mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b8e938acda | ||
|
|
b08d3f926e |
No files matched your search
@@ -184,12 +184,10 @@ Recipes are tested, copy-pasteable patterns for a task, one layer above the per-
|
||||
- **Description.** Written for someone who has never heard of Puter. Start with "Learn how to", put the action first, and say what the reader's app gains: "Learn how to add file sharing to your app with Puter.js, so users can share files in their Puter account with other people." No API names, no mechanism, no list of example cases, and no "not just …" contrast.
|
||||
- **Intro.** Explain the situation and why you'd do this, tied to something the reader already knows ("`puter.kv` is your database in Puter.js"). No "Use this when" bullet list.
|
||||
- **Structure.** Get to the first working call as early as possible, then add validation, rate limits, and sign-in after it. Explain a Puter concept the first time it appears, such as what `me.puter` is or that `puter.workers.exec()` opens the Puter sign-in window. Split recipes by what the reader needs, not by technique, and give a family of recipes a shared tag.
|
||||
- **Reading order.** A recipe reads top to bottom. Each code block uses only functions and variables from blocks above it, so the reader never has to search further down the page to understand it. Start with a small version that works, and have each section add one case. When a section changes a function from earlier, show the whole new version of that function there. If the code is built up over several sections, end with a `## Full Code` block that has all of it.
|
||||
- **Link to the reference for full option lists.** Mention only the options the recipe uses, then link to the function page for the rest ("All the sort options are listed on [`puter.fs.readdir()`](/FS/readdir/)."). A copied list of options, rules, or scopes goes stale when the reference changes.
|
||||
- **Limits.** Mention a limit only when it's the reason for a pattern the recipe uses, such as one wildcard subscription instead of one per key, and link to [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) or the function page for the number. Leave out limits that code following the recipe never reaches.
|
||||
- **Code.** Platform-neutral: just the code, with top-level `await`. No `<script>` or `<html>` wrapper, no IIFE.
|
||||
- **Tradeoffs.** Justify a choice with one scenario the reader can picture right away, in about two short paragraphs, or fold each point into the section where it matters. The description states the benefit only.
|
||||
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit the recipe needs goes in the section it applies to, and other facts belong on the linked reference page.
|
||||
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit goes in the section it applies to; other facts belong on the linked reference page.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -215,9 +215,7 @@
|
||||
// "bootstrapTables": true,
|
||||
"path": "",
|
||||
// Credentials. NOTE: snake_case here, unlike `s3.s3Config` below.
|
||||
// For dynamodb-local, any non-empty values work. On AWS,
|
||||
// `"useCredentialChain": true` uses the default credential chain
|
||||
// instead of keys.
|
||||
// For dynamodb-local, any non-empty values work.
|
||||
"aws": {
|
||||
"access_key": "",
|
||||
"secret_key": "",
|
||||
@@ -315,8 +313,7 @@
|
||||
// The tables ship either way and sit inert; this gates whether `/teams`
|
||||
// is registered at all, so with it off the paths do not exist rather than
|
||||
// existing and refusing. It is also the backout: turning it off removes
|
||||
// the feature without touching data. Teams ships as an extension; on,
|
||||
// the server refuses to boot unless one registers it.
|
||||
// the feature without touching data.
|
||||
"teams_enabled": false,
|
||||
//
|
||||
// Live teams one user may own. Soft-deleted ones do not count, so
|
||||
@@ -433,8 +430,6 @@
|
||||
"mistral": { "apiKey": "" },
|
||||
"xai": { "apiKey": "" },
|
||||
"moonshot": { "apiKey": "" },
|
||||
// MiniMax. One key powers chat and video generation (H3, Hailuo);
|
||||
// video calls go to the same host, beside the chat API's `/v1`.
|
||||
"minimax": {
|
||||
"apiKey": "",
|
||||
"apiBaseUrl": "https://api.minimax.io/v1"
|
||||
@@ -497,8 +492,6 @@
|
||||
"defaultVoiceId": "",
|
||||
"speechToSpeechModelId": ""
|
||||
},
|
||||
// Both AWS providers also take `"useCredentialChain": true` in place
|
||||
// of keys, like `dynamo.aws`.
|
||||
"aws-polly": {
|
||||
"access_key": "",
|
||||
"secret_key": "",
|
||||
|
||||
+1
-4
@@ -51,10 +51,7 @@ two pages can't be decoded into how fast the table grows; `openCursor` still
|
||||
reads a plain cursor. What a cursor wraps is an implementation detail per
|
||||
store:
|
||||
|
||||
- DynamoDB-backed lists wrap `LastEvaluatedKey`. A kv listing of another
|
||||
namespace (cross-app or through a share handle) resumes after the last entry
|
||||
it returned instead, since the scan can stop on a private one, and its cursor
|
||||
is sealed to that listing and padded to one length.
|
||||
- DynamoDB-backed lists wrap `LastEvaluatedKey`.
|
||||
- SQL-backed lists wrap a keyset position — `(sortValue, id)` of the last
|
||||
row — and the query seeks past it (`WHERE (col, id) > (?, ?) ORDER BY col,
|
||||
id`). Any SQL list gaining a cursor must have a deterministic `ORDER BY`
|
||||
|
||||
@@ -43,6 +43,7 @@ beforeAll(async () => {
|
||||
create_shortcut: true,
|
||||
payment_bypass: true,
|
||||
},
|
||||
teams_enabled: true,
|
||||
} as never);
|
||||
});
|
||||
|
||||
@@ -88,7 +89,7 @@ describe('whoami extension — handleWhoami', () => {
|
||||
return { seat, team };
|
||||
};
|
||||
|
||||
it('names no team where teams is off', async () => {
|
||||
it('names no team where teams is not installed', async () => {
|
||||
const { seat } = await seatOf('Acme Corp');
|
||||
const { res, captured } = makeRes();
|
||||
|
||||
@@ -97,7 +98,7 @@ describe('whoami extension — handleWhoami', () => {
|
||||
() => handleWhoami(makeReq(), res),
|
||||
);
|
||||
|
||||
// Off, so the seat is never looked up.
|
||||
// `extension.import` resolves the registry, which has no team.
|
||||
expect((captured.body as { team?: unknown }).team).toBeUndefined();
|
||||
});
|
||||
|
||||
|
||||
Generated
+1
-1
@@ -19099,7 +19099,7 @@
|
||||
},
|
||||
"src/puter-js": {
|
||||
"name": "@heyputer/puter.js",
|
||||
"version": "2.6.5",
|
||||
"version": "2.6.4",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@heyputer/kv.js": "^0.2.1",
|
||||
|
||||
@@ -24,8 +24,9 @@ import type { PuterServer } from './server.ts';
|
||||
import { createTestUser, setupTestServer } from './testUtil.ts';
|
||||
|
||||
/**
|
||||
* Teams is optional, so core has to work when neither the service nor the store
|
||||
* is registered, which is how this test server boots.
|
||||
* Teams is optional, so core has to work when neither the service nor the
|
||||
* store is registered. Both are dropped after boot here, which is the shape
|
||||
* core sees wherever the feature is not installed.
|
||||
*/
|
||||
describe('core without a teams registration', () => {
|
||||
let server: PuterServer;
|
||||
@@ -43,9 +44,13 @@ describe('core without a teams registration', () => {
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer({
|
||||
teams_enabled: true,
|
||||
// Without this the allowlist check returns before asking Teams.
|
||||
teams_allowed_email_domains: ['test.local'],
|
||||
} as never);
|
||||
// What the registry looks like where Teams is not installed.
|
||||
delete (server.services as { team?: unknown }).team;
|
||||
delete (server.stores as { team?: unknown }).team;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
@@ -57,12 +62,6 @@ describe('core without a teams registration', () => {
|
||||
expect(server.stores.team).toBeUndefined();
|
||||
});
|
||||
|
||||
it('refuses to boot when the config asks for teams', async () => {
|
||||
await expect(
|
||||
setupTestServer({ teams_enabled: true } as never),
|
||||
).rejects.toThrow(/teams_enabled/);
|
||||
});
|
||||
|
||||
it('deletes an account without asking what it was charged for', async () => {
|
||||
const user = await makeUser();
|
||||
await expect(
|
||||
|
||||
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
|
||||
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
|
||||
|
||||
/** Highest schema version the migration table can reach. */
|
||||
const CURRENT_SCHEMA_VERSION = 91;
|
||||
const CURRENT_SCHEMA_VERSION = 90;
|
||||
|
||||
/**
|
||||
* These suites migrate real files on disk. Idle they finish in well under a
|
||||
|
||||
@@ -129,7 +129,6 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
|
||||
[87, ['0092_share-entry-owner.sql']],
|
||||
[88, ['0093_fsentries-parent-uid-index.sql']],
|
||||
[89, ['0094_deleted-apps.sql']],
|
||||
[90, ['0095_kv-share-handle-levels.sql']],
|
||||
];
|
||||
|
||||
export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Share levels on `kv_share_handles`. See
|
||||
-- sqlite/0095_kv-share-handle-levels.sql for the rationale. No per-file
|
||||
-- applied-state tracking, so every step is guarded and a replay does nothing.
|
||||
|
||||
CALL _puter_add_col('kv_share_handles', 'revoked_by', '`revoked_by` varchar(8) DEFAULT NULL');
|
||||
CALL _puter_add_col('kv_share_handles', 'label', '`label` varchar(512) DEFAULT NULL');
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_indexes;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_kv_share_handle_indexes()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'kv_share_handles'
|
||||
AND INDEX_NAME = 'idx_kv_share_handles_grantee'
|
||||
) THEN
|
||||
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_grantee` (`grantee_user_id`, `id`);
|
||||
END IF;
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'kv_share_handles'
|
||||
AND INDEX_NAME = 'idx_kv_share_handles_app'
|
||||
) THEN
|
||||
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_app` (`app_uid`);
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_kv_share_handle_indexes();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_indexes;
|
||||
@@ -1,29 +0,0 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Share levels on kv_share_handles. See
|
||||
-- sqlite/0095_kv-share-handle-levels.sql for the rationale. Idempotent via
|
||||
-- IF NOT EXISTS; there is no per-file applied-state tracking.
|
||||
|
||||
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS revoked_by VARCHAR(8) DEFAULT NULL;
|
||||
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS label VARCHAR(512) DEFAULT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_grantee
|
||||
ON kv_share_handles (grantee_user_id, id);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_app
|
||||
ON kv_share_handles (app_uid);
|
||||
@@ -1,35 +0,0 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- What share levels need from `kv_share_handles`. The level itself is not a
|
||||
-- column: it is the root of the stored `permission`.
|
||||
--
|
||||
-- - `revoked_by` : `owner`, `holder` or `system` (the namespace app was
|
||||
-- deleted). Null while live, and on rows retired before
|
||||
-- this existed.
|
||||
-- - `label` : optional owner-set text the holder is shown.
|
||||
-- - the grantee index serves the holder's listing and their unshare, and
|
||||
-- the app index retiring a deleted app's handles.
|
||||
|
||||
ALTER TABLE `kv_share_handles` ADD COLUMN `revoked_by` TEXT DEFAULT NULL;
|
||||
ALTER TABLE `kv_share_handles` ADD COLUMN `label` TEXT DEFAULT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_grantee`
|
||||
ON `kv_share_handles` (`grantee_user_id`, `id`);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_app`
|
||||
ON `kv_share_handles` (`app_uid`);
|
||||
@@ -188,47 +188,6 @@ describe('DDBClient — item operations', () => {
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('puts only while its condition holds', async () => {
|
||||
const absent = {
|
||||
expression: 'attribute_not_exists(#pk)',
|
||||
names: { '#pk': 'pk' },
|
||||
};
|
||||
await client.put(
|
||||
TABLE,
|
||||
{ pk: 'u7', sk: 'once', v: 1 },
|
||||
{
|
||||
condition: absent,
|
||||
},
|
||||
);
|
||||
await expect(
|
||||
client.put(
|
||||
TABLE,
|
||||
{ pk: 'u7', sk: 'once', v: 2 },
|
||||
{
|
||||
condition: absent,
|
||||
},
|
||||
),
|
||||
).rejects.toMatchObject({ name: 'ConditionalCheckFailedException' });
|
||||
expect(
|
||||
(await client.get(TABLE, { pk: 'u7', sk: 'once' })).Item?.v,
|
||||
).toBe(1);
|
||||
|
||||
await client.put(
|
||||
TABLE,
|
||||
{ pk: 'u7', sk: 'once', v: 3 },
|
||||
{
|
||||
condition: {
|
||||
expression: '#v = :held',
|
||||
names: { '#v': 'v' },
|
||||
values: { ':held': 1 },
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(
|
||||
(await client.get(TABLE, { pk: 'u7', sk: 'once' })).Item?.v,
|
||||
).toBe(3);
|
||||
});
|
||||
|
||||
it('fetches keys from several tables in one batch', async () => {
|
||||
await client.put(TABLE, { pk: 'b1', sk: 'a', v: 1 });
|
||||
await client.put(TABLE, { pk: 'b1', sk: 'b', v: 2 });
|
||||
@@ -542,19 +501,6 @@ describe('DDBClient — credentialled configuration', () => {
|
||||
).toThrow('requires both `access_key` and `secret_key`');
|
||||
});
|
||||
|
||||
it('accepts the credential chain in place of keys', async () => {
|
||||
const client = new DDBClient({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
dynamo: {
|
||||
aws: { useCredentialChain: true, region: 'us-west-2' },
|
||||
endpoint: 'http://127.0.0.1:1',
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
|
||||
await expect(client.recreateClient()).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it('skips table creation on a managed deployment', async () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const client = new DDBClient({
|
||||
|
||||
@@ -43,10 +43,6 @@ import { once } from 'node:events';
|
||||
import { Agent as httpsAgent } from 'node:https';
|
||||
import { HttpError } from '../../core/http';
|
||||
import type { IConfig, IDynamoConfig } from '../../types';
|
||||
import {
|
||||
awsCredentialsFrom,
|
||||
type AwsCredentials,
|
||||
} from '../../util/awsCredentials.js';
|
||||
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
|
||||
import { runWithConcurrencyLimitSettled } from '../../util/concurrency.js';
|
||||
import { Span } from '../../util/span.js';
|
||||
@@ -252,19 +248,7 @@ export class DDBClient extends PuterClient {
|
||||
}
|
||||
|
||||
@Span('ddb.put', (table: string) => ({ 'db.table': table }))
|
||||
async put<T extends Record<string, unknown>>(
|
||||
table: string,
|
||||
item: T,
|
||||
opts?: {
|
||||
/** Refused (`ConditionalCheckFailedException`) unless it holds. */
|
||||
condition?: {
|
||||
expression: string;
|
||||
names?: Record<string, string>;
|
||||
values?: Record<string, unknown>;
|
||||
};
|
||||
},
|
||||
) {
|
||||
const condition = opts?.condition;
|
||||
async put<T extends Record<string, unknown>>(table: string, item: T) {
|
||||
const client = await this.#getDocumentClient();
|
||||
return sendRepairingValues(
|
||||
item,
|
||||
@@ -274,23 +258,6 @@ export class DDBClient extends PuterClient {
|
||||
TableName: table,
|
||||
Item: itemToWrite,
|
||||
ReturnConsumedCapacity: 'TOTAL',
|
||||
...(condition
|
||||
? {
|
||||
ConditionExpression: condition.expression,
|
||||
...(condition.names
|
||||
? {
|
||||
ExpressionAttributeNames:
|
||||
condition.names,
|
||||
}
|
||||
: {}),
|
||||
...(condition.values
|
||||
? {
|
||||
ExpressionAttributeValues:
|
||||
condition.values,
|
||||
}
|
||||
: {}),
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
),
|
||||
() => `a put to ${table}`,
|
||||
@@ -721,15 +688,17 @@ export class DDBClient extends PuterClient {
|
||||
}
|
||||
|
||||
#bindAwsClient() {
|
||||
const credentials = awsCredentialsFrom(this.#ddbConfig.aws);
|
||||
if (!credentials) {
|
||||
const accessKeyId = this.#ddbConfig.aws?.access_key;
|
||||
const secretAccessKey = this.#ddbConfig.aws?.secret_key;
|
||||
|
||||
if (!accessKeyId || !secretAccessKey) {
|
||||
throw new Error(
|
||||
'DynamoDB aws config requires both `access_key` and `secret_key`, or `useCredentialChain`',
|
||||
'DynamoDB aws config requires both `access_key` and `secret_key`',
|
||||
);
|
||||
}
|
||||
|
||||
this.#bindClient({
|
||||
credentials,
|
||||
credentials: { accessKeyId, secretAccessKey },
|
||||
region: this.#ddbConfig.aws?.region || 'us-west-2',
|
||||
endpoint: this.#ddbConfig.endpoint,
|
||||
});
|
||||
@@ -751,7 +720,7 @@ export class DDBClient extends PuterClient {
|
||||
region,
|
||||
endpoint,
|
||||
}: {
|
||||
credentials: AwsCredentials;
|
||||
credentials: { accessKeyId: string; secretAccessKey: string };
|
||||
region: string;
|
||||
endpoint?: string;
|
||||
}) {
|
||||
|
||||
@@ -397,8 +397,6 @@ export type EventMap = {
|
||||
user_id: number;
|
||||
user_uuid?: string;
|
||||
stripe_customer_id?: string | null;
|
||||
/** Apps the account owned, which cascaded away with it. */
|
||||
appUids?: string[];
|
||||
};
|
||||
|
||||
// ---- Team billing ---- the trigger, never the charge ----
|
||||
@@ -492,33 +490,13 @@ export type EventMap = {
|
||||
noShareKeys?: string[];
|
||||
/** Events handler runs behind the write; see `Actor.handlerDepth`. */
|
||||
handlerDepth?: number;
|
||||
/**
|
||||
* The user who made the change. Differs from `userId` when a share
|
||||
* holder wrote into the owner's namespace.
|
||||
*/
|
||||
actingUserId?: number;
|
||||
/** Made by a worker session or an events handler; see `viaWorkerOf`. */
|
||||
viaWorker?: true;
|
||||
};
|
||||
/**
|
||||
* A whole namespace was emptied. Namespace-level on purpose: `flush`'s own
|
||||
* key enumeration is truncated for a large namespace, so the keys it saw
|
||||
* are not the keys it removed.
|
||||
*/
|
||||
'kv.flushed': {
|
||||
namespace: string;
|
||||
userId: number;
|
||||
actingUserId?: number;
|
||||
viaWorker?: true;
|
||||
};
|
||||
/**
|
||||
* Share handles stopped being addresses, whoever retired them. `userId` is
|
||||
* one user who could have subscribed through them, their owner or their
|
||||
* holder. Listeners end what was made through them even where another grant
|
||||
* still covers the region. Local, like `permission.revoked`: one region
|
||||
* settling covers every other.
|
||||
*/
|
||||
'kv.share.retired': { handles: string[]; userId: number };
|
||||
'kv.flushed': { namespace: string; userId: number };
|
||||
|
||||
// ---- Notifications ----
|
||||
/**
|
||||
@@ -701,10 +679,6 @@ export type EventMap = {
|
||||
requestHash?: string;
|
||||
mime: string;
|
||||
};
|
||||
// Asked for a request whose host is outside every configured domain. A
|
||||
// listener that recognises the host sets `result.subdomain` to the site
|
||||
// it serves; an unclaimed host is answered with a 404.
|
||||
'site.resolveHost': { host: string; result: { subdomain: string | null } };
|
||||
// Asked before a hosted file is streamed. A listener that owns the site
|
||||
// sets `result.allowed = false` to withhold the entry; the visitor then
|
||||
// sees the same 404 as for a missing file.
|
||||
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types';
|
||||
|
||||
/**
|
||||
* HTTP-level coverage for `/auth/revoke-own-access-token` — the route that
|
||||
@@ -423,7 +424,7 @@ describe('2FA enrollment over HTTP', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
env = await setupPuterTestEnv({ teams_enabled: true } as IConfig);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(async () => {
|
||||
|
||||
@@ -49,7 +49,7 @@ import {
|
||||
} from '../../core/http/types.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { createSecret as otpCreateSecret } from '../../services/auth/OTPUtil.js';
|
||||
import { kvSharePermission } from '../../services/kvShare/kvShares.js';
|
||||
import { kvSharePermission } from '../../services/events/kvShares.js';
|
||||
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { resetCardVerificationStatusCache } from '../../util/cardFallback.js';
|
||||
@@ -2151,7 +2151,7 @@ describe('AuthController grant flows', () => {
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const permission = `events:background`;
|
||||
const permission = `service:tg-app:ii:read`;
|
||||
// The controller's grant call delegates through PermissionService
|
||||
// (which uses ALS Context.set), so wrap in runWithContext.
|
||||
const res = makeRes();
|
||||
@@ -2192,7 +2192,7 @@ describe('AuthController grant flows', () => {
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const permission = `events:background`;
|
||||
const permission = `service:tg-origin-app:ii:read`;
|
||||
const res = makeRes();
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
@@ -2229,7 +2229,7 @@ describe('AuthController grant flows', () => {
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const permission = `events:background`;
|
||||
const permission = `service:tr-origin-app:ii:read`;
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
@@ -2294,7 +2294,7 @@ describe('AuthController grant flows', () => {
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleRevokeUserApp(
|
||||
makeReq(
|
||||
{ app_uid: app.uid, permission: 'events:background' },
|
||||
{ app_uid: app.uid, permission: 'service:x:ii:read' },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
@@ -2318,12 +2318,12 @@ describe('AuthController grant flows', () => {
|
||||
|
||||
it('grant-user-app: 400 on non-string or oversized origin/app_uid/permission', async () => {
|
||||
const cases = [
|
||||
{ origin: { host: 'evil' }, permission: 'events:background' },
|
||||
{ origin: 'https://a.test', permission: ['events:background'] },
|
||||
{ app_uid: 12345, permission: 'events:background' },
|
||||
{ origin: { host: 'evil' }, permission: 'service:x:ii:read' },
|
||||
{ origin: 'https://a.test', permission: ['service:x:ii:read'] },
|
||||
{ app_uid: 12345, permission: 'service:x:ii:read' },
|
||||
{
|
||||
origin: `https://${'a'.repeat(5000)}.test`,
|
||||
permission: 'events:background',
|
||||
permission: 'service:x:ii:read',
|
||||
},
|
||||
];
|
||||
for (const body of cases) {
|
||||
@@ -2354,7 +2354,7 @@ describe('AuthController grant flows', () => {
|
||||
{ ownerUserId: target.id },
|
||||
);
|
||||
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:squat:ii:read';
|
||||
for (const handler of [
|
||||
'handleGrantUserApp',
|
||||
'handleRevokeUserApp',
|
||||
@@ -2398,7 +2398,7 @@ describe('AuthController grant flows', () => {
|
||||
{ ownerUserId: target.id },
|
||||
);
|
||||
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:squat-beside:ii:read';
|
||||
for (const handler of [
|
||||
'handleGrantUserApp',
|
||||
'handleRevokeUserApp',
|
||||
@@ -2451,7 +2451,7 @@ describe('AuthController grant flows', () => {
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:tp-origin:ii:read';
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
@@ -2487,7 +2487,7 @@ describe('AuthController grant flows', () => {
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:tm-app:ii:read';
|
||||
for (const body of [
|
||||
{ app_uid: app.uid, permission, meta: 'nope' },
|
||||
{ app_uid: app.uid, permission, meta: [1, 2] },
|
||||
@@ -2528,7 +2528,7 @@ describe('AuthController grant flows', () => {
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:tb-app:ii:read';
|
||||
const oversized = { blob: 'z'.repeat(5000) };
|
||||
for (const body of [
|
||||
{ app_uid: app.uid, permission, extra: oversized },
|
||||
@@ -2605,278 +2605,26 @@ describe('AuthController grant flows', () => {
|
||||
expect(res.body).toEqual({});
|
||||
});
|
||||
|
||||
it('grant-user-app: grants read and write share delegations on a region, never on the namespace', async () => {
|
||||
const appName = `tkvl-${uuidv4()}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestKvShareLevelsApp',
|
||||
index_url: `https://${appName}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
for (const root of ['kv-share-read', 'kv-share-write']) {
|
||||
const namespace = `manage:${root}:${issuer.uuid}:${app.uid}`;
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{ app_uid: app.uid, permission: namespace },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
|
||||
|
||||
const res = makeRes();
|
||||
await inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{ app_uid: app.uid, permission: `${namespace}:team` },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
res,
|
||||
),
|
||||
);
|
||||
expect(res.body).toEqual({});
|
||||
}
|
||||
const rows = await server.clients.db.read(
|
||||
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ? ORDER BY `permission`',
|
||||
[issuer.id, app.id],
|
||||
);
|
||||
expect(rows.map((r: { permission: string }) => r.permission)).toEqual([
|
||||
`manage:kv-share-read:${issuer.uuid}:${app.uid}:team`,
|
||||
`manage:kv-share-write:${issuer.uuid}:${app.uid}:team`,
|
||||
]);
|
||||
});
|
||||
|
||||
it('grant-user-app: refuses a share delegation whose region a new share could not name, writing nothing', async () => {
|
||||
const appName = `tkvc-${uuidv4()}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestKvShareCharsApp',
|
||||
index_url: `https://${appName}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
for (const region of ['Team', 'team:Abc', 'caf\u00E9', 'a b'])
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{
|
||||
app_uid: app.uid,
|
||||
permission: `manage:kv-share-read:${issuer.uuid}:${app.uid}:${region}`,
|
||||
},
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
|
||||
const rows = await server.clients.db.read(
|
||||
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
|
||||
[issuer.id, app.id],
|
||||
);
|
||||
expect(rows).toEqual([]);
|
||||
});
|
||||
|
||||
it('grant-user-app: refuses a permission outside the app-grantable families, writing nothing', async () => {
|
||||
const appName = `tng-${uuidv4()}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestNotGrantableApp',
|
||||
index_url: `https://${appName}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const other = await server.stores.app.create(
|
||||
{
|
||||
name: `tng-other-${uuidv4()}`,
|
||||
title: 'TestNotGrantableOther',
|
||||
index_url: `https://tng-other-${uuidv4()}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const refused = [
|
||||
['driver:puter-image-generation:generate'],
|
||||
[`user:${target.uuid}:email:read`],
|
||||
// One bad entry refuses the batch.
|
||||
['events:background', 'random-string'],
|
||||
// A share delegation names the app's own namespace only.
|
||||
[`manage:kv-share:${issuer.uuid}:${other.uid}:rooms`],
|
||||
];
|
||||
for (const permissions of refused) {
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{ app_uid: app.uid, permissions },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'permission_not_grantable',
|
||||
});
|
||||
}
|
||||
const rows = await server.clients.db.read(
|
||||
'SELECT 1 FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
|
||||
[issuer.id, app.id],
|
||||
);
|
||||
expect(rows).toEqual([]);
|
||||
});
|
||||
|
||||
it('grant-user-app: refuses a non-canonical spelling of a grantable permission', async () => {
|
||||
const appName = `tnc-${uuidv4()}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestNonCanonicalApp',
|
||||
index_url: `https://${appName}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
for (const permission of [
|
||||
`f\\s:/${issuer.username}/Documents:read`,
|
||||
`app-root-di\\r:${app.uid}:read`,
|
||||
`manag\\e:kv-share:${issuer.uuid}:${app.uid}:rooms`,
|
||||
]) {
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{ app_uid: app.uid, permission },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'permission_not_grantable' });
|
||||
}
|
||||
});
|
||||
|
||||
it('grant-user-app: refuses a share delegation over a prefix no share could be made of, as a share would be', async () => {
|
||||
const appName = `tkvp-${uuidv4().slice(0, 8)}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestKvSharePrefixApp',
|
||||
index_url: `https://${appName}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
for (const root of ['kv-share', 'kv-share-read', 'kv-share-write']) {
|
||||
// A share grant one character wider than the column.
|
||||
const head = `${root}:${issuer.uuid}:${app.uid}:`;
|
||||
const region = 'a'.repeat(256 - head.length);
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{
|
||||
app_uid: app.uid,
|
||||
permission: `manage:${head}${region}`,
|
||||
},
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
|
||||
}
|
||||
});
|
||||
|
||||
it('grant-user-app: a share delegation must name the app the request names, by uid, name or origin', async () => {
|
||||
const appName = `tkvn-${uuidv4()}`;
|
||||
const origin = `https://${appName}.example.test`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestKvShareNamedApp',
|
||||
index_url: `${origin}/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const other = await server.stores.app.create(
|
||||
{
|
||||
name: `tkvn-other-${uuidv4()}`,
|
||||
title: 'TestKvShareNamedOther',
|
||||
index_url: `https://tkvn-other-${uuidv4()}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
const grant = (requester: Record<string, string>, namespace: string) =>
|
||||
it('grant-user-app: 400 on a `permission` wider than the column it lands in', async () => {
|
||||
// 300 chars is under the 4096 input cap but over `varchar(255)`, so it
|
||||
// used to reach the INSERT and fault on MySQL/Postgres. The check runs
|
||||
// after the rewrite, so this has to be a permission no rewriter
|
||||
// shortens — and it has to reject before the app is resolved, since
|
||||
// this uid names no app.
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{
|
||||
...requester,
|
||||
permission: `manage:kv-share:${issuer.uuid}:${namespace}:rooms`,
|
||||
app_uid: `app-${uuidv4()}`,
|
||||
permission: `service:${'a'.repeat(300)}:ii:read`,
|
||||
},
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
);
|
||||
|
||||
for (const requester of [{ app_uid: app.name }, { origin }]) {
|
||||
await grant(requester, app.uid);
|
||||
await expect(grant(requester, other.uid)).rejects.toMatchObject({
|
||||
legacyCode: 'permission_not_grantable',
|
||||
});
|
||||
}
|
||||
// The app's name is not its namespace.
|
||||
await expect(
|
||||
grant({ app_uid: app.uid }, app.name),
|
||||
).rejects.toMatchObject({ legacyCode: 'permission_not_grantable' });
|
||||
const rows = await server.clients.db.read(
|
||||
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
|
||||
[issuer.id, app.id],
|
||||
);
|
||||
expect(rows.map((r: { permission: string }) => r.permission)).toEqual([
|
||||
`manage:kv-share:${issuer.uuid}:${app.uid}:rooms`,
|
||||
]);
|
||||
});
|
||||
|
||||
it('grant-user-app: 400 on a `permission` wider than the column it lands in', async () => {
|
||||
// Under the 4096 input cap but over `varchar(255)`, so it would reach
|
||||
// the INSERT and fault on MySQL/Postgres. The check runs after the
|
||||
// rewrite, so this has to be a grantable permission no rewriter
|
||||
// shortens.
|
||||
const appName = `tw-${uuidv4()}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: appName,
|
||||
title: 'TestWidthApp',
|
||||
index_url: `https://${appName}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
// The shares it delegates fit, at 255 characters; the consent's own
|
||||
// `manage:` arm takes it past the column.
|
||||
const head = `kv-share:${issuer.uuid}:${app.uid}:`;
|
||||
const permission = `manage:${head}${'a'.repeat(255 - head.length)}`;
|
||||
await expect(
|
||||
inCtx(issuerActor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq(
|
||||
{ app_uid: app.uid, permission },
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
makeRes(),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'bad_request',
|
||||
message: 'Invalid `permission`',
|
||||
});
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
|
||||
it('grant-user-app: an oversized permission that a rewriter shortens is accepted', async () => {
|
||||
@@ -2888,8 +2636,6 @@ describe('AuthController grant flows', () => {
|
||||
// on "please try again" for any deeply nested file. Exercised here
|
||||
// through a rewriter of our own, since the real fs rewriter resolves
|
||||
// the path through the fsentry store and this suite has no entries.
|
||||
// It rewrites a share delegation only because that is a grantable
|
||||
// shape no production rewriter touches.
|
||||
const appName = `tlp-${uuidv4()}`;
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
@@ -2900,15 +2646,13 @@ describe('AuthController grant flows', () => {
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
|
||||
// Over the column only by its `manage:` arm, so the shares it
|
||||
// delegates still fit.
|
||||
const head = `kv-share:${issuer.uuid}:${app.uid}:`;
|
||||
const longPermission = `manage:${head}${'a'.repeat(255 - head.length)}`;
|
||||
const shortPermission = `manage:${head}${uuidv4()}`;
|
||||
const prefix = `tlprw-${uuidv4()}`;
|
||||
const longPermission = `${prefix}:${'a'.repeat(300)}:read`;
|
||||
const shortPermission = `${prefix}:${uuidv4()}:read`;
|
||||
expect(longPermission.length).toBeGreaterThan(255);
|
||||
expect(shortPermission.length).toBeLessThanOrEqual(255);
|
||||
server.services.permission.registerRewriter({
|
||||
id: `test-shorten-${app.uid}`,
|
||||
id: `test-shorten-${prefix}`,
|
||||
matches: (permission: string) => permission === longPermission,
|
||||
rewrite: async () => shortPermission,
|
||||
});
|
||||
@@ -2968,8 +2712,7 @@ describe('AuthController grant flows', () => {
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
|
||||
// A grantable shape no production rewriter touches.
|
||||
const prefix = `manage:kv-share:${issuer.uuid}:${app.uid}:conc-${uuidv4()}`;
|
||||
const prefix = `conc-${uuidv4()}`;
|
||||
let release: () => void;
|
||||
let entered: () => void;
|
||||
const held = new Promise<void>((r) => {
|
||||
@@ -2997,7 +2740,7 @@ describe('AuthController grant flows', () => {
|
||||
makeReq(
|
||||
{
|
||||
app_uid: app.uid,
|
||||
permission: `${prefix}:write`,
|
||||
permission: `${prefix}:${app.uid}:write`,
|
||||
},
|
||||
{ actor: issuerActor },
|
||||
),
|
||||
@@ -3038,8 +2781,8 @@ describe('AuthController grant flows', () => {
|
||||
{ ownerUserId: issuer.id },
|
||||
);
|
||||
|
||||
const prefix = `manage:kv-share:${issuer.uuid}:${app.uid}:tardrw-${uuidv4()}`;
|
||||
const pseudoPermission = `${prefix}:write`;
|
||||
const prefix = `tardrw-${uuidv4()}`;
|
||||
const pseudoPermission = `${prefix}:${app.uid}:write`;
|
||||
const resolvedPermission = `fs:${uuidv4()}:write`;
|
||||
const NOTHING = 'nothing-in-particular';
|
||||
server.services.permission.registerRewriter({
|
||||
@@ -3280,7 +3023,7 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
|
||||
// phase A's pre-flight passes, phase C's re-check then fails, so the
|
||||
// leaf (and its auto-created parent) already exist by the time the
|
||||
// request unwinds.
|
||||
const prefix = `manage:kv-share:${user.uuid}:${appUid}:rbparent-${uuidv4()}`;
|
||||
const prefix = `rbparent-${uuidv4()}`;
|
||||
let calls = 0;
|
||||
server.services.permission.registerRewriter({
|
||||
id: prefix,
|
||||
@@ -3446,13 +3189,13 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
|
||||
});
|
||||
|
||||
it('refuses a `manage:` grant outright, creating nothing', async () => {
|
||||
it('does not apply `create` to a `manage:` grant — 404 unchanged, nothing created', async () => {
|
||||
const p = path(`manage-missing-${rand()}`);
|
||||
await expect(
|
||||
grant({ permission: `manage:fs:${p}:write` }),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'permission_not_grantable',
|
||||
statusCode: 404,
|
||||
legacyCode: 'subject_does_not_exist',
|
||||
});
|
||||
expect(await server.stores.fsEntry.getEntryByPath(p)).toBeFalsy();
|
||||
});
|
||||
@@ -3542,7 +3285,7 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
|
||||
// (so nothing is rejected before anything is created) but has grown
|
||||
// too wide by the time phase C re-runs it — modelling a grant that
|
||||
// only fails once the fs path from the same batch already exists.
|
||||
const prefix = `manage:kv-share:${user.uuid}:${appUid}:rbtest-${uuidv4()}`;
|
||||
const prefix = `rbtest-${uuidv4()}`;
|
||||
let calls = 0;
|
||||
server.services.permission.registerRewriter({
|
||||
id: `test-rollback-${prefix}`,
|
||||
@@ -3583,37 +3326,6 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('check-permissions answers an app asking about itself through ACL, so a folder grant covers what is inside', async () => {
|
||||
const dir = path(`acl-self-${rand()}`);
|
||||
const child = `${dir}/inner`;
|
||||
await server.services.fs.mkdir(user.id, {
|
||||
path: child,
|
||||
createMissingParents: true,
|
||||
});
|
||||
expect(await checkAsApp(`fs:${child}:read`)).toBe(false);
|
||||
|
||||
await grant({ permission: `fs:${dir}:write`, create: false });
|
||||
expect(await checkAsApp(`fs:${child}:read`)).toBe(true);
|
||||
expect(await checkAsApp(`fs:${child}:write`)).toBe(true);
|
||||
// Another user's home stays out of reach.
|
||||
expect(await checkAsApp(`fs:/${target.username}/Documents:read`)).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it('check-permissions answers an app asking about itself through ACL for a uid too', async () => {
|
||||
const dir = path(`acl-self-uid-${rand()}`);
|
||||
const child = await server.services.fs.mkdir(user.id, {
|
||||
path: `${dir}/inner`,
|
||||
createMissingParents: true,
|
||||
});
|
||||
expect(await checkAsApp(`fs:${child.uuid}:read`)).toBe(false);
|
||||
|
||||
await grant({ permission: `fs:${dir}:read`, create: false });
|
||||
expect(await checkAsApp(`fs:${child.uuid}:read`)).toBe(true);
|
||||
expect(await checkAsApp(`fs:${child.uuid}:write`)).toBe(false);
|
||||
});
|
||||
|
||||
it('check-permissions on a missing fs path answers false rather than 404ing, without poisoning a mixed batch', async () => {
|
||||
const missingPath = path(`.checkmissing-${rand()}`);
|
||||
// A held fs grant: the owning user holds it implicitly (is-owner), so
|
||||
@@ -4440,9 +4152,9 @@ describe('AuthController.handleCreateAccessToken + handleRevokeAccessToken', ()
|
||||
['an oversized permission string', ['x'.repeat(4097)]],
|
||||
[
|
||||
'an oversized `extra`',
|
||||
[['events:background', { a: 'y'.repeat(5000) }]],
|
||||
[['service:x:ii:read', { a: 'y'.repeat(5000) }]],
|
||||
],
|
||||
['a non-object `extra`', [['events:background', 'nope']]],
|
||||
['a non-object `extra`', [['service:x:ii:read', 'nope']]],
|
||||
['an empty permission string', ['']],
|
||||
])('rejects %s with 400', async (_label, permissions) => {
|
||||
await expect(
|
||||
@@ -7653,88 +7365,6 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
|
||||
]);
|
||||
});
|
||||
|
||||
it('check-permissions: answers false for share grants, which name what a holder is never told, but not for their manage arm', async () => {
|
||||
const { user: owner, actor: ownerActor } = await makeUserAndActor();
|
||||
const { user: holder, actor: holderActor } = await makeUserAndActor();
|
||||
const region = `notes-${uuidv4().slice(0, 8)}`;
|
||||
const grant = `kv-share-read:${owner.uuid}:os-global:${region}`;
|
||||
await inCtx(ownerActor, () =>
|
||||
server.services.permission.grantUserUserPermission(
|
||||
ownerActor,
|
||||
holder.username,
|
||||
grant,
|
||||
),
|
||||
);
|
||||
expect(await server.services.permission.check(holderActor, grant)).toBe(
|
||||
true,
|
||||
);
|
||||
|
||||
const asked = [
|
||||
grant,
|
||||
`kv-share:${owner.uuid}:os-global:${region}:inner`,
|
||||
`kv-share-write:${owner.uuid}:os-global:${region}`,
|
||||
];
|
||||
const res = makeRes();
|
||||
await inCtx(holderActor, () =>
|
||||
controller.handleCheckPermissions(
|
||||
makeReq({ permissions: asked }, { actor: holderActor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
expect(res.body).toEqual({
|
||||
permissions: Object.fromEntries(asked.map((p) => [p, false])),
|
||||
});
|
||||
|
||||
// What `puter.perms.check('kvShare', …)` asks.
|
||||
const manage = `manage:kv-share-read:${owner.uuid}:os-global:${region}`;
|
||||
const ownerRes = makeRes();
|
||||
await inCtx(ownerActor, () =>
|
||||
controller.handleCheckPermissions(
|
||||
makeReq({ permissions: [manage] }, { actor: ownerActor }),
|
||||
ownerRes,
|
||||
),
|
||||
);
|
||||
expect(ownerRes.body).toEqual({ permissions: { [manage]: true } });
|
||||
});
|
||||
|
||||
it('check-permissions: answers false for share grants asked by an app actor too', async () => {
|
||||
const { user: owner, actor: ownerActor } = await makeUserAndActor();
|
||||
const { user: holder, actor: holderActor } = await makeUserAndActor();
|
||||
const app = await server.stores.app.create(
|
||||
{
|
||||
name: `cpk-${uuidv4()}`,
|
||||
title: 'TestCheckPermsKvApp',
|
||||
index_url: 'https://check-perms-kv.example.test/index.html',
|
||||
},
|
||||
{ ownerUserId: holder.id },
|
||||
);
|
||||
const grant = `kv-share-read:${owner.uuid}:${app.uid}:notes-${uuidv4().slice(0, 8)}`;
|
||||
await inCtx(ownerActor, () =>
|
||||
server.services.permission.grantUserUserPermission(
|
||||
ownerActor,
|
||||
holder.username,
|
||||
grant,
|
||||
),
|
||||
);
|
||||
const appActor = makeActor({
|
||||
user: holderActor.user,
|
||||
app: { id: app.id, uid: app.uid },
|
||||
});
|
||||
// The holder's app in the share's namespace holds it.
|
||||
expect(await server.services.permission.check(appActor, grant)).toBe(
|
||||
true,
|
||||
);
|
||||
|
||||
const res = makeRes();
|
||||
await inCtx(appActor, () =>
|
||||
controller.handleCheckPermissions(
|
||||
makeReq({ permissions: [grant] }, { actor: appActor }),
|
||||
res,
|
||||
),
|
||||
);
|
||||
expect(res.body).toEqual({ permissions: { [grant]: false } });
|
||||
});
|
||||
|
||||
// What lets a permission request settle without a prompt. The second string
|
||||
// is a consent scope nothing implies, so the first's `true` is the grant.
|
||||
it('check-permissions: an app-under-user actor sees a grant made to that app', async () => {
|
||||
@@ -7949,7 +7579,7 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
|
||||
},
|
||||
{ ownerUserId: user.id },
|
||||
);
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:tl-app:ii:read';
|
||||
await inCtx(actor, () =>
|
||||
controller.handleGrantUserApp(
|
||||
makeReq({ app_uid: app.uid, permission, extra: {} }, { actor }),
|
||||
@@ -10585,7 +10215,7 @@ describe('AuthController — app-data grants', () => {
|
||||
await expect(
|
||||
post('handleGrantUserApp', {
|
||||
app_uid: grantee.uid,
|
||||
permission: `app-data:app-${uuidv4()}:kv:read`,
|
||||
permission: 'app-data:app-nope/:kv:read',
|
||||
}),
|
||||
).rejects.toMatchObject({ statusCode: 404 });
|
||||
});
|
||||
@@ -10652,7 +10282,7 @@ describe('AuthController — app-data grants', () => {
|
||||
|
||||
it('leaves unrelated permissions untouched by the new validation', async () => {
|
||||
const grantee = await makeAppRow();
|
||||
const permission = 'events:background';
|
||||
const permission = 'service:unrelated:ii:read';
|
||||
await post('handleGrantUserApp', {
|
||||
app_uid: grantee.uid,
|
||||
permission,
|
||||
|
||||
@@ -33,7 +33,7 @@ import {
|
||||
hasVerifiedPhone,
|
||||
} from '../../core/http/middleware/gates.js';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { isAppActor, isPlainUserActor, makeActor } from '../../core/actor.js';
|
||||
import { isPlainUserActor, makeActor } from '../../core/actor.js';
|
||||
import {
|
||||
checkRateLimit,
|
||||
consumeRouteRateLimit,
|
||||
@@ -88,16 +88,13 @@ import {
|
||||
import { promoteToVerifiedGroup } from '../../util/userProvisioning.js';
|
||||
import {
|
||||
assertBoundedManageGrant,
|
||||
assertNewShareSegments,
|
||||
assertPrefixFitsAccess,
|
||||
isKvSharePermission,
|
||||
} from '../../services/kvShare/kvShares.js';
|
||||
} from '../../services/events/kvShares.js';
|
||||
import {
|
||||
APP_DATA_PERMISSION_PREFIX,
|
||||
appDataSharingAllowed,
|
||||
parseAppDataPermission,
|
||||
} from '../../services/permission/appDataScopes.js';
|
||||
import { parseAppGrantablePermission } from '../../services/permission/appGrantable.js';
|
||||
import {
|
||||
assertCreatablePath,
|
||||
isOwnHomePath,
|
||||
@@ -3000,45 +2997,6 @@ export class AuthController extends PuterController {
|
||||
return typeof permission === 'string' && permission ? [permission] : [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuses any entry outside the families an app can be granted. Anything
|
||||
* else is a row the backend never checks for an app, or one wider than the
|
||||
* consent prompt can describe.
|
||||
*/
|
||||
async #assertAppGrantable(
|
||||
actor: Actor,
|
||||
appIdentifier: string,
|
||||
list: string[],
|
||||
): Promise<void> {
|
||||
const refuse = (entry: string) =>
|
||||
new HttpError(
|
||||
400,
|
||||
`Permission cannot be granted to an app: ${entry}`,
|
||||
{ legacyCode: 'permission_not_grantable' },
|
||||
);
|
||||
let appUid: string | undefined;
|
||||
for (const entry of list) {
|
||||
// A delegation over a whole key-value namespace has no bounded
|
||||
// description; refused with its own code.
|
||||
assertBoundedManageGrant(entry);
|
||||
const parsed = parseAppGrantablePermission(entry, actor.user!.uuid);
|
||||
if (!parsed) throw refuse(entry);
|
||||
if (parsed.family !== 'kvShare') continue;
|
||||
// Held to what a new share's prefix may be, so a consent never
|
||||
// covers shares that could not be made.
|
||||
assertNewShareSegments(parsed.segments);
|
||||
assertPrefixFitsAccess(
|
||||
parsed.ownerUuid,
|
||||
parsed.appUid,
|
||||
parsed.segments.join(':'),
|
||||
parsed.level,
|
||||
);
|
||||
// An app shares regions of its own namespace only.
|
||||
appUid ??= (await this.stores.app.resolveApp(appIdentifier))?.uid;
|
||||
if (parsed.appUid !== appUid) throw refuse(entry);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gate a cross-app data grant: the target must exist, must not have opted
|
||||
* out of sharing, and must be named. Also creates the target's AppData
|
||||
@@ -3217,7 +3175,6 @@ export class AuthController extends PuterController {
|
||||
}
|
||||
// 400 on a bad value, before anything else runs. Absent means create.
|
||||
const create = parseCreateFlag(req.body?.create);
|
||||
await this.#assertAppGrantable(req.actor!, app_uid, list);
|
||||
|
||||
// Validate every entry before writing or creating anything, so a bad
|
||||
// one in the list cannot leave a partially-granted set behind: the
|
||||
@@ -3230,6 +3187,10 @@ export class AuthController extends PuterController {
|
||||
kind: FsCreateKind;
|
||||
}> = [];
|
||||
for (const entry of list) {
|
||||
// A delegation over a whole key-value namespace has no bounded
|
||||
// description, so it is refused where it is asked for rather than
|
||||
// prompted for and then refused at use.
|
||||
assertBoundedManageGrant(entry);
|
||||
await this.#prepareAppDataGrant(req.actor!, entry);
|
||||
const plan = await this.#planFsPathCreate(
|
||||
req.actor!,
|
||||
@@ -3436,28 +3397,11 @@ export class AuthController extends PuterController {
|
||||
const unique = [...new Set(permissions)] as string[];
|
||||
const result: Record<string, boolean> = {};
|
||||
|
||||
// A share grant names its owner and prefix, which its holder is never
|
||||
// told, so asking about one would confirm a guess. `manage:` arms are
|
||||
// the app's own consent and stay answerable.
|
||||
const answerable: string[] = [];
|
||||
for (const permission of unique) {
|
||||
if (
|
||||
typeof permission === 'string' &&
|
||||
isKvSharePermission(permission)
|
||||
)
|
||||
result[permission] = false;
|
||||
else answerable.push(permission);
|
||||
}
|
||||
|
||||
// `fs:` reaches down a tree, and only ACL walks that. An app asking
|
||||
// about itself gets the same answer, so a grant on a parent folder
|
||||
// doesn't read as missing and prompt again.
|
||||
// `fs:` reaches down a tree, and only ACL walks that.
|
||||
const viaAcl = new Set(
|
||||
app_uid || isAppActor(actor)
|
||||
? answerable.filter((p) => aclAnswerableFsPermission(p))
|
||||
: [],
|
||||
app_uid ? unique.filter((p) => aclAnswerableFsPermission(p)) : [],
|
||||
);
|
||||
const viaScan = answerable.filter((p) => !viaAcl.has(p));
|
||||
const viaScan = unique.filter((p) => !viaAcl.has(p));
|
||||
|
||||
let granted: Map<string, boolean>;
|
||||
try {
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
} from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { installSeatStub } from '../../testFixtures/seatStub.js';
|
||||
|
||||
describe('account change second-factor verification', () => {
|
||||
@@ -14,7 +15,7 @@ describe('account change second-factor verification', () => {
|
||||
const links = new Map<string, string>();
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv();
|
||||
env = await setupPuterTestEnv({ teams_enabled: true } as IConfig);
|
||||
vi.spyOn(env.server.clients.email, 'send').mockImplementation(
|
||||
async (to, template, values) => {
|
||||
if (template === 'email_verification_link')
|
||||
|
||||
@@ -154,9 +154,8 @@ export class EventsController extends PuterController {
|
||||
// -- Cross-user key-value handles --------------------------------
|
||||
|
||||
/**
|
||||
* POST /events/kv-handles — hand another user a region of this account's
|
||||
* key-value data, `watch` unless the body asks for more. The older name for
|
||||
* `POST /kv/shares`, keeping its own body and response shapes.
|
||||
* POST /events/kv-handles — hand another user a watchable region of this
|
||||
* account's key-value data.
|
||||
*
|
||||
* An account session, or an app session holding a `manage:` delegation on
|
||||
* the region — the service tells those apart and gates each rather than the
|
||||
@@ -168,32 +167,12 @@ export class EventsController extends PuterController {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
requireVerified: true,
|
||||
})
|
||||
async mintKvHandle(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const body = bodyRecord(req);
|
||||
const share = await this.services.kvShare.mintKvHandle(
|
||||
actor,
|
||||
{
|
||||
prefix: body.prefix,
|
||||
recipient: {
|
||||
username:
|
||||
typeof body.granteeUsername === 'string'
|
||||
? body.granteeUsername
|
||||
: undefined,
|
||||
uid:
|
||||
typeof body.granteeUid === 'string'
|
||||
? body.granteeUid
|
||||
: undefined,
|
||||
},
|
||||
access: body.access,
|
||||
appUid: body.appUid,
|
||||
},
|
||||
// Callers of this route mint again to get the same handle back.
|
||||
{ defaultAccess: 'watch', keepLevelUnlessNamed: true },
|
||||
res.json(
|
||||
await this.services.events.mintKvHandle(actor, bodyRecord(req)),
|
||||
);
|
||||
res.json({ handle: share.handle, prefix: share.prefix });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -210,7 +189,7 @@ export class EventsController extends PuterController {
|
||||
const actor = this.#requireActor(req);
|
||||
const query = (req.query ?? {}) as Record<string, unknown>;
|
||||
|
||||
const page = await this.services.kvShare.listKvHandles(actor, {
|
||||
const page = await this.services.events.listKvHandles(actor, {
|
||||
limit: normalizeLimit(query.limit, {
|
||||
cap: KV_HANDLE_LIST_LIMIT_CAP,
|
||||
}),
|
||||
@@ -219,10 +198,7 @@ export class EventsController extends PuterController {
|
||||
});
|
||||
|
||||
res.json({
|
||||
items: page.items.map(({ holderUsername, ...row }) => ({
|
||||
...row,
|
||||
granteeUsername: holderUsername,
|
||||
})),
|
||||
items: page.items,
|
||||
...(page.cursor ? { cursor: page.cursor } : {}),
|
||||
...(page.total !== undefined ? { total: page.total } : {}),
|
||||
});
|
||||
@@ -241,10 +217,7 @@ export class EventsController extends PuterController {
|
||||
async revokeKvHandle(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
res.json(
|
||||
await this.services.kvShare.revokeKvHandle(
|
||||
actor,
|
||||
req.params.handle,
|
||||
),
|
||||
await this.services.events.revokeKvHandle(actor, req.params.handle),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -163,16 +163,6 @@ export const EVENTS_ACK_LIMIT = userWindow('events:ack', 600);
|
||||
*/
|
||||
export const EVENTS_KV_HANDLE_LIMIT = userWindow('events:kvHandles', 60);
|
||||
|
||||
/**
|
||||
* `kv.share.received` notices per hour, per owner and holder. A share past it
|
||||
* is still made, without a notice.
|
||||
*/
|
||||
export const KV_SHARE_NOTICE_LIMIT = userWindow(
|
||||
'kvShare:notices',
|
||||
5,
|
||||
60 * 60_000,
|
||||
);
|
||||
|
||||
/**
|
||||
* Live share handles one account may hold out at a time, across every app.
|
||||
*
|
||||
|
||||
@@ -27,7 +27,6 @@ import { EventsController } from './events/EventsController.js';
|
||||
import { FSController } from './fs/FSController.js';
|
||||
import { HomepageController } from './homepage/HomepageController.js';
|
||||
import { HostingController } from './hosting/HostingController.js';
|
||||
import { KvShareController } from './kvShare/KvShareController.js';
|
||||
import { LegacyFSController } from './fs/LegacyFSController.js';
|
||||
import { NotificationController } from './notification/NotificationController.js';
|
||||
import { OIDCController } from './oidc/OIDCController.js';
|
||||
@@ -59,7 +58,6 @@ export const puterControllers = {
|
||||
broadcast: BroadcastController,
|
||||
notification: NotificationController,
|
||||
events: EventsController,
|
||||
kvShare: KvShareController,
|
||||
share: ShareController,
|
||||
webdav: WebDAVController,
|
||||
oidc: OIDCController,
|
||||
|
||||
@@ -1,478 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import {
|
||||
createTestUser,
|
||||
setupPuterTestEnv,
|
||||
type PuterTestEnv,
|
||||
type TestUserCredentials,
|
||||
} from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { EVENTS_KV_HANDLE_LIMIT } from '../events/limits.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
|
||||
const call = (
|
||||
env: PuterTestEnv,
|
||||
method: string,
|
||||
path: string,
|
||||
token?: string,
|
||||
body?: unknown,
|
||||
) =>
|
||||
fetch(new URL(path, env.apiOrigin), {
|
||||
method,
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
...(token ? { authorization: `Bearer ${token}` } : {}),
|
||||
},
|
||||
...(body === undefined ? {} : { body: JSON.stringify(body) }),
|
||||
});
|
||||
|
||||
const resetBudgets = async (env: PuterTestEnv) => {
|
||||
for (const name of [env.users.user.username, env.users.other.username]) {
|
||||
const user = await env.server.stores.user.getByUsername(name);
|
||||
await env.server.clients.redis.del(
|
||||
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user!.id}`,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
describe('/kv/shares', () => {
|
||||
let env: PuterTestEnv;
|
||||
let ownerUuid: string;
|
||||
const req = (
|
||||
method: string,
|
||||
path: string,
|
||||
token?: string,
|
||||
body?: unknown,
|
||||
) => call(env, method, path, token, body);
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv({
|
||||
events: { enabled: true, kvHandles: true },
|
||||
kv: { shareHandles: true },
|
||||
unlimitedMetering: true,
|
||||
} as IConfig);
|
||||
const owner = await env.server.stores.user.getByUsername(
|
||||
env.users.user.username,
|
||||
);
|
||||
ownerUuid = owner!.uuid as string;
|
||||
}, BOOT_TIMEOUT_MS);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetBudgets(env);
|
||||
});
|
||||
|
||||
const shareWithOther = async (body: Record<string, unknown> = {}) => {
|
||||
const res = await req('POST', '/kv/shares', env.users.user.token, {
|
||||
prefix: `http-${uuidv4().slice(0, 8)}:`,
|
||||
recipient: env.users.other.username,
|
||||
...body,
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
return (await res.json()) as Record<string, unknown>;
|
||||
};
|
||||
|
||||
it('shares at read by default and answers in the owner`s shape', async () => {
|
||||
const shared = await shareWithOther({ label: 'Groceries' });
|
||||
expect(Object.keys(shared).sort()).toEqual(
|
||||
[
|
||||
'access',
|
||||
'appUid',
|
||||
'createdAt',
|
||||
'handle',
|
||||
'holder',
|
||||
'label',
|
||||
'prefix',
|
||||
].sort(),
|
||||
);
|
||||
expect(shared).toMatchObject({
|
||||
access: 'read',
|
||||
appUid: null,
|
||||
holder: env.users.other.username,
|
||||
label: 'Groceries',
|
||||
});
|
||||
});
|
||||
|
||||
it('takes the recipient as an object too, and refuses an address', async () => {
|
||||
await shareWithOther({
|
||||
recipient: { username: env.users.other.username },
|
||||
access: 'write',
|
||||
});
|
||||
const refused = await req('POST', '/kv/shares', env.users.user.token, {
|
||||
prefix: 'address:',
|
||||
recipient: '[email protected]',
|
||||
});
|
||||
expect(refused.status).toBe(400);
|
||||
await expect(refused.json()).resolves.toMatchObject({
|
||||
code: 'invalid_recipient',
|
||||
});
|
||||
});
|
||||
|
||||
it('lists what was shared out, and what was shared with the holder without the owner', async () => {
|
||||
const shared = await shareWithOther({ access: 'write', label: 'Trip' });
|
||||
|
||||
const byMe = await req(
|
||||
'GET',
|
||||
'/kv/shares?limit=200',
|
||||
env.users.user.token,
|
||||
);
|
||||
const byMePage = (await byMe.json()) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
};
|
||||
expect(
|
||||
byMePage.items.find((row) => row.handle === shared.handle),
|
||||
).toEqual({
|
||||
handle: shared.handle,
|
||||
prefix: shared.prefix,
|
||||
access: 'write',
|
||||
appUid: null,
|
||||
holder: env.users.other.username,
|
||||
label: 'Trip',
|
||||
createdAt: shared.createdAt,
|
||||
revokedAt: null,
|
||||
revokedBy: null,
|
||||
});
|
||||
|
||||
const withMe = await req(
|
||||
'GET',
|
||||
'/kv/shares/with-me?limit=200',
|
||||
env.users.other.token,
|
||||
);
|
||||
const withMeText = await withMe.text();
|
||||
const withMePage = JSON.parse(withMeText) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
};
|
||||
expect(
|
||||
withMePage.items.find((row) => row.handle === shared.handle),
|
||||
).toEqual({
|
||||
handle: shared.handle,
|
||||
access: 'write',
|
||||
appUid: null,
|
||||
label: 'Trip',
|
||||
createdAt: shared.createdAt,
|
||||
});
|
||||
expect(withMeText).not.toContain(env.users.user.username);
|
||||
expect(withMeText).not.toContain(ownerUuid);
|
||||
expect(withMeText).not.toContain(String(shared.prefix));
|
||||
});
|
||||
|
||||
it('lets the holder unshare, and shows the owner they did', async () => {
|
||||
const shared = await shareWithOther();
|
||||
const removed = await req(
|
||||
'DELETE',
|
||||
`/kv/shares/${shared.handle}`,
|
||||
env.users.other.token,
|
||||
);
|
||||
expect(removed.status).toBe(200);
|
||||
await expect(removed.json()).resolves.toMatchObject({
|
||||
handle: shared.handle,
|
||||
});
|
||||
|
||||
const byMe = (await (
|
||||
await req('GET', '/kv/shares?limit=200', env.users.user.token)
|
||||
).json()) as { items: Array<Record<string, unknown>> };
|
||||
expect(
|
||||
byMe.items.find((row) => row.handle === shared.handle),
|
||||
).toMatchObject({ revokedBy: 'holder' });
|
||||
});
|
||||
|
||||
it('keeps the owner and prefix out of the holder`s grant audit after they unshare', async () => {
|
||||
const shared = await shareWithOther({ access: 'write' });
|
||||
const segment = String(shared.prefix).slice(0, -1);
|
||||
const other = await env.server.stores.user.getByUsername(
|
||||
env.users.other.username,
|
||||
);
|
||||
// A row an earlier build filed under the holder.
|
||||
await env.server.stores.permission.auditUserUserPerm({
|
||||
holder_user_id: other!.id,
|
||||
issuer_user_id: other!.id,
|
||||
permission: `kv-share-write:${ownerUuid}:os-global:${segment}`,
|
||||
action: 'revoke',
|
||||
reason: 'kv share handle unshared',
|
||||
});
|
||||
const removed = await req(
|
||||
'DELETE',
|
||||
`/kv/shares/${shared.handle}`,
|
||||
env.users.other.token,
|
||||
);
|
||||
expect(removed.status).toBe(200);
|
||||
|
||||
// The revoke is filed under the owner, who granted it.
|
||||
const auditOf = async (token: string) => {
|
||||
const res = await req('GET', '/share/audit?limit=200', token);
|
||||
expect(res.status).toBe(200);
|
||||
return await res.text();
|
||||
};
|
||||
await vi.waitFor(async () => {
|
||||
const owners = JSON.parse(await auditOf(env.users.user.token)) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
};
|
||||
expect(owners.items).toContainEqual(
|
||||
expect.objectContaining({
|
||||
action: 'revoke',
|
||||
issuer: env.users.user.username,
|
||||
holder: env.users.other.username,
|
||||
permission: expect.stringContaining(segment),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
const holders = await auditOf(env.users.other.token);
|
||||
expect(holders).not.toContain(ownerUuid);
|
||||
expect(holders).not.toContain(segment);
|
||||
});
|
||||
|
||||
it('lets the owner revoke', async () => {
|
||||
const shared = await shareWithOther();
|
||||
const removed = await req(
|
||||
'DELETE',
|
||||
`/kv/shares/${shared.handle}`,
|
||||
env.users.user.token,
|
||||
);
|
||||
expect(removed.status).toBe(200);
|
||||
});
|
||||
|
||||
it('answers anyone else, and an unknown handle, with the same 404', async () => {
|
||||
const shared = await shareWithOther();
|
||||
const stranger = await createTestUser(env.server, {
|
||||
username: `kvs-${uuidv4().slice(0, 8)}`,
|
||||
password: 'pw-test-1234',
|
||||
});
|
||||
const strangers = await req(
|
||||
'DELETE',
|
||||
`/kv/shares/${shared.handle}`,
|
||||
stranger.token,
|
||||
);
|
||||
const unknownHandle = `kvh-${uuidv4()}`;
|
||||
const unknown = await req(
|
||||
'DELETE',
|
||||
`/kv/shares/${unknownHandle}`,
|
||||
env.users.user.token,
|
||||
);
|
||||
expect(strangers.status).toBe(404);
|
||||
expect(unknown.status).toBe(404);
|
||||
const a = (await strangers.json()) as Record<string, unknown>;
|
||||
const b = (await unknown.json()) as Record<string, unknown>;
|
||||
expect(a.code).toBe(b.code);
|
||||
expect(a.message).toBe(`No such handle: ${shared.handle}`);
|
||||
expect(b.message).toBe(`No such handle: ${unknownHandle}`);
|
||||
});
|
||||
|
||||
it('turns an anonymous caller away', async () => {
|
||||
const res = await req('GET', '/kv/shares');
|
||||
expect(res.status).toBeGreaterThanOrEqual(400);
|
||||
});
|
||||
});
|
||||
|
||||
describe('/events/kv-handles beside /kv/shares', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv({
|
||||
events: { enabled: true, kvHandles: true },
|
||||
kv: { shareHandles: true },
|
||||
unlimitedMetering: true,
|
||||
} as IConfig);
|
||||
}, BOOT_TIMEOUT_MS);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
it('keeps its own request and response shapes, at watch by default', async () => {
|
||||
const minted = await call(
|
||||
env,
|
||||
'POST',
|
||||
'/events/kv-handles',
|
||||
env.users.user.token,
|
||||
{
|
||||
granteeUsername: env.users.other.username,
|
||||
prefix: 'alias-shape:',
|
||||
},
|
||||
);
|
||||
expect(minted.status).toBe(200);
|
||||
const body = (await minted.json()) as Record<string, unknown>;
|
||||
expect(body).toEqual({ handle: body.handle, prefix: 'alias-shape:' });
|
||||
|
||||
const listed = (await (
|
||||
await call(
|
||||
env,
|
||||
'GET',
|
||||
'/events/kv-handles?limit=200',
|
||||
env.users.user.token,
|
||||
)
|
||||
).json()) as { items: Array<Record<string, unknown>> };
|
||||
const row = listed.items.find((one) => one.handle === body.handle);
|
||||
expect(row).toMatchObject({
|
||||
prefix: 'alias-shape:',
|
||||
appUid: 'os-global',
|
||||
granteeUsername: env.users.other.username,
|
||||
access: 'watch',
|
||||
revokedBy: null,
|
||||
});
|
||||
expect(row).not.toHaveProperty('holder');
|
||||
|
||||
// The alias revoke stays the owner's: the holder is told it is absent.
|
||||
const byHolder = await call(
|
||||
env,
|
||||
'DELETE',
|
||||
`/events/kv-handles/${body.handle}`,
|
||||
env.users.other.token,
|
||||
);
|
||||
expect(byHolder.status).toBe(404);
|
||||
});
|
||||
|
||||
it('hands back a share it finds at the level it has, unless the body names one', async () => {
|
||||
const shared = (await (
|
||||
await call(env, 'POST', '/kv/shares', env.users.user.token, {
|
||||
prefix: 'alias-keep:',
|
||||
recipient: env.users.other.username,
|
||||
access: 'write',
|
||||
})
|
||||
).json()) as Record<string, unknown>;
|
||||
const accessOf = async () => {
|
||||
const listed = (await (
|
||||
await call(
|
||||
env,
|
||||
'GET',
|
||||
'/kv/shares?limit=200',
|
||||
env.users.user.token,
|
||||
)
|
||||
).json()) as { items: Array<Record<string, unknown>> };
|
||||
return listed.items.find((one) => one.handle === shared.handle)
|
||||
?.access;
|
||||
};
|
||||
const mintAlias = (extra: Record<string, unknown> = {}) =>
|
||||
call(env, 'POST', '/events/kv-handles', env.users.user.token, {
|
||||
granteeUsername: env.users.other.username,
|
||||
prefix: 'alias-keep:',
|
||||
...extra,
|
||||
});
|
||||
|
||||
const again = await mintAlias();
|
||||
expect(again.status).toBe(200);
|
||||
await expect(again.json()).resolves.toEqual({
|
||||
handle: shared.handle,
|
||||
prefix: 'alias-keep:',
|
||||
});
|
||||
expect(await accessOf()).toBe('write');
|
||||
|
||||
expect((await mintAlias({ access: 'read' })).status).toBe(200);
|
||||
expect(await accessOf()).toBe('read');
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Sharing asks for a confirmed email and nothing else. Booted with an SMS
|
||||
* provider configured, so a phone requirement would bite here if one were ever
|
||||
* added to these routes.
|
||||
*/
|
||||
describe('verification on the mint routes', () => {
|
||||
let env: PuterTestEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv({
|
||||
events: { enabled: true, kvHandles: true },
|
||||
kv: { shareHandles: true },
|
||||
unlimitedMetering: true,
|
||||
strict_email_verification_required: true,
|
||||
prelude: { apiKey: 'test-key' },
|
||||
} as IConfig);
|
||||
}, BOOT_TIMEOUT_MS);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
const makeUser = async (
|
||||
patch: Record<string, unknown> = {},
|
||||
): Promise<TestUserCredentials> => {
|
||||
const username = `kvv${uuidv4().slice(0, 8)}`;
|
||||
const creds = await createTestUser(env.server, {
|
||||
username,
|
||||
password: 'pw-test-1234',
|
||||
});
|
||||
if (Object.keys(patch).length > 0) {
|
||||
const row = await env.server.stores.user.getByUsername(username);
|
||||
await env.server.stores.user.update(row!.id, patch);
|
||||
}
|
||||
return creds;
|
||||
};
|
||||
|
||||
const mintBoth = (owner: TestUserCredentials, prefix: string) =>
|
||||
Promise.all([
|
||||
call(env, 'POST', '/kv/shares', owner.token, {
|
||||
prefix: `${prefix}a:`,
|
||||
recipient: env.users.other.username,
|
||||
}),
|
||||
call(env, 'POST', '/events/kv-handles', owner.token, {
|
||||
prefix: `${prefix}b:`,
|
||||
granteeUsername: env.users.other.username,
|
||||
}),
|
||||
]);
|
||||
|
||||
it('refuses an account whose email is not confirmed, on both routes', async () => {
|
||||
const owner = await makeUser();
|
||||
for (const res of await mintBoth(owner, 'unconfirmed')) {
|
||||
expect(res.status).toBe(403);
|
||||
await expect(res.json()).resolves.toMatchObject({
|
||||
code: 'account_is_not_verified',
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('lets a confirmed account share with no phone and no card', async () => {
|
||||
const owner = await makeUser({
|
||||
email: `kvv-${uuidv4().slice(0, 8)}@example.invalid`,
|
||||
email_confirmed: 1,
|
||||
});
|
||||
for (const res of await mintBoth(owner, 'confirmed'))
|
||||
expect(res.status).toBe(200);
|
||||
});
|
||||
|
||||
it('never gates listing or unsharing', async () => {
|
||||
const owner = await makeUser();
|
||||
for (const path of ['/kv/shares', '/kv/shares/with-me'])
|
||||
expect((await call(env, 'GET', path, owner.token)).status).toBe(
|
||||
200,
|
||||
);
|
||||
const removed = await call(
|
||||
env,
|
||||
'DELETE',
|
||||
`/kv/shares/kvh-${uuidv4()}`,
|
||||
owner.token,
|
||||
);
|
||||
expect(removed.status).toBe(404);
|
||||
});
|
||||
});
|
||||
@@ -1,172 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Request, Response } from 'express';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { bodyRecord } from '../../core/http/requestBody.js';
|
||||
import { parseKvShareRecipient } from '../../services/kvShare/kvShares.js';
|
||||
import type { KvShareHandleListOptions } from '../../stores/events/KvShareHandleStore.js';
|
||||
import { KV_HANDLE_LIST_LIMIT_CAP } from '../../stores/events/KvShareHandleStore.js';
|
||||
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
|
||||
import { normalizeLimit } from '../../util/pagination.js';
|
||||
import { EVENTS_LIST_LIMIT } from '../events/limits.js';
|
||||
import { PuterController } from '../types.js';
|
||||
|
||||
/** The app-less namespace is no app, so the wire says `null` for it. */
|
||||
const publicAppUid = (appUid: string): string | null =>
|
||||
appUid === KV_GLOBAL_APP_KEY ? null : appUid;
|
||||
|
||||
const pageOptions = (req: Request): KvShareHandleListOptions => {
|
||||
const query = (req.query ?? {}) as Record<string, unknown>;
|
||||
return {
|
||||
limit: normalizeLimit(query.limit, { cap: KV_HANDLE_LIST_LIMIT_CAP }),
|
||||
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
|
||||
includeTotal: query.includeTotal === 'true',
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Sharing a region of one account's key-value data with another account.
|
||||
*
|
||||
* Gated like the events surface: `subdomain: 'api'` plus `allowAccessToken`,
|
||||
* with app-ness decided from `effectiveApp` in the service, which is where the
|
||||
* owner, holder, delegation and namespace rules live. Only sharing asks for a
|
||||
* confirmed email; listing, revoking and unsharing only narrow or report.
|
||||
*/
|
||||
@Controller('/kv')
|
||||
export class KvShareController extends PuterController {
|
||||
/**
|
||||
* POST /kv/shares — share a region with another account, `read` unless the
|
||||
* body asks otherwise. Sharing the same region with the same account again
|
||||
* changes its level or label in place.
|
||||
*/
|
||||
@Post('/shares', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
requireVerified: true,
|
||||
})
|
||||
async share(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const body = bodyRecord(req);
|
||||
const share = await this.services.kvShare.mintKvHandle(
|
||||
actor,
|
||||
{
|
||||
prefix: body.prefix,
|
||||
recipient: parseKvShareRecipient(body.recipient),
|
||||
access: body.access,
|
||||
appUid: body.appUid,
|
||||
label: body.label,
|
||||
},
|
||||
{ defaultAccess: 'read' },
|
||||
);
|
||||
res.json({
|
||||
handle: share.handle,
|
||||
prefix: share.prefix,
|
||||
access: share.access,
|
||||
appUid: publicAppUid(share.appUid),
|
||||
holder: share.holderUsername,
|
||||
label: share.label,
|
||||
createdAt: share.createdAt,
|
||||
});
|
||||
}
|
||||
|
||||
/** GET /kv/shares — what this account has shared out, retired included. */
|
||||
@Get('/shares', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
rateLimit: EVENTS_LIST_LIMIT,
|
||||
})
|
||||
async listSharedByMe(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const page = await this.services.kvShare.listKvHandles(
|
||||
actor,
|
||||
pageOptions(req),
|
||||
);
|
||||
res.json({
|
||||
items: page.items.map((row) => ({
|
||||
handle: row.handle,
|
||||
prefix: row.prefix,
|
||||
access: row.access,
|
||||
appUid: publicAppUid(row.appUid),
|
||||
holder: row.holderUsername,
|
||||
label: row.label,
|
||||
createdAt: row.createdAt,
|
||||
revokedAt: row.revokedAt,
|
||||
revokedBy: row.revokedBy,
|
||||
})),
|
||||
...(page.cursor ? { cursor: page.cursor } : {}),
|
||||
...(page.total !== undefined ? { total: page.total } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
/** GET /kv/shares/with-me — live shares this account holds. */
|
||||
@Get('/shares/with-me', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
rateLimit: EVENTS_LIST_LIMIT,
|
||||
})
|
||||
async listSharedWithMe(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const page = await this.services.kvShare.listKvSharesWithMe(
|
||||
actor,
|
||||
pageOptions(req),
|
||||
);
|
||||
res.json({
|
||||
items: page.items.map((row) => ({
|
||||
handle: row.handle,
|
||||
access: row.access,
|
||||
appUid: publicAppUid(row.appUid),
|
||||
label: row.label,
|
||||
createdAt: row.createdAt,
|
||||
})),
|
||||
...(page.cursor ? { cursor: page.cursor } : {}),
|
||||
...(page.total !== undefined ? { total: page.total } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* DELETE /kv/shares/:handle — the owner revokes, the holder gives it up,
|
||||
* and anyone else is told it is not there.
|
||||
*/
|
||||
@Delete('/shares/:handle', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
})
|
||||
async unshare(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
res.json(
|
||||
await this.services.kvShare.removeKvShare(actor, req.params.handle),
|
||||
);
|
||||
}
|
||||
|
||||
#requireActor(req: Request): Actor {
|
||||
const actor = req.actor;
|
||||
if (!actor?.user)
|
||||
throw new HttpError(401, 'Unauthorized', {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
return actor;
|
||||
}
|
||||
}
|
||||
@@ -23,7 +23,6 @@ import {
|
||||
makeActor,
|
||||
SYSTEM_ACTOR,
|
||||
userRelatedActor,
|
||||
viaWorkerOf,
|
||||
type Actor,
|
||||
} from './actor';
|
||||
|
||||
@@ -112,28 +111,3 @@ describe('assertResolvedActor', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('viaWorkerOf', () => {
|
||||
const user = { uuid: 'u-1', id: 1 };
|
||||
|
||||
it('reads a worker session or an events handler as server-side', () => {
|
||||
expect(
|
||||
viaWorkerOf(
|
||||
makeActor({ user, session: { uid: 's-1', kind: 'worker' } }),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(viaWorkerOf(makeActor({ user, handlerDepth: 1 }))).toBe(true);
|
||||
});
|
||||
|
||||
it('reads a browser session, an app or no actor at all as not', () => {
|
||||
expect(
|
||||
viaWorkerOf(
|
||||
makeActor({ user, session: { uid: 's-1', kind: 'web' } }),
|
||||
),
|
||||
).toBe(false);
|
||||
expect(viaWorkerOf(makeActor({ user, app: { uid: 'app-1' } }))).toBe(
|
||||
false,
|
||||
);
|
||||
expect(viaWorkerOf(undefined)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -239,10 +239,3 @@ export const userRelatedActor = (actor: Actor): Actor => {
|
||||
/** How many events handler runs led to a write by this actor. */
|
||||
export const handlerDepthOf = (actor: Actor | undefined | null): number =>
|
||||
actor?.handlerDepth ?? 0;
|
||||
|
||||
/**
|
||||
* Whether a write by this actor was made by server-side code running as the
|
||||
* account: a worker session, or an events handler.
|
||||
*/
|
||||
export const viaWorkerOf = (actor: Actor | undefined | null): boolean =>
|
||||
actor?.session?.kind === 'worker' || handlerDepthOf(actor) > 0;
|
||||
@@ -67,12 +67,6 @@ export interface KnownContextFields {
|
||||
* receive stops early and is never metered.
|
||||
*/
|
||||
abortSignal: AbortSignal;
|
||||
/**
|
||||
* Aborts the chat driver's upstream request. A client disconnect doesn't
|
||||
* fire it: the upstream bills a request it has accepted either way, so the
|
||||
* driver lets it finish and meters what it reports.
|
||||
*/
|
||||
upstreamSignal: AbortSignal;
|
||||
/**
|
||||
* When true, `ChatCompletionDriver.complete` treats a request-level
|
||||
* upstream 4xx as final instead of falling back to another provider — set
|
||||
@@ -90,7 +84,6 @@ const KNOWN_KEY_FLAGS: Record<keyof KnownContextFields, true> = {
|
||||
requestId: true,
|
||||
driverName: true,
|
||||
abortSignal: true,
|
||||
upstreamSignal: true,
|
||||
strictUpstreamErrors: true,
|
||||
};
|
||||
const KNOWN_KEYS: ReadonlySet<string> = new Set(Object.keys(KNOWN_KEY_FLAGS));
|
||||
|
||||
@@ -128,18 +128,6 @@ describe('createWwwRedirect', () => {
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('leaves www. alone on a host outside our domains', () => {
|
||||
const req = makeReq({
|
||||
subdomains: ['com', 'example', 'www'],
|
||||
host: 'www.example.com',
|
||||
});
|
||||
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
|
||||
true;
|
||||
const { out, next } = run(createWwwRedirect(config), req);
|
||||
expect(out.redirectArgs).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("passes through when config.domain isn't configured (no target to redirect to)", () => {
|
||||
const { out, next } = run(
|
||||
createWwwRedirect({} as IConfig),
|
||||
@@ -891,18 +879,6 @@ describe('createNativeAppStatic', () => {
|
||||
const withRoot = () =>
|
||||
createNativeAppStatic({ native_apps_root: root } as unknown as IConfig);
|
||||
|
||||
it('leaves a native app name alone on a host outside our domains', async () => {
|
||||
const req = staticReq({
|
||||
subdomains: ['com', 'example', 'editor'],
|
||||
path: '/index.html',
|
||||
});
|
||||
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
|
||||
true;
|
||||
const { out, next } = await runStatic(withRoot(), req);
|
||||
expect(out.sent).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('is a no-op when native_apps_root is unset', async () => {
|
||||
const { out, next } = await runStatic(
|
||||
createNativeAppStatic(config),
|
||||
|
||||
@@ -207,8 +207,6 @@ const dispatchClaimed = async (
|
||||
export const createWwwRedirect = (config: IConfig): RequestHandler => {
|
||||
const domain = (config.domain ?? '').toLowerCase();
|
||||
return (req, res, next) => {
|
||||
// `www.` on a host outside our domains belongs to that host's site.
|
||||
if (req.is_custom_domain) return next();
|
||||
const active = req.subdomains?.[req.subdomains.length - 1] ?? '';
|
||||
if (active !== 'www') return next();
|
||||
if (!domain) return next();
|
||||
@@ -307,7 +305,6 @@ export const createNativeAppStatic = (config: IConfig): RequestHandler => {
|
||||
return (_req, _res, next) => next();
|
||||
}
|
||||
return async (req, res, next) => {
|
||||
if (req.is_custom_domain) return next();
|
||||
const active = (
|
||||
req.subdomains?.[req.subdomains.length - 1] ?? ''
|
||||
).toLowerCase();
|
||||
|
||||
@@ -950,8 +950,7 @@ describe('resolvePrivateIdentity', () => {
|
||||
expect(out.userUid).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ignores an app-issued access token even for the target app', async () => {
|
||||
// A scoped token (e.g. a read URL) is not the account, whoever issued it.
|
||||
it('accepts an app-issued access token minted for the target app', async () => {
|
||||
const user = await makeUser();
|
||||
const targetUid = `app-target-${uuidv4()}`;
|
||||
const issuer = makeActor({
|
||||
@@ -971,11 +970,11 @@ describe('resolvePrivateIdentity', () => {
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
expectedAppUid: targetUid,
|
||||
});
|
||||
expect(out.source).toBe('none');
|
||||
expect(out.userUid).toBeUndefined();
|
||||
expect(out.source).toBe('session-cookie');
|
||||
expect(out.userUid).toBe(user.uuid);
|
||||
});
|
||||
|
||||
it('ignores a user-issued scoped access token', async () => {
|
||||
it('still accepts a user-issued access token, which is portable', async () => {
|
||||
const user = await makeUser();
|
||||
const issuer = makeActor({ user: { uuid: user.uuid } });
|
||||
const out = await resolvePrivateIdentity({
|
||||
@@ -984,46 +983,7 @@ describe('resolvePrivateIdentity', () => {
|
||||
actor: makeActor({
|
||||
user: { uuid: user.uuid },
|
||||
accessToken: { uid: `tok-${uuidv4()}`, issuer },
|
||||
}),
|
||||
}),
|
||||
authService,
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
expectedAppUid: `app-target-${uuidv4()}`,
|
||||
});
|
||||
expect(out.source).toBe('none');
|
||||
expect(out.userUid).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ignores a scoped access token when the host has no app', async () => {
|
||||
const user = await makeUser();
|
||||
const issuer = makeActor({ user: { uuid: user.uuid } });
|
||||
const out = await resolvePrivateIdentity({
|
||||
req: reqOf({
|
||||
cookies: {},
|
||||
actor: makeActor({
|
||||
user: { uuid: user.uuid },
|
||||
accessToken: { uid: `tok-${uuidv4()}`, issuer },
|
||||
}),
|
||||
}),
|
||||
authService,
|
||||
sessionCookieName: 'puter_auth_token',
|
||||
});
|
||||
expect(out.source).toBe('none');
|
||||
});
|
||||
|
||||
it('accepts a full-access token, which holds the account', async () => {
|
||||
const user = await makeUser();
|
||||
const issuer = makeActor({ user: { uuid: user.uuid } });
|
||||
const out = await resolvePrivateIdentity({
|
||||
req: reqOf({
|
||||
cookies: {},
|
||||
actor: makeActor({
|
||||
user: { uuid: user.uuid },
|
||||
accessToken: {
|
||||
uid: `tok-${uuidv4()}`,
|
||||
issuer,
|
||||
fullAccess: true,
|
||||
},
|
||||
session: { uid: 'sess-1' },
|
||||
}),
|
||||
}),
|
||||
authService,
|
||||
|
||||
@@ -25,7 +25,7 @@ import {
|
||||
} from '../../../services/auth/AuthService';
|
||||
import type { AppStore } from '../../../stores/app/AppStore.js';
|
||||
import type { IConfig } from '../../../types';
|
||||
import { isAccessTokenActor, isAccountContext, type Actor } from '../../actor';
|
||||
import type { Actor } from '../../actor';
|
||||
|
||||
/**
|
||||
* Support helpers for the private-app access gate — ported from v1's
|
||||
@@ -196,12 +196,10 @@ export async function resolveOwnedAppForHostedSite(opts: {
|
||||
appStore: Pick<AppStore, 'listOwnedByIndexUrlCandidates'>;
|
||||
config: PrivateHostingConfig;
|
||||
requirePrivate?: boolean;
|
||||
/** Match against this hosting-domain host instead of the request's. */
|
||||
host?: string;
|
||||
}): Promise<AppLike | null> {
|
||||
if (!opts.site?.user_id) return null;
|
||||
|
||||
const host = normalizeHost(opts.host ?? opts.req.hostname);
|
||||
const host = normalizeHost(opts.req.hostname);
|
||||
if (!host) return null;
|
||||
|
||||
const hostedSubdomain = subdomainFromHost(host, [
|
||||
@@ -217,7 +215,7 @@ export async function resolveOwnedAppForHostedSite(opts: {
|
||||
const hostCandidates = new Set<string>();
|
||||
hostCandidates.add(host);
|
||||
const headerHost =
|
||||
!opts.host && typeof opts.req.headers?.host === 'string'
|
||||
typeof opts.req.headers?.host === 'string'
|
||||
? opts.req.headers.host.trim().toLowerCase()
|
||||
: '';
|
||||
if (headerHost) hostCandidates.add(headerHost);
|
||||
@@ -460,15 +458,11 @@ export async function resolvePrivateIdentity(opts: {
|
||||
* "No app anywhere in the chain" is the portable case, so read `effectiveApp` —
|
||||
* an app-issued access token has no `app` of its own and would otherwise pass
|
||||
* as a plain session on any private host.
|
||||
*
|
||||
* A scoped access token (a read URL, whoever issued it) is never the account
|
||||
* here: the hosted cookie minted from it would outlive the token's revocation.
|
||||
*/
|
||||
function actorMatchesExpectedApp(
|
||||
actor: Actor,
|
||||
expectedAppUid: string | undefined,
|
||||
): boolean {
|
||||
if (isAccessTokenActor(actor) && !isAccountContext(actor)) return false;
|
||||
if (!expectedAppUid) return true;
|
||||
const app = actor.effectiveApp;
|
||||
if (!app) return true;
|
||||
|
||||
@@ -1030,133 +1030,6 @@ describe('createPuterSiteMiddleware — site.access.check', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('createPuterSiteMiddleware — hosts outside the configured domains', () => {
|
||||
const claims = new Map<string, string>();
|
||||
const resolveHost = (
|
||||
_key: string,
|
||||
data: { host: string; result: { subdomain: string | null } },
|
||||
) => {
|
||||
const sub = claims.get(data.host);
|
||||
if (sub) data.result.subdomain = sub;
|
||||
};
|
||||
|
||||
beforeAll(() => {
|
||||
server.clients.event.on('site.resolveHost', resolveHost as never);
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
server.clients.event.off('site.resolveHost', resolveHost as never);
|
||||
});
|
||||
|
||||
const customReq = (hostname: string, path = '/index.html') => {
|
||||
const req = makeReq({ hostname, path });
|
||||
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
|
||||
true;
|
||||
return req;
|
||||
};
|
||||
|
||||
const makeSite = async (opts: { suspended?: boolean } = {}) => {
|
||||
const owner = await makeUserWithHome();
|
||||
const homePath = `/${owner.username}`;
|
||||
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
|
||||
const sub = `cust-${Math.random().toString(36).slice(2, 8)}`;
|
||||
await server.stores.subdomain.create({
|
||||
userId: owner.id,
|
||||
subdomain: sub,
|
||||
rootDirId: homeEntry!.id,
|
||||
});
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/index.html`,
|
||||
Buffer.from('<html>custom</html>'),
|
||||
'text/html',
|
||||
);
|
||||
if (opts.suspended) {
|
||||
await server.stores.user.update(owner.id, { suspended: 1 });
|
||||
}
|
||||
const host = `${sub}.example.test`;
|
||||
claims.set(host, sub);
|
||||
return { owner, sub, host };
|
||||
};
|
||||
|
||||
it('serves the claimed site on its host', async () => {
|
||||
const { host } = await makeSite();
|
||||
const { out, next } = await runMiddleware(
|
||||
buildMiddleware(),
|
||||
customReq(host),
|
||||
);
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
expect(out.statusCode).toBe(200);
|
||||
expect(String(out.body)).toBe('<html>custom</html>');
|
||||
});
|
||||
|
||||
it('404s an unclaimed host instead of falling through to the API/GUI', async () => {
|
||||
const { out, next } = await runMiddleware(
|
||||
buildMiddleware(),
|
||||
customReq('nobody-claims-this.example.test'),
|
||||
);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(out.body).toBe('Site not found');
|
||||
});
|
||||
|
||||
it('404s when a listener throws', async () => {
|
||||
const host = 'throws.example.test';
|
||||
const boom = (_key: string, data: { host: string }) => {
|
||||
if (data.host === host) throw new Error('boom');
|
||||
};
|
||||
server.clients.event.on('site.resolveHost', boom as never);
|
||||
try {
|
||||
const { out, next } = await runMiddleware(
|
||||
buildMiddleware(),
|
||||
customReq(host),
|
||||
);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
expect(out.statusCode).toBe(404);
|
||||
} finally {
|
||||
server.clients.event.off('site.resolveHost', boom as never);
|
||||
}
|
||||
});
|
||||
|
||||
it('404s a claimed site whose owner is suspended', async () => {
|
||||
const { host } = await makeSite({ suspended: true });
|
||||
const { out } = await runMiddleware(buildMiddleware(), customReq(host));
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(out.body).toBe('Subdomain not found');
|
||||
});
|
||||
|
||||
it('404s a private app rather than serving it ungated', async () => {
|
||||
const { owner, sub, host } = await makeSite();
|
||||
await server.clients.db.write(
|
||||
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`, `is_private`) VALUES (?, ?, ?, ?, ?, ?)',
|
||||
[
|
||||
`app-${uuidv4()}`,
|
||||
sub,
|
||||
sub,
|
||||
`http://${sub}.site.puter.localhost/`,
|
||||
owner.id,
|
||||
1,
|
||||
],
|
||||
);
|
||||
const { out } = await runMiddleware(buildMiddleware(), customReq(host));
|
||||
expect(out.statusCode).toBe(404);
|
||||
expect(out.body).toBe('Site not found');
|
||||
expect(out.redirected).toBeUndefined();
|
||||
expect(out.cookies).toEqual({});
|
||||
});
|
||||
|
||||
it('passes the host through untouched when custom domains are not flagged', async () => {
|
||||
const { host } = await makeSite();
|
||||
const { out, next } = await runMiddleware(
|
||||
buildMiddleware(),
|
||||
makeReq({ hostname: host }),
|
||||
);
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
expect(out.statusCode).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('createPuterSiteMiddleware — .puter_site_config', () => {
|
||||
const setupSiteWithConfig = async (config: unknown) => {
|
||||
const owner = await makeUserWithHome();
|
||||
@@ -1381,7 +1254,12 @@ describe('createPuterSiteMiddleware — .puter_site_config', () => {
|
||||
errors: { '404': { file: '/index.html', status: 200 } },
|
||||
});
|
||||
const body = Buffer.from('real-page');
|
||||
await writeFile(owner.id, `${homePath}/page.html`, body, 'text/html');
|
||||
await writeFile(
|
||||
owner.id,
|
||||
`${homePath}/page.html`,
|
||||
body,
|
||||
'text/html',
|
||||
);
|
||||
|
||||
const mw = buildMiddleware();
|
||||
const { res, out } = makeRes();
|
||||
@@ -1495,9 +1373,9 @@ describe('createPuterSiteMiddleware — .puter_site_config', () => {
|
||||
|
||||
// SPA fallback still fires → cache served the deleted config.
|
||||
expect(second.out.statusCode).toBe(200);
|
||||
expect((second.out.body as Buffer | undefined)?.equals(body)).toBe(
|
||||
true,
|
||||
);
|
||||
expect(
|
||||
(second.out.body as Buffer | undefined)?.equals(body),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('ignores `errors` entries with status codes outside 4xx/5xx', async () => {
|
||||
|
||||
@@ -61,12 +61,12 @@ import {
|
||||
* - Private-app gate via `app.privateAccess.check` — marketplace extension
|
||||
* decides; default is denied + redirect to `app-center`
|
||||
* - Range / ETag / Last-Modified passthrough via `fsEntry.readContent`
|
||||
* - Hosts outside the configured domains (`custom_domains_enabled`) map to a site
|
||||
* via `site.resolveHost`; unclaimed ones 404. Public sites only.
|
||||
*
|
||||
* Deferred (not yet implemented):
|
||||
*
|
||||
* - `.at` username-based sites (UUIDv5-keyed `/user/Public`).
|
||||
* - Custom domains (subdomains table `domain` column) — requires host validation
|
||||
* to allow arbitrary hostnames first.
|
||||
*
|
||||
* Site config:
|
||||
*
|
||||
@@ -228,60 +228,29 @@ export const createPuterSiteMiddleware = (
|
||||
if (!host) return next();
|
||||
|
||||
const matched = matchHostingDomain(host);
|
||||
// A host outside every configured domain. Listeners may map it to a
|
||||
// site; an unclaimed one is answered here rather than falling through
|
||||
// to the API/GUI routers.
|
||||
const isCustomHost = !matched && Boolean(req.is_custom_domain);
|
||||
if (!matched && !isCustomHost) return next();
|
||||
if (!matched) return next();
|
||||
|
||||
let subdomain: string;
|
||||
if (isCustomHost) {
|
||||
const resolveEvent = {
|
||||
host,
|
||||
result: { subdomain: null as string | null },
|
||||
};
|
||||
try {
|
||||
await layers.clients.event.emitAndWait(
|
||||
'site.resolveHost',
|
||||
resolveEvent,
|
||||
{},
|
||||
);
|
||||
} catch {
|
||||
resolveEvent.result.subdomain = null;
|
||||
}
|
||||
const claimed = resolveEvent.result.subdomain;
|
||||
if (typeof claimed !== 'string' || !claimed) {
|
||||
res.status(404).type('text/plain').send('Site not found');
|
||||
// Bare hosting domain (e.g. `puter.site`) → redirect to the main site.
|
||||
if (host === matched) {
|
||||
if (domain) {
|
||||
res.redirect(302, `${req.protocol}://${domain}`);
|
||||
return;
|
||||
}
|
||||
subdomain = claimed;
|
||||
} else {
|
||||
const hostingDomain = matched as string;
|
||||
// Bare hosting domain (e.g. `puter.site`) → redirect to the main site.
|
||||
if (host === hostingDomain) {
|
||||
if (domain) {
|
||||
res.redirect(302, `${req.protocol}://${domain}`);
|
||||
return;
|
||||
}
|
||||
res.status(404).type('text/plain').send('Subdomain not found');
|
||||
return;
|
||||
}
|
||||
res.status(404).type('text/plain').send('Subdomain not found');
|
||||
return;
|
||||
}
|
||||
|
||||
// `host` is `<prefix>.<matched>`; subdomain is the left-most label.
|
||||
const prefix = host.slice(
|
||||
0,
|
||||
host.length - hostingDomain.length - 1,
|
||||
);
|
||||
subdomain = prefix.split('.')[0] || '';
|
||||
// `host` is `<prefix>.<matched>`; subdomain is the left-most label.
|
||||
const prefix = host.slice(0, host.length - matched.length - 1);
|
||||
const subdomain = prefix.split('.')[0] || '';
|
||||
|
||||
if (!subdomain || subdomain === 'www') {
|
||||
if (domain) {
|
||||
res.redirect(302, `${req.protocol}://${domain}`);
|
||||
return;
|
||||
}
|
||||
res.status(404).type('text/plain').send('Subdomain not found');
|
||||
if (!subdomain || subdomain === 'www') {
|
||||
if (domain) {
|
||||
res.redirect(302, `${req.protocol}://${domain}`);
|
||||
return;
|
||||
}
|
||||
res.status(404).type('text/plain').send('Subdomain not found');
|
||||
return;
|
||||
}
|
||||
|
||||
const site = (await layers.stores.subdomain.getBySubdomain(
|
||||
@@ -317,11 +286,6 @@ export const createPuterSiteMiddleware = (
|
||||
site: { user_id: site.user_id },
|
||||
appStore: layers.stores.app,
|
||||
config: hostingCfg,
|
||||
// App index URLs name the hosting-domain address, not the
|
||||
// custom host the visitor used.
|
||||
host: isCustomHost
|
||||
? `${subdomain}.${hostingDomains[0]}`
|
||||
: undefined,
|
||||
}),
|
||||
);
|
||||
const rootLookup =
|
||||
@@ -336,14 +300,7 @@ export const createPuterSiteMiddleware = (
|
||||
const isPrivateApp = Boolean(associatedApp?.is_private);
|
||||
const privateApp = isPrivateApp ? associatedApp : null;
|
||||
|
||||
if (isCustomHost) {
|
||||
// The private-app gate and its cookies are bound to the private
|
||||
// hosting domain, so these hosts serve public sites only.
|
||||
if (isPrivateApp) {
|
||||
res.status(404).type('text/plain').send('Site not found');
|
||||
return;
|
||||
}
|
||||
} else if (isPrivateApp) {
|
||||
if (isPrivateApp) {
|
||||
// Private apps must run on the private hosting domain. If a
|
||||
// visitor arrives via the public domain (puter.site), redirect
|
||||
// them to the equivalent private-host URL so the cookie scope
|
||||
@@ -491,7 +448,7 @@ export const createPuterSiteMiddleware = (
|
||||
res.redirect(302, cleanPath);
|
||||
return;
|
||||
}
|
||||
} else if (privateHostingDomains.has(matched as string)) {
|
||||
} else if (privateHostingDomains.has(matched)) {
|
||||
// Non-private content landed on the private hosting domain —
|
||||
// mirror of the private redirect above. Covers two cases:
|
||||
// - a paid app that just flipped to free (is_private 1→0)
|
||||
|
||||
@@ -35,7 +35,7 @@ describe('chat cancellation over HTTP', () => {
|
||||
let env: PuterTestEnv;
|
||||
let restoreFetch: () => void;
|
||||
let actor: Actor;
|
||||
let onRequest: (res: ServerResponse, body: string) => void;
|
||||
let onRequest: (res: ServerResponse) => void;
|
||||
|
||||
beforeAll(async () => {
|
||||
upstream = createServer((req, res) => {
|
||||
@@ -44,12 +44,8 @@ describe('chat cancellation over HTTP', () => {
|
||||
res.end(JSON.stringify({ models: [{ name: 'cancel-test' }] }));
|
||||
return;
|
||||
}
|
||||
let body = '';
|
||||
req.setEncoding('utf8');
|
||||
req.on('data', (chunk: string) => {
|
||||
body += chunk;
|
||||
});
|
||||
req.once('end', () => onRequest(res, body));
|
||||
req.resume();
|
||||
req.once('end', () => onRequest(res));
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
upstream.listen(0, '127.0.0.1', resolve),
|
||||
@@ -147,137 +143,24 @@ describe('chat cancellation over HTTP', () => {
|
||||
);
|
||||
};
|
||||
|
||||
const usage = { prompt_tokens: 3, completion_tokens: 5, total_tokens: 8 };
|
||||
const responsesUsage = {
|
||||
input_tokens: 3,
|
||||
output_tokens: 5,
|
||||
total_tokens: 8,
|
||||
input_tokens_details: { cached_tokens: 0 },
|
||||
output_tokens_details: { reasoning_tokens: 0 },
|
||||
};
|
||||
const response = {
|
||||
id: 'resp_1',
|
||||
object: 'response',
|
||||
status: 'completed',
|
||||
output: [
|
||||
{
|
||||
type: 'message',
|
||||
id: 'msg_1',
|
||||
role: 'assistant',
|
||||
status: 'completed',
|
||||
content: [
|
||||
{ type: 'output_text', text: 'done', annotations: [] },
|
||||
],
|
||||
},
|
||||
],
|
||||
usage: responsesUsage,
|
||||
};
|
||||
const sse = (events: unknown[]) =>
|
||||
events.map((event) => `data: ${JSON.stringify(event)}\n\n`).join('');
|
||||
|
||||
/**
|
||||
* A complete upstream answer, with usage, in the shape the request asked
|
||||
* for.
|
||||
*/
|
||||
const finishAnswer = (res: ServerResponse, body: string) => {
|
||||
const stream = (JSON.parse(body) as { stream?: boolean }).stream;
|
||||
const responses = res.req.url === '/v1/responses';
|
||||
if (!stream) {
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify(
|
||||
responses
|
||||
? response
|
||||
: {
|
||||
id: 'chatcmpl_1',
|
||||
object: 'chat.completion',
|
||||
created: 0,
|
||||
model: 'cancel-test',
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: {
|
||||
role: 'assistant',
|
||||
content: 'done',
|
||||
},
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage,
|
||||
},
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (!res.headersSent)
|
||||
res.setHeader('Content-Type', 'text/event-stream');
|
||||
res.end(
|
||||
responses
|
||||
? sse([
|
||||
{ type: 'response.output_text.delta', delta: 'done' },
|
||||
{ type: 'response.completed', response },
|
||||
])
|
||||
: `${sse([
|
||||
{
|
||||
id: 'chatcmpl_1',
|
||||
object: 'chat.completion.chunk',
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
delta: { content: 'done' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'chatcmpl_1',
|
||||
object: 'chat.completion.chunk',
|
||||
choices: [],
|
||||
usage,
|
||||
},
|
||||
])}data: [DONE]\n\n`,
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether a usage entry the upstream reported, not an estimate, was
|
||||
* recorded.
|
||||
*/
|
||||
const meteredReportedUsage = (spy: ReturnType<typeof vi.spyOn>) =>
|
||||
spy.mock.calls
|
||||
.flatMap((call) => (call[1] as { usageType: string }[]) ?? [])
|
||||
.some(
|
||||
(u) =>
|
||||
!u.usageType.includes('estimated_') &&
|
||||
/completion|output_tokens/.test(u.usageType),
|
||||
);
|
||||
|
||||
it.each(
|
||||
routes.flatMap((route) =>
|
||||
[false, true].map((stream) => ({ route, stream })),
|
||||
),
|
||||
)(
|
||||
'bills an upstream the caller left while waiting for headers: $route (stream=$stream)',
|
||||
'cancels upstream while waiting for headers: $route (stream=$stream)',
|
||||
async ({ route, stream }) => {
|
||||
let upstreamRes!: ServerResponse;
|
||||
let upstreamBody = '';
|
||||
let upstreamClosedEarly = false;
|
||||
let upstreamClosed = false;
|
||||
let received!: () => void;
|
||||
const ready = new Promise<void>((resolve) => {
|
||||
received = resolve;
|
||||
});
|
||||
onRequest = (res, body) => {
|
||||
upstreamRes = res;
|
||||
upstreamBody = body;
|
||||
onRequest = (res) => {
|
||||
res.once('close', () => {
|
||||
if (!res.writableFinished) upstreamClosedEarly = true;
|
||||
upstreamClosed = true;
|
||||
});
|
||||
received();
|
||||
};
|
||||
const metered = vi.spyOn(
|
||||
env.server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
const abort = new AbortController();
|
||||
const request = call(route, abort.signal, stream).catch(
|
||||
(error: unknown) => error,
|
||||
@@ -293,50 +176,36 @@ describe('chat cancellation over HTTP', () => {
|
||||
);
|
||||
}),
|
||||
]);
|
||||
abort.abort();
|
||||
expect(await request).toMatchObject({ name: 'AbortError' });
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
expect(upstreamClosedEarly).toBe(false);
|
||||
expect(
|
||||
await env.server.services.metering.getOutstandingHolds(
|
||||
actor,
|
||||
),
|
||||
).toBeGreaterThan(0);
|
||||
|
||||
finishAnswer(upstreamRes, upstreamBody);
|
||||
abort.abort();
|
||||
expect(await request).toMatchObject({ name: 'AbortError' });
|
||||
await expect.poll(() => upstreamClosed).toBe(true);
|
||||
await expect
|
||||
.poll(() =>
|
||||
env.server.services.metering.getOutstandingHolds(actor),
|
||||
)
|
||||
.toBe(0);
|
||||
expect(upstreamClosedEarly).toBe(false);
|
||||
expect(meteredReportedUsage(metered)).toBe(true);
|
||||
} finally {
|
||||
abort.abort();
|
||||
metered.mockRestore();
|
||||
upstream.closeAllConnections();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it.each(routes)(
|
||||
'bills an upstream the caller left after partial output: %s',
|
||||
'cancels an idle upstream after partial output: %s',
|
||||
async (route) => {
|
||||
let upstreamRes!: ServerResponse;
|
||||
let upstreamBody = '';
|
||||
let upstreamClosedEarly = false;
|
||||
onRequest = (res, body) => {
|
||||
upstreamRes = res;
|
||||
upstreamBody = body;
|
||||
let upstreamClosed = false;
|
||||
onRequest = (res) => {
|
||||
res.once('close', () => {
|
||||
if (!res.writableFinished) upstreamClosedEarly = true;
|
||||
upstreamClosed = true;
|
||||
});
|
||||
writeText(res);
|
||||
};
|
||||
const metered = vi.spyOn(
|
||||
env.server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
const abort = new AbortController();
|
||||
try {
|
||||
const response = await call(route, abort.signal);
|
||||
@@ -350,20 +219,14 @@ describe('chat cancellation over HTTP', () => {
|
||||
text += new TextDecoder().decode(chunk.value);
|
||||
}
|
||||
abort.abort();
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
expect(upstreamClosedEarly).toBe(false);
|
||||
|
||||
finishAnswer(upstreamRes, upstreamBody);
|
||||
await expect.poll(() => upstreamClosed).toBe(true);
|
||||
await expect
|
||||
.poll(() =>
|
||||
env.server.services.metering.getOutstandingHolds(actor),
|
||||
)
|
||||
.toBe(0);
|
||||
expect(upstreamClosedEarly).toBe(false);
|
||||
expect(meteredReportedUsage(metered)).toBe(true);
|
||||
} finally {
|
||||
abort.abort();
|
||||
metered.mockRestore();
|
||||
upstream.closeAllConnections();
|
||||
}
|
||||
},
|
||||
|
||||
@@ -47,8 +47,7 @@ import { ChatCompletionDriver } from './ChatCompletionDriver.js';
|
||||
import { clearUnhealthyRoutes, isRouteUnhealthy } from './utils/providerHealth.js';
|
||||
import { FakeChatProvider } from './providers/FakeChatProvider.js';
|
||||
import type { IChatCompleteResult } from './types.js';
|
||||
import { SDK_TIMEOUT_MS } from './utils/sdkClient.js';
|
||||
import type { AIChatStream } from './utils/Streaming.js';
|
||||
import { type AIChatStream, ChatStreamAbortedError } from './utils/Streaming.js';
|
||||
|
||||
let server: PuterServer;
|
||||
|
||||
@@ -906,7 +905,7 @@ describe('ChatCompletionDriver when the caller hangs up', () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('keeps generating after the hang-up and bills the reported usage', async () => {
|
||||
it('stops the generation and gives the hold back', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const metering = server.services.metering;
|
||||
const abort = new AbortController();
|
||||
@@ -915,7 +914,6 @@ describe('ChatCompletionDriver when the caller hangs up', () => {
|
||||
resume = r;
|
||||
});
|
||||
let providerError: unknown;
|
||||
let finished = false;
|
||||
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
@@ -923,13 +921,8 @@ describe('ChatCompletionDriver when the caller hangs up', () => {
|
||||
block.addText('first');
|
||||
await paused;
|
||||
try {
|
||||
// An upstream still generating after the caller left.
|
||||
for (let i = 0; i < 1000; i++) {
|
||||
await chatStream.waitForDrain();
|
||||
block.addText('more');
|
||||
}
|
||||
chatStream.end({ input_tokens: 3, output_tokens: 1001 });
|
||||
finished = true;
|
||||
// An upstream still generating: keeps writing until told.
|
||||
for (let i = 0; i < 1000; i++) block.addText('more');
|
||||
} catch (e) {
|
||||
providerError = e;
|
||||
throw e;
|
||||
@@ -937,7 +930,6 @@ describe('ChatCompletionDriver when the caller hangs up', () => {
|
||||
}) as never,
|
||||
);
|
||||
const metered = vi.spyOn(metering, 'batchIncrementUsages');
|
||||
const events = vi.spyOn(server.clients.event, 'emit');
|
||||
|
||||
const { stream } = (await withTestActor(() => {
|
||||
Context.set('abortSignal', abort.signal);
|
||||
@@ -958,120 +950,15 @@ describe('ChatCompletionDriver when the caller hangs up', () => {
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
|
||||
expect(providerError).toBeUndefined();
|
||||
expect(finished).toBe(true);
|
||||
// Nothing reaches a caller who has gone, including the usage line.
|
||||
expect(body).not.toContain('more');
|
||||
expect(body).not.toContain('"type":"usage"');
|
||||
await vi.waitFor(async () =>
|
||||
expect(await metering.getOutstandingHolds(actor)).toBe(0),
|
||||
);
|
||||
expect(
|
||||
meteredUsages(metered).some((u) =>
|
||||
u.usageType.includes('estimated_'),
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
events.mock.calls.some(
|
||||
([key, data]) =>
|
||||
key === 'ai.prompt.cost-calculated' &&
|
||||
(data as { usage?: { output_tokens?: number } }).usage
|
||||
?.output_tokens === 1001,
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('bills a stream whose caller left before the first token', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const abort = new AbortController();
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockImplementationOnce(
|
||||
async () => {
|
||||
// The caller hangs up while the upstream is still answering.
|
||||
abort.abort();
|
||||
return streamOf(async ({ chatStream }) => {
|
||||
chatStream
|
||||
.message()
|
||||
.contentBlock({ type: 'text' })
|
||||
.addText('answer');
|
||||
chatStream.end({ input_tokens: 3, output_tokens: 2 });
|
||||
});
|
||||
},
|
||||
);
|
||||
const events = vi.spyOn(server.clients.event, 'emit');
|
||||
const { stream } = (await withTestActor(() => {
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'go' }],
|
||||
stream: true,
|
||||
});
|
||||
}, actor)) as unknown as { stream: Readable };
|
||||
await drain(stream);
|
||||
expect(
|
||||
events.mock.calls.some(
|
||||
([key, data]) =>
|
||||
key === 'ai.prompt.cost-calculated' &&
|
||||
(data as { usage?: { output_tokens?: number } }).usage
|
||||
?.output_tokens === 2,
|
||||
),
|
||||
).toBe(true);
|
||||
await vi.waitFor(async () =>
|
||||
expect(await server.services.metering.getOutstandingHolds(actor)).toBe(0),
|
||||
);
|
||||
});
|
||||
|
||||
it('cuts off an upstream that outlives its caller by the SDK timeout', async () => {
|
||||
const actor = freeUser() as never;
|
||||
const metering = server.services.metering;
|
||||
const abort = new AbortController();
|
||||
let upstreamAborted = false;
|
||||
vi.spyOn(FakeChatProvider.prototype, 'complete').mockResolvedValueOnce(
|
||||
streamOf(async ({ chatStream }) => {
|
||||
chatStream
|
||||
.message()
|
||||
.contentBlock({ type: 'text' })
|
||||
.addText('x'.repeat(400));
|
||||
const signal = Context.get('upstreamSignal')!;
|
||||
// A stalled upstream: no more chunks until it's cut off.
|
||||
await new Promise((_resolve, reject) => {
|
||||
signal.addEventListener('abort', () => {
|
||||
upstreamAborted = true;
|
||||
reject(signal.reason);
|
||||
});
|
||||
});
|
||||
}) as never,
|
||||
);
|
||||
const metered = vi.spyOn(metering, 'batchIncrementUsages');
|
||||
const timers = vi.spyOn(globalThis, 'setTimeout');
|
||||
const { stream } = (await withTestActor(() => {
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return driver.complete({
|
||||
model: 'priced',
|
||||
messages: [{ role: 'user', content: 'go' }],
|
||||
stream: true,
|
||||
});
|
||||
}, actor)) as unknown as { stream: Readable };
|
||||
|
||||
abort.abort();
|
||||
const limit = timers.mock.calls.find(
|
||||
([, ms]) => ms === SDK_TIMEOUT_MS,
|
||||
);
|
||||
expect(limit).toBeDefined();
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
expect(upstreamAborted).toBe(false);
|
||||
|
||||
(limit![0] as () => void)();
|
||||
await drain(stream);
|
||||
|
||||
expect(upstreamAborted).toBe(true);
|
||||
expect(
|
||||
meteredUsages(metered).some((u) =>
|
||||
u.usageType.endsWith('estimated_output_tokens'),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(providerError).toBeInstanceOf(ChatStreamAbortedError);
|
||||
// Nobody is listening, so no error line is written for the abort.
|
||||
expect(body).not.toContain('"type":"error"');
|
||||
// The hold goes back once the usage write lands.
|
||||
await vi.waitFor(async () =>
|
||||
expect(await metering.getOutstandingHolds(actor)).toBe(0),
|
||||
);
|
||||
// What was generated before the hang-up is still charged.
|
||||
expect(metered).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -116,7 +116,6 @@ import {
|
||||
UNHEALTHY_TTL_SEC,
|
||||
} from './utils/providerHealth.js';
|
||||
import { CATALOG_TTL_MS } from './utils/cachedRemoteCatalog.js';
|
||||
import { SDK_TIMEOUT_MS } from './utils/sdkClient.js';
|
||||
import { fromFinishReason } from './utils/stopReason.js';
|
||||
import { AIChatStream } from './utils/Streaming.js';
|
||||
import {
|
||||
@@ -681,10 +680,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
throw new HttpError(401, 'Authentication required', {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
// The caller hanging up stops nothing that has reached an upstream: it
|
||||
// would bill the request anyway, so it runs to its usage report.
|
||||
const upstream = new AbortController();
|
||||
Context.set('upstreamSignal', upstream.signal);
|
||||
// AI charges recorded in this request before the completion started.
|
||||
const priorCharges = requestAiCosts().length;
|
||||
|
||||
@@ -885,8 +880,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
if (!useFakeProvider) {
|
||||
await this.#resolvePuterPaths(attemptProvider, args, actor);
|
||||
}
|
||||
// Last point where a caller who left costs nothing.
|
||||
abortSignal?.throwIfAborted();
|
||||
return attemptProvider.complete(
|
||||
{
|
||||
...args,
|
||||
@@ -1049,20 +1042,10 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
const init = res.init_chat_stream;
|
||||
const cleanup = res.finally_fn;
|
||||
|
||||
// A caller who leaves stops getting output, but the upstream keeps
|
||||
// going until it reports usage, with as long to finish as a
|
||||
// non-streaming completion gets.
|
||||
let detachedLimit: ReturnType<typeof setTimeout> | undefined;
|
||||
const detachStream = () => {
|
||||
chatStream.detach();
|
||||
detachedLimit = setTimeout(
|
||||
() => upstream.abort(),
|
||||
SDK_TIMEOUT_MS,
|
||||
);
|
||||
detachedLimit.unref?.();
|
||||
};
|
||||
if (abortSignal?.aborted) detachStream();
|
||||
else abortSignal?.addEventListener('abort', detachStream);
|
||||
// Providers also receive this signal to cancel while waiting for output.
|
||||
const abortStream = () => chatStream.abort();
|
||||
if (abortSignal?.aborted) abortStream();
|
||||
else abortSignal?.addEventListener('abort', abortStream);
|
||||
|
||||
// Intercept `chatStream.end(usage)` to fire complete + cost events
|
||||
// (mirrors the non-streaming branch). Clone usage so providers that
|
||||
@@ -1086,9 +1069,9 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
enrichedUsage.cost = charged();
|
||||
}
|
||||
}
|
||||
// A completion nobody received isn't announced, but it still
|
||||
// gets its cost row.
|
||||
if (!chatStream.detached) {
|
||||
// A cancelled stream isn't a completion, but whatever it
|
||||
// metered still gets its cost row.
|
||||
if (!chatStream.aborted) {
|
||||
this.clients.event.emit(
|
||||
'ai.prompt.complete',
|
||||
{
|
||||
@@ -1155,13 +1138,13 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
(async () => {
|
||||
try {
|
||||
await init({ chatStream });
|
||||
if (!passthrough.writableEnded) {
|
||||
if (!chatStream.aborted && !passthrough.writableEnded) {
|
||||
throw new Error('Stream ended before completion');
|
||||
}
|
||||
} catch (e) {
|
||||
if (!chatStream.detached) {
|
||||
if (!chatStream.aborted) {
|
||||
const attempt = toAttempt(model.id, model.provider!, e);
|
||||
chatStream.write(
|
||||
passthrough.write(
|
||||
`${JSON.stringify({
|
||||
type: 'error',
|
||||
message: sanitizeUpstreamMessage(
|
||||
@@ -1177,8 +1160,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
passthrough.end();
|
||||
} finally {
|
||||
clearInterval(renewHold);
|
||||
clearTimeout(detachedLimit);
|
||||
abortSignal?.removeEventListener('abort', detachStream);
|
||||
abortSignal?.removeEventListener('abort', abortStream);
|
||||
// Providers report usage the moment they meter it (see
|
||||
// `AIChatStream.reportUsage`); a stream that never got
|
||||
// there was never charged for.
|
||||
|
||||
@@ -25,10 +25,10 @@ import { AIChatStream } from '../utils/Streaming.js';
|
||||
import { FakeChatProvider } from './FakeChatProvider.js';
|
||||
|
||||
describe('fake chat streaming lifecycle', () => {
|
||||
it('cancels the simulated delay when the upstream signal aborts', async () => {
|
||||
it('cancels the simulated delay before writing output', async () => {
|
||||
await withTestActor(async () => {
|
||||
const abort = new AbortController();
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
const result = await new FakeChatProvider().complete({
|
||||
model: 'fake',
|
||||
stream: true,
|
||||
@@ -59,7 +59,7 @@ describe('fake chat streaming lifecycle', () => {
|
||||
.map((line) => JSON.parse(line));
|
||||
expect(events.map((event) => event.type)).toEqual(['text', 'usage']);
|
||||
});
|
||||
it('finishes without writing after the caller detaches', async () => {
|
||||
it('does not write after the chat stream was aborted independently', async () => {
|
||||
const result = await new FakeChatProvider().complete({
|
||||
model: 'fake',
|
||||
stream: true,
|
||||
@@ -67,9 +67,8 @@ describe('fake chat streaming lifecycle', () => {
|
||||
});
|
||||
const stream = new PassThrough();
|
||||
const chatStream = new AIChatStream({ stream });
|
||||
chatStream.detach();
|
||||
chatStream.abort();
|
||||
await result.init_chat_stream!({ chatStream });
|
||||
expect(stream.readableLength).toBe(0);
|
||||
expect(stream.writableEnded).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -101,11 +101,12 @@ export class FakeChatProvider implements IChatProvider {
|
||||
chatStream: AIChatStream;
|
||||
}) => {
|
||||
await delay(500, undefined, {
|
||||
signal: Context.get('upstreamSignal'),
|
||||
signal: Context.get('abortSignal'),
|
||||
});
|
||||
const response = await resp;
|
||||
if (chatStream.aborted) return;
|
||||
await chatStream.waitForDrain();
|
||||
chatStream.write(
|
||||
chatStream.stream.write(
|
||||
`${JSON.stringify({
|
||||
type: 'text',
|
||||
text: response.message.content[0].text,
|
||||
|
||||
@@ -175,7 +175,7 @@ export class OpenAICompatProvider implements IChatProvider {
|
||||
}
|
||||
}
|
||||
const actor = Context.get('actor');
|
||||
const signal = Context.get('upstreamSignal');
|
||||
const signal = Context.get('abortSignal');
|
||||
|
||||
let messages = args.messages;
|
||||
if (
|
||||
|
||||
@@ -311,7 +311,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
|
||||
const completion = await this.#openAi.chat.completions.create(
|
||||
completionParams,
|
||||
{ signal: Context.get('upstreamSignal') },
|
||||
{ signal: Context.get('abortSignal') },
|
||||
);
|
||||
|
||||
return OpenAiUtil.handle_completion_output({
|
||||
|
||||
@@ -326,7 +326,7 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
|
||||
const completion = await this.#openAi.responses.create(
|
||||
completionParams,
|
||||
{ signal: Context.get('upstreamSignal') },
|
||||
{ signal: Context.get('abortSignal') },
|
||||
);
|
||||
return OpenAiUtil.handle_completion_output_responses_api({
|
||||
usage_calculator: ({
|
||||
|
||||
@@ -320,7 +320,7 @@ describe('ClaudeProvider.complete request shape', () => {
|
||||
};
|
||||
|
||||
it.each([false, true])(
|
||||
'passes the upstream signal to the SDK with stream=%s',
|
||||
'passes cancellation to the SDK with stream=%s',
|
||||
async (stream) => {
|
||||
const { provider } = makeProvider();
|
||||
const abort = new AbortController();
|
||||
@@ -328,7 +328,7 @@ describe('ClaudeProvider.complete request shape', () => {
|
||||
messagesStreamMock.mockReturnValueOnce(makeStreamLike([]));
|
||||
else messagesCreateMock.mockResolvedValueOnce(baseResponse);
|
||||
await withTestActor(() => {
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return provider.complete({
|
||||
model: 'claude-sonnet-4-6',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
|
||||
@@ -389,10 +389,10 @@ export class ClaudeProvider implements IChatProvider {
|
||||
if (args.stream) {
|
||||
const completion = usesBeta
|
||||
? this.anthropic.beta.messages.stream(sdkParams as never, {
|
||||
signal: Context.get('upstreamSignal'),
|
||||
signal: Context.get('abortSignal'),
|
||||
})
|
||||
: this.anthropic.messages.stream(sdkParams as never, {
|
||||
signal: Context.get('upstreamSignal'),
|
||||
signal: Context.get('abortSignal'),
|
||||
});
|
||||
// Subscribed before the request is awaited: the SDK only queues
|
||||
// events for iterators that already exist.
|
||||
@@ -432,10 +432,10 @@ export class ClaudeProvider implements IChatProvider {
|
||||
try {
|
||||
const msg = await (usesBeta
|
||||
? this.anthropic.beta.messages.create(sdkParams as never, {
|
||||
signal: Context.get('upstreamSignal'),
|
||||
signal: Context.get('abortSignal'),
|
||||
})
|
||||
: this.anthropic.messages.create(sdkParams as never, {
|
||||
signal: Context.get('upstreamSignal'),
|
||||
signal: Context.get('abortSignal'),
|
||||
}));
|
||||
const { usage, costs, details, advisorModel } = this.#meter(
|
||||
(msg as Message).usage,
|
||||
|
||||
@@ -337,11 +337,11 @@ describe('gemini-4-argon catalog and routing', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('does not call Google when the upstream signal already aborted', async () => {
|
||||
it('does not call Google when the request was already cancelled', async () => {
|
||||
const abort = new AbortController();
|
||||
abort.abort();
|
||||
const error = await withTestActor(() => {
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return makeProvider().complete({
|
||||
model: 'gemini-4-argon',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
|
||||
@@ -213,12 +213,12 @@ describe('GroqAIProvider.complete request shape', () => {
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
};
|
||||
|
||||
it('passes the upstream signal to the SDK', async () => {
|
||||
it('passes the request cancellation signal to the SDK', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const abort = new AbortController();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
await withTestActor(() => {
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return provider.complete({
|
||||
model: 'openai/gpt-oss-20b',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
|
||||
@@ -229,19 +229,19 @@ describe('MistralAIProvider.complete request shape', () => {
|
||||
usage: { promptTokens: 1, completionTokens: 1 },
|
||||
};
|
||||
|
||||
it('passes the upstream signal to the SDK', async () => {
|
||||
it('passes the request cancellation signal to the SDK', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const abort = new AbortController();
|
||||
completeMock.mockResolvedValueOnce(baseCompletion);
|
||||
await withTestActor(() => {
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return provider.complete({
|
||||
model: 'mistral-small-latest',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
});
|
||||
});
|
||||
// Bounded by the SDK timeout as well, and still aborted with the
|
||||
// upstream signal.
|
||||
// Bounded by the SDK timeout as well, and still cancelled with the
|
||||
// request.
|
||||
const signal: AbortSignal = completeMock.mock.calls[0][1]?.signal;
|
||||
expect(signal.aborted).toBe(false);
|
||||
abort.abort();
|
||||
|
||||
@@ -243,8 +243,8 @@ export class MistralAIProvider implements IChatProvider {
|
||||
// signal is passed; a stream is bounded by its caller instead.
|
||||
{
|
||||
signal: stream
|
||||
? Context.get('upstreamSignal')
|
||||
: withSdkTimeout(Context.get('upstreamSignal')),
|
||||
? Context.get('abortSignal')
|
||||
: withSdkTimeout(Context.get('abortSignal')),
|
||||
},
|
||||
);
|
||||
} catch (e) {
|
||||
|
||||
@@ -927,7 +927,7 @@ describe('OpenAiChatProvider.complete error mapping', () => {
|
||||
|
||||
describe('provider request cancellation', () => {
|
||||
it.each([false, true])(
|
||||
'passes the upstream signal to the SDK with stream=%s',
|
||||
'passes cancellation to the SDK with stream=%s',
|
||||
async (stream) => {
|
||||
const { provider } = makeProvider();
|
||||
const abort = new AbortController();
|
||||
@@ -945,7 +945,7 @@ describe('provider request cancellation', () => {
|
||||
},
|
||||
);
|
||||
await withTestActor(() => {
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return provider.complete({
|
||||
model: 'gpt-4o-mini',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
|
||||
@@ -250,7 +250,7 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
|
||||
const completion = await this.#openAi.chat.completions.create(
|
||||
completionParams,
|
||||
{ signal: Context.get('upstreamSignal') },
|
||||
{ signal: Context.get('abortSignal') },
|
||||
);
|
||||
|
||||
return OpenAiUtil.handle_completion_output({
|
||||
|
||||
@@ -307,7 +307,7 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
|
||||
const completion = await this.#openAi.responses.create(
|
||||
completionParams,
|
||||
{ signal: Context.get('upstreamSignal') },
|
||||
{ signal: Context.get('abortSignal') },
|
||||
);
|
||||
return OpenAiUtil.handle_completion_output_responses_api({
|
||||
usage_calculator: ({ usage, webSearchCalls, setUsageCosts }) => {
|
||||
|
||||
@@ -97,10 +97,10 @@ export class XAIProvider implements IChatProvider {
|
||||
}
|
||||
: {}),
|
||||
} as ChatCompletionCreateParams,
|
||||
{ signal: Context.get('upstreamSignal') },
|
||||
{ signal: Context.get('abortSignal') },
|
||||
);
|
||||
} catch (e) {
|
||||
if (!Context.get('upstreamSignal')?.aborted) {
|
||||
if (!Context.get('abortSignal')?.aborted) {
|
||||
console.log('XAI AI process_input_messages error: ', e);
|
||||
}
|
||||
throw e;
|
||||
|
||||
@@ -651,9 +651,8 @@ export const create_chat_stream_handler =
|
||||
}
|
||||
}
|
||||
|
||||
Context.get('abortSignal')?.throwIfAborted();
|
||||
if (!finish_reason && !last_usage) {
|
||||
// Some SDKs end an aborted stream as if it were complete.
|
||||
Context.get('upstreamSignal')?.throwIfAborted();
|
||||
throw new Error('Stream ended before completion');
|
||||
}
|
||||
|
||||
@@ -825,11 +824,8 @@ export const create_chat_stream_handler_responses_api =
|
||||
}
|
||||
}
|
||||
|
||||
if (!completed) {
|
||||
// Some SDKs end an aborted stream as if it were complete.
|
||||
Context.get('upstreamSignal')?.throwIfAborted();
|
||||
throw new Error('Stream ended before completion');
|
||||
}
|
||||
Context.get('abortSignal')?.throwIfAborted();
|
||||
if (!completed) throw new Error('Stream ended before completion');
|
||||
|
||||
// TODO DS: this is a bit too abstracted... this is basically just doing the metering now
|
||||
// Missing usage is left undefined rather than fed to the calculator —
|
||||
|
||||
@@ -1919,7 +1919,7 @@ describe('cancelled upstream iterators', () => {
|
||||
const harness = makeCapturingChatStream();
|
||||
await expect(
|
||||
withTestActor(() => {
|
||||
Context.set('upstreamSignal', abort.signal);
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return handler({
|
||||
completion,
|
||||
usage_calculator: usageCalculator,
|
||||
@@ -1932,58 +1932,6 @@ describe('cancelled upstream iterators', () => {
|
||||
expect(usageCalculator).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
'chat completions',
|
||||
create_chat_stream_handler,
|
||||
[
|
||||
{ choices: [{ delta: { content: 'partial' } }] },
|
||||
{
|
||||
choices: [],
|
||||
usage: { prompt_tokens: 3, completion_tokens: 5 },
|
||||
},
|
||||
],
|
||||
],
|
||||
[
|
||||
'responses',
|
||||
create_chat_stream_handler_responses_api,
|
||||
[
|
||||
{ type: 'response.output_text.delta', delta: 'partial' },
|
||||
{
|
||||
type: 'response.completed',
|
||||
response: {
|
||||
status: 'completed',
|
||||
usage: { input_tokens: 3, output_tokens: 5 },
|
||||
},
|
||||
},
|
||||
],
|
||||
],
|
||||
])(
|
||||
'meters %s usage that arrived before the caller left',
|
||||
async (_name, handler, chunks) => {
|
||||
const abort = new AbortController();
|
||||
const usageCalculator = vi.fn(() => ({ output_tokens: 5 }));
|
||||
const completion = {
|
||||
async *[Symbol.asyncIterator]() {
|
||||
yield* chunks;
|
||||
abort.abort();
|
||||
},
|
||||
};
|
||||
const harness = makeCapturingChatStream();
|
||||
await withTestActor(() => {
|
||||
Context.set('abortSignal', abort.signal);
|
||||
return handler({
|
||||
completion,
|
||||
usage_calculator: usageCalculator,
|
||||
})({ chatStream: harness.chatStream });
|
||||
});
|
||||
expect(usageCalculator).toHaveBeenCalledTimes(1);
|
||||
expect(harness.chatStream.reportedUsage).toEqual({
|
||||
output_tokens: 5,
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
|
||||
@@ -169,9 +169,9 @@ export class AIChatStream {
|
||||
*/
|
||||
reportedUsage = null;
|
||||
|
||||
/** Set once the caller has gone away; see `detach`. */
|
||||
detached = false;
|
||||
#detachController = new AbortController();
|
||||
/** Set once the caller has gone away; see `abort`. */
|
||||
aborted = false;
|
||||
abortController = new AbortController();
|
||||
|
||||
/**
|
||||
* Stop/usage/context-management state for the end-of-stream `usage` line,
|
||||
@@ -199,31 +199,28 @@ export class AIChatStream {
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop writing output once the caller has gone away. The provider keeps
|
||||
* reading its upstream, so the usage the upstream reports still gets
|
||||
* metered.
|
||||
* Stop accepting output after cancellation; providers cancel their requests
|
||||
* too.
|
||||
*/
|
||||
detach() {
|
||||
this.detached = true;
|
||||
this.#detachController.abort();
|
||||
}
|
||||
|
||||
/** False once nothing will read the output. */
|
||||
#writable() {
|
||||
return !this.detached && !this.stream.destroyed;
|
||||
abort() {
|
||||
this.aborted = true;
|
||||
this.abortController.abort();
|
||||
}
|
||||
|
||||
async waitForDrain() {
|
||||
if (!this.#writable() || !this.stream.writableNeedDrain) return;
|
||||
this.#assertOpen();
|
||||
if (!this.stream.writableNeedDrain) return;
|
||||
await new Promise((resolve, reject) => {
|
||||
const signal = this.#detachController.signal;
|
||||
const cleanup = () => {
|
||||
this.stream.removeListener('drain', onDone);
|
||||
this.stream.removeListener('close', onDone);
|
||||
this.stream.removeListener('drain', onDrain);
|
||||
this.stream.removeListener('close', onAbort);
|
||||
this.stream.removeListener('error', onError);
|
||||
signal.removeEventListener('abort', onDone);
|
||||
this.abortController.signal.removeEventListener(
|
||||
'abort',
|
||||
onAbort,
|
||||
);
|
||||
};
|
||||
const onDone = () => {
|
||||
const onDrain = () => {
|
||||
cleanup();
|
||||
resolve();
|
||||
};
|
||||
@@ -231,16 +228,28 @@ export class AIChatStream {
|
||||
cleanup();
|
||||
reject(error);
|
||||
};
|
||||
this.stream.once('drain', onDone);
|
||||
this.stream.once('close', onDone);
|
||||
const onAbort = () => onError(new ChatStreamAbortedError());
|
||||
this.stream.once('drain', onDrain);
|
||||
this.stream.once('close', onAbort);
|
||||
this.stream.once('error', onError);
|
||||
signal.addEventListener('abort', onDone, { once: true });
|
||||
if (!this.#writable() || !this.stream.writableNeedDrain) onDone();
|
||||
this.abortController.signal.addEventListener('abort', onAbort, {
|
||||
once: true,
|
||||
});
|
||||
if (this.stream.destroyed || this.aborted) onAbort();
|
||||
else if (!this.stream.writableNeedDrain) onDrain();
|
||||
});
|
||||
this.#assertOpen();
|
||||
}
|
||||
|
||||
#assertOpen() {
|
||||
if (this.aborted) throw new ChatStreamAbortedError();
|
||||
}
|
||||
|
||||
/** @param {string} text */
|
||||
countOutput(text) {
|
||||
// Tool-call input arrives through here and is only written when the
|
||||
// block ends, so a long tool call has to notice the abort here too.
|
||||
this.#assertOpen();
|
||||
if (typeof text === 'string') this.outputChars += text.length;
|
||||
}
|
||||
|
||||
@@ -270,8 +279,9 @@ export class AIChatStream {
|
||||
* @param {{ alreadyCounted?: boolean }} [opts]
|
||||
*/
|
||||
writeChunk(chunk, { alreadyCounted = false } = {}) {
|
||||
this.#assertOpen();
|
||||
if (!alreadyCounted) this.outputChars += payloadChars(chunk);
|
||||
if (this.#writable()) this.stream.write(`${JSON.stringify(chunk)}\n`);
|
||||
this.stream.write(`${JSON.stringify(chunk)}\n`);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -374,7 +384,8 @@ export class AIChatStream {
|
||||
*/
|
||||
end(usage, extra) {
|
||||
this.reportUsage(usage);
|
||||
if (this.#writable()) {
|
||||
// Usage still counts after an abort; only the line has no reader.
|
||||
if (!this.aborted) {
|
||||
const stop = this.stop;
|
||||
const finish_reason = stop?.reason
|
||||
? toFinishReason(stop.reason)
|
||||
@@ -425,7 +436,16 @@ export class AIChatStream {
|
||||
return new AIChatMessageStream(this);
|
||||
}
|
||||
write(...args) {
|
||||
if (this.#writable()) this.stream.write(...args);
|
||||
this.#assertOpen();
|
||||
return this.stream.write(...args);
|
||||
}
|
||||
}
|
||||
|
||||
/** Thrown into a provider's read loop once its caller has gone away. */
|
||||
export class ChatStreamAbortedError extends Error {
|
||||
constructor() {
|
||||
super('Chat stream aborted: the caller disconnected');
|
||||
this.name = 'ChatStreamAbortedError';
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@ import Streaming, {
|
||||
AIChatStream,
|
||||
AIChatTextStream,
|
||||
AIChatToolUseStream,
|
||||
ChatStreamAbortedError,
|
||||
} from './Streaming.js';
|
||||
|
||||
// AIChatStream + friends emit newline-delimited JSON to an underlying
|
||||
@@ -188,27 +189,26 @@ describe('AIChatStream stop + usage-detail reporting', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// -- Detach --
|
||||
// ── Abort ──────────────────────────────────────────────────────────
|
||||
|
||||
describe('AIChatStream.detach', () => {
|
||||
it('stops writing but keeps counting output', () => {
|
||||
describe('AIChatStream.abort', () => {
|
||||
it('throws into the next text, reasoning or tool-call write', () => {
|
||||
const h = makeHarness();
|
||||
const text = h.chatStream.message().contentBlock({ type: 'text' });
|
||||
const tool = h.chatStream
|
||||
.message()
|
||||
.contentBlock({ type: 'tool_use', id: 't', name: 'f' });
|
||||
h.chatStream.detach();
|
||||
text.addText('abc');
|
||||
text.addReasoning('de');
|
||||
tool.addPartialJSON('{}');
|
||||
h.chatStream.write('raw');
|
||||
expect(h.chatStream.outputChars).toBe(7);
|
||||
h.chatStream.abort();
|
||||
expect(() => text.addText('x')).toThrow(ChatStreamAbortedError);
|
||||
expect(() => text.addReasoning('x')).toThrow(ChatStreamAbortedError);
|
||||
expect(() => tool.addPartialJSON('{')).toThrow(ChatStreamAbortedError);
|
||||
expect(() => h.chatStream.write('raw')).toThrow(ChatStreamAbortedError);
|
||||
expect(h.rawChunks()).toEqual([]);
|
||||
});
|
||||
|
||||
it('still records usage and ends the stream, writing nothing', () => {
|
||||
const h = makeHarness();
|
||||
h.chatStream.detach();
|
||||
h.chatStream.abort();
|
||||
h.chatStream.end({ output_tokens: 3 });
|
||||
expect(h.chatStream.reportedUsage).toEqual({ output_tokens: 3 });
|
||||
expect(h.rawChunks()).toEqual([]);
|
||||
@@ -448,25 +448,16 @@ describe('AIChatStream backpressure', () => {
|
||||
sink.destroy();
|
||||
});
|
||||
|
||||
it.each(['detach', 'close'])('releases a drain wait on %s', async (event) => {
|
||||
it.each(['abort', 'close', 'error'])('releases a drain wait on %s', async (event) => {
|
||||
const sink = new Writable({ highWaterMark: 1, write() {} });
|
||||
const chatStream = new AIChatStream({ stream: sink });
|
||||
chatStream.writeChunk({ type: 'text', text: 'hello' });
|
||||
const waiting = chatStream.waitForDrain();
|
||||
if (event === 'detach') chatStream.detach();
|
||||
const rejected = expect(waiting).rejects.toThrow(event === 'error' ? 'failed' : 'Chat stream aborted');
|
||||
if (event === 'abort') chatStream.abort();
|
||||
else if (event === 'error') sink.destroy(new Error('failed'));
|
||||
else sink.destroy();
|
||||
await waiting;
|
||||
expect(sink.listenerCount('drain')).toBe(0);
|
||||
await chatStream.waitForDrain();
|
||||
});
|
||||
|
||||
it('rejects a drain wait when the writable errors', async () => {
|
||||
const sink = new Writable({ highWaterMark: 1, write() {} });
|
||||
const chatStream = new AIChatStream({ stream: sink });
|
||||
chatStream.writeChunk({ type: 'text', text: 'hello' });
|
||||
const waiting = chatStream.waitForDrain();
|
||||
sink.destroy(new Error('failed'));
|
||||
await expect(waiting).rejects.toThrow('failed');
|
||||
await rejected;
|
||||
expect(sink.listenerCount('drain')).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -45,7 +45,6 @@ import { runWithContext } from '../../core/context.js';
|
||||
import { type Actor, SYSTEM_ACTOR } from '../../core/actor.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { withAiCostFactorHook } from '../integrationTestUtil.js';
|
||||
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import { CLOUDFLARE_IMAGE_GENERATION_MODELS } from './providers/cloudflare/models.js';
|
||||
import cloudflareModelSearchFixture from './providers/cloudflare/fixtures/model-search.json';
|
||||
@@ -1646,34 +1645,3 @@ it.each([undefined, null, '', ' ', 42, {}])(
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
describe('ImageGenerationDriver cost factor', () => {
|
||||
it('holds and meters at the AI cost factor', async () => {
|
||||
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
|
||||
const usageSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
const generate = async (charges: number) => {
|
||||
openaiImagesGenerateMock.mockResolvedValueOnce({
|
||||
data: [{ url: 'https://oai/img.png' }],
|
||||
});
|
||||
await withActor(() =>
|
||||
driver.generate({ model: 'gpt-image-2', prompt: 'hi' }),
|
||||
);
|
||||
await vi.waitFor(() =>
|
||||
expect(usageSpy).toHaveBeenCalledTimes(charges),
|
||||
);
|
||||
};
|
||||
await generate(1);
|
||||
await withAiCostFactorHook(server.clients.event, 2, () => generate(2));
|
||||
|
||||
const costs = (call: number) =>
|
||||
usageSpy.mock.calls[call]![1].map((u) => u.costOverride!);
|
||||
expect(costs(0).length).toBeGreaterThan(0);
|
||||
expect(costs(1)).toEqual(costs(0).map((c) => Math.round(c * 2)));
|
||||
const held = reserveSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held[0]).toBeGreaterThan(0);
|
||||
expect(held).toEqual([held[0], Math.round(held[0]! * 2)]);
|
||||
});
|
||||
});
|
||||
@@ -27,10 +27,6 @@ import { Mistral } from '@mistralai/mistralai';
|
||||
import { Actor } from '../../core/actor.js';
|
||||
import { Context } from '../../core/context.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import {
|
||||
awsCredentialsFrom,
|
||||
type AwsCredentials,
|
||||
} from '../../util/awsCredentials.js';
|
||||
import { mimeFromName } from '../../util/fileSigning.js';
|
||||
import {
|
||||
assertActorHasCredits,
|
||||
@@ -297,7 +293,11 @@ export class OCRDriver extends PuterDriver {
|
||||
|
||||
// Textract state — one client per region.
|
||||
#textractClients: Record<string, TextractClient> = {};
|
||||
#awsConfig: { credentials: AwsCredentials; region: string } | null = null;
|
||||
#awsConfig: {
|
||||
accessKeyId?: string;
|
||||
secretAccessKey?: string;
|
||||
region?: string;
|
||||
} | null = null;
|
||||
/** DetectDocumentText rates per region, from AWS's public price list. */
|
||||
#textractRates = textractRates(OCR_COSTS[TEXTRACT_PAGE_USAGE_TYPE]);
|
||||
|
||||
@@ -314,14 +314,16 @@ export class OCRDriver extends PuterDriver {
|
||||
Record<string, unknown> | undefined;
|
||||
const textractAws = (textract?.aws ?? textract) as
|
||||
Record<string, unknown> | undefined;
|
||||
const textractCredentials = awsCredentialsFrom(textractAws);
|
||||
const textractAccessKey = textractAws?.access_key as string | undefined;
|
||||
const textractSecretKey = textractAws?.secret_key as string | undefined;
|
||||
const textractRegion =
|
||||
(textractAws?.region as string | undefined) ??
|
||||
(textract?.region as string | undefined) ??
|
||||
DEFAULT_TEXTRACT_REGION;
|
||||
if (textractCredentials) {
|
||||
if (textractAccessKey && textractSecretKey) {
|
||||
this.#awsConfig = {
|
||||
credentials: textractCredentials,
|
||||
accessKeyId: textractAccessKey,
|
||||
secretAccessKey: textractSecretKey,
|
||||
region: textractRegion,
|
||||
};
|
||||
// Warm the configured region so the cost report has its rate.
|
||||
@@ -458,7 +460,10 @@ export class OCRDriver extends PuterDriver {
|
||||
const cached = this.#textractClients[region];
|
||||
if (cached) return cached;
|
||||
const client = new TextractClient({
|
||||
credentials: this.#awsConfig!.credentials,
|
||||
credentials: {
|
||||
accessKeyId: this.#awsConfig!.accessKeyId!,
|
||||
secretAccessKey: this.#awsConfig!.secretAccessKey!,
|
||||
},
|
||||
region,
|
||||
});
|
||||
this.#textractClients[region] = client;
|
||||
|
||||
@@ -48,11 +48,7 @@ import type { MeteringService } from '../../services/metering/MeteringService.js
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import type { VoiceChangerDriver } from './VoiceChangerDriver.js';
|
||||
import {
|
||||
callerError,
|
||||
readStreamResult,
|
||||
withAiCostFactorHook,
|
||||
} from '../integrationTestUtil.js';
|
||||
import { callerError, readStreamResult } from '../integrationTestUtil.js';
|
||||
import { VOICE_CHANGER_COSTS } from './costs.js';
|
||||
|
||||
const { webFetchMock } = vi.hoisted(() => ({ webFetchMock: vi.fn() }));
|
||||
@@ -356,35 +352,6 @@ describe('VoiceChangerDriver.convert success path', () => {
|
||||
expect(cost).toBe(perSecond * expectedSeconds);
|
||||
});
|
||||
|
||||
it('holds and meters at the AI cost factor', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const usageType = 'elevenlabs:eleven_multilingual_sts_v2:second';
|
||||
const metered = () =>
|
||||
incrementUsageSpy.mock.calls
|
||||
.filter(([, type]) => type === usageType)
|
||||
.map(([, , , cost]) => cost);
|
||||
const convert = async (charges: number) => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
okResponse(new ArrayBuffer(8), 'audio/mpeg'),
|
||||
);
|
||||
await readStreamResult(
|
||||
await withActor(actor, () =>
|
||||
driver.convert({
|
||||
audio: dataUrl(Buffer.alloc(32_000), 'audio/mpeg'),
|
||||
}),
|
||||
),
|
||||
);
|
||||
await vi.waitFor(() => expect(metered()).toHaveLength(charges));
|
||||
};
|
||||
await convert(1);
|
||||
await withAiCostFactorHook(server.clients.event, 2, () => convert(2));
|
||||
|
||||
const [base, factored] = metered();
|
||||
expect(factored).toBe(base! * 2);
|
||||
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held).toEqual([base, base! * 2]);
|
||||
});
|
||||
|
||||
it('meters nothing when ElevenLabs breaks off before any audio', async () => {
|
||||
const { actor } = await makeUser();
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
|
||||
@@ -49,7 +49,6 @@ import { PuterServer } from '../../server.js';
|
||||
import type { MeteringService } from '../../services/metering/MeteringService.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import { withAiCostFactorHook } from '../integrationTestUtil.js';
|
||||
import type { SpeechToTextDriver } from './SpeechToTextDriver.js';
|
||||
import { SPEECH_TO_TEXT_COSTS } from './costs.js';
|
||||
|
||||
@@ -683,34 +682,6 @@ describe('SpeechToTextDriver metering', () => {
|
||||
const creditCall = hasCreditsSpy.mock.calls[0]!;
|
||||
expect(creditCall[1]).toBe(expected);
|
||||
});
|
||||
|
||||
it('holds and meters at the AI cost factor', async () => {
|
||||
const { actor } = await makeUser();
|
||||
const usageType = 'openai:gpt-transcribe:second';
|
||||
const metered = () =>
|
||||
incrementUsageSpy.mock.calls
|
||||
.filter(([, type]) => type === usageType)
|
||||
.map(([, , , cost]) => cost);
|
||||
const transcribe = async (charges: number) => {
|
||||
transcriptionsCreateMock.mockResolvedValueOnce({ text: 'x' });
|
||||
await withActor(actor, () =>
|
||||
driver.transcribe({
|
||||
file: dataUrl(Buffer.alloc(32000, 0), 'audio/mp3'),
|
||||
model: 'gpt-transcribe',
|
||||
}),
|
||||
);
|
||||
await vi.waitFor(() => expect(metered()).toHaveLength(charges));
|
||||
};
|
||||
await transcribe(1);
|
||||
await withAiCostFactorHook(server.clients.event, 2, () =>
|
||||
transcribe(2),
|
||||
);
|
||||
|
||||
const [base, factored] = metered();
|
||||
expect(factored).toBe(base! * 2);
|
||||
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held).toEqual([base, base! * 2]);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Error paths ─────────────────────────────────────────────────────
|
||||
|
||||
@@ -44,10 +44,6 @@ import { runWithContext } from '../../core/context.js';
|
||||
import { SYSTEM_ACTOR } from '../../core/actor.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import {
|
||||
readStreamResult,
|
||||
withAiCostFactorHook,
|
||||
} from '../integrationTestUtil.js';
|
||||
import type { TTSDriver } from './TTSDriver.js';
|
||||
import pollyPriceList from './providers/awsPolly/fixtures/us-west-2.json';
|
||||
|
||||
@@ -692,40 +688,3 @@ describe('TTSDriver.synthesize error mapping', () => {
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('TTSDriver cost factor', () => {
|
||||
it('holds and meters at the AI cost factor', async () => {
|
||||
const incrementUsageSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'incrementUsage',
|
||||
);
|
||||
const hasCreditsSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'hasEnoughCredits',
|
||||
);
|
||||
const metered = () =>
|
||||
incrementUsageSpy.mock.calls.map(([, , , cost]) => cost!);
|
||||
const synthesize = async (charges: number) => {
|
||||
openaiSpeechCreateMock.mockResolvedValueOnce(openaiAudioResponse());
|
||||
await readStreamResult(
|
||||
await withActor(() =>
|
||||
driver.synthesize({
|
||||
text: 'hello there',
|
||||
provider: 'openai',
|
||||
}),
|
||||
),
|
||||
);
|
||||
await vi.waitFor(() => expect(metered()).toHaveLength(charges));
|
||||
};
|
||||
await synthesize(1);
|
||||
await withAiCostFactorHook(server.clients.event, 2, () =>
|
||||
synthesize(2),
|
||||
);
|
||||
|
||||
const [base, factored] = metered();
|
||||
expect(base).toBeGreaterThan(0);
|
||||
expect(factored).toBe(Math.round(base! * 2));
|
||||
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held).toEqual([base, Math.round(base! * 2)]);
|
||||
});
|
||||
});
|
||||
@@ -19,7 +19,6 @@
|
||||
|
||||
import { Context } from '../../core/context.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { awsCredentialsFrom } from '../../util/awsCredentials.js';
|
||||
import type { DriverStreamResult } from '../meta.js';
|
||||
import { PuterDriver } from '../types.js';
|
||||
import {
|
||||
@@ -242,13 +241,15 @@ export class TTSDriver extends PuterDriver {
|
||||
const polly = providers['aws-polly'];
|
||||
const pollyAws = (polly?.aws ?? polly) as
|
||||
Record<string, unknown> | undefined;
|
||||
const pollyCredentials = awsCredentialsFrom(pollyAws);
|
||||
if (pollyCredentials) {
|
||||
const pollyAccessKey = pollyAws?.access_key as string | undefined;
|
||||
const pollySecretKey = pollyAws?.secret_key as string | undefined;
|
||||
if (pollyAccessKey && pollySecretKey) {
|
||||
register(
|
||||
'aws-polly',
|
||||
() =>
|
||||
new AWSPollyTTSProvider(m, {
|
||||
credentials: pollyCredentials,
|
||||
access_key: pollyAccessKey,
|
||||
secret_key: pollySecretKey,
|
||||
region: (pollyAws?.region ?? polly?.region) as
|
||||
string | undefined,
|
||||
}),
|
||||
|
||||
@@ -101,7 +101,8 @@ const aiMetering = () =>
|
||||
|
||||
const makeProvider = (region?: string) =>
|
||||
new AWSPollyTTSProvider(aiMetering(), {
|
||||
credentials: { accessKeyId: 'AKIA-TEST', secretAccessKey: 'secret' },
|
||||
access_key: 'AKIA-TEST',
|
||||
secret_key: 'secret',
|
||||
region,
|
||||
});
|
||||
|
||||
|
||||
@@ -27,7 +27,6 @@ import {
|
||||
} from '@aws-sdk/client-polly';
|
||||
import { HttpError } from '../../../../core/http/HttpError.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { AwsCredentials } from '../../../../util/awsCredentials.js';
|
||||
import type { AiMeteringService } from '../../../util/aiCostFactor.js';
|
||||
import type { DriverStreamResult } from '../../../meta.js';
|
||||
import { SAMPLE_AUDIO_URL } from '../../../util/testMode.js';
|
||||
@@ -84,7 +83,8 @@ export class AWSPollyTTSProvider implements ITTSProvider {
|
||||
constructor(
|
||||
private readonly meteringService: AiMeteringService,
|
||||
private readonly config: {
|
||||
credentials: AwsCredentials;
|
||||
access_key: string;
|
||||
secret_key: string;
|
||||
region?: string;
|
||||
},
|
||||
) {
|
||||
@@ -115,7 +115,10 @@ export class AWSPollyTTSProvider implements ITTSProvider {
|
||||
}
|
||||
|
||||
this.clients[resolvedRegion] = new PollyClient({
|
||||
credentials: cfg.credentials,
|
||||
credentials: {
|
||||
accessKeyId: cfg.access_key,
|
||||
secretAccessKey: cfg.secret_key,
|
||||
},
|
||||
region: resolvedRegion,
|
||||
});
|
||||
|
||||
|
||||
@@ -46,7 +46,6 @@ import { type Actor, SYSTEM_ACTOR } from '../../core/actor.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import type { MeteringService } from '../../services/metering/MeteringService.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { withAiCostFactorHook } from '../integrationTestUtil.js';
|
||||
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
|
||||
import { TOGETHER_VIDEO_GENERATION_MODELS } from './providers/together/models.js';
|
||||
import type { VideoGenerationDriver } from './VideoGenerationDriver.js';
|
||||
@@ -103,11 +102,6 @@ beforeAll(async () => {
|
||||
// block, so this one alone must not register a provider.
|
||||
'gemini-video-generation': { apiKey: 'gem-key' },
|
||||
kling: { apiKey: 'kling-key' },
|
||||
// The chat block's base URL ends in `/v1`; video must drop it.
|
||||
minimax: {
|
||||
apiKey: 'mm-key',
|
||||
apiBaseUrl: 'https://api.minimax.io/v1',
|
||||
},
|
||||
},
|
||||
} as never);
|
||||
driver = server.drivers.aiVideo as unknown as VideoGenerationDriver;
|
||||
@@ -512,54 +506,6 @@ describe('VideoGenerationDriver.generate provider routing', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'minimax-h3',
|
||||
'minimax/minimax-h3',
|
||||
'minimax:minimax/minimax-h3',
|
||||
])('routes %s to the MiniMax provider from the shared minimax key', async (model) => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch');
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({ task_id: 'mm-task' })),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
task: {
|
||||
id: 'mm-task',
|
||||
status: 'succeeded',
|
||||
content: { url: 'https://minimax/out.mp4' },
|
||||
usage: { output_seconds: 5 },
|
||||
},
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
const result = await withActor(() =>
|
||||
driver.generate({ prompt: 'hi', model } as never),
|
||||
);
|
||||
|
||||
expect(result).toBe('https://minimax/out.mp4');
|
||||
expect(String(fetchSpy.mock.calls[0]![0])).toBe(
|
||||
'https://api.minimax.io/v2/video_generation',
|
||||
);
|
||||
expect(togetherVideosCreateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('lists the MiniMax models under the minimax-video-generation provider', async () => {
|
||||
const minimax = (await driver.models()).filter(
|
||||
(m) => m.provider === 'minimax-video-generation',
|
||||
);
|
||||
expect(minimax.map((m) => m.id).sort()).toEqual([
|
||||
'minimax-h3',
|
||||
'minimax-h3-highspeed',
|
||||
'minimax-h3-lite',
|
||||
'minimax-h3-max',
|
||||
'minimax-hailuo-02',
|
||||
'minimax-hailuo-2.3',
|
||||
'minimax-hailuo-2.3-fast',
|
||||
]);
|
||||
});
|
||||
|
||||
it('routes a known togetherai:minimax/video-01-director id to the Together provider', async () => {
|
||||
togetherVideosCreateMock.mockResolvedValueOnce({ id: 'tg-job' });
|
||||
togetherVideosRetrieveMock.mockResolvedValueOnce({
|
||||
@@ -1001,49 +947,6 @@ describe('VideoGenerationDriver.generate puter_output_path', () => {
|
||||
// ── Size unification ───────────────────────────────────────────────
|
||||
|
||||
describe('VideoGenerationDriver.generate size unification', () => {
|
||||
it.each(['912x512', '512x912'])(
|
||||
'selects and bills Hailuo 02 at 512p for %s',
|
||||
async (size) => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch');
|
||||
const incrementUsageSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'incrementUsage',
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({ task_id: 'mm-task' })),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({
|
||||
status: 'Success',
|
||||
file_id: 'file-1',
|
||||
})),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({
|
||||
file: { download_url: 'https://minimax/out.mp4' },
|
||||
})),
|
||||
);
|
||||
|
||||
await withActor(() => driver.generate({
|
||||
prompt: 'a ball rolls',
|
||||
model: 'minimax-hailuo-02',
|
||||
size,
|
||||
input_reference: 'https://example.com/first.png',
|
||||
} as never));
|
||||
|
||||
const body = JSON.parse(
|
||||
fetchSpy.mock.calls[0]![1]!.body as string,
|
||||
);
|
||||
expect(body.resolution).toBe('512P');
|
||||
expect(incrementUsageSpy).toHaveBeenCalledWith(
|
||||
SYSTEM_ACTOR,
|
||||
'minimax-video-generation:minimax-hailuo-02:per-video:512p:6',
|
||||
1,
|
||||
10 * 1_000_000,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('maps a WIDTHxHEIGHT size onto a tier plus aspect ratio for tier-based models', async () => {
|
||||
completeTogetherJob();
|
||||
|
||||
@@ -1077,77 +980,6 @@ describe('VideoGenerationDriver.generate size unification', () => {
|
||||
expect('ratio' in sent).toBe(false);
|
||||
});
|
||||
|
||||
it('treats a catalog with a 2k or 4k tier as tier-based', async () => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch');
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
code: 0,
|
||||
data: { id: 'kl-task', status: 'submitted' },
|
||||
}),
|
||||
),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
code: 0,
|
||||
data: [
|
||||
{
|
||||
id: 'kl-task',
|
||||
status: 'succeeded',
|
||||
outputs: [
|
||||
{
|
||||
type: 'video',
|
||||
url: 'https://kling/out.mp4',
|
||||
duration: '5',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
),
|
||||
);
|
||||
await withActor(() =>
|
||||
driver.generate({
|
||||
prompt: 'hi',
|
||||
model: 'kling-3.0',
|
||||
size: '1080x1920',
|
||||
} as never),
|
||||
);
|
||||
const sent = JSON.parse(
|
||||
(fetchSpy.mock.calls[0]![1] as RequestInit).body as string,
|
||||
);
|
||||
expect(sent.settings.resolution).toBe('1080p');
|
||||
expect(sent.settings.aspect_ratio).toBe('9:16');
|
||||
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({ task_id: 'mm-task' })),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
task: {
|
||||
id: 'mm-task',
|
||||
status: 'succeeded',
|
||||
content: { url: 'https://minimax/out.mp4' },
|
||||
},
|
||||
}),
|
||||
),
|
||||
);
|
||||
await withActor(() =>
|
||||
driver.generate({
|
||||
prompt: 'hi',
|
||||
model: 'minimax-h3-lite',
|
||||
size: '854x480',
|
||||
} as never),
|
||||
);
|
||||
const sentToMiniMax = JSON.parse(
|
||||
(fetchSpy.mock.calls[2]![1] as RequestInit).body as string,
|
||||
);
|
||||
expect(sentToMiniMax.resolution).toBe('480P');
|
||||
expect(sentToMiniMax.ratio).toBe('16:9');
|
||||
});
|
||||
|
||||
it('fills width/height from a WIDTHxHEIGHT size for pixel-sized models', async () => {
|
||||
completeTogetherJob();
|
||||
|
||||
@@ -1193,31 +1025,3 @@ describe('VideoGenerationDriver.generate size unification', () => {
|
||||
expect('height' in second).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('VideoGenerationDriver cost factor', () => {
|
||||
it('holds and meters at the AI cost factor', async () => {
|
||||
const incrementUsageSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'incrementUsage',
|
||||
);
|
||||
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
|
||||
const metered = () =>
|
||||
incrementUsageSpy.mock.calls.map(([, , , cost]) => cost!);
|
||||
const generate = async (charges: number) => {
|
||||
completeTogetherJob();
|
||||
await withActor(() =>
|
||||
driver.generate({ prompt: 'hi', model: DEFAULT_MODEL } as never),
|
||||
);
|
||||
await vi.waitFor(() => expect(metered()).toHaveLength(charges));
|
||||
};
|
||||
await generate(1);
|
||||
await withAiCostFactorHook(server.clients.event, 2, () => generate(2));
|
||||
|
||||
const [base, factored] = metered();
|
||||
expect(base).toBeGreaterThan(0);
|
||||
expect(factored).toBe(Math.round(base! * 2));
|
||||
const held = reserveSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held[0]).toBeGreaterThan(0);
|
||||
expect(held).toEqual([held[0], Math.round(held[0]! * 2)]);
|
||||
});
|
||||
});
|
||||
@@ -41,7 +41,6 @@ import { readProviderKey } from '../util/providerRegistry.js';
|
||||
import { BytePlusVideoProvider } from './providers/byteplus/BytePlusVideoProvider.js';
|
||||
import { GeminiVideoProvider } from './providers/gemini/GeminiVideoProvider.js';
|
||||
import { KlingVideoProvider } from './providers/kling/KlingVideoProvider.js';
|
||||
import { MiniMaxVideoProvider } from './providers/minimax/MiniMaxVideoProvider.js';
|
||||
import { TogetherVideoProvider } from './providers/together/TogetherVideoProvider.js';
|
||||
import type {
|
||||
IGenerateVideoParams,
|
||||
@@ -51,9 +50,7 @@ import type {
|
||||
|
||||
const DEFAULT_PROVIDER = 'together-video-generation';
|
||||
|
||||
/** A resolution tier such as `720p`, `2k` or `4k`, as opposed to pixels. */
|
||||
const isResolutionTier = (value: string): boolean =>
|
||||
/^(\d{3,4}p|\dk)$/i.test(value);
|
||||
const isResolutionTier = (value: string): boolean => /^\d{3,4}p$/i.test(value);
|
||||
|
||||
const parsePixelSize = (
|
||||
value: string | undefined,
|
||||
@@ -66,15 +63,14 @@ const parsePixelSize = (
|
||||
};
|
||||
|
||||
/** Resolution tier of the shorter side, e.g. 1920x1080 and 1080x1920 → 1080p. */
|
||||
const tierForPixels = (
|
||||
{ width, height }: { width: number; height: number },
|
||||
dimensions: readonly string[],
|
||||
) => {
|
||||
const tierForPixels = ({
|
||||
width,
|
||||
height,
|
||||
}: {
|
||||
width: number;
|
||||
height: number;
|
||||
}) => {
|
||||
const shortSide = Math.min(width, height);
|
||||
const exactTier = dimensions.find(
|
||||
(tier) => tier.toLowerCase() === `${shortSide}p`,
|
||||
);
|
||||
if (exactTier) return exactTier;
|
||||
if (shortSide <= 480) return '480p';
|
||||
if (shortSide <= 720) return '720p';
|
||||
if (shortSide <= 1080) return '1080p';
|
||||
@@ -85,7 +81,7 @@ const tierForPixels = (
|
||||
* Driver implementing the `puter-video-generation` interface.
|
||||
*
|
||||
* Manages multiple upstream providers (Together, BytePlus/Seedance, Gemini,
|
||||
* Kling, MiniMax) and handles model resolution, provider routing, and parameter
|
||||
* Kling) and handles model resolution, provider routing, and parameter
|
||||
* normalisation. Each provider is a plain `IVideoProvider` -- the driver
|
||||
* instantiates them from config on boot.
|
||||
*
|
||||
@@ -102,7 +98,6 @@ export class VideoGenerationDriver extends PuterDriver {
|
||||
'byteplus-video-generation',
|
||||
'gemini-video-generation',
|
||||
'kling-video-generation',
|
||||
'minimax-video-generation',
|
||||
];
|
||||
readonly isDefault = true;
|
||||
|
||||
@@ -298,7 +293,7 @@ export class VideoGenerationDriver extends PuterDriver {
|
||||
const catalogIsTiers = model.dimensions.every(isResolutionTier);
|
||||
const wanted =
|
||||
catalogIsTiers && requestedPixels
|
||||
? tierForPixels(requestedPixels, model.dimensions)
|
||||
? tierForPixels(requestedPixels)
|
||||
: requestedResolution;
|
||||
|
||||
// Case-insensitive so '4K' matches a catalog entry spelled '4k';
|
||||
@@ -473,26 +468,6 @@ export class VideoGenerationDriver extends PuterDriver {
|
||||
{ stores: this.stores, fs: this.services.fs },
|
||||
);
|
||||
}
|
||||
|
||||
// Falls back to the shared `minimax` (ai-chat) key. The chat block's
|
||||
// `apiBaseUrl` names the chat API under `/v1`; the provider keeps its
|
||||
// host, since the video API lives beside it.
|
||||
const minimaxVideoCfg = providers['minimax-video-generation'] as
|
||||
Record<string, unknown> | undefined;
|
||||
const minimaxSharedCfg = providers['minimax'] as
|
||||
Record<string, unknown> | undefined;
|
||||
const minimaxKey = readProviderKey(minimaxVideoCfg, minimaxSharedCfg);
|
||||
if (minimaxKey) {
|
||||
this.#providers['minimax-video-generation'] =
|
||||
new MiniMaxVideoProvider(
|
||||
{
|
||||
apiKey: minimaxKey,
|
||||
apiBaseUrl: (minimaxVideoCfg?.apiBaseUrl ??
|
||||
minimaxSharedCfg?.apiBaseUrl) as string | undefined,
|
||||
},
|
||||
m,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// -- Model map -----------------------------------------------------------
|
||||
|
||||
-66
@@ -1,66 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Integration test for the MiniMax video provider.
|
||||
*
|
||||
* Generates a 4s 480P clip on minimax-h3-lite ($0.10). Skipped when
|
||||
* `PUTER_TEST_AI_MINIMAX_API_KEY` is unset.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
makeMeteringStub,
|
||||
optionalEnv,
|
||||
skipUnlessEnv,
|
||||
withTestActor,
|
||||
} from '../../../integrationTestUtil.js';
|
||||
import { MiniMaxVideoProvider } from './MiniMaxVideoProvider.js';
|
||||
|
||||
const ENV_VAR = 'PUTER_TEST_AI_MINIMAX_API_KEY';
|
||||
// Task-based generation can take a couple of minutes.
|
||||
const VIDEO_TIMEOUT_MS = 5 * 60 * 1000;
|
||||
|
||||
describe.skipIf(skipUnlessEnv(ENV_VAR))(
|
||||
'MiniMaxVideoProvider (integration)',
|
||||
() => {
|
||||
it(
|
||||
'generates a video URL from minimax-h3-lite',
|
||||
{ timeout: VIDEO_TIMEOUT_MS },
|
||||
async () => {
|
||||
const provider = new MiniMaxVideoProvider(
|
||||
{ apiKey: optionalEnv(ENV_VAR)! },
|
||||
makeMeteringStub(),
|
||||
);
|
||||
|
||||
const url = await withTestActor(() =>
|
||||
provider.generate({
|
||||
model: 'minimax-h3-lite',
|
||||
prompt: 'a red ball rolls to the right',
|
||||
resolution: '480p',
|
||||
seconds: 4,
|
||||
}),
|
||||
);
|
||||
|
||||
expect(typeof url).toBe('string');
|
||||
expect((url as string).startsWith('https://')).toBe(true);
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
@@ -1,876 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Offline unit tests for MiniMaxVideoProvider.
|
||||
*
|
||||
* Boots a real PuterServer and constructs the provider against the live wired
|
||||
* `MeteringService`. MiniMax's task APIs are called through global `fetch`,
|
||||
* which is stubbed as the network boundary.
|
||||
*/
|
||||
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
type MockInstance,
|
||||
} from 'vitest';
|
||||
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import { PuterServer } from '../../../../server.js';
|
||||
import { setupTestServer } from '../../../../testUtil.js';
|
||||
import { withTestActor } from '../../../integrationTestUtil.js';
|
||||
import { withAiCostFactor } from '../../../util/aiCostFactor.js';
|
||||
import type { IGenerateVideoParams } from '../../types.js';
|
||||
import { MiniMaxVideoProvider } from './MiniMaxVideoProvider.js';
|
||||
import { MINIMAX_VIDEO_GENERATION_MODELS } from './models.js';
|
||||
|
||||
// -- Test harness ----------------------------------------------------
|
||||
|
||||
let server: PuterServer;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let remainingUsageSpy: MockInstance<MeteringService['getRemainingUsage']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
const aiMetering = () =>
|
||||
withAiCostFactor(
|
||||
server.services.metering,
|
||||
server.clients.event,
|
||||
'ai-video',
|
||||
);
|
||||
|
||||
const makeProvider = (config: { apiKey?: string; apiBaseUrl?: string } = {}) =>
|
||||
new MiniMaxVideoProvider(
|
||||
{
|
||||
apiKey: config.apiKey ?? 'test-key',
|
||||
...(config.apiBaseUrl ? { apiBaseUrl: config.apiBaseUrl } : {}),
|
||||
pollIntervalMs: 1,
|
||||
},
|
||||
aiMetering(),
|
||||
);
|
||||
|
||||
beforeEach(() => {
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
remainingUsageSpy = vi.spyOn(server.services.metering, 'getRemainingUsage');
|
||||
remainingUsageSpy.mockResolvedValue(Number.MAX_SAFE_INTEGER);
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
incrementUsageSpy.mockResolvedValue({} as never);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const jsonResponse = (body: unknown, status = 200) =>
|
||||
new Response(JSON.stringify(body), {
|
||||
status,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
|
||||
const ok = { status_code: 0, status_msg: 'success' };
|
||||
|
||||
const h3Task = (overrides: Record<string, unknown> = {}) => ({
|
||||
id: 'task-1',
|
||||
model: 'MiniMax-H3',
|
||||
status: 'succeeded',
|
||||
content: { url: 'https://minimax.example/out.mp4' },
|
||||
resolution: '768P',
|
||||
duration: 5,
|
||||
usage: { output_seconds: 5 },
|
||||
...overrides,
|
||||
});
|
||||
|
||||
/** Queue the v2 create response followed by task query responses. */
|
||||
const mockH3Flow = (...polled: unknown[]) => {
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse({ task_id: 'task-1' }));
|
||||
for (const task of polled) {
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse({ task }));
|
||||
}
|
||||
};
|
||||
|
||||
const hailuoTask = (status: string, fileId = 'file-1') => ({
|
||||
task_id: 'task-1',
|
||||
status,
|
||||
file_id: status === 'Success' ? fileId : '',
|
||||
base_resp: ok,
|
||||
});
|
||||
|
||||
/** Queue the v1 create, query and file retrieve responses. */
|
||||
const mockHailuoFlow = (...polled: unknown[]) => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({ task_id: 'task-1', base_resp: ok }),
|
||||
);
|
||||
for (const task of polled) {
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse(task));
|
||||
}
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({
|
||||
file: {
|
||||
file_id: 1,
|
||||
download_url: 'https://minimax.example/hailuo.mp4',
|
||||
},
|
||||
base_resp: ok,
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
const sentUrl = (callIndex = 0) => String(fetchSpy.mock.calls[callIndex]![0]);
|
||||
|
||||
const sentBody = (callIndex = 0): Record<string, any> =>
|
||||
JSON.parse(
|
||||
(fetchSpy.mock.calls[callIndex]![1] as RequestInit).body as string,
|
||||
);
|
||||
|
||||
const generate = (params: Partial<IGenerateVideoParams>) =>
|
||||
withTestActor(() =>
|
||||
makeProvider().generate({ prompt: 'a ball rolls', ...params }),
|
||||
);
|
||||
|
||||
const rate = (id: string, key: string) =>
|
||||
MINIMAX_VIDEO_GENERATION_MODELS.find((m) => m.id === id)!.costs![key]!;
|
||||
|
||||
// -- Construction / catalog ------------------------------------------
|
||||
|
||||
describe('MiniMaxVideoProvider construction and catalog', () => {
|
||||
it('throws when no apiKey is supplied', () => {
|
||||
expect(
|
||||
() => new MiniMaxVideoProvider({ apiKey: '' }, aiMetering()),
|
||||
).toThrow(/API key/i);
|
||||
});
|
||||
|
||||
it('defaults to MiniMax H3 and lists the seven models', () => {
|
||||
const provider = makeProvider();
|
||||
expect(provider.getDefaultModel()).toBe('minimax-h3');
|
||||
expect(provider.models().map((m) => m.id)).toEqual([
|
||||
'minimax-h3',
|
||||
'minimax-h3-highspeed',
|
||||
'minimax-h3-lite',
|
||||
'minimax-h3-max',
|
||||
'minimax-hailuo-2.3',
|
||||
'minimax-hailuo-2.3-fast',
|
||||
'minimax-hailuo-02',
|
||||
]);
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// -- H3 request shape ------------------------------------------------
|
||||
|
||||
describe('MiniMaxVideoProvider.generate H3 request shape', () => {
|
||||
it('creates a v2 task with a ratio and polls it', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
const result = await generate({
|
||||
model: 'minimax-h3-lite',
|
||||
seconds: 6,
|
||||
size: '480p',
|
||||
width: 1080,
|
||||
height: 1920,
|
||||
});
|
||||
|
||||
expect(result).toBe('https://minimax.example/out.mp4');
|
||||
expect(sentUrl()).toBe('https://api.minimax.io/v2/video_generation');
|
||||
const init = fetchSpy.mock.calls[0]![1] as RequestInit & {
|
||||
headers: Record<string, string>;
|
||||
};
|
||||
expect(init.method).toBe('POST');
|
||||
expect(init.headers.Authorization).toBe('Bearer test-key');
|
||||
expect(sentBody()).toEqual({
|
||||
model: 'MiniMax-H3-Lite',
|
||||
content: [{ type: 'text', text: 'a ball rolls' }],
|
||||
duration: 6,
|
||||
resolution: '480P',
|
||||
ratio: '9:16',
|
||||
});
|
||||
expect(sentUrl(1)).toBe(
|
||||
'https://api.minimax.io/v2/query/video_generation/task-1',
|
||||
);
|
||||
});
|
||||
|
||||
it('sends the model name as MiniMax spells it and 2K upper-cased', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({ model: 'MiniMax:minimax/minimax-h3', size: '2K' });
|
||||
expect(sentBody()).toMatchObject({
|
||||
model: 'MiniMax-H3',
|
||||
resolution: '2K',
|
||||
duration: 5,
|
||||
});
|
||||
});
|
||||
|
||||
it('defaults the ratio to 16:9 and ignores ratios MiniMax lacks', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({ model: 'minimax-h3', width: 1000, height: 300 });
|
||||
expect(sentBody().ratio).toBe('16:9');
|
||||
});
|
||||
|
||||
it('sends frames with roles and no ratio', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({
|
||||
model: 'minimax-h3',
|
||||
input_reference: 'data:image/png;base64,AAAA',
|
||||
last_frame: 'https://example.com/last.png',
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
});
|
||||
const body = sentBody();
|
||||
expect(body.content).toEqual([
|
||||
{ type: 'text', text: 'a ball rolls' },
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: { url: 'data:image/png;base64,AAAA' },
|
||||
role: 'first_frame',
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: { url: 'https://example.com/last.png' },
|
||||
role: 'last_frame',
|
||||
},
|
||||
]);
|
||||
expect('ratio' in body).toBe(false);
|
||||
});
|
||||
|
||||
it('takes a last frame on its own', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({
|
||||
model: 'minimax-h3',
|
||||
last_frame: 'https://example.com/last.png',
|
||||
});
|
||||
expect(
|
||||
sentBody().content.map((c: { role?: string }) => c.role),
|
||||
).toEqual([undefined, 'last_frame']);
|
||||
});
|
||||
|
||||
it('wraps raw base64 as a PNG data URI', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({ model: 'minimax-h3', input_reference: 'BBBB' });
|
||||
expect(sentBody().content[1].image_url.url).toBe(
|
||||
'data:image/png;base64,BBBB',
|
||||
);
|
||||
});
|
||||
|
||||
it('sends references with a ratio', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({
|
||||
model: 'minimax-h3-max',
|
||||
reference_images: ['https://example.com/a.png', 'CCCC'],
|
||||
width: 4,
|
||||
height: 3,
|
||||
});
|
||||
const body = sentBody();
|
||||
expect(body.content).toEqual([
|
||||
{ type: 'text', text: 'a ball rolls' },
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: { url: 'https://example.com/a.png' },
|
||||
role: 'reference_image',
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: { url: 'data:image/png;base64,CCCC' },
|
||||
role: 'reference_image',
|
||||
},
|
||||
]);
|
||||
expect(body.ratio).toBe('4:3');
|
||||
});
|
||||
|
||||
it("drops the chat API's /v1 from a base URL override", async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await withTestActor(() =>
|
||||
makeProvider({
|
||||
apiBaseUrl: 'https://api.minimaxi.com/v1/',
|
||||
}).generate({ prompt: 'hi' }),
|
||||
);
|
||||
expect(sentUrl()).toBe('https://api.minimaxi.com/v2/video_generation');
|
||||
});
|
||||
|
||||
it('keeps polling while the task is queued or running', async () => {
|
||||
mockH3Flow(
|
||||
h3Task({ status: 'queued', content: undefined, usage: {} }),
|
||||
h3Task({ status: 'running', content: undefined, usage: {} }),
|
||||
h3Task(),
|
||||
);
|
||||
const result = await generate({});
|
||||
expect(result).toBe('https://minimax.example/out.mp4');
|
||||
expect(fetchSpy).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
|
||||
it('treats a rate-limited poll as a missed poll', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse({ task_id: 'task-1' }));
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse(
|
||||
{
|
||||
type: 'error',
|
||||
error: {
|
||||
type: 'rate_limit_error',
|
||||
message: 'rate limit (1002)',
|
||||
},
|
||||
},
|
||||
429,
|
||||
),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse({ task: h3Task() }));
|
||||
const result = await generate({});
|
||||
expect(result).toBe('https://minimax.example/out.mp4');
|
||||
expect(fetchSpy).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it('returns the sample clip in test mode without calling MiniMax', async () => {
|
||||
const result = await generate({ test_mode: true });
|
||||
expect(result).toBe('https://assets.puter.site/txt2vid.mp4');
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// -- Hailuo request shape --------------------------------------------
|
||||
|
||||
describe('MiniMaxVideoProvider.generate Hailuo request shape', () => {
|
||||
it('creates a v1 task, polls it and exchanges the file id for a URL', async () => {
|
||||
mockHailuoFlow(hailuoTask('Preparing'), hailuoTask('Success'));
|
||||
const result = await generate({
|
||||
model: 'minimax-hailuo-2.3',
|
||||
seconds: 10,
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
});
|
||||
|
||||
expect(result).toBe('https://minimax.example/hailuo.mp4');
|
||||
expect(sentUrl()).toBe('https://api.minimax.io/v1/video_generation');
|
||||
expect(sentBody()).toEqual({
|
||||
model: 'MiniMax-Hailuo-2.3',
|
||||
prompt: 'a ball rolls',
|
||||
duration: 10,
|
||||
resolution: '768P',
|
||||
});
|
||||
expect(sentUrl(1)).toBe(
|
||||
'https://api.minimax.io/v1/query/video_generation?task_id=task-1',
|
||||
);
|
||||
expect(sentUrl(3)).toBe(
|
||||
'https://api.minimax.io/v1/files/retrieve?file_id=file-1',
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to 6 seconds at 1080p, which takes no 10-second clip', async () => {
|
||||
mockHailuoFlow(hailuoTask('Success'));
|
||||
await generate({
|
||||
model: 'minimax-hailuo-2.3-fast',
|
||||
seconds: 10,
|
||||
size: '1080p',
|
||||
input_reference: 'https://example.com/first.png',
|
||||
});
|
||||
expect(sentBody()).toMatchObject({
|
||||
model: 'MiniMax-Hailuo-2.3-Fast',
|
||||
duration: 6,
|
||||
resolution: '1080P',
|
||||
first_frame_image: 'https://example.com/first.png',
|
||||
});
|
||||
});
|
||||
|
||||
it.each(['768p', '1080p'])('sends first and last frames at %s', async (size) => {
|
||||
mockHailuoFlow(hailuoTask('Success'));
|
||||
await generate({
|
||||
model: 'minimax-hailuo-02',
|
||||
size,
|
||||
input_reference: 'DDDD',
|
||||
last_frame: 'https://example.com/last.png',
|
||||
});
|
||||
expect(sentBody()).toEqual({
|
||||
model: 'MiniMax-Hailuo-02',
|
||||
prompt: 'a ball rolls',
|
||||
duration: 6,
|
||||
resolution: size.toUpperCase(),
|
||||
first_frame_image: 'data:image/png;base64,DDDD',
|
||||
last_frame_image: 'https://example.com/last.png',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// -- Validation ------------------------------------------------------
|
||||
|
||||
describe('MiniMaxVideoProvider.generate rejects invalid input before calling MiniMax', () => {
|
||||
it.each<[string, Partial<IGenerateVideoParams>]>([
|
||||
['a blank prompt', { prompt: ' ' }],
|
||||
[
|
||||
'a prompt over 7000 characters on H3',
|
||||
{ model: 'minimax-h3', prompt: 'x'.repeat(7001) },
|
||||
],
|
||||
[
|
||||
'a prompt over 2000 characters on Hailuo',
|
||||
{ model: 'minimax-hailuo-2.3', prompt: 'x'.repeat(2001) },
|
||||
],
|
||||
[
|
||||
'a source video',
|
||||
{ model: 'minimax-h3', input_video: 'https://example.com/v.mp4' },
|
||||
],
|
||||
[
|
||||
'reference images on Hailuo',
|
||||
{
|
||||
model: 'minimax-hailuo-02',
|
||||
reference_images: ['https://example.com/r.png'],
|
||||
},
|
||||
],
|
||||
[
|
||||
'reference images with a first frame on H3',
|
||||
{
|
||||
model: 'minimax-h3',
|
||||
input_reference: 'https://example.com/f.png',
|
||||
reference_images: ['https://example.com/r.png'],
|
||||
},
|
||||
],
|
||||
[
|
||||
'more than 5 images on H3',
|
||||
{
|
||||
model: 'minimax-h3',
|
||||
reference_images: Array(6).fill('https://example.com/r.png'),
|
||||
},
|
||||
],
|
||||
[
|
||||
'more than 2 images on H3 Max',
|
||||
{
|
||||
model: 'minimax-h3-max',
|
||||
reference_images: Array(3).fill('https://example.com/r.png'),
|
||||
},
|
||||
],
|
||||
[
|
||||
'a last frame on Hailuo 2.3',
|
||||
{
|
||||
model: 'minimax-hailuo-2.3',
|
||||
input_reference: 'https://example.com/f.png',
|
||||
last_frame: 'https://example.com/l.png',
|
||||
},
|
||||
],
|
||||
[
|
||||
'a last frame without a first frame on Hailuo 02',
|
||||
{
|
||||
model: 'minimax-hailuo-02',
|
||||
last_frame: 'https://example.com/l.png',
|
||||
},
|
||||
],
|
||||
[
|
||||
'512p without a first frame on Hailuo 02',
|
||||
{ model: 'minimax-hailuo-02', size: '512p' },
|
||||
],
|
||||
[
|
||||
'a missing first frame on Hailuo 2.3 Fast',
|
||||
{ model: 'minimax-hailuo-2.3-fast' },
|
||||
],
|
||||
[
|
||||
'512p with first and last frames on Hailuo 02',
|
||||
{
|
||||
model: 'minimax-hailuo-02',
|
||||
size: '512p',
|
||||
input_reference: 'https://example.com/first.png',
|
||||
last_frame: 'https://example.com/last.png',
|
||||
},
|
||||
],
|
||||
[
|
||||
'a non-string image',
|
||||
{ model: 'minimax-h3', input_reference: 42 as never },
|
||||
],
|
||||
[
|
||||
'a non-array reference_images',
|
||||
{ model: 'minimax-h3', reference_images: 'x' as never },
|
||||
],
|
||||
])('rejects %s', async (_label, params) => {
|
||||
fetchSpy.mockRejectedValue(new Error('Unexpected MiniMax request'));
|
||||
const reserveCreditsSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCredits',
|
||||
);
|
||||
await expect(generate(params)).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(reserveCreditsSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('accepts a 7000-character prompt on H3', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({ model: 'minimax-h3', prompt: 'x'.repeat(7000) });
|
||||
expect(fetchSpy).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('accepts five images on H3', async () => {
|
||||
mockH3Flow(h3Task());
|
||||
await generate({
|
||||
model: 'minimax-h3',
|
||||
reference_images: Array(5).fill('https://example.com/r.png'),
|
||||
});
|
||||
expect(sentBody().content).toHaveLength(6);
|
||||
});
|
||||
});
|
||||
|
||||
// -- Billing ---------------------------------------------------------
|
||||
|
||||
describe('MiniMaxVideoProvider billing', () => {
|
||||
it.each<[Partial<IGenerateVideoParams>, string, string, number]>([
|
||||
[
|
||||
{ model: 'minimax-h3', seconds: 5 },
|
||||
'minimax-h3',
|
||||
'per-second:768p',
|
||||
5,
|
||||
],
|
||||
[
|
||||
{ model: 'minimax-h3', seconds: 8, size: '2k' },
|
||||
'minimax-h3',
|
||||
'per-second:2k',
|
||||
8,
|
||||
],
|
||||
[
|
||||
{ model: 'minimax-h3-lite', seconds: 4, size: '480p' },
|
||||
'minimax-h3-lite',
|
||||
'per-second:480p',
|
||||
4,
|
||||
],
|
||||
])(
|
||||
'meters %o per second under its cost key',
|
||||
async (params, id, key, secs) => {
|
||||
mockH3Flow(
|
||||
h3Task({ duration: secs, usage: { output_seconds: secs } }),
|
||||
);
|
||||
await generate(params);
|
||||
expect(incrementUsageSpy).toHaveBeenCalledTimes(1);
|
||||
const [, usageType, amount, cost] =
|
||||
incrementUsageSpy.mock.calls[0]!;
|
||||
expect(usageType).toBe(`minimax-video-generation:${id}:${key}`);
|
||||
expect(amount).toBe(secs);
|
||||
expect(cost).toBeCloseTo(secs * rate(id, key) * 1_000_000);
|
||||
},
|
||||
);
|
||||
|
||||
it('bills the seconds of output the task reports', async () => {
|
||||
mockH3Flow(h3Task({ duration: 5, usage: { output_seconds: 4 } }));
|
||||
await generate({ model: 'minimax-h3', seconds: 5 });
|
||||
const [, , amount, cost] = incrementUsageSpy.mock.calls[0]!;
|
||||
expect(amount).toBe(4);
|
||||
expect(cost).toBeCloseTo(4 * 8 * 1_000_000);
|
||||
});
|
||||
|
||||
it('bills the requested seconds when the task reports no usage', async () => {
|
||||
mockH3Flow(h3Task({ duration: undefined, usage: undefined }));
|
||||
await generate({ model: 'minimax-h3', seconds: 7 });
|
||||
expect(incrementUsageSpy.mock.calls[0]![2]).toBe(7);
|
||||
});
|
||||
|
||||
it('shortens an H3 clip to what the balance buys', async () => {
|
||||
remainingUsageSpy.mockResolvedValue(6 * 8 * 1_000_000);
|
||||
mockH3Flow(h3Task({ duration: 6, usage: { output_seconds: 6 } }));
|
||||
await generate({ model: 'minimax-h3', seconds: 10 });
|
||||
expect(sentBody().duration).toBe(6);
|
||||
});
|
||||
|
||||
it('rejects with insufficient_funds before creating an H3 task it cannot afford', async () => {
|
||||
remainingUsageSpy.mockResolvedValue(1000);
|
||||
await expect(generate({ model: 'minimax-h3' })).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each<[Partial<IGenerateVideoParams>, string, string]>([
|
||||
[
|
||||
{ model: 'minimax-hailuo-2.3', seconds: 10 },
|
||||
'minimax-hailuo-2.3',
|
||||
'per-video:768p:10',
|
||||
],
|
||||
[
|
||||
{ model: 'minimax-hailuo-02', size: '1080p' },
|
||||
'minimax-hailuo-02',
|
||||
'per-video:1080p:6',
|
||||
],
|
||||
[
|
||||
{
|
||||
model: 'minimax-hailuo-02',
|
||||
size: '512p',
|
||||
seconds: 10,
|
||||
input_reference: 'https://example.com/f.png',
|
||||
},
|
||||
'minimax-hailuo-02',
|
||||
'per-video:512p:10',
|
||||
],
|
||||
])('meters a Hailuo clip %o as one video', async (params, id, key) => {
|
||||
mockHailuoFlow(hailuoTask('Success'));
|
||||
await generate(params);
|
||||
const [, usageType, amount, cost] = incrementUsageSpy.mock.calls[0]!;
|
||||
expect(usageType).toBe(`minimax-video-generation:${id}:${key}`);
|
||||
expect(amount).toBe(1);
|
||||
expect(cost).toBe(rate(id, key) * 1_000_000);
|
||||
});
|
||||
|
||||
it('rejects with insufficient_funds before creating a Hailuo task it cannot afford', async () => {
|
||||
remainingUsageSpy.mockResolvedValue(1000);
|
||||
await expect(
|
||||
generate({ model: 'minimax-hailuo-2.3' }),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
legacyCode: 'insufficient_funds',
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// -- Failures --------------------------------------------------------
|
||||
|
||||
describe('MiniMaxVideoProvider failures', () => {
|
||||
it.each(['failed', 'cancelled'])(
|
||||
'maps a %s H3 task to upstream_failed and meters nothing',
|
||||
async (status) => {
|
||||
mockH3Flow(
|
||||
h3Task({
|
||||
status,
|
||||
content: undefined,
|
||||
error: { code: '1000', message: 'unknown error' },
|
||||
}),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
statusCode: 502,
|
||||
legacyCode: 'upstream_failed',
|
||||
});
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it('maps an H3 task failed by the content filter to moderation_flagged', async () => {
|
||||
mockH3Flow(
|
||||
h3Task({
|
||||
status: 'failed',
|
||||
content: undefined,
|
||||
error: {
|
||||
code: '1026',
|
||||
message: 'video description contains sensitive content',
|
||||
},
|
||||
}),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
code: 'moderation_flagged',
|
||||
});
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('carries a v2 parameter rejection with its status and code', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse(
|
||||
{
|
||||
type: 'error',
|
||||
error: {
|
||||
type: 'bad_request_error',
|
||||
message:
|
||||
'invalid params, model MiniMax-H3-Lite does not support resolution 2K (2013)',
|
||||
http_code: '400',
|
||||
},
|
||||
},
|
||||
400,
|
||||
),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
status: 400,
|
||||
code: '2013',
|
||||
provider: 'minimax',
|
||||
});
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('reports a model MiniMax no longer serves as upstream_failed', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse(
|
||||
{
|
||||
type: 'error',
|
||||
error: {
|
||||
type: 'bad_request_error',
|
||||
message:
|
||||
'invalid params, this model is not supported by /v2/video_generation (2013)',
|
||||
},
|
||||
},
|
||||
400,
|
||||
),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
statusCode: 502,
|
||||
legacyCode: 'upstream_failed',
|
||||
});
|
||||
});
|
||||
|
||||
it('hides the auth message behind a 401', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse(
|
||||
{
|
||||
type: 'error',
|
||||
error: {
|
||||
type: 'authorized_error',
|
||||
message:
|
||||
'login fail: carry the key in Authorization (1004)',
|
||||
},
|
||||
},
|
||||
401,
|
||||
),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
status: 401,
|
||||
message: 'MiniMax rejected the API key',
|
||||
});
|
||||
});
|
||||
|
||||
it('surfaces an empty balance as status 402 with the MiniMax code', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse(
|
||||
{
|
||||
type: 'error',
|
||||
error: {
|
||||
type: 'insufficient_balance_error',
|
||||
message: 'insufficient balance (1008)',
|
||||
},
|
||||
},
|
||||
402,
|
||||
),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
status: 402,
|
||||
code: '1008',
|
||||
provider: 'minimax',
|
||||
});
|
||||
});
|
||||
|
||||
it('maps a content block at submit to moderation_flagged', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse(
|
||||
{
|
||||
type: 'error',
|
||||
error: {
|
||||
type: 'unprocessable_entity_error',
|
||||
message: 'input new_sensitive (1026)',
|
||||
},
|
||||
},
|
||||
422,
|
||||
),
|
||||
);
|
||||
await expect(generate({})).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
code: 'moderation_flagged',
|
||||
});
|
||||
});
|
||||
|
||||
it('reads a v1 verdict out of base_resp under HTTP 200', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({
|
||||
task_id: '',
|
||||
base_resp: {
|
||||
status_code: 1008,
|
||||
status_msg: 'insufficient balance',
|
||||
},
|
||||
}),
|
||||
);
|
||||
await expect(
|
||||
generate({ model: 'minimax-hailuo-2.3' }),
|
||||
).rejects.toMatchObject({
|
||||
status: 402,
|
||||
code: '1008',
|
||||
provider: 'minimax',
|
||||
});
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('reports a Hailuo model MiniMax no longer serves as upstream_failed', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({
|
||||
task_id: '',
|
||||
base_resp: {
|
||||
status_code: 2013,
|
||||
status_msg: 'invalid params, incorrect model param input',
|
||||
},
|
||||
}),
|
||||
);
|
||||
await expect(
|
||||
generate({ model: 'minimax-hailuo-2.3' }),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 502,
|
||||
legacyCode: 'upstream_failed',
|
||||
});
|
||||
});
|
||||
|
||||
it('maps a failed Hailuo task to upstream_failed and meters nothing', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({ task_id: 'task-1', base_resp: ok }),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse(hailuoTask('Fail')));
|
||||
await expect(
|
||||
generate({ model: 'minimax-hailuo-2.3' }),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 502,
|
||||
legacyCode: 'upstream_failed',
|
||||
});
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('maps a Hailuo clip removed by the output filter to moderation_flagged', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({ task_id: 'task-1', base_resp: ok }),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({
|
||||
task_id: 'task-1',
|
||||
status: 'Fail',
|
||||
base_resp: {
|
||||
status_code: 1027,
|
||||
status_msg: 'output new_sensitive',
|
||||
},
|
||||
}),
|
||||
);
|
||||
await expect(
|
||||
generate({ model: 'minimax-hailuo-2.3' }),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
code: 'moderation_flagged',
|
||||
});
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('fails when the finished H3 task has no video', async () => {
|
||||
mockH3Flow(h3Task({ content: {} }));
|
||||
await expect(generate({})).rejects.toThrow(/video URL/);
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('fails when the Hailuo file has no download URL', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({ task_id: 'task-1', base_resp: ok }),
|
||||
);
|
||||
fetchSpy.mockResolvedValueOnce(jsonResponse(hailuoTask('Success')));
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
jsonResponse({ file: { file_id: 1 }, base_resp: ok }),
|
||||
);
|
||||
await expect(generate({ model: 'minimax-hailuo-2.3' })).rejects.toThrow(
|
||||
/video URL/,
|
||||
);
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -1,690 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { Actor } from '../../../../core/actor.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import { HttpError } from '../../../../core/http/HttpError.js';
|
||||
import { insufficientCreditsError } from '../../../../services/metering/enforcement.js';
|
||||
import type { CreditHold } from '../../../../services/metering/types.js';
|
||||
import type { AiMeteringService } from '../../../util/aiCostFactor.js';
|
||||
import { readBoundedBody } from '../../../util/fileInput.js';
|
||||
import { toUrlOrDataUri } from '../../../util/imageInput.js';
|
||||
import { sanitizeUpstreamMessage } from '../../../util/upstreamErrors.js';
|
||||
import { capSecondsToRemainingCredits } from '../../creditCap.js';
|
||||
import type { IGenerateVideoParams, IVideoModel } from '../../types.js';
|
||||
import { VideoProvider } from '../VideoProvider.js';
|
||||
import {
|
||||
pollUntilSettled,
|
||||
VIDEO_HOLD_TTL_MS,
|
||||
videoJobFailure,
|
||||
} from '../polling.js';
|
||||
import {
|
||||
MINIMAX_VIDEO_GENERATION_MODELS,
|
||||
MINIMAX_VIDEO_SPECS,
|
||||
type MiniMaxVideoSpec,
|
||||
} from './models.js';
|
||||
|
||||
const DEFAULT_TEST_VIDEO_URL = 'https://assets.puter.site/txt2vid.mp4';
|
||||
const DEFAULT_BASE_URL = 'https://api.minimax.io';
|
||||
const DEFAULT_MODEL = 'minimax-h3';
|
||||
const DEFAULT_POLL_INTERVAL_MS = 5_000;
|
||||
const REQUEST_TIMEOUT_MS = 60_000;
|
||||
const MAX_RESPONSE_BYTES = 1024 * 1024;
|
||||
const PROVIDER = 'minimax';
|
||||
|
||||
const ASPECT_RATIOS = ['21:9', '16:9', '4:3', '1:1', '3:4', '9:16'];
|
||||
const DEFAULT_ASPECT_RATIO = '16:9';
|
||||
|
||||
// MiniMax's business codes, from the trailing "(NNNN)" of a v2 error message
|
||||
// or `base_resp.status_code` of a v1 reply.
|
||||
// https://platform.minimax.io/docs/api-reference/errorcode
|
||||
const AUTH_CODES = new Set([1004, 2049]);
|
||||
const BALANCE_CODES = new Set([1008, 2053, 2056, 2067]);
|
||||
const RATE_LIMIT_CODES = new Set([1002, 1039, 1041, 1045]);
|
||||
const MODERATION_CODES = new Set([1026, 1027]);
|
||||
const MODEL_UNAVAILABLE_CODES = new Set([2022, 2061, 2068, 2069]);
|
||||
const INVALID_PARAM_CODES = new Set([2013, 2014]);
|
||||
/** A 2013 about the model itself, which means the catalog has drifted. */
|
||||
const UNSUPPORTED_MODEL_PATTERN =
|
||||
/this model is not supported|incorrect model param/i;
|
||||
|
||||
type MiniMaxVideoConfig = {
|
||||
apiKey: string;
|
||||
apiBaseUrl?: string;
|
||||
/** Test hook: polling cadence for task status checks. */
|
||||
pollIntervalMs?: number;
|
||||
};
|
||||
|
||||
interface MiniMaxEnvelope {
|
||||
base_resp?: { status_code?: number; status_msg?: string };
|
||||
error?: { type?: string; message?: string; http_code?: string };
|
||||
}
|
||||
|
||||
interface H3Task {
|
||||
id: string;
|
||||
status: 'queued' | 'running' | 'succeeded' | 'failed' | 'cancelled';
|
||||
content?: { url?: string };
|
||||
error?: { code?: string; message?: string };
|
||||
duration?: number;
|
||||
usage?: { output_seconds?: number };
|
||||
}
|
||||
|
||||
interface HailuoTask {
|
||||
task_id?: string;
|
||||
status?: 'Preparing' | 'Queueing' | 'Processing' | 'Success' | 'Fail';
|
||||
file_id?: string;
|
||||
}
|
||||
|
||||
const badRequest = (message: string) =>
|
||||
new HttpError(400, message, { legacyCode: 'bad_request' });
|
||||
|
||||
const present = (value: unknown): boolean =>
|
||||
value !== undefined && value !== null;
|
||||
|
||||
const statusForCode = (code: number, httpStatus: number): number => {
|
||||
if (AUTH_CODES.has(code)) return 401;
|
||||
if (BALANCE_CODES.has(code)) return 402;
|
||||
if (RATE_LIMIT_CODES.has(code)) return 429;
|
||||
if (INVALID_PARAM_CODES.has(code)) return 400;
|
||||
return httpStatus >= 400 ? httpStatus : 502;
|
||||
};
|
||||
|
||||
/** The code a v2 error message ends with, e.g. "invalid params (2013)". */
|
||||
const parseCode = (message: string): number | undefined => {
|
||||
const match = message.match(/\((\d{4,5})\)\s*$/);
|
||||
return match ? Number.parseInt(match[1], 10) : undefined;
|
||||
};
|
||||
|
||||
/** A request MiniMax refused, shaped for `translateProviderError`. */
|
||||
const minimaxError = (
|
||||
code: number | undefined,
|
||||
message: string,
|
||||
status: number,
|
||||
): Error => {
|
||||
// A model MiniMax doesn't serve is Puter's pick, so the catalog has
|
||||
// drifted from MiniMax's: our fault, not the caller's, and worth
|
||||
// someone's attention.
|
||||
if (
|
||||
code !== undefined &&
|
||||
(MODEL_UNAVAILABLE_CODES.has(code) ||
|
||||
(code === 2013 && UNSUPPORTED_MODEL_PATTERN.test(message)))
|
||||
) {
|
||||
return new HttpError(
|
||||
502,
|
||||
'MiniMax does not support the model Puter requested',
|
||||
{
|
||||
legacyCode: 'upstream_failed',
|
||||
alarmSeverity: 'warning',
|
||||
fields: {
|
||||
provider: PROVIDER,
|
||||
upstreamCode: String(code),
|
||||
upstreamMessage: sanitizeUpstreamMessage(message),
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
if (code !== undefined && MODERATION_CODES.has(code)) {
|
||||
return new HttpError(
|
||||
400,
|
||||
sanitizeUpstreamMessage(message) ||
|
||||
'MiniMax rejected the request under its content policy',
|
||||
{
|
||||
legacyCode: 'bad_request',
|
||||
code: 'moderation_flagged',
|
||||
fields: { provider: PROVIDER, upstreamCode: String(code) },
|
||||
},
|
||||
);
|
||||
}
|
||||
// MiniMax's auth messages point at its own console; the caller can't act
|
||||
// on them.
|
||||
const isAuthFailure = code !== undefined && AUTH_CODES.has(code);
|
||||
return Object.assign(
|
||||
new Error(isAuthFailure ? 'MiniMax rejected the API key' : message),
|
||||
{
|
||||
status:
|
||||
code === undefined
|
||||
? status >= 400
|
||||
? status
|
||||
: 502
|
||||
: statusForCode(code, status),
|
||||
code: code === undefined ? undefined : String(code),
|
||||
provider: PROVIDER,
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* MiniMax video generation (H3 and Hailuo).
|
||||
*
|
||||
* H3 models go through the v2 task API: a `content` list of text, frames and
|
||||
* references creates a task that is polled until it leaves queued/running, and
|
||||
* the clip's URL comes back on the task. Billing is per second of output at a
|
||||
* rate set by resolution. Hailuo models go through the v1 API, which takes a
|
||||
* prompt and frame images, is polled on its own status endpoint and hands back
|
||||
* a file id to exchange for a download URL. They are billed per clip, by
|
||||
* resolution and length.
|
||||
* https://platform.minimax.io/docs/api-reference/video-generation-v2-create
|
||||
*/
|
||||
export class MiniMaxVideoProvider extends VideoProvider {
|
||||
#apiKey: string;
|
||||
#baseUrl: string;
|
||||
#pollIntervalMs: number;
|
||||
#meteringService: AiMeteringService;
|
||||
|
||||
constructor(
|
||||
config: MiniMaxVideoConfig,
|
||||
meteringService: AiMeteringService,
|
||||
) {
|
||||
super();
|
||||
if (!config.apiKey) {
|
||||
throw new Error('MiniMax video generation requires an API key');
|
||||
}
|
||||
this.#apiKey = config.apiKey;
|
||||
// The shared chat block's base URL ends in the chat API's `/v1`; the
|
||||
// video API lives beside it on the same host.
|
||||
this.#baseUrl = (config.apiBaseUrl ?? DEFAULT_BASE_URL)
|
||||
.replace(/\/+$/, '')
|
||||
.replace(/\/v1$/, '');
|
||||
this.#pollIntervalMs =
|
||||
config.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS;
|
||||
this.#meteringService = meteringService;
|
||||
}
|
||||
|
||||
getDefaultModel(): string {
|
||||
return DEFAULT_MODEL;
|
||||
}
|
||||
|
||||
models(): IVideoModel[] {
|
||||
return MINIMAX_VIDEO_GENERATION_MODELS;
|
||||
}
|
||||
|
||||
async generate(params: IGenerateVideoParams): Promise<unknown> {
|
||||
const {
|
||||
prompt,
|
||||
model: requestedModel,
|
||||
seconds,
|
||||
duration,
|
||||
size,
|
||||
resolution,
|
||||
width,
|
||||
height,
|
||||
input_reference: inputReference,
|
||||
last_frame: lastFrame,
|
||||
reference_images: referenceImages,
|
||||
input_video: inputVideo,
|
||||
test_mode: testMode,
|
||||
} = params ?? {};
|
||||
|
||||
if (typeof prompt !== 'string' || !prompt.trim()) {
|
||||
throw badRequest('prompt must be a non-empty string');
|
||||
}
|
||||
const model = this.#getModel(requestedModel);
|
||||
const spec = MINIMAX_VIDEO_SPECS[model.id];
|
||||
if (prompt.length > spec.maxPromptChars) {
|
||||
throw badRequest(
|
||||
`${model.id} takes a prompt of at most ${spec.maxPromptChars} characters`,
|
||||
);
|
||||
}
|
||||
if (present(inputVideo)) {
|
||||
throw badRequest(`${model.id} does not accept a source video`);
|
||||
}
|
||||
|
||||
const resolutionKey = this.#resolution(size ?? resolution, model);
|
||||
const firstFrame = this.#image(inputReference, 'input_reference');
|
||||
const endFrame = this.#image(lastFrame, 'last_frame');
|
||||
const references = this.#references(referenceImages);
|
||||
this.#checkInputs(spec, model, resolutionKey, {
|
||||
firstFrame,
|
||||
endFrame,
|
||||
references,
|
||||
});
|
||||
|
||||
if (testMode) {
|
||||
return DEFAULT_TEST_VIDEO_URL;
|
||||
}
|
||||
|
||||
const actor = Context.get('actor') as Actor | undefined;
|
||||
if (!actor) {
|
||||
throw new HttpError(401, 'Authentication required', {
|
||||
legacyCode: 'unauthorized',
|
||||
});
|
||||
}
|
||||
|
||||
const requestedSeconds =
|
||||
this.#coerceSeconds(seconds ?? duration) ??
|
||||
model.durationSeconds![0];
|
||||
const aspectRatio = this.#aspectRatio(width, height);
|
||||
const images = { firstFrame, endFrame, references };
|
||||
|
||||
if (spec.api === 'v2') {
|
||||
return this.#generateH3(actor, model, spec, {
|
||||
prompt,
|
||||
resolutionKey,
|
||||
requestedSeconds,
|
||||
aspectRatio,
|
||||
...images,
|
||||
});
|
||||
}
|
||||
return this.#generateHailuo(actor, model, spec, {
|
||||
prompt,
|
||||
resolutionKey,
|
||||
requestedSeconds,
|
||||
...images,
|
||||
});
|
||||
}
|
||||
|
||||
// -- H3 (v2 task API, per second) ---------------------------------------
|
||||
|
||||
async #generateH3(
|
||||
actor: Actor,
|
||||
model: IVideoModel,
|
||||
spec: MiniMaxVideoSpec,
|
||||
input: {
|
||||
prompt: string;
|
||||
resolutionKey: string;
|
||||
requestedSeconds: number;
|
||||
aspectRatio: string | undefined;
|
||||
firstFrame?: string;
|
||||
endFrame?: string;
|
||||
references: string[];
|
||||
},
|
||||
): Promise<string> {
|
||||
const costKey = `per-second:${input.resolutionKey}`;
|
||||
const centsPerSecond = model.costs?.[costKey];
|
||||
if (!centsPerSecond) {
|
||||
throw new Error(
|
||||
`No pricing configured for video model ${model.id} at ${costKey}`,
|
||||
);
|
||||
}
|
||||
const usageType = `minimax-video-generation:${model.id}:${costKey}`;
|
||||
const { seconds: cappedSeconds, hold } =
|
||||
await capSecondsToRemainingCredits({
|
||||
metering: this.#meteringService,
|
||||
actor,
|
||||
usageType,
|
||||
perSecondMicroCents: centsPerSecond * 1_000_000,
|
||||
requestedSeconds: input.requestedSeconds,
|
||||
allowedSeconds: model.durationSeconds,
|
||||
});
|
||||
|
||||
try {
|
||||
const content: Array<Record<string, unknown>> = [
|
||||
{ type: 'text', text: input.prompt },
|
||||
];
|
||||
if (input.firstFrame) {
|
||||
content.push({
|
||||
type: 'image_url',
|
||||
image_url: { url: input.firstFrame },
|
||||
role: 'first_frame',
|
||||
});
|
||||
}
|
||||
if (input.endFrame) {
|
||||
content.push({
|
||||
type: 'image_url',
|
||||
image_url: { url: input.endFrame },
|
||||
role: 'last_frame',
|
||||
});
|
||||
}
|
||||
for (const url of input.references) {
|
||||
content.push({
|
||||
type: 'image_url',
|
||||
image_url: { url },
|
||||
role: 'reference_image',
|
||||
});
|
||||
}
|
||||
const body: Record<string, unknown> = {
|
||||
model: spec.apiModel,
|
||||
content,
|
||||
duration: cappedSeconds,
|
||||
resolution: input.resolutionKey.toUpperCase(),
|
||||
};
|
||||
// A clip from frames takes their shape; MiniMax requires a ratio
|
||||
// for everything else.
|
||||
if (!input.firstFrame && !input.endFrame) {
|
||||
body.ratio = input.aspectRatio ?? DEFAULT_ASPECT_RATIO;
|
||||
}
|
||||
|
||||
const created = await this.#request<{ task_id?: string }>(
|
||||
'POST',
|
||||
'/v2/video_generation',
|
||||
body,
|
||||
);
|
||||
if (!created.task_id) {
|
||||
throw new Error('MiniMax response did not include a task id');
|
||||
}
|
||||
const task = await pollUntilSettled<H3Task>({
|
||||
provider: PROVIDER,
|
||||
providerLabel: 'MiniMax',
|
||||
intervalMs: this.#pollIntervalMs,
|
||||
fetch: async () => {
|
||||
const { task } = await this.#request<{ task?: H3Task }>(
|
||||
'GET',
|
||||
`/v2/query/video_generation/${encodeURIComponent(created.task_id!)}`,
|
||||
);
|
||||
if (!task) {
|
||||
throw new Error('MiniMax task query returned no task');
|
||||
}
|
||||
return task;
|
||||
},
|
||||
isPending: (task) =>
|
||||
task.status === 'queued' || task.status === 'running',
|
||||
});
|
||||
if (task.status !== 'succeeded') {
|
||||
throw videoJobFailure(
|
||||
PROVIDER,
|
||||
task.error?.message || `Video generation ${task.status}`,
|
||||
task.error?.code,
|
||||
);
|
||||
}
|
||||
const url = task.content?.url;
|
||||
if (typeof url !== 'string' || !url.trim()) {
|
||||
throw new Error('MiniMax response did not include a video URL');
|
||||
}
|
||||
|
||||
// MiniMax bills the seconds of output the task reports.
|
||||
const outputSeconds = task.usage?.output_seconds ?? task.duration;
|
||||
const billedSeconds =
|
||||
typeof outputSeconds === 'number' &&
|
||||
Number.isFinite(outputSeconds) &&
|
||||
outputSeconds > 0
|
||||
? Math.max(1, Math.round(outputSeconds))
|
||||
: cappedSeconds;
|
||||
await this.#meteringService.incrementUsage(
|
||||
actor,
|
||||
usageType,
|
||||
billedSeconds,
|
||||
Math.round(billedSeconds * centsPerSecond * 1_000_000),
|
||||
);
|
||||
return url;
|
||||
} finally {
|
||||
await hold.release();
|
||||
}
|
||||
}
|
||||
|
||||
// -- Hailuo (v1 API, per clip) ------------------------------------------
|
||||
|
||||
async #generateHailuo(
|
||||
actor: Actor,
|
||||
model: IVideoModel,
|
||||
spec: MiniMaxVideoSpec,
|
||||
input: {
|
||||
prompt: string;
|
||||
resolutionKey: string;
|
||||
requestedSeconds: number;
|
||||
firstFrame?: string;
|
||||
endFrame?: string;
|
||||
},
|
||||
): Promise<string> {
|
||||
// A clip length the resolution doesn't offer falls back to the first
|
||||
// one it does, the way the driver treats any unsupported duration.
|
||||
const offered = (secs: number) =>
|
||||
model.costs?.[`per-video:${input.resolutionKey}:${secs}`];
|
||||
const clipSeconds = offered(input.requestedSeconds)
|
||||
? input.requestedSeconds
|
||||
: model.durationSeconds!.find(offered)!;
|
||||
const costKey = `per-video:${input.resolutionKey}:${clipSeconds}`;
|
||||
const centsPerVideo = model.costs![costKey]!;
|
||||
const usageType = `minimax-video-generation:${model.id}:${costKey}`;
|
||||
const microCents = centsPerVideo * 1_000_000;
|
||||
const hold: CreditHold | null =
|
||||
await this.#meteringService.reserveAiCredits(
|
||||
actor,
|
||||
usageType,
|
||||
microCents,
|
||||
{ ttlMs: VIDEO_HOLD_TTL_MS },
|
||||
);
|
||||
if (!hold) {
|
||||
throw insufficientCreditsError();
|
||||
}
|
||||
|
||||
try {
|
||||
const body: Record<string, unknown> = {
|
||||
model: spec.apiModel,
|
||||
prompt: input.prompt,
|
||||
duration: clipSeconds,
|
||||
resolution: input.resolutionKey.toUpperCase(),
|
||||
};
|
||||
if (input.firstFrame) body.first_frame_image = input.firstFrame;
|
||||
if (input.endFrame) body.last_frame_image = input.endFrame;
|
||||
|
||||
const created = await this.#request<{ task_id?: string }>(
|
||||
'POST',
|
||||
'/v1/video_generation',
|
||||
body,
|
||||
);
|
||||
if (!created.task_id) {
|
||||
throw new Error('MiniMax response did not include a task id');
|
||||
}
|
||||
const task = await pollUntilSettled<HailuoTask>({
|
||||
provider: PROVIDER,
|
||||
providerLabel: 'MiniMax',
|
||||
intervalMs: this.#pollIntervalMs,
|
||||
fetch: () =>
|
||||
this.#request<HailuoTask>(
|
||||
'GET',
|
||||
`/v1/query/video_generation?task_id=${encodeURIComponent(created.task_id!)}`,
|
||||
),
|
||||
isPending: (task) =>
|
||||
task.status === 'Preparing' ||
|
||||
task.status === 'Queueing' ||
|
||||
task.status === 'Processing',
|
||||
});
|
||||
if (task.status !== 'Success' || !task.file_id) {
|
||||
throw videoJobFailure(
|
||||
PROVIDER,
|
||||
'MiniMax video generation failed',
|
||||
);
|
||||
}
|
||||
// The clip is a file in MiniMax's store; its download link is
|
||||
// minted on request and works for an hour.
|
||||
const { file } = await this.#request<{
|
||||
file?: { download_url?: string };
|
||||
}>(
|
||||
'GET',
|
||||
`/v1/files/retrieve?file_id=${encodeURIComponent(task.file_id)}`,
|
||||
);
|
||||
const url = file?.download_url;
|
||||
if (typeof url !== 'string' || !url.trim()) {
|
||||
throw new Error('MiniMax response did not include a video URL');
|
||||
}
|
||||
|
||||
await this.#meteringService.incrementUsage(
|
||||
actor,
|
||||
usageType,
|
||||
1,
|
||||
microCents,
|
||||
);
|
||||
return url;
|
||||
} finally {
|
||||
await hold.release();
|
||||
}
|
||||
}
|
||||
|
||||
// -- Validation ---------------------------------------------------------
|
||||
|
||||
#checkInputs(
|
||||
spec: MiniMaxVideoSpec,
|
||||
model: IVideoModel,
|
||||
resolutionKey: string,
|
||||
inputs: {
|
||||
firstFrame?: string;
|
||||
endFrame?: string;
|
||||
references: string[];
|
||||
},
|
||||
) {
|
||||
const { firstFrame, endFrame, references } = inputs;
|
||||
if (spec.firstFrameRequired && !firstFrame) {
|
||||
throw badRequest(`${model.id} requires an input_reference image`);
|
||||
}
|
||||
if (endFrame && !spec.lastFrame) {
|
||||
throw badRequest(`${model.id} does not support last_frame`);
|
||||
}
|
||||
if (endFrame && !firstFrame && spec.api === 'v1') {
|
||||
throw badRequest(
|
||||
'last_frame requires a first-frame image via input_reference',
|
||||
);
|
||||
}
|
||||
if (references.length > 0) {
|
||||
if (!spec.references) {
|
||||
throw badRequest(
|
||||
`${model.id} does not support reference_images`,
|
||||
);
|
||||
}
|
||||
if (firstFrame || endFrame) {
|
||||
throw badRequest(
|
||||
'reference_images cannot be combined with input_reference or last_frame',
|
||||
);
|
||||
}
|
||||
}
|
||||
const imageCount =
|
||||
(firstFrame ? 1 : 0) + (endFrame ? 1 : 0) + references.length;
|
||||
if (imageCount > spec.maxImages) {
|
||||
throw badRequest(
|
||||
`${model.id} takes at most ${spec.maxImages} image(s), frames and references together`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
spec.imageOnlyResolutions?.includes(resolutionKey) &&
|
||||
(!firstFrame || endFrame)
|
||||
) {
|
||||
throw badRequest(
|
||||
`${model.id} renders ${resolutionKey} only from an input_reference image without last_frame`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async #request<T>(
|
||||
method: 'GET' | 'POST',
|
||||
path: string,
|
||||
body?: Record<string, unknown>,
|
||||
): Promise<T> {
|
||||
const response = await fetch(`${this.#baseUrl}${path}`, {
|
||||
method,
|
||||
headers: {
|
||||
Authorization: `Bearer ${this.#apiKey}`,
|
||||
...(body ? { 'Content-Type': 'application/json' } : {}),
|
||||
},
|
||||
...(body ? { body: JSON.stringify(body) } : {}),
|
||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
const raw = await readBoundedBody(response, MAX_RESPONSE_BYTES, () =>
|
||||
minimaxError(undefined, 'MiniMax response too large', 502),
|
||||
);
|
||||
let payload: MiniMaxEnvelope = {};
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(raw.toString('utf8'));
|
||||
if (parsed && typeof parsed === 'object') {
|
||||
payload = parsed as MiniMaxEnvelope;
|
||||
}
|
||||
} catch {
|
||||
// Not JSON; the status alone decides.
|
||||
}
|
||||
// A v1 reply carries its verdict in `base_resp` under HTTP 200; a v2
|
||||
// reply uses the HTTP status and an OpenAI-style `error`.
|
||||
const v1Code = payload.base_resp?.status_code;
|
||||
if (typeof v1Code === 'number' && v1Code !== 0) {
|
||||
throw minimaxError(
|
||||
v1Code,
|
||||
payload.base_resp?.status_msg ||
|
||||
`MiniMax request failed (code ${v1Code})`,
|
||||
response.status,
|
||||
);
|
||||
}
|
||||
if (!response.ok) {
|
||||
const message =
|
||||
payload.error?.message ||
|
||||
`MiniMax request failed (status ${response.status})`;
|
||||
throw minimaxError(parseCode(message), message, response.status);
|
||||
}
|
||||
return payload as unknown as T;
|
||||
}
|
||||
|
||||
#getModel(requestedModel?: string): IVideoModel {
|
||||
const wanted = (requestedModel ?? '').trim().toLowerCase();
|
||||
return (
|
||||
MINIMAX_VIDEO_GENERATION_MODELS.find(
|
||||
(m) =>
|
||||
m.id === wanted ||
|
||||
m.puterId === wanted ||
|
||||
m.aliases?.includes(wanted),
|
||||
) ??
|
||||
MINIMAX_VIDEO_GENERATION_MODELS.find((m) => m.id === DEFAULT_MODEL)!
|
||||
);
|
||||
}
|
||||
|
||||
#resolution(candidate: unknown, model: IVideoModel): string {
|
||||
const tiers = model.dimensions!;
|
||||
if (typeof candidate === 'string') {
|
||||
const wanted = candidate.trim().toLowerCase();
|
||||
if (tiers.includes(wanted)) return wanted;
|
||||
}
|
||||
return tiers[0];
|
||||
}
|
||||
|
||||
/** An image as MiniMax takes it: a URL or a data URI. */
|
||||
#image(value: unknown, label: string): string | undefined {
|
||||
if (!present(value)) return undefined;
|
||||
if (typeof value !== 'string' || !value.trim()) {
|
||||
throw badRequest(
|
||||
`${label} must be an image URL, data URI or base64`,
|
||||
);
|
||||
}
|
||||
return toUrlOrDataUri(value);
|
||||
}
|
||||
|
||||
#references(value: unknown): string[] {
|
||||
if (!present(value)) return [];
|
||||
if (!Array.isArray(value)) {
|
||||
throw badRequest('reference_images must be an array');
|
||||
}
|
||||
return value.map((image) => this.#image(image, 'reference_images')!);
|
||||
}
|
||||
|
||||
#coerceSeconds(value: unknown): number | undefined {
|
||||
const numeric =
|
||||
typeof value === 'string' ? Number.parseInt(value, 10) : value;
|
||||
return typeof numeric === 'number' &&
|
||||
Number.isFinite(numeric) &&
|
||||
numeric > 0
|
||||
? Math.round(numeric)
|
||||
: undefined;
|
||||
}
|
||||
|
||||
/** Snap width/height to one of MiniMax's aspect ratios. */
|
||||
#aspectRatio(width?: number, height?: number): string | undefined {
|
||||
if (
|
||||
typeof width !== 'number' ||
|
||||
typeof height !== 'number' ||
|
||||
!Number.isFinite(width) ||
|
||||
!Number.isFinite(height) ||
|
||||
width <= 0 ||
|
||||
height <= 0
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
const gcd = (a: number, b: number): number =>
|
||||
b === 0 ? a : gcd(b, a % b);
|
||||
const w = Math.round(width);
|
||||
const h = Math.round(height);
|
||||
const d = gcd(w, h) || 1;
|
||||
const candidate = `${w / d}:${h / d}`;
|
||||
// Other ratios are left to the 16:9 default.
|
||||
return ASPECT_RATIOS.includes(candidate) ? candidate : undefined;
|
||||
}
|
||||
}
|
||||
@@ -1,234 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import type { IVideoModel } from '../../types.js';
|
||||
|
||||
/** MiniMax-specific model behavior the shared IVideoModel shape can't carry. */
|
||||
export interface MiniMaxVideoSpec {
|
||||
/** The model name as MiniMax spells it on the wire. */
|
||||
apiModel: string;
|
||||
/**
|
||||
* `v2`: the H3 task API, billed per second of output. `v1`: the Hailuo API,
|
||||
* billed per clip, whose result is fetched through a file id.
|
||||
*/
|
||||
api: 'v1' | 'v2';
|
||||
/**
|
||||
* Frames and reference images together. MiniMax charges per image above
|
||||
* this quota, so more are refused rather than billed.
|
||||
*/
|
||||
maxImages: number;
|
||||
/** Whether `reference_images` are taken (H3 reference-to-video). */
|
||||
references: boolean;
|
||||
firstFrameRequired?: boolean;
|
||||
lastFrame: boolean;
|
||||
/** Resolutions MiniMax renders only from a first frame. */
|
||||
imageOnlyResolutions?: string[];
|
||||
maxPromptChars: number;
|
||||
}
|
||||
|
||||
// Duration ladder for the driver's normalization: element 0 is the fallback
|
||||
// default, the rest enumerate the model's contiguous valid range.
|
||||
const seconds = (def: number, min: number, max: number): number[] => [
|
||||
def,
|
||||
...Array.from({ length: max - min + 1 }, (_, i) => min + i).filter(
|
||||
(s) => s !== def,
|
||||
),
|
||||
];
|
||||
|
||||
// Prices from https://platform.minimax.io/docs/guides/pricing-video. H3 models
|
||||
// are per second of output, keyed `per-second:<resolution>`; Hailuo models are
|
||||
// per clip, keyed `per-video:<resolution>:<seconds>`, and a key's absence means
|
||||
// the model doesn't offer that combination. `default-duration-per-video` is
|
||||
// the default clip at the default resolution, for cross-provider sorting and
|
||||
// display only.
|
||||
export const MINIMAX_VIDEO_GENERATION_MODELS: IVideoModel[] = [
|
||||
{
|
||||
id: 'minimax-h3',
|
||||
puterId: 'minimax:minimax/minimax-h3',
|
||||
name: 'MiniMax H3',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-second:768p': 8,
|
||||
'per-second:2k': 13,
|
||||
'default-duration-per-video': 40,
|
||||
},
|
||||
durationSeconds: seconds(5, 4, 15),
|
||||
dimensions: ['768p', '2k'],
|
||||
fps: [24],
|
||||
},
|
||||
{
|
||||
id: 'minimax-h3-highspeed',
|
||||
puterId: 'minimax:minimax/minimax-h3-highspeed',
|
||||
name: 'MiniMax H3 Highspeed',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-second:768p': 6,
|
||||
'default-duration-per-video': 30,
|
||||
},
|
||||
durationSeconds: seconds(5, 4, 15),
|
||||
dimensions: ['768p'],
|
||||
fps: [24],
|
||||
},
|
||||
{
|
||||
id: 'minimax-h3-lite',
|
||||
puterId: 'minimax:minimax/minimax-h3-lite',
|
||||
name: 'MiniMax H3 Lite',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-second:768p': 4,
|
||||
'per-second:480p': 2.5,
|
||||
'default-duration-per-video': 20,
|
||||
},
|
||||
durationSeconds: seconds(5, 4, 15),
|
||||
dimensions: ['768p', '480p'],
|
||||
fps: [24],
|
||||
},
|
||||
{
|
||||
id: 'minimax-h3-max',
|
||||
puterId: 'minimax:minimax/minimax-h3-max',
|
||||
name: 'MiniMax H3 Max',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-second:768p': 8,
|
||||
'per-second:480p': 5,
|
||||
'default-duration-per-video': 40,
|
||||
},
|
||||
durationSeconds: seconds(5, 5, 15),
|
||||
dimensions: ['768p', '480p'],
|
||||
fps: [24],
|
||||
},
|
||||
{
|
||||
id: 'minimax-hailuo-2.3',
|
||||
puterId: 'minimax:minimax/minimax-hailuo-2.3',
|
||||
name: 'MiniMax Hailuo 2.3',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-video:768p:6': 28,
|
||||
'per-video:768p:10': 56,
|
||||
'per-video:1080p:6': 49,
|
||||
'default-duration-per-video': 28,
|
||||
},
|
||||
durationSeconds: [6, 10],
|
||||
dimensions: ['768p', '1080p'],
|
||||
fps: [24],
|
||||
},
|
||||
{
|
||||
id: 'minimax-hailuo-2.3-fast',
|
||||
puterId: 'minimax:minimax/minimax-hailuo-2.3-fast',
|
||||
name: 'MiniMax Hailuo 2.3 Fast',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-video:768p:6': 19,
|
||||
'per-video:768p:10': 32,
|
||||
'per-video:1080p:6': 33,
|
||||
'default-duration-per-video': 19,
|
||||
},
|
||||
durationSeconds: [6, 10],
|
||||
dimensions: ['768p', '1080p'],
|
||||
fps: [24],
|
||||
},
|
||||
{
|
||||
id: 'minimax-hailuo-02',
|
||||
puterId: 'minimax:minimax/minimax-hailuo-02',
|
||||
name: 'MiniMax Hailuo 02',
|
||||
costs_currency: 'usd-cents',
|
||||
output_cost_key: 'default-duration-per-video',
|
||||
costs: {
|
||||
'per-video:768p:6': 28,
|
||||
'per-video:768p:10': 56,
|
||||
'per-video:1080p:6': 49,
|
||||
'per-video:512p:6': 10,
|
||||
'per-video:512p:10': 15,
|
||||
'default-duration-per-video': 28,
|
||||
},
|
||||
durationSeconds: [6, 10],
|
||||
dimensions: ['768p', '1080p', '512p'],
|
||||
fps: [24],
|
||||
},
|
||||
];
|
||||
|
||||
// Capabilities per https://platform.minimax.io/docs/api-reference/video-generation-v2-create
|
||||
// (H3) and the text-, image- and frames-to-video pages under
|
||||
// https://platform.minimax.io/docs/api-reference/video-generation-t2v (Hailuo).
|
||||
export const MINIMAX_VIDEO_SPECS: Record<string, MiniMaxVideoSpec> = {
|
||||
'minimax-h3': {
|
||||
apiModel: 'MiniMax-H3',
|
||||
api: 'v2',
|
||||
maxImages: 5,
|
||||
references: true,
|
||||
lastFrame: true,
|
||||
maxPromptChars: 7000,
|
||||
},
|
||||
'minimax-h3-highspeed': {
|
||||
apiModel: 'MiniMax-H3-Highspeed',
|
||||
api: 'v2',
|
||||
maxImages: 5,
|
||||
references: true,
|
||||
lastFrame: true,
|
||||
maxPromptChars: 7000,
|
||||
},
|
||||
'minimax-h3-lite': {
|
||||
apiModel: 'MiniMax-H3-Lite',
|
||||
api: 'v2',
|
||||
maxImages: 5,
|
||||
references: true,
|
||||
lastFrame: true,
|
||||
maxPromptChars: 7000,
|
||||
},
|
||||
'minimax-h3-max': {
|
||||
apiModel: 'MiniMax-H3-Max',
|
||||
api: 'v2',
|
||||
maxImages: 2,
|
||||
references: true,
|
||||
lastFrame: true,
|
||||
maxPromptChars: 7000,
|
||||
},
|
||||
'minimax-hailuo-2.3': {
|
||||
apiModel: 'MiniMax-Hailuo-2.3',
|
||||
api: 'v1',
|
||||
maxImages: 1,
|
||||
references: false,
|
||||
lastFrame: false,
|
||||
maxPromptChars: 2000,
|
||||
},
|
||||
'minimax-hailuo-2.3-fast': {
|
||||
apiModel: 'MiniMax-Hailuo-2.3-Fast',
|
||||
api: 'v1',
|
||||
maxImages: 1,
|
||||
references: false,
|
||||
firstFrameRequired: true,
|
||||
lastFrame: false,
|
||||
maxPromptChars: 2000,
|
||||
},
|
||||
'minimax-hailuo-02': {
|
||||
apiModel: 'MiniMax-Hailuo-02',
|
||||
api: 'v1',
|
||||
maxImages: 2,
|
||||
references: false,
|
||||
lastFrame: true,
|
||||
imageOnlyResolutions: ['512p'],
|
||||
maxPromptChars: 2000,
|
||||
},
|
||||
};
|
||||
@@ -30,7 +30,6 @@
|
||||
* a test file.
|
||||
*/
|
||||
|
||||
import type { EventClient } from '../clients/event/EventClient.js';
|
||||
import { translateProviderError } from '../controllers/drivers/DriverController.js';
|
||||
import type { Actor } from '../core/actor.js';
|
||||
import { SYSTEM_ACTOR, makeActor } from '../core/actor.js';
|
||||
@@ -61,23 +60,6 @@ export const skipUnlessEnv = (name: string): boolean => !optionalEnv(name);
|
||||
*/
|
||||
export const INTEGRATION_TEST_TIMEOUT_MS = 90_000;
|
||||
|
||||
/** Runs `fn` with every AI driver's cost factor hooked to `factor`. */
|
||||
export const withAiCostFactorHook = async <T>(
|
||||
events: EventClient,
|
||||
factor: number,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<T> => {
|
||||
const hook = (_key: string, event: { factor: number }) => {
|
||||
event.factor = factor;
|
||||
};
|
||||
events.on('ai.cost.factor.*', hook);
|
||||
try {
|
||||
return await fn();
|
||||
} finally {
|
||||
events.off('ai.cost.factor.*', hook);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Returns a no-op MeteringService stub. Real metering would write to DynamoDB /
|
||||
* Redis, which integration tests for AI providers don't care about — we just
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -14,10 +14,7 @@ import {
|
||||
APP_DATA_KV_METHOD_OPS,
|
||||
appDataPermission,
|
||||
} from '../../services/permission/appDataScopes.ts';
|
||||
import { kvNamespace } from '../../stores/systemKv/SystemKVStore.ts';
|
||||
import { createTestUser, setupTestServer } from '../../testUtil.ts';
|
||||
import type { IConfig } from '../../types.ts';
|
||||
import { sealCursor, unsealCursor } from '../../util/pagination.ts';
|
||||
import { KV_COSTS } from './costs.ts';
|
||||
import type { KVStoreDriver } from './KVStoreDriver.ts';
|
||||
import {
|
||||
@@ -1555,301 +1552,6 @@ describe('KVStoreDriver', () => {
|
||||
expect(own).toContain('token');
|
||||
});
|
||||
|
||||
// -- Cross-app cursors ----------------------------------------------
|
||||
|
||||
const secret = () =>
|
||||
(target as unknown as { config: IConfig }).config.jwt_secret_v2;
|
||||
|
||||
/** A cursor as the server reads it, its key decoded, or null. */
|
||||
const opened = (cursor: string) => {
|
||||
const payload = unsealCursor(cursor, secret());
|
||||
return payload
|
||||
? {
|
||||
...payload,
|
||||
k: Buffer.from(String(payload.k), 'base64url').toString(
|
||||
'utf8',
|
||||
),
|
||||
}
|
||||
: null;
|
||||
};
|
||||
|
||||
/** Everything a cursor shows to the server and to a base64 decoder. */
|
||||
const revealed = (cursor: string) =>
|
||||
[
|
||||
JSON.stringify(opened(cursor)),
|
||||
Buffer.from(cursor, 'base64').toString('utf8'),
|
||||
Buffer.from(cursor, 'base64url').toString('utf8'),
|
||||
].join('\n');
|
||||
|
||||
const encode = (payload: unknown) =>
|
||||
Buffer.from(JSON.stringify(payload)).toString('base64');
|
||||
|
||||
type Page = { items: string[]; cursor?: string; total?: number };
|
||||
|
||||
const writeAs = (
|
||||
actorForCall: Actor,
|
||||
key: string,
|
||||
disableSharing = false,
|
||||
) =>
|
||||
inCtx(
|
||||
() =>
|
||||
target.set({
|
||||
key,
|
||||
value: key,
|
||||
...(disableSharing
|
||||
? { optConfig: { disableSharing: true } }
|
||||
: {}),
|
||||
}),
|
||||
actorForCall,
|
||||
);
|
||||
|
||||
it('resumes a cross-app page after a private entry it skipped, padding every cursor to one length', async () => {
|
||||
const { owner, calendar, contacts, calendarActor, contactsActor } =
|
||||
await setup();
|
||||
await writeAs(contactsActor, 'a-hidden-oauth-token-name', true);
|
||||
await writeAs(contactsActor, 'b');
|
||||
await writeAs(contactsActor, 'c-a-much-longer-public-key-name');
|
||||
await grant(owner, calendar.uid, appDataPermission(contacts.uid));
|
||||
const page = (cursor?: string) =>
|
||||
crossApp(calendarActor, contacts.uid, (optConfig) =>
|
||||
target.list({
|
||||
as: 'keys',
|
||||
limit: 1,
|
||||
...(cursor ? { cursor } : {}),
|
||||
optConfig,
|
||||
}),
|
||||
) as Promise<Page>;
|
||||
|
||||
// The scan's first row is the private one.
|
||||
const first = await page();
|
||||
expect(first.items).toEqual(['b']);
|
||||
expect(revealed(first.cursor!)).not.toContain('hidden-oauth');
|
||||
expect(opened(first.cursor!)?.k).toBe('b');
|
||||
|
||||
const second = await page(first.cursor);
|
||||
expect(second.items).toEqual(['c-a-much-longer-public-key-name']);
|
||||
expect(second.cursor!.length).toBe(first.cursor!.length);
|
||||
});
|
||||
|
||||
// Over a thousand writes and reads: slow when the whole suite runs.
|
||||
it('pads a cursor left on a long run of private entries to the same length', async () => {
|
||||
const { owner, calendar, contacts, calendarActor, contactsActor } =
|
||||
await setup();
|
||||
// More than one call reads past, so the page ends on a hidden row.
|
||||
const hidden = Array.from(
|
||||
{ length: 1100 },
|
||||
(_, i) => `h-hidden-${String(i).padStart(4, '0')}`,
|
||||
);
|
||||
for (let at = 0; at < hidden.length; at += 1000)
|
||||
await inCtx(
|
||||
() =>
|
||||
target.batchPut({
|
||||
items: hidden
|
||||
.slice(at, at + 1000)
|
||||
.map((key) => ({ key, value: 1 })),
|
||||
optConfig: { disableSharing: true },
|
||||
}),
|
||||
contactsActor,
|
||||
);
|
||||
await writeAs(contactsActor, 'z');
|
||||
await grant(owner, calendar.uid, appDataPermission(contacts.uid));
|
||||
|
||||
const keys: string[] = [];
|
||||
const cursors: string[] = [];
|
||||
let emptyPages = 0;
|
||||
let cursor: string | undefined;
|
||||
for (let pages = 0; pages < 10; pages++) {
|
||||
const page = (await crossApp(
|
||||
calendarActor,
|
||||
contacts.uid,
|
||||
(optConfig) =>
|
||||
target.list({
|
||||
as: 'keys',
|
||||
limit: 1,
|
||||
...(cursor ? { cursor } : {}),
|
||||
optConfig,
|
||||
}),
|
||||
)) as Page;
|
||||
keys.push(...page.items);
|
||||
if (page.items.length === 0) emptyPages++;
|
||||
cursor = page.cursor;
|
||||
if (!cursor) break;
|
||||
cursors.push(cursor);
|
||||
expect(
|
||||
Buffer.from(cursor, 'base64').toString('utf8'),
|
||||
).not.toContain('h-hidden');
|
||||
}
|
||||
expect(keys).toEqual(['entry', 'z']);
|
||||
expect(emptyPages).toBeGreaterThan(0);
|
||||
expect(new Set(cursors.map((c) => c.length)).size).toBe(1);
|
||||
}, 30_000);
|
||||
|
||||
describe('paging a namespace with private entries in it', () => {
|
||||
// Lengths differ, so an unpadded cursor would show which it is.
|
||||
const PUBLIC = [
|
||||
'entry',
|
||||
'k1',
|
||||
'k3-longer',
|
||||
'k4',
|
||||
'k6-longest-of-all',
|
||||
];
|
||||
const PRIVATE = ['k2', 'k5', 'k7', 'k8'];
|
||||
|
||||
const seeded = async () => {
|
||||
const fixture = await setup();
|
||||
for (const key of PUBLIC.slice(1))
|
||||
await writeAs(fixture.contactsActor, key);
|
||||
for (const key of PRIVATE)
|
||||
await writeAs(fixture.contactsActor, key, true);
|
||||
await grant(
|
||||
fixture.owner,
|
||||
fixture.calendar.uid,
|
||||
appDataPermission(fixture.contacts.uid),
|
||||
);
|
||||
return fixture;
|
||||
};
|
||||
|
||||
it.each(
|
||||
[false, true].flatMap((reverse) =>
|
||||
[false, true].map((includeTotal) => ({
|
||||
reverse,
|
||||
includeTotal,
|
||||
})),
|
||||
),
|
||||
)(
|
||||
'returns every public entry once, reverse $reverse, includeTotal $includeTotal',
|
||||
async ({ reverse, includeTotal }) => {
|
||||
const { contacts, calendarActor } = await seeded();
|
||||
const keys: string[] = [];
|
||||
const lengths = new Set<number>();
|
||||
let cursor: string | undefined;
|
||||
let pages = 0;
|
||||
do {
|
||||
const page = (await crossApp(
|
||||
calendarActor,
|
||||
contacts.uid,
|
||||
(optConfig) =>
|
||||
target.list({
|
||||
as: 'keys',
|
||||
limit: 2,
|
||||
...(reverse ? { reverse } : {}),
|
||||
...(includeTotal ? { includeTotal } : {}),
|
||||
...(cursor ? { cursor } : {}),
|
||||
optConfig,
|
||||
}),
|
||||
)) as Page;
|
||||
keys.push(...page.items);
|
||||
if (includeTotal)
|
||||
expect(page.total).toBe(PUBLIC.length);
|
||||
cursor = page.cursor;
|
||||
if (cursor) {
|
||||
lengths.add(cursor.length);
|
||||
const position = opened(cursor);
|
||||
expect(position).not.toBeNull();
|
||||
expect(PRIVATE).not.toContain(position!.k);
|
||||
}
|
||||
pages++;
|
||||
} while (cursor && pages < 10);
|
||||
|
||||
expect(keys).toEqual(
|
||||
reverse ? [...PUBLIC].reverse() : PUBLIC,
|
||||
);
|
||||
expect(lengths.size).toBe(1);
|
||||
},
|
||||
);
|
||||
|
||||
it('still reads a plain cursor from before they were sealed, and seals the next one', async () => {
|
||||
const { owner, contacts, calendarActor } = await seeded();
|
||||
const namespace = kvNamespace(owner.user.uuid!, contacts.uid);
|
||||
const list = (cursor: unknown, limit = 10) =>
|
||||
crossApp(calendarActor, contacts.uid, (optConfig) =>
|
||||
target.list({ as: 'keys', limit, cursor, optConfig }),
|
||||
) as Promise<Page>;
|
||||
|
||||
expect(
|
||||
(await list(encode({ namespace, key: 'k1' }))).items,
|
||||
).toEqual(PUBLIC.slice(2));
|
||||
expect(
|
||||
(
|
||||
await list(
|
||||
encode({
|
||||
reverse: true,
|
||||
key: { namespace, key: 'k4' },
|
||||
}),
|
||||
)
|
||||
).items,
|
||||
).toEqual(PUBLIC.slice(0, 3).reverse());
|
||||
|
||||
const next = await list(encode({ namespace, key: 'k1' }), 1);
|
||||
expect(next.items).toEqual(['k3-longer']);
|
||||
expect(opened(next.cursor!)).toMatchObject({ k: 'k3-longer' });
|
||||
});
|
||||
|
||||
it('refuses a plain or sealed cursor for another namespace', async () => {
|
||||
const { owner, calendar, contacts, calendarActor } =
|
||||
await seeded();
|
||||
const other = await makeRealApp(owner.user.id!);
|
||||
await writeAs(asApp(owner, other), 'k1');
|
||||
await writeAs(asApp(owner, other), 'k2');
|
||||
await grant(owner, calendar.uid, appDataPermission(other.uid));
|
||||
const fromOther = (
|
||||
(await crossApp(calendarActor, other.uid, (optConfig) =>
|
||||
target.list({ as: 'keys', limit: 1, optConfig }),
|
||||
)) as Page
|
||||
).cursor!;
|
||||
await writeAs(calendarActor, 'own-1');
|
||||
await writeAs(calendarActor, 'own-2');
|
||||
const ownCursor = (
|
||||
(await inCtx(
|
||||
() => target.list({ as: 'keys', limit: 1 }),
|
||||
calendarActor,
|
||||
)) as Page
|
||||
).cursor!;
|
||||
const elsewhere = kvNamespace(owner.user.uuid!, other.uid);
|
||||
const stranger = kvNamespace('someone-else', contacts.uid);
|
||||
|
||||
const forged: unknown[] = [
|
||||
fromOther,
|
||||
encode({ namespace: elsewhere, key: 'k1' }),
|
||||
encode({ namespace: stranger, key: 'k1' }),
|
||||
{ namespace: elsewhere, key: 'k1' },
|
||||
encode({
|
||||
reverse: true,
|
||||
key: { namespace: elsewhere, key: 'k1' },
|
||||
}),
|
||||
encode({
|
||||
namespace: kvNamespace(owner.user.uuid!, contacts.uid),
|
||||
key: 'k1',
|
||||
extra: true,
|
||||
}),
|
||||
sealCursor({ id: 1 }, secret()),
|
||||
sealCursor(
|
||||
{
|
||||
s: elsewhere,
|
||||
k: Buffer.from('k1').toString('base64url'),
|
||||
},
|
||||
secret(),
|
||||
),
|
||||
ownCursor,
|
||||
];
|
||||
for (const cursor of forged)
|
||||
await expect(
|
||||
crossApp(calendarActor, contacts.uid, (optConfig) =>
|
||||
target.list({
|
||||
as: 'keys',
|
||||
limit: 1,
|
||||
cursor,
|
||||
optConfig,
|
||||
}),
|
||||
),
|
||||
JSON.stringify(cursor),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
message: 'invalid kv: cursor',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses cross-app writes and deletes against a private entry', async () => {
|
||||
const { owner, calendar, contacts, calendarActor, contactsActor } =
|
||||
await setup();
|
||||
|
||||
@@ -32,29 +32,9 @@ import {
|
||||
appDataPermission,
|
||||
appDataSharingAllowed,
|
||||
} from '../../services/permission/appDataScopes.js';
|
||||
import {
|
||||
KV_GLOBAL_APP_KEY,
|
||||
KV_MAX_KEY_BYTES,
|
||||
kvNamespace,
|
||||
type KVOpts,
|
||||
type KVUsage,
|
||||
} from '../../stores/systemKv/SystemKVStore.js';
|
||||
import { KV_SHARE_HANDLE_WIDTHS } from '../../stores/events/columnWidths.js';
|
||||
import type { KVOpts, KVUsage } from '../../stores/systemKv/SystemKVStore.js';
|
||||
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
|
||||
import { isKvHandleId } from '../../services/events/subjects.js';
|
||||
import {
|
||||
KV_HANDLE_KEY_MAX_BYTES,
|
||||
KV_SHARE_PREFIX_MAX_BYTES,
|
||||
kvHandlesDisabled,
|
||||
kvShareLevelForMethod,
|
||||
kvSharePermission,
|
||||
} from '../../services/kvShare/kvShares.js';
|
||||
import {
|
||||
decodeCursor,
|
||||
normalizeLimit,
|
||||
sealCursor,
|
||||
unsealCursor,
|
||||
} from '../../util/pagination.js';
|
||||
import { normalizeLimit } from '../../util/pagination.js';
|
||||
import { KV_COSTS } from './costs.js';
|
||||
import {
|
||||
KV_MAX_BATCH_PUT_ITEMS,
|
||||
@@ -62,143 +42,16 @@ import {
|
||||
KV_MAX_LIST_LIMIT,
|
||||
} from './limits.js';
|
||||
|
||||
/** `optConfig` as every KV method takes it. */
|
||||
interface KvOptConfig {
|
||||
/** Another app's namespace, which the user let this app reach. */
|
||||
appUuid?: string;
|
||||
/** Writes the entry private to the app that wrote it. */
|
||||
disableSharing?: boolean;
|
||||
/**
|
||||
* A share handle: the call reaches that region of its owner's data, with
|
||||
* keys relative to it.
|
||||
*/
|
||||
handle?: string;
|
||||
/** `set` only: write only when no live entry holds the key. */
|
||||
onlyIfAbsent?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every KV method's argument object, as far as option resolution cares: the
|
||||
* namespace override, plus the expiry parameters that can delete an entry.
|
||||
*/
|
||||
type KvCallArgs = {
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string; disableSharing?: boolean };
|
||||
expireAt?: unknown;
|
||||
ttl?: unknown;
|
||||
};
|
||||
|
||||
/**
|
||||
* Store options plus, for a call through a share handle, the handle and its
|
||||
* region's prefix: keys go in under it and come back out relative to it.
|
||||
*/
|
||||
type DriverKvOpts = KVOpts & { handle?: string; handlePrefix?: string };
|
||||
|
||||
type KvListResult =
|
||||
| unknown[]
|
||||
| { items: unknown[]; cursor?: string; total?: number };
|
||||
|
||||
/**
|
||||
* A listing of another namespace, cross-app or through a handle: what its
|
||||
* cursors are sealed to, the namespace a position rebuilds in, and the prefix
|
||||
* its keys are relative to.
|
||||
*/
|
||||
interface ForeignListing {
|
||||
bind: string;
|
||||
namespace: string;
|
||||
prefix: string;
|
||||
/** Cross-app cursors were plain before they were sealed. */
|
||||
acceptsPlain: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Plaintext bytes a foreign listing's cursor is padded to: a key at the store's
|
||||
* limit in base64url, plus room for the binding and the envelope.
|
||||
*/
|
||||
const FOREIGN_CURSOR_BYTES = Math.ceil((KV_MAX_KEY_BYTES * 4) / 3) + 256;
|
||||
|
||||
/** Whether `value` is a store position, `{ namespace, key }`, in `namespace`. */
|
||||
const isPositionIn = (value: unknown, namespace: string): boolean => {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value))
|
||||
return false;
|
||||
const { namespace: named, key, ...rest } = value as Record<string, unknown>;
|
||||
return (
|
||||
named === namespace &&
|
||||
typeof key === 'string' &&
|
||||
Object.keys(rest).length === 0
|
||||
);
|
||||
};
|
||||
|
||||
// -- Share-handle errors ----------------------------------------------
|
||||
|
||||
/**
|
||||
* Unknown, retired, not the caller's, owner gone or suspended, or namespace app
|
||||
* deleted: one body for all of them, so no call can tell them apart.
|
||||
*/
|
||||
const handleNotFound = (): HttpError =>
|
||||
new HttpError(404, 'kv: no such share handle', {
|
||||
legacyCode: 'kv_handle_not_found',
|
||||
});
|
||||
|
||||
/** Only ever told to the handle's holder. */
|
||||
const handleAccessDenied = (): HttpError =>
|
||||
new HttpError(403, 'kv: this share handle does not allow this call', {
|
||||
legacyCode: 'kv_handle_access_denied',
|
||||
});
|
||||
|
||||
const handleOptionInvalid = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'kv_handle_option_invalid' });
|
||||
|
||||
const handleKeyInvalid = (): HttpError =>
|
||||
new HttpError(
|
||||
400,
|
||||
`kv: a key used with a share handle must be a non-empty string of at most ${KV_HANDLE_KEY_MAX_BYTES} bytes`,
|
||||
{ legacyCode: 'kv_handle_key_invalid' },
|
||||
);
|
||||
|
||||
/** The key limit, so no pattern length can tell how long the prefix is. */
|
||||
const handlePatternInvalid = (): HttpError =>
|
||||
new HttpError(
|
||||
400,
|
||||
`kv: a pattern used with a share handle may be at most ${KV_HANDLE_KEY_MAX_BYTES} bytes`,
|
||||
{ legacyCode: 'kv_handle_key_invalid' },
|
||||
);
|
||||
|
||||
const invalidCursor = (): HttpError =>
|
||||
new HttpError(400, 'invalid kv: cursor', { legacyCode: 'bad_request' });
|
||||
|
||||
/** A stored key outside the handle's region, which a scan can't return. */
|
||||
const outsideRegion = (): HttpError =>
|
||||
new HttpError(500, 'kv: internal error', { legacyCode: 'internal_error' });
|
||||
|
||||
const viaHandle = (args?: KvCallArgs): boolean =>
|
||||
args?.optConfig?.handle !== undefined && args.optConfig.handle !== null;
|
||||
|
||||
/** `onlyIfAbsent` as a boolean, refusing anything else a raw body sends. */
|
||||
const parseOnlyIfAbsent = (value: unknown): boolean => {
|
||||
if (value === undefined || value === null || value === false) return false;
|
||||
if (value === true) return true;
|
||||
throw new HttpError(400, 'kv: onlyIfAbsent must be a boolean', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* A store error with every stored key it names put back the way the caller
|
||||
* named it, so the handle's prefix never leaves in a message.
|
||||
*/
|
||||
const relativizeError = (
|
||||
err: unknown,
|
||||
prefix: string,
|
||||
keys: readonly string[],
|
||||
): unknown => {
|
||||
if (!(err instanceof Error) || typeof err.message !== 'string') return err;
|
||||
let message = err.message;
|
||||
for (const key of keys)
|
||||
message = message.split(`${prefix}${key}`).join(key);
|
||||
if (message !== err.message) err.message = message;
|
||||
return err;
|
||||
};
|
||||
|
||||
/**
|
||||
* Methods that stay available to an account with nothing left of its budget.
|
||||
* Each one only ever reduces what the account is storing, and turning those
|
||||
@@ -288,25 +141,13 @@ export class KVStoreDriver extends PuterDriver {
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* A key as the caller sent it. Through a handle it is relative to the
|
||||
* region, and held to the handle's own fixed limit.
|
||||
*/
|
||||
#coerceKey(key: unknown, args?: KvCallArgs): string {
|
||||
#coerceKey(key: unknown): string {
|
||||
if (key === null || key === undefined) {
|
||||
throw new HttpError(400, 'Missing `key`', {
|
||||
legacyCode: 'bad_request',
|
||||
}); // legacyCode for backward compatibility with old error handling in controllers
|
||||
}
|
||||
const str = typeof key === 'string' ? key : String(key);
|
||||
if (viaHandle(args)) {
|
||||
if (
|
||||
str === '' ||
|
||||
Buffer.byteLength(str, 'utf8') > KV_HANDLE_KEY_MAX_BYTES
|
||||
)
|
||||
throw handleKeyInvalid();
|
||||
return str;
|
||||
}
|
||||
if (str === '')
|
||||
throw new HttpError(400, 'Missing `key`', {
|
||||
legacyCode: 'bad_request',
|
||||
@@ -314,38 +155,7 @@ export class KVStoreDriver extends PuterDriver {
|
||||
return str;
|
||||
}
|
||||
|
||||
/** The key as stored: through a handle, placed under its prefix. */
|
||||
#storeKey(key: string, opts: DriverKvOpts): string {
|
||||
return opts.handlePrefix === undefined
|
||||
? key
|
||||
: `${opts.handlePrefix}${key}`;
|
||||
}
|
||||
|
||||
/** A stored key as a handle's caller names it; fails closed outside it. */
|
||||
#relativeKey(key: unknown, prefix: string): string {
|
||||
if (typeof key !== 'string' || !key.startsWith(prefix))
|
||||
throw outsideRegion();
|
||||
return key.slice(prefix.length);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one store call. Through a handle, an error naming a stored key names
|
||||
* the caller's relative key instead.
|
||||
*/
|
||||
async #inRegion<T>(
|
||||
opts: DriverKvOpts,
|
||||
keys: readonly string[],
|
||||
call: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
if (opts.handlePrefix === undefined) return call();
|
||||
try {
|
||||
return await call();
|
||||
} catch (err) {
|
||||
throw relativizeError(err, opts.handlePrefix, keys);
|
||||
}
|
||||
}
|
||||
|
||||
async #opts(method: string, args: KvCallArgs): Promise<DriverKvOpts> {
|
||||
async #opts(method: string, args: KvCallArgs): Promise<KVOpts> {
|
||||
const actor = Context.get('actor') as Actor | undefined;
|
||||
|
||||
// Every method resolves its options here first, so this is the one
|
||||
@@ -360,8 +170,6 @@ export class KVStoreDriver extends PuterDriver {
|
||||
);
|
||||
}
|
||||
|
||||
if (viaHandle(args)) return this.#handleOpts(actor, method, args);
|
||||
|
||||
const appUuid = args.optConfig?.appUuid;
|
||||
// Through the issuer chain, not `actor.app`: an access-token actor
|
||||
// carries no app of its own, so keying off `app` would read a token an
|
||||
@@ -390,214 +198,6 @@ export class KVStoreDriver extends PuterDriver {
|
||||
return { actor, namespaceAppUuid: appUuid };
|
||||
}
|
||||
|
||||
/**
|
||||
* A call through a share handle, resolved to the owner's namespace. The
|
||||
* handle row only names the region and whose it is; the grant, checked here
|
||||
* on every call, is the authority. One check covers every key of the call,
|
||||
* since they all sit under the handle's prefix.
|
||||
*/
|
||||
async #handleOpts(
|
||||
actor: Actor | undefined,
|
||||
method: string,
|
||||
args: KvCallArgs,
|
||||
): Promise<DriverKvOpts> {
|
||||
if (this.config.kv?.shareHandles !== true) throw kvHandlesDisabled();
|
||||
|
||||
const optConfig = args.optConfig ?? {};
|
||||
const level = kvShareLevelForMethod(method);
|
||||
if (optConfig.appUuid || optConfig.disableSharing)
|
||||
throw handleOptionInvalid(
|
||||
'kv: `handle` cannot be combined with `appUuid` or `disableSharing`',
|
||||
);
|
||||
if (level === null)
|
||||
throw handleOptionInvalid(
|
||||
`kv: \`${method}\` is not available through a share handle`,
|
||||
);
|
||||
|
||||
const handle: unknown = optConfig.handle;
|
||||
if (typeof handle !== 'string')
|
||||
throw handleOptionInvalid('kv: `handle` must be a string');
|
||||
const userId = actor?.user?.id;
|
||||
if (
|
||||
!actor ||
|
||||
typeof userId !== 'number' ||
|
||||
handle.length > KV_SHARE_HANDLE_WIDTHS.handle.max ||
|
||||
!isKvHandleId(handle)
|
||||
)
|
||||
throw handleNotFound();
|
||||
|
||||
const live = await this.stores.kvShareHandle.getLive(handle);
|
||||
if (!live || live.ownerSuspended) throw handleNotFound();
|
||||
const { row, ownerUuid } = live;
|
||||
// Bound to its grantee: a handle id is an address, not a bearer
|
||||
// credential, so another holder's overlapping grant doesn't reach it.
|
||||
if (userId !== row.granteeUserId && userId !== row.ownerUserId)
|
||||
throw handleNotFound();
|
||||
|
||||
// Side by side, so the app lookup costs no round trip of its own.
|
||||
const [appGone, allowed] = await Promise.all([
|
||||
row.appUid === KV_GLOBAL_APP_KEY
|
||||
? false
|
||||
: this.stores.app.getByUid(row.appUid).then((app) => !app),
|
||||
this.services.permission.check(
|
||||
actor,
|
||||
kvSharePermission(ownerUuid, row.appUid, row.keyPrefix, level),
|
||||
),
|
||||
]);
|
||||
if (appGone) throw handleNotFound();
|
||||
if (!allowed) {
|
||||
// Asked only on the way out: whoever can still watch the region
|
||||
// is its holder, and only they may learn the level is too low.
|
||||
const holds = await this.services.permission.check(
|
||||
actor,
|
||||
kvSharePermission(
|
||||
ownerUuid,
|
||||
row.appUid,
|
||||
row.keyPrefix,
|
||||
'watch',
|
||||
),
|
||||
);
|
||||
throw holds ? handleAccessDenied() : handleNotFound();
|
||||
}
|
||||
// Past the cap only a watch handle can be, reached through a wider
|
||||
// grant; the fixed key limit holders are shown assumes the cap.
|
||||
if (
|
||||
Buffer.byteLength(row.keyPrefix, 'utf8') > KV_SHARE_PREFIX_MAX_BYTES
|
||||
)
|
||||
throw handleAccessDenied();
|
||||
|
||||
return {
|
||||
actor,
|
||||
namespaceUser: { uuid: ownerUuid, id: row.ownerUserId },
|
||||
namespaceAppUuid: row.appUid,
|
||||
handle,
|
||||
handlePrefix: row.keyPrefix,
|
||||
};
|
||||
}
|
||||
|
||||
/** Set when a listing covers another namespace, cross-app or by handle. */
|
||||
#foreignListing(opts: DriverKvOpts): ForeignListing | undefined {
|
||||
if (opts.handle !== undefined)
|
||||
return {
|
||||
bind: opts.handle,
|
||||
namespace: kvNamespace(
|
||||
opts.namespaceUser!.uuid,
|
||||
opts.namespaceAppUuid!,
|
||||
),
|
||||
prefix: opts.handlePrefix!,
|
||||
acceptsPlain: false,
|
||||
};
|
||||
if (!opts.namespaceAppUuid) return undefined;
|
||||
const namespace = kvNamespace(
|
||||
opts.actor!.user.uuid!,
|
||||
opts.namespaceAppUuid,
|
||||
);
|
||||
return { bind: namespace, namespace, prefix: '', acceptsPlain: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* A foreign listing's cursor as a store position, rebuilt in the listing's
|
||||
* own namespace and region. Only a cursor sealed to this listing opens, or
|
||||
* a plain cross-app one naming the same namespace.
|
||||
*/
|
||||
#cursorIn(
|
||||
cursor: unknown,
|
||||
listing: ForeignListing,
|
||||
): Record<string, unknown> | undefined {
|
||||
if (cursor === undefined || cursor === null) return undefined;
|
||||
if (typeof cursor === 'string') {
|
||||
if (cursor.trim() === '') return undefined;
|
||||
const sealed = unsealCursor(
|
||||
cursor.trim(),
|
||||
this.config.jwt_secret_v2,
|
||||
);
|
||||
if (sealed) {
|
||||
const { s, k, r, ...rest } = sealed;
|
||||
if (
|
||||
s !== listing.bind ||
|
||||
typeof k !== 'string' ||
|
||||
(r !== undefined && r !== true) ||
|
||||
Object.keys(rest).length > 0
|
||||
)
|
||||
throw invalidCursor();
|
||||
const position = {
|
||||
namespace: listing.namespace,
|
||||
key: `${listing.prefix}${Buffer.from(k, 'base64url').toString('utf8')}`,
|
||||
};
|
||||
return r ? { reverse: true, key: position } : position;
|
||||
}
|
||||
} else if (typeof cursor !== 'object') throw invalidCursor();
|
||||
if (!listing.acceptsPlain) throw invalidCursor();
|
||||
|
||||
const plain = decodeCursor(
|
||||
cursor as string | Record<string, unknown>,
|
||||
'kv: cursor',
|
||||
);
|
||||
if (plain === undefined || isPositionIn(plain, listing.namespace))
|
||||
return plain;
|
||||
const { reverse, key, ...rest } = plain;
|
||||
if (
|
||||
reverse === true &&
|
||||
Object.keys(rest).length === 0 &&
|
||||
isPositionIn(key, listing.namespace)
|
||||
)
|
||||
return plain;
|
||||
throw invalidCursor();
|
||||
}
|
||||
|
||||
/**
|
||||
* The store's next-page position, sealed to the listing and padded to one
|
||||
* length, with its key relative to the region.
|
||||
*/
|
||||
#cursorOut(cursor: string, listing: ForeignListing): string | undefined {
|
||||
const decoded = decodeCursor(cursor);
|
||||
const reverse = decoded?.reverse === true;
|
||||
const position = (reverse ? decoded?.key : decoded) as
|
||||
| { namespace?: unknown; key?: unknown }
|
||||
| undefined;
|
||||
if (position?.namespace !== listing.namespace) throw outsideRegion();
|
||||
const key = this.#relativeKey(position.key, listing.prefix);
|
||||
return sealCursor(
|
||||
{
|
||||
s: listing.bind,
|
||||
k: Buffer.from(key, 'utf8').toString('base64url'),
|
||||
...(reverse ? { r: true } : {}),
|
||||
},
|
||||
this.config.jwt_secret_v2,
|
||||
{ padTo: FOREIGN_CURSOR_BYTES },
|
||||
);
|
||||
}
|
||||
|
||||
/** A foreign listing as its caller sees it: keys relative, cursor sealed. */
|
||||
#foreignResult(
|
||||
res: KvListResult,
|
||||
as: string | undefined,
|
||||
listing: ForeignListing,
|
||||
): KvListResult {
|
||||
const { prefix } = listing;
|
||||
const relative = (items: unknown[]): unknown[] => {
|
||||
if (as === 'values' || prefix === '') return items;
|
||||
if (as === 'keys')
|
||||
return items.map((key) => this.#relativeKey(key, prefix));
|
||||
return items.map((entry) => {
|
||||
const { key, value } = entry as {
|
||||
key: unknown;
|
||||
value: unknown;
|
||||
};
|
||||
return { key: this.#relativeKey(key, prefix), value };
|
||||
});
|
||||
};
|
||||
if (Array.isArray(res)) return relative(res);
|
||||
const cursor = res.cursor
|
||||
? this.#cursorOut(res.cursor, listing)
|
||||
: undefined;
|
||||
return {
|
||||
items: relative(res.items),
|
||||
...(cursor ? { cursor } : {}),
|
||||
...(res.total !== undefined ? { total: res.total } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
async #assertCrossAppKvAccess(
|
||||
actor: Actor,
|
||||
targetAppUid: string,
|
||||
@@ -722,7 +322,7 @@ export class KVStoreDriver extends PuterDriver {
|
||||
|
||||
async get(args: {
|
||||
key: unknown;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const { key } = args;
|
||||
if (key === undefined || key === null) {
|
||||
@@ -743,55 +343,45 @@ export class KVStoreDriver extends PuterDriver {
|
||||
|
||||
if (Array.isArray(key)) {
|
||||
if (key.length === 0) return [];
|
||||
const coerced = key.map((k) => this.#coerceKey(k, args));
|
||||
const { res, usage } = await this.#inRegion(opts, coerced, () =>
|
||||
this.stores.kv.get(
|
||||
{ key: coerced.map((k) => this.#storeKey(k, opts)) },
|
||||
opts,
|
||||
),
|
||||
const coerced = key.map((k) => this.#coerceKey(k));
|
||||
const { res, usage } = await this.stores.kv.get(
|
||||
{ key: coerced },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
}
|
||||
|
||||
const coerced = this.#coerceKey(key, args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.get({ key: this.#storeKey(coerced, opts) }, opts),
|
||||
const { res, usage } = await this.stores.kv.get(
|
||||
{ key: this.#coerceKey(key) },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
}
|
||||
|
||||
/**
|
||||
* `false` only for an `onlyIfAbsent` write that found a live entry, which
|
||||
* is left as it was.
|
||||
*/
|
||||
async set(args: {
|
||||
key: unknown;
|
||||
value: unknown;
|
||||
expireAt?: number | null;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string; disableSharing?: boolean };
|
||||
}): Promise<boolean> {
|
||||
const { key, value, expireAt } = args;
|
||||
const coerced = this.#coerceKey(key, args);
|
||||
const coerced = this.#coerceKey(key);
|
||||
if (value === undefined)
|
||||
throw new HttpError(400, 'Missing `value`', {
|
||||
legacyCode: 'bad_request',
|
||||
}); // legacyCode for backward compatibility with old error handling in controllers
|
||||
const onlyIfAbsent = parseOnlyIfAbsent(args.optConfig?.onlyIfAbsent);
|
||||
|
||||
const opts = await this.#opts('set', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.set(
|
||||
{
|
||||
key: this.#storeKey(coerced, opts),
|
||||
value,
|
||||
expireAt,
|
||||
disableSharing: args.optConfig?.disableSharing,
|
||||
onlyIfAbsent,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.set(
|
||||
{
|
||||
key: coerced,
|
||||
value,
|
||||
expireAt,
|
||||
disableSharing: args.optConfig?.disableSharing,
|
||||
},
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -799,7 +389,7 @@ export class KVStoreDriver extends PuterDriver {
|
||||
|
||||
async batchPut(args: {
|
||||
items: Array<{ key: string; value: unknown; expireAt?: number | null }>;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string; disableSharing?: boolean };
|
||||
}): Promise<boolean> {
|
||||
const { items } = args;
|
||||
if (!Array.isArray(items) || items.length === 0) {
|
||||
@@ -814,27 +404,17 @@ export class KVStoreDriver extends PuterDriver {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
// A batch write takes no condition, so it can't promise this per item.
|
||||
if (parseOnlyIfAbsent(args.optConfig?.onlyIfAbsent)) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'kv: onlyIfAbsent is not available on a batch write',
|
||||
{ legacyCode: 'invalid_request' },
|
||||
);
|
||||
}
|
||||
|
||||
const coerced = items.map((item) => ({
|
||||
key: this.#coerceKey(item.key, args),
|
||||
key: this.#coerceKey(item.key),
|
||||
value: item.value,
|
||||
expireAt: item.expireAt,
|
||||
}));
|
||||
|
||||
const opts = await this.#opts('batchPut', args);
|
||||
// Per-item expiry, which the top-level scan in `#opts` cannot see. A
|
||||
// handle's `write` level already carries expiries.
|
||||
// Per-item expiry, which the top-level scan in `#opts` cannot see.
|
||||
if (
|
||||
opts.namespaceAppUuid &&
|
||||
opts.handlePrefix === undefined &&
|
||||
coerced.some(
|
||||
(item) => item.expireAt !== undefined && item.expireAt !== null,
|
||||
)
|
||||
@@ -844,20 +424,12 @@ export class KVStoreDriver extends PuterDriver {
|
||||
opts.namespaceAppUuid,
|
||||
);
|
||||
}
|
||||
const { res, usage } = await this.#inRegion(
|
||||
const { res, usage } = await this.stores.kv.batchPut(
|
||||
{
|
||||
items: coerced,
|
||||
disableSharing: args.optConfig?.disableSharing,
|
||||
},
|
||||
opts,
|
||||
coerced.map((item) => item.key),
|
||||
() =>
|
||||
this.stores.kv.batchPut(
|
||||
{
|
||||
items: coerced.map((item) => ({
|
||||
...item,
|
||||
key: this.#storeKey(item.key, opts),
|
||||
})),
|
||||
disableSharing: args.optConfig?.disableSharing,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -865,13 +437,11 @@ export class KVStoreDriver extends PuterDriver {
|
||||
|
||||
async del(args: {
|
||||
key: unknown;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<boolean> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
const opts = await this.#opts('del', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.del({ key: this.#storeKey(coerced, opts) }, opts),
|
||||
);
|
||||
const { res, usage } = await this.stores.kv.del({ key: coerced }, opts);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
}
|
||||
@@ -885,7 +455,7 @@ export class KVStoreDriver extends PuterDriver {
|
||||
includeTotal?: boolean;
|
||||
fetchUntilFull?: boolean;
|
||||
reverse?: boolean;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const opts = await this.#opts('list', args);
|
||||
// Naming what it holds is how an account with nothing left decides what
|
||||
@@ -899,14 +469,6 @@ export class KVStoreDriver extends PuterDriver {
|
||||
this.config,
|
||||
);
|
||||
}
|
||||
const handlePrefix = opts.handlePrefix;
|
||||
const listing = this.#foreignListing(opts);
|
||||
if (
|
||||
handlePrefix !== undefined &&
|
||||
typeof args.pattern === 'string' &&
|
||||
Buffer.byteLength(args.pattern, 'utf8') > KV_HANDLE_KEY_MAX_BYTES
|
||||
)
|
||||
throw handlePatternInvalid();
|
||||
const { res, usage } = await this.stores.kv.list(
|
||||
{
|
||||
as: args.as,
|
||||
@@ -914,26 +476,20 @@ export class KVStoreDriver extends PuterDriver {
|
||||
cap: KV_MAX_LIST_LIMIT,
|
||||
label: 'kv: limit',
|
||||
}),
|
||||
cursor: listing
|
||||
? this.#cursorIn(args.cursor, listing)
|
||||
: args.cursor,
|
||||
cursor: args.cursor,
|
||||
pattern: args.pattern,
|
||||
offset: args.offset,
|
||||
includeTotal: args.includeTotal,
|
||||
fetchUntilFull: args.fetchUntilFull,
|
||||
reverse: args.reverse,
|
||||
...(handlePrefix === undefined
|
||||
? {}
|
||||
: { keyPrefix: handlePrefix }),
|
||||
},
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
if (!listing) return res;
|
||||
return this.#foreignResult(res as KvListResult, args.as, listing);
|
||||
return res;
|
||||
}
|
||||
|
||||
async flush(args: { optConfig?: KvOptConfig }): Promise<boolean> {
|
||||
async flush(args: { optConfig?: { appUuid?: string } }): Promise<boolean> {
|
||||
const opts = await this.#opts('flush', args);
|
||||
const { res, usage } = await this.stores.kv.flush(opts);
|
||||
this.#meter(opts.actor, usage);
|
||||
@@ -943,9 +499,9 @@ export class KVStoreDriver extends PuterDriver {
|
||||
async incr(args: {
|
||||
key: unknown;
|
||||
pathAndAmountMap: Record<string, number>;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (
|
||||
!args.pathAndAmountMap ||
|
||||
typeof args.pathAndAmountMap !== 'object'
|
||||
@@ -955,14 +511,9 @@ export class KVStoreDriver extends PuterDriver {
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('incr', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.incr(
|
||||
{
|
||||
key: this.#storeKey(coerced, opts),
|
||||
pathAndAmountMap: args.pathAndAmountMap,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.incr(
|
||||
{ key: coerced, pathAndAmountMap: args.pathAndAmountMap },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -971,9 +522,9 @@ export class KVStoreDriver extends PuterDriver {
|
||||
async decr(args: {
|
||||
key: unknown;
|
||||
pathAndAmountMap: Record<string, number>;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (
|
||||
!args.pathAndAmountMap ||
|
||||
typeof args.pathAndAmountMap !== 'object'
|
||||
@@ -983,14 +534,9 @@ export class KVStoreDriver extends PuterDriver {
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('decr', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.decr(
|
||||
{
|
||||
key: this.#storeKey(coerced, opts),
|
||||
pathAndAmountMap: args.pathAndAmountMap,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.decr(
|
||||
{ key: coerced, pathAndAmountMap: args.pathAndAmountMap },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -999,23 +545,18 @@ export class KVStoreDriver extends PuterDriver {
|
||||
async expireAt(args: {
|
||||
key: unknown;
|
||||
timestamp: number;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<boolean> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (typeof args.timestamp !== 'number') {
|
||||
throw new HttpError(400, '`timestamp` must be a number', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('expireAt', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.expireAt(
|
||||
{
|
||||
key: this.#storeKey(coerced, opts),
|
||||
timestamp: args.timestamp,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.expireAt(
|
||||
{ key: coerced, timestamp: args.timestamp },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -1024,20 +565,18 @@ export class KVStoreDriver extends PuterDriver {
|
||||
async expire(args: {
|
||||
key: unknown;
|
||||
ttl: number;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<boolean> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (typeof args.ttl !== 'number') {
|
||||
throw new HttpError(400, '`ttl` must be a number (seconds)', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('expire', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.expire(
|
||||
{ key: this.#storeKey(coerced, opts), ttl: args.ttl },
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.expire(
|
||||
{ key: coerced, ttl: args.ttl },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -1047,24 +586,22 @@ export class KVStoreDriver extends PuterDriver {
|
||||
key: unknown;
|
||||
pathAndValueMap: Record<string, unknown>;
|
||||
ttl?: number | null;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (!args.pathAndValueMap || typeof args.pathAndValueMap !== 'object') {
|
||||
throw new HttpError(400, 'Missing or invalid `pathAndValueMap`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('update', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.update(
|
||||
{
|
||||
key: this.#storeKey(coerced, opts),
|
||||
pathAndValueMap: args.pathAndValueMap,
|
||||
ttl: args.ttl,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.update(
|
||||
{
|
||||
key: coerced,
|
||||
pathAndValueMap: args.pathAndValueMap,
|
||||
ttl: args.ttl,
|
||||
},
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -1073,23 +610,18 @@ export class KVStoreDriver extends PuterDriver {
|
||||
async add(args: {
|
||||
key: unknown;
|
||||
pathAndValueMap: Record<string, unknown>;
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (!args.pathAndValueMap || typeof args.pathAndValueMap !== 'object') {
|
||||
throw new HttpError(400, 'Missing or invalid `pathAndValueMap`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('add', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.add(
|
||||
{
|
||||
key: this.#storeKey(coerced, opts),
|
||||
pathAndValueMap: args.pathAndValueMap,
|
||||
},
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.add(
|
||||
{ key: coerced, pathAndValueMap: args.pathAndValueMap },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
return res;
|
||||
@@ -1098,27 +630,23 @@ export class KVStoreDriver extends PuterDriver {
|
||||
async remove(args: {
|
||||
key: unknown;
|
||||
paths: string[];
|
||||
optConfig?: KvOptConfig;
|
||||
optConfig?: { appUuid?: string };
|
||||
}): Promise<unknown> {
|
||||
const coerced = this.#coerceKey(args.key, args);
|
||||
const coerced = this.#coerceKey(args.key);
|
||||
if (!Array.isArray(args.paths) || args.paths.length === 0) {
|
||||
throw new HttpError(400, 'Missing or invalid `paths`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const opts = await this.#opts('remove', args);
|
||||
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
|
||||
this.stores.kv.remove(
|
||||
{ key: this.#storeKey(coerced, opts), paths: args.paths },
|
||||
opts,
|
||||
),
|
||||
const { res, usage } = await this.stores.kv.remove(
|
||||
{ key: coerced, paths: args.paths },
|
||||
opts,
|
||||
);
|
||||
this.#meter(opts.actor, usage);
|
||||
// The result is the entry's value, so a delete-only grant gets none. A
|
||||
// handle's `write` level reads too.
|
||||
// The result is the entry's value, so a delete-only grant gets none.
|
||||
if (
|
||||
opts.namespaceAppUuid &&
|
||||
opts.handlePrefix === undefined &&
|
||||
!(await this.services.permission.check(
|
||||
opts.actor!,
|
||||
appDataPermission(opts.namespaceAppUuid, 'kv', 'get'),
|
||||
|
||||
@@ -36,7 +36,6 @@ import {
|
||||
} from '../../services/notification/notificationAudience.js';
|
||||
import {
|
||||
NOTIFICATION_VALUE_MAX_BYTES,
|
||||
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
|
||||
resolveNotificationWrite,
|
||||
type NotificationTypeName,
|
||||
} from '../../services/notification/notificationTypes.js';
|
||||
@@ -222,13 +221,7 @@ export class NotificationDriver extends PuterDriver {
|
||||
scope: MailboxScope | null;
|
||||
filter?: string;
|
||||
onlyUnacknowledged?: boolean;
|
||||
excludeTypes?: readonly string[];
|
||||
} = { limit, scope };
|
||||
// The whole mailbox, or a slice naming no app, is the desktop's view of
|
||||
// it, which leaves out the types only an app has a use for. A slice
|
||||
// naming the app still returns them.
|
||||
if (scope === null || scope.appUid === undefined)
|
||||
query.excludeTypes = OUTSIDE_ACCOUNT_MAILBOX_TYPES;
|
||||
|
||||
switch (predicateName) {
|
||||
case 'unseen':
|
||||
|
||||
@@ -441,7 +441,6 @@ describe('PuterServer host header validation — permissive modes', () => {
|
||||
allow_no_host_header: false,
|
||||
custom_domains_enabled: true,
|
||||
allow_nipio_domains: true,
|
||||
static_hosting_domain: 'site.puter.localhost',
|
||||
} as unknown as IConfig,
|
||||
{ listen: true },
|
||||
);
|
||||
@@ -451,13 +450,11 @@ describe('PuterServer host header validation — permissive modes', () => {
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
it('hands an unknown host to site hosting when custom domains are enabled', async () => {
|
||||
it('lets an unknown host through when custom domains are enabled', async () => {
|
||||
const res = await rawRequest(port, '/', {
|
||||
host: 'my-own-domain.example',
|
||||
});
|
||||
// No site claims it, so it 404s instead of reaching the GUI.
|
||||
expect(res.status).toBe(404);
|
||||
expect(res.body).toBe('Site not found');
|
||||
expect(res.status).not.toBe(400);
|
||||
});
|
||||
|
||||
it('accepts nip.io hosts when they are opted in', async () => {
|
||||
|
||||
@@ -210,20 +210,6 @@ export class PuterServer {
|
||||
[this.#config, this.clients, this.stores, this.services],
|
||||
);
|
||||
|
||||
// Core's seat rules skip silently without Teams, so asking for it with
|
||||
// nothing registered refuses to boot rather than serve without them.
|
||||
if (
|
||||
this.#config.teams_enabled === true &&
|
||||
!(
|
||||
(this.stores as { team?: unknown }).team &&
|
||||
(this.services as { team?: unknown }).team
|
||||
)
|
||||
) {
|
||||
throw new Error(
|
||||
'`teams_enabled` is on, but no extension registered the team store and service',
|
||||
);
|
||||
}
|
||||
|
||||
// Wire the rate-limiter to its configured backend now that clients
|
||||
// and stores exist. Memory is the default; `redis` needs a redis
|
||||
// client, `kv` needs the system KV store (DynamoDB-backed).
|
||||
|
||||
@@ -51,9 +51,8 @@ import {
|
||||
EVENTS_UNSUBSCRIBE_VERB,
|
||||
type DeliveryEnvelope,
|
||||
type EventSocket,
|
||||
type KvDispatchOptions,
|
||||
} from './EventsService.js';
|
||||
import { kvSharePermission } from '../kvShare/kvShares.js';
|
||||
import { kvSharePermission } from './kvShares.js';
|
||||
import { FILTER_EVALUATIONS_PER_EVENT } from './matcher.js';
|
||||
import { SUBSCRIPTION_CACHE_TTL_MS } from './subscriptionCache.js';
|
||||
import { kvAnchorToken } from './subjects.js';
|
||||
@@ -204,15 +203,6 @@ let handles: Map<string, KvShareHandle>;
|
||||
|
||||
const kvShareHandleStore = {
|
||||
getByHandle: async (handle: string) => handles.get(handle) ?? null,
|
||||
getLive: async (handle: string) => {
|
||||
const row = handles.get(handle);
|
||||
if (!row || row.revokedAt !== null) return null;
|
||||
return {
|
||||
row,
|
||||
ownerUuid: `user-${row.ownerUserId}`,
|
||||
ownerSuspended: false,
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
/** Apps the cross-app gate can be asked about, and what they share. */
|
||||
@@ -2118,7 +2108,6 @@ it('sends only the projected shape, never an internal row', async () => {
|
||||
'subject',
|
||||
'ts',
|
||||
'uid',
|
||||
'viaWorker',
|
||||
]);
|
||||
expect(sent[0].envelope.event).not.toHaveProperty('userId');
|
||||
});
|
||||
@@ -2565,47 +2554,6 @@ describe('delivering a kv change', () => {
|
||||
expect(sent).toEqual([]);
|
||||
});
|
||||
|
||||
it('tells a subscriber the change was theirs only when they made it', async () => {
|
||||
vi.useFakeTimers();
|
||||
await subscribeKv(`kv:${OWN_APP}:cart`);
|
||||
const write = (options: KvDispatchOptions) =>
|
||||
service.dispatchKv(
|
||||
{
|
||||
userId,
|
||||
namespace: `v1:user-${userId}:${OWN_APP}`,
|
||||
keys: ['cart'],
|
||||
op: 'set',
|
||||
},
|
||||
options,
|
||||
);
|
||||
|
||||
// One window each: writes to one key inside a window coalesce.
|
||||
for (const options of [
|
||||
{ actingUserId: userId },
|
||||
{ actingUserId: userId + 1 },
|
||||
{},
|
||||
{ actingUserId: userId, viaWorker: true },
|
||||
]) {
|
||||
await write(options);
|
||||
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
|
||||
}
|
||||
|
||||
expect(
|
||||
sent.map(({ envelope }) => {
|
||||
const { self, viaWorker } = envelope.event as {
|
||||
self: boolean;
|
||||
viaWorker: boolean;
|
||||
};
|
||||
return { self, viaWorker };
|
||||
}),
|
||||
).toEqual([
|
||||
{ self: true, viaWorker: false },
|
||||
{ self: false, viaWorker: false },
|
||||
{ self: false, viaWorker: false },
|
||||
{ self: true, viaWorker: true },
|
||||
]);
|
||||
});
|
||||
|
||||
it('sends only the projected shape', async () => {
|
||||
vi.useFakeTimers();
|
||||
await subscribeKv(`kv:${OWN_APP}:cart`);
|
||||
@@ -2621,7 +2569,6 @@ describe('delivering a kv change', () => {
|
||||
'seq',
|
||||
'subject',
|
||||
'ts',
|
||||
'viaWorker',
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -2896,7 +2843,6 @@ describe('the cross-app kv gate', () => {
|
||||
'seq',
|
||||
'subject',
|
||||
'ts',
|
||||
'viaWorker',
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -3005,10 +2951,8 @@ describe('cross-user kv handles', () => {
|
||||
appUid: OWN_APP,
|
||||
keyPrefix: PREFIX,
|
||||
permission,
|
||||
label: null,
|
||||
createdAt: 0,
|
||||
revokedAt: null,
|
||||
revokedBy: null,
|
||||
...overrides,
|
||||
};
|
||||
handles.set(row.handle, row);
|
||||
|
||||
@@ -31,6 +31,9 @@ import {
|
||||
EVENTS_HANDLER_DEPTH,
|
||||
EVENTS_HANDLER_PUBLISH_BATCH,
|
||||
EVENTS_HANDLER_PUBLISH_LIMIT,
|
||||
EVENTS_KV_HANDLE_LIMIT,
|
||||
EVENTS_KV_HANDLES_PER_APP,
|
||||
EVENTS_KV_HANDLES_PER_USER,
|
||||
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT,
|
||||
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT,
|
||||
EVENTS_KV_VALUE_MAX_BYTES,
|
||||
@@ -50,6 +53,7 @@ import {
|
||||
isAccessTokenActor,
|
||||
isAccountContext,
|
||||
makeActor,
|
||||
userRelatedActor,
|
||||
type Actor,
|
||||
} from '../../core/actor.js';
|
||||
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
|
||||
@@ -58,12 +62,15 @@ import type {
|
||||
ReanchorInput,
|
||||
SuspendedReason,
|
||||
} from '../../stores/events/DurableSubscriptionStore.js';
|
||||
import type { LiveKvShareHandle } from '../../stores/events/KvShareHandleStore.js';
|
||||
import {
|
||||
DurableQuotaRaceLost,
|
||||
HANDLER_SETTLE_BATCH,
|
||||
isSuspendedReason,
|
||||
} from '../../stores/events/DurableSubscriptionStore.js';
|
||||
import type {
|
||||
KvShareHandle,
|
||||
KvShareHandleListOptions,
|
||||
} from '../../stores/events/KvShareHandleStore.js';
|
||||
import {
|
||||
HANDLER_NAME_MAX_LENGTH,
|
||||
hashContent,
|
||||
@@ -91,7 +98,10 @@ import {
|
||||
type SubscriptionTarget,
|
||||
} from '../../stores/events/types.js';
|
||||
import type { FSEntry } from '../../stores/fs/FSEntry.js';
|
||||
import { parseKvNamespace } from '../../stores/systemKv/SystemKVStore.js';
|
||||
import {
|
||||
KV_GLOBAL_APP_KEY,
|
||||
parseKvNamespace,
|
||||
} from '../../stores/systemKv/SystemKVStore.js';
|
||||
import { awaitLater } from '../../util/concurrency.js';
|
||||
import {
|
||||
openCursor,
|
||||
@@ -101,17 +111,13 @@ import {
|
||||
import type { AclMode, ResourceDescriptor } from '../acl/ACLService.js';
|
||||
import { resolveNode } from '../fs/resolveNode.js';
|
||||
import { maskUnderAnchor } from '../fs/sharePathMask.js';
|
||||
import { actorPlanId, assertActorHasCredits } from '../metering/enforcement.js';
|
||||
import { assertActorHasCredits } from '../metering/enforcement.js';
|
||||
import {
|
||||
canViewNotification,
|
||||
notificationRowScope,
|
||||
ownedAppUids,
|
||||
} from '../notification/notificationAudience.js';
|
||||
import { notificationsFoldInEnabled } from '../notification/notificationSocket.js';
|
||||
import {
|
||||
inAccountMailbox,
|
||||
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
|
||||
} from '../notification/notificationTypes.js';
|
||||
import {
|
||||
appSocketRoom,
|
||||
type SocketSpecifier,
|
||||
@@ -190,10 +196,18 @@ import {
|
||||
} from './registry.js';
|
||||
import { SubscriptionCache } from './subscriptionCache.js';
|
||||
import {
|
||||
assertBoundedManageGrant,
|
||||
assertShareableAppUid,
|
||||
assertShareablePermission,
|
||||
assertShareablePrefix,
|
||||
kvShareAppDelegateImplicator,
|
||||
kvShareGrantCovers,
|
||||
kvShareManageNamespaceRoot,
|
||||
kvShareManagePermission,
|
||||
kvShareOwnerImplicator,
|
||||
kvSharePermission,
|
||||
relativeToKvShareRoot,
|
||||
} from '../kvShare/kvShares.js';
|
||||
} from './kvShares.js';
|
||||
import {
|
||||
KV_MATCH_SEPARATOR,
|
||||
NOTIF_MATCH_SEPARATOR,
|
||||
@@ -202,7 +216,6 @@ import {
|
||||
isKvToken,
|
||||
isNotifToken,
|
||||
kvHandleFromSubject,
|
||||
notifMatchSpansEveryApp,
|
||||
parseSubject,
|
||||
type FsOp,
|
||||
type ParsedSubject,
|
||||
@@ -419,6 +432,35 @@ export interface DurableSubscriptionView extends SubscriptionView {
|
||||
suspendedReason: string | null;
|
||||
}
|
||||
|
||||
/** Body of the handle-minting surface. The grantee is named either way. */
|
||||
export interface MintKvHandleRequest {
|
||||
granteeUsername?: unknown;
|
||||
granteeUid?: unknown;
|
||||
appUid?: unknown;
|
||||
prefix?: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* What minting returns. Only the handle and the region it covers: the owner
|
||||
* already knows the rest, and the grantee is handed this verbatim, so anything
|
||||
* else here would be something the handle exists to not say.
|
||||
*/
|
||||
export interface MintedKvHandle {
|
||||
handle: string;
|
||||
prefix: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* One handle as its owner sees it, revoked ones included — they are the record
|
||||
* of what was shared and when it stopped.
|
||||
*/
|
||||
export interface KvShareHandleView extends MintedKvHandle {
|
||||
appUid: string;
|
||||
granteeUsername: string | null;
|
||||
createdAt: number;
|
||||
revokedAt: number | null;
|
||||
}
|
||||
|
||||
export type VerbAck<T extends object> =
|
||||
| ({ ok: true } & T)
|
||||
| { ok: false; error: { code: string; message: string } };
|
||||
@@ -534,8 +576,6 @@ export interface FsDispatchOptions {
|
||||
actingUserId?: number;
|
||||
/** Handler runs behind the write; see `Actor.handlerDepth`. */
|
||||
handlerDepth?: number;
|
||||
/** The write was made by server-side code; see `viaWorkerOf`. */
|
||||
viaWorker?: boolean;
|
||||
/** Present only for a move: where the entry lived before it, lazily. */
|
||||
movedFrom?: {
|
||||
path: string;
|
||||
@@ -556,8 +596,6 @@ export interface FsDispatchOptions {
|
||||
export interface KvDispatchOptions {
|
||||
/** Who performed the write, for the `self` flag. */
|
||||
actingUserId?: number;
|
||||
/** The write was made by server-side code; see `viaWorkerOf`. */
|
||||
viaWorker?: boolean;
|
||||
/** See {@link FsDispatchOptions.forwarded}. */
|
||||
forwarded?: true;
|
||||
/** Carried over from the emitting region so both copies share one id/ts. */
|
||||
@@ -595,9 +633,6 @@ export interface KvDispatchInput {
|
||||
noShareKeys?: readonly string[];
|
||||
/** Handler runs behind the write; see `Actor.handlerDepth`. */
|
||||
handlerDepth?: number;
|
||||
/** Who made the change; the listener hands it on as an option. */
|
||||
actingUserId?: number;
|
||||
viaWorker?: true;
|
||||
}
|
||||
|
||||
interface RouteLimits {
|
||||
@@ -796,6 +831,19 @@ const handlerAppRequired = (): HttpError =>
|
||||
legacyCode: 'events_handler_app_required',
|
||||
});
|
||||
|
||||
/**
|
||||
* Listing and revoking are the owner's, and an app's only where the handle is
|
||||
* the app's own: its namespace, its user, and a delegation it still holds.
|
||||
* Anything else — another app, a token an app issued, a handle outside the
|
||||
* namespace — reads as somebody else's surface.
|
||||
*/
|
||||
const handleOwnerOnly = (): HttpError =>
|
||||
new HttpError(
|
||||
403,
|
||||
'Only the account, or the app whose namespace this is, may manage share handles',
|
||||
{ legacyCode: 'events_kv_handle_owner_only' },
|
||||
);
|
||||
|
||||
/** A scoped token, whoever issued it: neither the account nor the app. */
|
||||
const eventsWorkerOwnerOnly = (): HttpError =>
|
||||
new HttpError(
|
||||
@@ -804,6 +852,62 @@ const eventsWorkerOwnerOnly = (): HttpError =>
|
||||
{ legacyCode: 'events_worker_owner_only' },
|
||||
);
|
||||
|
||||
/**
|
||||
* An app minting is delegation, and the `manage:` grant on the region is the
|
||||
* consent for it. Without one there is nothing authorizing the app to dispose
|
||||
* of its user's data, whatever the user themselves could do here.
|
||||
*/
|
||||
const handleNotDelegated = (): HttpError =>
|
||||
new HttpError(
|
||||
403,
|
||||
'This app has not been given permission to share this data',
|
||||
{ legacyCode: 'events_kv_handle_not_delegated' },
|
||||
);
|
||||
|
||||
/**
|
||||
* A delegation is the app's to hold, not to pass on: a token it minted may
|
||||
* carry the `manage:` permission and still not mint through it. The user's own
|
||||
* token is not this case — it acts for the user, who needs no delegation.
|
||||
*/
|
||||
const handleAccessTokenForbidden = (): HttpError =>
|
||||
new HttpError(403, 'An access token may not mint a share handle', {
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
|
||||
/**
|
||||
* An app addresses exactly one key-value namespace — its own, under its user —
|
||||
* so that is the only one it can hand a region of out.
|
||||
*/
|
||||
const handleOutsideNamespace = (): HttpError =>
|
||||
new HttpError(403, 'An app may only share its own key-value data', {
|
||||
legacyCode: 'events_kv_handle_outside_namespace',
|
||||
});
|
||||
|
||||
/**
|
||||
* A temporary account holds no share handles: the grant outlives the session
|
||||
* that made it, and there is no account left behind it to take one back.
|
||||
*/
|
||||
const kvHandleNeedsAccount = (): HttpError =>
|
||||
new HttpError(403, 'A temporary account may not share its key-value data', {
|
||||
legacyCode: 'events_kv_handle_requires_account',
|
||||
});
|
||||
|
||||
/**
|
||||
* Past a handle ceiling. One code for both of them — the remedy is the same,
|
||||
* revoke something — with the message saying which one it was.
|
||||
*/
|
||||
const handleQuotaReached = (
|
||||
limit: number,
|
||||
scope: 'account' | 'app',
|
||||
): HttpError =>
|
||||
new HttpError(
|
||||
409,
|
||||
scope === 'app'
|
||||
? `An app may hold out ${limit} share handles for one account`
|
||||
: `An account may hold out ${limit} share handles at a time`,
|
||||
{ legacyCode: 'events_kv_handle_limit_reached' },
|
||||
);
|
||||
|
||||
/**
|
||||
* Deploying an app's code is the developer's, and an app token cannot borrow
|
||||
* its user's ownership of some other app. Same answer for an app that is not
|
||||
@@ -1370,17 +1474,6 @@ export class EventsService extends PuterService {
|
||||
});
|
||||
});
|
||||
|
||||
// A retired handle is a dead address: what was made through it ends,
|
||||
// even where another grant still covers the region. Awaited by the
|
||||
// emitter, so a revoke returns once its subscriptions are settled.
|
||||
this.clients.event.on('kv.share.retired', async (_key, data) => {
|
||||
try {
|
||||
await this.settleRetiredKvHandles(data);
|
||||
} catch (err) {
|
||||
console.warn('[events] retired-handle settle failed', err);
|
||||
}
|
||||
});
|
||||
|
||||
// A deleted app's rows can never deliver again — the grant identity
|
||||
// they are re-checked under no longer resolves — so they are torn
|
||||
// down rather than left to cost their holder an anchor slot forever.
|
||||
@@ -1414,17 +1507,9 @@ export class EventsService extends PuterService {
|
||||
// a service. Same posture as the FS hook: post-commit, and nothing here
|
||||
// can fail the write that produced it.
|
||||
this.clients.event.on('kv.mutated', (_key, data) => {
|
||||
const input = data as KvDispatchInput;
|
||||
void (async () => {
|
||||
try {
|
||||
await this.dispatchKv(input, {
|
||||
actingUserId: input.actingUserId,
|
||||
viaWorker: input.viaWorker === true,
|
||||
});
|
||||
} catch (err) {
|
||||
console.warn('[events] kv dispatch failed', err);
|
||||
}
|
||||
})();
|
||||
void this.dispatchKv(data as KvDispatchInput).catch((err) => {
|
||||
console.warn('[events] kv dispatch failed', err);
|
||||
});
|
||||
});
|
||||
|
||||
// Emitted once the row is written, so a delivery never names a
|
||||
@@ -1435,6 +1520,19 @@ export class EventsService extends PuterService {
|
||||
});
|
||||
});
|
||||
|
||||
// Owning a key-value namespace is holding every share grant over it,
|
||||
// which is what lets its owner mint a handle on their own data.
|
||||
this.services.permission.registerImplicator(kvShareOwnerImplicator());
|
||||
|
||||
// Lets an app-under-user actor exercise a share grant its user holds,
|
||||
// scoped to the app the grant's namespace names.
|
||||
this.services.permission.registerImplicator(
|
||||
kvShareAppDelegateImplicator({
|
||||
userHolds: (actor, permission) =>
|
||||
this.services.permission.check(actor, permission),
|
||||
}),
|
||||
);
|
||||
|
||||
this.#armExpirySweep();
|
||||
this.#armPendingSweep();
|
||||
this.#armCreditSweep();
|
||||
@@ -1836,8 +1934,15 @@ export class EventsService extends PuterService {
|
||||
}
|
||||
|
||||
/** The plan this actor is on, or `null` where there are no plans. */
|
||||
#planId(actor: Actor): Promise<string | null> {
|
||||
return actorPlanId(this.services.metering, actor);
|
||||
async #planId(actor: Actor): Promise<string | null> {
|
||||
const metering = this.services.metering;
|
||||
if (!metering) return null;
|
||||
try {
|
||||
return (await metering.getActorSubscription(actor)).id;
|
||||
} catch (err) {
|
||||
console.warn('[events] could not resolve a plan', err);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1942,16 +2047,11 @@ export class EventsService extends PuterService {
|
||||
);
|
||||
|
||||
// One extra row answers "is there another page" without a count.
|
||||
// The generic slice is the desktop's view of the mailbox, so it leaves
|
||||
// out the types kept off the desktop, as the live delivery does.
|
||||
const rows = await this.stores.notification.listScoped(user.id, {
|
||||
audience: scope.audience,
|
||||
appUid: scope.appUid,
|
||||
after: Number.isFinite(cursored) ? cursored : null,
|
||||
limit: limit + 1,
|
||||
...(scope.appUid === undefined
|
||||
? { excludeTypes: OUTSIDE_ACCOUNT_MAILBOX_TYPES }
|
||||
: {}),
|
||||
});
|
||||
|
||||
const page = rows.slice(0, limit);
|
||||
@@ -2299,6 +2399,373 @@ export class EventsService extends PuterService {
|
||||
return { appUid, removed, suspended };
|
||||
}
|
||||
|
||||
// -- Cross-user key-value handles --------------------------------
|
||||
|
||||
/**
|
||||
* Hand another user a watchable region of this account's key-value data.
|
||||
*
|
||||
* Two writes, in this order: the grant, which is the authorization and
|
||||
* carries its own `manage:` check and its own refusal to grant to yourself,
|
||||
* and then the handle, which is only a name for it. A handle whose grant
|
||||
* never landed would be a name for nothing; a grant whose handle never
|
||||
* landed is unaddressable, since a handle is the only thing that can name
|
||||
* this family.
|
||||
*
|
||||
* An app may mint too, bounded the way sharing bounds an app handing out
|
||||
* its user's files: authority is the user's, asked of the user behind the
|
||||
* actor and never of the app, and reach is what the credential itself
|
||||
* holds. Here reach is structural — an app addresses one namespace — so it
|
||||
* is asserted rather than worked out, and the app's own consent to delegate
|
||||
* is a `manage:` grant on the region.
|
||||
*/
|
||||
async mintKvHandle(
|
||||
actor: Actor,
|
||||
request: MintKvHandleRequest,
|
||||
): Promise<MintedKvHandle> {
|
||||
if (!this.enabled) throw disabled();
|
||||
if (!this.kvHandlesEnabled) throw kvShareHandleDisabled();
|
||||
|
||||
const owner = actor.user;
|
||||
if (!owner?.uuid || owner.id === undefined) throw disabled();
|
||||
// `undefined` is an app that could not be resolved, not the absence of
|
||||
// one — reading it as an account session is what would hand an app a
|
||||
// surface bounded on the app it is acting as.
|
||||
const app = actor.effectiveApp;
|
||||
if (app === undefined) throw handleOwnerOnly();
|
||||
// An account's surface is the account's, and a token scoped to less
|
||||
// than its issuer holds is not the account. A token an app issued is
|
||||
// refused by the delegation check below, which has its own answer.
|
||||
if (app === null && !isAccountContext(actor)) throw handleOwnerOnly();
|
||||
|
||||
// The budget is the user's, so an app spends its user's slots rather
|
||||
// than a machine-rate allowance of its own.
|
||||
await this.#spendHandleBudget(owner.id);
|
||||
|
||||
const keyPrefix = assertShareablePrefix(request?.prefix);
|
||||
const appUid = this.#mintNamespace(app, request);
|
||||
const grantee = await this.#resolveGrantee(request);
|
||||
|
||||
const permission = assertShareablePermission(
|
||||
kvSharePermission(owner.uuid, appUid, keyPrefix),
|
||||
);
|
||||
// Before the idempotent answer below, or an app that lost its
|
||||
// delegation could read back a handle it may no longer hand out.
|
||||
if (app) await this.#assertKvShareDelegated(actor, permission);
|
||||
|
||||
// Resolved before the idempotent answer below, so a plan that holds
|
||||
// none is refused whatever it asks for. Kept behind the delegation
|
||||
// check above: an app with no delegation must not learn anything
|
||||
// about the account's plan.
|
||||
const quota = await this.#handleQuota(actor);
|
||||
|
||||
// The same region minted again for the same grantee is the same
|
||||
// capability, not a second one — handing back what already exists is
|
||||
// what keeps two handles from ever sharing one permission row. The
|
||||
// ceiling counts slots, and this takes none, so it is not asked.
|
||||
const existing = await this.stores.kvShareHandle.findLive({
|
||||
ownerUserId: owner.id,
|
||||
granteeUserId: grantee.id,
|
||||
appUid,
|
||||
keyPrefix,
|
||||
});
|
||||
if (existing)
|
||||
return { handle: existing.handle, prefix: existing.keyPrefix };
|
||||
|
||||
await this.#assertHandleCeiling(owner.id, appUid, quota);
|
||||
|
||||
// The user behind the app, so the authority checked is theirs and the
|
||||
// issuer recorded is them. Which app acted is carried separately: it
|
||||
// belongs on the audit row, not in the grant's authority.
|
||||
await this.services.permission.grantUserUserPermission(
|
||||
userRelatedActor(actor),
|
||||
grantee.username,
|
||||
permission,
|
||||
{},
|
||||
{ reason: 'kv share handle', appUid: app?.uid },
|
||||
);
|
||||
|
||||
const row = await this.stores.kvShareHandle.mint({
|
||||
ownerUserId: owner.id,
|
||||
granteeUserId: grantee.id,
|
||||
appUid,
|
||||
keyPrefix,
|
||||
permission,
|
||||
});
|
||||
return { handle: row.handle, prefix: row.keyPrefix };
|
||||
}
|
||||
|
||||
/**
|
||||
* Which namespace a mint may name. An app reaches `v1:<user>:<its own app>`
|
||||
* and nothing else, so a request naming another one is refused rather than
|
||||
* quietly minted somewhere the app cannot even write.
|
||||
*/
|
||||
#mintNamespace(
|
||||
app: Actor['effectiveApp'],
|
||||
request: MintKvHandleRequest,
|
||||
): string {
|
||||
const named = parseAppUid(request?.appUid);
|
||||
if (!app) return assertShareableAppUid(named ?? KV_GLOBAL_APP_KEY);
|
||||
if (named !== null && named !== app.uid) throw handleOutsideNamespace();
|
||||
return app.uid;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this app was given the region to hand out. The `manage:` grant is
|
||||
* the user's consent, and prefix implication makes one taken on a region
|
||||
* cover the keys beneath it — so a handle deeper than the consent is still
|
||||
* inside it, and one above it is not.
|
||||
*/
|
||||
async #assertKvShareDelegated(
|
||||
actor: Actor,
|
||||
permission: string,
|
||||
): Promise<void> {
|
||||
// Consequential enough to require the app's own session, whatever a
|
||||
// token it minted happens to carry.
|
||||
if (isAccessTokenActor(actor)) throw handleAccessTokenForbidden();
|
||||
// The consent surface refuses a namespace-root delegation because no
|
||||
// prompt can describe it; refused here too, so a row written any other
|
||||
// way cannot authorize a mint.
|
||||
assertBoundedManageGrant(kvShareManagePermission(permission));
|
||||
const delegated = await this.services.permission.canManagePermission(
|
||||
actor,
|
||||
permission,
|
||||
);
|
||||
if (!delegated) throw handleNotDelegated();
|
||||
// `canManagePermission` walks ancestors, so a namespace-root grant
|
||||
// written by any path other than the consent surface would otherwise
|
||||
// authorize a mint anywhere in the namespace, bounded region or not.
|
||||
const unbounded = await this.services.permission.check(
|
||||
actor,
|
||||
kvShareManageNamespaceRoot(permission),
|
||||
);
|
||||
if (unbounded) throw handleNotDelegated();
|
||||
}
|
||||
|
||||
/**
|
||||
* Take a shared region back.
|
||||
*
|
||||
* The grant goes first — the subtree with it, since a deeper grant would
|
||||
* leave access to part of a region that has just been withdrawn — and the
|
||||
* handle is retired after. Removing the grant is what actually settles the
|
||||
* subscriptions, and it is the only half that stops delivery, so it is the
|
||||
* half that must not be able to fail after the other has succeeded. In that
|
||||
* order both steps are idempotent and a failure anywhere is a retry: the
|
||||
* reverse leaves a handle nothing can address standing on a grant
|
||||
* everything still passes, and no way to try again.
|
||||
*
|
||||
* Not gated on the feature flag. Withdrawing access only ever narrows, and
|
||||
* an install that turned handles off must still be able to retire the ones
|
||||
* it has.
|
||||
*/
|
||||
async revokeKvHandle(
|
||||
actor: Actor,
|
||||
handle: unknown,
|
||||
): Promise<{ handle: string; revokedAt: number }> {
|
||||
if (!this.enabled) throw disabled();
|
||||
const owner = actor.user;
|
||||
if (owner?.id === undefined) throw disabled();
|
||||
// `undefined` is an app that could not be resolved, not the absence of
|
||||
// one — reading it as an account session is what would hand an app the
|
||||
// account's whole surface.
|
||||
const app = actor.effectiveApp;
|
||||
if (app === undefined) throw handleOwnerOnly();
|
||||
// A delegation is the app's to hold, not to pass on, so a token an app
|
||||
// issued is refused here as it is on the mint path — and a user's own
|
||||
// token acts for the user only when it carries the user's whole reach.
|
||||
if (isAccessTokenActor(actor) && !isAccountContext(actor))
|
||||
throw handleOwnerOnly();
|
||||
|
||||
await this.#spendHandleBudget(owner.id);
|
||||
|
||||
const named = typeof handle === 'string' ? handle : '';
|
||||
const share = await this.stores.kvShareHandle.getByHandle(named);
|
||||
// Unknown and somebody else's read the same way, so revoking cannot be
|
||||
// used to find out that a handle exists.
|
||||
if (!share || share.ownerUserId !== owner.id)
|
||||
throw unknownKvShareHandle(named);
|
||||
// An app reaches one namespace, so a handle outside it is not its to
|
||||
// take back — checked after the owner test, which must answer first.
|
||||
if (app && share.appUid !== app.uid) throw handleOwnerOnly();
|
||||
// The same consent the mint took: a withdrawn delegation withdraws the
|
||||
// app's surface on the region with it.
|
||||
if (app) await this.#assertKvShareDelegated(actor, share.permission);
|
||||
|
||||
const grantee = await this.stores.user.getById(share.granteeUserId);
|
||||
if (!grantee?.username)
|
||||
throw new HttpError(500, 'The grantee of this handle is gone', {
|
||||
legacyCode: 'internal_error',
|
||||
});
|
||||
await this.services.permission.revokeUserUserPermissionSubtree(
|
||||
userRelatedActor(actor),
|
||||
grantee.username,
|
||||
share.permission,
|
||||
{ reason: 'kv share handle revoked', appUid: app?.uid },
|
||||
);
|
||||
|
||||
const revoked = await this.stores.kvShareHandle.retire(named, owner.id);
|
||||
if (!revoked?.revokedAt) throw unknownKvShareHandle(named);
|
||||
|
||||
// A narrower handle to the same grantee shares this one's permission
|
||||
// row whenever its region nests under it, and the subtree revoke
|
||||
// above just withdrew that row — so its grant is already gone even
|
||||
// though its own row still reads live.
|
||||
await this.#retireCoveredHandles(revoked);
|
||||
|
||||
return { handle: revoked.handle, revokedAt: revoked.revokedAt };
|
||||
}
|
||||
|
||||
/**
|
||||
* Every other live handle to this grantee that the just-withdrawn grant
|
||||
* covered. Coverage runs on the permission string, whose third component is
|
||||
* the namespace app uid, so a covered handle is always inside the same
|
||||
* one.
|
||||
*/
|
||||
async #retireCoveredHandles(revoked: KvShareHandle): Promise<void> {
|
||||
const siblings =
|
||||
await this.stores.kvShareHandle.listLiveForOwnerAndGrantee(
|
||||
revoked.ownerUserId,
|
||||
revoked.granteeUserId,
|
||||
);
|
||||
await Promise.all(
|
||||
siblings
|
||||
.filter((row) =>
|
||||
kvShareGrantCovers(revoked.permission, row.permission),
|
||||
)
|
||||
.map((row) =>
|
||||
this.stores.kvShareHandle.retire(
|
||||
row.handle,
|
||||
revoked.ownerUserId,
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* What this account has shared out of its key-value data, retired handles
|
||||
* included: they are the only record of what was shared and when it
|
||||
* stopped. The grantee has no listing of their own — what they hold is the
|
||||
* handle they were given.
|
||||
*/
|
||||
async listKvHandles(
|
||||
actor: Actor,
|
||||
options: KvShareHandleListOptions = {},
|
||||
): Promise<PageResult<KvShareHandleView>> {
|
||||
if (!this.enabled) throw disabled();
|
||||
const owner = actor.user;
|
||||
if (owner?.id === undefined) throw disabled();
|
||||
const app = actor.effectiveApp;
|
||||
if (app === undefined) throw handleOwnerOnly();
|
||||
if (isAccessTokenActor(actor) && !isAccountContext(actor))
|
||||
throw handleOwnerOnly();
|
||||
|
||||
// An app sees its own namespace and nothing else; an account session
|
||||
// sees across apps, which is what makes the account the surface for a
|
||||
// row whose app is long gone. Listing disposes of nothing, so the gate
|
||||
// is app-ness and the namespace rather than the region's delegation.
|
||||
const page = await this.stores.kvShareHandle.listForOwner(owner.id, {
|
||||
limit: options.limit,
|
||||
cursor: options.cursor,
|
||||
includeTotal: options.includeTotal,
|
||||
appUid: app?.uid ?? null,
|
||||
});
|
||||
const grantees = await this.stores.user.getByIds([
|
||||
...new Set(page.items.map((row) => row.granteeUserId)),
|
||||
]);
|
||||
|
||||
return {
|
||||
...page,
|
||||
items: page.items.map((row) => ({
|
||||
handle: row.handle,
|
||||
prefix: row.keyPrefix,
|
||||
appUid: row.appUid,
|
||||
granteeUsername:
|
||||
grantees.get(row.granteeUserId)?.username ?? null,
|
||||
createdAt: row.createdAt,
|
||||
revokedAt: row.revokedAt,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
/** Who a mint is for. Named by username or uuid; unknown reads as absent. */
|
||||
async #resolveGrantee(
|
||||
request: MintKvHandleRequest,
|
||||
): Promise<{ id: number; username: string }> {
|
||||
const username =
|
||||
typeof request?.granteeUsername === 'string'
|
||||
? request.granteeUsername.trim()
|
||||
: '';
|
||||
const uid =
|
||||
typeof request?.granteeUid === 'string'
|
||||
? request.granteeUid.trim()
|
||||
: '';
|
||||
if (!username && !uid)
|
||||
throw badRequest(
|
||||
'Name the grantee with `granteeUsername` or `granteeUid`',
|
||||
'bad_request',
|
||||
);
|
||||
|
||||
const user = username
|
||||
? await this.stores.user.getByUsername(username)
|
||||
: await this.stores.user.getByUuid(uid);
|
||||
if (!user?.username || user.id === undefined)
|
||||
throw new HttpError(404, 'user_does_not_exist', {
|
||||
legacyCode: 'subject_does_not_exist',
|
||||
});
|
||||
return { id: user.id, username: user.username };
|
||||
}
|
||||
|
||||
/**
|
||||
* How many handles this caller's plan lets the account hold out, in total
|
||||
* and in one namespace. Resolved before the idempotent answer on the mint
|
||||
* path, so a plan that may hold none is refused whatever it asks for.
|
||||
*
|
||||
* A deployment with no metering has no plans to read, and is held to the
|
||||
* paid caps.
|
||||
*/
|
||||
async #handleQuota(actor: Actor): Promise<TieredQuota> {
|
||||
const plan = await this.#planId(actor);
|
||||
if (plan !== null && limitFor(EVENTS_KV_HANDLES_PER_USER, plan) === 0)
|
||||
throw kvHandleNeedsAccount();
|
||||
return {
|
||||
perUser: limitFor(EVENTS_KV_HANDLES_PER_USER, plan),
|
||||
perApp: limitFor(EVENTS_KV_HANDLES_PER_APP, plan),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this account may hold out another share handle, in total and in
|
||||
* the namespace this mint names. Retired ones do not count: the row stays
|
||||
* as the record of what was shared, not as a slot. Check-then-act: two
|
||||
* mints racing past it can both proceed, which the mint budget bounds well
|
||||
* enough for an abuse backstop.
|
||||
*/
|
||||
async #assertHandleCeiling(
|
||||
userId: number,
|
||||
appUid: string,
|
||||
quota: TieredQuota,
|
||||
): Promise<void> {
|
||||
const live = await this.stores.kvShareHandle.countLiveForOwner(
|
||||
userId,
|
||||
appUid,
|
||||
);
|
||||
if (live.total >= quota.perUser)
|
||||
throw handleQuotaReached(quota.perUser, 'account');
|
||||
// The global namespace is the account's own shelf rather than an
|
||||
// app's, so the account cap is its only bound — which is also what
|
||||
// keeps that cap reachable without naming a namespace.
|
||||
if (appUid !== KV_GLOBAL_APP_KEY && live.forApp >= quota.perApp)
|
||||
throw handleQuotaReached(quota.perApp, 'app');
|
||||
}
|
||||
|
||||
async #spendHandleBudget(userId: number): Promise<void> {
|
||||
const ok = await checkRateLimit(
|
||||
`${EVENTS_KV_HANDLE_LIMIT.scope}:${userId}`,
|
||||
EVENTS_KV_HANDLE_LIMIT.limit,
|
||||
EVENTS_KV_HANDLE_LIMIT.window,
|
||||
);
|
||||
if (!ok) throw tooManyCalls();
|
||||
}
|
||||
|
||||
// -- Suspension --------------------------------------------------
|
||||
|
||||
/**
|
||||
@@ -3080,11 +3547,9 @@ export class EventsService extends PuterService {
|
||||
* keyspace that is matters: the row is indexed under the **owner**, because
|
||||
* a write only ever knows whose namespace it touched.
|
||||
*
|
||||
* Authority is the grant, never the handle: an actor holding the grant at
|
||||
* any level may subscribe, and one who does not is told the handle is not
|
||||
* there rather than that it is theirs to want. The handle is also bound to
|
||||
* its holder, so another user whose own grant overlaps the region cannot
|
||||
* use it.
|
||||
* Authority is the grant, never the handle: an actor holding the mirrored
|
||||
* permission may subscribe, and one who does not is told the handle is not
|
||||
* there rather than that it is theirs to want.
|
||||
*/
|
||||
async #resolveKvHandleSubscribe(
|
||||
actor: Actor,
|
||||
@@ -3096,31 +3561,17 @@ export class EventsService extends PuterService {
|
||||
const share = await this.stores.kvShareHandle.getByHandle(handle);
|
||||
if (!share || share.revokedAt !== null)
|
||||
throw unknownKvShareHandle(handle);
|
||||
if (
|
||||
actor.user?.id !== share.granteeUserId &&
|
||||
actor.user?.id !== share.ownerUserId
|
||||
)
|
||||
throw unknownKvShareHandle(handle);
|
||||
|
||||
const owner = await this.stores.user.getById(share.ownerUserId);
|
||||
if (!owner?.uuid) throw unknownKvShareHandle(handle);
|
||||
|
||||
// Watching is the lowest level, so every level answers it; storing it
|
||||
// rather than the handle's own grant is what lets a level change leave
|
||||
// the subscription alone.
|
||||
const watch = kvSharePermission(
|
||||
owner.uuid,
|
||||
share.appUid,
|
||||
share.keyPrefix,
|
||||
'watch',
|
||||
);
|
||||
const held = await kvSharedRegionAuthorized(
|
||||
actor,
|
||||
watch,
|
||||
share.permission,
|
||||
this.#kvShareDeps(),
|
||||
);
|
||||
if (!held) throw unknownKvShareHandle(handle);
|
||||
|
||||
const owner = await this.stores.user.getById(share.ownerUserId);
|
||||
if (!owner?.uuid) throw unknownKvShareHandle(handle);
|
||||
|
||||
const anchor = resolveKvHandleAnchor(parsed, {
|
||||
ownerUserUuid: owner.uuid,
|
||||
appUid: share.appUid,
|
||||
@@ -3139,7 +3590,7 @@ export class EventsService extends PuterService {
|
||||
ownerUserId: share.ownerUserId,
|
||||
// The grant string rather than a mode: it is what a revoke names,
|
||||
// and what the delivery re-check asks again.
|
||||
permission: watch,
|
||||
permission: share.permission,
|
||||
subject: anchor.subject,
|
||||
};
|
||||
}
|
||||
@@ -3265,7 +3716,6 @@ export class EventsService extends PuterService {
|
||||
id: options.id ?? randomUUID(),
|
||||
ts: options.ts ?? Date.now(),
|
||||
handlerDepth: options.handlerDepth,
|
||||
viaWorker: options.viaWorker === true,
|
||||
};
|
||||
|
||||
const { local, remote } =
|
||||
@@ -3283,7 +3733,6 @@ export class EventsService extends PuterService {
|
||||
family: 'fs',
|
||||
ownerUserId,
|
||||
actingUserId: options.actingUserId,
|
||||
...(context.viaWorker ? { viaWorker: true as const } : {}),
|
||||
id: context.id,
|
||||
ts: context.ts,
|
||||
fs: {
|
||||
@@ -3374,7 +3823,6 @@ export class EventsService extends PuterService {
|
||||
id: options.forwarded && options.id ? options.id : randomUUID(),
|
||||
ts,
|
||||
handlerDepth: input.handlerDepth,
|
||||
viaWorker: options.viaWorker === true,
|
||||
...(noShareKeys.has(kvKey) ? { noShare: true as const } : {}),
|
||||
}));
|
||||
const carried: Array<{ value: unknown } | undefined> = [];
|
||||
@@ -3417,9 +3865,6 @@ export class EventsService extends PuterService {
|
||||
family: 'kv',
|
||||
ownerUserId,
|
||||
actingUserId: options.actingUserId,
|
||||
...(context.viaWorker
|
||||
? { viaWorker: true as const }
|
||||
: {}),
|
||||
id: context.id,
|
||||
ts: context.ts,
|
||||
kv: {
|
||||
@@ -3486,7 +3931,7 @@ export class EventsService extends PuterService {
|
||||
subject,
|
||||
context,
|
||||
forKey,
|
||||
options.actingUserId,
|
||||
ownerUserId,
|
||||
(matched) =>
|
||||
this.#kvStillAuthorized(
|
||||
matched,
|
||||
@@ -3587,7 +4032,6 @@ export class EventsService extends PuterService {
|
||||
ancestors: async () => item.fs!.movedFrom!.ancestors,
|
||||
},
|
||||
actingUserId: item.actingUserId,
|
||||
viaWorker: item.viaWorker === true,
|
||||
forwarded: true,
|
||||
id: item.id,
|
||||
ts: item.ts,
|
||||
@@ -3624,7 +4068,6 @@ export class EventsService extends PuterService {
|
||||
},
|
||||
{
|
||||
actingUserId: item.actingUserId,
|
||||
viaWorker: item.viaWorker === true,
|
||||
forwarded: true,
|
||||
id: item.id,
|
||||
ts: item.ts,
|
||||
@@ -3680,14 +4123,10 @@ export class EventsService extends PuterService {
|
||||
);
|
||||
if (watched.length === 0) return;
|
||||
|
||||
const fetched = await this.stores.eventSubscription.getForTokens(
|
||||
const rows = await this.stores.eventSubscription.getForTokens(
|
||||
userId,
|
||||
watched,
|
||||
);
|
||||
// A subscription naming no app is the desktop's view of the mailbox.
|
||||
const rows = inAccountMailbox(context.type)
|
||||
? fetched
|
||||
: fetched.filter((row) => !notifMatchSpansEveryApp(row.match));
|
||||
if (rows.length === 0) return;
|
||||
|
||||
await this.#route(subject, context, rows, userId, (matched) =>
|
||||
@@ -4055,27 +4494,14 @@ export class EventsService extends PuterService {
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether one row's holder may still watch the region it was made on. The
|
||||
* handle must be live and theirs, so a retired one goes silent within the
|
||||
* handle cache's lifetime even where a wider grant covers the region. The
|
||||
* grant check doesn't vary by key, so it's cached under the row's anchor.
|
||||
* Whether one row's holder may still watch the region it was made on.
|
||||
*
|
||||
* Held in the cross-event cache under the row's anchor, which for a shared
|
||||
* region is the whole of what it can address: nothing above the handle is
|
||||
* nameable, so the answer does not vary by key and one evaluation covers
|
||||
* every event under it until a grant or a revoke moves the generation.
|
||||
*/
|
||||
async #kvShareHolds(row: DispatchSubscription): Promise<boolean> {
|
||||
const handle = kvHandleFromSubject(row.subject);
|
||||
if (handle === null) return false;
|
||||
let live: LiveKvShareHandle | null;
|
||||
try {
|
||||
live = await this.stores.kvShareHandle.getLive(handle);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
!live ||
|
||||
(live.row.granteeUserId !== row.holderUserId &&
|
||||
live.row.ownerUserId !== row.holderUserId)
|
||||
)
|
||||
return false;
|
||||
|
||||
const identity = await this.#grantIdentity(row);
|
||||
if (!identity) return false;
|
||||
|
||||
@@ -4247,37 +4673,6 @@ export class EventsService extends PuterService {
|
||||
return suspended.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Take out of service every durable subscription one user, the share's
|
||||
* owner or its holder, made through these share handles, with their
|
||||
* backlogs. One holder-index read, bounded by the user's subscription cap.
|
||||
* Session rows are left to the delivery re-check, which reads the handle's
|
||||
* liveness.
|
||||
*/
|
||||
async settleRetiredKvHandles(retired: {
|
||||
handles: readonly string[];
|
||||
userId: number;
|
||||
}): Promise<number> {
|
||||
if (!this.enabled || retired.handles.length === 0) return 0;
|
||||
const held = await this.stores.durableSubscription.listActiveForHolder(
|
||||
retired.userId,
|
||||
null,
|
||||
{ includeSuspended: true },
|
||||
);
|
||||
const handles = new Set(retired.handles);
|
||||
const settling = held.filter((row) => {
|
||||
const handle = kvHandleFromSubject(row.subject);
|
||||
return handle !== null && handles.has(handle);
|
||||
});
|
||||
if (settling.length === 0) return 0;
|
||||
|
||||
const suspended = await this.#suspend(settling, 'permission_revoked', {
|
||||
override: RESUMABLE_REASONS,
|
||||
});
|
||||
await this.#notifyEnded(suspended, 'permission_revoked');
|
||||
return suspended.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Revoke the reused `events:handlers` worker session for (holder, app), if
|
||||
* one was ever minted. Best-effort: a lookup or revoke failure must not
|
||||
|
||||
@@ -224,7 +224,6 @@ describe('an app connection', () => {
|
||||
'subject',
|
||||
'ts',
|
||||
'uid',
|
||||
'viaWorker',
|
||||
]);
|
||||
});
|
||||
|
||||
|
||||
@@ -81,8 +81,6 @@ export interface ForwardEvent {
|
||||
ownerUserId: number;
|
||||
/** Who caused it, so `self` is right on the far side. */
|
||||
actingUserId?: number;
|
||||
/** Made by server-side code, so `viaWorker` is right on the far side. */
|
||||
viaWorker?: true;
|
||||
/** The emitter's event id and timestamp, kept so the two copies match. */
|
||||
id: string;
|
||||
ts: number;
|
||||
|
||||
@@ -2044,34 +2044,6 @@ describe('a kv value crossing to another region', () => {
|
||||
|
||||
expect(eventsIn(west)[0].kv).toMatchObject({ value: { items: 3 } });
|
||||
});
|
||||
|
||||
it('carries who wrote it, so `self` and `viaWorker` hold on the far side', async () => {
|
||||
const west = makeRegion('west', ['east'], forwardCfg);
|
||||
const east = makeRegion('east', ['west'], forwardCfg);
|
||||
await subscribeKv(east, 'socket-writer');
|
||||
await announced(east);
|
||||
|
||||
await west.events.dispatchKv(
|
||||
{
|
||||
userId,
|
||||
namespace: `v1:user-${userId}:${APP}`,
|
||||
keys: ['cart'],
|
||||
op: 'set',
|
||||
},
|
||||
{ actingUserId: userId + 1, viaWorker: true },
|
||||
);
|
||||
await posted(west);
|
||||
await arrived(east);
|
||||
|
||||
expect(eventsIn(west)[0]).toMatchObject({
|
||||
actingUserId: userId + 1,
|
||||
viaWorker: true,
|
||||
});
|
||||
expect(east.sent[0].event).toMatchObject({
|
||||
self: false,
|
||||
viaWorker: true,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('a session subscription in another region', () => {
|
||||
@@ -2086,33 +2058,10 @@ describe('a session subscription in another region', () => {
|
||||
await dispatch(west);
|
||||
|
||||
await arrived(east);
|
||||
expect(east.sent[0].event).toMatchObject({
|
||||
op: 'write',
|
||||
self: true,
|
||||
viaWorker: false,
|
||||
});
|
||||
expect(east.sent[0].event).toMatchObject({ op: 'write', self: true });
|
||||
expect(west.sent).toEqual([]);
|
||||
});
|
||||
|
||||
it('carries a server-side write as one on the far side', async () => {
|
||||
const west = makeRegion('west', ['east'], forwardCfg);
|
||||
const east = makeRegion('east', ['west'], forwardCfg);
|
||||
await subscribeSession(east);
|
||||
await posted(east);
|
||||
|
||||
await west.events.dispatchFs('fs.write.file', entry(), {
|
||||
actingUserId: userId,
|
||||
viaWorker: true,
|
||||
ancestors: async () => ancestors(),
|
||||
});
|
||||
|
||||
await arrived(east);
|
||||
expect(east.sent[0].event).toMatchObject({
|
||||
self: true,
|
||||
viaWorker: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('delivers to a subscriber who is not the writer', async () => {
|
||||
const west = makeRegion('west', ['east'], forwardCfg);
|
||||
const east = makeRegion('east', ['west'], forwardCfg);
|
||||
|
||||
@@ -42,10 +42,7 @@ import { runWithContext } from '../../core/context.js';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import type { DeliveryEnvelope } from './EventsService.js';
|
||||
import {
|
||||
kvShareManagePermission,
|
||||
kvSharePermission,
|
||||
} from '../kvShare/kvShares.js';
|
||||
import { kvShareManagePermission, kvSharePermission } from './kvShares.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
const SOCKET_ID = 'kv-delegate-socket';
|
||||
@@ -67,7 +64,6 @@ let appId: number;
|
||||
let appActor: Actor;
|
||||
|
||||
const events = () => env.server.services.events;
|
||||
const shares = () => env.server.services.kvShare;
|
||||
const permissions = () => env.server.services.permission;
|
||||
|
||||
const settled = (count = 1) =>
|
||||
@@ -115,15 +111,11 @@ const undelegate = (prefix = PREFIX) =>
|
||||
);
|
||||
|
||||
const mint = (request: Record<string, unknown> = {}, actor = appActor) =>
|
||||
shares().mintKvHandle(
|
||||
actor,
|
||||
{
|
||||
prefix: PREFIX,
|
||||
...request,
|
||||
recipient: { username: guest.username },
|
||||
},
|
||||
{ defaultAccess: 'watch' },
|
||||
);
|
||||
events().mintKvHandle(actor, {
|
||||
granteeUsername: guest.username,
|
||||
prefix: PREFIX,
|
||||
...request,
|
||||
});
|
||||
|
||||
const clearRows = async () => {
|
||||
await env.server.clients.db.write(`DELETE FROM \`${TABLE}\``, []);
|
||||
@@ -199,7 +191,7 @@ describe('an app minting without consent', () => {
|
||||
legacyCode: 'events_kv_handle_not_delegated',
|
||||
});
|
||||
// The share itself is untouched: what was refused is the app asking.
|
||||
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
|
||||
const page = await events().listKvHandles(owner.actor, { limit: 100 });
|
||||
expect(page.items).toContainEqual(
|
||||
expect.objectContaining({ handle, revokedAt: null }),
|
||||
);
|
||||
@@ -413,14 +405,14 @@ describe('an app minting with consent', () => {
|
||||
|
||||
it('shows up in the owner`s listing of what they have shared out', async () => {
|
||||
const { handle } = await mint();
|
||||
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
|
||||
const page = await events().listKvHandles(owner.actor, { limit: 100 });
|
||||
|
||||
expect(page.items).toContainEqual(
|
||||
expect.objectContaining({
|
||||
handle,
|
||||
prefix: PREFIX,
|
||||
appUid,
|
||||
holderUsername: guest.username,
|
||||
granteeUsername: guest.username,
|
||||
revokedAt: null,
|
||||
}),
|
||||
);
|
||||
@@ -462,7 +454,7 @@ describe('a handle an app minted', () => {
|
||||
handlerName: 'onChange',
|
||||
});
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
const [row] = (await env.server.clients.db.pread(
|
||||
@@ -548,7 +540,7 @@ describe('a grantee subscribing through their own app actor', () => {
|
||||
await settled();
|
||||
expect(delivered[0].subId).toBe(sub.subId);
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
delivered.length = 0;
|
||||
|
||||
await appWrites(`${PREFIX}live:2`, 2);
|
||||
@@ -581,7 +573,7 @@ describe('an app managing what it minted', () => {
|
||||
handlerName: 'onChange',
|
||||
});
|
||||
|
||||
await shares().revokeKvHandle(appActor, handle);
|
||||
await events().revokeKvHandle(appActor, handle);
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
const [row] = (await env.server.clients.db.pread(
|
||||
@@ -600,7 +592,7 @@ describe('an app managing what it minted', () => {
|
||||
),
|
||||
).resolves.toBe(false);
|
||||
|
||||
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
|
||||
const page = await events().listKvHandles(owner.actor, { limit: 100 });
|
||||
expect(
|
||||
page.items.find((one) => one.handle === handle)?.revokedAt,
|
||||
).toBeTypeOf('number');
|
||||
@@ -615,10 +607,10 @@ describe('an app managing what it minted', () => {
|
||||
const prefix = `${PREFIX}session-in-app-${uuidv4().slice(0, 8)}:`;
|
||||
const { handle } = await mint({ appUid, prefix }, owner.actor);
|
||||
|
||||
const revoked = await shares().revokeKvHandle(appActor, handle);
|
||||
const revoked = await events().revokeKvHandle(appActor, handle);
|
||||
expect(revoked.handle).toBe(handle);
|
||||
|
||||
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
|
||||
const page = await events().listKvHandles(owner.actor, { limit: 100 });
|
||||
expect(
|
||||
page.items.find((one) => one.handle === handle)?.revokedAt,
|
||||
).toBeTypeOf('number');
|
||||
@@ -632,7 +624,7 @@ describe('an app managing what it minted', () => {
|
||||
);
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(appActor, handle),
|
||||
events().revokeKvHandle(appActor, handle),
|
||||
).rejects.toMatchObject({ legacyCode: 'events_kv_handle_owner_only' });
|
||||
});
|
||||
|
||||
@@ -655,7 +647,7 @@ describe('an app managing what it minted', () => {
|
||||
});
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(siblingActor, handle),
|
||||
events().revokeKvHandle(siblingActor, handle),
|
||||
).rejects.toMatchObject({ legacyCode: 'events_kv_handle_owner_only' });
|
||||
});
|
||||
|
||||
@@ -673,7 +665,7 @@ describe('an app managing what it minted', () => {
|
||||
});
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(tokenActor, handle),
|
||||
events().revokeKvHandle(tokenActor, handle),
|
||||
).rejects.toMatchObject({ legacyCode: 'events_kv_handle_owner_only' });
|
||||
});
|
||||
|
||||
@@ -687,12 +679,12 @@ describe('an app managing what it minted', () => {
|
||||
await undelegate(prefix);
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(appActor, handle),
|
||||
events().revokeKvHandle(appActor, handle),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'events_kv_handle_not_delegated',
|
||||
});
|
||||
|
||||
const page = await shares().listKvHandles(owner.actor, {
|
||||
const page = await events().listKvHandles(owner.actor, {
|
||||
limit: 100,
|
||||
});
|
||||
expect(
|
||||
@@ -706,7 +698,7 @@ describe('an app managing what it minted', () => {
|
||||
await mint({ appUid: 'os-global', prefix: sessionPrefix }, owner.actor);
|
||||
await mint({ prefix: appPrefix });
|
||||
|
||||
const appPage = await shares().listKvHandles(appActor, {
|
||||
const appPage = await events().listKvHandles(appActor, {
|
||||
limit: 100,
|
||||
includeTotal: true,
|
||||
});
|
||||
@@ -716,7 +708,7 @@ describe('an app managing what it minted', () => {
|
||||
false,
|
||||
);
|
||||
|
||||
const ownerPage = await shares().listKvHandles(owner.actor, {
|
||||
const ownerPage = await events().listKvHandles(owner.actor, {
|
||||
limit: 100,
|
||||
includeTotal: true,
|
||||
});
|
||||
@@ -729,7 +721,7 @@ describe('an app managing what it minted', () => {
|
||||
const key = `rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${owner.id}`;
|
||||
|
||||
const before = await env.server.clients.redis.zcard(key);
|
||||
await shares().revokeKvHandle(appActor, handle);
|
||||
await events().revokeKvHandle(appActor, handle);
|
||||
const after = await env.server.clients.redis.zcard(key);
|
||||
|
||||
expect(after).toBe(before + 1);
|
||||
@@ -737,14 +729,14 @@ describe('an app managing what it minted', () => {
|
||||
|
||||
it('answers another user`s handle in the same namespace as absent, not as forbidden', async () => {
|
||||
const prefix = `cross-user-${uuidv4().slice(0, 8)}:`;
|
||||
const { handle } = await shares().mintKvHandle(
|
||||
guest.actor,
|
||||
{ recipient: { username: owner.username }, appUid, prefix },
|
||||
{ defaultAccess: 'watch' },
|
||||
);
|
||||
const { handle } = await events().mintKvHandle(guest.actor, {
|
||||
granteeUsername: owner.username,
|
||||
appUid,
|
||||
prefix,
|
||||
});
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(appActor, handle),
|
||||
events().revokeKvHandle(appActor, handle),
|
||||
).rejects.toMatchObject({ legacyCode: 'subject_does_not_exist' });
|
||||
});
|
||||
|
||||
@@ -752,7 +744,7 @@ describe('an app managing what it minted', () => {
|
||||
const prefix = `${PREFIX}revoke-audit-probe-${uuidv4().slice(0, 8)}:`;
|
||||
const { handle } = await mint({ prefix });
|
||||
|
||||
await shares().revokeKvHandle(appActor, handle);
|
||||
await events().revokeKvHandle(appActor, handle);
|
||||
|
||||
const row = await vi.waitFor(async () => {
|
||||
const [found] = (await env.server.clients.db.pread(
|
||||
|
||||
@@ -60,7 +60,6 @@ let stranger: TestUser;
|
||||
let delivered: DeliveryEnvelope[];
|
||||
|
||||
const events = () => env.server.services.events;
|
||||
const shares = () => env.server.services.kvShare;
|
||||
|
||||
const settled = (count = 1) =>
|
||||
vi.waitFor(() => expect(delivered.length).toBeGreaterThanOrEqual(count), {
|
||||
@@ -89,11 +88,10 @@ const ownerWrites = (key: string, value: unknown): Promise<unknown> =>
|
||||
);
|
||||
|
||||
const mint = (prefix = PREFIX) =>
|
||||
shares().mintKvHandle(
|
||||
owner.actor,
|
||||
{ prefix, recipient: { username: guest.username } },
|
||||
{ defaultAccess: 'watch' },
|
||||
);
|
||||
events().mintKvHandle(owner.actor, {
|
||||
granteeUsername: guest.username,
|
||||
prefix,
|
||||
});
|
||||
|
||||
/** A durable `single` row, which is the only kind that holds a backlog. */
|
||||
const subscribeDurable = async (handle: string) =>
|
||||
@@ -184,7 +182,7 @@ describe('revoking a handle', () => {
|
||||
);
|
||||
expect(await ownerWatches()).toBe(true);
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
await suspendedRow(sub.subId);
|
||||
|
||||
// The anchor leaves the *owner's* watched set, which is the keyspace a
|
||||
@@ -213,7 +211,7 @@ describe('revoking a handle', () => {
|
||||
await settled();
|
||||
expect(delivered[0].subId).toBe(sub.subId);
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
delivered.length = 0;
|
||||
|
||||
await ownerWrites(`${PREFIX}live:2`, 2);
|
||||
@@ -231,21 +229,16 @@ describe('revoking a handle', () => {
|
||||
const settle = vi.spyOn(events(), 'settleRevokedGrant');
|
||||
|
||||
try {
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
await suspendedRow(sub.subId);
|
||||
|
||||
// One grant settle for the holder, and a retired-handle settle
|
||||
// for each side, each a single indexed read of that user's rows —
|
||||
// no lookup keyed on the handle anywhere.
|
||||
// One announcement, one settle pass, one indexed read of the
|
||||
// holder's rows — no lookup keyed on the handle anywhere.
|
||||
expect(settle).toHaveBeenCalledTimes(1);
|
||||
expect(
|
||||
byHolder.mock.calls.map(([userId]) => userId).sort(),
|
||||
).toEqual([guest.id, guest.id, owner.id].sort());
|
||||
for (const call of byHolder.mock.calls)
|
||||
expect(call.slice(1)).toEqual([
|
||||
null,
|
||||
{ includeSuspended: true },
|
||||
]);
|
||||
expect(byHolder).toHaveBeenCalledTimes(1);
|
||||
expect(byHolder).toHaveBeenCalledWith(guest.id, null, {
|
||||
includeSuspended: true,
|
||||
});
|
||||
} finally {
|
||||
byHolder.mockRestore();
|
||||
settle.mockRestore();
|
||||
@@ -257,7 +250,7 @@ describe('revoking a handle', () => {
|
||||
const { handle } = await mint();
|
||||
const deeper = await mint(`${PREFIX}messages:`);
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
|
||||
// The subtree goes with the region: a grant left underneath would keep
|
||||
// access to part of what was just taken back.
|
||||
@@ -278,12 +271,12 @@ describe('revoking a handle', () => {
|
||||
const deeper = await mint(`${PREFIX}messages:`);
|
||||
expect((await countLive()).total).toBe(before.total + 2);
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
|
||||
// The deeper handle's grant died with the subtree revoke above (proven
|
||||
// by the sibling case), so its row reading live would disagree with
|
||||
// reality — both in the listing and in the slot it appears to hold.
|
||||
const page = await shares().listKvHandles(owner.actor, {
|
||||
const page = await events().listKvHandles(owner.actor, {
|
||||
limit: 100,
|
||||
});
|
||||
const deeperRow = page.items.find((one) => one.handle === deeper.handle);
|
||||
@@ -296,9 +289,9 @@ describe('revoking a handle', () => {
|
||||
const { handle } = await mint();
|
||||
const sibling = await mint('workspace:other:');
|
||||
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
|
||||
const page = await shares().listKvHandles(owner.actor, {
|
||||
const page = await events().listKvHandles(owner.actor, {
|
||||
limit: 100,
|
||||
});
|
||||
const siblingRow = page.items.find(
|
||||
@@ -310,7 +303,7 @@ describe('revoking a handle', () => {
|
||||
it('lets a fresh handle over the same region work, and leaves the old one dead', async () => {
|
||||
await clearRows();
|
||||
const first = await mint();
|
||||
await shares().revokeKvHandle(owner.actor, first.handle);
|
||||
await events().revokeKvHandle(owner.actor, first.handle);
|
||||
|
||||
const second = await mint();
|
||||
expect(second.handle).not.toBe(first.handle);
|
||||
@@ -341,7 +334,7 @@ describe('revoking a handle', () => {
|
||||
: owner.actor;
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(actor, handle),
|
||||
events().revokeKvHandle(actor, handle),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'subject_does_not_exist',
|
||||
});
|
||||
@@ -352,8 +345,8 @@ describe('revoking a handle', () => {
|
||||
// between them leaves a retry to finish the job — which only works if
|
||||
// reaching a retired handle again succeeds.
|
||||
const { handle } = await mint();
|
||||
const first = await shares().revokeKvHandle(owner.actor, handle);
|
||||
const again = await shares().revokeKvHandle(owner.actor, handle);
|
||||
const first = await events().revokeKvHandle(owner.actor, handle);
|
||||
const again = await events().revokeKvHandle(owner.actor, handle);
|
||||
expect(again).toEqual(first);
|
||||
});
|
||||
|
||||
@@ -372,7 +365,7 @@ describe('revoking a handle', () => {
|
||||
.spyOn(store, 'retire')
|
||||
.mockRejectedValueOnce(new Error('write failed'));
|
||||
await expect(
|
||||
shares().revokeKvHandle(owner.actor, handle),
|
||||
events().revokeKvHandle(owner.actor, handle),
|
||||
).rejects.toThrow('write failed');
|
||||
failing.mockRestore();
|
||||
|
||||
@@ -384,7 +377,7 @@ describe('revoking a handle', () => {
|
||||
expect(delivered).toEqual([]);
|
||||
|
||||
// And the retry converges rather than answering "no such handle".
|
||||
const revoked = await shares().revokeKvHandle(owner.actor, handle);
|
||||
const revoked = await events().revokeKvHandle(owner.actor, handle);
|
||||
expect(revoked.handle).toBe(handle);
|
||||
expect(await retire(handle, owner.id)).toMatchObject({
|
||||
revokedAt: revoked.revokedAt,
|
||||
@@ -405,7 +398,7 @@ describe('revoking a handle', () => {
|
||||
});
|
||||
|
||||
await expect(
|
||||
shares().revokeKvHandle(appActor, handle),
|
||||
events().revokeKvHandle(appActor, handle),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'events_kv_handle_owner_only',
|
||||
});
|
||||
@@ -415,9 +408,9 @@ describe('revoking a handle', () => {
|
||||
describe('the owner`s handle listing', () => {
|
||||
it('shows what was shared, retired handles included', async () => {
|
||||
const { handle } = await mint();
|
||||
await shares().revokeKvHandle(owner.actor, handle);
|
||||
await events().revokeKvHandle(owner.actor, handle);
|
||||
|
||||
const page = await shares().listKvHandles(owner.actor, {
|
||||
const page = await events().listKvHandles(owner.actor, {
|
||||
includeTotal: true,
|
||||
});
|
||||
const row = page.items.find((one) => one.handle === handle);
|
||||
@@ -425,9 +418,7 @@ describe('the owner`s handle listing', () => {
|
||||
expect(row).toMatchObject({
|
||||
prefix: PREFIX,
|
||||
appUid: 'os-global',
|
||||
holderUsername: guest.username,
|
||||
access: 'watch',
|
||||
revokedBy: 'owner',
|
||||
granteeUsername: guest.username,
|
||||
});
|
||||
expect(row?.revokedAt).toBeTypeOf('number');
|
||||
expect(page.total).toBe(page.items.length + (page.cursor ? 1 : 0));
|
||||
@@ -435,7 +426,7 @@ describe('the owner`s handle listing', () => {
|
||||
|
||||
it('shows the grantee nothing of the owner`s', async () => {
|
||||
await mint();
|
||||
const page = await shares().listKvHandles(guest.actor);
|
||||
const page = await events().listKvHandles(guest.actor);
|
||||
expect(page.items).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -443,11 +434,11 @@ describe('the owner`s handle listing', () => {
|
||||
await mint();
|
||||
await mint();
|
||||
|
||||
const first = await shares().listKvHandles(owner.actor, { limit: 1 });
|
||||
const first = await events().listKvHandles(owner.actor, { limit: 1 });
|
||||
expect(first.items).toHaveLength(1);
|
||||
expect(first.cursor).toBeTypeOf('string');
|
||||
|
||||
const second = await shares().listKvHandles(owner.actor, {
|
||||
const second = await events().listKvHandles(owner.actor, {
|
||||
limit: 1,
|
||||
cursor: first.cursor,
|
||||
});
|
||||
|
||||
@@ -46,7 +46,7 @@ import {
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { DEFAULT_FREE_SUBSCRIPTION } from '../metering/consts.js';
|
||||
import type { DeliveryEnvelope } from './EventsService.js';
|
||||
import { kvSharePermission } from '../kvShare/kvShares.js';
|
||||
import { kvSharePermission } from './kvShares.js';
|
||||
import { isKvHandleId, kvAnchorToken } from './subjects.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
@@ -68,7 +68,6 @@ let stranger: TestUser;
|
||||
let delivered: DeliveryEnvelope[];
|
||||
|
||||
const events = () => env.server.services.events;
|
||||
const shares = () => env.server.services.kvShare;
|
||||
|
||||
const settled = (count = 1) =>
|
||||
vi.waitFor(() => expect(delivered.length).toBeGreaterThanOrEqual(count), {
|
||||
@@ -98,34 +97,14 @@ const ownerWrites = (key: string, value: unknown): Promise<unknown> =>
|
||||
);
|
||||
|
||||
const mint = (
|
||||
{
|
||||
granteeUsername = guest.username,
|
||||
...request
|
||||
}: Record<string, unknown> = {},
|
||||
request: Record<string, unknown> = {},
|
||||
actor: Actor = owner.actor,
|
||||
) =>
|
||||
shares().mintKvHandle(
|
||||
actor,
|
||||
{
|
||||
prefix: PREFIX,
|
||||
...request,
|
||||
recipient: { username: granteeUsername as string },
|
||||
},
|
||||
{ defaultAccess: 'watch' },
|
||||
);
|
||||
|
||||
/** An app row under a fresh uid, since a share may only name an app that exists. */
|
||||
const registerApp = async (
|
||||
on: PuterTestEnv = env,
|
||||
ownerUserId: number = owner.id,
|
||||
): Promise<string> => {
|
||||
const uid = `app-${uuidv4()}`;
|
||||
await on.server.clients.db.write(
|
||||
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
|
||||
[uid, uid, uid, `https://${uid}.example/`, ownerUserId],
|
||||
);
|
||||
return uid;
|
||||
};
|
||||
events().mintKvHandle(actor, {
|
||||
granteeUsername: guest.username,
|
||||
prefix: PREFIX,
|
||||
...request,
|
||||
});
|
||||
|
||||
const clearRows = async () => {
|
||||
await env.server.clients.db.write(`DELETE FROM \`${TABLE}\``, []);
|
||||
@@ -169,12 +148,7 @@ describe('minting a handle', () => {
|
||||
const minted = await mint();
|
||||
|
||||
expect(isKvHandleId(minted.handle)).toBe(true);
|
||||
expect(minted).toMatchObject({
|
||||
handle: minted.handle,
|
||||
prefix: PREFIX,
|
||||
access: 'watch',
|
||||
holderUsername: guest.username,
|
||||
});
|
||||
expect(minted).toEqual({ handle: minted.handle, prefix: PREFIX });
|
||||
const wire = JSON.stringify(minted);
|
||||
expect(wire).not.toContain(owner.uuid);
|
||||
expect(wire).not.toContain(owner.username);
|
||||
@@ -325,8 +299,7 @@ describe('minting a handle', () => {
|
||||
});
|
||||
|
||||
it('lets a paid namespace fill the account cap and then refuses other namespaces', async () => {
|
||||
const namespace = await registerApp();
|
||||
const another = await registerApp();
|
||||
const namespace = 'app-perapp-probe';
|
||||
try {
|
||||
for (let i = 0; i < EVENTS_KV_HANDLES_PER_APP.limit; i++) {
|
||||
const prefix = `perapp:${i}:`;
|
||||
@@ -345,20 +318,20 @@ describe('minting a handle', () => {
|
||||
legacyCode: 'events_kv_handle_limit_reached',
|
||||
});
|
||||
await expect(
|
||||
mint({ appUid: another, prefix: 'still:room:' }),
|
||||
mint({ appUid: 'app-another-probe', prefix: 'still:room:' }),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'events_kv_handle_limit_reached',
|
||||
});
|
||||
} finally {
|
||||
await env.server.clients.db.write(
|
||||
'DELETE FROM `kv_share_handles` WHERE `owner_user_id` = ? AND `app_uid` IN (?, ?)',
|
||||
[owner.id, namespace, another],
|
||||
[owner.id, namespace, 'app-another-probe'],
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('hands back an existing handle at the cap, taking no slot', async () => {
|
||||
const namespace = await registerApp();
|
||||
const namespace = 'app-idempotent-probe';
|
||||
const prefix = 'idempotent:cap:';
|
||||
try {
|
||||
const first = await mint({ appUid: namespace, prefix });
|
||||
@@ -776,7 +749,7 @@ describe('what a plan lets an account hold', () => {
|
||||
|
||||
it('holds a free account to the free per-app cap, with room left in the account cap', async () => {
|
||||
await setEmail(`${tiered.users.user.username}@example.invalid`);
|
||||
const namespace = await registerApp(tiered, tieredUserId);
|
||||
const namespace = 'app-tiered-free-probe';
|
||||
const cap =
|
||||
EVENTS_KV_HANDLES_PER_APP.bySubscription[DEFAULT_FREE_SUBSCRIPTION];
|
||||
const grantee = await tiered.server.stores.user.getByUsername(
|
||||
@@ -810,7 +783,7 @@ describe('what a plan lets an account hold', () => {
|
||||
// another namespace still succeeds.
|
||||
const accepted = await mintTiered(
|
||||
'still:room:',
|
||||
await registerApp(tiered, tieredUserId),
|
||||
'app-tiered-free-other',
|
||||
);
|
||||
expect(accepted.status).toBe(200);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,451 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { PermissionUtil } from '../permission/permissionUtil.js';
|
||||
import {
|
||||
assertBoundedManageGrant,
|
||||
assertShareableAppUid,
|
||||
assertShareablePermission,
|
||||
assertShareablePrefix,
|
||||
keyPrefixSegments,
|
||||
kvShareAppDelegateImplicator,
|
||||
kvShareGrantCovers,
|
||||
kvShareManagePermission,
|
||||
kvShareOwnerImplicator,
|
||||
kvSharePermission,
|
||||
mintKvHandleId,
|
||||
normalizeKeyPrefix,
|
||||
relativeToKvShareRoot,
|
||||
} from './kvShares.js';
|
||||
import { isKvHandleId, kvHandleFromSubject } from './subjects.js';
|
||||
|
||||
const OWNER = '2a1b0c9d-0000-4000-8000-000000000001';
|
||||
const OTHER = '2a1b0c9d-0000-4000-8000-000000000002';
|
||||
const APP = 'app-1234';
|
||||
const OTHER_APP = 'app-5678';
|
||||
|
||||
const userActor = (uuid: string): Actor =>
|
||||
({ user: { uuid }, effectiveApp: null }) as unknown as Actor;
|
||||
|
||||
const appActor = (uuid: string, appUid: string): Actor =>
|
||||
({
|
||||
user: { uuid },
|
||||
app: { uid: appUid },
|
||||
effectiveApp: { uid: appUid },
|
||||
}) as unknown as Actor;
|
||||
|
||||
describe('the share permission family', () => {
|
||||
it('names the owner, the app and the granted prefix', () => {
|
||||
expect(kvSharePermission(OWNER, APP, 'workspace:abc:')).toBe(
|
||||
`kv-share:${OWNER}:${APP}:workspace:abc`,
|
||||
);
|
||||
});
|
||||
|
||||
it('makes key segments components, so a deeper key is a descendant', () => {
|
||||
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
const deeper = kvSharePermission(
|
||||
OWNER,
|
||||
APP,
|
||||
'workspace:abc:messages:1',
|
||||
);
|
||||
|
||||
// The exploder walks parents of the string being checked, so implication
|
||||
// over key segments needs nothing of its own.
|
||||
expect(deeper.startsWith(`${granted}:`)).toBe(true);
|
||||
});
|
||||
|
||||
it('escapes a `:` inside a component rather than splitting on it', () => {
|
||||
const permission = kvSharePermission(OWNER, 'app:odd', 'cart:');
|
||||
expect(PermissionUtil.split(permission)).toEqual([
|
||||
'kv-share',
|
||||
OWNER,
|
||||
'app:odd',
|
||||
'cart',
|
||||
]);
|
||||
});
|
||||
|
||||
it('reads a withdrawn grant as covering itself and everything under it', () => {
|
||||
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
expect(kvShareGrantCovers(granted, granted)).toBe(true);
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:messages:'),
|
||||
),
|
||||
).toBe(true);
|
||||
// A sibling that merely shares a text prefix is a different region.
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OWNER, APP, 'workspace:abcdef:'),
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OTHER, APP, 'workspace:abc:'),
|
||||
),
|
||||
).toBe(false);
|
||||
// Coverage never crosses namespaces: the app uid is what keeps a
|
||||
// revoke inside the one the handle was minted in.
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OWNER, OTHER_APP, 'workspace:abc:messages:'),
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('never reads a withdrawn child as covering its own parent', () => {
|
||||
// Revoking the deeper of two grants must not read as covering the
|
||||
// shallower one — coverage only ever runs downward.
|
||||
const child = kvSharePermission(OWNER, APP, 'workspace:abc:messages:');
|
||||
const parent = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
expect(kvShareGrantCovers(child, parent)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('granted prefixes', () => {
|
||||
it('always ends on the key delimiter', () => {
|
||||
expect(normalizeKeyPrefix('workspace:abc')).toBe('workspace:abc:');
|
||||
expect(normalizeKeyPrefix('workspace:abc:')).toBe('workspace:abc:');
|
||||
expect(keyPrefixSegments('workspace:abc:')).toEqual([
|
||||
'workspace',
|
||||
'abc',
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['', 'the whole namespace'],
|
||||
[':', 'delimiters only'],
|
||||
['workspace:*', 'a pattern'],
|
||||
['workspace:a?c', 'a single-character pattern'],
|
||||
// Normalizing would drop the empty segment and grant `a:b:` instead —
|
||||
// a region other than the one asked for.
|
||||
['workspace::abc:', 'an empty key segment'],
|
||||
[':workspace:abc:', 'a leading empty segment'],
|
||||
// `escape_permission_component` leaves `\\` alone while escaping `:`,
|
||||
// so the stored prefix and the permission string would disagree and
|
||||
// the share would silently never fire.
|
||||
['work\\space:abc:', 'a backslash'],
|
||||
])('refuses %s (%s)', (prefix) => {
|
||||
let thrown: unknown;
|
||||
try {
|
||||
assertShareablePrefix(prefix);
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(HttpError);
|
||||
expect((thrown as HttpError).legacyCode).toBe(
|
||||
'invalid_kv_share_prefix',
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses a region too deep to fit the grant column', () => {
|
||||
const deep = kvSharePermission(OWNER, APP, 'a'.repeat(300));
|
||||
expect(() => assertShareablePermission(deep)).toThrow(HttpError);
|
||||
expect(
|
||||
assertShareablePermission(
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:'),
|
||||
),
|
||||
).toBeTypeOf('string');
|
||||
});
|
||||
|
||||
it('keeps the trailing delimiter optional', () => {
|
||||
expect(assertShareablePrefix('workspace:abc')).toBe('workspace:abc:');
|
||||
expect(assertShareablePrefix('workspace:abc:')).toBe('workspace:abc:');
|
||||
});
|
||||
});
|
||||
|
||||
describe('granted namespaces', () => {
|
||||
it('takes an app uid, and the app-less namespace', () => {
|
||||
expect(assertShareableAppUid(APP)).toBe(APP);
|
||||
expect(assertShareableAppUid('os-global')).toBe('os-global');
|
||||
});
|
||||
|
||||
it('refuses one past the column it is stored in', () => {
|
||||
// A longer one reaches the insert as a write the column cannot hold.
|
||||
let thrown: unknown;
|
||||
try {
|
||||
assertShareableAppUid('a'.repeat(41));
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(HttpError);
|
||||
expect((thrown as HttpError).statusCode).toBe(400);
|
||||
});
|
||||
|
||||
it('takes both shapes a real app uid comes in', () => {
|
||||
// Minted (`app-<uuidv4>`) and derived from an origin (`app-<uuidv5>`)
|
||||
// — 40 characters either way, which is the column exactly.
|
||||
const minted = `app-${uuidv4()}`;
|
||||
expect(assertShareableAppUid(minted)).toBe(minted);
|
||||
expect(minted).toHaveLength(40);
|
||||
});
|
||||
|
||||
it('refuses a handle, which shares the slot but is not a namespace', () => {
|
||||
expect(() => assertShareableAppUid(mintKvHandleId())).toThrow(
|
||||
HttpError,
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses anything else short enough to fit the column', () => {
|
||||
// The slot ends up in a subject and in a permission string, so a
|
||||
// namespace carrying a delimiter of either grammar is refused before
|
||||
// it can be composed into one.
|
||||
for (const uid of [
|
||||
'os-globals',
|
||||
'workspace:abc',
|
||||
'app',
|
||||
'app-',
|
||||
'app-a:b',
|
||||
'app-a#b',
|
||||
'v1:someone:app-x',
|
||||
'',
|
||||
])
|
||||
expect(() => assertShareableAppUid(uid)).toThrow(HttpError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('keys inside a granted region', () => {
|
||||
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
|
||||
it('are named relative to the granted root', () => {
|
||||
expect(
|
||||
relativeToKvShareRoot(permission, 'workspace:abc:messages:1'),
|
||||
).toBe('messages:1');
|
||||
});
|
||||
|
||||
it('read as outside when they are', () => {
|
||||
expect(
|
||||
relativeToKvShareRoot(permission, 'workspace:abcdef:x'),
|
||||
).toBeNull();
|
||||
expect(relativeToKvShareRoot(permission, 'workspace:abc')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('a delegation request', () => {
|
||||
const region = kvShareManagePermission(
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:'),
|
||||
);
|
||||
|
||||
it('is the manage arm of the grant it would let an app issue', () => {
|
||||
expect(region).toBe(`manage:kv-share:${OWNER}:${APP}:workspace:abc`);
|
||||
expect(assertBoundedManageGrant(region)).toBe(region);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['manage:kv-share', 'the family itself'],
|
||||
[`manage:kv-share:${OWNER}`, 'an owner and no namespace'],
|
||||
[`manage:kv-share:${OWNER}:${APP}`, 'a whole namespace'],
|
||||
])('refuses %s (%s)', (permission) => {
|
||||
let thrown: unknown;
|
||||
try {
|
||||
assertBoundedManageGrant(permission);
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(HttpError);
|
||||
expect((thrown as HttpError).legacyCode).toBe(
|
||||
'invalid_kv_share_prefix',
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves permissions from other families alone', () => {
|
||||
const unrelated = `manage:fs:${OWNER}:write`;
|
||||
expect(assertBoundedManageGrant(unrelated)).toBe(unrelated);
|
||||
// The read arm of this family is not a delegation, so it is not this
|
||||
// check's to bound.
|
||||
expect(assertBoundedManageGrant(`kv-share:${OWNER}:${APP}`)).toBeTypeOf(
|
||||
'string',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('handle ids', () => {
|
||||
it('are told apart from an app uid in the same slot', () => {
|
||||
const handle = mintKvHandleId();
|
||||
expect(isKvHandleId(handle)).toBe(true);
|
||||
expect(isKvHandleId(APP)).toBe(false);
|
||||
expect(isKvHandleId('kvh-')).toBe(false);
|
||||
});
|
||||
|
||||
it('carry nothing about the owner', () => {
|
||||
const handle = mintKvHandleId();
|
||||
expect(handle).not.toContain(OWNER);
|
||||
expect(mintKvHandleId()).not.toBe(handle);
|
||||
});
|
||||
|
||||
it('are recoverable from the subject a row stored', () => {
|
||||
const handle = mintKvHandleId();
|
||||
expect(kvHandleFromSubject(`kv:${handle}:messages:*`)).toBe(handle);
|
||||
expect(kvHandleFromSubject(`kv:${APP}:messages:*`)).toBeNull();
|
||||
expect(kvHandleFromSubject(`fs:${handle}:write`)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('the owner implicator', () => {
|
||||
const implicator = kvShareOwnerImplicator();
|
||||
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
|
||||
it('answers the share family and its manage arm', () => {
|
||||
expect(implicator.matches(permission)).toBe(true);
|
||||
expect(implicator.matches(`manage:${permission}`)).toBe(true);
|
||||
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
|
||||
});
|
||||
|
||||
it('holds for the owner named in the permission, and nobody else', () => {
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OWNER), permission }),
|
||||
).toEqual({});
|
||||
expect(
|
||||
implicator.check({
|
||||
actor: userActor(OWNER),
|
||||
permission: `manage:${permission}`,
|
||||
}),
|
||||
).toEqual({});
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OTHER), permission }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('reads the owner past a `manage` key segment', () => {
|
||||
// Stripping every `manage:` rather than the leading one would rewrite
|
||||
// the string the owner is read out of.
|
||||
const nested = kvSharePermission(OWNER, APP, 'manage:secrets:');
|
||||
expect(implicator.matches(nested)).toBe(true);
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OWNER), permission: nested }),
|
||||
).toEqual({});
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OTHER), permission: nested }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('never answers for an app or a token acting through the owner', () => {
|
||||
const appActor = {
|
||||
user: { uuid: OWNER },
|
||||
app: { uid: APP },
|
||||
} as unknown as Actor;
|
||||
expect(
|
||||
implicator.check({ actor: appActor, permission }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('the app delegate implicator', () => {
|
||||
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
|
||||
it('answers the share family, never its manage arm, nor other families', () => {
|
||||
const implicator = kvShareAppDelegateImplicator({
|
||||
userHolds: vi.fn(),
|
||||
});
|
||||
expect(implicator.matches(permission)).toBe(true);
|
||||
expect(implicator.matches(`manage:${permission}`)).toBe(false);
|
||||
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
|
||||
});
|
||||
|
||||
it("grants when the actor's own app matches the namespace app and the user holds the grant", async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = appActor(OWNER, APP);
|
||||
|
||||
await expect(implicator.check({ actor, permission })).resolves.toEqual(
|
||||
{},
|
||||
);
|
||||
|
||||
expect(userHolds).toHaveBeenCalledTimes(1);
|
||||
const [calledActor, calledPermission] = userHolds.mock.calls[0];
|
||||
expect(calledActor.app).toBeUndefined();
|
||||
expect(calledActor.effectiveApp).toBeNull();
|
||||
expect(calledActor.user).toEqual({ uuid: OWNER });
|
||||
expect(calledPermission).toBe(permission);
|
||||
});
|
||||
|
||||
it('refuses when the user does not hold the grant', async () => {
|
||||
const implicator = kvShareAppDelegateImplicator({
|
||||
userHolds: vi.fn().mockResolvedValue(false),
|
||||
});
|
||||
await expect(
|
||||
implicator.check({ actor: appActor(OWNER, APP), permission }),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("refuses when the actor's app differs from the namespace app", async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
|
||||
await expect(
|
||||
implicator.check({
|
||||
actor: appActor(OWNER, OTHER_APP),
|
||||
permission,
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses a plain user actor with no app', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
|
||||
await expect(
|
||||
implicator.check({ actor: userActor(OWNER), permission }),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses an access-token actor', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = {
|
||||
user: { uuid: OWNER },
|
||||
app: { uid: APP },
|
||||
accessToken: { uid: 't' },
|
||||
} as unknown as Actor;
|
||||
|
||||
await expect(
|
||||
implicator.check({ actor, permission }),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses a permission with no key segments or no app', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = appActor(OWNER, APP);
|
||||
|
||||
await expect(
|
||||
implicator.check({
|
||||
actor,
|
||||
permission: PermissionUtil.join('kv-share', OWNER, APP),
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
await expect(
|
||||
implicator.check({
|
||||
actor,
|
||||
permission: PermissionUtil.join('kv-share', OWNER),
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,327 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import {
|
||||
type Actor,
|
||||
isAppActor,
|
||||
isPlainUserActor,
|
||||
userRelatedActor,
|
||||
} from '../../core/actor.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { KV_SHARE_HANDLE_WIDTHS } from '../../stores/events/columnWidths.js';
|
||||
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
|
||||
import {
|
||||
MANAGE_PERM_PREFIX,
|
||||
PERMISSION_MAX_LEN,
|
||||
} from '../permission/consts.js';
|
||||
import type { PermissionImplicator } from '../permission/permissionUtil.js';
|
||||
import { PermissionUtil } from '../permission/permissionUtil.js';
|
||||
import {
|
||||
isKvHandleId,
|
||||
KV_HANDLE_PREFIX,
|
||||
KV_KEY_SEPARATOR,
|
||||
} from './subjects.js';
|
||||
|
||||
/**
|
||||
* Letting one user watch a region of another's key-value namespace.
|
||||
*
|
||||
* The grant is an ordinary user-to-user permission, which is the whole point:
|
||||
* `manage:` answers "may I delegate this", the exploder's prefix implication
|
||||
* makes a grant on `workspace:abc:` answer every key beneath it, and revocation
|
||||
* settling and the delivery re-check key off the stored string exactly as they
|
||||
* do for a shared folder. Putting the ACL on the key-value item instead would
|
||||
* take part in none of that, and would tax every write to the item besides.
|
||||
*
|
||||
* The permission names the owner, so its components are the three things a
|
||||
* shared region is:
|
||||
*
|
||||
* kv-share:<ownerUserUuid>:<appUid>:<keySegment>...
|
||||
*
|
||||
* Key segments are components rather than one escaped blob because that is what
|
||||
* makes prefix implication work without a rule of its own: the check a deep key
|
||||
* runs is a descendant of the grant string, and the existing parent walk finds
|
||||
* it. The grantee never sees this string — the handle is the name on the wire.
|
||||
*/
|
||||
|
||||
/** Root of the cross-user key-value share namespace. */
|
||||
export const KV_SHARE_PERMISSION_PREFIX = 'kv-share';
|
||||
|
||||
/** Width of the `app_uid` column the handle row stores its namespace in. */
|
||||
export const KV_SHARE_APP_UID_MAX_LENGTH = KV_SHARE_HANDLE_WIDTHS.appUid.max;
|
||||
|
||||
export const mintKvHandleId = (): string =>
|
||||
`${KV_HANDLE_PREFIX}${randomUUID()}`;
|
||||
|
||||
/** The key segments a prefix contributes to a permission string. */
|
||||
export const keyPrefixSegments = (keyPrefix: string): string[] =>
|
||||
keyPrefix.split(KV_KEY_SEPARATOR).filter((segment) => segment.length > 0);
|
||||
|
||||
/**
|
||||
* A granted prefix as it is stored: always ending on the key delimiter, so
|
||||
* `keyPrefix + relative` is a whole key and the region never accidentally
|
||||
* includes the key that names it.
|
||||
*/
|
||||
export const normalizeKeyPrefix = (keyPrefix: string): string => {
|
||||
const segments = keyPrefixSegments(keyPrefix);
|
||||
return segments.length === 0
|
||||
? ''
|
||||
: `${segments.join(KV_KEY_SEPARATOR)}${KV_KEY_SEPARATOR}`;
|
||||
};
|
||||
|
||||
export const kvSharePermission = (
|
||||
ownerUserUuid: string,
|
||||
appUid: string,
|
||||
keyPrefix: string,
|
||||
): string =>
|
||||
PermissionUtil.join(
|
||||
KV_SHARE_PERMISSION_PREFIX,
|
||||
ownerUserUuid,
|
||||
appUid,
|
||||
...keyPrefixSegments(keyPrefix),
|
||||
);
|
||||
|
||||
export const isKvSharePermission = (permission: string): boolean =>
|
||||
permission === KV_SHARE_PERMISSION_PREFIX ||
|
||||
permission.startsWith(`${KV_SHARE_PERMISSION_PREFIX}:`);
|
||||
|
||||
/**
|
||||
* Whether withdrawing `revoked` takes `held` with it. Prefix implication read
|
||||
* backwards: a grant answers every check at or beneath it, so withdrawing it
|
||||
* puts every one of those in question.
|
||||
*/
|
||||
export const kvShareGrantCovers = (revoked: string, held: string): boolean =>
|
||||
held === revoked || held.startsWith(`${revoked}:`);
|
||||
|
||||
/** The granted root a share grant names, read back as a key prefix. */
|
||||
export const kvShareGrantPrefix = (permission: string): string =>
|
||||
normalizeKeyPrefix(
|
||||
PermissionUtil.split(permission).slice(3).join(KV_KEY_SEPARATOR),
|
||||
);
|
||||
|
||||
/**
|
||||
* A key inside a granted region, named the way its holder addresses it, or
|
||||
* `null` for one outside. The handle is the granted root, so everything the
|
||||
* holder is shown is relative to it — an absolute key would name a namespace
|
||||
* they cannot address and were never told about.
|
||||
*/
|
||||
export const relativeToKvShareRoot = (
|
||||
permission: string,
|
||||
key: string,
|
||||
): string | null => {
|
||||
const prefix = kvShareGrantPrefix(permission);
|
||||
if (!prefix || !key.startsWith(prefix)) return null;
|
||||
return key.slice(prefix.length);
|
||||
};
|
||||
|
||||
// -- Minting input ----------------------------------------------------
|
||||
|
||||
export const invalidPrefix = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'invalid_kv_share_prefix' });
|
||||
|
||||
export const invalidAppUid = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'bad_request' });
|
||||
|
||||
/**
|
||||
* The region a handle may be minted on. A pattern is refused because a handle
|
||||
* names a region rather than selecting within one, and the namespace root is
|
||||
* refused because a handle over everything the app holds is not a bounded
|
||||
* capability — it is the app's data, and sharing that is a different decision
|
||||
* with a different consent.
|
||||
*/
|
||||
export const assertShareablePrefix = (keyPrefix: unknown): string => {
|
||||
if (typeof keyPrefix !== 'string')
|
||||
throw invalidPrefix('`prefix` must be a string');
|
||||
if (keyPrefix.includes('*') || keyPrefix.includes('?'))
|
||||
throw invalidPrefix('A share prefix is a key prefix, not a pattern');
|
||||
// `escape_permission_component` escapes `:` as `\C` but leaves `\` alone,
|
||||
// so a prefix carrying one desynchronizes the stored `key_prefix` from the
|
||||
// permission string and the share silently never fires.
|
||||
if (keyPrefix.includes('\\'))
|
||||
throw invalidPrefix('A share prefix may not contain a backslash');
|
||||
// Normalizing drops empty segments, so `a::b:` would silently become a
|
||||
// grant on `a:b:` — a region other than the one asked for. Refused rather
|
||||
// than rewritten; only the trailing delimiter is optional.
|
||||
const written = keyPrefix.split(KV_KEY_SEPARATOR);
|
||||
if (written[written.length - 1] === '') written.pop();
|
||||
if (written.some((segment) => segment === ''))
|
||||
throw invalidPrefix('A share prefix may not have an empty key segment');
|
||||
|
||||
const normalized = normalizeKeyPrefix(keyPrefix);
|
||||
if (normalized === '')
|
||||
throw invalidPrefix('A share prefix may not be the whole namespace');
|
||||
return normalized;
|
||||
};
|
||||
|
||||
/**
|
||||
* The shape every app uid takes, minted or derived from an origin. Narrow on
|
||||
* purpose: `:` and `#` delimit the anchor token and permission grammars, so
|
||||
* excluding them matters as much as excluding the handle prefix does.
|
||||
*/
|
||||
const APP_UID_SHAPE = /^app-[A-Za-z0-9_-]{1,64}$/;
|
||||
|
||||
/** Fixed namespaces with no app of their own. */
|
||||
const FIXED_KV_NAMESPACES: readonly string[] = [KV_GLOBAL_APP_KEY];
|
||||
|
||||
/**
|
||||
* The namespace a handle may be minted over: the caller's own app slot, or the
|
||||
* app-less one. Bounded by the column that stores it, shaped like a real app
|
||||
* uid or one of the platform's fixed namespaces, and never handle-shaped, since
|
||||
* the two share the app slot of a `kv:` subject.
|
||||
*/
|
||||
export const assertShareableAppUid = (appUid: string): string => {
|
||||
if (appUid.length > KV_SHARE_APP_UID_MAX_LENGTH)
|
||||
throw invalidAppUid(
|
||||
`\`appUid\` may not exceed ${KV_SHARE_APP_UID_MAX_LENGTH} characters`,
|
||||
);
|
||||
if (isKvHandleId(appUid))
|
||||
throw invalidAppUid('A share handle does not name a namespace');
|
||||
if (!APP_UID_SHAPE.test(appUid) && !FIXED_KV_NAMESPACES.includes(appUid))
|
||||
throw invalidAppUid('`appUid` does not name a namespace');
|
||||
return appUid;
|
||||
};
|
||||
|
||||
/**
|
||||
* The grant a mint would issue, refused when it would not fit the column every
|
||||
* permission table declares. Key prefixes run to a kilobyte and permissions do
|
||||
* not, so the real bound on how deep a region may be is this one.
|
||||
*/
|
||||
export const assertShareablePermission = (permission: string): string => {
|
||||
if (permission.length > PERMISSION_MAX_LEN)
|
||||
throw invalidPrefix(
|
||||
`A share prefix must leave the grant under ${PERMISSION_MAX_LEN} characters`,
|
||||
);
|
||||
return permission;
|
||||
};
|
||||
|
||||
// -- Delegation -------------------------------------------------------
|
||||
|
||||
/** The delegation arm of a share grant: authority to hand the region out. */
|
||||
export const kvShareManagePermission = (permission: string): string =>
|
||||
PermissionUtil.join(
|
||||
MANAGE_PERM_PREFIX,
|
||||
...PermissionUtil.split(permission),
|
||||
);
|
||||
|
||||
export const KV_SHARE_MANAGE_PREFIX = `${MANAGE_PERM_PREFIX}:${KV_SHARE_PERMISSION_PREFIX}`;
|
||||
|
||||
export const isKvShareManagePermission = (permission: string): boolean =>
|
||||
permission === KV_SHARE_MANAGE_PREFIX ||
|
||||
permission.startsWith(`${KV_SHARE_MANAGE_PREFIX}:`);
|
||||
|
||||
/**
|
||||
* A delegation must name a subtree. Consent on the namespace root reads as "let
|
||||
* this app hand out anything it has stored for you", which is not a bounded
|
||||
* capability and so is not something a prompt can put to a user.
|
||||
*/
|
||||
export const assertBoundedManageGrant = (permission: string): string => {
|
||||
if (!isKvShareManagePermission(permission)) return permission;
|
||||
const [, , owner, appUid, ...segments] = PermissionUtil.split(permission);
|
||||
if (!owner || !appUid || segments.length === 0)
|
||||
throw invalidPrefix(
|
||||
'A share delegation must name a region, not the whole namespace',
|
||||
);
|
||||
return permission;
|
||||
};
|
||||
|
||||
/**
|
||||
* The unbounded `manage:` grant a bounded delegation's own permission descends
|
||||
* from. The consent surface refuses to ever write this row, but
|
||||
* `canManagePermission` walks ancestors to decide "may I delegate this", so if
|
||||
* it exists by any other path it would silently authorize a mint anywhere in
|
||||
* the namespace — this is what lets the mint path notice it regardless.
|
||||
*/
|
||||
export const kvShareManageNamespaceRoot = (permission: string): string => {
|
||||
const [, owner, appUid] = PermissionUtil.split(permission);
|
||||
return PermissionUtil.join(
|
||||
MANAGE_PERM_PREFIX,
|
||||
KV_SHARE_PERMISSION_PREFIX,
|
||||
owner,
|
||||
appUid,
|
||||
);
|
||||
};
|
||||
|
||||
// -- Permission rules -------------------------------------------------
|
||||
|
||||
/**
|
||||
* The permission a `manage:` arm delegates over. Only the leading component is
|
||||
* stripped: a key segment may itself be `manage`, and taking those out would
|
||||
* change which string the owner is read from.
|
||||
*/
|
||||
const withoutManageArm = (permission: string): string =>
|
||||
permission.startsWith(`${MANAGE_PERM_PREFIX}:`)
|
||||
? permission.slice(MANAGE_PERM_PREFIX.length + 1)
|
||||
: permission;
|
||||
|
||||
/**
|
||||
* Owning the namespace is holding every share grant over it, and being able to
|
||||
* issue them. Without this the owner cannot mint a handle on their own data:
|
||||
* `grantUserUserPermission` asks `canManagePermission` first, and there is no
|
||||
* row anywhere saying a user may manage what is already theirs.
|
||||
*
|
||||
* Restricted to plain user actors, as the filesystem's `is-owner` is. An app
|
||||
* disposing of a region of its user's namespace is delegation, which is the
|
||||
* `manage:` grant's job rather than this one's.
|
||||
*/
|
||||
export const kvShareOwnerImplicator = (): PermissionImplicator => ({
|
||||
id: 'kv-share-is-owner',
|
||||
shortcut: true,
|
||||
matches: (permission: string): boolean =>
|
||||
isKvSharePermission(withoutManageArm(permission)),
|
||||
check: ({ actor, permission }): unknown => {
|
||||
if (!isPlainUserActor(actor)) return undefined;
|
||||
const uuid = actor.user?.uuid;
|
||||
if (!uuid) return undefined;
|
||||
|
||||
const owner = PermissionUtil.split(withoutManageArm(permission))[1];
|
||||
return owner === uuid ? {} : undefined;
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
* Letting an app-under-user actor exercise a `kv-share:` grant its user holds,
|
||||
* bounded to the app the grant's namespace names.
|
||||
*
|
||||
* Only the read arm is matched — never `manage:kv-share:…` — because answering
|
||||
* the manage arm here would read as the unbounded namespace-root delegation and
|
||||
* break minting for everyone (see `kvShareManageNamespaceRoot`). The
|
||||
* namespace-app check keeps one app from reading a region shared with another
|
||||
* app of the same user.
|
||||
*/
|
||||
export const kvShareAppDelegateImplicator = (deps: {
|
||||
userHolds: (actor: Actor, permission: string) => Promise<boolean>;
|
||||
}): PermissionImplicator => ({
|
||||
id: 'kv-share-app-delegate',
|
||||
shortcut: true,
|
||||
matches: (permission: string): boolean => isKvSharePermission(permission),
|
||||
check: async ({ actor, permission }): Promise<unknown> => {
|
||||
if (!isAppActor(actor)) return undefined;
|
||||
const app = actor.effectiveApp;
|
||||
if (!app?.uid) return undefined;
|
||||
|
||||
const [, owner, namespaceApp, ...segments] =
|
||||
PermissionUtil.split(permission);
|
||||
if (!owner || !namespaceApp || segments.length === 0) return undefined;
|
||||
if (namespaceApp !== app.uid) return undefined;
|
||||
|
||||
return (await deps.userHolds(userRelatedActor(actor), permission))
|
||||
? {}
|
||||
: undefined;
|
||||
},
|
||||
});
|
||||
@@ -43,20 +43,10 @@ const PROJECTED_KEYS = [
|
||||
'subject',
|
||||
'ts',
|
||||
'uid',
|
||||
'viaWorker',
|
||||
];
|
||||
|
||||
/** A KV event names a key where an FS event names a node. */
|
||||
const PROJECTED_KV_KEYS = [
|
||||
'id',
|
||||
'key',
|
||||
'op',
|
||||
'self',
|
||||
'seq',
|
||||
'subject',
|
||||
'ts',
|
||||
'viaWorker',
|
||||
];
|
||||
const PROJECTED_KV_KEYS = ['id', 'key', 'op', 'self', 'seq', 'subject', 'ts'];
|
||||
|
||||
const entry = {
|
||||
id: 42,
|
||||
@@ -205,24 +195,11 @@ describe('the kv subject', () => {
|
||||
op: 'set',
|
||||
key: 'cart:items',
|
||||
self: true,
|
||||
viaWorker: false,
|
||||
ts: 1_700_000_002,
|
||||
seq: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it('says whether server-side code made the change, on kv and fs alike', () => {
|
||||
expect(
|
||||
subject().project({ ...kvDelivery, viaWorker: true }),
|
||||
).toMatchObject({ viaWorker: true });
|
||||
expect(
|
||||
lookupFsSubject('fs.write.file')!.project({
|
||||
...delivery,
|
||||
viaWorker: true,
|
||||
}),
|
||||
).toMatchObject({ viaWorker: true });
|
||||
});
|
||||
|
||||
it('names the namespace app, never the namespace itself', () => {
|
||||
const projected = subject().project(kvDelivery) as Record<
|
||||
string,
|
||||
|
||||
@@ -64,11 +64,6 @@ export interface ProjectedFsEvent extends ProjectedEventBase {
|
||||
path: string;
|
||||
/** Present only on a `move` a subscribed folder lost the node from. */
|
||||
from?: string;
|
||||
/**
|
||||
* The change was made by server-side code running as the account (a worker,
|
||||
* or an events handler) rather than by the account directly.
|
||||
*/
|
||||
viaWorker: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -85,8 +80,6 @@ export interface ProjectedKvEvent extends ProjectedEventBase {
|
||||
* value is small enough to inline; an `expire` never carries one.
|
||||
*/
|
||||
value?: unknown;
|
||||
/** See {@link ProjectedFsEvent.viaWorker}. */
|
||||
viaWorker: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -149,8 +142,6 @@ export interface EventContextBase {
|
||||
ts: number;
|
||||
/** Handler runs behind the change. Never projected. */
|
||||
handlerDepth?: number;
|
||||
/** Made by server-side code running as the account; see `viaWorkerOf`. */
|
||||
viaWorker?: boolean;
|
||||
}
|
||||
|
||||
/** What dispatch knows about one committed filesystem change. */
|
||||
@@ -293,7 +284,6 @@ const fsProject =
|
||||
path: delivery.entry.path,
|
||||
...(delivery.movedFrom ? { from: delivery.movedFrom.path } : {}),
|
||||
self: delivery.self,
|
||||
viaWorker: delivery.viaWorker === true,
|
||||
ts: delivery.ts,
|
||||
seq: delivery.seq,
|
||||
});
|
||||
@@ -319,7 +309,6 @@ const kvProject = (delivery: KvDeliveryContext): ProjectedKvEvent => ({
|
||||
key: delivery.kvKey,
|
||||
...(delivery.value !== undefined ? { value: delivery.value } : {}),
|
||||
self: delivery.self,
|
||||
viaWorker: delivery.viaWorker === true,
|
||||
ts: delivery.ts,
|
||||
seq: delivery.seq,
|
||||
});
|
||||
@@ -440,10 +429,6 @@ export const UNPUBLISHED_INTERNAL_EVENTS = [
|
||||
event: 'kv.flushed',
|
||||
reason: 'No subject addresses "the whole namespace went": the grammar anchors on a key or a prefix, and fanning a flush out per key would report keys a truncated enumeration never saw.',
|
||||
},
|
||||
{
|
||||
event: 'kv.share.retired',
|
||||
reason: 'A share handle stopping is not a change to any key; it ends the subscriptions made through the handle rather than being delivered to them.',
|
||||
},
|
||||
] as const satisfies readonly UnpublishedInternalEvent[];
|
||||
|
||||
// Every `fs.*`/`kv.*`/`notif.*` bus key needs a decision. A new key that is in
|
||||
|
||||
@@ -194,10 +194,6 @@ export const notifMatchOn = (
|
||||
audience: NotificationAudience | string,
|
||||
): string => `${ref}${NOTIF_MATCH_SEPARATOR}${audience}`;
|
||||
|
||||
/** Whether a notif filter spans every app: a session's generic slice. */
|
||||
export const notifMatchSpansEveryApp = (match: string | null): boolean =>
|
||||
match?.startsWith(`*${NOTIF_MATCH_SEPARATOR}`) === true;
|
||||
|
||||
/** Which family a stored row belongs to, without re-parsing its subject. */
|
||||
export const isKvToken = (token: string): boolean =>
|
||||
token.startsWith(KV_TOKEN_PREFIX);
|
||||
|
||||
@@ -425,7 +425,6 @@ describe('the call an owed delivery makes', () => {
|
||||
'subject',
|
||||
'ts',
|
||||
'uid',
|
||||
'viaWorker',
|
||||
]);
|
||||
expect(call.body.event).toMatchObject({
|
||||
path: `${anchor}/note.txt`,
|
||||
|
||||
@@ -41,7 +41,6 @@ import {
|
||||
isAccountContext,
|
||||
isAppActor,
|
||||
isPlainUserActor,
|
||||
viaWorkerOf,
|
||||
} from '../../core/actor.js';
|
||||
import { Context } from '../../core/context.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
@@ -4795,7 +4794,6 @@ export class FSService extends PuterService {
|
||||
const actor = Context.get('actor') as Actor | undefined;
|
||||
const actingUserId = actor?.user?.id;
|
||||
const handlerDepth = handlerDepthOf(actor);
|
||||
const viaWorker = viaWorkerOf(actor);
|
||||
const movedFromChain = movedFrom && {
|
||||
path: movedFrom.path,
|
||||
ancestors: () => fsEntry.getAncestorChain(movedFrom.path),
|
||||
@@ -4809,7 +4807,6 @@ export class FSService extends PuterService {
|
||||
events.dispatchFs(key, entry, {
|
||||
actingUserId,
|
||||
handlerDepth,
|
||||
viaWorker,
|
||||
ancestors: () => ancestors(entry),
|
||||
...(movedFromChain
|
||||
? { movedFrom: movedFromChain }
|
||||
|
||||
@@ -33,7 +33,6 @@ import { EventForwardService } from './events/EventForwardService';
|
||||
import { EventsService } from './events/EventsService';
|
||||
import { AppFeedbackService } from './feedback/AppFeedbackService';
|
||||
import { FSService } from './fs/FSService';
|
||||
import { KvShareService } from './kvShare/KvShareService';
|
||||
import { ServerHealthService } from './health/ServerHealthService';
|
||||
import { PuterHomepageService } from './homepage/PuterHomepageService';
|
||||
import { LocalWorkerService } from './localworker/LocalWorkerService';
|
||||
@@ -75,7 +74,6 @@ declare module './types' {
|
||||
suggestedApps: SuggestedAppsService;
|
||||
app: AppService;
|
||||
socket: SocketService;
|
||||
kvShare: KvShareService;
|
||||
events: EventsService;
|
||||
eventForward: EventForwardService;
|
||||
notification: NotificationService;
|
||||
@@ -123,6 +121,7 @@ export const puterServices = {
|
||||
// Declared after `fs` — account teardown tears the user's filesystem down
|
||||
// first.
|
||||
userAccount: UserAccountService,
|
||||
// Leaf: team + user stores only.
|
||||
// Reaches `team` and `socket` at call time only.
|
||||
signup: SignupService,
|
||||
// AppPermissionService + SubdomainPermissionService register permission
|
||||
@@ -136,9 +135,6 @@ export const puterServices = {
|
||||
// Reaches `auth` and `permission` at call time only.
|
||||
app: AppService,
|
||||
socket: SocketService,
|
||||
// Registers the share-grant implicators with `permission`, and reaches
|
||||
// `permission`, `metering` and `notification` at call time only.
|
||||
kvShare: KvShareService,
|
||||
// Delivers through `socket` and resolves paths through `fs`, so it follows
|
||||
// both; `fs` reaches back for dispatch at call time only.
|
||||
events: EventsService,
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,365 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Withdrawing a share against a replica that lags the primary, and the MySQL
|
||||
* matching rules a share's grants and rows meet. Runs on SQLite, and also on
|
||||
* MySQL when `PUTER_TEST_MYSQL_PORT` names one.
|
||||
*/
|
||||
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import mysql from 'mysql2/promise';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import { EVENTS_KV_HANDLE_LIMIT } from '../../controllers/events/limits.js';
|
||||
import { makeActor, type Actor } from '../../core/actor.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import {
|
||||
kvShareGrantAtEveryLevel,
|
||||
kvSharePermission,
|
||||
type KvShareAccess,
|
||||
} from './kvShares.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 300_000;
|
||||
const GLOBAL = 'os-global';
|
||||
const MYSQL = {
|
||||
host: '127.0.0.1',
|
||||
port: Number(process.env.PUTER_TEST_MYSQL_PORT ?? 3307),
|
||||
user: 'root',
|
||||
password: 'puter',
|
||||
};
|
||||
const MIGRATIONS = fileURLToPath(
|
||||
new URL('../../clients/database/migrations/mysql', import.meta.url),
|
||||
);
|
||||
const ENGINES = [
|
||||
'sqlite',
|
||||
...(process.env.PUTER_TEST_MYSQL_PORT ? ['mysql'] : []),
|
||||
] as const;
|
||||
|
||||
interface TestUser {
|
||||
actor: Actor;
|
||||
username: string;
|
||||
id: number;
|
||||
uuid: string;
|
||||
}
|
||||
|
||||
describe.each(ENGINES)('key-value shares on %s', (engine) => {
|
||||
const database = `kv_share_${Date.now().toString(36)}`;
|
||||
let env: PuterTestEnv;
|
||||
let owner: TestUser;
|
||||
let guest: TestUser;
|
||||
|
||||
const db = () => env.server.clients.db;
|
||||
const shares = () => env.server.services.kvShare;
|
||||
const permissions = () => env.server.services.permission;
|
||||
const handles = () => env.server.stores.kvShareHandle;
|
||||
const flat = () => env.server.stores.permission;
|
||||
|
||||
const userFor = async (username: string): Promise<TestUser> => {
|
||||
const row = await env.server.stores.user.getByUsername(username);
|
||||
return {
|
||||
actor: makeActor({ user: row as never }),
|
||||
username,
|
||||
id: row!.id,
|
||||
uuid: row!.uuid as string,
|
||||
};
|
||||
};
|
||||
|
||||
const region = (label: string) => `${label}-${uuidv4().slice(0, 8)}:`;
|
||||
|
||||
const share = (prefix: string, access: KvShareAccess) =>
|
||||
shares().mintKvHandle(
|
||||
owner.actor,
|
||||
{ prefix, recipient: { username: guest.username }, access },
|
||||
{ defaultAccess: 'read' },
|
||||
);
|
||||
|
||||
const grantRows = async (prefix: string) =>
|
||||
(
|
||||
await db().pread(
|
||||
'SELECT `permission` FROM `user_to_user_permissions` ' +
|
||||
'WHERE `holder_user_id` = ? AND `permission` IN (?, ?, ?)',
|
||||
[
|
||||
guest.id,
|
||||
...kvShareGrantAtEveryLevel(
|
||||
kvSharePermission(owner.uuid, GLOBAL, prefix),
|
||||
),
|
||||
],
|
||||
)
|
||||
).map((row) => String(row.permission));
|
||||
|
||||
/**
|
||||
* From here until the returned function runs, replica reads see the grant
|
||||
* table as it is now and primary reads see it as it really is.
|
||||
*/
|
||||
const lagReplica = async () => {
|
||||
await db().write('DROP TABLE IF EXISTS `u2u_lagging`');
|
||||
await db().write(
|
||||
'CREATE TABLE `u2u_lagging` AS SELECT * FROM `user_to_user_permissions`',
|
||||
);
|
||||
const read = db().read.bind(db());
|
||||
const replica = vi
|
||||
.spyOn(db(), 'read')
|
||||
.mockImplementation((query, params) =>
|
||||
read(
|
||||
query.replaceAll(
|
||||
'`user_to_user_permissions`',
|
||||
'`u2u_lagging`',
|
||||
),
|
||||
params,
|
||||
),
|
||||
);
|
||||
// SQLite's own primary read goes through `read`, so route it past the
|
||||
// lagging view explicitly.
|
||||
const primary = vi
|
||||
.spyOn(db(), 'pread')
|
||||
.mockImplementation((query, params) => read(query, params));
|
||||
return async () => {
|
||||
replica.mockRestore();
|
||||
primary.mockRestore();
|
||||
await db().write('DROP TABLE `u2u_lagging`');
|
||||
};
|
||||
};
|
||||
|
||||
/** No row, no flat entry, and no write reaching a later nested share. */
|
||||
const expectWithdrawn = async (prefix: string) => {
|
||||
expect(await grantRows(prefix)).toEqual([]);
|
||||
expect(
|
||||
await flat().getFlatUserPerms(
|
||||
guest.id,
|
||||
kvShareGrantAtEveryLevel(
|
||||
kvSharePermission(owner.uuid, GLOBAL, prefix),
|
||||
),
|
||||
),
|
||||
).toEqual([]);
|
||||
const nested = await share(`${prefix}nested:`, 'watch');
|
||||
await expect(
|
||||
runWithContext({ actor: guest.actor }, () =>
|
||||
env.server.drivers.kvStore.set({
|
||||
key: 'k',
|
||||
value: 1,
|
||||
optConfig: { handle: nested.handle },
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'kv_handle_access_denied' });
|
||||
};
|
||||
|
||||
beforeAll(async () => {
|
||||
const flags = {
|
||||
events: { enabled: true, kvHandles: true },
|
||||
kv: { shareHandles: true },
|
||||
unlimitedMetering: true,
|
||||
};
|
||||
if (engine === 'mysql') {
|
||||
const admin = await mysql.createConnection(MYSQL);
|
||||
await admin.query(`CREATE DATABASE \`${database}\``);
|
||||
await admin.end();
|
||||
env = await setupPuterTestEnv({
|
||||
...flags,
|
||||
database: {
|
||||
engine: 'mysql',
|
||||
...MYSQL,
|
||||
database,
|
||||
migrationPaths: [MIGRATIONS],
|
||||
},
|
||||
} as never);
|
||||
} else {
|
||||
env = await setupPuterTestEnv(flags as IConfig);
|
||||
}
|
||||
owner = await userFor(env.users.user.username);
|
||||
guest = await userFor(env.users.other.username);
|
||||
}, BOOT_TIMEOUT_MS);
|
||||
|
||||
afterAll(async () => {
|
||||
await env?.shutdown();
|
||||
if (engine !== 'mysql') return;
|
||||
const admin = await mysql.createConnection(MYSQL);
|
||||
await admin.query(`DROP DATABASE IF EXISTS \`${database}\``);
|
||||
await admin.end();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
for (const user of [owner, guest])
|
||||
await env.server.clients.redis.del(
|
||||
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user.id}`,
|
||||
);
|
||||
});
|
||||
|
||||
describe('an owner revoke while the replica lags', () => {
|
||||
it('withdraws a share minted inside the lag', async () => {
|
||||
const prefix = region('lag-mint');
|
||||
const thaw = await lagReplica();
|
||||
try {
|
||||
const { handle } = await share(prefix, 'write');
|
||||
await shares().removeKvShare(owner.actor, handle);
|
||||
} finally {
|
||||
await thaw();
|
||||
}
|
||||
await expectWithdrawn(prefix);
|
||||
});
|
||||
|
||||
it('withdraws the write grant of a read share raised inside the lag', async () => {
|
||||
const prefix = region('lag-raise');
|
||||
const { handle } = await share(prefix, 'read');
|
||||
const thaw = await lagReplica();
|
||||
try {
|
||||
await share(prefix, 'write');
|
||||
await shares().removeKvShare(owner.actor, handle);
|
||||
} finally {
|
||||
await thaw();
|
||||
}
|
||||
await expectWithdrawn(prefix);
|
||||
});
|
||||
|
||||
it('clears a flat entry under the region that no row backs', async () => {
|
||||
const prefix = region('orphan');
|
||||
const { handle } = await share(prefix, 'read');
|
||||
const orphan = kvSharePermission(
|
||||
owner.uuid,
|
||||
GLOBAL,
|
||||
`${prefix}deep:`,
|
||||
'write',
|
||||
);
|
||||
await flat().setFlatUserPerm(guest.id, orphan, {
|
||||
permission: orphan,
|
||||
issuer_user_id: owner.id,
|
||||
deleted: false,
|
||||
});
|
||||
|
||||
await shares().removeKvShare(owner.actor, handle);
|
||||
|
||||
expect(await flat().getFlatUserPerms(guest.id, [orphan])).toEqual(
|
||||
[],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('matching a share`s prefix', () => {
|
||||
it('finds a live share by its exact bytes only', async () => {
|
||||
const prefix = region('exact');
|
||||
await share(prefix, 'read');
|
||||
const input = {
|
||||
ownerUserId: owner.id,
|
||||
granteeUserId: guest.id,
|
||||
appUid: GLOBAL,
|
||||
keyPrefix: prefix,
|
||||
};
|
||||
expect(await handles().findLive(input)).not.toBeNull();
|
||||
expect(
|
||||
await handles().findLive({
|
||||
...input,
|
||||
keyPrefix: prefix.toUpperCase(),
|
||||
}),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it.skipIf(engine !== 'mysql')(
|
||||
'uses no operator MySQL warns is deprecated, whatever the connection charset',
|
||||
async () => {
|
||||
const pread = vi.spyOn(db(), 'pread');
|
||||
let call: unknown[] | undefined;
|
||||
try {
|
||||
await handles().findLive({
|
||||
ownerUserId: owner.id,
|
||||
granteeUserId: guest.id,
|
||||
appUid: GLOBAL,
|
||||
keyPrefix: region('warn'),
|
||||
});
|
||||
call = pread.mock.calls.find(([query]) =>
|
||||
query.includes('`key_prefix`'),
|
||||
);
|
||||
} finally {
|
||||
pread.mockRestore();
|
||||
}
|
||||
expect(call).toBeDefined();
|
||||
|
||||
// The text protocol, so the statement is parsed (and warned
|
||||
// about) as it runs.
|
||||
const connection = await mysql.createConnection({
|
||||
...MYSQL,
|
||||
database,
|
||||
charset: 'LATIN1_SWEDISH_CI',
|
||||
});
|
||||
try {
|
||||
await connection.query(
|
||||
String(call![0]),
|
||||
call![1] as never[],
|
||||
);
|
||||
const [warnings] = await connection.query('SHOW WARNINGS');
|
||||
expect(warnings).toEqual([]);
|
||||
} finally {
|
||||
await connection.end();
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe('revoking a grant the database spells differently', () => {
|
||||
it.skipIf(engine !== 'mysql')(
|
||||
'clears the flat entry under the spelling it removed',
|
||||
async () => {
|
||||
const tag = uuidv4().slice(0, 8);
|
||||
const stored = kvSharePermission(
|
||||
owner.uuid,
|
||||
GLOBAL,
|
||||
`Notes-${tag}:`,
|
||||
'read',
|
||||
);
|
||||
const asked = kvSharePermission(
|
||||
owner.uuid,
|
||||
GLOBAL,
|
||||
`notes-${tag}:`,
|
||||
'read',
|
||||
);
|
||||
await permissions().grantUserUserPermission(
|
||||
owner.actor,
|
||||
guest.username,
|
||||
stored,
|
||||
);
|
||||
expect(await permissions().check(guest.actor, stored)).toBe(
|
||||
true,
|
||||
);
|
||||
|
||||
const removed = await permissions().revokeUserUserPermissions(
|
||||
owner.actor,
|
||||
guest.username,
|
||||
[asked],
|
||||
);
|
||||
|
||||
expect(removed).toEqual([stored]);
|
||||
expect(
|
||||
await flat().getFlatUserPerms(guest.id, [stored]),
|
||||
).toEqual([]);
|
||||
expect(await permissions().check(guest.actor, stored)).toBe(
|
||||
false,
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,549 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* What the recipient of a key-value share sees and controls: the notification
|
||||
* that tells their app a share arrived, kept off their desktop, and the share
|
||||
* blocks that refuse one.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import {
|
||||
EVENTS_COALESCE_WINDOW_MS,
|
||||
EVENTS_KV_HANDLE_LIMIT,
|
||||
KV_SHARE_NOTICE_LIMIT,
|
||||
} from '../../controllers/events/limits.js';
|
||||
import { makeActor, type Actor } from '../../core/actor.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import type { DeliveryEnvelope } from '../events/EventsService.js';
|
||||
import type { KvShareAccess } from './kvShares.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
|
||||
interface TestUser {
|
||||
actor: Actor;
|
||||
username: string;
|
||||
id: number;
|
||||
uuid: string;
|
||||
}
|
||||
|
||||
/** One server, its two users, and a fresh app the owner can share data of. */
|
||||
const bootEnv = (config: Record<string, unknown>) => {
|
||||
const state = {
|
||||
env: null as unknown as PuterTestEnv,
|
||||
owner: null as unknown as TestUser,
|
||||
guest: null as unknown as TestUser,
|
||||
};
|
||||
|
||||
const userFor = async (username: string): Promise<TestUser> => {
|
||||
const row = await state.env.server.stores.user.getByUsername(username);
|
||||
return {
|
||||
actor: makeActor({ user: row as never }),
|
||||
username,
|
||||
id: row!.id,
|
||||
uuid: row!.uuid as string,
|
||||
};
|
||||
};
|
||||
|
||||
beforeAll(async () => {
|
||||
state.env = await setupPuterTestEnv({
|
||||
events: { enabled: true, kvHandles: true },
|
||||
kv: { shareHandles: true },
|
||||
unlimitedMetering: true,
|
||||
...config,
|
||||
} as IConfig);
|
||||
state.owner = await userFor(state.env.users.user.username);
|
||||
state.guest = await userFor(state.env.users.other.username);
|
||||
}, BOOT_TIMEOUT_MS);
|
||||
|
||||
afterAll(async () => {
|
||||
await state.env?.shutdown();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
for (const user of [state.owner, state.guest])
|
||||
await state.env.server.clients.redis.del(
|
||||
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user.id}`,
|
||||
);
|
||||
await state.env.server.clients.redis.del(
|
||||
`rate:${KV_SHARE_NOTICE_LIMIT.scope}:${state.owner.id}:${state.guest.id}`,
|
||||
);
|
||||
});
|
||||
|
||||
const makeApp = async () => {
|
||||
const name = `kv-recipient-${uuidv4().slice(0, 8)}`;
|
||||
return state.env.server.stores.app.create(
|
||||
{
|
||||
name,
|
||||
title: 'Recipient App',
|
||||
index_url: `https://${name}.example.test/index.html`,
|
||||
},
|
||||
{ ownerUserId: state.owner.id },
|
||||
);
|
||||
};
|
||||
|
||||
const share = (
|
||||
prefix: string,
|
||||
access: KvShareAccess,
|
||||
opts: { appUid?: string; label?: unknown } = {},
|
||||
) =>
|
||||
state.env.server.services.kvShare.mintKvHandle(
|
||||
state.owner.actor,
|
||||
{
|
||||
prefix,
|
||||
recipient: { username: state.guest.username },
|
||||
access,
|
||||
...(opts.appUid ? { appUid: opts.appUid } : {}),
|
||||
...(opts.label !== undefined ? { label: opts.label } : {}),
|
||||
},
|
||||
{ defaultAccess: 'read' },
|
||||
);
|
||||
|
||||
/** The holder's share notifications naming one handle. */
|
||||
const notificationsFor = async (handle: string) =>
|
||||
(
|
||||
await state.env.server.clients.db.pread(
|
||||
'SELECT `uid`, `app_uid`, `audience`, `value` FROM `notification` ' +
|
||||
"WHERE `user_id` = ? AND `type` = 'kv.share.received'",
|
||||
[state.guest.id],
|
||||
)
|
||||
)
|
||||
.map((row) => ({
|
||||
uid: String(row.uid),
|
||||
appUid: row.app_uid,
|
||||
audience: row.audience,
|
||||
value: (typeof row.value === 'string'
|
||||
? JSON.parse(row.value)
|
||||
: row.value) as Record<string, unknown>,
|
||||
}))
|
||||
.filter((row) => row.value.handle === handle);
|
||||
|
||||
return { state, makeApp, share, notificationsFor };
|
||||
};
|
||||
|
||||
const region = (label: string) => `${label}-${uuidv4().slice(0, 8)}:`;
|
||||
|
||||
describe('the share notification', () => {
|
||||
const { state, makeApp, share, notificationsFor } = bootEnv({});
|
||||
|
||||
it('reaches the holder once, in the app the data belongs to, without the owner', async () => {
|
||||
const app = await makeApp();
|
||||
const prefix = region('notify');
|
||||
const shared = await share(prefix, 'write', {
|
||||
appUid: app.uid,
|
||||
label: 'Groceries',
|
||||
});
|
||||
|
||||
const rows = await notificationsFor(shared.handle);
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({
|
||||
appUid: app.uid,
|
||||
audience: 'app-user',
|
||||
});
|
||||
expect(rows[0].value).toEqual({
|
||||
type: 'kv.share.received',
|
||||
handle: shared.handle,
|
||||
access: 'write',
|
||||
label: 'Groceries',
|
||||
createdAt: shared.createdAt,
|
||||
});
|
||||
const wire = JSON.stringify(rows);
|
||||
expect(wire).not.toContain(state.owner.uuid);
|
||||
expect(wire).not.toContain(state.owner.username);
|
||||
expect(wire).not.toContain(prefix);
|
||||
});
|
||||
|
||||
it('is not sent again for a re-share, a level change or a label change', async () => {
|
||||
const app = await makeApp();
|
||||
const prefix = region('again');
|
||||
const { handle } = await share(prefix, 'read', { appUid: app.uid });
|
||||
await share(prefix, 'read', { appUid: app.uid });
|
||||
await share(prefix, 'write', { appUid: app.uid });
|
||||
await share(prefix, 'write', { appUid: app.uid, label: 'Renamed' });
|
||||
expect(await notificationsFor(handle)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('is sent at most five times an hour per owner and holder, and the share is made anyway', async () => {
|
||||
const app = await makeApp();
|
||||
const prefix = region('churn');
|
||||
const handles: string[] = [];
|
||||
for (let i = 0; i < KV_SHARE_NOTICE_LIMIT.limit + 2; i++) {
|
||||
const { handle } = await share(prefix, 'read', { appUid: app.uid });
|
||||
handles.push(handle);
|
||||
await state.env.server.services.kvShare.removeKvShare(
|
||||
state.owner.actor,
|
||||
handle,
|
||||
);
|
||||
}
|
||||
expect(new Set(handles).size).toBe(handles.length);
|
||||
const counts = await Promise.all(
|
||||
handles.map(
|
||||
async (handle) => (await notificationsFor(handle)).length,
|
||||
),
|
||||
);
|
||||
expect(counts.reduce((sum, n) => sum + n, 0)).toBe(
|
||||
KV_SHARE_NOTICE_LIMIT.limit,
|
||||
);
|
||||
expect(counts.slice(KV_SHARE_NOTICE_LIMIT.limit)).toEqual([0, 0]);
|
||||
});
|
||||
|
||||
it('is not sent for a share with no app', async () => {
|
||||
const { handle } = await share(region('global'), 'read');
|
||||
expect(await notificationsFor(handle)).toEqual([]);
|
||||
});
|
||||
|
||||
it('never reaches the desktop wire, and the mint stands when it fails', async () => {
|
||||
const app = await makeApp();
|
||||
const emit = vi.spyOn(state.env.server.clients.event, 'emit');
|
||||
try {
|
||||
const { handle } = await share(region('wire'), 'read', {
|
||||
appUid: app.uid,
|
||||
});
|
||||
const [row] = await notificationsFor(handle);
|
||||
const pushed = emit.mock.calls.filter(
|
||||
([key, data]) =>
|
||||
key === 'outer.gui.notif.message' &&
|
||||
JSON.stringify(data).includes(row.uid),
|
||||
);
|
||||
expect(pushed).toEqual([]);
|
||||
} finally {
|
||||
emit.mockRestore();
|
||||
}
|
||||
|
||||
const notify = vi
|
||||
.spyOn(state.env.server.services.notification, 'notify')
|
||||
.mockRejectedValueOnce(new Error('mailbox down'));
|
||||
try {
|
||||
await expect(
|
||||
share(region('failing'), 'read', { appUid: app.uid }),
|
||||
).resolves.toMatchObject({ access: 'read' });
|
||||
} finally {
|
||||
notify.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('stays out of the whole-mailbox listing, but not out of the app`s slice', async () => {
|
||||
const app = await makeApp();
|
||||
const { handle } = await share(region('select'), 'read', {
|
||||
appUid: app.uid,
|
||||
});
|
||||
const [row] = await notificationsFor(handle);
|
||||
const select = (actor: Actor, args: Record<string, unknown> = {}) =>
|
||||
runWithContext({ actor }, () =>
|
||||
state.env.server.drivers.notifications.select(args),
|
||||
) as Promise<Array<{ uid: unknown }>>;
|
||||
const uids = (rows: Array<{ uid: unknown }>) =>
|
||||
rows.map((one) => one.uid);
|
||||
|
||||
expect(uids(await select(state.guest.actor))).not.toContain(row.uid);
|
||||
expect(
|
||||
uids(
|
||||
await select(state.guest.actor, { subject: 'notif:app-user' }),
|
||||
),
|
||||
).not.toContain(row.uid);
|
||||
expect(
|
||||
uids(
|
||||
await select(state.guest.actor, {
|
||||
subject: `notif:${app.uid}:app-user`,
|
||||
}),
|
||||
),
|
||||
).toContain(row.uid);
|
||||
const appActor = makeActor({
|
||||
user: state.guest.actor.user as never,
|
||||
app: { uid: app.uid, id: app.id },
|
||||
});
|
||||
expect(uids(await select(appActor))).toContain(row.uid);
|
||||
});
|
||||
|
||||
it('is left out of the unread replay a desktop gets on connecting', async () => {
|
||||
const app = await makeApp();
|
||||
const { handle } = await share(region('replay'), 'read', {
|
||||
appUid: app.uid,
|
||||
});
|
||||
const [row] = await notificationsFor(handle);
|
||||
const emit = vi.spyOn(state.env.server.clients.event, 'emit');
|
||||
try {
|
||||
state.env.server.clients.event.emit(
|
||||
'web.socket.user-connected',
|
||||
{ user: { id: state.guest.id } } as never,
|
||||
{},
|
||||
);
|
||||
// The replay is debounced by two seconds.
|
||||
await new Promise((resolve) => setTimeout(resolve, 2_300));
|
||||
const replayed = emit.mock.calls.filter(
|
||||
([key, data]) =>
|
||||
key === 'outer.gui.notif.unreads' &&
|
||||
JSON.stringify(data).includes(row.uid),
|
||||
);
|
||||
expect(replayed).toEqual([]);
|
||||
} finally {
|
||||
emit.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('the share notification with notification dispatch on', () => {
|
||||
const { state, makeApp, share, notificationsFor } = bootEnv({
|
||||
events: {
|
||||
enabled: true,
|
||||
kvHandles: true,
|
||||
notificationsFoldIn: true,
|
||||
},
|
||||
});
|
||||
|
||||
it('reaches the app`s notif:app-user subscription and never the desktop', async () => {
|
||||
const app = await makeApp();
|
||||
const delivered: DeliveryEnvelope[] = [];
|
||||
state.env.server.services.events.onDelivered = (envelope) =>
|
||||
delivered.push(envelope);
|
||||
const appActor = makeActor({
|
||||
user: state.guest.actor.user as never,
|
||||
app: { uid: app.uid, id: app.id },
|
||||
});
|
||||
await state.env.server.services.events.subscribe(
|
||||
appActor,
|
||||
'kv-recipient-socket',
|
||||
{ subject: 'notif:app-user' },
|
||||
);
|
||||
const send = vi.spyOn(state.env.server.services.socket, 'send');
|
||||
try {
|
||||
const { handle } = await share(region('folded'), 'read', {
|
||||
appUid: app.uid,
|
||||
});
|
||||
const [row] = await notificationsFor(handle);
|
||||
|
||||
await vi.waitFor(
|
||||
() =>
|
||||
expect(
|
||||
delivered.some(
|
||||
(one) =>
|
||||
(one.event as { uid?: string }).uid === row.uid,
|
||||
),
|
||||
).toBe(true),
|
||||
{ timeout: EVENTS_COALESCE_WINDOW_MS * 12, interval: 25 },
|
||||
);
|
||||
const desktop = send.mock.calls.filter(
|
||||
([, key, data]) =>
|
||||
key === 'notif.message' &&
|
||||
JSON.stringify(data).includes(row.uid),
|
||||
);
|
||||
expect(desktop).toEqual([]);
|
||||
} finally {
|
||||
send.mockRestore();
|
||||
await state.env.server.services.events.reapSocket(
|
||||
state.guest.id,
|
||||
'kv-recipient-socket',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('stays out of the desktop`s own notif:app-user feed, live and fetched', async () => {
|
||||
const events = state.env.server.services.events;
|
||||
const app = await makeApp();
|
||||
const delivered: DeliveryEnvelope[] = [];
|
||||
events.onDelivered = (envelope) => delivered.push(envelope);
|
||||
const appActor = makeActor({
|
||||
user: state.guest.actor.user as never,
|
||||
app: { uid: app.uid, id: app.id },
|
||||
});
|
||||
const { sub: desktopSub } = await events.subscribe(
|
||||
state.guest.actor,
|
||||
'kv-recipient-desktop',
|
||||
{ subject: 'notif:app-user' },
|
||||
);
|
||||
const { sub: appSub } = await events.subscribe(
|
||||
appActor,
|
||||
'kv-recipient-app',
|
||||
{ subject: 'notif:app-user' },
|
||||
);
|
||||
const deliveredTo = (subId: string, uid: string) =>
|
||||
delivered.some(
|
||||
(one) =>
|
||||
one.subId === subId &&
|
||||
(one.event as { uid?: string }).uid === uid,
|
||||
);
|
||||
// Deliveries on one subject within the coalesce window collapse to the
|
||||
// last, so each notification is waited out before the next is sent.
|
||||
const waitFor = (check: () => boolean) =>
|
||||
vi.waitFor(() => expect(check()).toBe(true), {
|
||||
timeout: EVENTS_COALESCE_WINDOW_MS * 12,
|
||||
interval: 25,
|
||||
});
|
||||
try {
|
||||
const { handle } = await share(region('desktop'), 'read', {
|
||||
appUid: app.uid,
|
||||
});
|
||||
const [row] = await notificationsFor(handle);
|
||||
await waitFor(() => deliveredTo(appSub.subId, row.uid));
|
||||
|
||||
// Another app-user row about the same app still reaches the desktop.
|
||||
const ended = (await state.env.server.services.notification.notify(
|
||||
[state.guest.id],
|
||||
{ title: 'Subscription ended' },
|
||||
{ type: 'app.events.ended', appUid: app.uid },
|
||||
))!;
|
||||
await waitFor(() => deliveredTo(desktopSub.subId, ended));
|
||||
expect(deliveredTo(desktopSub.subId, row.uid)).toBe(false);
|
||||
|
||||
const fetched = async (subject: string) =>
|
||||
(
|
||||
await events.fetchMissed(state.guest.actor, { subject })
|
||||
).items.map((item) => item.uid);
|
||||
const generic = await fetched('notif:app-user');
|
||||
expect(generic).toContain(ended);
|
||||
expect(generic).not.toContain(row.uid);
|
||||
expect(await fetched(`notif:${app.uid}:app-user`)).toContain(
|
||||
row.uid,
|
||||
);
|
||||
} finally {
|
||||
for (const socketId of ['kv-recipient-desktop', 'kv-recipient-app'])
|
||||
await events.reapSocket(state.guest.id, socketId);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('recipient share blocks', () => {
|
||||
const { state, makeApp, share, notificationsFor } = bootEnv({});
|
||||
|
||||
it('refuse a share from a sender the recipient blocked', async () => {
|
||||
await state.env.server.stores.userBlock.create(
|
||||
state.guest.id,
|
||||
state.owner.id,
|
||||
);
|
||||
try {
|
||||
const app = await makeApp();
|
||||
await expect(
|
||||
share(region('blocked'), 'read', { appUid: app.uid }),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
legacyCode: 'recipient_not_accepting_shares',
|
||||
});
|
||||
const listed =
|
||||
await state.env.server.services.kvShare.listKvHandles(
|
||||
state.owner.actor,
|
||||
{ limit: 200 },
|
||||
);
|
||||
expect(
|
||||
listed.items.filter((row) => row.appUid === app.uid),
|
||||
).toEqual([]);
|
||||
} finally {
|
||||
await state.env.server.stores.userBlock.deleteByPair(
|
||||
state.guest.id,
|
||||
state.owner.id,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('refuse a share when the recipient refuses everyone', async () => {
|
||||
await state.env.server.stores.user.updateMetadata(state.guest.id, {
|
||||
blockAllShares: true,
|
||||
});
|
||||
try {
|
||||
await expect(share(region('all'), 'watch')).rejects.toMatchObject({
|
||||
legacyCode: 'recipient_not_accepting_shares',
|
||||
});
|
||||
} finally {
|
||||
await state.env.server.stores.user.updateMetadata(state.guest.id, {
|
||||
blockAllShares: false,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('leave a share made before the block to be narrowed, its label cleared, and revoked', async () => {
|
||||
const app = await makeApp();
|
||||
const prefix = region('before');
|
||||
const { handle } = await share(prefix, 'write', {
|
||||
appUid: app.uid,
|
||||
label: 'Before',
|
||||
});
|
||||
await state.env.server.stores.userBlock.create(
|
||||
state.guest.id,
|
||||
state.owner.id,
|
||||
);
|
||||
try {
|
||||
await expect(
|
||||
share(prefix, 'read', { appUid: app.uid, label: null }),
|
||||
).resolves.toMatchObject({
|
||||
handle,
|
||||
access: 'read',
|
||||
label: null,
|
||||
});
|
||||
await expect(
|
||||
state.env.server.services.kvShare.removeKvShare(
|
||||
state.owner.actor,
|
||||
handle,
|
||||
),
|
||||
).resolves.toMatchObject({ handle });
|
||||
// Still only the notification from before the block.
|
||||
expect(await notificationsFor(handle)).toHaveLength(1);
|
||||
} finally {
|
||||
await state.env.server.stores.userBlock.deleteByPair(
|
||||
state.guest.id,
|
||||
state.owner.id,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
['a raised level', { access: 'write' as const }],
|
||||
['a new label', { label: 'Later' }],
|
||||
[
|
||||
'a lower level with a new label',
|
||||
{ access: 'watch' as const, label: 'Later' },
|
||||
],
|
||||
])('refuse %s on a share made before the block', async (_name, change) => {
|
||||
const app = await makeApp();
|
||||
const prefix = region('raise');
|
||||
const { handle } = await share(prefix, 'read', { appUid: app.uid });
|
||||
await state.env.server.stores.userBlock.create(
|
||||
state.guest.id,
|
||||
state.owner.id,
|
||||
);
|
||||
try {
|
||||
await expect(
|
||||
share(prefix, 'access' in change ? change.access : 'read', {
|
||||
appUid: app.uid,
|
||||
...('label' in change ? { label: change.label } : {}),
|
||||
}),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'recipient_not_accepting_shares',
|
||||
});
|
||||
const row =
|
||||
await state.env.server.stores.kvShareHandle.getByHandle(handle);
|
||||
expect(row).toMatchObject({ revokedAt: null, label: null });
|
||||
expect(row?.permission.startsWith('kv-share-read:')).toBe(true);
|
||||
} finally {
|
||||
await state.env.server.stores.userBlock.deleteByPair(
|
||||
state.guest.id,
|
||||
state.owner.id,
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,868 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { PermissionUtil } from '../permission/permissionUtil.js';
|
||||
import {
|
||||
assertBoundedManageGrant,
|
||||
assertNewShareSegments,
|
||||
assertPrefixFitsAccess,
|
||||
assertShareableAppUid,
|
||||
assertShareablePrefix,
|
||||
isKvShareManagePermission,
|
||||
isKvSharePermission,
|
||||
keyPrefixSegments,
|
||||
KV_SHARE_PREFIX_MAX_BYTES,
|
||||
kvShareAccessOf,
|
||||
kvShareAppDelegateImplicator,
|
||||
kvShareGrantAtEveryLevel,
|
||||
kvShareGrantCovers,
|
||||
kvShareGrantPrefix,
|
||||
kvShareLevelForMethod,
|
||||
kvShareLevelsExploder,
|
||||
kvShareManageNamespaceRoots,
|
||||
kvShareManagePermission,
|
||||
kvShareOwnerImplicator,
|
||||
kvSharePermission,
|
||||
mintKvHandleId,
|
||||
normalizeKeyPrefix,
|
||||
parseKvShareAccess,
|
||||
parseKvShareLabel,
|
||||
parseKvShareRecipient,
|
||||
relativeToKvShareRoot,
|
||||
withKvShareAccess,
|
||||
} from './kvShares.js';
|
||||
import { APP_DATA_KV_METHOD_OPS } from '../permission/appDataScopes.js';
|
||||
import { isKvHandleId, kvHandleFromSubject } from '../events/subjects.js';
|
||||
|
||||
const OWNER = '2a1b0c9d-0000-4000-8000-000000000001';
|
||||
const OTHER = '2a1b0c9d-0000-4000-8000-000000000002';
|
||||
const APP = 'app-1234';
|
||||
const OTHER_APP = 'app-5678';
|
||||
|
||||
const userActor = (uuid: string): Actor =>
|
||||
({ user: { uuid }, effectiveApp: null }) as unknown as Actor;
|
||||
|
||||
const appActor = (uuid: string, appUid: string): Actor =>
|
||||
({
|
||||
user: { uuid },
|
||||
app: { uid: appUid },
|
||||
effectiveApp: { uid: appUid },
|
||||
}) as unknown as Actor;
|
||||
|
||||
describe('the share permission family', () => {
|
||||
it('names the owner, the app and the granted prefix', () => {
|
||||
expect(kvSharePermission(OWNER, APP, 'workspace:abc:')).toBe(
|
||||
`kv-share:${OWNER}:${APP}:workspace:abc`,
|
||||
);
|
||||
});
|
||||
|
||||
it('makes key segments components, so a deeper key is a descendant', () => {
|
||||
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
const deeper = kvSharePermission(
|
||||
OWNER,
|
||||
APP,
|
||||
'workspace:abc:messages:1',
|
||||
);
|
||||
|
||||
// The exploder walks parents of the string being checked, so implication
|
||||
// over key segments needs nothing of its own.
|
||||
expect(deeper.startsWith(`${granted}:`)).toBe(true);
|
||||
});
|
||||
|
||||
it('escapes a `:` inside a component rather than splitting on it', () => {
|
||||
const permission = kvSharePermission(OWNER, 'app:odd', 'cart:');
|
||||
expect(PermissionUtil.split(permission)).toEqual([
|
||||
'kv-share',
|
||||
OWNER,
|
||||
'app:odd',
|
||||
'cart',
|
||||
]);
|
||||
});
|
||||
|
||||
it('reads a withdrawn grant as covering itself and everything under it', () => {
|
||||
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
expect(kvShareGrantCovers(granted, granted)).toBe(true);
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:messages:'),
|
||||
),
|
||||
).toBe(true);
|
||||
// A sibling that merely shares a text prefix is a different region.
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OWNER, APP, 'workspace:abcdef:'),
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OTHER, APP, 'workspace:abc:'),
|
||||
),
|
||||
).toBe(false);
|
||||
// Coverage never crosses namespaces: the app uid is what keeps a
|
||||
// revoke inside the one the handle was minted in.
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
granted,
|
||||
kvSharePermission(OWNER, OTHER_APP, 'workspace:abc:messages:'),
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('never reads a withdrawn child as covering its own parent', () => {
|
||||
// Revoking the deeper of two grants must not read as covering the
|
||||
// shallower one — coverage only ever runs downward.
|
||||
const child = kvSharePermission(OWNER, APP, 'workspace:abc:messages:');
|
||||
const parent = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
expect(kvShareGrantCovers(child, parent)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('granted prefixes', () => {
|
||||
it('always ends on the key delimiter', () => {
|
||||
expect(normalizeKeyPrefix('workspace:abc')).toBe('workspace:abc:');
|
||||
expect(normalizeKeyPrefix('workspace:abc:')).toBe('workspace:abc:');
|
||||
expect(keyPrefixSegments('workspace:abc:')).toEqual([
|
||||
'workspace',
|
||||
'abc',
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['', 'the whole namespace'],
|
||||
[':', 'delimiters only'],
|
||||
['workspace:*', 'a pattern'],
|
||||
['workspace:a?c', 'a single-character pattern'],
|
||||
// Normalizing would drop the empty segment and grant `a:b:` instead —
|
||||
// a region other than the one asked for.
|
||||
['workspace::abc:', 'an empty key segment'],
|
||||
[':workspace:abc:', 'a leading empty segment'],
|
||||
// `escape_permission_component` leaves `\\` alone while escaping `:`,
|
||||
// so the stored prefix and the permission string would disagree and
|
||||
// the share would silently never fire.
|
||||
['work\\space:abc:', 'a backslash'],
|
||||
])('refuses %s (%s)', (prefix) => {
|
||||
let thrown: unknown;
|
||||
try {
|
||||
assertShareablePrefix(prefix);
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(HttpError);
|
||||
expect((thrown as HttpError).legacyCode).toBe(
|
||||
'invalid_kv_share_prefix',
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses a region too deep to fit the grant column', () => {
|
||||
expect(() =>
|
||||
assertPrefixFitsAccess(OWNER, APP, 'a'.repeat(300), 'watch'),
|
||||
).toThrow(HttpError);
|
||||
expect(
|
||||
assertPrefixFitsAccess(OWNER, APP, 'workspace:abc:', 'watch'),
|
||||
).toBe(kvSharePermission(OWNER, APP, 'workspace:abc:'));
|
||||
});
|
||||
|
||||
it('keeps the trailing delimiter optional', () => {
|
||||
expect(assertShareablePrefix('workspace:abc')).toBe('workspace:abc:');
|
||||
expect(assertShareablePrefix('workspace:abc:')).toBe('workspace:abc:');
|
||||
});
|
||||
});
|
||||
|
||||
describe('granted namespaces', () => {
|
||||
it('takes an app uid, and the app-less namespace', () => {
|
||||
expect(assertShareableAppUid(APP)).toBe(APP);
|
||||
expect(assertShareableAppUid('os-global')).toBe('os-global');
|
||||
});
|
||||
|
||||
it('refuses one past the column it is stored in', () => {
|
||||
// A longer one reaches the insert as a write the column cannot hold.
|
||||
let thrown: unknown;
|
||||
try {
|
||||
assertShareableAppUid('a'.repeat(41));
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(HttpError);
|
||||
expect((thrown as HttpError).statusCode).toBe(400);
|
||||
});
|
||||
|
||||
it('takes both shapes a real app uid comes in', () => {
|
||||
// Minted (`app-<uuidv4>`) and derived from an origin (`app-<uuidv5>`)
|
||||
// — 40 characters either way, which is the column exactly.
|
||||
const minted = `app-${uuidv4()}`;
|
||||
expect(assertShareableAppUid(minted)).toBe(minted);
|
||||
expect(minted).toHaveLength(40);
|
||||
});
|
||||
|
||||
it('refuses a handle, which shares the slot but is not a namespace', () => {
|
||||
expect(() => assertShareableAppUid(mintKvHandleId())).toThrow(
|
||||
HttpError,
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses anything else short enough to fit the column', () => {
|
||||
// The slot ends up in a subject and in a permission string, so a
|
||||
// namespace carrying a delimiter of either grammar is refused before
|
||||
// it can be composed into one.
|
||||
for (const uid of [
|
||||
'os-globals',
|
||||
'workspace:abc',
|
||||
'app',
|
||||
'app-',
|
||||
'app-a:b',
|
||||
'app-a#b',
|
||||
'v1:someone:app-x',
|
||||
'',
|
||||
])
|
||||
expect(() => assertShareableAppUid(uid)).toThrow(HttpError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('keys inside a granted region', () => {
|
||||
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
|
||||
it('are named relative to the granted root', () => {
|
||||
expect(
|
||||
relativeToKvShareRoot(permission, 'workspace:abc:messages:1'),
|
||||
).toBe('messages:1');
|
||||
});
|
||||
|
||||
it('read as outside when they are', () => {
|
||||
expect(
|
||||
relativeToKvShareRoot(permission, 'workspace:abcdef:x'),
|
||||
).toBeNull();
|
||||
expect(relativeToKvShareRoot(permission, 'workspace:abc')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('a delegation request', () => {
|
||||
const region = kvShareManagePermission(
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:'),
|
||||
);
|
||||
|
||||
it('is the manage arm of the grant it would let an app issue', () => {
|
||||
expect(region).toBe(`manage:kv-share:${OWNER}:${APP}:workspace:abc`);
|
||||
expect(assertBoundedManageGrant(region)).toBe(region);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['manage:kv-share', 'the family itself'],
|
||||
[`manage:kv-share:${OWNER}`, 'an owner and no namespace'],
|
||||
[`manage:kv-share:${OWNER}:${APP}`, 'a whole namespace'],
|
||||
])('refuses %s (%s)', (permission) => {
|
||||
let thrown: unknown;
|
||||
try {
|
||||
assertBoundedManageGrant(permission);
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(HttpError);
|
||||
expect((thrown as HttpError).legacyCode).toBe(
|
||||
'invalid_kv_share_prefix',
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves permissions from other families alone', () => {
|
||||
const unrelated = `manage:fs:${OWNER}:write`;
|
||||
expect(assertBoundedManageGrant(unrelated)).toBe(unrelated);
|
||||
// The read arm of this family is not a delegation, so it is not this
|
||||
// check's to bound.
|
||||
expect(assertBoundedManageGrant(`kv-share:${OWNER}:${APP}`)).toBeTypeOf(
|
||||
'string',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('handle ids', () => {
|
||||
it('are told apart from an app uid in the same slot', () => {
|
||||
const handle = mintKvHandleId();
|
||||
expect(isKvHandleId(handle)).toBe(true);
|
||||
expect(isKvHandleId(APP)).toBe(false);
|
||||
expect(isKvHandleId('kvh-')).toBe(false);
|
||||
});
|
||||
|
||||
it('carry nothing about the owner', () => {
|
||||
const handle = mintKvHandleId();
|
||||
expect(handle).not.toContain(OWNER);
|
||||
expect(mintKvHandleId()).not.toBe(handle);
|
||||
});
|
||||
|
||||
it('are recoverable from the subject a row stored', () => {
|
||||
const handle = mintKvHandleId();
|
||||
expect(kvHandleFromSubject(`kv:${handle}:messages:*`)).toBe(handle);
|
||||
expect(kvHandleFromSubject(`kv:${APP}:messages:*`)).toBeNull();
|
||||
expect(kvHandleFromSubject(`fs:${handle}:write`)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('the owner implicator', () => {
|
||||
const implicator = kvShareOwnerImplicator();
|
||||
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
|
||||
it('answers the share family and its manage arm', () => {
|
||||
expect(implicator.matches(permission)).toBe(true);
|
||||
expect(implicator.matches(`manage:${permission}`)).toBe(true);
|
||||
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
|
||||
});
|
||||
|
||||
it('holds for the owner named in the permission, and nobody else', () => {
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OWNER), permission }),
|
||||
).toEqual({});
|
||||
expect(
|
||||
implicator.check({
|
||||
actor: userActor(OWNER),
|
||||
permission: `manage:${permission}`,
|
||||
}),
|
||||
).toEqual({});
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OTHER), permission }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('reads the owner past a `manage` key segment', () => {
|
||||
// Stripping every `manage:` rather than the leading one would rewrite
|
||||
// the string the owner is read out of.
|
||||
const nested = kvSharePermission(OWNER, APP, 'manage:secrets:');
|
||||
expect(implicator.matches(nested)).toBe(true);
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OWNER), permission: nested }),
|
||||
).toEqual({});
|
||||
expect(
|
||||
implicator.check({ actor: userActor(OTHER), permission: nested }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('never answers for an app or a token acting through the owner', () => {
|
||||
const appActor = {
|
||||
user: { uuid: OWNER },
|
||||
app: { uid: APP },
|
||||
} as unknown as Actor;
|
||||
expect(
|
||||
implicator.check({ actor: appActor, permission }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('the app delegate implicator', () => {
|
||||
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
|
||||
|
||||
it('answers the share family, never its manage arm, nor other families', () => {
|
||||
const implicator = kvShareAppDelegateImplicator({
|
||||
userHolds: vi.fn(),
|
||||
});
|
||||
expect(implicator.matches(permission)).toBe(true);
|
||||
expect(implicator.matches(`manage:${permission}`)).toBe(false);
|
||||
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
|
||||
});
|
||||
|
||||
it("grants when the actor's own app matches the namespace app and the user holds the grant", async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = appActor(OWNER, APP);
|
||||
|
||||
await expect(implicator.check({ actor, permission })).resolves.toEqual(
|
||||
{},
|
||||
);
|
||||
|
||||
expect(userHolds).toHaveBeenCalledTimes(1);
|
||||
const [calledActor, calledPermission] = userHolds.mock.calls[0];
|
||||
expect(calledActor.app).toBeUndefined();
|
||||
expect(calledActor.effectiveApp).toBeNull();
|
||||
expect(calledActor.user).toEqual({ uuid: OWNER });
|
||||
expect(calledPermission).toBe(permission);
|
||||
});
|
||||
|
||||
it('refuses when the user does not hold the grant', async () => {
|
||||
const implicator = kvShareAppDelegateImplicator({
|
||||
userHolds: vi.fn().mockResolvedValue(false),
|
||||
});
|
||||
await expect(
|
||||
implicator.check({ actor: appActor(OWNER, APP), permission }),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("refuses when the actor's app differs from the namespace app", async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
|
||||
await expect(
|
||||
implicator.check({
|
||||
actor: appActor(OWNER, OTHER_APP),
|
||||
permission,
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses a plain user actor with no app', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
|
||||
await expect(
|
||||
implicator.check({ actor: userActor(OWNER), permission }),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses an access-token actor', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = {
|
||||
user: { uuid: OWNER },
|
||||
app: { uid: APP },
|
||||
accessToken: { uid: 't' },
|
||||
} as unknown as Actor;
|
||||
|
||||
await expect(
|
||||
implicator.check({ actor, permission }),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('asks the user once for a deep option and the ancestors its check already asked', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(false);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = appActor(OWNER, APP);
|
||||
const memo = new Map<string, unknown>();
|
||||
const deep = kvSharePermission(OWNER, APP, 'a:b:c:', 'read');
|
||||
for (const permission of [
|
||||
deep,
|
||||
kvSharePermission(OWNER, APP, 'a:b:c:', 'write'),
|
||||
kvSharePermission(OWNER, APP, 'a:b:', 'read'),
|
||||
kvSharePermission(OWNER, APP, 'a:', 'write'),
|
||||
])
|
||||
await expect(
|
||||
implicator.check({ actor, permission, memo }),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).toHaveBeenCalledTimes(1);
|
||||
|
||||
// A lower level, or a region beside it, is a question of its own.
|
||||
await implicator.check({
|
||||
actor,
|
||||
permission: kvSharePermission(OWNER, APP, 'a:b:', 'watch'),
|
||||
memo,
|
||||
});
|
||||
await implicator.check({
|
||||
actor,
|
||||
permission: kvSharePermission(OWNER, APP, 'a:x:', 'read'),
|
||||
memo,
|
||||
});
|
||||
expect(userHolds).toHaveBeenCalledTimes(3);
|
||||
|
||||
// Nothing carries over to another scan.
|
||||
await implicator.check({ actor, permission: deep });
|
||||
expect(userHolds).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
|
||||
it('refuses a permission with no key segments or no app', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const implicator = kvShareAppDelegateImplicator({ userHolds });
|
||||
const actor = appActor(OWNER, APP);
|
||||
|
||||
await expect(
|
||||
implicator.check({
|
||||
actor,
|
||||
permission: PermissionUtil.join('kv-share', OWNER, APP),
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
await expect(
|
||||
implicator.check({
|
||||
actor,
|
||||
permission: PermissionUtil.join('kv-share', OWNER),
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
expect(userHolds).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
/** The refusal a call threw, for asserting on its code. */
|
||||
const refusal = (fn: () => unknown): HttpError => {
|
||||
try {
|
||||
fn();
|
||||
} catch (err) {
|
||||
if (err instanceof HttpError) return err;
|
||||
throw err;
|
||||
}
|
||||
throw new Error('expected a refusal');
|
||||
};
|
||||
|
||||
describe('share levels', () => {
|
||||
const REGION = 'workspace:abc:';
|
||||
|
||||
it('put the level in the root, leaving watch grants as they were', () => {
|
||||
expect(kvSharePermission(OWNER, APP, REGION)).toBe(
|
||||
`kv-share:${OWNER}:${APP}:workspace:abc`,
|
||||
);
|
||||
expect(kvSharePermission(OWNER, APP, REGION, 'read')).toBe(
|
||||
`kv-share-read:${OWNER}:${APP}:workspace:abc`,
|
||||
);
|
||||
expect(kvSharePermission(OWNER, APP, REGION, 'write')).toBe(
|
||||
`kv-share-write:${OWNER}:${APP}:workspace:abc`,
|
||||
);
|
||||
});
|
||||
|
||||
it('read the level back off any root, and nothing else as the family', () => {
|
||||
for (const access of ['watch', 'read', 'write'] as const) {
|
||||
const permission = kvSharePermission(OWNER, APP, REGION, access);
|
||||
expect(isKvSharePermission(permission)).toBe(true);
|
||||
expect(kvShareAccessOf(permission)).toBe(access);
|
||||
expect(kvShareGrantPrefix(permission)).toBe(REGION);
|
||||
expect(
|
||||
relativeToKvShareRoot(permission, `${REGION}messages:1`),
|
||||
).toBe('messages:1');
|
||||
}
|
||||
expect(isKvSharePermission(`kv-shared:${OWNER}`)).toBe(false);
|
||||
expect(isKvSharePermission(`kv-share-admin:${OWNER}`)).toBe(false);
|
||||
expect(kvShareAccessOf(`fs:${OWNER}:read`)).toBeNull();
|
||||
});
|
||||
|
||||
it('swap only the root when moving a grant between levels', () => {
|
||||
const read = kvSharePermission(OWNER, APP, REGION, 'read');
|
||||
expect(withKvShareAccess(read, 'write')).toBe(
|
||||
kvSharePermission(OWNER, APP, REGION, 'write'),
|
||||
);
|
||||
expect(kvShareGrantAtEveryLevel(read)).toEqual([
|
||||
kvSharePermission(OWNER, APP, REGION, 'watch'),
|
||||
read,
|
||||
kvSharePermission(OWNER, APP, REGION, 'write'),
|
||||
]);
|
||||
});
|
||||
|
||||
it('cover a region withdrawn at one level at every level', () => {
|
||||
const writeRegion = kvSharePermission(OWNER, APP, REGION, 'write');
|
||||
const watchInside = kvSharePermission(OWNER, APP, `${REGION}messages:`);
|
||||
expect(kvShareGrantCovers(writeRegion, watchInside)).toBe(true);
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
kvSharePermission(OWNER, APP, REGION, 'read'),
|
||||
kvSharePermission(OWNER, APP, REGION, 'write'),
|
||||
),
|
||||
).toBe(true);
|
||||
// Still never upward, and never across namespaces.
|
||||
expect(kvShareGrantCovers(watchInside, writeRegion)).toBe(false);
|
||||
expect(
|
||||
kvShareGrantCovers(
|
||||
writeRegion,
|
||||
kvSharePermission(OWNER, OTHER_APP, REGION, 'write'),
|
||||
),
|
||||
).toBe(false);
|
||||
// Nothing outside the family is ever covered.
|
||||
expect(kvShareGrantCovers(writeRegion, `fs:${OWNER}:read`)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the level ladder exploder', () => {
|
||||
const exploder = kvShareLevelsExploder();
|
||||
const at = (access: 'watch' | 'read' | 'write') =>
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:', access);
|
||||
|
||||
it('lets a check at a level be answered by every level above it', async () => {
|
||||
expect(await exploder.explode({ permission: at('watch') })).toEqual([
|
||||
at('watch'),
|
||||
at('read'),
|
||||
at('write'),
|
||||
]);
|
||||
expect(await exploder.explode({ permission: at('read') })).toEqual([
|
||||
at('read'),
|
||||
at('write'),
|
||||
]);
|
||||
expect(exploder.matches(at('write'))).toBe(false);
|
||||
});
|
||||
|
||||
it('ladders a manage arm the same way', async () => {
|
||||
expect(
|
||||
await exploder.explode({ permission: `manage:${at('read')}` }),
|
||||
).toEqual([`manage:${at('read')}`, `manage:${at('write')}`]);
|
||||
});
|
||||
|
||||
it('never ladders a string with no key segment', async () => {
|
||||
for (const permission of [
|
||||
PermissionUtil.join('kv-share', OWNER, APP),
|
||||
PermissionUtil.join('manage', 'kv-share-read', OWNER, APP),
|
||||
PermissionUtil.join('kv-share', OWNER),
|
||||
'kv-share',
|
||||
])
|
||||
expect(await exploder.explode({ permission })).toEqual([
|
||||
permission,
|
||||
]);
|
||||
});
|
||||
|
||||
it('leaves other families alone', () => {
|
||||
expect(exploder.matches(`fs:${OWNER}:read`)).toBe(false);
|
||||
expect(exploder.matches(`app-data:${APP}:kv:read`)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the level a driver method needs', () => {
|
||||
it('is read for get and list, and write for every other mapped op', () => {
|
||||
expect(kvShareLevelForMethod('get')).toBe('read');
|
||||
expect(kvShareLevelForMethod('list')).toBe('read');
|
||||
for (const method of [
|
||||
'set',
|
||||
'batchPut',
|
||||
'add',
|
||||
'incr',
|
||||
'decr',
|
||||
'update',
|
||||
'del',
|
||||
'remove',
|
||||
'expire',
|
||||
'expireAt',
|
||||
])
|
||||
expect(kvShareLevelForMethod(method)).toBe('write');
|
||||
});
|
||||
|
||||
it('refuses flush and anything the op table does not name', () => {
|
||||
expect(kvShareLevelForMethod('flush')).toBeNull();
|
||||
expect(kvShareLevelForMethod('toString')).toBeNull();
|
||||
expect(kvShareLevelForMethod('somethingNew')).toBeNull();
|
||||
});
|
||||
|
||||
it('covers every method the op table knows', () => {
|
||||
for (const [method, op] of Object.entries(APP_DATA_KV_METHOD_OPS))
|
||||
expect(kvShareLevelForMethod(method) === null).toBe(op === null);
|
||||
});
|
||||
});
|
||||
|
||||
describe('share request parsing', () => {
|
||||
it('takes the three levels and a fallback, and refuses anything else', () => {
|
||||
expect(parseKvShareAccess(undefined, 'read')).toBe('read');
|
||||
expect(parseKvShareAccess(null, 'watch')).toBe('watch');
|
||||
expect(parseKvShareAccess('write', 'read')).toBe('write');
|
||||
for (const value of ['delete', 'WRITE', '', 2, {}])
|
||||
expect(
|
||||
refusal(() => parseKvShareAccess(value, 'read')).legacyCode,
|
||||
).toBe('kv_share_access_invalid');
|
||||
});
|
||||
|
||||
it('caps a prefix at what the grant column leaves for its owner, app and level', () => {
|
||||
const owner = uuidv4();
|
||||
const app = `app-${uuidv4()}`;
|
||||
// 255 less the root, the owner uuid, the app uid and two separators.
|
||||
const longest = { watch: 169, read: 164, write: 163 } as const;
|
||||
for (const access of ['watch', 'read', 'write'] as const) {
|
||||
const fits = `${'a'.repeat(longest[access] - 1)}:`;
|
||||
const over = `${'a'.repeat(longest[access])}:`;
|
||||
expect(
|
||||
assertPrefixFitsAccess(owner, app, fits, access),
|
||||
).toHaveLength(255);
|
||||
const refused = refusal(() =>
|
||||
assertPrefixFitsAccess(owner, app, over, access),
|
||||
);
|
||||
expect(refused.legacyCode).toBe('invalid_kv_share_prefix');
|
||||
expect(refused.message).toContain(`${longest[access]} characters`);
|
||||
}
|
||||
// The app-less namespace has a shorter name, so more room.
|
||||
expect(() =>
|
||||
assertPrefixFitsAccess(
|
||||
owner,
|
||||
'os-global',
|
||||
`${'a'.repeat(193)}:`,
|
||||
'write',
|
||||
),
|
||||
).not.toThrow();
|
||||
});
|
||||
|
||||
it('holds a multibyte read or write prefix to the byte cap', () => {
|
||||
const multibyte = `${'é'.repeat(KV_SHARE_PREFIX_MAX_BYTES / 2)}:`;
|
||||
expect(
|
||||
refusal(() => assertPrefixFitsAccess(OWNER, APP, multibyte, 'read'))
|
||||
.legacyCode,
|
||||
).toBe('invalid_kv_share_prefix');
|
||||
expect(() =>
|
||||
assertPrefixFitsAccess(OWNER, APP, multibyte, 'watch'),
|
||||
).not.toThrow();
|
||||
});
|
||||
|
||||
it('takes a recipient as a username or an object, never an address', () => {
|
||||
expect(parseKvShareRecipient(' alice ')).toEqual({
|
||||
username: 'alice',
|
||||
});
|
||||
expect(parseKvShareRecipient({ username: 'alice' })).toEqual({
|
||||
username: 'alice',
|
||||
});
|
||||
expect(parseKvShareRecipient({ uid: OWNER })).toEqual({ uid: OWNER });
|
||||
for (const value of [
|
||||
'[email protected]',
|
||||
{ username: 'a@b' },
|
||||
'',
|
||||
{},
|
||||
[],
|
||||
42,
|
||||
null,
|
||||
undefined,
|
||||
])
|
||||
expect(refusal(() => parseKvShareRecipient(value)).legacyCode).toBe(
|
||||
'invalid_recipient',
|
||||
);
|
||||
});
|
||||
|
||||
it('trims a label, reads empty as none, and keeps absent apart from null', () => {
|
||||
expect(parseKvShareLabel(undefined)).toBeUndefined();
|
||||
expect(parseKvShareLabel(null)).toBeNull();
|
||||
expect(parseKvShareLabel(' ')).toBeNull();
|
||||
expect(parseKvShareLabel(' Japan 2027 · from dan ')).toBe(
|
||||
'Japan 2027 · from dan',
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['a control character', 'trip\u0007'],
|
||||
['a newline', 'trip\nplans'],
|
||||
['a line separator', 'trip\u2028plans'],
|
||||
['a paragraph separator', 'trip\u2029plans'],
|
||||
['a bidi override', 'trip\u202Eplans'],
|
||||
['a bidi isolate', 'trip\u2067plans'],
|
||||
['a left-to-right mark', 'trip\u200Eplans'],
|
||||
['a right-to-left mark', 'trip\u200Fplans'],
|
||||
['an Arabic letter mark', 'trip\u061Cplans'],
|
||||
['a zero-width space', 'trip\u200Bplans'],
|
||||
['a zero-width joiner', 'trip\u200Dplans'],
|
||||
['a zero-width non-joiner', 'trip\u200Cplans'],
|
||||
['a word joiner', 'trip\u2060plans'],
|
||||
['a soft hyphen', 'trip\u00ADplans'],
|
||||
['a byte order mark', 'trip\uFEFFplans'],
|
||||
['a tag character', 'trip\u{E0041}plans'],
|
||||
['an interlinear annotation anchor', 'trip\uFFF9plans'],
|
||||
['four combining marks on one character', 'e\u0301\u0302\u0303\u0304'],
|
||||
['a non-string', 42],
|
||||
['too many characters', 'x'.repeat(129)],
|
||||
['too many bytes', '😀'.repeat(129)],
|
||||
])('refuses a label with %s', (_case, value) => {
|
||||
expect(refusal(() => parseKvShareLabel(value)).legacyCode).toBe(
|
||||
'kv_share_label_invalid',
|
||||
);
|
||||
});
|
||||
|
||||
it('holds a new share`s key segments to lowercase letters, digits, `_`, `.` and `-`', () => {
|
||||
expect(() =>
|
||||
assertNewShareSegments(['room_1.a-b', 'x9']),
|
||||
).not.toThrow();
|
||||
for (const segment of ['Room', 'café', 'a b', 'a*', 'ß', 'İ'])
|
||||
expect(
|
||||
refusal(() => assertNewShareSegments(['ok', segment]))
|
||||
.legacyCode,
|
||||
).toBe('invalid_kv_share_prefix');
|
||||
});
|
||||
|
||||
it('takes a label at the character cap', () => {
|
||||
expect(parseKvShareLabel('x'.repeat(128))).toBe('x'.repeat(128));
|
||||
});
|
||||
|
||||
it('takes up to three combining marks on a character, and emoji with a variation selector', () => {
|
||||
expect(parseKvShareLabel('e\u0301\u0302\u0303 caf\u00E9')).toBe(
|
||||
'e\u0301\u0302\u0303 caf\u00E9',
|
||||
);
|
||||
expect(parseKvShareLabel('Trip \u2764\uFE0F')).toBe(
|
||||
'Trip \u2764\uFE0F',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('delegations at every level', () => {
|
||||
it('are bounded the same way at every level', () => {
|
||||
for (const access of ['watch', 'read', 'write'] as const) {
|
||||
const region = kvShareManagePermission(
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:', access),
|
||||
);
|
||||
expect(isKvShareManagePermission(region)).toBe(true);
|
||||
expect(assertBoundedManageGrant(region)).toBe(region);
|
||||
const root = PermissionUtil.join(
|
||||
'manage',
|
||||
access === 'watch' ? 'kv-share' : `kv-share-${access}`,
|
||||
OWNER,
|
||||
APP,
|
||||
);
|
||||
expect(
|
||||
refusal(() => assertBoundedManageGrant(root)).legacyCode,
|
||||
).toBe('invalid_kv_share_prefix');
|
||||
}
|
||||
});
|
||||
|
||||
it('name the namespace root at every level for the unbounded check', () => {
|
||||
expect(
|
||||
kvShareManageNamespaceRoots(
|
||||
kvSharePermission(OWNER, APP, 'workspace:abc:', 'read'),
|
||||
),
|
||||
).toEqual([
|
||||
`manage:kv-share:${OWNER}:${APP}`,
|
||||
`manage:kv-share-read:${OWNER}:${APP}`,
|
||||
`manage:kv-share-write:${OWNER}:${APP}`,
|
||||
]);
|
||||
});
|
||||
|
||||
it('are answered for the owner at every level, and never by the app delegate', () => {
|
||||
const owner = kvShareOwnerImplicator();
|
||||
const delegate = kvShareAppDelegateImplicator({ userHolds: vi.fn() });
|
||||
for (const access of ['read', 'write'] as const) {
|
||||
const permission = kvSharePermission(
|
||||
OWNER,
|
||||
APP,
|
||||
'workspace:abc:',
|
||||
access,
|
||||
);
|
||||
expect(owner.matches(permission)).toBe(true);
|
||||
expect(owner.matches(`manage:${permission}`)).toBe(true);
|
||||
expect(
|
||||
owner.check({ actor: userActor(OWNER), permission }),
|
||||
).toEqual({});
|
||||
expect(
|
||||
owner.check({ actor: userActor(OTHER), permission }),
|
||||
).toBeUndefined();
|
||||
expect(delegate.matches(permission)).toBe(true);
|
||||
expect(delegate.matches(`manage:${permission}`)).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('let a holder`s app use a read or write grant only in its own namespace', async () => {
|
||||
const userHolds = vi.fn().mockResolvedValue(true);
|
||||
const delegate = kvShareAppDelegateImplicator({ userHolds });
|
||||
const permission = kvSharePermission(
|
||||
OWNER,
|
||||
APP,
|
||||
'workspace:abc:',
|
||||
'write',
|
||||
);
|
||||
await expect(
|
||||
delegate.check({ actor: appActor(OTHER, APP), permission }),
|
||||
).resolves.toEqual({});
|
||||
await expect(
|
||||
delegate.check({ actor: appActor(OTHER, OTHER_APP), permission }),
|
||||
).resolves.toBeUndefined();
|
||||
await expect(
|
||||
delegate.check({
|
||||
actor: appActor(OTHER, OTHER_APP),
|
||||
permission: kvSharePermission(
|
||||
OWNER,
|
||||
'os-global',
|
||||
'workspace:abc:',
|
||||
'read',
|
||||
),
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -1,628 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import {
|
||||
type Actor,
|
||||
isAppActor,
|
||||
isPlainUserActor,
|
||||
userRelatedActor,
|
||||
} from '../../core/actor.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { KV_SHARE_HANDLE_WIDTHS } from '../../stores/events/columnWidths.js';
|
||||
import {
|
||||
KV_GLOBAL_APP_KEY,
|
||||
KV_MAX_KEY_BYTES,
|
||||
} from '../../stores/systemKv/SystemKVStore.js';
|
||||
import {
|
||||
MANAGE_PERM_PREFIX,
|
||||
PERMISSION_MAX_LEN,
|
||||
} from '../permission/consts.js';
|
||||
import { APP_DATA_KV_METHOD_OPS } from '../permission/appDataScopes.js';
|
||||
import type {
|
||||
PermissionExploder,
|
||||
PermissionImplicator,
|
||||
} from '../permission/permissionUtil.js';
|
||||
import { PermissionUtil } from '../permission/permissionUtil.js';
|
||||
import {
|
||||
isKvHandleId,
|
||||
KV_HANDLE_PREFIX,
|
||||
KV_KEY_SEPARATOR,
|
||||
} from '../events/subjects.js';
|
||||
|
||||
/**
|
||||
* Letting one user watch, read or write a region of another's key-value
|
||||
* namespace.
|
||||
*
|
||||
* The grant is an ordinary user-to-user permission, which is the whole point:
|
||||
* `manage:` answers "may I delegate this", the exploder's prefix implication
|
||||
* makes a grant on `workspace:abc:` answer every key beneath it, and revocation
|
||||
* settling and the delivery re-check key off the stored string exactly as they
|
||||
* do for a shared folder. Putting the ACL on the key-value item instead would
|
||||
* take part in none of that, and would tax every write to the item besides.
|
||||
*
|
||||
* The permission names the owner, so its components are the three things a
|
||||
* shared region is, under a root that names the access level:
|
||||
*
|
||||
* kv-share:<ownerUserUuid>:<appUid>:<keySegment>... watch
|
||||
* kv-share-read:<ownerUserUuid>:<appUid>:<keySegment>... read
|
||||
* kv-share-write:<ownerUserUuid>:<appUid>:<keySegment>... write
|
||||
*
|
||||
* The level is in the root rather than a component so the permission system
|
||||
* enforces it, and so the strings `kv-share:` grants already use keep their
|
||||
* meaning.
|
||||
*
|
||||
* Key segments are components rather than one escaped blob because that is what
|
||||
* makes prefix implication work without a rule of its own: the check a deep key
|
||||
* runs is a descendant of the grant string, and the existing parent walk finds
|
||||
* it. The grantee never sees this string — the handle is the name on the wire.
|
||||
*/
|
||||
|
||||
/** Root of the cross-user key-value share namespace, at the watch level. */
|
||||
export const KV_SHARE_PERMISSION_PREFIX = 'kv-share';
|
||||
|
||||
/** Access levels, lowest first. Each implies the ones before it. */
|
||||
export const KV_SHARE_ACCESS_LEVELS = ['watch', 'read', 'write'] as const;
|
||||
export type KvShareAccess = (typeof KV_SHARE_ACCESS_LEVELS)[number];
|
||||
|
||||
const KV_SHARE_ROOTS: Record<KvShareAccess, string> = {
|
||||
watch: KV_SHARE_PERMISSION_PREFIX,
|
||||
read: `${KV_SHARE_PERMISSION_PREFIX}-read`,
|
||||
write: `${KV_SHARE_PERMISSION_PREFIX}-write`,
|
||||
};
|
||||
|
||||
/** The permission root a level is granted under. */
|
||||
export const kvShareRoot = (access: KvShareAccess): string =>
|
||||
KV_SHARE_ROOTS[access];
|
||||
|
||||
const ACCESS_BY_ROOT = new Map<string, KvShareAccess>(
|
||||
KV_SHARE_ACCESS_LEVELS.map((access) => [KV_SHARE_ROOTS[access], access]),
|
||||
);
|
||||
|
||||
/**
|
||||
* Byte ceiling on a `read` or `write` prefix, which the fixed relative-key
|
||||
* limit assumes. The grant column binds first for every prefix a new share may
|
||||
* name; this one only bites on a multibyte prefix shared before that rule.
|
||||
*/
|
||||
export const KV_SHARE_PREFIX_MAX_BYTES = 256;
|
||||
|
||||
/**
|
||||
* Longest key a call through a handle may name, relative to the handle, in
|
||||
* UTF-8 bytes: what the store's key limit leaves after the longest prefix, so
|
||||
* the limit a holder hits says nothing about how long the prefix is.
|
||||
*/
|
||||
export const KV_HANDLE_KEY_MAX_BYTES =
|
||||
KV_MAX_KEY_BYTES - KV_SHARE_PREFIX_MAX_BYTES;
|
||||
|
||||
/** Width of the `app_uid` column the handle row stores its namespace in. */
|
||||
export const KV_SHARE_APP_UID_MAX_LENGTH = KV_SHARE_HANDLE_WIDTHS.appUid.max;
|
||||
|
||||
export const mintKvHandleId = (): string =>
|
||||
`${KV_HANDLE_PREFIX}${randomUUID()}`;
|
||||
|
||||
/** The key segments a prefix contributes to a permission string. */
|
||||
export const keyPrefixSegments = (keyPrefix: string): string[] =>
|
||||
keyPrefix.split(KV_KEY_SEPARATOR).filter((segment) => segment.length > 0);
|
||||
|
||||
/**
|
||||
* A granted prefix as it is stored: always ending on the key delimiter, so
|
||||
* `keyPrefix + relative` is a whole key and the region never accidentally
|
||||
* includes the key that names it.
|
||||
*/
|
||||
export const normalizeKeyPrefix = (keyPrefix: string): string => {
|
||||
const segments = keyPrefixSegments(keyPrefix);
|
||||
return segments.length === 0
|
||||
? ''
|
||||
: `${segments.join(KV_KEY_SEPARATOR)}${KV_KEY_SEPARATOR}`;
|
||||
};
|
||||
|
||||
export const kvSharePermission = (
|
||||
ownerUserUuid: string,
|
||||
appUid: string,
|
||||
keyPrefix: string,
|
||||
access: KvShareAccess = 'watch',
|
||||
): string =>
|
||||
PermissionUtil.join(
|
||||
KV_SHARE_ROOTS[access],
|
||||
ownerUserUuid,
|
||||
appUid,
|
||||
...keyPrefixSegments(keyPrefix),
|
||||
);
|
||||
|
||||
/** The family root a permission starts with, or `null` outside the family. */
|
||||
const kvShareRootOf = (permission: string): string | null => {
|
||||
const end = permission.indexOf(':');
|
||||
const head = end === -1 ? permission : permission.slice(0, end);
|
||||
return ACCESS_BY_ROOT.has(head) ? head : null;
|
||||
};
|
||||
|
||||
export const isKvSharePermission = (permission: string): boolean =>
|
||||
kvShareRootOf(permission) !== null;
|
||||
|
||||
/** The access level a share grant carries, or `null` outside the family. */
|
||||
export const kvShareAccessOf = (permission: string): KvShareAccess | null => {
|
||||
const root = kvShareRootOf(permission);
|
||||
return root === null ? null : (ACCESS_BY_ROOT.get(root) ?? null);
|
||||
};
|
||||
|
||||
/** The same grant at another level: only the root changes. */
|
||||
export const withKvShareAccess = (
|
||||
permission: string,
|
||||
access: KvShareAccess,
|
||||
): string => {
|
||||
const root = kvShareRootOf(permission);
|
||||
if (root === null) return permission;
|
||||
return `${KV_SHARE_ROOTS[access]}${permission.slice(root.length)}`;
|
||||
};
|
||||
|
||||
/** The grant at every level, for withdrawing a region whatever it was. */
|
||||
export const kvShareGrantAtEveryLevel = (permission: string): string[] =>
|
||||
KV_SHARE_ACCESS_LEVELS.map((access) =>
|
||||
withKvShareAccess(permission, access),
|
||||
);
|
||||
|
||||
/**
|
||||
* Whether withdrawing `revoked` takes `held` with it. Prefix implication read
|
||||
* backwards: a grant answers every check at or beneath it, so withdrawing it
|
||||
* puts every one of those in question. Compared below the root, because a
|
||||
* region withdrawn at one level is in question at every level.
|
||||
*/
|
||||
export const kvShareGrantCovers = (revoked: string, held: string): boolean => {
|
||||
const revokedRoot = kvShareRootOf(revoked);
|
||||
const heldRoot = kvShareRootOf(held);
|
||||
if (revokedRoot === null || heldRoot === null) return false;
|
||||
const region = revoked.slice(revokedRoot.length);
|
||||
const inside = held.slice(heldRoot.length);
|
||||
return inside === region || inside.startsWith(`${region}:`);
|
||||
};
|
||||
|
||||
/**
|
||||
* The level a key-value driver method needs through a share, or `null` for a
|
||||
* method no share reaches. Derived from the cross-app op table, so a method
|
||||
* missing from it fails closed.
|
||||
*/
|
||||
export const kvShareLevelForMethod = (method: string): KvShareAccess | null => {
|
||||
if (!Object.prototype.hasOwnProperty.call(APP_DATA_KV_METHOD_OPS, method))
|
||||
return null;
|
||||
const op = APP_DATA_KV_METHOD_OPS[method];
|
||||
if (op === null) return null;
|
||||
return op === 'get' || op === 'list' ? 'read' : 'write';
|
||||
};
|
||||
|
||||
/** The granted root a share grant names, read back as a key prefix. */
|
||||
export const kvShareGrantPrefix = (permission: string): string =>
|
||||
normalizeKeyPrefix(
|
||||
PermissionUtil.split(permission).slice(3).join(KV_KEY_SEPARATOR),
|
||||
);
|
||||
|
||||
/**
|
||||
* A key inside a granted region, named the way its holder addresses it, or
|
||||
* `null` for one outside. The handle is the granted root, so everything the
|
||||
* holder is shown is relative to it — an absolute key would name a namespace
|
||||
* they cannot address and were never told about.
|
||||
*/
|
||||
export const relativeToKvShareRoot = (
|
||||
permission: string,
|
||||
key: string,
|
||||
): string | null => {
|
||||
const prefix = kvShareGrantPrefix(permission);
|
||||
if (!prefix || !key.startsWith(prefix)) return null;
|
||||
return key.slice(prefix.length);
|
||||
};
|
||||
|
||||
/**
|
||||
* `kv.shareHandles` is off: no `read` or `write` shares, and no calls through
|
||||
* one.
|
||||
*/
|
||||
export const kvHandlesDisabled = (): HttpError =>
|
||||
new HttpError(
|
||||
403,
|
||||
'Sharing key-value data for reading or writing is not available',
|
||||
{ legacyCode: 'kv_handles_disabled' },
|
||||
);
|
||||
|
||||
// -- Minting input ----------------------------------------------------
|
||||
|
||||
export const invalidPrefix = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'invalid_kv_share_prefix' });
|
||||
|
||||
const invalidAccess = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'kv_share_access_invalid' });
|
||||
|
||||
/** The level a mint asks for, `fallback` when it names none. */
|
||||
export const parseKvShareAccess = (
|
||||
value: unknown,
|
||||
fallback: KvShareAccess,
|
||||
): KvShareAccess => {
|
||||
if (value === undefined || value === null) return fallback;
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
!(KV_SHARE_ACCESS_LEVELS as readonly string[]).includes(value)
|
||||
)
|
||||
throw invalidAccess('`access` must be `watch`, `read` or `write`');
|
||||
return value as KvShareAccess;
|
||||
};
|
||||
|
||||
/**
|
||||
* The grant a share of `keyPrefix` at `access` would issue, refused with
|
||||
* `invalid_kv_share_prefix` when it would not fit the permission column. The
|
||||
* room left depends on the owner uuid, the app uid and the level's root, so the
|
||||
* longest prefix differs per namespace and level.
|
||||
*/
|
||||
export const assertPrefixFitsAccess = (
|
||||
ownerUuid: string,
|
||||
appUid: string,
|
||||
keyPrefix: string,
|
||||
access: KvShareAccess,
|
||||
): string => {
|
||||
const permission = kvSharePermission(ownerUuid, appUid, keyPrefix, access);
|
||||
const prefix = normalizeKeyPrefix(keyPrefix);
|
||||
const room = PERMISSION_MAX_LEN - (permission.length - prefix.length);
|
||||
const overBytes =
|
||||
access !== 'watch' &&
|
||||
Buffer.byteLength(prefix, 'utf8') > KV_SHARE_PREFIX_MAX_BYTES;
|
||||
if (permission.length > PERMISSION_MAX_LEN || overBytes)
|
||||
throw invalidPrefix(
|
||||
`With \`${access}\` access, a share prefix in this namespace may be at most ${room} characters` +
|
||||
(access === 'watch'
|
||||
? ''
|
||||
: ` and ${KV_SHARE_PREFIX_MAX_BYTES} bytes`),
|
||||
);
|
||||
return permission;
|
||||
};
|
||||
|
||||
const invalidRecipient = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'invalid_recipient' });
|
||||
|
||||
/** Who a share is for, as a username or a user uuid. */
|
||||
export interface KvShareRecipient {
|
||||
username?: string;
|
||||
uid?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A recipient as `POST /kv/shares` takes it: a username, `{ username }` or `{
|
||||
* uid }`. An address is refused rather than looked up, so email recipients can
|
||||
* later mean what they mean for file shares.
|
||||
*/
|
||||
export const parseKvShareRecipient = (value: unknown): KvShareRecipient => {
|
||||
const named = (username: unknown): KvShareRecipient => {
|
||||
if (typeof username !== 'string' || username.trim() === '')
|
||||
throw invalidRecipient('`recipient` must name a user');
|
||||
if (username.includes('@'))
|
||||
throw invalidRecipient(
|
||||
'`recipient` must be a username, not an email address',
|
||||
);
|
||||
return { username: username.trim() };
|
||||
};
|
||||
if (typeof value === 'string') return named(value);
|
||||
if (value && typeof value === 'object' && !Array.isArray(value)) {
|
||||
const { username, uid } = value as Record<string, unknown>;
|
||||
if (username !== undefined) return named(username);
|
||||
if (typeof uid === 'string' && uid.trim() !== '')
|
||||
return { uid: uid.trim() };
|
||||
}
|
||||
throw invalidRecipient(
|
||||
'`recipient` must be a username, `{ username }` or `{ uid }`',
|
||||
);
|
||||
};
|
||||
|
||||
const invalidLabel = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'kv_share_label_invalid' });
|
||||
|
||||
export const KV_SHARE_LABEL_MAX_CHARS = 128;
|
||||
export const KV_SHARE_LABEL_MAX_BYTES = 512;
|
||||
|
||||
/**
|
||||
* Control and format characters (bidi marks, overrides and isolates, zero-width
|
||||
* characters, BOM, tags) and line or paragraph separators.
|
||||
*/
|
||||
const LABEL_FORBIDDEN = /[\p{Cc}\p{Cf}\p{Zl}\p{Zp}]/u;
|
||||
|
||||
/** Combining marks a label may stack on one character. */
|
||||
export const KV_SHARE_LABEL_MAX_COMBINING_MARKS = 3;
|
||||
const LABEL_MARK_RUN = new RegExp(
|
||||
`\\p{M}{${KV_SHARE_LABEL_MAX_COMBINING_MARKS + 1},}`,
|
||||
'u',
|
||||
);
|
||||
|
||||
/**
|
||||
* An owner-set label. `undefined` leaves the stored one alone, `null` clears
|
||||
* it, and a string that trims to nothing reads as `null`.
|
||||
*/
|
||||
export const parseKvShareLabel = (
|
||||
value: unknown,
|
||||
): string | null | undefined => {
|
||||
if (value === undefined) return undefined;
|
||||
if (value === null) return null;
|
||||
if (typeof value !== 'string')
|
||||
throw invalidLabel('`label` must be a string');
|
||||
const label = value.trim();
|
||||
if (label === '') return null;
|
||||
if (LABEL_FORBIDDEN.test(label))
|
||||
throw invalidLabel(
|
||||
'`label` may not contain control, formatting, line-break or text-direction characters',
|
||||
);
|
||||
if (LABEL_MARK_RUN.test(label))
|
||||
throw invalidLabel(
|
||||
`\`label\` may not stack more than ${KV_SHARE_LABEL_MAX_COMBINING_MARKS} combining marks on one character`,
|
||||
);
|
||||
if (
|
||||
[...label].length > KV_SHARE_LABEL_MAX_CHARS ||
|
||||
Buffer.byteLength(label, 'utf8') > KV_SHARE_LABEL_MAX_BYTES
|
||||
)
|
||||
throw invalidLabel(
|
||||
`\`label\` may not exceed ${KV_SHARE_LABEL_MAX_CHARS} characters or ${KV_SHARE_LABEL_MAX_BYTES} bytes`,
|
||||
);
|
||||
return label;
|
||||
};
|
||||
|
||||
export const invalidAppUid = (message: string): HttpError =>
|
||||
new HttpError(400, message, { legacyCode: 'bad_request' });
|
||||
|
||||
/**
|
||||
* The region a handle may be minted on. A pattern is refused because a handle
|
||||
* names a region rather than selecting within one, and the namespace root is
|
||||
* refused because a handle over everything the app holds is not a bounded
|
||||
* capability — it is the app's data, and sharing that is a different decision
|
||||
* with a different consent.
|
||||
*/
|
||||
export const assertShareablePrefix = (keyPrefix: unknown): string => {
|
||||
if (typeof keyPrefix !== 'string')
|
||||
throw invalidPrefix('`prefix` must be a string');
|
||||
if (keyPrefix.includes('*') || keyPrefix.includes('?'))
|
||||
throw invalidPrefix('A share prefix is a key prefix, not a pattern');
|
||||
// `escape_permission_component` escapes `:` as `\C` but leaves `\` alone,
|
||||
// so a prefix carrying one desynchronizes the stored `key_prefix` from the
|
||||
// permission string and the share silently never fires.
|
||||
if (keyPrefix.includes('\\'))
|
||||
throw invalidPrefix('A share prefix may not contain a backslash');
|
||||
// Normalizing drops empty segments, so `a::b:` would silently become a
|
||||
// grant on `a:b:` — a region other than the one asked for. Refused rather
|
||||
// than rewritten; only the trailing delimiter is optional.
|
||||
const written = keyPrefix.split(KV_KEY_SEPARATOR);
|
||||
if (written[written.length - 1] === '') written.pop();
|
||||
if (written.some((segment) => segment === ''))
|
||||
throw invalidPrefix('A share prefix may not have an empty key segment');
|
||||
|
||||
const normalized = normalizeKeyPrefix(keyPrefix);
|
||||
if (normalized === '')
|
||||
throw invalidPrefix('A share prefix may not be the whole namespace');
|
||||
return normalized;
|
||||
};
|
||||
|
||||
const NEW_SHARE_SEGMENT = /^[a-z0-9_.-]+$/;
|
||||
|
||||
/**
|
||||
* The key segments a new share or share consent may name. One spelling per
|
||||
* region, so no two grants differ only in case or accents, which some databases
|
||||
* compare as equal. Existing shares are not held to it.
|
||||
*/
|
||||
export const assertNewShareSegments = (segments: readonly string[]): void => {
|
||||
if (segments.some((segment) => !NEW_SHARE_SEGMENT.test(segment)))
|
||||
throw invalidPrefix(
|
||||
'A share prefix may only contain lowercase letters (a-z), digits, `_`, `.` and `-` between its `:` separators',
|
||||
);
|
||||
};
|
||||
|
||||
/** A new share over a prefix that an existing share spells differently. */
|
||||
export const kvSharePrefixCaseConflict = (): HttpError =>
|
||||
invalidPrefix(
|
||||
'This user already has a share of a prefix that differs from this one only in letter case',
|
||||
);
|
||||
|
||||
/**
|
||||
* The shape every app uid takes, minted or derived from an origin. Narrow on
|
||||
* purpose: `:` and `#` delimit the anchor token and permission grammars, so
|
||||
* excluding them matters as much as excluding the handle prefix does.
|
||||
*/
|
||||
export const APP_UID_SHAPE = /^app-[A-Za-z0-9_-]{1,64}$/;
|
||||
|
||||
/** Fixed namespaces with no app of their own. */
|
||||
const FIXED_KV_NAMESPACES: readonly string[] = [KV_GLOBAL_APP_KEY];
|
||||
|
||||
/**
|
||||
* The namespace a handle may be minted over: the caller's own app slot, or the
|
||||
* app-less one. Bounded by the column that stores it, shaped like a real app
|
||||
* uid or one of the platform's fixed namespaces, and never handle-shaped, since
|
||||
* the two share the app slot of a `kv:` subject.
|
||||
*/
|
||||
export const assertShareableAppUid = (appUid: string): string => {
|
||||
if (appUid.length > KV_SHARE_APP_UID_MAX_LENGTH)
|
||||
throw invalidAppUid(
|
||||
`\`appUid\` may not exceed ${KV_SHARE_APP_UID_MAX_LENGTH} characters`,
|
||||
);
|
||||
if (isKvHandleId(appUid))
|
||||
throw invalidAppUid('A share handle does not name a namespace');
|
||||
if (!APP_UID_SHAPE.test(appUid) && !FIXED_KV_NAMESPACES.includes(appUid))
|
||||
throw invalidAppUid('`appUid` does not name a namespace');
|
||||
return appUid;
|
||||
};
|
||||
|
||||
// -- Delegation -------------------------------------------------------
|
||||
|
||||
/** The delegation arm of a share grant: authority to hand the region out. */
|
||||
export const kvShareManagePermission = (permission: string): string =>
|
||||
PermissionUtil.join(
|
||||
MANAGE_PERM_PREFIX,
|
||||
...PermissionUtil.split(permission),
|
||||
);
|
||||
|
||||
export const isKvShareManagePermission = (permission: string): boolean =>
|
||||
PermissionUtil.isManage(permission) &&
|
||||
isKvSharePermission(permission.slice(MANAGE_PERM_PREFIX.length + 1));
|
||||
|
||||
/**
|
||||
* A delegation must name a subtree. Consent on the namespace root reads as "let
|
||||
* this app hand out anything it has stored for you", which is not a bounded
|
||||
* capability and so is not something a prompt can put to a user.
|
||||
*/
|
||||
export const assertBoundedManageGrant = (permission: string): string => {
|
||||
if (!isKvShareManagePermission(permission)) return permission;
|
||||
const [, , owner, appUid, ...segments] = PermissionUtil.split(permission);
|
||||
if (!owner || !appUid || segments.length === 0)
|
||||
throw invalidPrefix(
|
||||
'A share delegation must name a region, not the whole namespace',
|
||||
);
|
||||
return permission;
|
||||
};
|
||||
|
||||
/**
|
||||
* The unbounded `manage:` grants, one per level, that a bounded delegation's
|
||||
* own permission sits in. The consent surface refuses to ever write these rows,
|
||||
* but `canManagePermission` walks ancestors to decide "may I delegate this", so
|
||||
* one that exists by any other path would silently authorize a mint anywhere in
|
||||
* the namespace. This is what lets the mint path notice it regardless.
|
||||
*/
|
||||
export const kvShareManageNamespaceRoots = (permission: string): string[] => {
|
||||
const [, owner, appUid] = PermissionUtil.split(permission);
|
||||
return KV_SHARE_ACCESS_LEVELS.map((access) =>
|
||||
PermissionUtil.join(
|
||||
MANAGE_PERM_PREFIX,
|
||||
KV_SHARE_ROOTS[access],
|
||||
owner,
|
||||
appUid,
|
||||
),
|
||||
);
|
||||
};
|
||||
|
||||
// -- Permission rules -------------------------------------------------
|
||||
|
||||
/**
|
||||
* The level ladder: a check at one level is also answered by a grant at any
|
||||
* level above it, `manage:` arms included. Only a string with at least one key
|
||||
* segment is laddered, so no level of a namespace-root grant answers another.
|
||||
*/
|
||||
export const kvShareLevelsExploder = (): PermissionExploder => ({
|
||||
id: 'kv-share-levels',
|
||||
matches: (permission: string): boolean => {
|
||||
const access = kvShareAccessOf(
|
||||
PermissionUtil.stripManageArms(permission),
|
||||
);
|
||||
return access !== null && access !== 'write';
|
||||
},
|
||||
explode: ({ permission }): string[] => {
|
||||
const grant = PermissionUtil.stripManageArms(permission);
|
||||
const arms = permission.slice(0, permission.length - grant.length);
|
||||
const access = kvShareAccessOf(grant);
|
||||
// Owner, app, and at least one key segment after the root.
|
||||
if (access === null || grant.split(':').length < 4) return [permission];
|
||||
const above = KV_SHARE_ACCESS_LEVELS.slice(
|
||||
KV_SHARE_ACCESS_LEVELS.indexOf(access) + 1,
|
||||
);
|
||||
return [
|
||||
permission,
|
||||
...above.map(
|
||||
(level) => `${arms}${withKvShareAccess(grant, level)}`,
|
||||
),
|
||||
];
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
* The permission a `manage:` arm delegates over. Only the leading component is
|
||||
* stripped: a key segment may itself be `manage`, and taking those out would
|
||||
* change which string the owner is read from.
|
||||
*/
|
||||
const withoutManageArm = (permission: string): string =>
|
||||
permission.startsWith(`${MANAGE_PERM_PREFIX}:`)
|
||||
? permission.slice(MANAGE_PERM_PREFIX.length + 1)
|
||||
: permission;
|
||||
|
||||
/**
|
||||
* Owning the namespace is holding every share grant over it, and being able to
|
||||
* issue them. Without this the owner cannot mint a handle on their own data:
|
||||
* `grantUserUserPermission` asks `canManagePermission` first, and there is no
|
||||
* row anywhere saying a user may manage what is already theirs.
|
||||
*
|
||||
* Restricted to plain user actors, as the filesystem's `is-owner` is. An app
|
||||
* disposing of a region of its user's namespace is delegation, which is the
|
||||
* `manage:` grant's job rather than this one's.
|
||||
*/
|
||||
export const kvShareOwnerImplicator = (): PermissionImplicator => ({
|
||||
id: 'kv-share-is-owner',
|
||||
shortcut: true,
|
||||
matches: (permission: string): boolean =>
|
||||
isKvSharePermission(withoutManageArm(permission)),
|
||||
check: ({ actor, permission }): unknown => {
|
||||
if (!isPlainUserActor(actor)) return undefined;
|
||||
const uuid = actor.user?.uuid;
|
||||
if (!uuid) return undefined;
|
||||
|
||||
const owner = PermissionUtil.split(withoutManageArm(permission))[1];
|
||||
return owner === uuid ? {} : undefined;
|
||||
},
|
||||
});
|
||||
|
||||
/** Scan memo key for the grants a user check already refused. */
|
||||
const DELEGATE_REFUSED = 'kv-share-app-delegate:refused';
|
||||
|
||||
/**
|
||||
* Whether a check of `asked` also asks `option`: `option` is `asked` or an
|
||||
* ancestor of it, at the same level or above.
|
||||
*/
|
||||
const kvShareCheckReaches = (asked: string, option: string): boolean => {
|
||||
const askedAccess = kvShareAccessOf(asked);
|
||||
const optionAccess = kvShareAccessOf(option);
|
||||
if (askedAccess === null || optionAccess === null) return false;
|
||||
return (
|
||||
KV_SHARE_ACCESS_LEVELS.indexOf(optionAccess) >=
|
||||
KV_SHARE_ACCESS_LEVELS.indexOf(askedAccess) &&
|
||||
kvShareGrantCovers(option, asked)
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Letting an app-under-user actor exercise a share grant its user holds, at any
|
||||
* level, bounded to the app the grant's namespace names.
|
||||
*
|
||||
* Only the grant itself is matched — never `manage:kv-share…` — because
|
||||
* answering the manage arm here would read as the unbounded namespace-root
|
||||
* delegation and break minting for everyone (see
|
||||
* `kvShareManageNamespaceRoots`). The namespace-app check keeps one app from
|
||||
* reading a region shared with another app of the same user.
|
||||
*/
|
||||
export const kvShareAppDelegateImplicator = (deps: {
|
||||
userHolds: (actor: Actor, permission: string) => Promise<boolean>;
|
||||
}): PermissionImplicator => ({
|
||||
id: 'kv-share-app-delegate',
|
||||
shortcut: true,
|
||||
matches: (permission: string): boolean => isKvSharePermission(permission),
|
||||
check: async ({ actor, permission, memo }): Promise<unknown> => {
|
||||
if (!isAppActor(actor)) return undefined;
|
||||
const app = actor.effectiveApp;
|
||||
if (!app?.uid) return undefined;
|
||||
|
||||
const [, owner, namespaceApp, ...segments] =
|
||||
PermissionUtil.split(permission);
|
||||
if (!owner || !namespaceApp || segments.length === 0) return undefined;
|
||||
if (namespaceApp !== app.uid) return undefined;
|
||||
|
||||
// A scan offers the deepest option first, and the user check of one
|
||||
// also asks its ancestors at its level or above, so its refusal
|
||||
// answers those without asking again.
|
||||
const refused = memo?.get(DELEGATE_REFUSED) as string[] | undefined;
|
||||
if (refused?.some((asked) => kvShareCheckReaches(asked, permission)))
|
||||
return undefined;
|
||||
if (await deps.userHolds(userRelatedActor(actor), permission))
|
||||
return {};
|
||||
memo?.set(DELEGATE_REFUSED, [...(refused ?? []), permission]);
|
||||
return undefined;
|
||||
},
|
||||
});
|
||||
@@ -223,23 +223,6 @@ export const actorOnPaidPlan = async (
|
||||
return subscriptionSatisfies(subscription.id, true);
|
||||
};
|
||||
|
||||
/**
|
||||
* The actor's plan id, for tiered limits. `null` with no metering to ask, or
|
||||
* when the lookup fails, which the tier tables read as their default.
|
||||
*/
|
||||
export const actorPlanId = async (
|
||||
metering: SubscriptionMetering | undefined,
|
||||
actor: Actor,
|
||||
): Promise<string | null> => {
|
||||
if (!metering) return null;
|
||||
try {
|
||||
return (await metering.getActorSubscription(actor)).id;
|
||||
} catch (err) {
|
||||
console.warn('[metering] could not resolve a plan', err);
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Where a plan gate sits, so a `subscription.gate.<surface>` listener can waive
|
||||
* it for a call it recognizes. `surface` is the key segment after
|
||||
|
||||
@@ -28,8 +28,6 @@ import {
|
||||
} from './notificationSocket.js';
|
||||
import {
|
||||
findNotificationType,
|
||||
inAccountMailbox,
|
||||
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
|
||||
resolveNotificationWrite,
|
||||
} from './notificationTypes.js';
|
||||
import type { NotificationTypeName } from './notificationTypes.js';
|
||||
@@ -245,7 +243,6 @@ export class NotificationService extends PuterService {
|
||||
* message the GUI has always listened for.
|
||||
*/
|
||||
deliverOverSocket(context: NotifEventContext): void {
|
||||
if (!inAccountMailbox(context.type)) return;
|
||||
this.#socket().message(
|
||||
context.userId,
|
||||
context.uid,
|
||||
@@ -302,9 +299,7 @@ export class NotificationService extends PuterService {
|
||||
payload: Record<string, unknown>,
|
||||
scope: { type: string; audience: string; appUid: string | null },
|
||||
): Promise<void> {
|
||||
const desktop = inAccountMailbox(scope.type);
|
||||
if (!notificationsFoldInEnabled(this.config)) {
|
||||
if (!desktop) return;
|
||||
this.#socket().message(userId, uid, payload);
|
||||
this.#markDelivered(uid, userId);
|
||||
return;
|
||||
@@ -314,7 +309,6 @@ export class NotificationService extends PuterService {
|
||||
// looked up only on this path, and user rows are cached.
|
||||
const user = await this.stores.user.getById(userId);
|
||||
if (!user?.uuid) {
|
||||
if (!desktop) return;
|
||||
this.#socket().message(userId, uid, payload);
|
||||
this.#markDelivered(uid, userId);
|
||||
return;
|
||||
@@ -338,7 +332,7 @@ export class NotificationService extends PuterService {
|
||||
},
|
||||
{},
|
||||
);
|
||||
if (desktop) this.#markDelivered(uid, userId);
|
||||
this.#markDelivered(uid, userId);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -378,7 +372,6 @@ export class NotificationService extends PuterService {
|
||||
const rows = await this.stores.notification.listByUserId(userId, {
|
||||
filter: 'unseen',
|
||||
limit: UNREAD_REPLAY_LIMIT,
|
||||
excludeTypes: OUTSIDE_ACCOUNT_MAILBOX_TYPES,
|
||||
});
|
||||
if (rows.length === 0) return;
|
||||
|
||||
|
||||
@@ -20,9 +20,7 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
NOTIFICATION_TYPES,
|
||||
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
|
||||
findNotificationType,
|
||||
inAccountMailbox,
|
||||
resolveNotificationWrite,
|
||||
} from './notificationTypes.js';
|
||||
|
||||
@@ -130,28 +128,3 @@ describe('resolveNotificationWrite', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('types kept off the account mailbox', () => {
|
||||
it('are only the ones an app alone has a use for', () => {
|
||||
expect(OUTSIDE_ACCOUNT_MAILBOX_TYPES).toEqual(['kv.share.received']);
|
||||
for (const entry of NOTIFICATION_TYPES)
|
||||
expect(inAccountMailbox(entry.type)).toBe(
|
||||
entry.type !== 'kv.share.received',
|
||||
);
|
||||
// An unclassified legacy row still belongs on the desktop.
|
||||
expect(inAccountMailbox('')).toBe(true);
|
||||
expect(inAccountMailbox(undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it('include the key-value share notice, which always names its app', () => {
|
||||
expect(findNotificationType('kv.share.received')).toMatchObject({
|
||||
audience: 'app-user',
|
||||
appScoped: true,
|
||||
groupable: false,
|
||||
accountMailbox: false,
|
||||
});
|
||||
expect(() =>
|
||||
resolveNotificationWrite('kv.share.received', null),
|
||||
).toThrow(/requires an app uid/);
|
||||
});
|
||||
});
|
||||
Loaded 100 of 336 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user