mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
20
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fec1d1916f | ||
|
|
da369039e0 | ||
|
|
64c3b3c0ce | ||
|
|
e9a4c7a461 | ||
|
|
be9cd9eb89 | ||
|
|
8b64cc5076 | ||
|
|
d3869b7846 | ||
|
|
91230697cd | ||
|
|
56e4856518 | ||
|
|
8acd73675c | ||
|
|
fc4af97885 | ||
|
|
7cec710014 | ||
|
|
48cd4ea355 | ||
|
|
a110be0340 | ||
|
|
93798cba93 | ||
|
|
848e336455 | ||
|
|
a9a8df6409 | ||
|
|
0a8b37ae56 | ||
|
|
23782dec12 | ||
|
|
6a6cf78c03 |
No files matched your search
@@ -161,11 +161,38 @@ An overview page answers "what is this API and is it what I need?", and a functi
|
||||
|
||||
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
|
||||
|
||||
### Function page layout
|
||||
|
||||
A function page opens with one sentence that says what the function does, in its own paragraph, such as "Given a prompt, generate an image using AI." Put any more detail in the paragraphs after it.
|
||||
|
||||
After the intro, a function page has these `##` sections, in this order:
|
||||
|
||||
1. Syntax
|
||||
2. Parameters
|
||||
3. Return value
|
||||
4. Errors, if the function has any
|
||||
5. Sections for specific topics, if the page needs them, with `## Model-specific features` last
|
||||
6. Examples
|
||||
|
||||
[chat.md](src/docs/src/AI/chat.md) follows this layout.
|
||||
|
||||
- **Options as a table.** Document an options object as a table with Option, Type, and Description columns. Give each option its own row, even when two options go together, such as `width` and `height`. Order the rows by how often developers use them, most common first, without group rows. In chat.md that's the core options, then reasoning, tools, response handling, request control, and output.
|
||||
- **Common options in the table, model-specific behavior in its own section.** A page whose function works with many models or providers, such as [chat.md](src/docs/src/AI/chat.md) and [txt2img.md](src/docs/src/AI/txt2img.md), splits its options this way:
|
||||
- **The options table** under Parameters lists the options that work with every model, and describes each one the way it works for every model.
|
||||
- **`## Model-specific features`**, the last section before Examples, covers everything else. It has a `###` for each model family or provider, named after it, such as "OpenAI models" or "Cloudflare models". Under it go the options only some models accept, and the common options a model handles differently, such as a narrower range, other allowed values, or an option it ignores.
|
||||
- **Write there only what's different.** A `###` lists an option only when that model or provider adds it or handles it differently. An option that works the same everywhere stays out, even if the provider supports it, and a model or provider with nothing different gets no `###`.
|
||||
- **Model-specific features is the only place for model-specific details.** Every limit, format, allowed value, default, price, or quirk that applies to some models goes there, in that model's or provider's `###`, and nowhere else. The options table and topic sections (such as Image inputs or Video input) describe what's true for every model and link to Model-specific features for the rest. Don't add a table or sentence there that compares providers or repeats a model's limits. Each copy is another source of truth to keep in sync, and copies drift when a model changes.
|
||||
- **Link the two.** Directly under the options table, put `Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).` A row whose option behaves differently on some models can also end with `See [Model-specific features](#model-specific-features).`
|
||||
- **A long topic gets its own `##` section.** When explaining an option or feature takes more than a sentence or two, such as function calling or output dimensions, give it a section between Errors and Examples and link to it from the options table. This keeps the reference sections short.
|
||||
- **Usage patterns are examples.** A way of using the API, such as stopping a stream, goes in a runnable example that the relevant section links to ("See the [stop a stream example](/playground/ai-chat-stop-stream/)."), not in a section of its own.
|
||||
- **Each example has a `###` heading** under `## Examples`, so every example shows in the table of contents.
|
||||
|
||||
### Writing docs
|
||||
|
||||
These apply to reference pages and recipes alike.
|
||||
|
||||
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
|
||||
- **Document Puter as it runs on puter.com.** The docs are for developers building on production Puter. Leave out behavior that only happens on a self-hosted or differently configured server, such as a provider that isn't set up there.
|
||||
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
|
||||
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
|
||||
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
|
||||
|
||||
@@ -63,6 +63,9 @@ store:
|
||||
sealed cursor: the last `id` it returned, and the last `(replaced_at, id)`
|
||||
among rows up to that id. Each is its own indexed read, merged into one
|
||||
page, so a row replaced after a client read it comes back from that cursor.
|
||||
`from=end` with no cursor answers an empty page whose cursor holds the
|
||||
slice's newest `id` and the slice's latest `(replaced_at, id)` as the
|
||||
replacement mark.
|
||||
|
||||
Cursors that carry a sort also pin it: a request that passes a cursor plus a
|
||||
conflicting sort is rejected with 400.
|
||||
|
||||
@@ -318,3 +318,13 @@ describe('AbstractDatabaseClient — SQL generation', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
it('builds an insert for batchWrite and shares it with insert', async () => {
|
||||
const client = new FixtureDatabaseClient([]);
|
||||
const entry = client.insertEntry('widget', { name: "reader's", enabled: true });
|
||||
expect(entry.values).toEqual(["reader's", true]);
|
||||
expect(entry.statement).toContain('VALUES (?, ?)');
|
||||
await client.insert('widget', { name: "reader's", enabled: true });
|
||||
expect(client.writes).toEqual([{ query: entry.statement, params: entry.values }]);
|
||||
});
|
||||
@@ -110,14 +110,23 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
tableName: string,
|
||||
data: Record<string, unknown>,
|
||||
): Promise<WriteResult> {
|
||||
const entry = this.insertEntry(tableName, data);
|
||||
return this.write(
|
||||
entry.statement + this.returningIdClause(),
|
||||
entry.values,
|
||||
);
|
||||
}
|
||||
|
||||
/** An insert that can join other writes in a transaction. */
|
||||
insertEntry(tableName: string, data: Record<string, unknown>): BatchEntry {
|
||||
const cols = Object.keys(data);
|
||||
const values = Object.values(data);
|
||||
const sql =
|
||||
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
|
||||
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
|
||||
`VALUES (${cols.map(() => '?').join(', ')})` +
|
||||
this.returningIdClause();
|
||||
return this.write(sql, values);
|
||||
return {
|
||||
statement:
|
||||
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
|
||||
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
|
||||
`VALUES (${cols.map(() => '?').join(', ')})`,
|
||||
values: Object.values(data),
|
||||
};
|
||||
}
|
||||
|
||||
/** Whether `read()` goes to a separate replica rather than the primary. */
|
||||
|
||||
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
|
||||
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
|
||||
|
||||
/** Highest schema version the migration table can reach. */
|
||||
const CURRENT_SCHEMA_VERSION = 96;
|
||||
const CURRENT_SCHEMA_VERSION = 98;
|
||||
|
||||
/**
|
||||
* These suites migrate real files on disk. Idle they finish in well under a
|
||||
@@ -735,6 +735,74 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
|
||||
expect(columns.map((c) => String(c.name))).toEqual(['send_key']);
|
||||
});
|
||||
|
||||
it('creates event_user_handlers with its unique and holder indexes', async () => {
|
||||
const columns = await client.read(
|
||||
'PRAGMA table_info(`event_user_handlers`)',
|
||||
);
|
||||
expect(columns.map((c) => String(c.name))).toEqual([
|
||||
'id',
|
||||
'app_uid',
|
||||
'holder_user_id',
|
||||
'name',
|
||||
'source',
|
||||
'source_hash',
|
||||
'created_at',
|
||||
'updated_at',
|
||||
'scope_since',
|
||||
]);
|
||||
const indexes = await client.read(
|
||||
'PRAGMA index_list(`event_user_handlers`)',
|
||||
);
|
||||
expect(
|
||||
indexes.find(
|
||||
(i) => i.name === 'idx_event_user_handlers_app_holder_name',
|
||||
),
|
||||
).toMatchObject({ unique: 1 });
|
||||
expect(
|
||||
indexes.find((i) => i.name === 'idx_event_user_handlers_holder_app'),
|
||||
).toMatchObject({ unique: 0 });
|
||||
const columnsOf = async (index: string) =>
|
||||
(await client.read(`PRAGMA index_info(\`${index}\`)`)).map((c) =>
|
||||
String(c.name),
|
||||
);
|
||||
expect(await columnsOf('idx_event_user_handlers_app_holder_name')).toEqual(
|
||||
['app_uid', 'holder_user_id', 'name'],
|
||||
);
|
||||
expect(await columnsOf('idx_event_user_handlers_holder_app')).toEqual([
|
||||
'holder_user_id',
|
||||
'app_uid',
|
||||
]);
|
||||
});
|
||||
|
||||
it('leaves event_handlers as it was', async () => {
|
||||
const columns = await client.read('PRAGMA table_info(`event_handlers`)');
|
||||
expect(columns.map((c) => String(c.name))).toEqual([
|
||||
'id',
|
||||
'app_uid',
|
||||
'name',
|
||||
'source',
|
||||
'source_hash',
|
||||
'created_at',
|
||||
'updated_at',
|
||||
]);
|
||||
});
|
||||
|
||||
it('adds event_subscriptions.handler_scope, app scope for existing rows', async () => {
|
||||
const columns = await client.read(
|
||||
'PRAGMA table_info(`event_subscriptions`)',
|
||||
);
|
||||
expect(columns.find((c) => c.name === 'handler_scope')).toMatchObject({
|
||||
notnull: 1,
|
||||
dflt_value: '0',
|
||||
});
|
||||
const indexes = await client.read(
|
||||
'PRAGMA index_list(`event_subscriptions`)',
|
||||
);
|
||||
expect(indexes.map((i) => String(i.name))).toContain(
|
||||
'idx_event_subscriptions_holder',
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves an already-migrated database untouched on a second boot', async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
|
||||
const path = join(dir, 'nested', 'puter.sqlite');
|
||||
|
||||
@@ -135,6 +135,8 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
|
||||
[93, ['0098_notification-groups.sql']],
|
||||
[94, ['0099_kv-share-handle-notify.sql']],
|
||||
[95, ['0100_notification-send-key.sql']],
|
||||
[96, ['0101_user-scoped-handlers.sql']],
|
||||
[97, ['0102_notification-scope-id-index.sql']],
|
||||
];
|
||||
|
||||
export class SqliteDatabaseClient extends AbstractDatabaseClient {
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Handlers an app's user published into their own scope, kept apart from
|
||||
-- `event_handlers` so code that predates them never reads them. See
|
||||
-- sqlite/0101_user-scoped-handlers.sql for the column rationale.
|
||||
--
|
||||
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
|
||||
-- and a guarded column add. There is no per-file applied-state tracking, so a
|
||||
-- replay has to be a no-op.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS `event_user_handlers` (
|
||||
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
|
||||
-- Matches `apps`.`uid` exactly, charset included, so an equality against
|
||||
-- one never falls back to a conversion.
|
||||
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
|
||||
`holder_user_id` int unsigned NOT NULL,
|
||||
`name` varchar(128) NOT NULL,
|
||||
`source` mediumtext NOT NULL,
|
||||
`source_hash` char(64) NOT NULL,
|
||||
`created_at` bigint NOT NULL,
|
||||
`updated_at` bigint NOT NULL,
|
||||
-- When the scope first held a handler; kept across handler churn, so the
|
||||
-- rent sweep sees the scope's age, not its oldest remaining handler's.
|
||||
`scope_since` bigint NOT NULL,
|
||||
PRIMARY KEY (`id`),
|
||||
UNIQUE KEY `idx_event_user_handlers_app_holder_name` (`app_uid`, `holder_user_id`, `name`),
|
||||
KEY `idx_event_user_handlers_holder_app` (`holder_user_id`, `app_uid`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
-- Guarded here rather than through `_puter_add_col`, so the file also applies
|
||||
-- on its own to a database that never ran the earlier migrations.
|
||||
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_handler_scope;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_event_subscriptions_handler_scope()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'event_subscriptions'
|
||||
AND COLUMN_NAME = 'handler_scope'
|
||||
) THEN
|
||||
ALTER TABLE `event_subscriptions`
|
||||
ADD COLUMN `handler_scope` tinyint unsigned NOT NULL DEFAULT 0,
|
||||
ALGORITHM=INSTANT;
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_event_subscriptions_handler_scope();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_handler_scope;
|
||||
@@ -0,0 +1,41 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- `notification` (`user_id`, `audience`, `app_uid`, `id`). See
|
||||
-- sqlite/0102_notification-scope-id-index.sql for the rationale. No per-file
|
||||
-- applied-state tracking, so the index is added only when missing and a
|
||||
-- replay does nothing.
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_id_index;
|
||||
DELIMITER //
|
||||
CREATE PROCEDURE _puter_add_notification_scope_id_index()
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'notification'
|
||||
AND INDEX_NAME = 'idx_notification_scope_id'
|
||||
) THEN
|
||||
ALTER TABLE `notification` ADD INDEX `idx_notification_scope_id`
|
||||
(`user_id`, `audience`, `app_uid`, `id`), ALGORITHM=INPLACE, LOCK=NONE;
|
||||
END IF;
|
||||
END//
|
||||
DELIMITER ;
|
||||
|
||||
CALL _puter_add_notification_scope_id_index();
|
||||
|
||||
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_id_index;
|
||||
@@ -0,0 +1,39 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Handlers an app's user published into their own scope. See
|
||||
-- sqlite/0101_user-scoped-handlers.sql for the column rationale.
|
||||
-- Idempotent via IF NOT EXISTS.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS event_user_handlers (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
app_uid VARCHAR(40) NOT NULL,
|
||||
holder_user_id INTEGER NOT NULL,
|
||||
name VARCHAR(128) NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
source_hash VARCHAR(64) NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
updated_at BIGINT NOT NULL,
|
||||
scope_since BIGINT NOT NULL
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_event_user_handlers_app_holder_name
|
||||
ON event_user_handlers (app_uid, holder_user_id, name);
|
||||
CREATE INDEX IF NOT EXISTS idx_event_user_handlers_holder_app
|
||||
ON event_user_handlers (holder_user_id, app_uid);
|
||||
|
||||
ALTER TABLE event_subscriptions ADD COLUMN IF NOT EXISTS handler_scope smallint NOT NULL DEFAULT 0;
|
||||
@@ -0,0 +1,23 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- notification (user_id, audience, app_uid, id). See
|
||||
-- sqlite/0102_notification-scope-id-index.sql for the rationale. Idempotent via
|
||||
-- IF NOT EXISTS.
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_notification_scope_id
|
||||
ON notification (user_id, audience, app_uid, id);
|
||||
@@ -0,0 +1,48 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Handlers an app's user published into their own scope, in a table of their
|
||||
-- own so `event_handlers` stays the app owner's set. Same columns as
|
||||
-- `event_handlers` (see 0076_event-handlers.sql) plus the holder.
|
||||
--
|
||||
-- - `holder_user_id` : the user whose scope this is. A name is unique per
|
||||
-- (app, holder), so the same name can live in the app's
|
||||
-- set and in each holder's.
|
||||
-- - `(holder_user_id, app_uid)` : a holder's scopes, for listing and for the
|
||||
-- rent count.
|
||||
--
|
||||
-- `event_subscriptions.handler_scope` says which table a row's `handler_name`
|
||||
-- refers to: 0 the app's set, 1 the row holder's scope.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS `event_user_handlers` (
|
||||
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
"app_uid" TEXT NOT NULL,
|
||||
"holder_user_id" INTEGER NOT NULL,
|
||||
"name" TEXT NOT NULL,
|
||||
"source" TEXT NOT NULL,
|
||||
"source_hash" TEXT NOT NULL,
|
||||
"created_at" INTEGER NOT NULL, -- unix seconds
|
||||
"updated_at" INTEGER NOT NULL,
|
||||
"scope_since" INTEGER NOT NULL -- unix seconds; the scope's first handler, kept across churn
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS `idx_event_user_handlers_app_holder_name`
|
||||
ON `event_user_handlers` (`app_uid`, `holder_user_id`, `name`);
|
||||
CREATE INDEX IF NOT EXISTS `idx_event_user_handlers_holder_app`
|
||||
ON `event_user_handlers` (`holder_user_id`, `app_uid`);
|
||||
|
||||
ALTER TABLE `event_subscriptions` ADD COLUMN `handler_scope` INTEGER NOT NULL DEFAULT 0;
|
||||
@@ -0,0 +1,22 @@
|
||||
-- Copyright (C) 2024-present Puter Technologies Inc.
|
||||
--
|
||||
-- This file is part of Puter.
|
||||
--
|
||||
-- Puter is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as published
|
||||
-- by the Free Software Foundation, either version 3 of the License, or
|
||||
-- (at your option) any later version.
|
||||
--
|
||||
-- This program is distributed in the hope that it will be useful,
|
||||
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
-- GNU Affero General Public License for more details.
|
||||
--
|
||||
-- You should have received a copy of the GNU Affero General Public License
|
||||
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
-- Index a mailbox slice by row id, so a fetch finds the newest row of a slice,
|
||||
-- and pages forward through one, with a seek rather than a scan of the slice.
|
||||
|
||||
CREATE INDEX IF NOT EXISTS `idx_notification_scope_id`
|
||||
ON `notification` (`user_id`, `audience`, `app_uid`, `id`);
|
||||
@@ -22,7 +22,6 @@ import type {
|
||||
import { Actor } from '../../core';
|
||||
import type { UsageInput } from '../../services/metering/types';
|
||||
import { FSEntry } from '../../stores/fs/FSEntry';
|
||||
import type { SignupUtm } from '../../util/signupUtm';
|
||||
|
||||
// GUI write events spread an entry plus per-event metadata into `response`.
|
||||
// The exact field set varies by emit site (FSController / LegacyFSController /
|
||||
@@ -201,6 +200,16 @@ export type EventMap = {
|
||||
};
|
||||
|
||||
// ---- Auth / signup ----
|
||||
/**
|
||||
* Which signup methods are open, emitted via `emitAndWait` before every
|
||||
* signup and whenever the GUI reads the provider list. A listener sets
|
||||
* `oidcOnly` to refuse form and temp signups, leaving OIDC and team
|
||||
* provisioning open; `message` is what a refused form signup is told.
|
||||
*/
|
||||
'puter.signup.policy': {
|
||||
oidcOnly: boolean;
|
||||
message: string | null;
|
||||
};
|
||||
'puter.signup.validate': {
|
||||
allow: boolean;
|
||||
email?: string;
|
||||
@@ -251,8 +260,6 @@ export type EventMap = {
|
||||
is_temp?: boolean;
|
||||
/** The bonus code `puter.signup-bonus.validate` accepted, if any. */
|
||||
bonus_code?: string;
|
||||
/** The `utm_*` params the signup carried, if any. Client-supplied. */
|
||||
utm?: SignupUtm;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
// Signup bonus codes are pure mechanism here: an extension decides what a
|
||||
@@ -549,6 +556,14 @@ export type EventMap = {
|
||||
'kv.share.retired': { handles: string[]; userId: number };
|
||||
|
||||
// ---- Notifications ----
|
||||
/** An app message was acknowledged after its state committed. */
|
||||
'notif.acknowledged': {
|
||||
userId: number;
|
||||
uid: string;
|
||||
type: string;
|
||||
audience: string;
|
||||
appUid: string;
|
||||
};
|
||||
/**
|
||||
* A notification row exists. Emitted after the insert commits — nothing
|
||||
* pushes a uid before there is a row to name — and carries the scope tuple
|
||||
@@ -582,6 +597,8 @@ export type EventMap = {
|
||||
value: Record<string, unknown>;
|
||||
/** An earlier notification rewritten in place. */
|
||||
updated?: true;
|
||||
/** Replaced an earlier notification in its group. */
|
||||
replaced?: true;
|
||||
/** Names this one send, for a receipt asked about it. */
|
||||
sendId: string;
|
||||
/** The same `sendId` may be emitted again, as a delayed send is. */
|
||||
@@ -669,9 +686,18 @@ export type EventMap = {
|
||||
// and going away (1→0, whether by removing the last handler or by the
|
||||
// destroy route). `actor.user` is the app's owner, not the caller — a
|
||||
// developer session publishing for an app it owns is the common case, but
|
||||
// billing follows ownership.
|
||||
'events.worker.create': { actor: Actor; appUid: string };
|
||||
'events.worker.destroy': { actor: Actor; appUid: string };
|
||||
// billing follows ownership. For `scope: 'user'` the unit is one user's
|
||||
// handlers in one app instead, and `actor.user` is that user.
|
||||
'events.worker.create': {
|
||||
actor: Actor;
|
||||
appUid: string;
|
||||
scope: 'app' | 'user';
|
||||
};
|
||||
'events.worker.destroy': {
|
||||
actor: Actor;
|
||||
appUid: string;
|
||||
scope: 'app' | 'user';
|
||||
};
|
||||
|
||||
// ---- Outer / GUI broadcast ----
|
||||
'outer.cacheUpdate': {
|
||||
@@ -1035,11 +1061,10 @@ export type EventKey = keyof EventMap & string;
|
||||
// Generates a wildcard for every non-final dot-separated prefix of K.
|
||||
export type WildcardPrefixes<K extends string> =
|
||||
K extends `${infer Head}.${infer Tail}`
|
||||
?
|
||||
| `${Head}.*`
|
||||
| (Tail extends `${string}.${string}`
|
||||
? `${Head}.${WildcardPrefixes<Tail>}`
|
||||
: never)
|
||||
? | `${Head}.*`
|
||||
| (Tail extends `${string}.${string}`
|
||||
? `${Head}.${WildcardPrefixes<Tail>}`
|
||||
: never)
|
||||
: never;
|
||||
|
||||
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
|
||||
@@ -1050,10 +1075,7 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
|
||||
|
||||
/** Acked, not acked, no socket anywhere, or sockets only in a peer region. */
|
||||
export type NotifDeliveryOutcome =
|
||||
| 'received'
|
||||
| 'unreceived'
|
||||
| 'no-socket'
|
||||
| 'remote-only';
|
||||
'received' | 'unreceived' | 'no-socket' | 'remote-only';
|
||||
|
||||
/** One socket's receipt for a live notification. */
|
||||
export interface NotifDeliveryResponder {
|
||||
|
||||
@@ -23,16 +23,20 @@
|
||||
*/
|
||||
|
||||
import type { fetch as undiciFetch } from 'undici';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import {
|
||||
DispatcherInvokeTransport,
|
||||
EVENTS_DEPLOYED_HEADER,
|
||||
EVENTS_DEADLINE_MARGIN_MS,
|
||||
EVENTS_DISPATCH_BATCH_PATH,
|
||||
EVENTS_DISPATCH_ERROR_HEADER,
|
||||
EVENTS_DISPATCH_PATH,
|
||||
EVENTS_ERROR_HEADER,
|
||||
EVENTS_HANDLED_HEADER,
|
||||
EVENTS_HOLDER_HEADER,
|
||||
EVENTS_INVOKE_TIMEOUT_MS,
|
||||
EVENTS_WORKER_LEASE_FACTOR,
|
||||
EventsWorkerInvokerClient,
|
||||
INVOKE_CONNECTIONS,
|
||||
SCHEDULE_INVOKE_CONNECTIONS,
|
||||
@@ -343,6 +347,46 @@ describe('DispatcherInvokeTransport with every connection busy', () => {
|
||||
held[INVOKE_CONNECTIONS].answer();
|
||||
await expect(after).resolves.toMatchObject({ status: 200 });
|
||||
});
|
||||
|
||||
it('rechecks the guard once a connection frees, and drops a superseded call', async () => {
|
||||
const { transport, held, answerAll } = await saturated();
|
||||
|
||||
let current = true;
|
||||
const guard = vi.fn(async () => current);
|
||||
const waiting = transport.send({ ...CALL, beforeSend: guard });
|
||||
current = false;
|
||||
held[0].answer();
|
||||
|
||||
await expect(waiting).resolves.toEqual({
|
||||
status: null,
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
expect(guard).toHaveBeenCalledTimes(1);
|
||||
expect(held).toHaveLength(INVOKE_CONNECTIONS);
|
||||
await answerAll();
|
||||
});
|
||||
|
||||
it('fails a call whose guard throws after the wait, and frees its slot', async () => {
|
||||
const { transport, held, answerAll } = await saturated();
|
||||
|
||||
const waiting = transport.send({
|
||||
...CALL,
|
||||
beforeSend: async () => {
|
||||
throw new Error('lease store down');
|
||||
},
|
||||
});
|
||||
held[0].answer();
|
||||
await expect(waiting).rejects.toThrow('lease store down');
|
||||
expect(held).toHaveLength(INVOKE_CONNECTIONS);
|
||||
|
||||
await answerAll();
|
||||
const after = transport.send(CALL);
|
||||
await vi.waitFor(() =>
|
||||
expect(held).toHaveLength(INVOKE_CONNECTIONS + 1),
|
||||
);
|
||||
held[INVOKE_CONNECTIONS].answer();
|
||||
await expect(after).resolves.toMatchObject({ status: 200 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('DispatcherInvokeTransport schedule lane', () => {
|
||||
@@ -767,6 +811,138 @@ describe('DispatcherInvokeTransport batches', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it('leaves a call superseded while gathered out of the batch', async () => {
|
||||
const { fetchImpl, batches, singles } = streamingFetch();
|
||||
const transport = transportFor(fetchImpl, { batchWindowMs: 20 });
|
||||
let current = true;
|
||||
const sent = [
|
||||
transport.send(CALL),
|
||||
transport.send({ ...CALL, beforeSend: async () => current }),
|
||||
transport.send({ ...CALL, script: 'evw-other' }),
|
||||
];
|
||||
current = false;
|
||||
await vi.waitFor(() => expect(batches).toHaveLength(1));
|
||||
expect(batches[0].body.calls.map((call) => call.script)).toEqual([
|
||||
'evw-test',
|
||||
'evw-other',
|
||||
]);
|
||||
await expect(sent[1]).resolves.toEqual({
|
||||
status: null,
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
batches[0].answer({ i: 0, status: 200, handled: true });
|
||||
batches[0].answer({ i: 1, status: 200, handled: true });
|
||||
await expect(Promise.all([sent[0], sent[2]])).resolves.toEqual([
|
||||
{ status: 200, handled: true },
|
||||
{ status: 200, handled: true },
|
||||
]);
|
||||
expect(singles).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('sends nothing when every gathered call was superseded', async () => {
|
||||
const fetchImpl = vi.fn(
|
||||
stubFetch(200, { [EVENTS_HANDLED_HEADER]: '1' }),
|
||||
);
|
||||
const transport = transportFor(fetchImpl as unknown as FetchImpl);
|
||||
const superseded = { ...CALL, beforeSend: async () => false };
|
||||
await expect(
|
||||
Promise.all([
|
||||
transport.send(superseded),
|
||||
transport.send(superseded),
|
||||
]),
|
||||
).resolves.toEqual([
|
||||
{ status: null, error: 'invocation superseded' },
|
||||
{ status: null, error: 'invocation superseded' },
|
||||
]);
|
||||
expect(fetchImpl).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('fails only the call whose guard throws at flush', async () => {
|
||||
const { fetchImpl, batches, singles } = streamingFetch();
|
||||
const transport = transportFor(fetchImpl);
|
||||
const failing = transport.send({
|
||||
...CALL,
|
||||
beforeSend: async () => {
|
||||
throw new Error('lease store down');
|
||||
},
|
||||
});
|
||||
const kept = transport.send(CALL);
|
||||
await expect(failing).rejects.toThrow('lease store down');
|
||||
// One call left, so it goes alone.
|
||||
await expect(kept).resolves.toEqual({ status: 200, handled: true });
|
||||
expect(batches).toHaveLength(0);
|
||||
expect(singles).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('rechecks the guard before resending a batch call on its own', async () => {
|
||||
const urls: string[] = [];
|
||||
let current = true;
|
||||
const fetchImpl = (async (url: string) => {
|
||||
urls.push(url);
|
||||
if (url.endsWith(EVENTS_DISPATCH_BATCH_PATH)) {
|
||||
// Superseded while the batch was out.
|
||||
current = false;
|
||||
return new Response('no-such-route', {
|
||||
status: 404,
|
||||
headers: {
|
||||
[EVENTS_DISPATCH_ERROR_HEADER]: 'no-such-route',
|
||||
},
|
||||
});
|
||||
}
|
||||
return new Response(null, {
|
||||
status: 200,
|
||||
headers: { [EVENTS_HANDLED_HEADER]: '1' },
|
||||
});
|
||||
}) as unknown as FetchImpl;
|
||||
const transport = transportFor(fetchImpl);
|
||||
|
||||
await expect(
|
||||
Promise.all([
|
||||
transport.send({ ...CALL, beforeSend: async () => current }),
|
||||
transport.send(CALL),
|
||||
]),
|
||||
).resolves.toEqual([
|
||||
{ status: null, error: 'invocation superseded' },
|
||||
{ status: 200, handled: true },
|
||||
]);
|
||||
expect(urls.map((url) => new URL(url).pathname)).toEqual([
|
||||
EVENTS_DISPATCH_BATCH_PATH,
|
||||
EVENTS_DISPATCH_PATH,
|
||||
]);
|
||||
});
|
||||
|
||||
it('answers a schedule run superseded in the batch as the early check does', async () => {
|
||||
const { fetchImpl, batches } = streamingFetch();
|
||||
const client = new EventsWorkerInvokerClient({
|
||||
events: { invokeTimeoutMs: 5_000 },
|
||||
} as unknown as IConfig);
|
||||
client.setTransport(transportFor(fetchImpl, { batchWindowMs: 20 }));
|
||||
let current = true;
|
||||
const guard = vi.fn(async () => current);
|
||||
|
||||
const run = client.invoke(REQUEST, {
|
||||
lane: 'schedule',
|
||||
beforeSend: guard,
|
||||
});
|
||||
const other = client.invoke(REQUEST);
|
||||
const another = client.invoke(REQUEST);
|
||||
await vi.waitFor(() => expect(guard).toHaveBeenCalledTimes(1));
|
||||
current = false;
|
||||
|
||||
await vi.waitFor(() => expect(batches).toHaveLength(1));
|
||||
expect(batches[0].body.calls).toHaveLength(2);
|
||||
await expect(run).resolves.toEqual(
|
||||
await client.invoke(REQUEST, { beforeSend: async () => false }),
|
||||
);
|
||||
await expect(run).resolves.toMatchObject({
|
||||
outcome: 'unavailable',
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
batches[0].answer({ i: 0, status: 200, handled: true });
|
||||
batches[0].answer({ i: 1, status: 200, handled: true });
|
||||
await Promise.all([other, another]);
|
||||
});
|
||||
|
||||
it('builds each body when the batch goes out', async () => {
|
||||
const { fetchImpl, batches } = streamingFetch();
|
||||
const transport = transportFor(fetchImpl);
|
||||
@@ -823,6 +999,26 @@ describe('DispatcherInvokeTransport batches', () => {
|
||||
});
|
||||
|
||||
describe('EventsWorkerInvokerClient invoke options', () => {
|
||||
it('does not send a superseded invocation', async () => {
|
||||
const client = makeClient();
|
||||
const send = vi.fn(async () => ({ status: 200, handled: true }));
|
||||
client.setTransport({ send });
|
||||
const result = await client.invoke(REQUEST, { beforeSend: async () => false });
|
||||
expect(result.outcome).toBe('unavailable');
|
||||
expect(send).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rechecks before resending after a deployment', async () => {
|
||||
const client = makeClient();
|
||||
let current = true;
|
||||
const send = vi.fn(async () => ({ status: null, dispatchReason: 'missing' }));
|
||||
client.setTransport({ send });
|
||||
client.setMissHandler(async () => { current = false; return 'deployed'; });
|
||||
const result = await client.invoke(REQUEST, { beforeSend: async () => current });
|
||||
expect(result.outcome).toBe('unavailable');
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('stamps the event as each attempt goes out', async () => {
|
||||
const bodies: string[] = [];
|
||||
const client = makeClient();
|
||||
@@ -1302,3 +1498,217 @@ describe('outcomeForStatus', () => {
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe('EventsWorkerInvokerClient overall deadline', () => {
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
const LEASE_MS = EVENTS_INVOKE_TIMEOUT_MS * EVENTS_WORKER_LEASE_FACTOR;
|
||||
|
||||
/**
|
||||
* A transport on a fake clock: each send costs what `cost` says, and
|
||||
* answers a miss until the deployed header, then a handled 200.
|
||||
*/
|
||||
const timedTransport = (cost: (deployed: boolean) => number) => {
|
||||
const sends: Array<{ deployed: boolean; timeoutMs: number }> = [];
|
||||
const transport = {
|
||||
send: async (call: {
|
||||
deployed?: boolean;
|
||||
timeoutMs: number;
|
||||
}): Promise<unknown> => {
|
||||
const deployed = call.deployed === true;
|
||||
sends.push({ deployed, timeoutMs: call.timeoutMs });
|
||||
// A send never outlasts its own timeout.
|
||||
vi.advanceTimersByTime(
|
||||
Math.min(cost(deployed), call.timeoutMs * 1.5),
|
||||
);
|
||||
return deployed
|
||||
? { status: 200, handled: true }
|
||||
: {
|
||||
status: null,
|
||||
dispatchReason: 'missing',
|
||||
error: 'dispatcher: missing (404)',
|
||||
};
|
||||
},
|
||||
};
|
||||
return { transport, sends };
|
||||
};
|
||||
|
||||
const run = async (
|
||||
cost: (deployed: boolean) => number,
|
||||
deployMs: number,
|
||||
) => {
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
const { transport, sends } = timedTransport(cost);
|
||||
const client = makeClient();
|
||||
client.setTransport(transport as never);
|
||||
client.setMissHandler(async () => {
|
||||
vi.advanceTimersByTime(deployMs);
|
||||
return 'deployed';
|
||||
});
|
||||
const started = Date.now();
|
||||
const result = await client.invoke(REQUEST);
|
||||
return { result, sends, elapsed: Date.now() - started };
|
||||
};
|
||||
|
||||
it('leaves a fast miss as it was: a full-timeout resend', async () => {
|
||||
const { result, sends } = await run(() => 100, 2_000);
|
||||
expect(result).toEqual({ outcome: 'settled', status: 200 });
|
||||
expect(sends).toEqual([
|
||||
{ deployed: false, timeoutMs: EVENTS_INVOKE_TIMEOUT_MS },
|
||||
{ deployed: true, timeoutMs: EVENTS_INVOKE_TIMEOUT_MS },
|
||||
]);
|
||||
});
|
||||
|
||||
it('shortens the resend after a slow deploy, so the call ends before the lease', async () => {
|
||||
// A miss that took 12 s, a deploy that took 30 s, and a resend that
|
||||
// would use everything it is given.
|
||||
const { result, sends, elapsed } = await run(
|
||||
(deployed) => (deployed ? 1e9 : 12_000),
|
||||
30_000,
|
||||
);
|
||||
expect(result.outcome).toBe('settled');
|
||||
expect(sends[1].deployed).toBe(true);
|
||||
expect(sends[1].timeoutMs).toBeLessThan(EVENTS_INVOKE_TIMEOUT_MS);
|
||||
expect(elapsed).toBeLessThanOrEqual(
|
||||
LEASE_MS - EVENTS_DEADLINE_MARGIN_MS,
|
||||
);
|
||||
});
|
||||
|
||||
it('gives up on a deploy that never settles before the lease, leaving it running', async () => {
|
||||
vi.useFakeTimers();
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const { transport, sends } = timedTransport(() => 0);
|
||||
const client = makeClient();
|
||||
client.setTransport(transport as never);
|
||||
let rejectDeploy!: (err: Error) => void;
|
||||
client.setMissHandler(
|
||||
() =>
|
||||
new Promise<string>((_resolve, reject) => {
|
||||
rejectDeploy = reject;
|
||||
}),
|
||||
);
|
||||
let finished: unknown = null;
|
||||
const pending = client.invoke(REQUEST).then((r) => (finished = r));
|
||||
|
||||
// Up to where the call must have ended, and no further.
|
||||
await vi.advanceTimersByTimeAsync(LEASE_MS - EVENTS_DEADLINE_MARGIN_MS);
|
||||
expect(finished).toMatchObject({
|
||||
outcome: 'unavailable',
|
||||
status: null,
|
||||
});
|
||||
await pending;
|
||||
expect(sends).toHaveLength(1);
|
||||
// The deploy was not cancelled; its late failure is logged, not thrown.
|
||||
rejectDeploy(new Error('stalled'));
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
'[events] deploy failed late',
|
||||
expect.any(Error),
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it('does not resend when the deploy leaves less than the margin', async () => {
|
||||
const { result, sends } = await run(
|
||||
() => 100,
|
||||
LEASE_MS - EVENTS_DEADLINE_MARGIN_MS * 2 + 1_000,
|
||||
);
|
||||
expect(sends).toHaveLength(1);
|
||||
expect(result).toMatchObject({ outcome: 'unavailable', status: null });
|
||||
});
|
||||
});
|
||||
|
||||
describe('the holder of a user scope on the wire', () => {
|
||||
const HOLDER = 'user-uuid-1';
|
||||
|
||||
it('sends the holder header on a single invoke, and none for an app scope', async () => {
|
||||
const seen: Array<Record<string, string>> = [];
|
||||
const fetchImpl = (async (_url: string, init?: RequestInit) => {
|
||||
seen.push((init?.headers ?? {}) as Record<string, string>);
|
||||
return new Response(null, {
|
||||
status: 200,
|
||||
headers: { [EVENTS_HANDLED_HEADER]: '1' },
|
||||
});
|
||||
}) as unknown as FetchImpl;
|
||||
const client = makeClient();
|
||||
client.setTransport(
|
||||
new DispatcherInvokeTransport('http://dispatcher', 's', {
|
||||
fetchImpl,
|
||||
batchSize: 1,
|
||||
}),
|
||||
);
|
||||
|
||||
await client.invoke({ ...REQUEST, holder: HOLDER });
|
||||
await client.invoke(REQUEST);
|
||||
|
||||
expect(seen[0][EVENTS_HOLDER_HEADER]).toBe(HOLDER);
|
||||
expect(EVENTS_HOLDER_HEADER in seen[1]).toBe(false);
|
||||
});
|
||||
|
||||
it('puts a holder field on a batch call of a user scope only', async () => {
|
||||
const bodies: Array<{ calls: Array<Record<string, unknown>> }> = [];
|
||||
const fetchImpl = (async (_url: string, init?: RequestInit) => {
|
||||
bodies.push(JSON.parse(String(init?.body)));
|
||||
return new Response(
|
||||
'{"i":0,"status":200,"handled":true}\n{"i":1,"status":200,"handled":true}\n',
|
||||
{ status: 200 },
|
||||
);
|
||||
}) as unknown as FetchImpl;
|
||||
const transport = new DispatcherInvokeTransport(
|
||||
'http://dispatcher',
|
||||
's',
|
||||
{ fetchImpl },
|
||||
);
|
||||
|
||||
await Promise.all([
|
||||
transport.send({ ...CALL, holder: HOLDER }),
|
||||
transport.send(CALL),
|
||||
]);
|
||||
|
||||
expect(bodies).toHaveLength(1);
|
||||
expect(bodies[0].calls[0].holder).toBe(HOLDER);
|
||||
expect('holder' in bodies[0].calls[1]).toBe(false);
|
||||
});
|
||||
|
||||
it('hands the holder to the miss handler, and nothing for an app scope', async () => {
|
||||
const seen: Array<{ holder?: string }> = [];
|
||||
const client = makeClient();
|
||||
client.setTransport(
|
||||
new DispatcherInvokeTransport('http://dispatcher', 's', {
|
||||
fetchImpl: missUntilDeployedFetch([]),
|
||||
batchSize: 1,
|
||||
}),
|
||||
);
|
||||
client.setMissHandler(async (_appUid, _script, options) => {
|
||||
seen.push(options);
|
||||
return 'deployed';
|
||||
});
|
||||
|
||||
await client.invoke({ ...REQUEST, holder: HOLDER });
|
||||
await client.invoke(REQUEST);
|
||||
|
||||
expect(seen[0].holder).toBe(HOLDER);
|
||||
expect('holder' in seen[1]).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps the holder out of the body the script sees', async () => {
|
||||
const bodies: string[] = [];
|
||||
const client = makeClient();
|
||||
client.setTransport({
|
||||
send: async (call) => {
|
||||
bodies.push(call.body);
|
||||
expect(call.holder).toBe(HOLDER);
|
||||
return { status: 200, handled: true };
|
||||
},
|
||||
});
|
||||
await client.invoke({ ...REQUEST, holder: HOLDER });
|
||||
expect(Object.keys(JSON.parse(bodies[0])).sort()).toEqual([
|
||||
'ctx',
|
||||
'event',
|
||||
'handler',
|
||||
'token',
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -35,6 +35,7 @@ import { PuterClient } from '../types.js';
|
||||
* x-puter-internal-auth: <internal secret>
|
||||
* x-puter-events-script: <script the handler set deploys as>
|
||||
* x-puter-events-app: <app uid, for the deploy-on-miss callback>
|
||||
* x-puter-events-holder: <holder uuid of a user scope; absent for app scope>
|
||||
* x-puter-events-key: <key derived for that script>
|
||||
* { handler, event, ctx, token }
|
||||
*
|
||||
@@ -48,7 +49,7 @@ import { PuterClient } from '../types.js';
|
||||
*
|
||||
* POST <events dispatcher>/invoke-batch
|
||||
* x-puter-internal-auth: <internal secret>
|
||||
* { calls: [{ script, appUid, key, deployed?, body, waitMs }] }
|
||||
* { calls: [{ script, appUid, holder?, key, deployed?, body, waitMs }] }
|
||||
*
|
||||
* The dispatcher runs a few at a time, in order, and streams NDJSON: `{ i,
|
||||
* started: true }` when a call goes to its worker, then `{ i, status, handled?,
|
||||
@@ -127,6 +128,12 @@ const PLATFORM_ERROR_CODES = new Set([
|
||||
*/
|
||||
export const EVENTS_DEPLOYED_HEADER = 'x-puter-events-deployed';
|
||||
|
||||
/**
|
||||
* Names the user whose scope a script belongs to, so a miss deploys it from
|
||||
* their handlers. Absent for an app's own scope.
|
||||
*/
|
||||
export const EVENTS_HOLDER_HEADER = 'x-puter-events-holder';
|
||||
|
||||
/**
|
||||
* Dispatch-error reasons this backend can resolve itself by deploying, rather
|
||||
* than wait on the rehydrate callback finding a backend behind the dispatcher's
|
||||
@@ -150,6 +157,11 @@ const ALREADY_DEPLOYED_MISS_REASONS = new Set(['deploy-timeout']);
|
||||
/** Whose call asked for the deploy, for the share of deploys it may take. */
|
||||
export interface WorkerMissOptions {
|
||||
lane?: InvokeLane;
|
||||
/**
|
||||
* Holder uuid of the user scope the script belongs to; absent for app
|
||||
* scope.
|
||||
*/
|
||||
holder?: string;
|
||||
}
|
||||
|
||||
/** What a self-deploy attempt did, and to which app/script it applies. */
|
||||
@@ -159,9 +171,55 @@ export type WorkerMissHandler = (
|
||||
options: WorkerMissOptions,
|
||||
) => Promise<'deployed' | string>;
|
||||
|
||||
/**
|
||||
* `work`'s result, or null once `ms` has passed. The work keeps running, and a
|
||||
* failure after the cutoff is logged since nothing else would see it.
|
||||
*/
|
||||
const withinTime = async <T>(
|
||||
work: Promise<T>,
|
||||
ms: number,
|
||||
): Promise<T | null> => {
|
||||
let timedOut = false;
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
const limit = new Promise<null>((resolve) => {
|
||||
timer = setTimeout(
|
||||
() => {
|
||||
timedOut = true;
|
||||
resolve(null);
|
||||
},
|
||||
Math.max(0, ms),
|
||||
);
|
||||
});
|
||||
void (async () => {
|
||||
try {
|
||||
await work;
|
||||
} catch (err) {
|
||||
if (timedOut) console.warn('[events] deploy failed late', err);
|
||||
}
|
||||
})();
|
||||
try {
|
||||
return await Promise.race([work, limit]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
};
|
||||
|
||||
/** How long a handler has to answer before the attempt is abandoned. */
|
||||
export const EVENTS_INVOKE_TIMEOUT_MS = 30_000;
|
||||
|
||||
/**
|
||||
* How many invoke timeouts a delivery's worker lease lasts. The lease store
|
||||
* reads this, and `invoke()` keeps a whole call, deploy and resend included,
|
||||
* inside it.
|
||||
*/
|
||||
export const EVENTS_WORKER_LEASE_FACTOR = 2;
|
||||
|
||||
/**
|
||||
* Left unspent at the end of the lease, and the least a resend is worth
|
||||
* starting with: below it the call is given up as `unavailable` and retried.
|
||||
*/
|
||||
export const EVENTS_DEADLINE_MARGIN_MS = 5_000;
|
||||
|
||||
/**
|
||||
* Connections held open to the dispatcher, so a busy fleet is not
|
||||
* re-handshaking. Also how many calls one transport makes at once: a call past
|
||||
@@ -194,6 +252,8 @@ export interface WorkerInvokeRequest {
|
||||
/** Script the app's current handler set deploys as. */
|
||||
script: string;
|
||||
appUid: string;
|
||||
/** Holder uuid, when the handler lives in a user scope. */
|
||||
holder?: string;
|
||||
handler: string;
|
||||
/** The subscriber's app session token. */
|
||||
token: string;
|
||||
@@ -207,6 +267,8 @@ export interface WorkerInvokeRequest {
|
||||
export interface EventsInvokeCall {
|
||||
script: string;
|
||||
appUid: string;
|
||||
/** Holder uuid of the user scope the script belongs to. */
|
||||
holder?: string;
|
||||
/** Value the script compares against its own invoke-key binding. */
|
||||
key: string;
|
||||
/** Serialized `{ handler, event, ctx, token }`. */
|
||||
@@ -220,6 +282,11 @@ export interface EventsInvokeCall {
|
||||
* connection first, so a send time stamped into it is the real one.
|
||||
*/
|
||||
render?: () => string;
|
||||
/**
|
||||
* Rechecked by a transport that waits before sending; false drops the call
|
||||
* with {@link supersededAnswer}.
|
||||
*/
|
||||
beforeSend?: () => Promise<boolean>;
|
||||
}
|
||||
|
||||
/** What one attempt at delivering an invocation came back with. */
|
||||
@@ -255,10 +322,7 @@ export interface EventsInvokeTransport {
|
||||
* handler); retried like `retriable` but never the handler's fault.
|
||||
*/
|
||||
export type WorkerInvokeOutcome =
|
||||
| 'settled'
|
||||
| 'terminal'
|
||||
| 'retriable'
|
||||
| 'unavailable';
|
||||
'settled' | 'terminal' | 'retriable' | 'unavailable';
|
||||
|
||||
export interface WorkerInvokeResult {
|
||||
outcome: WorkerInvokeOutcome;
|
||||
@@ -275,6 +339,8 @@ export interface WorkerInvokeResult {
|
||||
/** How one invocation is sent. */
|
||||
export interface WorkerInvokeOptions {
|
||||
lane?: InvokeLane;
|
||||
/** Rechecked before each send, including a resend after deployment. */
|
||||
beforeSend?: () => Promise<boolean>;
|
||||
/** Applied to the event as each attempt is sent. */
|
||||
stampEvent?: (event: unknown) => unknown;
|
||||
}
|
||||
@@ -282,6 +348,12 @@ export interface WorkerInvokeOptions {
|
||||
const NO_TRANSPORT =
|
||||
'no events dispatcher configured (url and internal secret)';
|
||||
|
||||
/** What a call whose `beforeSend` said no answers instead of being sent. */
|
||||
const supersededAnswer = (): EventsInvokeAnswer => ({
|
||||
status: null,
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
|
||||
export class EventsWorkerInvokerClient extends PuterClient {
|
||||
/** Set only where invocations do not leave the process — local development. */
|
||||
#transport: EventsInvokeTransport | null = null;
|
||||
@@ -329,25 +401,68 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
ctx: request.ctx,
|
||||
token: request.token,
|
||||
});
|
||||
const send = (deployed: boolean): Promise<EventsInvokeAnswer> =>
|
||||
transport.send({
|
||||
const timeoutMs = this.#timeoutMs();
|
||||
// The whole call, deploy and resend included, ends before the lease a
|
||||
// delivery holds would let it be claimed again mid-run.
|
||||
// A short configured timeout gets a proportionally short margin.
|
||||
const margin = Math.min(EVENTS_DEADLINE_MARGIN_MS, timeoutMs / 2);
|
||||
const deadline =
|
||||
Date.now() + timeoutMs * EVENTS_WORKER_LEASE_FACTOR - margin;
|
||||
const send = async (
|
||||
deployed: boolean,
|
||||
sendTimeoutMs = timeoutMs,
|
||||
): Promise<EventsInvokeAnswer> => {
|
||||
if (options.beforeSend && !(await options.beforeSend()))
|
||||
return supersededAnswer();
|
||||
return transport.send({
|
||||
script: request.script,
|
||||
appUid: request.appUid,
|
||||
...(request.holder ? { holder: request.holder } : {}),
|
||||
key: request.key,
|
||||
body: render(),
|
||||
timeoutMs: this.#timeoutMs(),
|
||||
timeoutMs: sendTimeoutMs,
|
||||
...(deployed ? { deployed: true } : {}),
|
||||
...(options.lane ? { lane: options.lane } : {}),
|
||||
...(stampEvent ? { render } : {}),
|
||||
...(options.beforeSend
|
||||
? { beforeSend: options.beforeSend }
|
||||
: {}),
|
||||
});
|
||||
};
|
||||
|
||||
// A send may wait for a connection for half its timeout before the
|
||||
// run's own timeout starts, so it is given what is left divided by that.
|
||||
const resend = (): Promise<EventsInvokeAnswer> | null => {
|
||||
const remaining = deadline - Date.now();
|
||||
if (remaining < margin) return null;
|
||||
return send(
|
||||
true,
|
||||
Math.floor(
|
||||
Math.min(
|
||||
timeoutMs,
|
||||
remaining / (1 + CONNECTION_WAIT_SHARE),
|
||||
),
|
||||
),
|
||||
);
|
||||
};
|
||||
const outOfTime = (): WorkerInvokeResult => ({
|
||||
outcome: 'unavailable',
|
||||
status: null,
|
||||
error: 'out of time to resend after deploy',
|
||||
});
|
||||
|
||||
let result = await send(false);
|
||||
if (this.#missHandler && selfDeployable(result)) {
|
||||
const outcome = await this.#missHandler(
|
||||
request.appUid,
|
||||
request.script,
|
||||
{ lane: options.lane },
|
||||
// The deploy may be shared with other callers, so it is left
|
||||
// running when this call runs out of time for it.
|
||||
const outcome = await withinTime(
|
||||
this.#missHandler(request.appUid, request.script, {
|
||||
lane: options.lane,
|
||||
...(request.holder ? { holder: request.holder } : {}),
|
||||
}),
|
||||
deadline - Date.now() - margin,
|
||||
);
|
||||
if (outcome === null) return outOfTime();
|
||||
if (outcome !== 'deployed')
|
||||
return {
|
||||
outcome: 'unavailable',
|
||||
@@ -355,7 +470,9 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
error: `deploy: ${outcome}`,
|
||||
...(outcome === DEPLOY_BUSY ? { deployBusy: true } : {}),
|
||||
};
|
||||
result = await send(true);
|
||||
const resent = await resend();
|
||||
if (!resent) return outOfTime();
|
||||
result = resent;
|
||||
} else if (
|
||||
result.status === null &&
|
||||
result.dispatchReason &&
|
||||
@@ -363,7 +480,9 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
) {
|
||||
// No deploy to trigger — just give propagation the one retry
|
||||
// it usually needs, without spending an upload on it.
|
||||
result = await send(true);
|
||||
const resent = await resend();
|
||||
if (!resent) return outOfTime();
|
||||
result = resent;
|
||||
}
|
||||
|
||||
const { status, handled, error, timedOut, retryAfterMs } = result;
|
||||
@@ -434,6 +553,8 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
interface BatchedCall {
|
||||
call: EventsInvokeCall;
|
||||
resolve: (answer: EventsInvokeAnswer) => void;
|
||||
/** For a `beforeSend` that threw. */
|
||||
reject: (err: unknown) => void;
|
||||
/** When its wait for a connection began, for what is left of it. */
|
||||
waitingSince: number;
|
||||
}
|
||||
@@ -504,9 +625,14 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
error: 'no free connection to the events dispatcher',
|
||||
};
|
||||
try {
|
||||
if (this.#batchSize === 1) return await this.#sendOne(call);
|
||||
return await new Promise<EventsInvokeAnswer>((resolve) => {
|
||||
this.#batch.push({ call, resolve, waitingSince });
|
||||
if (this.#batchSize === 1) {
|
||||
// The wait for a connection may have outlived the call.
|
||||
if (call.beforeSend && !(await call.beforeSend()))
|
||||
return supersededAnswer();
|
||||
return await this.#sendOne(call);
|
||||
}
|
||||
return await new Promise<EventsInvokeAnswer>((resolve, reject) => {
|
||||
this.#batch.push({ call, resolve, reject, waitingSince });
|
||||
if (this.#batch.length >= this.#batchSize) this.#flush();
|
||||
else
|
||||
this.#batchTimer ??= setTimeout(
|
||||
@@ -526,6 +652,12 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
const batch = this.#batch;
|
||||
this.#batch = [];
|
||||
if (batch.length === 0) return;
|
||||
if (batch.some(({ call }) => call.beforeSend))
|
||||
void this.#sendStillCurrent(batch);
|
||||
else this.#sendGathered(batch);
|
||||
}
|
||||
|
||||
#sendGathered(batch: BatchedCall[]): void {
|
||||
if (batch.length === 1 || Date.now() < this.#batchUnsupportedUntil) {
|
||||
for (const { call, resolve } of batch)
|
||||
void this.#sendOne(call).then(resolve);
|
||||
@@ -534,6 +666,26 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
void this.#sendBatch(batch);
|
||||
}
|
||||
|
||||
/** Drops calls superseded while they were gathered, then sends the rest. */
|
||||
async #sendStillCurrent(batch: BatchedCall[]): Promise<void> {
|
||||
const checked = await Promise.all(
|
||||
batch.map(async (entry) => {
|
||||
if (!entry.call.beforeSend) return entry;
|
||||
try {
|
||||
if (await entry.call.beforeSend()) return entry;
|
||||
entry.resolve(supersededAnswer());
|
||||
} catch (err) {
|
||||
entry.reject(err);
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
);
|
||||
const current = checked.filter(
|
||||
(entry): entry is BatchedCall => entry !== null,
|
||||
);
|
||||
if (current.length > 0) this.#sendGathered(current);
|
||||
}
|
||||
|
||||
/** One call on the single route. */
|
||||
async #sendOne(call: EventsInvokeCall): Promise<EventsInvokeAnswer> {
|
||||
try {
|
||||
@@ -546,6 +698,9 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
'x-puter-internal-auth': this.#secret,
|
||||
'x-puter-events-script': call.script,
|
||||
'x-puter-events-app': call.appUid,
|
||||
...(call.holder
|
||||
? { [EVENTS_HOLDER_HEADER]: call.holder }
|
||||
: {}),
|
||||
'x-puter-events-key': call.key,
|
||||
...(call.deployed
|
||||
? { [EVENTS_DEPLOYED_HEADER]: '1' }
|
||||
@@ -585,13 +740,15 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
let open = batch.length;
|
||||
const answered = batch.map(() => false);
|
||||
const timers: ReturnType<typeof setTimeout>[] = [];
|
||||
const settle = (i: number, answer: EventsInvokeAnswer): void => {
|
||||
const finish = (i: number, done: (entry: BatchedCall) => void) => {
|
||||
if (answered[i]) return;
|
||||
answered[i] = true;
|
||||
clearTimeout(timers[i]);
|
||||
batch[i].resolve(answer);
|
||||
done(batch[i]);
|
||||
if (--open === 0) hangUp.abort();
|
||||
};
|
||||
const settle = (i: number, answer: EventsInvokeAnswer): void =>
|
||||
finish(i, (entry) => entry.resolve(answer));
|
||||
const settleRest = (answer: EventsInvokeAnswer): void => {
|
||||
for (let i = 0; i < batch.length; i++) settle(i, answer);
|
||||
};
|
||||
@@ -610,6 +767,7 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
calls: batch.map(({ call }, i) => ({
|
||||
script: call.script,
|
||||
appUid: call.appUid,
|
||||
...(call.holder ? { holder: call.holder } : {}),
|
||||
key: call.key,
|
||||
...(call.deployed ? { deployed: true } : {}),
|
||||
body: call.render ? call.render() : call.body,
|
||||
@@ -673,8 +831,22 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
this.#batchUnsupportedUntil =
|
||||
Date.now() + BATCH_UNSUPPORTED_RETRY_MS;
|
||||
batch.forEach(({ call }, i) => {
|
||||
if (!answered[i])
|
||||
if (answered[i]) return;
|
||||
const { beforeSend } = call;
|
||||
if (!beforeSend) {
|
||||
void this.#sendOne(call).then((a) => settle(i, a));
|
||||
return;
|
||||
}
|
||||
// The batch round trip may have outlived the call.
|
||||
void (async () => {
|
||||
try {
|
||||
if (!(await beforeSend()))
|
||||
return settle(i, supersededAnswer());
|
||||
} catch (err) {
|
||||
return finish(i, (entry) => entry.reject(err));
|
||||
}
|
||||
settle(i, await this.#sendOne(call));
|
||||
})();
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1073,62 +1073,6 @@ describe('AuthController.handleSignup', () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('announces the utm params a form signup carried', async () => {
|
||||
const baseline = heardSignupSuccess.length;
|
||||
const username = `utm_${uniq()}`;
|
||||
await controller.handleSignup(
|
||||
makeReq({
|
||||
username,
|
||||
email: `${username}@test.local`,
|
||||
password: 'correct-horse-battery',
|
||||
utm_source: 'newsletter',
|
||||
utm_campaign: 'launch',
|
||||
utm_id: 'not-tracked',
|
||||
}),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
const evt = heardSignupSuccess
|
||||
.slice(baseline)
|
||||
.find((e) => e.username === username);
|
||||
expect(evt?.utm).toEqual({ source: 'newsletter', campaign: 'launch' });
|
||||
expect(evt).not.toHaveProperty('password');
|
||||
});
|
||||
|
||||
it('announces utm on a temp signup without exposing its placeholder email', async () => {
|
||||
const baseline = heardSignupSuccess.length;
|
||||
const res = makeRes();
|
||||
await controller.handleSignup(
|
||||
makeReq({ is_temp: true, utm_source: 'ads' }),
|
||||
res,
|
||||
);
|
||||
|
||||
const evt = heardSignupSuccess
|
||||
.slice(baseline)
|
||||
.find((e) => e.is_temp === true);
|
||||
expect(evt?.utm).toEqual({ source: 'ads' });
|
||||
expect(evt?.email).toBeNull();
|
||||
});
|
||||
|
||||
it('leaves utm off the event when the signup carried none', async () => {
|
||||
const baseline = heardSignupSuccess.length;
|
||||
const username = `noutm_${uniq()}`;
|
||||
await controller.handleSignup(
|
||||
makeReq({
|
||||
username,
|
||||
email: `${username}@test.local`,
|
||||
password: 'correct-horse-battery',
|
||||
}),
|
||||
makeRes(),
|
||||
);
|
||||
|
||||
const evt = heardSignupSuccess
|
||||
.slice(baseline)
|
||||
.find((e) => e.username === username);
|
||||
expect(evt).toBeDefined();
|
||||
expect(evt).not.toHaveProperty('utm');
|
||||
});
|
||||
|
||||
it('still allows claiming a pseudo-user row when registration is disabled', async () => {
|
||||
const authConfig = server.controllers.auth.config as {
|
||||
disable_user_signup?: boolean;
|
||||
|
||||
@@ -77,7 +77,6 @@ import {
|
||||
isAbsentBonusCode,
|
||||
normalizeBonusCode,
|
||||
} from '../../util/signupBonus.js';
|
||||
import { readSignupUtm } from '../../util/signupUtm.js';
|
||||
import { isTemporaryPasswordExpired } from '../../util/temporaryPassword.js';
|
||||
import { buildUserDetails, scrubSensitive } from '../../util/userDetails.js';
|
||||
import {
|
||||
@@ -889,7 +888,6 @@ export class AuthController extends PuterController {
|
||||
isTemp: is_temp,
|
||||
fingerprint,
|
||||
bonusCode,
|
||||
utm: readSignupUtm(body),
|
||||
});
|
||||
await this.#completeLogin(req, res, user);
|
||||
}
|
||||
|
||||
@@ -64,6 +64,8 @@ const EVENTS_SCRIPT_REGEX = new RegExp(`^${EVENTS_WORKER_PREFIX}[a-f0-9]{32}$`);
|
||||
|
||||
/** Matches the edge dispatcher's own validation of the app uid it forwards. */
|
||||
const APP_UID_REGEX = /^app-[A-Za-z0-9_-]{1,64}$/;
|
||||
/** A user uuid. */
|
||||
const HOLDER_REGEX = /^[A-Za-z0-9-]{1,64}$/;
|
||||
|
||||
@Controller('/events')
|
||||
export class EventsController extends PuterController {
|
||||
@@ -132,7 +134,8 @@ export class EventsController extends PuterController {
|
||||
*
|
||||
* Stateless: no subscription, no stored position, nothing written. The
|
||||
* cursor comes back to the caller and goes out again as `cursor` (or
|
||||
* `after`, the name it first shipped under).
|
||||
* `after`, the name it first shipped under). `from=end` with no cursor
|
||||
* starts at the newest row instead of the oldest.
|
||||
*/
|
||||
@Get('/fetch', {
|
||||
subdomain: 'api',
|
||||
@@ -143,9 +146,16 @@ export class EventsController extends PuterController {
|
||||
async fetch(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const query = (req.query ?? {}) as Record<string, unknown>;
|
||||
const from = query.from as 'start' | 'end' | undefined;
|
||||
if (from !== undefined && from !== 'start' && from !== 'end') {
|
||||
throw new HttpError(400, '`from` must be `start` or `end`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
|
||||
const page = await this.services.events.fetchMissed(actor, {
|
||||
subject: typeof query.subject === 'string' ? query.subject : '',
|
||||
from,
|
||||
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
|
||||
after: typeof query.after === 'string' ? query.after : undefined,
|
||||
limit: normalizeLimit(query.limit, {
|
||||
@@ -226,6 +236,17 @@ export class EventsController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
@Post('/schedules/update', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
})
|
||||
async updateSchedule(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const { scheduleId, changes } = bodyRecord(req);
|
||||
res.json(await this.#schedules().update(actor, scheduleId, changes));
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /events/schedules/remove — end one schedule. An id the caller does
|
||||
* not hold reads as absent.
|
||||
@@ -402,6 +423,7 @@ export class EventsController extends PuterController {
|
||||
handlers: await this.services.events.listHandlers(actor, {
|
||||
appUid:
|
||||
typeof query.appUid === 'string' ? query.appUid : undefined,
|
||||
scope: query.scope,
|
||||
}),
|
||||
});
|
||||
}
|
||||
@@ -500,6 +522,15 @@ export class EventsController extends PuterController {
|
||||
throw new HttpError(400, 'Missing or invalid `script`/`appUid`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
// Absent or empty is the app's own scope.
|
||||
const holder = body.holder === undefined ? '' : body.holder;
|
||||
if (
|
||||
typeof holder !== 'string' ||
|
||||
!(holder === '' || HOLDER_REGEX.test(holder))
|
||||
)
|
||||
throw new HttpError(400, 'Invalid `holder`', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
|
||||
const deployer = this.#eventsWorkerDeployer();
|
||||
if (!deployer.enabled) {
|
||||
@@ -507,7 +538,13 @@ export class EventsController extends PuterController {
|
||||
return;
|
||||
}
|
||||
|
||||
const outcome = await deployer.ensure(appUid, script);
|
||||
const scope = await deployer.scopeFor(appUid, holder);
|
||||
if (!scope) {
|
||||
res.status(404).json({ deployed: false, reason: 'no-holder' });
|
||||
return;
|
||||
}
|
||||
|
||||
const outcome = await deployer.ensure(scope, script);
|
||||
if (outcome === 'deployed') {
|
||||
res.json({ deployed: true });
|
||||
return;
|
||||
@@ -540,14 +577,22 @@ export class EventsController extends PuterController {
|
||||
override onServerStart(): void {
|
||||
const deployer = this.#eventsWorkerDeployer();
|
||||
if (!deployer.enabled) return;
|
||||
this.services.events.useEagerDeploy((scope, script) =>
|
||||
deployer.deployEagerly(scope, script),
|
||||
);
|
||||
if (this.config.workers?.localServer) {
|
||||
this.services.events.useWorkerTransport(
|
||||
new LocalEventsInvokeTransport(this.services, deployer),
|
||||
);
|
||||
return;
|
||||
}
|
||||
this.services.events.useWorkerMissHandler((appUid, script, options) =>
|
||||
deployer.ensure(appUid, script, options),
|
||||
this.services.events.useWorkerMissHandler(
|
||||
async (appUid, script, options) => {
|
||||
const scope = await deployer.scopeFor(appUid, options.holder);
|
||||
return scope
|
||||
? deployer.ensure(scope, script, options)
|
||||
: 'no-holder';
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -34,12 +34,21 @@ import {
|
||||
EVENTS_DURABLE_SUBSCRIPTIONS_MAX,
|
||||
EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP,
|
||||
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER,
|
||||
EVENTS_HANDLER_PUBLISH_LIMIT,
|
||||
EVENTS_HANDLER_SOURCE_MAX_BYTES,
|
||||
EVENTS_HANDLERS_PER_APP,
|
||||
EVENTS_KV_HANDLES_PER_APP,
|
||||
EVENTS_KV_HANDLES_PER_USER,
|
||||
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT,
|
||||
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT,
|
||||
EVENTS_SINGLE_DELIVERY_LIMIT,
|
||||
EVENTS_USER_HANDLERS_PER_SCOPE,
|
||||
EVENTS_USER_SCOPE_SOURCE_MAX_BYTES,
|
||||
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
|
||||
EVENTS_WORKER_DEPLOYS_PER_HOUR,
|
||||
EVENTS_WORKER_INVOCATION_LIMIT,
|
||||
EVENTS_WORKER_SOURCE_MAX_BYTES,
|
||||
RENTED_UNITS_PER_USER,
|
||||
limitFor,
|
||||
tiered,
|
||||
type TieredLimit,
|
||||
@@ -183,3 +192,42 @@ describe('the delivery budgets', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('the handler limits', () => {
|
||||
it('pins the app-scope numbers the quotas page lists', () => {
|
||||
expect(EVENTS_HANDLERS_PER_APP).toBe(100);
|
||||
expect(EVENTS_HANDLER_SOURCE_MAX_BYTES).toBe(64 * 1024);
|
||||
expect(EVENTS_WORKER_SOURCE_MAX_BYTES).toBe(5 * 1024 * 1024);
|
||||
expect(EVENTS_WORKER_DEPLOYS_PER_HOUR).toBe(30);
|
||||
expect(EVENTS_HANDLER_PUBLISH_LIMIT.limit).toBe(60);
|
||||
expect(EVENTS_HANDLER_PUBLISH_LIMIT.window).toBe(60_000);
|
||||
});
|
||||
|
||||
it('pins the user-scope numbers the quotas page lists', () => {
|
||||
expect(EVENTS_USER_HANDLERS_PER_SCOPE).toBe(20);
|
||||
expect(EVENTS_USER_SCOPE_SOURCE_MAX_BYTES).toBe(1024 * 1024);
|
||||
expect(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR).toBe(10);
|
||||
});
|
||||
|
||||
it('keeps a user scope tighter than an app scope', () => {
|
||||
expect(EVENTS_USER_HANDLERS_PER_SCOPE).toBeLessThan(
|
||||
EVENTS_HANDLERS_PER_APP,
|
||||
);
|
||||
expect(EVENTS_USER_SCOPE_SOURCE_MAX_BYTES).toBeLessThan(
|
||||
EVENTS_WORKER_SOURCE_MAX_BYTES,
|
||||
);
|
||||
expect(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR).toBeLessThan(
|
||||
EVENTS_WORKER_DEPLOYS_PER_HOUR,
|
||||
);
|
||||
});
|
||||
|
||||
it('pins the rented-unit cap by plan', () => {
|
||||
expect(RENTED_UNITS_PER_USER.limit).toBe(100);
|
||||
expect(limitFor(RENTED_UNITS_PER_USER, DEFAULT_FREE_SUBSCRIPTION)).toBe(
|
||||
16,
|
||||
);
|
||||
expect(limitFor(RENTED_UNITS_PER_USER, DEFAULT_TEMP_SUBSCRIPTION)).toBe(
|
||||
16,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -269,6 +269,12 @@ export const KV_SHARE_NOTIFY_LIMITS = [
|
||||
*/
|
||||
export const EVENTS_HANDLERS_PER_APP = 100;
|
||||
|
||||
/**
|
||||
* Handlers one user may hold in their own scope of one app. Each is a script of
|
||||
* its own, so the cap is tighter than the app's.
|
||||
*/
|
||||
export const EVENTS_USER_HANDLERS_PER_SCOPE = 20;
|
||||
|
||||
/** Longest a serialized handler may be. */
|
||||
export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
|
||||
|
||||
@@ -280,6 +286,9 @@ export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
|
||||
*/
|
||||
export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
|
||||
|
||||
/** Source one user scope may hold, all its handlers combined. */
|
||||
export const EVENTS_USER_SCOPE_SOURCE_MAX_BYTES = 1024 * 1024;
|
||||
|
||||
/**
|
||||
* Deploys one app's events worker may trigger per hour.
|
||||
*
|
||||
@@ -289,6 +298,16 @@ export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
|
||||
*/
|
||||
export const EVENTS_WORKER_DEPLOYS_PER_HOUR = 30;
|
||||
|
||||
/** Deploys one user scope may trigger per hour. */
|
||||
export const EVENTS_USER_WORKER_DEPLOYS_PER_HOUR = 10;
|
||||
|
||||
/**
|
||||
* Units one user may rent at once: ordinary workers, user scopes, and the
|
||||
* events workers of apps they own. Only creating a unit is refused; holdings
|
||||
* over the cap are kept.
|
||||
*/
|
||||
export const RENTED_UNITS_PER_USER = tiered(100, 16, 16);
|
||||
|
||||
/**
|
||||
* Handlers one `publishAll` may carry. A build step publishes its whole set in
|
||||
* one call, and the set is capped by what an app may hold anyway.
|
||||
|
||||
@@ -75,6 +75,7 @@ const fakeSchedules = () =>
|
||||
total: 3,
|
||||
})),
|
||||
get: vi.fn(async () => view),
|
||||
update: vi.fn(async () => view),
|
||||
remove: vi.fn(async () => {}),
|
||||
receivePeerChanges: vi.fn(async () => ({ applied: 2 })),
|
||||
prepareSendAt: vi.fn(),
|
||||
@@ -129,6 +130,10 @@ const routes: Array<[string, (response: Response) => Promise<void>]> = [
|
||||
(r) =>
|
||||
events.getSchedule(req({ params: { scheduleId: SCHEDULE_ID } }), r),
|
||||
],
|
||||
[
|
||||
'POST /events/schedules/update',
|
||||
(r) => events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes: { every: '2h' } } }), r),
|
||||
],
|
||||
[
|
||||
'POST /events/schedules/remove',
|
||||
(r) =>
|
||||
@@ -250,6 +255,16 @@ describe('with services.schedules', () => {
|
||||
expect(sent.body).toEqual(view);
|
||||
});
|
||||
|
||||
it('updates through update and answers its view', async () => {
|
||||
const schedules = fakeSchedules();
|
||||
setSchedules(schedules);
|
||||
const { response, sent } = res();
|
||||
const changes = { every: '2h' };
|
||||
await events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes } }), response);
|
||||
expect(schedules.update).toHaveBeenCalledWith(actor, SCHEDULE_ID, changes);
|
||||
expect(sent.body).toEqual(view);
|
||||
});
|
||||
|
||||
it('removes through remove and answers {}', async () => {
|
||||
const schedules = fakeSchedules();
|
||||
setSchedules(schedules);
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import type { Request, Response } from 'express';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { makeActor } from '../../core/actor.js';
|
||||
import type {
|
||||
ScheduleServiceContract,
|
||||
ScheduleView,
|
||||
} from '../../core/schedules.js';
|
||||
import { EventsController } from './EventsController.js';
|
||||
|
||||
const actor = makeActor({
|
||||
user: { id: 1, uuid: 'user-1', username: 'user1' } as never,
|
||||
});
|
||||
const request = (body: unknown) => ({ actor, body }) as Request;
|
||||
const controller = (schedules?: Pick<ScheduleServiceContract, 'update'>) =>
|
||||
new EventsController(
|
||||
{} as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
{ schedules } as never,
|
||||
{} as never,
|
||||
);
|
||||
|
||||
describe('update schedule route contract', () => {
|
||||
it('answers events_disabled when schedules are unavailable', async () => {
|
||||
await expect(
|
||||
controller().updateSchedule(
|
||||
request({ scheduleId: 'id', changes: {} }),
|
||||
{} as Response,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'events_disabled',
|
||||
statusCode: 503,
|
||||
noAlarm: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('passes the actor, id and changes through and returns the updated view', async () => {
|
||||
const view = {
|
||||
scheduleId: 'id',
|
||||
kind: 'recurring',
|
||||
state: 'suspended',
|
||||
} as ScheduleView;
|
||||
const changes = { cron: '0 9 * * *', tz: 'UTC' };
|
||||
const calls: unknown[][] = [];
|
||||
const events = controller({
|
||||
update: async (...args) => {
|
||||
calls.push(args);
|
||||
return view;
|
||||
},
|
||||
});
|
||||
let sent: unknown;
|
||||
await events.updateSchedule(request({ scheduleId: 'id', changes }), {
|
||||
json: (value: unknown) => {
|
||||
sent = value;
|
||||
},
|
||||
} as Response);
|
||||
expect(calls).toEqual([[actor, 'id', changes]]);
|
||||
expect(sent).toBe(view);
|
||||
});
|
||||
|
||||
it('preserves validation and sending errors from the implementation', async () => {
|
||||
const error = {
|
||||
code: 'events_schedule_sending',
|
||||
message: 'Already sending',
|
||||
};
|
||||
await expect(
|
||||
controller({
|
||||
update: async () => {
|
||||
throw error;
|
||||
},
|
||||
}).updateSchedule(
|
||||
request({ scheduleId: 'id', changes: {} }),
|
||||
{} as Response,
|
||||
),
|
||||
).rejects.toBe(error);
|
||||
});
|
||||
});
|
||||
@@ -25,14 +25,35 @@
|
||||
|
||||
import MockRedis from 'ioredis-mock';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { EventsInvokeCall } from '../../clients/events/EventsWorkerInvokerClient.js';
|
||||
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
|
||||
import type { fetch as undiciFetch } from 'undici';
|
||||
import {
|
||||
DispatcherInvokeTransport,
|
||||
EVENTS_DEPLOYED_HEADER,
|
||||
EVENTS_DISPATCH_ERROR_HEADER,
|
||||
EVENTS_HANDLED_HEADER,
|
||||
EventsWorkerInvokerClient,
|
||||
type EventsInvokeCall,
|
||||
} from '../../clients/events/EventsWorkerInvokerClient.js';
|
||||
import { INTERNAL_DEPLOY_TARGET } from '../../drivers/workers/WorkerDriver.js';
|
||||
import {
|
||||
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
|
||||
EVENTS_WORKER_DEPLOYS_PER_HOUR,
|
||||
} from './limits.js';
|
||||
import {
|
||||
EventsWorkerDeployer,
|
||||
LocalEventsInvokeTransport,
|
||||
type EventsDeployOptions,
|
||||
type EventsDeployOutcome,
|
||||
} from './workerDeploy.js';
|
||||
import { eventsWorkerScript } from '../../services/events/workerRuntime.js';
|
||||
import {
|
||||
eventsWorkerScript,
|
||||
userHandlerScript,
|
||||
} from '../../services/events/workerRuntime.js';
|
||||
import {
|
||||
appHandlerScope,
|
||||
hashContent,
|
||||
type HandlerScope,
|
||||
} from '../../stores/events/EventHandlerStore.js';
|
||||
import { handlerSetHash } from '../../services/events/workerSource.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
|
||||
@@ -51,14 +72,22 @@ interface Owner {
|
||||
type Layers = ConstructorParameters<typeof EventsWorkerDeployer>[0];
|
||||
|
||||
/** Stub layers whose only real dependency is a fresh redis mock. */
|
||||
const makeLayers = (owners: Record<string, Owner | null>) => {
|
||||
const makeLayers = (
|
||||
owners: Record<string, Owner | null>,
|
||||
events: Record<string, unknown> = {},
|
||||
) => {
|
||||
const redis = new MockRedis.Cluster(['redis://localhost:7001']);
|
||||
const created: string[] = [];
|
||||
const targets: Array<Record<string, unknown>> = [];
|
||||
/** Apps whose handler sources were read, once per read. */
|
||||
const sourceReads: string[] = [];
|
||||
const layers = {
|
||||
config: {
|
||||
events: { workerRuntime: true, internalSecret: 'sekret' },
|
||||
events: {
|
||||
workerRuntime: true,
|
||||
internalSecret: 'sekret',
|
||||
...events,
|
||||
},
|
||||
// `#namespace()` needs a local-worker shape to resolve without
|
||||
// throwing — this stub never reaches a real Cloudflare or
|
||||
// Miniflare deploy either way.
|
||||
@@ -68,7 +97,7 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
|
||||
stores: {
|
||||
eventHandler: {
|
||||
setForApp: async () => [{ name: 'h', sourceHash: SOURCE_HASH }],
|
||||
allForApp: async (appUid: string) => {
|
||||
allForApp: async ({ appUid }: { appUid: string }) => {
|
||||
sourceReads.push(appUid);
|
||||
return [
|
||||
{
|
||||
@@ -91,6 +120,18 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
|
||||
},
|
||||
},
|
||||
user: {
|
||||
getByUuid: async (uuid: string) => {
|
||||
const owner = Object.values(owners).find(
|
||||
(o) => `uuid-${o?.ownerUserId}` === uuid,
|
||||
);
|
||||
if (!owner) return null;
|
||||
return {
|
||||
id: owner.ownerUserId,
|
||||
uuid,
|
||||
username: `user-${owner.ownerUserId}`,
|
||||
suspended: owner.suspended ?? false,
|
||||
};
|
||||
},
|
||||
getById: async (id: number) => {
|
||||
const owner = Object.values(owners).find(
|
||||
(o) => o?.ownerUserId === id,
|
||||
@@ -108,8 +149,12 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
|
||||
services: {},
|
||||
drivers: {
|
||||
workers: {
|
||||
create: async (args: { workerName: string }) => {
|
||||
create: async (args: {
|
||||
workerName: string;
|
||||
[INTERNAL_DEPLOY_TARGET]?: Record<string, unknown>;
|
||||
}) => {
|
||||
created.push(args.workerName);
|
||||
targets.push(args[INTERNAL_DEPLOY_TARGET] ?? {});
|
||||
return { success: true, errors: [] };
|
||||
},
|
||||
},
|
||||
@@ -118,6 +163,7 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
|
||||
return {
|
||||
layers: layers as unknown as Layers,
|
||||
created,
|
||||
targets,
|
||||
redis,
|
||||
sourceReads,
|
||||
};
|
||||
@@ -136,8 +182,8 @@ describe('EventsWorkerDeployer.ensure', () => {
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
|
||||
const [a, b] = await Promise.all([
|
||||
deployer.ensure('app-a', SCRIPT),
|
||||
deployer.ensure('app-b', SCRIPT),
|
||||
deployer.ensure(appHandlerScope('app-a'), SCRIPT),
|
||||
deployer.ensure(appHandlerScope('app-b'), SCRIPT),
|
||||
]);
|
||||
expect(a).toBe('deployed');
|
||||
expect(b).toBe('no-owner');
|
||||
@@ -154,7 +200,9 @@ describe('EventsWorkerDeployer.ensure', () => {
|
||||
String(EVENTS_WORKER_DEPLOYS_PER_HOUR),
|
||||
);
|
||||
|
||||
expect(await deployer.ensure('app-c', SCRIPT)).toBe('throttled');
|
||||
expect(await deployer.ensure(appHandlerScope('app-c'), SCRIPT)).toBe(
|
||||
'throttled',
|
||||
);
|
||||
expect(sourceReads).toEqual([]);
|
||||
expect(created).toEqual([]);
|
||||
});
|
||||
@@ -169,16 +217,41 @@ describe('EventsWorkerDeployer.ensure', () => {
|
||||
'',
|
||||
);
|
||||
|
||||
expect(await deployer.ensure('app-e', stale)).toBe('stale');
|
||||
expect(await deployer.ensure(appHandlerScope('app-e'), stale)).toBe(
|
||||
'stale',
|
||||
);
|
||||
expect(sourceReads).toEqual([]);
|
||||
const hour = new Date().toISOString().slice(0, 13);
|
||||
expect(await redis.get(`ev:deploys:{app-e}:${hour}`)).toBeNull();
|
||||
});
|
||||
|
||||
it('passes events.workerCpuMs to the deploy target', async () => {
|
||||
const { layers, targets } = makeLayers(
|
||||
{ 'app-f': { ownerUserId: 6 } },
|
||||
{ workerCpuMs: 50 },
|
||||
);
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
expect(await deployer.ensure(appHandlerScope('app-f'), SCRIPT)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(targets[0]!.cpuMs).toBe(50);
|
||||
});
|
||||
|
||||
it('passes no cpuMs when events.workerCpuMs is unset', async () => {
|
||||
const { layers, targets } = makeLayers({ 'app-g': { ownerUserId: 7 } });
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
expect(await deployer.ensure(appHandlerScope('app-g'), SCRIPT)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(targets[0]!.cpuMs).toBeUndefined();
|
||||
});
|
||||
|
||||
it('deploys normally under the cap', async () => {
|
||||
const { layers } = makeLayers({ 'app-d': { ownerUserId: 4 } });
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
expect(await deployer.ensure('app-d', SCRIPT)).toBe('deployed');
|
||||
expect(await deployer.ensure(appHandlerScope('app-d'), SCRIPT)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -209,16 +282,22 @@ describe('the schedules` deploy gate', () => {
|
||||
it('is skipped without schedules, or with schedules that have no gate', async () => {
|
||||
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
|
||||
await expect(
|
||||
new EventsWorkerDeployer(layers).ensure('app-x', SCRIPT, {
|
||||
lane: 'schedule',
|
||||
}),
|
||||
new EventsWorkerDeployer(layers).ensure(
|
||||
appHandlerScope('app-x'),
|
||||
SCRIPT,
|
||||
{
|
||||
lane: 'schedule',
|
||||
},
|
||||
),
|
||||
).resolves.toBe('deployed');
|
||||
const gateless = new EventsWorkerDeployer({
|
||||
...layers,
|
||||
services: { ...layers.services, schedules: {} },
|
||||
} as unknown as Layers);
|
||||
await expect(
|
||||
gateless.ensure('app-x', SCRIPT, { lane: 'schedule' }),
|
||||
gateless.ensure(appHandlerScope('app-x'), SCRIPT, {
|
||||
lane: 'schedule',
|
||||
}),
|
||||
).resolves.toBe('deployed');
|
||||
expect(created).toEqual([SCRIPT, SCRIPT]);
|
||||
});
|
||||
@@ -227,11 +306,13 @@ describe('the schedules` deploy gate', () => {
|
||||
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
|
||||
const { deployer, gated } = withGate(layers);
|
||||
|
||||
await expect(deployer.ensure('app-x', SCRIPT)).resolves.toBe(
|
||||
'deployed',
|
||||
);
|
||||
await expect(
|
||||
deployer.ensure('app-x', SCRIPT, { lane: 'schedule' }),
|
||||
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
|
||||
).resolves.toBe('deployed');
|
||||
await expect(
|
||||
deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
|
||||
lane: 'schedule',
|
||||
}),
|
||||
).resolves.toBe('deployed');
|
||||
expect(gated).toEqual([
|
||||
{ script: SCRIPT, capped: false },
|
||||
@@ -245,11 +326,13 @@ describe('the schedules` deploy gate', () => {
|
||||
const { deployer } = withGate(layers, (capped) => capped);
|
||||
|
||||
await expect(
|
||||
deployer.ensure('app-x', SCRIPT, { lane: 'schedule' }),
|
||||
deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
|
||||
lane: 'schedule',
|
||||
}),
|
||||
).resolves.toBe('busy');
|
||||
await expect(deployer.ensure('app-x', SCRIPT)).resolves.toBe(
|
||||
'deployed',
|
||||
);
|
||||
await expect(
|
||||
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
|
||||
).resolves.toBe('deployed');
|
||||
expect(created).toEqual([SCRIPT]);
|
||||
});
|
||||
|
||||
@@ -257,12 +340,12 @@ describe('the schedules` deploy gate', () => {
|
||||
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
|
||||
const { deployer, gated } = withGate(layers, (capped) => capped);
|
||||
|
||||
const scheduled = deployer.ensure('app-x', SCRIPT, {
|
||||
const scheduled = deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
|
||||
lane: 'schedule',
|
||||
});
|
||||
const deliveries = [
|
||||
deployer.ensure('app-x', SCRIPT),
|
||||
deployer.ensure('app-x', SCRIPT),
|
||||
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
|
||||
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
|
||||
];
|
||||
await expect(scheduled).resolves.toBe('busy');
|
||||
await expect(Promise.all(deliveries)).resolves.toEqual([
|
||||
@@ -277,8 +360,8 @@ describe('the schedules` deploy gate', () => {
|
||||
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
|
||||
const { deployer, gated } = withGate(layers, (capped) => capped);
|
||||
|
||||
const delivery = deployer.ensure('app-x', SCRIPT);
|
||||
const scheduled = deployer.ensure('app-x', SCRIPT, {
|
||||
const delivery = deployer.ensure(appHandlerScope('app-x'), SCRIPT);
|
||||
const scheduled = deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
|
||||
lane: 'schedule',
|
||||
});
|
||||
await expect(Promise.all([delivery, scheduled])).resolves.toEqual([
|
||||
@@ -312,6 +395,7 @@ describe('LocalEventsInvokeTransport', () => {
|
||||
resident = true;
|
||||
return 'deployed';
|
||||
},
|
||||
scopeFor: async (appUid: string) => appHandlerScope(appUid),
|
||||
} as unknown as EventsWorkerDeployer,
|
||||
);
|
||||
let renders = 0;
|
||||
@@ -329,3 +413,484 @@ describe('LocalEventsInvokeTransport', () => {
|
||||
expect(bodies).toEqual(['{"n":1}', '{"n":2}']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('deploy-on-miss of an evicted script', () => {
|
||||
it('redeploys the evicted script, then resends the run with the deployed marker', async () => {
|
||||
const { layers, created } = makeLayers({ 'app-m': { ownerUserId: 8 } });
|
||||
// The namespace only has the script once the deploy lands: before
|
||||
// that the dispatcher answers a miss.
|
||||
let resident = false;
|
||||
layers.drivers.workers.create = (async (args: {
|
||||
workerName: string;
|
||||
}) => {
|
||||
created.push(args.workerName);
|
||||
resident = true;
|
||||
return { success: true, errors: [] };
|
||||
}) as never;
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
|
||||
const sent: boolean[] = [];
|
||||
const fetchImpl = (async (_url: string, init?: RequestInit) => {
|
||||
const headers = (init?.headers ?? {}) as Record<string, string>;
|
||||
sent.push(headers[EVENTS_DEPLOYED_HEADER] === '1');
|
||||
if (resident)
|
||||
return new Response(null, {
|
||||
status: 200,
|
||||
headers: { [EVENTS_HANDLED_HEADER]: '1' },
|
||||
});
|
||||
return new Response(null, {
|
||||
status: 404,
|
||||
headers: { [EVENTS_DISPATCH_ERROR_HEADER]: 'missing' },
|
||||
});
|
||||
}) as unknown as typeof undiciFetch;
|
||||
|
||||
const client = new EventsWorkerInvokerClient({
|
||||
events: { workerRuntime: true },
|
||||
} as IConfig);
|
||||
client.setTransport(
|
||||
new DispatcherInvokeTransport('http://dispatcher', 's', {
|
||||
fetchImpl,
|
||||
}),
|
||||
);
|
||||
client.setMissHandler((appUid, script, options) =>
|
||||
deployer.ensure(appHandlerScope(appUid), script, options),
|
||||
);
|
||||
|
||||
const result = await client.invoke({
|
||||
script: SCRIPT,
|
||||
appUid: 'app-m',
|
||||
handler: 'h',
|
||||
token: 't',
|
||||
key: 'k',
|
||||
event: {},
|
||||
ctx: {},
|
||||
});
|
||||
|
||||
expect(result).toEqual({ outcome: 'settled', status: 200 });
|
||||
expect(created).toEqual([SCRIPT]);
|
||||
// One miss, then the resend carrying the deployed marker.
|
||||
expect(sent).toEqual([false, true]);
|
||||
});
|
||||
});
|
||||
|
||||
// The redis mock is shared across tests, so each one uses its own app and code.
|
||||
let fresh = 0;
|
||||
const nextApp = () => `app-u${++fresh}`;
|
||||
const userScope = (holder: number, appUid: string) => ({ appUid, holder });
|
||||
const sourceN = (label: string) =>
|
||||
`async () => { await ${label}${++fresh}(); }`;
|
||||
const scriptOf = (name: string, source: string) =>
|
||||
userHandlerScript({ name, sourceHash: hashContent(source) }, '');
|
||||
|
||||
type Sets = Record<number, Array<{ name: string; source: string }>>;
|
||||
|
||||
/** Layers whose handler sets are per holder (0 is the app's own). */
|
||||
const withSets = (layers: Layers, sets: Sets) => {
|
||||
const handlersOf = (scope: { appUid: string; holder: number }) =>
|
||||
(sets[scope.holder] ?? []).map((h) => ({
|
||||
appUid: scope.appUid,
|
||||
name: h.name,
|
||||
source: h.source,
|
||||
sourceHash: hashContent(h.source),
|
||||
createdAt: 0,
|
||||
updatedAt: 0,
|
||||
}));
|
||||
const store = layers.stores.eventHandler as unknown as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
store.setForApp = async (scope: { appUid: string; holder: number }) =>
|
||||
handlersOf(scope).map(({ name, sourceHash }) => ({ name, sourceHash }));
|
||||
store.allForApp = async (scope: { appUid: string; holder: number }) =>
|
||||
handlersOf(scope);
|
||||
};
|
||||
|
||||
const holders = (...ids: number[]) =>
|
||||
Object.fromEntries(
|
||||
ids.map((id) => [`holder-${id}`, { ownerUserId: id }]),
|
||||
) as Record<string, Owner>;
|
||||
|
||||
describe('EventsWorkerDeployer in a user scope', () => {
|
||||
it('deploys the one handler whose script was asked for, as the holder', async () => {
|
||||
const app = nextApp();
|
||||
const a = sourceN('a');
|
||||
const b = sourceN('b');
|
||||
const { layers, created, targets } = makeLayers(holders(10));
|
||||
withSets(layers, {
|
||||
10: [
|
||||
{ name: 'a', source: a },
|
||||
{ name: 'b', source: b },
|
||||
],
|
||||
});
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
|
||||
const script = scriptOf('b', b);
|
||||
expect(await deployer.ensure(userScope(10, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
|
||||
expect(created).toEqual([script]);
|
||||
const target = targets[0]!;
|
||||
expect(target.tags).toEqual([`events-app:${app}`, 'user:uuid-10']);
|
||||
// One handler's script: the other one's code is not in it.
|
||||
expect(target.source).toContain(b);
|
||||
expect(target.source).not.toContain(a);
|
||||
});
|
||||
|
||||
it('answers a script no handler of the scope matches as stale, before the throttle', async () => {
|
||||
const app = nextApp();
|
||||
const { layers, redis, created } = makeLayers(holders(10));
|
||||
withSets(layers, { 10: [{ name: 'a', source: sourceN('a') }] });
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
|
||||
expect(
|
||||
await deployer.ensure(
|
||||
userScope(10, app),
|
||||
scriptOf('b', sourceN('b')),
|
||||
),
|
||||
).toBe('stale');
|
||||
expect(created).toEqual([]);
|
||||
const hour = new Date().toISOString().slice(0, 13);
|
||||
expect(await redis.get(`ev:deploys:{${app}}:10:${hour}`)).toBeNull();
|
||||
});
|
||||
|
||||
it('answers a scope with no handlers as no-handlers', async () => {
|
||||
const { layers } = makeLayers(holders(10));
|
||||
withSets(layers, {});
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
expect(
|
||||
await deployer.ensure(
|
||||
userScope(10, nextApp()),
|
||||
scriptOf('a', sourceN('a')),
|
||||
),
|
||||
).toBe('no-handlers');
|
||||
});
|
||||
|
||||
it('does not deploy for a holder that is suspended or gone', async () => {
|
||||
const source = sourceN('a');
|
||||
const { layers, created } = makeLayers({
|
||||
'holder-10': { ownerUserId: 10, suspended: true },
|
||||
});
|
||||
withSets(layers, {
|
||||
10: [{ name: 'a', source }],
|
||||
11: [{ name: 'a', source }],
|
||||
});
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
const script = scriptOf('a', source);
|
||||
const app = nextApp();
|
||||
|
||||
expect(await deployer.ensure(userScope(10, app), script)).toBe(
|
||||
'no-owner',
|
||||
);
|
||||
expect(await deployer.ensure(userScope(11, app), script)).toBe(
|
||||
'no-owner',
|
||||
);
|
||||
expect(created).toEqual([]);
|
||||
});
|
||||
|
||||
it('gives each scope its own hourly cap, apart from the app scope', async () => {
|
||||
const app = nextApp();
|
||||
const source = sourceN('a');
|
||||
const { layers, created } = makeLayers({
|
||||
...holders(10, 11),
|
||||
[app]: { ownerUserId: 12 },
|
||||
});
|
||||
withSets(layers, {
|
||||
0: [{ name: 'a', source }],
|
||||
10: [{ name: 'a', source }],
|
||||
11: [{ name: 'a', source }],
|
||||
});
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
const script = scriptOf('a', source);
|
||||
|
||||
for (let i = 0; i < EVENTS_USER_WORKER_DEPLOYS_PER_HOUR; i++)
|
||||
expect(await deployer.ensure(userScope(10, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(await deployer.ensure(userScope(10, app), script)).toBe(
|
||||
'throttled',
|
||||
);
|
||||
expect(created).toHaveLength(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR);
|
||||
|
||||
// Another holder of the same app, and the app's own scope, still go.
|
||||
expect(await deployer.ensure(userScope(11, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(await deployer.ensure(appHandlerScope(app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
});
|
||||
|
||||
it('collapses concurrent deploys of one script across holders into one upload', async () => {
|
||||
const app = nextApp();
|
||||
const source = sourceN('a');
|
||||
const { layers, created } = makeLayers(holders(10, 11));
|
||||
withSets(layers, {
|
||||
10: [{ name: 'a', source }],
|
||||
11: [{ name: 'a', source }],
|
||||
});
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
const script = scriptOf('a', source);
|
||||
|
||||
await Promise.all([
|
||||
deployer.ensure(userScope(10, app), script),
|
||||
deployer.ensure(userScope(10, app), script),
|
||||
deployer.ensure(userScope(11, app), script),
|
||||
deployer.ensure(userScope(11, nextApp()), script),
|
||||
]);
|
||||
expect(created).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('names a script for identical code the same under any holder and app', async () => {
|
||||
const source = sourceN('a');
|
||||
const { layers, created } = makeLayers(holders(10, 11));
|
||||
withSets(layers, {
|
||||
10: [{ name: '@x', source }],
|
||||
11: [{ name: '@x', source }],
|
||||
});
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
const script = scriptOf('@x', source);
|
||||
|
||||
expect(await deployer.ensure(userScope(10, nextApp()), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(await deployer.ensure(userScope(11, nextApp()), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(created).toEqual([script, script]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the deployed marker', () => {
|
||||
const setup = () => {
|
||||
const source = sourceN('a');
|
||||
const made = makeLayers(holders(10, 11));
|
||||
withSets(made.layers, {
|
||||
10: [{ name: 'a', source }],
|
||||
11: [{ name: 'a', source }],
|
||||
});
|
||||
return {
|
||||
...made,
|
||||
deployer: new EventsWorkerDeployer(made.layers),
|
||||
script: scriptOf('a', source),
|
||||
app: nextApp(),
|
||||
};
|
||||
};
|
||||
|
||||
it('is left by a user-scope deploy with a ttl under a day', async () => {
|
||||
const { deployer, redis, script, app } = setup();
|
||||
await deployer.ensure(userScope(10, app), script);
|
||||
const ttl = await redis.ttl(`ev:deployed:${script}`);
|
||||
expect(ttl).toBeGreaterThan(0);
|
||||
expect(ttl).toBeLessThan(24 * 60 * 60);
|
||||
});
|
||||
|
||||
it('lets an eager deploy skip the upload another holder already made', async () => {
|
||||
const { deployer, created, script, app } = setup();
|
||||
expect(await deployer.deployEagerly(userScope(10, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(await deployer.deployEagerly(userScope(11, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(created).toEqual([script]);
|
||||
});
|
||||
|
||||
it('spends no throttle on a skipped eager deploy', async () => {
|
||||
const { deployer, redis, script, app } = setup();
|
||||
await deployer.deployEagerly(userScope(10, app), script);
|
||||
await deployer.deployEagerly(userScope(11, app), script);
|
||||
const hour = new Date().toISOString().slice(0, 13);
|
||||
expect(await redis.get(`ev:deploys:{${app}}:11:${hour}`)).toBeNull();
|
||||
});
|
||||
|
||||
it('never skips a deploy-on-miss', async () => {
|
||||
const { deployer, created, script, app } = setup();
|
||||
await deployer.deployEagerly(userScope(10, app), script);
|
||||
expect(await deployer.ensure(userScope(11, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(created).toEqual([script, script]);
|
||||
});
|
||||
|
||||
it('is not left by an app-scope deploy', async () => {
|
||||
const app = nextApp();
|
||||
const source = sourceN('a');
|
||||
const { layers, redis, created } = makeLayers({
|
||||
[app]: { ownerUserId: 4 },
|
||||
});
|
||||
withSets(layers, { 0: [{ name: 'a', source }] });
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
const script = scriptOf('a', source);
|
||||
await deployer.ensure(appHandlerScope(app), script);
|
||||
expect(created).toEqual([script]);
|
||||
expect(await redis.get(`ev:deployed:${script}`)).toBeNull();
|
||||
});
|
||||
|
||||
it('still deploys eagerly when the marker cannot be read', async () => {
|
||||
const { deployer, redis, created, script, app } = setup();
|
||||
await deployer.deployEagerly(userScope(10, app), script);
|
||||
const get = redis.get.bind(redis);
|
||||
redis.get = (async (key: string) => {
|
||||
if (key.startsWith('ev:deployed:')) throw new Error('down');
|
||||
return get(key);
|
||||
}) as never;
|
||||
expect(await deployer.deployEagerly(userScope(11, app), script)).toBe(
|
||||
'deployed',
|
||||
);
|
||||
expect(created).toEqual([script, script]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('scopeFor', () => {
|
||||
it('reads no holder as the app scope, and resolves a uuid to the user scope', async () => {
|
||||
const { layers } = makeLayers(holders(10));
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
expect(await deployer.scopeFor('app-u')).toEqual({
|
||||
appUid: 'app-u',
|
||||
holder: 0,
|
||||
});
|
||||
expect(await deployer.scopeFor('app-u', '')).toEqual({
|
||||
appUid: 'app-u',
|
||||
holder: 0,
|
||||
});
|
||||
expect(await deployer.scopeFor('app-u', 'uuid-10')).toEqual({
|
||||
appUid: 'app-u',
|
||||
holder: 10,
|
||||
});
|
||||
expect(await deployer.scopeFor('app-u', 'uuid-99')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('LocalEventsInvokeTransport in a user scope', () => {
|
||||
it('deploys the holder`s script on a miss and refuses an unknown holder', async () => {
|
||||
const source = sourceN('a');
|
||||
const { layers, created } = makeLayers(holders(10));
|
||||
withSets(layers, { 10: [{ name: 'a', source }] });
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
let resident = false;
|
||||
const transport = new LocalEventsInvokeTransport(
|
||||
{
|
||||
localworkerservice: {
|
||||
dispatchEventsWorker: async () => {
|
||||
if (!resident) return null;
|
||||
return new Response(null, {
|
||||
status: 200,
|
||||
headers: { 'x-puter-events-handled': '1' },
|
||||
});
|
||||
},
|
||||
},
|
||||
} as never,
|
||||
{
|
||||
ensure: async (
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
options: EventsDeployOptions,
|
||||
) => {
|
||||
const outcome = await deployer.ensure(
|
||||
scope,
|
||||
script,
|
||||
options,
|
||||
);
|
||||
resident = outcome === 'deployed';
|
||||
return outcome;
|
||||
},
|
||||
scopeFor: (appUid: string, holder?: string) =>
|
||||
deployer.scopeFor(appUid, holder),
|
||||
} as unknown as EventsWorkerDeployer,
|
||||
);
|
||||
const script = scriptOf('a', source);
|
||||
const call: EventsInvokeCall = {
|
||||
script,
|
||||
appUid: nextApp(),
|
||||
holder: 'uuid-10',
|
||||
key: 'k',
|
||||
body: '{}',
|
||||
timeoutMs: 5000,
|
||||
};
|
||||
|
||||
await expect(transport.send(call)).resolves.toMatchObject({
|
||||
status: 200,
|
||||
});
|
||||
expect(created).toEqual([script]);
|
||||
|
||||
resident = false;
|
||||
await expect(
|
||||
transport.send({ ...call, holder: 'uuid-99' }),
|
||||
).resolves.toMatchObject({ status: null, error: 'deploy: no-holder' });
|
||||
expect(created).toEqual([script]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('concurrent eager deploys of one script', () => {
|
||||
const setup = () => {
|
||||
const source = sourceN('a');
|
||||
const made = makeLayers(holders(10, 11));
|
||||
withSets(made.layers, {
|
||||
10: [{ name: 'a', source }],
|
||||
11: [{ name: 'a', source }],
|
||||
});
|
||||
return {
|
||||
...made,
|
||||
script: scriptOf('a', source),
|
||||
app: nextApp(),
|
||||
};
|
||||
};
|
||||
|
||||
it('upload once when two holders publish the same new code together', async () => {
|
||||
const { layers, created, script, app } = setup();
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
|
||||
const outcomes = await Promise.all([
|
||||
deployer.deployEagerly(userScope(10, app), script),
|
||||
deployer.deployEagerly(userScope(11, app), script),
|
||||
]);
|
||||
|
||||
expect(outcomes).toEqual(['deployed', 'deployed']);
|
||||
expect(created).toEqual([script]);
|
||||
});
|
||||
|
||||
it('skip the upload when the marker appears while they wait on the gate', async () => {
|
||||
const { layers, redis, created, script, app } = setup();
|
||||
// Another node's deploy of the script finishes before the gate opens.
|
||||
const gated = new EventsWorkerDeployer({
|
||||
...layers,
|
||||
services: {
|
||||
schedules: {
|
||||
gateDeploy: async (
|
||||
_script: string,
|
||||
_options: { capped: boolean },
|
||||
deploy: () => Promise<EventsDeployOutcome>,
|
||||
) => {
|
||||
await redis.set(`ev:deployed:${script}`, '1');
|
||||
return deploy();
|
||||
},
|
||||
},
|
||||
},
|
||||
} as unknown as Layers);
|
||||
|
||||
expect(
|
||||
await gated.ensure(userScope(10, app), script, { eager: true }),
|
||||
).toBe('deployed');
|
||||
expect(created).toEqual([]);
|
||||
// A miss still uploads, whatever the marker says.
|
||||
expect(await gated.ensure(userScope(11, app), script)).toBe('deployed');
|
||||
expect(created).toEqual([script]);
|
||||
});
|
||||
|
||||
it('make a miss that joins an eager deploy deploy after it', async () => {
|
||||
const { layers, redis, created, script, app } = setup();
|
||||
const deployer = new EventsWorkerDeployer(layers);
|
||||
await redis.set(`ev:deployed:${script}`, '1');
|
||||
|
||||
const eager = deployer.ensure(userScope(10, app), script, {
|
||||
eager: true,
|
||||
});
|
||||
const miss = deployer.ensure(userScope(11, app), script);
|
||||
|
||||
expect(await eager).toBe('deployed');
|
||||
expect(await miss).toBe('deployed');
|
||||
// The eager one skipped on the marker; the miss uploaded.
|
||||
expect(created).toEqual([script]);
|
||||
});
|
||||
});
|
||||
@@ -42,23 +42,33 @@ import {
|
||||
eventsInvokeKey,
|
||||
eventsWorkerScope,
|
||||
eventsWorkerScript,
|
||||
userHandlerScript,
|
||||
} from '../../services/events/workerRuntime.js';
|
||||
import {
|
||||
generateEventsWorkerSource,
|
||||
handlerSetHash,
|
||||
} from '../../services/events/workerSource.js';
|
||||
import {
|
||||
appHandlerScope,
|
||||
type HandlerScope,
|
||||
} from '../../stores/events/EventHandlerStore.js';
|
||||
import type { puterStores } from '../../stores/index.js';
|
||||
import type { IConfig, LayerInstances } from '../../types.js';
|
||||
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
|
||||
import {
|
||||
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
|
||||
EVENTS_WORKER_DEPLOYS_PER_HOUR,
|
||||
} from './limits.js';
|
||||
|
||||
/**
|
||||
* Deploys an app's events worker from its published handlers.
|
||||
*
|
||||
* Deployed the first time a delivery needs it and again on eviction — a publish
|
||||
* only writes rows. Scripts carry no worker token and no `subdomains` row: not
|
||||
* one of the owner's workers, not addressable from the internet, not listed or
|
||||
* deleted as if it were. A republished set leaves the old script behind rather
|
||||
* than deleting it; redeploying a still-live one is always safe.
|
||||
* only writes rows (a user-scope publish also deploys eagerly). An app's scope
|
||||
* is one script for the whole set; a user scope is one script per handler,
|
||||
* deployed as the holder. Scripts carry no worker token and no `subdomains`
|
||||
* row: not one of the owner's workers, not addressable from the internet, not
|
||||
* listed or deleted as if it were. A republished set leaves the old script
|
||||
* behind rather than deleting it; redeploying a still-live one is always safe.
|
||||
*/
|
||||
|
||||
/** Why a deploy could not happen, for the callers that answer differently. */
|
||||
@@ -74,6 +84,11 @@ export type EventsDeployOutcome =
|
||||
/** Whose call asked for the deploy: a schedule run's is held to a share. */
|
||||
export interface EventsDeployOptions {
|
||||
lane?: InvokeLane;
|
||||
/**
|
||||
* A user-scope publish's deploy, which skips the upload when the script's
|
||||
* deployed marker is already set. A deploy-on-miss is never eager.
|
||||
*/
|
||||
eager?: boolean;
|
||||
}
|
||||
|
||||
interface DeployLayers {
|
||||
@@ -86,14 +101,22 @@ interface DeployLayers {
|
||||
}
|
||||
|
||||
/**
|
||||
* `ev:deploys:{<appUid>}:<hour>` bucket TTL — comfortably past the hour it
|
||||
* counts.
|
||||
* `ev:deploys:{<appUid>}[:<holder>]:<hour>` bucket TTL — comfortably past the
|
||||
* hour it counts.
|
||||
*/
|
||||
const DEPLOY_THROTTLE_TTL_SECONDS = 2 * 60 * 60;
|
||||
|
||||
/** How long `ev:deployed:<script>` says a script was uploaded. Under a day. */
|
||||
const DEPLOYED_MARKER_TTL_SECONDS = 20 * 60 * 60;
|
||||
|
||||
const deployedMarkerKey = (script: string): string => `ev:deployed:${script}`;
|
||||
|
||||
const isUserScope = (scope: HandlerScope): boolean => scope.holder !== 0;
|
||||
|
||||
interface InFlightDeploy {
|
||||
/** A schedule run's, which the region's deploy slots may refuse. */
|
||||
capped: boolean;
|
||||
eager: boolean;
|
||||
promise: Promise<EventsDeployOutcome>;
|
||||
}
|
||||
|
||||
@@ -120,38 +143,76 @@ export class EventsWorkerDeployer {
|
||||
}
|
||||
|
||||
/**
|
||||
* Deploy `script`, which must be what the app's handlers currently hash to.
|
||||
* A request for anything else is `stale` — the set moved on, and the next
|
||||
* delivery resolves the new script rather than this one being revived.
|
||||
* Deploy `script`, which must be what the scope's handlers currently hash
|
||||
* to: the whole set for an app, one handler for a user scope. A request for
|
||||
* anything else is `stale` — the handlers moved on, and the next delivery
|
||||
* resolves the new script rather than this one being revived. Always
|
||||
* uploads; a deploy-on-miss means whatever said it was there is wrong.
|
||||
*/
|
||||
ensure(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
options: EventsDeployOptions = {},
|
||||
): Promise<EventsDeployOutcome> {
|
||||
const key = `${appUid}|${script}`;
|
||||
// A user scope's script is the same bytes whoever holds it, so
|
||||
// concurrent deploys of it from any holder or app are one.
|
||||
const key = isUserScope(scope)
|
||||
? `user|${script}`
|
||||
: `${scope.appUid}|${scope.holder}|${script}`;
|
||||
const capped = options.lane === 'schedule';
|
||||
const existing = this.#inFlight.get(key);
|
||||
if (!existing) return this.#start(key, appUid, script, options);
|
||||
if (!existing) return this.#start(key, scope, script, options);
|
||||
// An eager deploy may have skipped the upload, which a miss must not
|
||||
// take for done.
|
||||
if (existing.eager && !options.eager)
|
||||
return this.#afterEager(
|
||||
existing.promise,
|
||||
key,
|
||||
scope,
|
||||
script,
|
||||
options,
|
||||
);
|
||||
if (capped || !existing.capped) return existing.promise;
|
||||
return this.#afterCapped(
|
||||
existing.promise,
|
||||
key,
|
||||
appUid,
|
||||
script,
|
||||
options,
|
||||
);
|
||||
return this.#afterCapped(existing.promise, key, scope, script, options);
|
||||
}
|
||||
|
||||
/**
|
||||
* {@link ensure} for the deploy a user-scope publish starts, which skips the
|
||||
* upload when another holder's deploy of the same script left its marker.
|
||||
* Only this path reads the marker.
|
||||
*/
|
||||
async deployEagerly(
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
): Promise<EventsDeployOutcome> {
|
||||
if (await this.#markedDeployed(script)) return 'deployed';
|
||||
return this.ensure(scope, script, { eager: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* The scope a script request names: the app's own, or the user scope of
|
||||
* `holder` (a user uuid). Null for a holder that does not exist.
|
||||
*/
|
||||
async scopeFor(
|
||||
appUid: string,
|
||||
holder?: string | null,
|
||||
): Promise<HandlerScope | null> {
|
||||
if (!holder) return appHandlerScope(appUid);
|
||||
const user = await this.#layers.stores.user.getByUuid(holder);
|
||||
if (!user?.id) return null;
|
||||
return { appUid, holder: Number(user.id) };
|
||||
}
|
||||
|
||||
#start(
|
||||
key: string,
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
options: EventsDeployOptions,
|
||||
): Promise<EventsDeployOutcome> {
|
||||
const entry: InFlightDeploy = {
|
||||
capped: options.lane === 'schedule',
|
||||
promise: this.#gated(appUid, script, options).finally(() => {
|
||||
eager: options.eager === true,
|
||||
promise: this.#gated(scope, script, options).finally(() => {
|
||||
if (this.#inFlight.get(key) === entry)
|
||||
this.#inFlight.delete(key);
|
||||
}),
|
||||
@@ -160,6 +221,20 @@ export class EventsWorkerDeployer {
|
||||
return entry.promise;
|
||||
}
|
||||
|
||||
/** A miss that found an eager deploy in flight deploys after it. */
|
||||
async #afterEager(
|
||||
eager: Promise<EventsDeployOutcome>,
|
||||
key: string,
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
options: EventsDeployOptions,
|
||||
): Promise<EventsDeployOutcome> {
|
||||
await eager;
|
||||
const current = this.#inFlight.get(key);
|
||||
if (current && !current.eager) return current.promise;
|
||||
return this.#start(key, scope, script, options);
|
||||
}
|
||||
|
||||
/**
|
||||
* A delivery that joined a schedule run's deploy: `busy` means no deploy
|
||||
* started, and a delivery is never refused a slot, so it asks again.
|
||||
@@ -167,7 +242,7 @@ export class EventsWorkerDeployer {
|
||||
async #afterCapped(
|
||||
capped: Promise<EventsDeployOutcome>,
|
||||
key: string,
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
options: EventsDeployOptions,
|
||||
): Promise<EventsDeployOutcome> {
|
||||
@@ -175,7 +250,7 @@ export class EventsWorkerDeployer {
|
||||
if (outcome !== 'busy') return outcome;
|
||||
const current = this.#inFlight.get(key);
|
||||
if (current && !current.capped) return current.promise;
|
||||
return this.#start(key, appUid, script, options);
|
||||
return this.#start(key, scope, script, options);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -185,11 +260,12 @@ export class EventsWorkerDeployer {
|
||||
* may be refused with `busy`. A delivery's deploy is never refused.
|
||||
*/
|
||||
#gated(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
script: string,
|
||||
options: EventsDeployOptions,
|
||||
): Promise<EventsDeployOutcome> {
|
||||
const deploy = () => this.#deploy(appUid, script);
|
||||
const deploy = () =>
|
||||
this.#deploy(scope, script, options.eager === true);
|
||||
return (
|
||||
this.#layers.services.schedules?.gateDeploy?.(
|
||||
script,
|
||||
@@ -200,28 +276,55 @@ export class EventsWorkerDeployer {
|
||||
}
|
||||
|
||||
async #deploy(
|
||||
appUid: string,
|
||||
handlerScope: HandlerScope,
|
||||
script: string,
|
||||
eager: boolean,
|
||||
): Promise<EventsDeployOutcome> {
|
||||
const { appUid } = handlerScope;
|
||||
const { stores, drivers } = this.#layers;
|
||||
const scope = eventsWorkerScope(this.#layers.config);
|
||||
const backend = eventsWorkerScope(this.#layers.config);
|
||||
const userScope = isUserScope(handlerScope);
|
||||
|
||||
// Names and hashes are enough to tell a stale request, so the set's
|
||||
// sources are read and built only once the throttle has let it through.
|
||||
const set = await stores.eventHandler.setForApp(appUid);
|
||||
const set = await stores.eventHandler.setForApp(handlerScope);
|
||||
if (set.length === 0) return 'no-handlers';
|
||||
// Two apps that published byte-identical sets share one script, which
|
||||
// is sound: the source is the same, and everything per-app arrives with
|
||||
// the invocation. Whoever deploys first owns it.
|
||||
if (eventsWorkerScript(handlerSetHash(set), scope) !== script)
|
||||
// the invocation. Whoever deploys first owns it. A user scope's script
|
||||
// is one handler's, so it is shared the same way across holders.
|
||||
const wanted = userScope
|
||||
? set.find((entry) => userHandlerScript(entry, backend) === script)
|
||||
: undefined;
|
||||
if (
|
||||
userScope
|
||||
? !wanted
|
||||
: eventsWorkerScript(handlerSetHash(set), backend) !== script
|
||||
)
|
||||
return 'stale';
|
||||
|
||||
if (await this.#tooManyDeploysThisHour(appUid)) return 'throttled';
|
||||
// Asked again inside the gate, where another deploy of this script may
|
||||
// have just finished.
|
||||
if (eager && userScope && (await this.#markedDeployed(script)))
|
||||
return 'deployed';
|
||||
|
||||
const handlers = await stores.eventHandler.allForApp(appUid);
|
||||
if (await this.#tooManyDeploysThisHour(handlerScope))
|
||||
return 'throttled';
|
||||
|
||||
const all = await stores.eventHandler.allForApp(handlerScope);
|
||||
const handlers = wanted
|
||||
? all.filter(
|
||||
(handler) =>
|
||||
handler.name === wanted.name &&
|
||||
handler.sourceHash === wanted.sourceHash,
|
||||
)
|
||||
: all;
|
||||
const generated = generateEventsWorkerSource(handlers);
|
||||
// The set can move between the two reads.
|
||||
if (eventsWorkerScript(generated.setHash, scope) !== script)
|
||||
if (
|
||||
handlers.length === 0 ||
|
||||
eventsWorkerScript(generated.setHash, backend) !== script
|
||||
)
|
||||
return 'stale';
|
||||
if (generated.tooLarge) {
|
||||
console.error(
|
||||
@@ -235,7 +338,7 @@ export class EventsWorkerDeployer {
|
||||
generated.broken.join(', '),
|
||||
);
|
||||
|
||||
const owner = await this.#owner(appUid);
|
||||
const owner = await this.#owner(handlerScope);
|
||||
if (!owner) return 'no-owner';
|
||||
|
||||
const secret = this.#layers.config.events?.internalSecret;
|
||||
@@ -264,6 +367,7 @@ export class EventsWorkerDeployer {
|
||||
runtime: 'events',
|
||||
source: generated.source,
|
||||
omitOwnerToken: true,
|
||||
cpuMs: this.#layers.config.events?.workerCpuMs,
|
||||
// What the script compares an invocation against.
|
||||
// Derived from the script name, so a redeploy of
|
||||
// the same set binds the same key.
|
||||
@@ -294,34 +398,74 @@ export class EventsWorkerDeployer {
|
||||
);
|
||||
return 'failed';
|
||||
}
|
||||
if (userScope) await this.#markDeployed(script);
|
||||
return 'deployed';
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this app has already deployed `EVENTS_WORKER_DEPLOYS_PER_HOUR`
|
||||
* times this hour. A flapping set (publish, remove, republish) provisions
|
||||
* upstream on every miss, and this is the ceiling on that loop.
|
||||
*/
|
||||
async #tooManyDeploysThisHour(appUid: string): Promise<boolean> {
|
||||
const redis = this.#layers.clients.redis;
|
||||
const hour = new Date().toISOString().slice(0, 13);
|
||||
const key = `ev:deploys:{${appUid}}:${hour}`;
|
||||
const count = await redis.incr(key);
|
||||
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
|
||||
return count > EVENTS_WORKER_DEPLOYS_PER_HOUR;
|
||||
async #markedDeployed(script: string): Promise<boolean> {
|
||||
try {
|
||||
return !!(await this.#layers.clients.redis.get(
|
||||
deployedMarkerKey(script),
|
||||
));
|
||||
} catch (err) {
|
||||
console.warn('[events] deployed marker unreadable', err);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** The app owner, as an actor the deploy can run as. */
|
||||
async #markDeployed(script: string): Promise<void> {
|
||||
try {
|
||||
await this.#layers.clients.redis.set(
|
||||
deployedMarkerKey(script),
|
||||
'1',
|
||||
'EX',
|
||||
DEPLOYED_MARKER_TTL_SECONDS,
|
||||
);
|
||||
} catch (err) {
|
||||
console.warn('[events] could not mark a script deployed', err);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this scope has already deployed its hourly cap this hour. A
|
||||
* flapping set (publish, remove, republish) provisions upstream on every
|
||||
* miss, and this is the ceiling on that loop. An app's counter is per app;
|
||||
* a user scope has its own, so one user cannot spend another's.
|
||||
*/
|
||||
async #tooManyDeploysThisHour(scope: HandlerScope): Promise<boolean> {
|
||||
const redis = this.#layers.clients.redis;
|
||||
const hour = new Date().toISOString().slice(0, 13);
|
||||
const userScope = isUserScope(scope);
|
||||
const key = userScope
|
||||
? `ev:deploys:{${scope.appUid}}:${scope.holder}:${hour}`
|
||||
: `ev:deploys:{${scope.appUid}}:${hour}`;
|
||||
const count = await redis.incr(key);
|
||||
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
|
||||
return (
|
||||
count >
|
||||
(userScope
|
||||
? EVENTS_USER_WORKER_DEPLOYS_PER_HOUR
|
||||
: EVENTS_WORKER_DEPLOYS_PER_HOUR)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The user a scope's deploy runs as, as an actor: the app's owner, or the
|
||||
* holder of a user scope.
|
||||
*/
|
||||
async #owner(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
): Promise<
|
||||
| (Actor & { user: { id: number; uuid: string; username: string } })
|
||||
| null
|
||||
> {
|
||||
const { stores } = this.#layers;
|
||||
const app = await stores.app.getByUid(appUid);
|
||||
const ownerUserId = Number(app?.owner_user_id);
|
||||
if (!app || !Number.isFinite(ownerUserId)) return null;
|
||||
let ownerUserId = scope.holder;
|
||||
if (!isUserScope(scope)) {
|
||||
const app = await stores.app.getByUid(scope.appUid);
|
||||
ownerUserId = Number(app?.owner_user_id);
|
||||
if (!app || !Number.isFinite(ownerUserId)) return null;
|
||||
}
|
||||
|
||||
const user = await stores.user.getById(ownerUserId);
|
||||
if (!user?.id || !user.uuid || !user.username || user.suspended)
|
||||
@@ -402,11 +546,14 @@ export class LocalEventsInvokeTransport {
|
||||
const resident = await this.#dispatch(request(), call.timeoutMs);
|
||||
if (resident !== null) return resident;
|
||||
|
||||
const outcome = await this.#deployer.ensure(
|
||||
const scope = await this.#deployer.scopeFor(
|
||||
call.appUid,
|
||||
call.script,
|
||||
{ lane: call.lane },
|
||||
call.holder,
|
||||
);
|
||||
if (!scope) return { status: null, error: 'deploy: no-holder' };
|
||||
const outcome = await this.#deployer.ensure(scope, call.script, {
|
||||
lane: call.lane,
|
||||
});
|
||||
if (outcome !== 'deployed')
|
||||
return {
|
||||
status: null,
|
||||
|
||||
@@ -244,7 +244,10 @@ describe('OIDCController GET /auth/oidc/providers', () => {
|
||||
await callRoute('get', '/auth/oidc/providers', makeReq({}), res);
|
||||
// We configured a single `custom` provider; the live service
|
||||
// walks the config and returns its id.
|
||||
expect(captured.body).toEqual({ providers: ['custom'] });
|
||||
expect(captured.body).toEqual({
|
||||
providers: ['custom'],
|
||||
signupOidcOnly: false,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2386,46 +2389,4 @@ describe('OIDC signup bonus codes', () => {
|
||||
expect(captured.redirectUrl).toContain('message=signup_blocked');
|
||||
expect(captured.redirectUrl).toContain('bonusCode=startup3mabcd1234');
|
||||
});
|
||||
|
||||
it('carries shaped utm params in the signed state', async () => {
|
||||
const decoded = await startState({
|
||||
flow: 'signup',
|
||||
utm_source: 'ads',
|
||||
utm_campaign: 'launch',
|
||||
utm_id: 'not-tracked',
|
||||
utm_medium: '',
|
||||
});
|
||||
expect(decoded.utm).toEqual({ source: 'ads', campaign: 'launch' });
|
||||
});
|
||||
|
||||
it('leaves utm out of the state when none were sent', async () => {
|
||||
const decoded = await startState({ flow: 'signup' });
|
||||
expect(decoded).not.toHaveProperty('utm');
|
||||
});
|
||||
|
||||
it('announces the state utm on the new account, re-shaped', async () => {
|
||||
const email = `utm-${rand()}@test.local`;
|
||||
stubIdp(`sub-${rand()}`, email);
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
utm: { source: 'ads', medium: 7, extra: 'x' },
|
||||
});
|
||||
const { res, captured } = makeRes();
|
||||
await callRoute(
|
||||
'get',
|
||||
'/auth/oidc/callback/signup',
|
||||
makeReq({ query: { code: 'c', state } }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
const deadline = Date.now() + 2000;
|
||||
let success: Record<string, unknown> | undefined;
|
||||
while (!success && Date.now() < deadline) {
|
||||
success = heardSignupSuccess.find((e) => e.email === email);
|
||||
if (!success) await new Promise((r) => setTimeout(r, 10));
|
||||
}
|
||||
expect(success?.utm).toEqual({ source: 'ads' });
|
||||
});
|
||||
});
|
||||
@@ -33,11 +33,6 @@ import {
|
||||
import type { UserRow } from '../../stores/user/UserStore.js';
|
||||
import { sessionCookieFlags } from '../../util/cookieFlags.js';
|
||||
import { normalizeBonusCode } from '../../util/signupBonus.js';
|
||||
import {
|
||||
normalizeSignupUtm,
|
||||
readSignupUtm,
|
||||
type SignupUtm,
|
||||
} from '../../util/signupUtm.js';
|
||||
import { parseMaskedSharePath } from '../../services/fs/sharePathMask.js';
|
||||
import {
|
||||
SHARE_DEEP_LINK_ITEMS_LIMIT,
|
||||
@@ -93,7 +88,6 @@ function resolutionErrorCode(code: string | undefined): string {
|
||||
interface OIDCSignupOptions {
|
||||
referrer?: string | null;
|
||||
bonusCode?: string | null;
|
||||
utm?: SignupUtm | null;
|
||||
}
|
||||
|
||||
function signupOptionsFromState(
|
||||
@@ -104,7 +98,6 @@ function signupOptionsFromState(
|
||||
// Normalized when the state was signed; re-checked since this is the
|
||||
// value signup acts on.
|
||||
bonusCode: normalizeBonusCode(stateDecoded.bonus_code),
|
||||
utm: normalizeSignupUtm(stateDecoded.utm),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -403,9 +396,12 @@ export class OIDCController extends PuterController {
|
||||
},
|
||||
},
|
||||
async (_req: Request, res: Response) => {
|
||||
const providers =
|
||||
await this.services.oidc.getEnabledProviderIds();
|
||||
res.json({ providers });
|
||||
const [providers, policy] = await Promise.all([
|
||||
this.services.oidc.getEnabledProviderIds(),
|
||||
this.services.signup.signupPolicy(),
|
||||
]);
|
||||
// The signup window drops its email form when this is set.
|
||||
res.json({ providers, signupOidcOnly: policy.oidcOnly });
|
||||
},
|
||||
);
|
||||
|
||||
@@ -502,10 +498,8 @@ export class OIDCController extends PuterController {
|
||||
? req.query.bonusCode[0]
|
||||
: req.query.bonusCode;
|
||||
const bonusCode = normalizeBonusCode(rawBonusCode);
|
||||
const utm = readSignupUtm(req.query);
|
||||
if (flow === 'login' || flow === 'signup') {
|
||||
if (bonusCode) statePayload.bonus_code = bonusCode;
|
||||
if (utm) statePayload.utm = utm;
|
||||
if (bonusCode && (flow === 'login' || flow === 'signup')) {
|
||||
statePayload.bonus_code = bonusCode;
|
||||
}
|
||||
if (embeddedInPopup && msgId) {
|
||||
statePayload.embedded_in_popup = true;
|
||||
@@ -896,7 +890,7 @@ if (window.opener) {
|
||||
provider,
|
||||
userinfo as { sub: string; email?: string },
|
||||
signup.referrer ?? null,
|
||||
{ bonusCode: signup.bonusCode ?? null, utm: signup.utm ?? null },
|
||||
{ bonusCode: signup.bonusCode ?? null },
|
||||
);
|
||||
// A concurrent callback (a second tab, a provider retry) created the
|
||||
// account between step 2 and the insert. Nothing went wrong for the
|
||||
|
||||
@@ -19,8 +19,9 @@
|
||||
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import path from 'node:path';
|
||||
import type { Request, Response } from 'express';
|
||||
import express, { type Request, type Response } from 'express';
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import type { RouteDescriptor } from '../../core/http/types.js';
|
||||
@@ -111,6 +112,7 @@ const callGetHandler = async (
|
||||
let captured: { filename?: string; root?: string } = {};
|
||||
const req = {} as Request;
|
||||
const res = {
|
||||
setHeader: vi.fn(),
|
||||
sendFile: vi.fn((filename: string, opts: { root: string }) => {
|
||||
captured = { filename, root: opts.root };
|
||||
}),
|
||||
@@ -238,6 +240,48 @@ describe('StaticAssetsController puterjs_root', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── push-sw.js caching ──────────────────────────────────────────────
|
||||
|
||||
/** Serves the route's handler over real HTTP and returns the response. */
|
||||
const fetchThroughExpress = async (router: PuterRouter, routePath: string) => {
|
||||
const route = findRoute(router, 'get', routePath);
|
||||
if (!route) throw new Error(`No GET ${routePath} registered`);
|
||||
const app = express();
|
||||
app.get(routePath, (req, res, next) => route.handler(req, res, next));
|
||||
const listener = app.listen(0);
|
||||
await new Promise((resolve) => listener.once('listening', resolve));
|
||||
try {
|
||||
const { port } = listener.address() as AddressInfo;
|
||||
return await fetch(`http://127.0.0.1:${port}${routePath}`);
|
||||
} finally {
|
||||
listener.close();
|
||||
}
|
||||
};
|
||||
|
||||
describe('StaticAssetsController push-sw.js', () => {
|
||||
it.each([
|
||||
['client_libs_root', 'puter.js/push-sw.js'],
|
||||
['puterjs_root', 'push-sw.js'],
|
||||
] as const)(
|
||||
'lets the CDN keep it for five minutes from %s',
|
||||
async (configKey, file) => {
|
||||
const root = makeTempDir();
|
||||
mkdirSync(path.dirname(path.join(root, file)), { recursive: true });
|
||||
writeFileSync(path.join(root, file), '/* worker */');
|
||||
const router = buildController({ [configKey]: root });
|
||||
|
||||
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
|
||||
const response = await fetchThroughExpress(router, route);
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.headers.get('cache-control')).toBe(
|
||||
'public, max-age=0, s-maxage=300',
|
||||
);
|
||||
expect(await response.text()).toBe('/* worker */');
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
// ── gui_assets_root ─────────────────────────────────────────────────
|
||||
|
||||
describe('StaticAssetsController gui_assets_root', () => {
|
||||
|
||||
@@ -23,6 +23,9 @@ import path from 'node:path';
|
||||
import { PuterController } from '../types.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter';
|
||||
|
||||
// Sites import push-sw.js by URL, so the CDN refetches it within five minutes.
|
||||
const PUSH_WORKER_CACHE_CONTROL = 'public, max-age=0, s-maxage=300';
|
||||
|
||||
/**
|
||||
* Static asset routes.
|
||||
*
|
||||
@@ -58,6 +61,7 @@ export class StaticAssetsController extends PuterController {
|
||||
// `/v2/push-sw.js` is the path older docs gave.
|
||||
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
|
||||
router.get(route, { subdomain: 'js' }, (_req, res) => {
|
||||
res.setHeader('Cache-Control', PUSH_WORKER_CACHE_CONTROL);
|
||||
res.sendFile('puter.js/push-sw.js', { root });
|
||||
});
|
||||
}
|
||||
@@ -108,6 +112,10 @@ export class StaticAssetsController extends PuterController {
|
||||
});
|
||||
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
|
||||
router.get(route, { subdomain: 'js' }, (_req, res) => {
|
||||
res.setHeader(
|
||||
'Cache-Control',
|
||||
PUSH_WORKER_CACHE_CONTROL,
|
||||
);
|
||||
res.sendFile('push-sw.js', { root: puterjsRoot });
|
||||
});
|
||||
}
|
||||
|
||||
@@ -69,6 +69,8 @@ export interface ScheduleView {
|
||||
appUid: string;
|
||||
/** `null` once ended: the handler binding goes with the schedule. */
|
||||
handlerName: string | null;
|
||||
/** Set only when the handler lives in the holder's own scope. */
|
||||
handlerScope?: 'user';
|
||||
contextKeys: string[] | null;
|
||||
contextHash: string | null;
|
||||
/** Interval of an `every` schedule. */
|
||||
@@ -168,8 +170,16 @@ export interface ScheduleServiceContract {
|
||||
request: ScheduleListRequest,
|
||||
): Promise<PageResult<ScheduleView>>;
|
||||
get(actor: Actor, scheduleId: unknown): Promise<ScheduleView>;
|
||||
update(
|
||||
actor: Actor,
|
||||
scheduleId: unknown,
|
||||
changes: unknown,
|
||||
): Promise<ScheduleView>;
|
||||
remove(actor: Actor, scheduleId: unknown): Promise<void>;
|
||||
receivePeerChanges(body: unknown): Promise<{ applied: number }>;
|
||||
/** `results`: each change's index outcome, in order (`null` if dropped). */
|
||||
receivePeerChanges(
|
||||
body: unknown,
|
||||
): Promise<{ applied: number; results?: Array<number | null> }>;
|
||||
/** Checks a caller supplying `sendAt`; `events_disabled` when off. */
|
||||
prepareSendAt(actor: Actor, sendAt: unknown): Promise<DeferredSendPlan>;
|
||||
createDeferredAppMessage(
|
||||
|
||||
@@ -463,10 +463,26 @@ describe('ImageGenerationDriver.generate provider routing', () => {
|
||||
driver.generate({ prompt: 'a landscape' }),
|
||||
);
|
||||
expect(openaiImagesGenerateMock).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ model: 'gpt-image-2' }),
|
||||
expect.objectContaining({ model: 'gpt-image-2.6-flare' }),
|
||||
);
|
||||
});
|
||||
|
||||
it.each(['gpt-image-2.6-flare', 'gpt-image-2.6-sunburst'])(
|
||||
'routes %s and its OpenAI alias', async (model) => {
|
||||
for (const id of [model, `openai/${model}`]) {
|
||||
openaiImagesGenerateMock.mockResolvedValueOnce({
|
||||
data: [{ url: 'https://oai/image.png' }],
|
||||
});
|
||||
await withDriverName('ai-image', () => driver.generate({
|
||||
model: id, prompt: 'a landscape',
|
||||
}));
|
||||
expect(openaiImagesGenerateMock).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ model }),
|
||||
);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('rejects an excluded model regardless of caller casing', async () => {
|
||||
await expect(withActor(() =>
|
||||
driver.generate({
|
||||
|
||||
@@ -150,9 +150,9 @@ describe('OpenAiImageProvider construction', () => {
|
||||
// ── Model catalog ───────────────────────────────────────────────────
|
||||
|
||||
describe('OpenAiImageProvider model catalog', () => {
|
||||
it('returns gpt-image-2 as the default', () => {
|
||||
it('returns gpt-image-2.6-flare as the default', () => {
|
||||
const provider = makeProvider();
|
||||
expect(provider.getDefaultModel()).toBe('gpt-image-2');
|
||||
expect(provider.getDefaultModel()).toBe('gpt-image-2.6-flare');
|
||||
});
|
||||
|
||||
it('drops the deprecated gpt-image-1 family and chatgpt-image-latest', () => {
|
||||
@@ -235,6 +235,56 @@ describe('OpenAiImageProvider.generate argument validation', () => {
|
||||
// ── Credit gate ─────────────────────────────────────────────────────
|
||||
|
||||
describe('OpenAiImageProvider.generate credit gate', () => {
|
||||
it('waits for the real usage write before returning an image', async () => {
|
||||
const actor = makeActor({
|
||||
user: {
|
||||
id: 62,
|
||||
uuid: `image-write-${Date.now()}`,
|
||||
username: 'metered',
|
||||
},
|
||||
});
|
||||
const redis = server.clients.redis as unknown as {
|
||||
meterIncr: (...args: string[]) => Promise<unknown>;
|
||||
};
|
||||
const realIncr = redis.meterIncr.bind(redis);
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
let blocked = false;
|
||||
vi.spyOn(redis, 'meterIncr').mockImplementation(async (...args) => {
|
||||
if (!blocked && args[5].includes(`:actor:${actor.user!.uuid}:`)) {
|
||||
blocked = true;
|
||||
await held;
|
||||
}
|
||||
return realIncr(...args);
|
||||
});
|
||||
generateMock.mockResolvedValueOnce({
|
||||
data: [{ url: 'https://image.example/result.png' }],
|
||||
});
|
||||
const provider = makeProvider();
|
||||
let finished = false;
|
||||
const generating = withTestActor(
|
||||
() =>
|
||||
provider.generate({
|
||||
model: 'gpt-image-2',
|
||||
prompt: 'an image',
|
||||
ratio: { w: 1024, h: 1024 },
|
||||
}),
|
||||
actor,
|
||||
).then((result) => {
|
||||
finished = true;
|
||||
return result;
|
||||
});
|
||||
try {
|
||||
await vi.waitFor(() => expect(blocked).toBe(true));
|
||||
expect(finished).toBe(false);
|
||||
} finally {
|
||||
release();
|
||||
await generating;
|
||||
}
|
||||
});
|
||||
|
||||
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
@@ -857,3 +907,64 @@ it('keeps a landscape aspect hint landscape at gpt-image-2 sizes', async () => {
|
||||
expect(w).toBeGreaterThan(h);
|
||||
expect(Math.abs(w / h - 1.7)).toBeLessThan(0.1);
|
||||
});
|
||||
|
||||
|
||||
describe('OpenAiImageProvider GPT Image 2.6', () => {
|
||||
it.each(['gpt-image-2.6-flare', 'gpt-image-2.6-sunburst'])(
|
||||
'%s generates and edits with token-based billing', async (model) => {
|
||||
const provider = makeProvider();
|
||||
const response = {
|
||||
data: [{ b64_json: 'AAAA' }],
|
||||
usage: {
|
||||
input_tokens: 100,
|
||||
output_tokens: 800,
|
||||
input_tokens_details: { text_tokens: 100, image_tokens: 0 },
|
||||
},
|
||||
};
|
||||
generateMock.mockResolvedValue(response);
|
||||
editMock.mockResolvedValue(response);
|
||||
for (const input_image of [undefined, 'data:image/png;base64,AAAA']) {
|
||||
const result = await withTestActor(() => provider.generate({
|
||||
model, prompt: 'hi', quality: 'xhigh',
|
||||
ratio: { w: 1024, h: 1024 }, input_image,
|
||||
}));
|
||||
expect(result).toBe('data:image/png;base64,AAAA');
|
||||
}
|
||||
for (const mock of [generateMock, editMock]) {
|
||||
expect(mock.mock.calls[0]![0]).toMatchObject({
|
||||
model, quality: 'xhigh', size: '1024x1024',
|
||||
});
|
||||
}
|
||||
const [, entries] = batchIncrementUsagesSpy.mock.calls[0]!;
|
||||
expect(entries).toEqual(expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
usageType: `openai:${model}:xhigh:1024x1024:output`,
|
||||
costOverride: 2_400_000,
|
||||
}),
|
||||
]));
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
['low', 1024, 1024, 823],
|
||||
['medium', 1024, 1024, 1449],
|
||||
['high', 1024, 1024, 1756],
|
||||
['xhigh', 1024, 1024, 3122],
|
||||
['max', 1024, 1024, 7024],
|
||||
['auto', 1024, 1024, 1449],
|
||||
['low', 2048, 1088, 571],
|
||||
])('estimates %s at %sx%s from the 2.6 calculator', async (quality, w, h, tokens) => {
|
||||
generateMock.mockResolvedValue({ data: [{ b64_json: 'AAAA' }] });
|
||||
await withTestActor(() => makeProvider().generate({
|
||||
model: 'gpt-image-2.6-flare', prompt: 'hi', quality: String(quality),
|
||||
ratio: { w: Number(w), h: Number(h) },
|
||||
}));
|
||||
const [, entries] = batchIncrementUsagesSpy.mock.calls[0]!;
|
||||
const output = (entries as Array<{ usageType: string; costOverride: number }>)
|
||||
.find((entry) => entry.usageType.endsWith(':output'))!;
|
||||
expect(output.usageType).toBe(`openai:gpt-image-2.6-flare:${quality}:${w}x${h}:output`);
|
||||
// The metering conversion rounds fractional microcents upward.
|
||||
expect(output.costOverride).toBeGreaterThanOrEqual(Number(tokens) * 3000);
|
||||
expect(output.costOverride).toBeLessThanOrEqual(Number(tokens) * 3000 + 1);
|
||||
});
|
||||
});
|
||||
@@ -56,8 +56,8 @@ interface OpenAIImageUsage {
|
||||
|
||||
/**
|
||||
* OpenAI image generation provider for v2. Supports the GPT Image models
|
||||
* (gpt-image-2, -2.5-sunburst, -2.5-flare), including image-to-image editing
|
||||
* via `input_images` (the `images.edit` endpoint).
|
||||
* (gpt-image-2 and the 2.5/2.6 Sunburst and Flare models), including
|
||||
* image-to-image editing via `input_images` (the `images.edit` endpoint).
|
||||
*/
|
||||
export class OpenAiImageProvider implements IImageProvider {
|
||||
#meteringService: AiMeteringService;
|
||||
@@ -109,7 +109,7 @@ export class OpenAiImageProvider implements IImageProvider {
|
||||
}
|
||||
|
||||
getDefaultModel(): string {
|
||||
return 'gpt-image-2';
|
||||
return 'gpt-image-2.6-flare';
|
||||
}
|
||||
|
||||
async generate({
|
||||
@@ -293,7 +293,10 @@ export class OpenAiImageProvider implements IImageProvider {
|
||||
});
|
||||
}
|
||||
if (usageEntries.length) {
|
||||
this.#meteringService.batchIncrementUsages(actor, usageEntries);
|
||||
await this.#meteringService.batchIncrementUsages(
|
||||
actor,
|
||||
usageEntries,
|
||||
);
|
||||
}
|
||||
|
||||
const url =
|
||||
@@ -601,17 +604,36 @@ export class OpenAiImageProvider implements IImageProvider {
|
||||
height: number,
|
||||
quality?: string,
|
||||
): number {
|
||||
const factors = model.startsWith('gpt-image-2.5')
|
||||
? OpenAiImageProvider.#GPT_IMAGE_2_5_LATENT_FACTORS
|
||||
: OpenAiImageProvider.#GPT_IMAGE_2_LATENT_FACTORS;
|
||||
const isGptImage26 = model.startsWith('gpt-image-2.6');
|
||||
const factors =
|
||||
isGptImage26 || model.startsWith('gpt-image-2.5')
|
||||
? OpenAiImageProvider.#GPT_IMAGE_2_5_LATENT_FACTORS
|
||||
: OpenAiImageProvider.#GPT_IMAGE_2_LATENT_FACTORS;
|
||||
const factor = factors[quality ?? ''] ?? factors.medium;
|
||||
const longEdge = Math.max(width, height);
|
||||
const shortEdge = Math.min(width, height);
|
||||
const shortLatent = Math.round((factor * shortEdge) / longEdge);
|
||||
const shortGrid = (factor * shortEdge) / longEdge;
|
||||
const floorGrid = Math.floor(shortGrid);
|
||||
const shortLatent =
|
||||
isGptImage26 && shortGrid - floorGrid === 0.5
|
||||
? floorGrid + (floorGrid % 2)
|
||||
: Math.round(shortGrid);
|
||||
const latentW = width >= height ? factor : shortLatent;
|
||||
const latentH = width >= height ? shortLatent : factor;
|
||||
const baseArea = latentW * latentH;
|
||||
return Math.ceil((baseArea * (2_000_000 + width * height)) / 4_000_000);
|
||||
const tokens = Math.ceil(
|
||||
(baseArea * (2_000_000 + width * height)) / 4_000_000,
|
||||
);
|
||||
// GPT Image 2.6 bills more tokens at low and medium quality.
|
||||
const estimatedQuality =
|
||||
quality && quality in factors ? quality : 'medium';
|
||||
const multiplier =
|
||||
estimatedQuality === 'low'
|
||||
? 4.2
|
||||
: estimatedQuality === 'medium'
|
||||
? 3.3
|
||||
: 1;
|
||||
return isGptImage26 ? Math.round(tokens * multiplier) : tokens;
|
||||
}
|
||||
|
||||
#estimateOutputCostFromTokens(
|
||||
|
||||
@@ -20,6 +20,46 @@
|
||||
import type { IImageModel } from '../../types.js';
|
||||
|
||||
export const OPEN_AI_IMAGE_GENERATION_MODELS: IImageModel[] = [
|
||||
{
|
||||
puterId: 'openai:openai/gpt-image-2.6-sunburst',
|
||||
id: 'gpt-image-2.6-sunburst',
|
||||
aliases: ['openai/gpt-image-2.6-sunburst'],
|
||||
name: 'GPT Image 2.6 Sunburst',
|
||||
version: '2.6',
|
||||
costs_currency: 'usd-cents',
|
||||
index_cost_key: 'low:1024x1024',
|
||||
costs: {
|
||||
// Text tokens (per 1M tokens)
|
||||
text_input: 500, // $5.00
|
||||
text_cached_input: 125, // $1.25
|
||||
// Image tokens (per 1M tokens)
|
||||
image_input: 800, // $8.00
|
||||
image_cached_input: 200, // $2.00
|
||||
image_output: 3000, // $30.00
|
||||
'low:1024x1024': 2.469,
|
||||
},
|
||||
allowedQualityLevels: ['low', 'medium', 'high', 'xhigh', 'max', 'auto'],
|
||||
},
|
||||
{
|
||||
puterId: 'openai:openai/gpt-image-2.6-flare',
|
||||
id: 'gpt-image-2.6-flare',
|
||||
aliases: ['openai/gpt-image-2.6-flare'],
|
||||
name: 'GPT Image 2.6 Flare',
|
||||
version: '2.6',
|
||||
costs_currency: 'usd-cents',
|
||||
index_cost_key: 'low:1024x1024',
|
||||
costs: {
|
||||
// Text tokens (per 1M tokens)
|
||||
text_input: 500, // $5.00
|
||||
text_cached_input: 125, // $1.25
|
||||
// Image tokens (per 1M tokens)
|
||||
image_input: 800, // $8.00
|
||||
image_cached_input: 200, // $2.00
|
||||
image_output: 3000, // $30.00
|
||||
'low:1024x1024': 2.469,
|
||||
},
|
||||
allowedQualityLevels: ['low', 'medium', 'high', 'xhigh', 'max', 'auto'],
|
||||
},
|
||||
{
|
||||
puterId: 'openai:openai/gpt-image-2.5-sunburst',
|
||||
id: 'gpt-image-2.5-sunburst',
|
||||
|
||||
@@ -31,7 +31,7 @@ import {
|
||||
normalizeRawBase64ImageString,
|
||||
validateIconDataUrl,
|
||||
} from '../../util/appIcon.js';
|
||||
import { toAppReadView } from '../../util/appView.js';
|
||||
import { isOriginBootstrapApp, toAppReadView } from '../../util/appView.js';
|
||||
import { awaitLater } from '../../util/concurrency.js';
|
||||
import { isUniqueViolation } from '../../util/dbError.js';
|
||||
import {
|
||||
@@ -1088,22 +1088,6 @@ export class AppDriver extends PuterDriver {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Origin-bootstrap detection: rows auto-created when an unknown origin
|
||||
* first needed an app row (no human-supplied metadata). Marker is `name ===
|
||||
* uid && title === uid` and a description starting with "App created from
|
||||
* origin ". Only these rows are eligible for same-owner merging — refusing
|
||||
* to merge arbitrary same-owner apps prevents accidental data loss.
|
||||
*/
|
||||
#isOriginBootstrapApp(app) {
|
||||
if (!app || typeof app !== 'object') return false;
|
||||
if (typeof app.uid !== 'string' || !app.uid) return false;
|
||||
if (app.name !== app.uid) return false;
|
||||
if (app.title !== app.uid) return false;
|
||||
if (typeof app.description !== 'string') return false;
|
||||
return app.description.startsWith('App created from origin ');
|
||||
}
|
||||
|
||||
// -- Canonical-uid alias kvstore pair -----------------------------
|
||||
//
|
||||
// After a merge, the source app uid → canonical uid mapping is
|
||||
@@ -1266,7 +1250,7 @@ export class AppDriver extends PuterDriver {
|
||||
if (
|
||||
Number.isInteger(conflictOwnerUserId) &&
|
||||
conflictOwnerUserId === user.id &&
|
||||
!this.#isOriginBootstrapApp(appToJoin)
|
||||
!isOriginBootstrapApp(appToJoin)
|
||||
) {
|
||||
// Prevent merging arbitrary same-owner apps; only allow the
|
||||
// auto-created origin bootstrap row to be absorbed.
|
||||
|
||||
@@ -188,7 +188,11 @@ describe('app.message with live delivery over the socket wire', () => {
|
||||
captureBus(state.server, 'outer.gui.notif.message', async () => {
|
||||
created = (await withActor(asApp(holder, app), () =>
|
||||
state.driver.create(
|
||||
message({ text: 'Body', channel: 'replies' }),
|
||||
message({
|
||||
text: 'Body',
|
||||
priority: 'high',
|
||||
channel: 'replies',
|
||||
}),
|
||||
),
|
||||
)) as Record<string, unknown>;
|
||||
}).then((seen) => (desktop = seen)),
|
||||
@@ -200,7 +204,7 @@ describe('app.message with live delivery over the socket wire', () => {
|
||||
title: 'Hello',
|
||||
text: 'Body',
|
||||
channel: 'replies',
|
||||
priority: 'normal',
|
||||
priority: 'high',
|
||||
ttl: 3600,
|
||||
app: { uid: app.uid, name: 'App message test' },
|
||||
});
|
||||
@@ -212,6 +216,7 @@ describe('app.message with live delivery over the socket wire', () => {
|
||||
app_uid: app.uid,
|
||||
audience: 'app-user',
|
||||
type: 'app.message',
|
||||
value: expect.objectContaining({ priority: 'high' }),
|
||||
});
|
||||
// Not silent, but sent for the app alone: the desktop wire stays quiet.
|
||||
expect(written).toEqual([
|
||||
@@ -220,7 +225,7 @@ describe('app.message with live delivery over the socket wire', () => {
|
||||
uid,
|
||||
type: 'app.message',
|
||||
appUid: app.uid,
|
||||
}),
|
||||
value: expect.objectContaining({ priority: 'high' }), }),
|
||||
]);
|
||||
expect(desktop).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -207,6 +207,7 @@ describe('app.message to the other party of a share', () => {
|
||||
const written = await captureBus('notif.written', async () => {
|
||||
sent = await send(inApp(owner, app), {
|
||||
title: 'New message',
|
||||
priority: 'high',
|
||||
to: { handle },
|
||||
});
|
||||
});
|
||||
@@ -222,7 +223,7 @@ describe('app.message to the other party of a share', () => {
|
||||
expect(row!.value).toEqual({
|
||||
type: 'app.message',
|
||||
title: 'New message',
|
||||
priority: 'normal',
|
||||
priority: 'high',
|
||||
ttl: 3600,
|
||||
app: { uid: app.uid, name: 'Handle App' },
|
||||
});
|
||||
|
||||
@@ -165,6 +165,7 @@ const fakeSchedules = (plan: Partial<DeferredSendPlan>) =>
|
||||
createFor: vi.fn(),
|
||||
list: vi.fn(),
|
||||
get: vi.fn(),
|
||||
update: vi.fn(),
|
||||
remove: vi.fn(),
|
||||
receivePeerChanges: vi.fn(),
|
||||
prepareSendAt: vi.fn(async (): Promise<DeferredSendPlan> => ({
|
||||
@@ -270,7 +271,9 @@ describe('notify({ sendAt }) with services.schedules', () => {
|
||||
const schedules = fakeSchedules({ sendAt: at });
|
||||
setSchedules(schedules);
|
||||
|
||||
await expect(notify({ sendAt: at, text: 'later' })).resolves.toEqual({
|
||||
await expect(
|
||||
notify({ sendAt: at, text: 'later', priority: 'high' }),
|
||||
).resolves.toEqual({
|
||||
scheduled: { scheduleId: 'sched-1', sendAt: at },
|
||||
});
|
||||
expect(schedules.prepareSendAt).toHaveBeenCalledWith(
|
||||
@@ -283,6 +286,7 @@ describe('notify({ sendAt }) with services.schedules', () => {
|
||||
message: expect.objectContaining({
|
||||
title: 'Hi',
|
||||
text: 'later',
|
||||
priority: 'high',
|
||||
}),
|
||||
},
|
||||
);
|
||||
@@ -318,11 +322,20 @@ describe('deferredAppMessages.write', () => {
|
||||
const key = uuidv4();
|
||||
const id = await claimed(token);
|
||||
|
||||
await expect(write(key, txFor(id, token))).resolves.toBe('delivered');
|
||||
await expect(
|
||||
write(key, txFor(id, token), {
|
||||
payload: JSON.stringify({
|
||||
message: { title: 'Later', priority: 'high' },
|
||||
}),
|
||||
}),
|
||||
).resolves.toBe('delivered');
|
||||
const rows = await mailbox();
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({ uid: key, send_key: key });
|
||||
expect(valueOf(rows[0])).toMatchObject({ title: 'Later' });
|
||||
expect(valueOf(rows[0])).toMatchObject({
|
||||
title: 'Later',
|
||||
priority: 'high',
|
||||
});
|
||||
});
|
||||
|
||||
it('writes nothing when the guard no longer holds: another claim took it', async () => {
|
||||
|
||||
@@ -859,3 +859,32 @@ describe('NotificationDriver — rows written before the scope columns', () => {
|
||||
expect(await selectUids(holder.actor)).toEqual([uid]);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe('NotificationDriver displayed notification catch-up', () => {
|
||||
it('selects named ids within the app and user scope', async () => {
|
||||
const box = await mailbox();
|
||||
await server.services.notification.markAcknowledged(box.rows.appUser, box.holder.userId);
|
||||
const rows = await withActor(box.ownAppActor, () => driver.select({
|
||||
subject: 'notif:app-user',
|
||||
uids: Object.values(box.rows),
|
||||
limit: 50,
|
||||
}));
|
||||
expect(rows).toEqual([expect.objectContaining({
|
||||
uid: box.rows.appUser, acknowledged: expect.any(Number),
|
||||
})]);
|
||||
});
|
||||
|
||||
it.each([Array(51).fill('one'), [''], [7], 'one', ['x'.repeat(129)]])(
|
||||
'rejects an invalid or oversized id batch: %s', async (uids) => {
|
||||
const { actor } = await makeUser();
|
||||
await expect(withActor(actor, () => driver.select({ uids })))
|
||||
.rejects.toMatchObject({ statusCode: 400 });
|
||||
},
|
||||
);
|
||||
|
||||
it('returns no rows for an empty id batch', async () => {
|
||||
const { actor } = await makeUser();
|
||||
expect(await withActor(actor, () => driver.select({ uids: [] }))).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -46,7 +46,7 @@ import {
|
||||
ownedAppUids,
|
||||
} from '../../services/notification/notificationAudience.js';
|
||||
import {
|
||||
NOTIFICATION_VALUE_MAX_BYTES,
|
||||
assertNotificationValueFits,
|
||||
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
|
||||
resolveNotificationWrite,
|
||||
type NotificationTypeName,
|
||||
@@ -120,19 +120,6 @@ interface MailboxScope {
|
||||
appUid: string | null | undefined;
|
||||
}
|
||||
|
||||
const assertValueFits = (value: Record<string, unknown>): void => {
|
||||
if (
|
||||
Buffer.byteLength(JSON.stringify(value), 'utf8') >
|
||||
NOTIFICATION_VALUE_MAX_BYTES
|
||||
) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
`\`value\` may not exceed ${NOTIFICATION_VALUE_MAX_BYTES} bytes`,
|
||||
{ legacyCode: 'notification_value_too_large' },
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const typeNotAllowed = (message: string): HttpError =>
|
||||
new HttpError(403, message, {
|
||||
legacyCode: 'forbidden',
|
||||
@@ -277,7 +264,7 @@ export class NotificationDriver extends PuterDriver {
|
||||
value,
|
||||
);
|
||||
}
|
||||
assertValueFits(value);
|
||||
assertNotificationValueFits(value);
|
||||
if (type === APP_MESSAGE_TYPE) {
|
||||
throw typeNotAllowed(`Only apps can create ${APP_MESSAGE_TYPE}`);
|
||||
}
|
||||
@@ -342,13 +329,34 @@ export class NotificationDriver extends PuterDriver {
|
||||
: predicate;
|
||||
|
||||
const scope = this.#resolveScope(actor, args.subject);
|
||||
const uids = args.uids;
|
||||
if (
|
||||
uids !== undefined &&
|
||||
(!Array.isArray(uids) ||
|
||||
uids.length > 50 ||
|
||||
uids.some(
|
||||
(uid) =>
|
||||
typeof uid !== 'string' ||
|
||||
uid.length === 0 ||
|
||||
uid.length > 128,
|
||||
))
|
||||
) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'uids must contain at most 50 notification ids',
|
||||
{
|
||||
legacyCode: 'bad_request',
|
||||
},
|
||||
);
|
||||
}
|
||||
const query: {
|
||||
limit: number;
|
||||
scope: MailboxScope | null;
|
||||
filter?: string;
|
||||
onlyUnacknowledged?: boolean;
|
||||
excludeTypes?: readonly string[];
|
||||
} = { limit, scope };
|
||||
uids?: string[];
|
||||
} = { limit, scope, ...(uids ? { uids: uids as string[] } : {}) };
|
||||
// The whole mailbox, or a slice naming no app, is the desktop's view of
|
||||
// it, which leaves out the types only an app has a use for. A slice
|
||||
// naming the app still returns them.
|
||||
@@ -430,7 +438,7 @@ export class NotificationDriver extends PuterDriver {
|
||||
}
|
||||
const handle = to === undefined ? undefined : parseNotifyTarget(to);
|
||||
const message = parseAppMessage(fields);
|
||||
assertValueFits(value);
|
||||
assertNotificationValueFits(value);
|
||||
const timing =
|
||||
sendAt === undefined || sendAt === null
|
||||
? null
|
||||
@@ -539,9 +547,7 @@ export class NotificationDriver extends PuterDriver {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* For extensions. A field, not a method, so `/drivers/call` can't reach it.
|
||||
*/
|
||||
/** For extensions. A field, not a method, so `/drivers/call` can't reach it. */
|
||||
readonly deferredAppMessages = {
|
||||
write: (input: DeferredWriteInput, tx: DeferredWriteTx) =>
|
||||
this.#writeDeferredAppMessage(input, tx),
|
||||
|
||||
@@ -129,6 +129,27 @@ describe('pipeStreamResult', () => {
|
||||
expect(warn).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('adds at most two close listeners to the response', async () => {
|
||||
// The response already carries close listeners from metering,
|
||||
// compression and tracing; Node warns past ten.
|
||||
let added = -1;
|
||||
server = createServer((_req, res) => {
|
||||
const before = res.listenerCount('close');
|
||||
pipeStreamResult(Readable.from(['a', 'b']), res, 'test.run');
|
||||
added = res.listenerCount('close') - before;
|
||||
});
|
||||
await new Promise<void>((resolve) => server!.listen(0, resolve));
|
||||
const { port } = server.address() as AddressInfo;
|
||||
await new Promise<void>((resolve) =>
|
||||
get(`http://127.0.0.1:${port}/`, (res) => {
|
||||
res.resume();
|
||||
res.on('end', () => resolve());
|
||||
}),
|
||||
);
|
||||
expect(added).toBeGreaterThan(0);
|
||||
expect(added).toBeLessThanOrEqual(2);
|
||||
});
|
||||
|
||||
it('drops the source, without throwing, when the client left before the result was ready', async () => {
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const source = Readable.from(['late']);
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { pipeline, type Readable } from 'node:stream';
|
||||
import type { Readable } from 'node:stream';
|
||||
|
||||
// Only what's used here, so callers on another @types/node (or express's
|
||||
// Response) still fit.
|
||||
@@ -32,24 +32,32 @@ type ResponseSink = NodeJS.WritableStream & {
|
||||
* mid-body destroys the response, so the client sees an aborted transfer rather
|
||||
* than a clean end, and the failure is logged once instead of surfacing as an
|
||||
* unhandled stream error. A client that hangs up stops the source.
|
||||
*
|
||||
* `pipe` rather than `pipeline`: pipeline puts seven close listeners on the
|
||||
* response, and with the ones metering, compression and tracing add, every
|
||||
* streamed response tripped Node's ten-listener warning.
|
||||
*/
|
||||
export function pipeStreamResult(
|
||||
stream: Readable,
|
||||
res: ResponseSink,
|
||||
label: string,
|
||||
): void {
|
||||
// The caller hung up while the driver ran; `pipeline` throws on a closed
|
||||
// destination, so drop the source instead.
|
||||
// The caller hung up while the driver ran: nothing will read the source.
|
||||
if (res.closed || res.destroyed) {
|
||||
stream.destroy();
|
||||
return;
|
||||
}
|
||||
pipeline(stream, res, (err) => {
|
||||
if (!err) return;
|
||||
stream.once('error', (err) => {
|
||||
stream.unpipe(res);
|
||||
if (!res.destroyed) res.destroy();
|
||||
// The client hanging up is not a failure of ours.
|
||||
if ((err as { code?: string }).code === 'ERR_STREAM_PREMATURE_CLOSE')
|
||||
return;
|
||||
console.warn(`[drivers] ${label} stream failed mid-response:`, err);
|
||||
});
|
||||
// A response that closes before the source ended is a client hanging up.
|
||||
res.once('close', () => {
|
||||
if (stream.readableEnded) return;
|
||||
stream.unpipe(res);
|
||||
stream.destroy();
|
||||
});
|
||||
res.once('error', () => stream.destroy());
|
||||
stream.pipe(res);
|
||||
}
|
||||
@@ -754,6 +754,50 @@ describe('WorkerDriver.create with a configured deploy backend', () => {
|
||||
expect(putCalls()).toHaveLength(1);
|
||||
});
|
||||
|
||||
const uploadedMetadata = () => {
|
||||
const [, init] = putCalls().at(-1)!;
|
||||
const form = (init as RequestInit).body as FormData;
|
||||
return JSON.parse(form.get('metadata') as string);
|
||||
};
|
||||
|
||||
it('sends limits.cpu_ms in the upload metadata when cpuMs is set', async () => {
|
||||
const { user, actor } = await makeUser();
|
||||
const path = `/${user.username}/cpu.worker.js`;
|
||||
await writeSource(actor, user.id, path, 'export default {}');
|
||||
const scriptName = `dt-cpu-${user.username}`;
|
||||
await inCtx(actor, () =>
|
||||
target.create({
|
||||
appId: '',
|
||||
workerName: scriptName,
|
||||
filePath: path,
|
||||
[INTERNAL_DEPLOY_TARGET]: { scriptName, cpuMs: 50 },
|
||||
}),
|
||||
);
|
||||
expect(uploadedMetadata().limits).toEqual({ cpu_ms: 50 });
|
||||
});
|
||||
|
||||
it('sends no limits key without cpuMs, and none on an ordinary create', async () => {
|
||||
const { user, actor } = await makeUser();
|
||||
const path = `/${user.username}/nocpu.worker.js`;
|
||||
await writeSource(actor, user.id, path, 'export default {}');
|
||||
const scriptName = `dt-nocpu-${user.username}`;
|
||||
await deployTo(actor, path, scriptName);
|
||||
expect(uploadedMetadata()).not.toHaveProperty('limits');
|
||||
|
||||
const name = `plain-${user.username}`;
|
||||
await inCtx(actor, () =>
|
||||
target.create({ appId: '', workerName: name, filePath: path }),
|
||||
);
|
||||
const metadata = uploadedMetadata();
|
||||
expect(metadata).not.toHaveProperty('limits');
|
||||
expect(Object.keys(metadata)).toEqual([
|
||||
'body_part',
|
||||
'compatibility_flags',
|
||||
'compatibility_date',
|
||||
'bindings',
|
||||
]);
|
||||
});
|
||||
|
||||
it('refuses a file another user owns, even with manage on it', async () => {
|
||||
const owner = await makeUser();
|
||||
const deployer = await makeUser();
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
// however the interactions with the Puter API are
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
@@ -13,6 +14,7 @@ import {
|
||||
import { Actor, makeActor as resolveActor } from '../../core/actor.js';
|
||||
import { runWithContext } from '../../core/context.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { DEFAULT_FREE_SUBSCRIPTION } from '../../services/metering/consts.js';
|
||||
import {
|
||||
setupPuterTestEnv,
|
||||
setupTestServer,
|
||||
@@ -797,6 +799,174 @@ describe('WorkerDriver', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('rented-unit cap', () => {
|
||||
const FREE_CAP = 16;
|
||||
const PAID_CAP = 100;
|
||||
|
||||
// The cap sits after the deploy-credentials gate, so the driver needs
|
||||
// credentials to reach it; a create under the cap then fails on the
|
||||
// missing source file, which is what tells it past the cap check.
|
||||
let originalWorkersConfig: unknown;
|
||||
let originalEvents: unknown;
|
||||
beforeAll(() => {
|
||||
const driverConfig = (
|
||||
target as unknown as { config: Record<string, unknown> }
|
||||
).config;
|
||||
originalEvents = driverConfig.events;
|
||||
driverConfig.events = { userHandlers: true };
|
||||
originalWorkersConfig = driverConfig.workers;
|
||||
driverConfig.workers = {
|
||||
XAUTHKEY: 'test-key',
|
||||
ACCOUNTID: 'test-account',
|
||||
};
|
||||
});
|
||||
afterAll(() => {
|
||||
const driverConfig = (
|
||||
target as unknown as { config: Record<string, unknown> }
|
||||
).config;
|
||||
driverConfig.workers = originalWorkersConfig;
|
||||
driverConfig.events = originalEvents;
|
||||
});
|
||||
|
||||
const holding = (
|
||||
plan: string,
|
||||
held: { workers?: number; scopes?: number; owned?: number },
|
||||
) => {
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'getActorSubscription',
|
||||
).mockResolvedValue({ id: plan } as never);
|
||||
vi.spyOn(
|
||||
server.stores.subdomain,
|
||||
'countByUserIdAndPrefix',
|
||||
).mockResolvedValue(held.workers ?? 0);
|
||||
vi.spyOn(
|
||||
server.stores.eventHandler,
|
||||
'countUserScopesWithHandlersForUser',
|
||||
).mockResolvedValue(held.scopes ?? 0);
|
||||
vi.spyOn(
|
||||
server.stores.eventHandler,
|
||||
'countAppsWithHandlersForOwner',
|
||||
).mockResolvedValue(held.owned ?? 0);
|
||||
};
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const createWorker = () =>
|
||||
inCtx(() =>
|
||||
target.create({
|
||||
appId: 'test-app',
|
||||
workerName: 'capped',
|
||||
filePath: '/test.js',
|
||||
}),
|
||||
);
|
||||
|
||||
it('keeps the flat 100-worker cap over ordinary workers with user scopes off', async () => {
|
||||
const driverConfig = (
|
||||
target as unknown as { config: Record<string, unknown> }
|
||||
).config;
|
||||
driverConfig.events = { userHandlers: false };
|
||||
try {
|
||||
// 16 workers is fine here, and scopes and owned apps don't count.
|
||||
holding(DEFAULT_FREE_SUBSCRIPTION, {
|
||||
workers: 16,
|
||||
scopes: 50,
|
||||
owned: 50,
|
||||
});
|
||||
const list = vi
|
||||
.spyOn(server.stores.subdomain, 'listByUserIdAndPrefix')
|
||||
.mockResolvedValue(
|
||||
Array.from({ length: 16 }, () => ({})) as never,
|
||||
);
|
||||
await expect(createWorker()).rejects.not.toMatchObject({
|
||||
statusCode: 403,
|
||||
});
|
||||
|
||||
list.mockResolvedValue(
|
||||
Array.from({ length: 100 }, () => ({})) as never,
|
||||
);
|
||||
await expect(createWorker()).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
message: 'Worker limit reached (max 100)',
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
} finally {
|
||||
driverConfig.events = { userHandlers: true };
|
||||
}
|
||||
});
|
||||
|
||||
it('refuses a free user one unit past the cap, across all three sources', async () => {
|
||||
for (const held of [
|
||||
{ workers: FREE_CAP },
|
||||
{ scopes: FREE_CAP },
|
||||
{ owned: FREE_CAP },
|
||||
{ workers: 10, scopes: 4, owned: 2 },
|
||||
]) {
|
||||
holding(DEFAULT_FREE_SUBSCRIPTION, held);
|
||||
await expect(createWorker()).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
legacyCode: 'forbidden',
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('lets a free user create up to the cap', async () => {
|
||||
holding(DEFAULT_FREE_SUBSCRIPTION, {
|
||||
workers: 10,
|
||||
scopes: 3,
|
||||
owned: 2,
|
||||
});
|
||||
await expect(createWorker()).rejects.not.toMatchObject({
|
||||
statusCode: 403,
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses a paid user the unit after 100', async () => {
|
||||
holding('paid-plan', { workers: 90, scopes: 5, owned: 5 });
|
||||
await expect(createWorker()).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
});
|
||||
holding('paid-plan', { workers: 90, scopes: 5, owned: 4 });
|
||||
await expect(createWorker()).rejects.not.toMatchObject({
|
||||
statusCode: 403,
|
||||
});
|
||||
expect(PAID_CAP).toBe(100);
|
||||
});
|
||||
|
||||
it('keeps a free user over the cap, redeploys what they hold, and refuses a new one', async () => {
|
||||
holding(DEFAULT_FREE_SUBSCRIPTION, { workers: 20 });
|
||||
const user = actor.user;
|
||||
const exists = vi
|
||||
.spyOn(server.stores.subdomain, 'getBySubdomain')
|
||||
.mockImplementation(
|
||||
async (name: string) =>
|
||||
(name === 'workers.puter.capped'
|
||||
? { user_id: user.id, uuid: 'existing-uuid' }
|
||||
: null) as never,
|
||||
);
|
||||
const noApp = makeActor({ app: undefined });
|
||||
const create = (workerName: string) =>
|
||||
inCtx(
|
||||
() =>
|
||||
target.create({
|
||||
appId: '',
|
||||
workerName,
|
||||
filePath: '/test.js',
|
||||
}),
|
||||
noApp,
|
||||
);
|
||||
|
||||
await expect(create('capped')).rejects.not.toMatchObject({
|
||||
statusCode: 403,
|
||||
});
|
||||
await expect(create('another')).rejects.toMatchObject({
|
||||
statusCode: 403,
|
||||
});
|
||||
exists.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('admission bypass', () => {
|
||||
const unverified = (): Actor =>
|
||||
makeActor({
|
||||
|
||||
@@ -43,6 +43,7 @@ import { PuterDriver } from '../types.js';
|
||||
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
|
||||
import { awaitLater } from '../../util/concurrency.js';
|
||||
import { isUniqueViolation } from '../../util/dbError.js';
|
||||
import { assertRentedUnitRoom } from '../../util/rentedUnits.js';
|
||||
import { loadFileInput } from '../util/fileInput.js';
|
||||
import {
|
||||
normalizeLimit,
|
||||
@@ -57,6 +58,7 @@ const CF_BASE_URL = 'https://api.cloudflare.com/client/v4/accounts';
|
||||
export const WORKER_COMPATIBILITY_DATE = '2025-07-15';
|
||||
export const WORKER_COMPATIBILITY_FLAGS = ['global_fetch_strictly_public'];
|
||||
const WORKER_NAME_REGEX = /^[a-zA-Z0-9_-]+$/;
|
||||
/** Cap on ordinary workers while user-scope handlers are off. */
|
||||
const MAX_WORKERS_PER_USER = 100;
|
||||
|
||||
/**
|
||||
@@ -179,12 +181,17 @@ export interface InternalDeployTarget {
|
||||
* scripts belong to what.
|
||||
*/
|
||||
tags?: string[];
|
||||
/**
|
||||
* Per-invocation CPU limit in milliseconds, sent with the upload. Unset
|
||||
* sends no limit. Ignored by the local runtime.
|
||||
*/
|
||||
cpuMs?: number;
|
||||
}
|
||||
|
||||
const MAX_SOURCE_SIZE = 10 * 1024 * 1024; // 10 MB
|
||||
// How far to scan an app's child apps when resolving which workers it may
|
||||
// see. A user is capped at MAX_WORKERS_PER_USER workers, so only that many
|
||||
// child apps can actually own one; the ceiling is generous slack over that.
|
||||
// see. A user is capped at their plan's rented units, so only that many
|
||||
// child apps can actually own a worker; the ceiling is generous slack over that.
|
||||
const CHILD_APP_SCAN_LIMIT = 1000;
|
||||
|
||||
// -- Preamble --------------------------------------------------------
|
||||
@@ -452,26 +459,34 @@ export class WorkerDriver extends PuterDriver {
|
||||
|
||||
this.#requireCfConfig();
|
||||
|
||||
// Quota check — count existing workers.puter.* subdomains owned by user
|
||||
//
|
||||
// A deploy target creates no such row, so this count can neither see
|
||||
// its scripts nor be grown by them; running it would only make an
|
||||
// unrelated worker quota decide whether these deploy. Whatever bounds
|
||||
// them has to be counted where they actually live, and belongs to the
|
||||
// caller that owns their lifecycle.
|
||||
const existingWorkers =
|
||||
skipAdmission || deployTarget
|
||||
? []
|
||||
: await this.stores.subdomain.listByUserIdAndPrefix(
|
||||
actor.user.id,
|
||||
WORKER_SUBDOMAIN_PREFIX,
|
||||
);
|
||||
if (existingWorkers.length >= MAX_WORKERS_PER_USER) {
|
||||
throw new HttpError(
|
||||
403,
|
||||
`Worker limit reached (max ${MAX_WORKERS_PER_USER})`,
|
||||
{ legacyCode: 'forbidden' },
|
||||
);
|
||||
// With user scopes on, a new worker is one more rented unit under the
|
||||
// plan's cap, and redeploying an existing worker is never refused.
|
||||
// Deploy targets create no worker row and are bounded by their caller.
|
||||
if (this.config.events?.userHandlers === true) {
|
||||
if (!skipAdmission && !deployTarget && !existingSub) {
|
||||
await assertRentedUnitRoom(
|
||||
{
|
||||
stores: this.stores,
|
||||
metering: this.services.metering,
|
||||
},
|
||||
actor,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
const existingWorkers =
|
||||
skipAdmission || deployTarget
|
||||
? []
|
||||
: await this.stores.subdomain.listByUserIdAndPrefix(
|
||||
actor.user.id,
|
||||
WORKER_SUBDOMAIN_PREFIX,
|
||||
);
|
||||
if (existingWorkers.length >= MAX_WORKERS_PER_USER) {
|
||||
throw new HttpError(
|
||||
403,
|
||||
`Worker limit reached (max ${MAX_WORKERS_PER_USER})`,
|
||||
{ legacyCode: 'forbidden' },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Read source file. loadFileInput runs the read-ACL check internally
|
||||
@@ -940,6 +955,7 @@ export class WorkerDriver extends PuterDriver {
|
||||
compatibility_flags: WORKER_COMPATIBILITY_FLAGS,
|
||||
compatibility_date: WORKER_COMPATIBILITY_DATE,
|
||||
...(target?.tags ? { tags: target.tags } : {}),
|
||||
...(target?.cpuMs ? { limits: { cpu_ms: target.cpuMs } } : {}),
|
||||
bindings: [
|
||||
// A script deployed without a token gets no binding at all,
|
||||
// rather than an empty one for its code to find.
|
||||
|
||||
@@ -1492,6 +1492,104 @@ describe('PuterServer keep-alive timeout', () => {
|
||||
});
|
||||
|
||||
describe('PuterServer lifecycle hooks', () => {
|
||||
it('fails startup before running later hooks when a start hook rejects', async () => {
|
||||
const failure = new Error('startup failed');
|
||||
const later = vi.fn();
|
||||
class FailingStart extends JobScheduler {
|
||||
override async onServerStart() {
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
class LaterStart extends JobScheduler {
|
||||
override async onServerStart() {
|
||||
later();
|
||||
}
|
||||
}
|
||||
extension.registerService('failingStart', FailingStart as never);
|
||||
extension.registerService('laterStart', LaterStart as never);
|
||||
try {
|
||||
await expect(setupTestServer()).rejects.toBe(failure);
|
||||
expect(later).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
delete (extensionStore.services as Record<string, unknown>)
|
||||
.failingStart;
|
||||
delete (extensionStore.services as Record<string, unknown>)
|
||||
.laterStart;
|
||||
}
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
'continues teardown after failing hooks with listener callback %s',
|
||||
async (prepareFirst) => {
|
||||
const port = await allocateEphemeralPort();
|
||||
const server = await setupTestServer(
|
||||
{ port } as unknown as IConfig,
|
||||
{ listen: true },
|
||||
);
|
||||
const logged = vi
|
||||
.spyOn(console, 'error')
|
||||
.mockImplementation(() => {});
|
||||
const ran: string[] = [];
|
||||
class FailingHooks {
|
||||
onServerPrepareShutdown() {
|
||||
ran.push('prepare:failed');
|
||||
throw new Error('prepare failed');
|
||||
}
|
||||
async onServerShutdown() {
|
||||
ran.push('shutdown:failed');
|
||||
throw new Error('shutdown failed');
|
||||
}
|
||||
}
|
||||
(server.services as Record<string, unknown>).fails =
|
||||
new FailingHooks();
|
||||
(server.services as Record<string, unknown>).after = {
|
||||
onServerPrepareShutdown: () => ran.push('prepare:after'),
|
||||
onServerShutdown: () => ran.push('shutdown:after'),
|
||||
};
|
||||
(server.clients as Record<string, unknown>).last = {
|
||||
onServerShutdown: () => ran.push('shutdown:client'),
|
||||
};
|
||||
try {
|
||||
if (prepareFirst) {
|
||||
await server.prepareShutdown();
|
||||
await vi.waitFor(() =>
|
||||
expect(ran).toContain('prepare:after'),
|
||||
);
|
||||
}
|
||||
await server.shutdown();
|
||||
expect(ran).toEqual([
|
||||
'prepare:failed',
|
||||
'prepare:after',
|
||||
'shutdown:failed',
|
||||
'shutdown:after',
|
||||
'shutdown:client',
|
||||
]);
|
||||
expect(logged).toHaveBeenCalledWith(
|
||||
'[server] onServerPrepareShutdown failed in FailingHooks',
|
||||
expect.any(Error),
|
||||
);
|
||||
expect(logged).toHaveBeenCalledWith(
|
||||
'[server] onServerShutdown failed in FailingHooks',
|
||||
expect.any(Error),
|
||||
);
|
||||
await expect(
|
||||
new Promise<void>((resolve, reject) => {
|
||||
const socket = net.connect(port, '127.0.0.1');
|
||||
socket.on('connect', () => {
|
||||
socket.destroy();
|
||||
resolve();
|
||||
});
|
||||
socket.on('error', reject);
|
||||
}),
|
||||
).rejects.toMatchObject({ code: 'ECONNREFUSED' });
|
||||
} finally {
|
||||
delete (server.services as Record<string, unknown>).fails;
|
||||
await server.shutdown();
|
||||
logged.mockRestore();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('runs drain-start and prepare-shutdown bottom-up, once, before tearing down', async () => {
|
||||
const port = await allocateEphemeralPort();
|
||||
const server = await setupTestServer({ port } as unknown as IConfig, {
|
||||
@@ -1541,6 +1639,73 @@ describe('PuterServer lifecycle hooks', () => {
|
||||
'shutdown:client',
|
||||
]);
|
||||
});
|
||||
|
||||
it('keeps accepting connections until every drain-start hook settles, without one holding up another', async () => {
|
||||
const port = await allocateEphemeralPort();
|
||||
const server = await setupTestServer({ port } as unknown as IConfig, {
|
||||
listen: true,
|
||||
});
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => (release = resolve));
|
||||
let otherRan = false;
|
||||
(server.services as unknown as Record<string, unknown>).holdsListener =
|
||||
{ onServerDrainStart: () => held };
|
||||
(server.drivers as unknown as Record<string, unknown>).other = {
|
||||
onServerDrainStart: () => {
|
||||
otherRan = true;
|
||||
},
|
||||
};
|
||||
|
||||
const connect = () =>
|
||||
new Promise<void>((resolve, reject) => {
|
||||
const socket = net.connect(port, '127.0.0.1');
|
||||
socket.on('connect', () => {
|
||||
socket.destroy();
|
||||
resolve();
|
||||
});
|
||||
socket.on('error', reject);
|
||||
});
|
||||
|
||||
const prepared = server.prepareShutdown({ drainMs: 1_000 });
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(otherRan).toBe(true);
|
||||
await expect(connect()).resolves.toBeUndefined();
|
||||
|
||||
release();
|
||||
await prepared;
|
||||
await expect(connect()).rejects.toMatchObject({
|
||||
code: 'ECONNREFUSED',
|
||||
});
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
it('runs every drain-start hook when one of them throws', async () => {
|
||||
const port = await allocateEphemeralPort();
|
||||
const server = await setupTestServer({ port } as unknown as IConfig, {
|
||||
listen: true,
|
||||
});
|
||||
vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
const ran: string[] = [];
|
||||
(server.stores as unknown as Record<string, unknown>).fails = {
|
||||
onServerDrainStart: () => {
|
||||
throw new Error('drain hook failed');
|
||||
},
|
||||
};
|
||||
(server.services as unknown as Record<string, unknown>).after = {
|
||||
onServerDrainStart: () => {
|
||||
ran.push('after');
|
||||
},
|
||||
};
|
||||
|
||||
await server.prepareShutdown({ drainMs: 1_000 });
|
||||
await server.shutdown();
|
||||
|
||||
expect(ran).toEqual(['after']);
|
||||
expect(console.error).toHaveBeenCalledWith(
|
||||
expect.stringContaining('onServerDrainStart failed'),
|
||||
expect.any(Error),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('PuterServer shutdown during prepare', () => {
|
||||
|
||||
+47
-13
@@ -1743,12 +1743,7 @@ export class PuterServer {
|
||||
*/
|
||||
async #runHook(
|
||||
order: 'bottomUp' | 'topDown',
|
||||
hook:
|
||||
| 'onServerStart'
|
||||
| 'onServerDrainStart'
|
||||
| 'onServerPrepareShutdown'
|
||||
| 'onServerShutdown',
|
||||
info?: ServerDrainInfo,
|
||||
hook: 'onServerStart' | 'onServerPrepareShutdown' | 'onServerShutdown',
|
||||
) {
|
||||
const layers = [
|
||||
this.clients,
|
||||
@@ -1760,11 +1755,47 @@ export class PuterServer {
|
||||
if (order === 'topDown') layers.reverse();
|
||||
for (const layer of layers) {
|
||||
for (const instance of Object.values(layer) as WithLifecycle[]) {
|
||||
await instance[hook]?.(info);
|
||||
try {
|
||||
await instance[hook]?.();
|
||||
} catch (err) {
|
||||
if (hook === 'onServerStart') throw err;
|
||||
console.error(
|
||||
`[server] ${hook} failed in ${instance.constructor.name}`,
|
||||
err,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run every `onServerDrainStart` concurrently, so one hook holding the
|
||||
* listener open doesn't delay the others, and one failing doesn't skip the
|
||||
* rest.
|
||||
*/
|
||||
async #runDrainStartHooks(info: ServerDrainInfo) {
|
||||
const instances = [
|
||||
this.clients,
|
||||
this.stores,
|
||||
this.services,
|
||||
this.controllers,
|
||||
this.drivers,
|
||||
].flatMap((layer) => Object.values(layer) as WithLifecycle[]);
|
||||
const results = await Promise.allSettled(
|
||||
instances.map(async (instance) =>
|
||||
instance.onServerDrainStart?.(info),
|
||||
),
|
||||
);
|
||||
results.forEach((result, i) => {
|
||||
if (result.status === 'rejected') {
|
||||
console.error(
|
||||
`[server] onServerDrainStart failed in ${instances[i].constructor.name}`,
|
||||
result.reason,
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#prepareShutdownHooks: Promise<void> | null = null;
|
||||
|
||||
/**
|
||||
@@ -1782,16 +1813,19 @@ export class PuterServer {
|
||||
|
||||
async prepareShutdown(info: ServerDrainInfo = {}) {
|
||||
if (this.#server) {
|
||||
try {
|
||||
await this.#runHook('bottomUp', 'onServerDrainStart', info);
|
||||
} catch (err) {
|
||||
console.error('[server] onServerDrainStart failed', err);
|
||||
}
|
||||
await this.#runDrainStartHooks(info);
|
||||
this.#server.close(async () => {
|
||||
console.log(
|
||||
'PuterServer has stopped accepting new connections',
|
||||
);
|
||||
await this.#runPrepareShutdownHooks();
|
||||
try {
|
||||
await this.#runPrepareShutdownHooks();
|
||||
} catch (err) {
|
||||
console.error(
|
||||
'[server] prepare-shutdown hooks failed',
|
||||
err,
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1164,31 +1164,6 @@ describe('OIDCService.createUserFromOIDC', () => {
|
||||
server.clients.event.off('puter.signup.success', onSignup);
|
||||
});
|
||||
|
||||
it('announces the utm it was given', async () => {
|
||||
const email = `utm-${crypto.randomBytes(4).toString('hex')}@example.com`;
|
||||
const signups: Array<Record<string, unknown>> = [];
|
||||
const onSignup = (_k: string, data: unknown) =>
|
||||
signups.push(data as Record<string, unknown>);
|
||||
server.clients.event.on('puter.signup.success', onSignup);
|
||||
try {
|
||||
const result = await runWithContext({ req }, () =>
|
||||
oidc().createUserFromOIDC(
|
||||
'google',
|
||||
{ sub: `utm-sub-${email}`, email, email_verified: true },
|
||||
null,
|
||||
{ utm: { source: 'ads', campaign: 'launch' } },
|
||||
),
|
||||
);
|
||||
expect(result.success).toBe(true);
|
||||
expect(signups[0]?.utm).toEqual({
|
||||
source: 'ads',
|
||||
campaign: 'launch',
|
||||
});
|
||||
} finally {
|
||||
server.clients.event.off('puter.signup.success', onSignup);
|
||||
}
|
||||
});
|
||||
|
||||
it('retries the generated username when its home path is already occupied', async () => {
|
||||
// The RNG is input to the candidate generator, not the code under
|
||||
// test: compute the same first candidate the service will try.
|
||||
|
||||
@@ -30,7 +30,6 @@ import { Context } from '../../core';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import crypto from 'node:crypto';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import type { SignupUtm } from '../../util/signupUtm.js';
|
||||
import { verifyOidcIdToken, type JwksCacheEntry } from './oidcIdToken';
|
||||
|
||||
const GOOGLE_DISCOVERY_URL =
|
||||
@@ -493,17 +492,13 @@ export class OIDCService extends PuterService {
|
||||
* re-resolve rather than surface an error — see
|
||||
* `#resolveOrCreateOIDCUser`.
|
||||
*
|
||||
* `bonusCode` must already be canonical (`normalizeBonusCode`) and `utm`
|
||||
* already shaped (`normalizeSignupUtm`).
|
||||
* `bonusCode` must already be canonical (`normalizeBonusCode`).
|
||||
*/
|
||||
async createUserFromOIDC(
|
||||
providerId: string,
|
||||
claims: OIDCUserInfo,
|
||||
referrer?: string | null,
|
||||
{
|
||||
bonusCode = null,
|
||||
utm = null,
|
||||
}: { bonusCode?: string | null; utm?: SignupUtm | null } = {},
|
||||
{ bonusCode = null }: { bonusCode?: string | null } = {},
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
user?: UserRow;
|
||||
@@ -691,7 +686,6 @@ export class OIDCService extends PuterService {
|
||||
bonusCode,
|
||||
emailConfirmed: true,
|
||||
saveAccount: true,
|
||||
utm,
|
||||
});
|
||||
|
||||
return { success: true, user };
|
||||
|
||||
@@ -1149,3 +1149,154 @@ describe('BroadcastService — same-cluster pub/sub fan-out', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('BroadcastService shutdown drain', () => {
|
||||
let server: PuterServer;
|
||||
let broadcast: BroadcastService;
|
||||
|
||||
beforeEach(async () => {
|
||||
axiosRequestMock.mockReset();
|
||||
axiosRequestMock.mockResolvedValue({ status: 200, data: 'ok' });
|
||||
server = await setupTestServer({
|
||||
broadcast: {
|
||||
webhook: { peerId: SELF_PEER_ID, secret: SELF_SECRET },
|
||||
peers: [
|
||||
{
|
||||
peerId: PEER_ID,
|
||||
webhook: true,
|
||||
webhook_url: PEER_URL,
|
||||
webhook_secret: PEER_SECRET,
|
||||
},
|
||||
],
|
||||
outbound_flush_ms: 10,
|
||||
},
|
||||
} as never);
|
||||
broadcast = server.services.broadcast as unknown as BroadcastService;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
vi.useRealTimers();
|
||||
await broadcast.onServerPrepareShutdown();
|
||||
await broadcast.onServerShutdown();
|
||||
await server.shutdown();
|
||||
});
|
||||
|
||||
const emit = (hash: string) =>
|
||||
server.clients.event.emit(
|
||||
'outer.fs.write-hash' as never,
|
||||
{ hash, uuid: hash } as never,
|
||||
{},
|
||||
);
|
||||
|
||||
it.each(['onServerPrepareShutdown', 'onServerShutdown'] as const)(
|
||||
'waits for the active flush and sends its tail in %s',
|
||||
async (hook) => {
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
let releaseTail!: () => void;
|
||||
const tailHeld = new Promise<void>((resolve) => {
|
||||
releaseTail = resolve;
|
||||
});
|
||||
axiosRequestMock
|
||||
.mockImplementationOnce(async () => {
|
||||
await held;
|
||||
return { status: 200, data: 'ok' };
|
||||
})
|
||||
.mockImplementationOnce(async () => {
|
||||
await tailHeld;
|
||||
return { status: 200, data: 'ok' };
|
||||
});
|
||||
emit('first');
|
||||
await vi.waitFor(() =>
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(1),
|
||||
);
|
||||
emit('tail');
|
||||
let finished = false;
|
||||
const draining = broadcast[hook]().then(() => {
|
||||
finished = true;
|
||||
});
|
||||
try {
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(finished).toBe(false);
|
||||
release();
|
||||
await vi.waitFor(() =>
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(2),
|
||||
);
|
||||
expect(finished).toBe(false);
|
||||
} finally {
|
||||
release();
|
||||
releaseTail();
|
||||
await draining;
|
||||
}
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(2);
|
||||
expect(axiosRequestMock.mock.calls[1][0].data).toContain('tail');
|
||||
},
|
||||
);
|
||||
|
||||
it('keeps normal timer flushing after prepare on an idle server', async () => {
|
||||
await broadcast.onServerPrepareShutdown();
|
||||
vi.useFakeTimers();
|
||||
axiosRequestMock.mockClear();
|
||||
emit('during-drain');
|
||||
await vi.advanceTimersByTimeAsync(10);
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(1);
|
||||
expect(axiosRequestMock.mock.calls[0][0].data).toContain(
|
||||
'during-drain',
|
||||
);
|
||||
});
|
||||
|
||||
it('sends events queued after prepare during shutdown and clears the timer', async () => {
|
||||
await broadcast.onServerPrepareShutdown();
|
||||
vi.useFakeTimers();
|
||||
axiosRequestMock.mockClear();
|
||||
emit('after-prepare');
|
||||
await broadcast.onServerShutdown();
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(1);
|
||||
expect(axiosRequestMock.mock.calls[0][0].data).toContain(
|
||||
'after-prepare',
|
||||
);
|
||||
axiosRequestMock.mockClear();
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(axiosRequestMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(['onServerPrepareShutdown', 'onServerShutdown'] as const)(
|
||||
'bounds a hung flush in %s',
|
||||
async (hook) => {
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
axiosRequestMock.mockImplementationOnce(async () => {
|
||||
await held;
|
||||
return { status: 200, data: 'ok' };
|
||||
});
|
||||
emit('hung');
|
||||
await vi.waitFor(() =>
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(1),
|
||||
);
|
||||
vi.useFakeTimers();
|
||||
emit('tail');
|
||||
let finished = false;
|
||||
const draining = broadcast[hook]().then(() => {
|
||||
finished = true;
|
||||
});
|
||||
try {
|
||||
await vi.advanceTimersByTimeAsync(4_999);
|
||||
expect(finished).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
await draining;
|
||||
release();
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(
|
||||
hook === 'onServerPrepareShutdown' ? 2 : 1,
|
||||
);
|
||||
} finally {
|
||||
release();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -21,6 +21,8 @@ import axios from 'axios';
|
||||
import { createHmac, randomUUID, timingSafeEqual } from 'node:crypto';
|
||||
import { Agent as HttpsAgent } from 'node:https';
|
||||
import { IBroadcastPeerConfig } from '../../types.js';
|
||||
import { settleWithin } from '../../util/concurrency.js';
|
||||
import { FINAL_FLUSH_TIMEOUT_MS } from '../../util/gracefulShutdown.js';
|
||||
import { PuterService } from '../types.js';
|
||||
|
||||
// -- Wire types ------------------------------------------------------
|
||||
@@ -121,7 +123,8 @@ export class BroadcastService extends PuterService {
|
||||
/** Per peer, serialized events it has not accepted yet, oldest first. */
|
||||
#outboundBacklogs = new Map<IBroadcastPeerConfig, string[]>();
|
||||
#outboundFlushTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
#outboundIsFlushing = false;
|
||||
#outboundFlush: Promise<void> | null = null;
|
||||
#outboundDraining = false;
|
||||
|
||||
#webhookReplayWindowSeconds = 300;
|
||||
#outboundFlushMs = 2000;
|
||||
@@ -165,21 +168,40 @@ export class BroadcastService extends PuterService {
|
||||
}
|
||||
|
||||
override async onServerPrepareShutdown(): Promise<void> {
|
||||
await this.#drainOutboundEvents();
|
||||
if (this.#redisSub) {
|
||||
try {
|
||||
await this.#redisSub.unsubscribe('pubsub');
|
||||
this.#redisSub.quit();
|
||||
} catch (err) {
|
||||
console.warn('[broadcast] unsubscribe failed', err);
|
||||
} finally {
|
||||
this.#redisSub = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
override async onServerShutdown(): Promise<void> {
|
||||
this.#outboundDraining = true;
|
||||
await this.#drainOutboundEvents();
|
||||
}
|
||||
|
||||
async #drainOutboundEvents(): Promise<void> {
|
||||
if (this.#outboundFlushTimer) {
|
||||
clearTimeout(this.#outboundFlushTimer);
|
||||
this.#outboundFlushTimer = null;
|
||||
}
|
||||
// Best-effort drain — try one final flush so events queued near
|
||||
// shutdown make it out.
|
||||
try {
|
||||
await this.#flushOutboundEvents();
|
||||
} catch (err) {
|
||||
console.warn('[broadcast] final flush failed', err);
|
||||
}
|
||||
if (this.#redisSub) {
|
||||
await this.#redisSub.unsubscribe('pubsub');
|
||||
this.#redisSub.quit();
|
||||
this.#redisSub = null;
|
||||
const deadline = Date.now() + FINAL_FLUSH_TIMEOUT_MS;
|
||||
const drain = async () => {
|
||||
try {
|
||||
await this.#outboundFlush;
|
||||
if (Date.now() < deadline) await this.#flushOutboundEvents();
|
||||
} catch (err) {
|
||||
console.warn('[broadcast] final flush failed', err);
|
||||
}
|
||||
};
|
||||
if (!(await settleWithin(drain(), FINAL_FLUSH_TIMEOUT_MS))) {
|
||||
console.warn('[broadcast] final flush timed out');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -453,24 +475,27 @@ export class BroadcastService extends PuterService {
|
||||
}
|
||||
|
||||
#scheduleOutboundFlush(): void {
|
||||
if (this.#outboundFlushTimer) return;
|
||||
this.#outboundFlushTimer = setTimeout(() => {
|
||||
if (this.#outboundDraining || this.#outboundFlushTimer) return;
|
||||
this.#outboundFlushTimer = setTimeout(async () => {
|
||||
this.#outboundFlushTimer = null;
|
||||
void this.#flushOutboundEvents().catch((err) => {
|
||||
try {
|
||||
await this.#flushOutboundEvents();
|
||||
} catch (err) {
|
||||
console.warn('[broadcast] outbound flush failed', err);
|
||||
});
|
||||
}
|
||||
}, this.#outboundFlushMs);
|
||||
}
|
||||
|
||||
async #flushOutboundEvents(): Promise<void> {
|
||||
if (
|
||||
this.#outboundIsFlushing ||
|
||||
(this.#outboundQueue.size === 0 &&
|
||||
this.#outboundBacklogs.size === 0)
|
||||
)
|
||||
return;
|
||||
#flushOutboundEvents(): Promise<void> {
|
||||
if (this.#outboundFlush) return this.#outboundFlush;
|
||||
if (this.#outboundQueue.size === 0 && this.#outboundBacklogs.size === 0)
|
||||
return Promise.resolve();
|
||||
|
||||
this.#outboundIsFlushing = true;
|
||||
this.#outboundFlush = this.#sendOutboundEvents();
|
||||
return this.#outboundFlush;
|
||||
}
|
||||
|
||||
async #sendOutboundEvents(): Promise<void> {
|
||||
try {
|
||||
const fresh = this.#outboundQueue;
|
||||
this.#outboundQueue = new Set();
|
||||
@@ -504,7 +529,7 @@ export class BroadcastService extends PuterService {
|
||||
for (const result of flushed)
|
||||
if (result.status === 'rejected') throw result.reason;
|
||||
} finally {
|
||||
this.#outboundIsFlushing = false;
|
||||
this.#outboundFlush = null;
|
||||
// Anything that arrived during flush, or is still owed to a peer,
|
||||
// gets the next tick.
|
||||
if (
|
||||
|
||||
@@ -185,7 +185,11 @@ const aclService = () => ({
|
||||
!denied.has(resource.path) && !deniedHolders.has(actor.user?.id ?? -1),
|
||||
getSafeAclError: async (_actor: Actor, resource: { path: string }) =>
|
||||
denied.get(resource.path) === 'forbidden'
|
||||
? { status: 403, message: 'Forbidden', fields: { code: 'forbidden' } }
|
||||
? {
|
||||
status: 403,
|
||||
message: 'Forbidden',
|
||||
fields: { code: 'forbidden' },
|
||||
}
|
||||
: {
|
||||
status: 404,
|
||||
message: 'Subject does not exist',
|
||||
@@ -281,8 +285,8 @@ interface ForwardedCall {
|
||||
}
|
||||
|
||||
/**
|
||||
* Each service gets its own outbox. A delivery still in flight when a test
|
||||
* ends must land in that test's record, not in the next one's.
|
||||
* Each service gets its own outbox. A delivery still in flight when a test ends
|
||||
* must land in that test's record, not in the next one's.
|
||||
*/
|
||||
const buildService = (
|
||||
config: IConfig,
|
||||
@@ -350,10 +354,7 @@ const buildService = (
|
||||
acl: aclService(),
|
||||
permission: permissionService(),
|
||||
metering: {
|
||||
bufferIncrementUsages: (
|
||||
actor: Actor,
|
||||
usages: UsageInput[],
|
||||
) => {
|
||||
bufferIncrementUsages: (actor: Actor, usages: UsageInput[]) => {
|
||||
for (const usage of usages)
|
||||
lines.push({
|
||||
userUuid: actor.user?.uuid,
|
||||
@@ -597,6 +598,13 @@ describe('the feature switch', () => {
|
||||
nearPrimary: true,
|
||||
leaseMs: 15 * 60_000,
|
||||
},
|
||||
{
|
||||
name: 'events.userHandlers.gc',
|
||||
every: 60 * 60_000,
|
||||
scope: 'fleet',
|
||||
nearPrimary: true,
|
||||
leaseMs: 15 * 60_000,
|
||||
},
|
||||
{
|
||||
name: 'events.subscriptions.creditSweep',
|
||||
every: 15 * 60_000,
|
||||
@@ -1256,7 +1264,9 @@ describe('matching', () => {
|
||||
it('stops delivering to a durable row the moment it expires', async () => {
|
||||
const { documents, file } = seedTree();
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const row = (over: Partial<DurableSubscription>): DurableSubscription => ({
|
||||
const row = (
|
||||
over: Partial<DurableSubscription>,
|
||||
): DurableSubscription => ({
|
||||
subId: `durable-${seq}-${over.expiresAt}`,
|
||||
token: `f#${documents.uid}`,
|
||||
ownerUserId: userId,
|
||||
@@ -1405,9 +1415,9 @@ describe('matching', () => {
|
||||
await dispatch(closed);
|
||||
await flush();
|
||||
|
||||
expect(sent.map((s) => (s.envelope.event as { uid: string }).uid)).toEqual([
|
||||
reachable.uid,
|
||||
]);
|
||||
expect(
|
||||
sent.map((s) => (s.envelope.event as { uid: string }).uid),
|
||||
).toEqual([reachable.uid]);
|
||||
});
|
||||
|
||||
it('publishes nothing for an event with no registry entry', async () => {
|
||||
@@ -1449,9 +1459,9 @@ describe('matching', () => {
|
||||
await flush();
|
||||
|
||||
expect(sent).toHaveLength(1);
|
||||
expect(
|
||||
(sent[0].envelope.event as { path: string }).path,
|
||||
).toBe(inside.path);
|
||||
expect((sent[0].envelope.event as { path: string }).path).toBe(
|
||||
inside.path,
|
||||
);
|
||||
});
|
||||
|
||||
it('does not reach a sibling that only shares the missing name as a prefix', async () => {
|
||||
@@ -1523,8 +1533,9 @@ describe('coalescing', () => {
|
||||
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
|
||||
|
||||
expect(sent).toHaveLength(2);
|
||||
expect(sent.map((s) => (s.envelope.event as { uid: string }).uid).sort())
|
||||
.toEqual([file.uid, second.uid].sort());
|
||||
expect(
|
||||
sent.map((s) => (s.envelope.event as { uid: string }).uid).sort(),
|
||||
).toEqual([file.uid, second.uid].sort());
|
||||
});
|
||||
|
||||
it('counts exactly what it delivered', async () => {
|
||||
@@ -1670,9 +1681,9 @@ describe('limits', () => {
|
||||
}
|
||||
}
|
||||
|
||||
await expect(subscribe(`fs:${documents.uid}`)).resolves.toMatchObject(
|
||||
{ subId: expect.any(String) },
|
||||
);
|
||||
await expect(subscribe(`fs:${documents.uid}`)).resolves.toMatchObject({
|
||||
subId: expect.any(String),
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses unsubscribes past their own per-minute budget', async () => {
|
||||
@@ -1741,9 +1752,7 @@ describe('limits', () => {
|
||||
),
|
||||
);
|
||||
|
||||
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(
|
||||
1,
|
||||
);
|
||||
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
|
||||
expect(results.filter((r) => r.status === 'rejected')).toHaveLength(
|
||||
attempts - 1,
|
||||
);
|
||||
@@ -1940,9 +1949,9 @@ describe('limits', () => {
|
||||
).toHaveLength(EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT);
|
||||
// 200 evaluated, 20 never reached: everything past the delivery cap is
|
||||
// told it missed something, itself capped.
|
||||
expect(
|
||||
sent.filter((s) => s.envelope.event.op === 'gap'),
|
||||
).toHaveLength(EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT);
|
||||
expect(sent.filter((s) => s.envelope.event.op === 'gap')).toHaveLength(
|
||||
EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT,
|
||||
);
|
||||
expect(
|
||||
sent.find((s) => s.envelope.event.op === 'gap')?.envelope.event,
|
||||
).toMatchObject({ reason: 'filter_evaluation_limit' });
|
||||
@@ -1993,8 +2002,7 @@ describe('the socket surface', () => {
|
||||
fire: (event, ...args) =>
|
||||
(
|
||||
handlers.get(event) as
|
||||
| ((...a: unknown[]) => void)
|
||||
| undefined
|
||||
((...a: unknown[]) => void) | undefined
|
||||
)?.(...args),
|
||||
};
|
||||
};
|
||||
@@ -2175,10 +2183,14 @@ it('names the delivery channel the clients listen on', () => {
|
||||
|
||||
describe('asking a kv subscription for the value', () => {
|
||||
it('is recorded on the row and reported in the view', async () => {
|
||||
const { sub } = await service.subscribe(appActorFor(OWN_APP), socketId, {
|
||||
subject: `kv:${OWN_APP}:cart`,
|
||||
includeValue: true,
|
||||
});
|
||||
const { sub } = await service.subscribe(
|
||||
appActorFor(OWN_APP),
|
||||
socketId,
|
||||
{
|
||||
subject: `kv:${OWN_APP}:cart`,
|
||||
includeValue: true,
|
||||
},
|
||||
);
|
||||
expect(sub.includeValue).toBe(true);
|
||||
|
||||
const plain = await subscribeKv(`kv:${OWN_APP}:cart`);
|
||||
@@ -2261,10 +2273,7 @@ describe('resolving a kv subject', () => {
|
||||
});
|
||||
|
||||
it('refuses a pattern the grammar cannot enumerate from a key', async () => {
|
||||
for (const subject of [
|
||||
`kv:${OWN_APP}:ca*rt`,
|
||||
`kv:${OWN_APP}:cart?`,
|
||||
])
|
||||
for (const subject of [`kv:${OWN_APP}:ca*rt`, `kv:${OWN_APP}:cart?`])
|
||||
await expect(subscribeKv(subject)).rejects.toSatisfy(
|
||||
(err: unknown) =>
|
||||
isHttpError(err) && err.legacyCode === 'invalid_kv_pattern',
|
||||
@@ -2379,8 +2388,7 @@ describe('delivering a kv change', () => {
|
||||
|
||||
const eventFor = (subId: string) =>
|
||||
sent.find((one) => one.envelope.subId === subId)?.envelope.event as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
Record<string, unknown> | undefined;
|
||||
|
||||
it('hands the value to the row that asked for it and to no other', async () => {
|
||||
vi.useFakeTimers();
|
||||
@@ -2495,7 +2503,9 @@ describe('delivering a kv change', () => {
|
||||
await dispatchKv(['cart'], { values: ['kept'] });
|
||||
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
|
||||
|
||||
const deliveries = sent.filter((one) => one.envelope.event.op === 'set');
|
||||
const deliveries = sent.filter(
|
||||
(one) => one.envelope.event.op === 'set',
|
||||
);
|
||||
const gaps = sent.filter((one) => one.envelope.event.op === 'gap');
|
||||
expect(deliveries).toHaveLength(128);
|
||||
expect(gaps).toHaveLength(1);
|
||||
@@ -2542,12 +2552,12 @@ describe('delivering a kv change', () => {
|
||||
await dispatchKv(['cart']);
|
||||
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
|
||||
|
||||
expect(sent.filter((one) => one.envelope.event.op === 'set')).toHaveLength(
|
||||
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT.limit,
|
||||
);
|
||||
expect(sent.filter((one) => one.envelope.event.op === 'gap')).toHaveLength(
|
||||
1,
|
||||
);
|
||||
expect(
|
||||
sent.filter((one) => one.envelope.event.op === 'set'),
|
||||
).toHaveLength(EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT.limit);
|
||||
expect(
|
||||
sent.filter((one) => one.envelope.event.op === 'gap'),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('uses the free KV limit when owner policy lookup fails', async () => {
|
||||
@@ -2558,7 +2568,9 @@ describe('delivering a kv change', () => {
|
||||
await dispatchKv(['cart']);
|
||||
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
|
||||
|
||||
expect(sent.filter((one) => one.envelope.event.op === 'set')).toHaveLength(
|
||||
expect(
|
||||
sent.filter((one) => one.envelope.event.op === 'set'),
|
||||
).toHaveLength(
|
||||
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT.bySubscription[
|
||||
DEFAULT_FREE_SUBSCRIPTION
|
||||
],
|
||||
@@ -2571,7 +2583,9 @@ describe('delivering a kv change', () => {
|
||||
await seedKvSubscriptions(
|
||||
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT.bySubscription[
|
||||
DEFAULT_FREE_SUBSCRIPTION
|
||||
] - 128 + 1,
|
||||
] -
|
||||
128 +
|
||||
1,
|
||||
{ includeValue: true, match: 'elsewhere', tag: 'filtered-out' },
|
||||
);
|
||||
|
||||
@@ -2593,7 +2607,9 @@ describe('delivering a kv change', () => {
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
|
||||
|
||||
const deliveries = sent.filter((one) => one.envelope.event.op === 'set');
|
||||
const deliveries = sent.filter(
|
||||
(one) => one.envelope.event.op === 'set',
|
||||
);
|
||||
expect(deliveries).toHaveLength(128);
|
||||
expect(
|
||||
deliveries.every(
|
||||
@@ -2819,8 +2835,7 @@ describe('what a kv dispatch costs', () => {
|
||||
});
|
||||
|
||||
describe('the cross-app kv gate', () => {
|
||||
const grantRead = (app: string) =>
|
||||
grants.add(`app-data:${app}:kv:read`);
|
||||
const grantRead = (app: string) => grants.add(`app-data:${app}:kv:read`);
|
||||
|
||||
const crossAppService = () =>
|
||||
buildService({
|
||||
@@ -2884,7 +2899,8 @@ describe('the cross-app kv gate', () => {
|
||||
({ service } = crossAppService());
|
||||
|
||||
await expect(subscribeKv(`kv:${OTHER_APP}:cart`)).rejects.toSatisfy(
|
||||
(err: unknown) => isHttpError(err) && err.legacyCode === 'forbidden',
|
||||
(err: unknown) =>
|
||||
isHttpError(err) && err.legacyCode === 'forbidden',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -2895,8 +2911,7 @@ describe('the cross-app kv gate', () => {
|
||||
|
||||
await expect(subscribeKv('kv:app-gone:cart')).rejects.toSatisfy(
|
||||
(err: unknown) =>
|
||||
isHttpError(err) &&
|
||||
err.legacyCode === 'subject_does_not_exist',
|
||||
isHttpError(err) && err.legacyCode === 'subject_does_not_exist',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -2910,7 +2925,8 @@ describe('the cross-app kv gate', () => {
|
||||
});
|
||||
|
||||
await expect(subscribeKv(`kv:${OTHER_APP}:cart`)).rejects.toSatisfy(
|
||||
(err: unknown) => isHttpError(err) && err.legacyCode === 'forbidden',
|
||||
(err: unknown) =>
|
||||
isHttpError(err) && err.legacyCode === 'forbidden',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -3185,9 +3201,7 @@ describe('cross-user kv handles', () => {
|
||||
expect(off.kvHandlesEnabled).toBe(false);
|
||||
expect(service.kvHandlesEnabled).toBe(true);
|
||||
|
||||
await expect(
|
||||
subscribeAsGuest(`kv:${handle}:*`, off),
|
||||
).rejects.toSatisfy(
|
||||
await expect(subscribeAsGuest(`kv:${handle}:*`, off)).rejects.toSatisfy(
|
||||
(err: unknown) =>
|
||||
isHttpError(err) &&
|
||||
err.legacyCode === 'events_kv_handles_disabled',
|
||||
@@ -3395,10 +3409,13 @@ describe('cross-user kv handles', () => {
|
||||
it.each([
|
||||
['an unknown handle', () => undefined],
|
||||
['a revoked handle', () => mintHandle({ revokedAt: 1 })],
|
||||
['a handle whose grant the caller does not hold', () => {
|
||||
mintHandle();
|
||||
grants.delete(permission);
|
||||
}],
|
||||
[
|
||||
'a handle whose grant the caller does not hold',
|
||||
() => {
|
||||
mintHandle();
|
||||
grants.delete(permission);
|
||||
},
|
||||
],
|
||||
])('answers %s as absent', async (_case, setUp) => {
|
||||
setUp();
|
||||
await expect(subscribeAsGuest(`kv:${handle}:*`)).rejects.toSatisfy(
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -19,20 +19,29 @@
|
||||
|
||||
/**
|
||||
* What deleting an app leaves behind on the events side: its durable rows,
|
||||
* their backlog and its published handlers can never deliver again — the
|
||||
* grant identity they are re-checked under no longer resolves — so
|
||||
* `settleDeletedApp` tears them down on the `app.changed` bus key rather than
|
||||
* leaving them to cost their holder an anchor slot forever. Also the hourly
|
||||
* sweep that catches whatever neither this nor `workers.destroy` reached.
|
||||
* their backlog and its published handlers can never deliver again — the grant
|
||||
* identity they are re-checked under no longer resolves — so `settleDeletedApp`
|
||||
* tears them down on the `app.changed` bus key rather than leaving them to cost
|
||||
* their holder an anchor slot forever. Also the hourly sweep that catches
|
||||
* whatever neither this nor `workers.destroy` reached.
|
||||
*/
|
||||
|
||||
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
import { testJobContext } from '../jobs/jobsTestUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
|
||||
import { EVENTS_WORKER_SESSION_NAME } from './workerRuntime.js';
|
||||
import { appHandlerScope } from '../../stores/events/EventHandlerStore.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
|
||||
@@ -126,10 +135,13 @@ const emitDeleted = async (appUid: string): Promise<void> => {
|
||||
const publish = (
|
||||
token: string,
|
||||
body: { appUid: string; name: string; source: string },
|
||||
): Promise<ApiResponse> => call('POST', '/events/handlers/publish', token, body);
|
||||
): Promise<ApiResponse> =>
|
||||
call('POST', '/events/handlers/publish', token, body);
|
||||
|
||||
/** A durable `single` subscription bound to a published handler, with the
|
||||
* grants a background delivery needs. Returns the subscribed subId. */
|
||||
/**
|
||||
* A durable `single` subscription bound to a published handler, with the grants
|
||||
* a background delivery needs. Returns the subscribed subId.
|
||||
*/
|
||||
const subscribeBackgroundHandler = async (
|
||||
app: { uid: string; token: string },
|
||||
handlerName: string,
|
||||
@@ -163,6 +175,87 @@ const subscribeBackgroundHandler = async (
|
||||
return subscribed.body.subId as string;
|
||||
};
|
||||
|
||||
describe('user scopes of a deleted app or account', () => {
|
||||
const publishUserHandler = (
|
||||
appUid: string,
|
||||
holder: number,
|
||||
name = 'mine',
|
||||
) =>
|
||||
env.server.stores.eventHandler.publish({
|
||||
scope: { appUid, holder },
|
||||
name,
|
||||
source: SOURCE,
|
||||
broadcast: false,
|
||||
});
|
||||
const userHandlers = (appUid: string, holder: number) =>
|
||||
env.server.stores.eventHandler.listForApp({ appUid, holder });
|
||||
|
||||
let destroyed: Array<{ appUid: string; scope: string; userId: number }>;
|
||||
beforeAll(() => {
|
||||
env.server.clients.event.on('events.worker.destroy', (_key, event) => {
|
||||
destroyed?.push({
|
||||
appUid: event.appUid,
|
||||
scope: event.scope,
|
||||
userId: event.actor.user.id,
|
||||
});
|
||||
});
|
||||
});
|
||||
beforeEach(async () => {
|
||||
destroyed = [];
|
||||
await env.server.clients.db.write(
|
||||
'DELETE FROM `event_user_handlers`',
|
||||
[],
|
||||
);
|
||||
});
|
||||
|
||||
it('removes every holder`s scope with the app, announcing each as the holder', async () => {
|
||||
const app = await makeApp(userId);
|
||||
await publishUserHandler(app.uid, userId);
|
||||
await publishUserHandler(app.uid, otherUserId);
|
||||
await publishUserHandler(app.uid, otherUserId, 'second');
|
||||
const untouched = await makeApp(userId);
|
||||
await publishUserHandler(untouched.uid, userId);
|
||||
|
||||
await emitDeleted(app.uid);
|
||||
|
||||
expect(await userHandlers(app.uid, userId)).toEqual([]);
|
||||
expect(await userHandlers(app.uid, otherUserId)).toEqual([]);
|
||||
expect(
|
||||
destroyed
|
||||
.filter((event) => event.scope === 'user')
|
||||
.map((event) => [event.appUid, event.userId])
|
||||
.sort(),
|
||||
).toEqual(
|
||||
[
|
||||
[app.uid, userId],
|
||||
[app.uid, otherUserId],
|
||||
].sort(),
|
||||
);
|
||||
// Another app's scope is not touched.
|
||||
expect(await userHandlers(untouched.uid, userId)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('removes a deleted account`s scopes in every app, and no one else`s', async () => {
|
||||
const first = await makeApp(userId);
|
||||
const second = await makeApp(userId);
|
||||
for (const app of [first, second]) {
|
||||
await publishUserHandler(app.uid, otherUserId);
|
||||
await publishUserHandler(app.uid, userId);
|
||||
}
|
||||
|
||||
await env.server.clients.event.emitAndWait(
|
||||
'user.delete',
|
||||
{ user_id: otherUserId, appUids: [] },
|
||||
{},
|
||||
);
|
||||
|
||||
for (const app of [first, second]) {
|
||||
expect(await userHandlers(app.uid, otherUserId)).toEqual([]);
|
||||
expect(await userHandlers(app.uid, userId)).toHaveLength(1);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('an app deleted while it has events state', () => {
|
||||
it('clears the rows, the backlog and the handlers when the app is deleted', async () => {
|
||||
const app = await makeApp(userId);
|
||||
@@ -187,15 +280,17 @@ describe('an app deleted while it has events state', () => {
|
||||
|
||||
await emitDeleted(app.uid);
|
||||
|
||||
const listed = await env.server.stores.durableSubscription.listForHolder(
|
||||
userId,
|
||||
{ appUid: app.uid },
|
||||
);
|
||||
const listed =
|
||||
await env.server.stores.durableSubscription.listForHolder(userId, {
|
||||
appUid: app.uid,
|
||||
});
|
||||
expect(listed.items).toEqual([]);
|
||||
expect(await env.server.stores.pendingDelivery.depth(subId)).toBe(0);
|
||||
expect(await env.server.stores.eventHandler.listForApp(app.uid)).toEqual(
|
||||
[],
|
||||
);
|
||||
expect(
|
||||
await env.server.stores.eventHandler.listForApp(
|
||||
appHandlerScope(app.uid),
|
||||
),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("clears the rows and handlers of a deleted account's apps, and the account's subscription keys", async () => {
|
||||
@@ -218,14 +313,16 @@ describe('an app deleted while it has events state', () => {
|
||||
{},
|
||||
);
|
||||
|
||||
const listed = await env.server.stores.durableSubscription.listForHolder(
|
||||
userId,
|
||||
{ appUid: app.uid },
|
||||
);
|
||||
const listed =
|
||||
await env.server.stores.durableSubscription.listForHolder(userId, {
|
||||
appUid: app.uid,
|
||||
});
|
||||
expect(listed.items).toEqual([]);
|
||||
expect(await env.server.stores.eventHandler.listForApp(app.uid)).toEqual(
|
||||
[],
|
||||
);
|
||||
expect(
|
||||
await env.server.stores.eventHandler.listForApp(
|
||||
appHandlerScope(app.uid),
|
||||
),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
await env.server.stores.eventSubscription.isRegionWarm(goneUserId),
|
||||
).toBe(false);
|
||||
@@ -260,12 +357,10 @@ describe('an app deleted while it has events state', () => {
|
||||
expiresAt: null,
|
||||
});
|
||||
|
||||
const { actor: ownerActor } = await env.server.services.auth.authenticate(
|
||||
env.users.user.token,
|
||||
);
|
||||
const { actor: otherActor } = await env.server.services.auth.authenticate(
|
||||
env.users.other.token,
|
||||
);
|
||||
const { actor: ownerActor } =
|
||||
await env.server.services.auth.authenticate(env.users.user.token);
|
||||
const { actor: otherActor } =
|
||||
await env.server.services.auth.authenticate(env.users.other.token);
|
||||
const ownerToken = await env.server.services.auth.createWorkerAppToken(
|
||||
ownerActor!,
|
||||
app.uid,
|
||||
|
||||
@@ -616,6 +616,210 @@ describe('reading again from a saved cursor', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('starting from the end', () => {
|
||||
const SUBJECT = 'notif:app-user';
|
||||
|
||||
const message = async (title: string, inApp = appUid): Promise<string> =>
|
||||
(
|
||||
await env.server.stores.notification.create({
|
||||
userId,
|
||||
value: { type: 'app.message', title },
|
||||
type: 'app.message',
|
||||
audience: 'app-user',
|
||||
appUid: inApp,
|
||||
group: `group-${uuidv4()}`,
|
||||
})
|
||||
).uid as string;
|
||||
|
||||
const fromEnd = async (token = appToken): Promise<string> => {
|
||||
const end = await fetchPage(token, { subject: SUBJECT, from: 'end' });
|
||||
expect(end.status).toBe(200);
|
||||
expect(end.body.items).toEqual([]);
|
||||
expect(end.body.cursor).toBeTruthy();
|
||||
return end.body.cursor!;
|
||||
};
|
||||
|
||||
it('answers a cursor on an empty slice that returns what comes after', async () => {
|
||||
await clearMailboxes();
|
||||
const cursor = await fromEnd();
|
||||
|
||||
const later = await message('later');
|
||||
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
|
||||
expect(uidsOf(next)).toEqual([later]);
|
||||
});
|
||||
|
||||
it('skips what is there and returns only newer rows', async () => {
|
||||
await clearMailboxes();
|
||||
for (let i = 0; i < 3; i++) await message(`before ${i}`);
|
||||
const cursor = await fromEnd();
|
||||
|
||||
const quiet = await fetchPage(appToken, { subject: SUBJECT, cursor });
|
||||
expect(quiet.body.items).toEqual([]);
|
||||
expect(quiet.body.cursor).toBeUndefined();
|
||||
|
||||
const later = [await message('after 0'), await message('after 1')];
|
||||
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
|
||||
expect(uidsOf(next)).toEqual(later);
|
||||
});
|
||||
|
||||
it('returns a row replaced after the cursor was taken', async () => {
|
||||
await clearMailboxes();
|
||||
const old = await message('old');
|
||||
await message('older still unchanged');
|
||||
const cursor = await fromEnd();
|
||||
|
||||
await env.server.services.notification.notifyUpdate(
|
||||
old,
|
||||
userId,
|
||||
{ title: 'replaced' },
|
||||
{ type: 'app.message', replace: true },
|
||||
);
|
||||
|
||||
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
|
||||
expect(uidsOf(next)).toEqual([old]);
|
||||
expect(next.body.items![0]).toMatchObject({
|
||||
replaced: true,
|
||||
notification: { title: 'replaced' },
|
||||
});
|
||||
});
|
||||
|
||||
it('returns a replacement stamped by a clock behind the server`s', async () => {
|
||||
await clearMailboxes();
|
||||
const behind = Date.now() - 60_000;
|
||||
const [early, late] = [await message('early'), await message('late')];
|
||||
const replace = (uid: string, title: string, replacedAt: number) =>
|
||||
env.server.stores.notification.updateValue(
|
||||
uid,
|
||||
userId,
|
||||
{ type: 'app.message', title },
|
||||
{ onlyUnseen: true, replacedAt },
|
||||
);
|
||||
await replace(early, 'replaced before', behind - 1_000);
|
||||
const cursor = await fromEnd();
|
||||
|
||||
await replace(late, 'replaced after', behind);
|
||||
|
||||
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
|
||||
expect(uidsOf(next)).toEqual([late]);
|
||||
expect(next.body.items![0]).toMatchObject({
|
||||
notification: { title: 'replaced after' },
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps to the slice the subject names', async () => {
|
||||
await clearMailboxes();
|
||||
await message('own before');
|
||||
const cursor = await fromEnd();
|
||||
|
||||
await message('other app', otherAppUid);
|
||||
await seed(userId, { title: 'account row' });
|
||||
const own = await message('own after');
|
||||
|
||||
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
|
||||
expect(uidsOf(next)).toEqual([own]);
|
||||
});
|
||||
|
||||
it('answers no cursor for a slice the caller cannot see', async () => {
|
||||
await clearMailboxes();
|
||||
await message('other app', otherAppUid);
|
||||
|
||||
for (const [token, subject] of [
|
||||
[appToken, `notif:${otherAppUid}:app-user`],
|
||||
[appToken, `notif:${userUuid}:account`],
|
||||
[appReadUrlToken, SUBJECT],
|
||||
] as const) {
|
||||
const page = await fetchPage(token, { subject, from: 'end' });
|
||||
expect(page.status).toBe(200);
|
||||
expect(page.body.items).toEqual([]);
|
||||
expect(page.body.cursor).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('reads from the cursor when one is given', async () => {
|
||||
await clearMailboxes();
|
||||
const uids = [await message('m0'), await message('m1')];
|
||||
const first = await fetchPage(appToken, { subject: SUBJECT, limit: 1 });
|
||||
|
||||
const page = await fetchPage(appToken, {
|
||||
subject: SUBJECT,
|
||||
cursor: first.body.cursor,
|
||||
from: 'end',
|
||||
});
|
||||
expect(uidsOf(page)).toEqual([uids[1]]);
|
||||
});
|
||||
|
||||
it('rejects a `from` that is neither start nor end', async () => {
|
||||
const page = await fetchPage(appToken, {
|
||||
subject: SUBJECT,
|
||||
from: 'middle',
|
||||
});
|
||||
expect(page.status).toBe(400);
|
||||
|
||||
const start = await fetchPage(appToken, {
|
||||
subject: SUBJECT,
|
||||
from: 'start',
|
||||
});
|
||||
expect(start.status).toBe(200);
|
||||
});
|
||||
|
||||
it('starts the SDK feed without catch-up in one read', async () => {
|
||||
await clearMailboxes();
|
||||
for (let i = 0; i < 3; i++) await message(`before ${i}`);
|
||||
|
||||
const appActor = (await env.server.services.auth.authenticate(appToken))
|
||||
.actor!;
|
||||
const reads: Array<Record<string, unknown>> = [];
|
||||
let receive: (delivery: { event: unknown }) => void = () => {};
|
||||
const titles: string[] = [];
|
||||
const feed = new NotificationFeed(
|
||||
{
|
||||
authToken: appToken,
|
||||
events: {
|
||||
onLocal: async (
|
||||
_subject: string,
|
||||
handler: typeof receive,
|
||||
) => {
|
||||
receive = handler;
|
||||
return { off: async () => {} };
|
||||
},
|
||||
fetch: async (options: Record<string, unknown>) => {
|
||||
reads.push(options);
|
||||
return env.server.services.events.fetchMissed(
|
||||
appActor,
|
||||
options,
|
||||
);
|
||||
},
|
||||
},
|
||||
} as never,
|
||||
(notification: { title: string | null }) => {
|
||||
titles.push(String(notification.title));
|
||||
},
|
||||
{ catchUp: false },
|
||||
);
|
||||
|
||||
try {
|
||||
await feed.start();
|
||||
expect(reads).toEqual([
|
||||
{ subject: SUBJECT, limit: 200, from: 'end' },
|
||||
]);
|
||||
|
||||
await message('after start');
|
||||
receive({ event: { op: 'gap', reason: 'reconnect' } });
|
||||
await vi.waitFor(() => expect(reads).toHaveLength(2));
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
} finally {
|
||||
await feed.off();
|
||||
}
|
||||
|
||||
expect(reads[1]).toEqual({
|
||||
subject: SUBJECT,
|
||||
limit: 200,
|
||||
cursor: expect.any(String),
|
||||
});
|
||||
expect(titles).toEqual(['after start']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('cross-user isolation', () => {
|
||||
it('returns nothing of another account, whatever the subject names', async () => {
|
||||
await clearMailboxes();
|
||||
|
||||
@@ -217,6 +217,97 @@ describe('the addressed peer queue', () => {
|
||||
).toEqual(['b']);
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
'waits for an active send and ships its tail after failure %s',
|
||||
async (failFirst) => {
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
let releaseTail!: () => void;
|
||||
const tailHeld = new Promise<void>((resolve) => {
|
||||
releaseTail = resolve;
|
||||
});
|
||||
const sent: ForwardItem[][] = [];
|
||||
const { queue } = makeQueue({
|
||||
maxItems: 1,
|
||||
flushMs: 60_000,
|
||||
send: async (_peerId, items) => {
|
||||
sent.push(items);
|
||||
if (sent.length === 1) {
|
||||
await held;
|
||||
if (failFirst) throw new Error('peer unavailable');
|
||||
} else {
|
||||
await tailHeld;
|
||||
}
|
||||
},
|
||||
});
|
||||
queue.push('east', delivery('first'));
|
||||
await vi.waitFor(() => expect(sent).toHaveLength(1));
|
||||
queue.push('east', delivery('tail'));
|
||||
let finished = false;
|
||||
const draining = queue.flushAll().then(() => {
|
||||
finished = true;
|
||||
});
|
||||
try {
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(finished).toBe(false);
|
||||
release();
|
||||
await vi.waitFor(() => expect(sent).toHaveLength(2));
|
||||
expect(finished).toBe(false);
|
||||
} finally {
|
||||
release();
|
||||
releaseTail();
|
||||
await draining;
|
||||
queue.stop();
|
||||
}
|
||||
expect(sent).toEqual([[delivery('first')], [delivery('tail')]]);
|
||||
},
|
||||
);
|
||||
|
||||
it('releases the peer after a send throws synchronously', async () => {
|
||||
const send = vi.fn(() => {
|
||||
throw new Error('send failed');
|
||||
});
|
||||
const { queue } = makeQueue({ maxItems: 1, send });
|
||||
queue.push('east', delivery('first'));
|
||||
await queue.flushAll();
|
||||
queue.push('east', delivery('second'));
|
||||
await queue.flushAll();
|
||||
expect(send).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('bounds a final drain while a send is hung', async () => {
|
||||
vi.useFakeTimers();
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
const send = vi.fn(async () => {
|
||||
await held;
|
||||
});
|
||||
const { queue } = makeQueue({ maxItems: 1, send });
|
||||
try {
|
||||
queue.push('east', delivery('first'));
|
||||
queue.push('east', delivery('tail'));
|
||||
let finished = false;
|
||||
const draining = queue.flushAll().then(() => {
|
||||
finished = true;
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(4_999);
|
||||
expect(finished).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
await draining;
|
||||
queue.stop();
|
||||
release();
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
release();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
it('sends nothing more once it is stopped', async () => {
|
||||
const { queue, sent } = makeQueue();
|
||||
queue.stop();
|
||||
|
||||
@@ -18,6 +18,8 @@
|
||||
*/
|
||||
|
||||
import type { EventKey, KvOp } from '../../clients/event/types.js';
|
||||
import { settleWithin } from '../../util/concurrency.js';
|
||||
import { FINAL_FLUSH_TIMEOUT_MS } from '../../util/gracefulShutdown.js';
|
||||
import type { DeliverableEvent } from './registry.js';
|
||||
|
||||
/**
|
||||
@@ -86,7 +88,7 @@ export interface ForwardWatch {
|
||||
/** One committed change, replayed against another region's session rows. */
|
||||
export interface ForwardEvent {
|
||||
kind: 'event';
|
||||
family: 'fs' | 'kv';
|
||||
family: 'fs' | 'kv' | 'notif';
|
||||
ownerUserId: number;
|
||||
/** Who caused it, so `self` is right on the far side. */
|
||||
actingUserId?: number;
|
||||
@@ -108,6 +110,17 @@ export interface ForwardEvent {
|
||||
ancestors: Array<{ uid: string; path: string }>;
|
||||
};
|
||||
};
|
||||
notif?: {
|
||||
userUuid: string;
|
||||
uid: string;
|
||||
type: string;
|
||||
audience: string;
|
||||
appUid: string | null;
|
||||
value: Record<string, unknown>;
|
||||
dismissed?: true;
|
||||
replaced?: true;
|
||||
handlerDepth?: number;
|
||||
};
|
||||
kv?: {
|
||||
userUuid: string;
|
||||
appUid: string;
|
||||
@@ -204,7 +217,7 @@ interface PeerQueue {
|
||||
items: ForwardItem[];
|
||||
bytes: number;
|
||||
timer: ReturnType<typeof setTimeout> | null;
|
||||
flushing: boolean;
|
||||
flushing: Promise<void> | null;
|
||||
}
|
||||
|
||||
export class PeerForwardQueue {
|
||||
@@ -272,19 +285,27 @@ export class PeerForwardQueue {
|
||||
}
|
||||
|
||||
/** Ship whatever is held for one peer. */
|
||||
async flush(peerId: string): Promise<void> {
|
||||
flush(peerId: string): Promise<void> {
|
||||
const queue = this.#peers.get(peerId);
|
||||
if (!queue || queue.flushing || queue.items.length === 0) return;
|
||||
if (!queue) return Promise.resolve();
|
||||
if (queue.flushing) return queue.flushing;
|
||||
if (queue.items.length === 0) return Promise.resolve();
|
||||
if (queue.timer) {
|
||||
clearTimeout(queue.timer);
|
||||
queue.timer = null;
|
||||
}
|
||||
|
||||
queue.flushing = true;
|
||||
queue.flushing = this.#send(peerId, queue);
|
||||
return queue.flushing;
|
||||
}
|
||||
|
||||
async #send(peerId: string, queue: PeerQueue): Promise<void> {
|
||||
const batch = queue.items;
|
||||
queue.items = [];
|
||||
queue.bytes = 0;
|
||||
try {
|
||||
// Register the in-flight promise before a send can throw synchronously.
|
||||
await Promise.resolve();
|
||||
await this.#options.send(peerId, batch);
|
||||
} catch (err) {
|
||||
// A lost batch is a lost at-most-once delivery, and a `single`
|
||||
@@ -292,7 +313,7 @@ export class PeerForwardQueue {
|
||||
// hand the client an event whose moment has passed.
|
||||
console.warn(`[events] forward to ${peerId} failed`, err);
|
||||
} finally {
|
||||
queue.flushing = false;
|
||||
queue.flushing = null;
|
||||
// Anything that arrived mid-flush gets the next window.
|
||||
if (queue.items.length > 0 && !queue.timer && !this.#stopped) {
|
||||
queue.timer = setTimeout(() => {
|
||||
@@ -305,7 +326,16 @@ export class PeerForwardQueue {
|
||||
}
|
||||
|
||||
async flushAll(): Promise<void> {
|
||||
await Promise.all([...this.#peers.keys()].map((id) => this.flush(id)));
|
||||
const deadline = Date.now() + FINAL_FLUSH_TIMEOUT_MS;
|
||||
const work = Promise.allSettled(
|
||||
[...this.#peers.entries()].map(async ([id, queue]) => {
|
||||
await queue.flushing;
|
||||
if (Date.now() < deadline) await this.flush(id);
|
||||
}),
|
||||
);
|
||||
if (!(await settleWithin(work, FINAL_FLUSH_TIMEOUT_MS))) {
|
||||
console.warn('[events] final flush timed out');
|
||||
}
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
@@ -323,7 +353,7 @@ export class PeerForwardQueue {
|
||||
items: [],
|
||||
bytes: 0,
|
||||
timer: null,
|
||||
flushing: false,
|
||||
flushing: null,
|
||||
};
|
||||
this.#peers.set(peerId, queue);
|
||||
return queue;
|
||||
@@ -349,7 +379,7 @@ const isGapMarker = (item: ForwardItem): boolean =>
|
||||
*/
|
||||
const shed = (queue: PeerQueue, count: number): ForwardItem[] => {
|
||||
const dropped: ForwardItem[] = [];
|
||||
for (let i = 0; i < queue.items.length && dropped.length < count;) {
|
||||
for (let i = 0; i < queue.items.length && dropped.length < count; ) {
|
||||
if (isGapMarker(queue.items[i])) {
|
||||
i++;
|
||||
continue;
|
||||
@@ -372,7 +402,7 @@ const shed = (queue: PeerQueue, count: number): ForwardItem[] => {
|
||||
const shedBytes = (queue: PeerQueue, maxBytesHeld: number): ForwardItem[] => {
|
||||
const dropped: ForwardItem[] = [];
|
||||
let remaining = queue.bytes;
|
||||
for (let i = 0; i < queue.items.length && remaining > maxBytesHeld;) {
|
||||
for (let i = 0; i < queue.items.length && remaining > maxBytesHeld; ) {
|
||||
if (isGapMarker(queue.items[i])) {
|
||||
i++;
|
||||
continue;
|
||||
|
||||
@@ -402,7 +402,7 @@ const makeRegion = (
|
||||
fields: { code: 'subject_does_not_exist' },
|
||||
}),
|
||||
},
|
||||
notification: { notify: vi.fn() },
|
||||
notification: { notify: vi.fn(), deliverOverSocket: vi.fn() },
|
||||
metering: {
|
||||
bufferIncrementUsages: (
|
||||
actor: Actor,
|
||||
@@ -2401,3 +2401,68 @@ describe('a durable row created in another region', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe('notification dismissals across connected pages', () => {
|
||||
const config = { events: { enabled: true, notificationsFoldIn: true } } as Partial<IConfig>;
|
||||
const appUid = 'notification-app';
|
||||
const input = (uid = 'one') => ({
|
||||
userId, userUuid: `user-${userId}`, uid,
|
||||
type: 'app.message', audience: 'app-user', appUid,
|
||||
value: {}, createdAt: Date.now(), dismissed: true as const,
|
||||
});
|
||||
|
||||
it('reaches local and remote session pages without content or a forwarding loop', async () => {
|
||||
const west = makeRegion('west', ['east'], config);
|
||||
const east = makeRegion('east', ['west'], config);
|
||||
for (const [region, socket] of [[west, 'local'], [east, 'remote']] as const) {
|
||||
await region.events.subscribe(actorFor(appUid), socket, { subject: 'notif:app-user' });
|
||||
}
|
||||
await posted(east);
|
||||
await quiet(100);
|
||||
await west.events.dispatchNotif({ ...input(), value: { title: 'private' } });
|
||||
await arrived(west);
|
||||
await arrived(east);
|
||||
for (const region of [west, east]) {
|
||||
expect(region.sent).toHaveLength(1);
|
||||
expect(region.sent[0].event).toMatchObject({ op: 'dismiss', uid: 'one', appUid });
|
||||
expect(region.sent[0].event).not.toHaveProperty('notification');
|
||||
}
|
||||
expect(east.sent[0].event.id).toBe(west.sent[0].event.id);
|
||||
const forwarded = west.posts.flatMap((p) => p.batch.items)
|
||||
.find((item): item is ForwardEvent => item.kind === 'event');
|
||||
expect(forwarded?.notif?.value).toEqual({});
|
||||
expect(east.posts.flatMap((p) => p.batch.items).filter((i) => i.kind === 'event')).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps separate dismissed ids in the same coalescing window', async () => {
|
||||
const region = makeRegion('west', [], config);
|
||||
await region.events.subscribe(actorFor(appUid), 'local', { subject: 'notif:app-user' });
|
||||
await region.events.dispatchNotif(input('one'));
|
||||
await region.events.dispatchNotif(input('two'));
|
||||
await arrived(region, 2);
|
||||
expect(region.sent.map((item) => 'uid' in item.event && item.event.uid)).toEqual(['one', 'two']);
|
||||
});
|
||||
|
||||
it('does not deliver another app, account audience or another user to an app page', async () => {
|
||||
const region = makeRegion('west', [], config);
|
||||
await region.events.subscribe(actorFor(appUid), 'local', { subject: 'notif:app-user' });
|
||||
await region.events.dispatchNotif({ ...input(), appUid: 'other-app' });
|
||||
await region.events.dispatchNotif({ ...input(), audience: 'account' });
|
||||
await region.events.dispatchNotif({ ...input(), userId: userId + 1, userUuid: 'other-user' });
|
||||
await quiet(350);
|
||||
expect(region.sent).toEqual([]);
|
||||
});
|
||||
|
||||
it('also forwards postings through the same session path', async () => {
|
||||
const west = makeRegion('west', ['east'], config);
|
||||
const east = makeRegion('east', ['west'], config);
|
||||
await east.events.subscribe(actorFor(appUid), 'remote', { subject: 'notif:app-user' });
|
||||
await posted(east);
|
||||
await quiet(100);
|
||||
const { dismissed, ...postedInput } = input();
|
||||
await west.events.dispatchNotif({ ...postedInput, value: { title: 'Hello' } });
|
||||
await arrived(east);
|
||||
expect(east.sent[0].event).toMatchObject({ op: 'post', uid: 'one', notification: { title: 'Hello' } });
|
||||
});
|
||||
});
|
||||
@@ -45,7 +45,10 @@ import {
|
||||
EVENTS_WORKER_SOURCE_MAX_BYTES,
|
||||
} from '../../controllers/events/limits.js';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
import { hashContent } from '../../stores/events/EventHandlerStore.js';
|
||||
import {
|
||||
appHandlerScope,
|
||||
hashContent,
|
||||
} from '../../stores/events/EventHandlerStore.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
|
||||
import type { DurableSubscriptionView } from './EventsService.js';
|
||||
@@ -532,7 +535,7 @@ describe('what a publish reads', () => {
|
||||
if (sql.includes(BY_NAME_SQL) && !landed) {
|
||||
landed = true;
|
||||
await env.server.stores.eventHandler.publish({
|
||||
appUid,
|
||||
scope: appHandlerScope(appUid),
|
||||
name: 'ingestUpload',
|
||||
source: NEXT_SOURCE,
|
||||
replace: true,
|
||||
@@ -590,7 +593,7 @@ describe('what a publish reads', () => {
|
||||
pread.mockRestore();
|
||||
}
|
||||
const standing = await env.server.stores.eventHandler.getByName(
|
||||
appUid,
|
||||
appHandlerScope(appUid),
|
||||
'ingestUpload',
|
||||
);
|
||||
expect(standing?.sourceHash).toBe(hashContent(THIRD_SOURCE));
|
||||
@@ -603,7 +606,7 @@ describe('what a publish reads', () => {
|
||||
if (sql.includes(BY_NAME_SQL) && !beaten) {
|
||||
beaten = true;
|
||||
await env.server.stores.eventHandler.publish({
|
||||
appUid,
|
||||
scope: appHandlerScope(appUid),
|
||||
name: 'ingestUpload',
|
||||
source: THIRD_SOURCE,
|
||||
});
|
||||
@@ -659,7 +662,7 @@ describe('what a publish reads', () => {
|
||||
pread.mockRestore();
|
||||
}
|
||||
await expect(
|
||||
env.server.stores.eventHandler.countForApp(appUid),
|
||||
env.server.stores.eventHandler.countForApp(appHandlerScope(appUid)),
|
||||
).resolves.toBe(0);
|
||||
});
|
||||
|
||||
|
||||
@@ -93,24 +93,25 @@ export interface ProjectedKvEvent extends ProjectedEventBase {
|
||||
/**
|
||||
* What a notification subscriber receives.
|
||||
*
|
||||
* `id` is the row's own uid rather than a per-dispatch uuid: a client that
|
||||
* fetched a missed notification and then received the live copy has to be able
|
||||
* to tell they are the same thing, and the mailbox uid is the only identity
|
||||
* both halves share. `uid` carries it under the name the mailbox verbs
|
||||
* (`mark-ack`, `mark-shown`) take.
|
||||
* For postings, `id` is the row's own uid: a client that fetched a missed
|
||||
* notification and then received the live copy has to be able to tell they are
|
||||
* the same thing, and the mailbox uid is the only identity both halves share.
|
||||
* `uid` carries it under the name the mailbox verbs (`mark-ack`, `mark-shown`)
|
||||
* take.
|
||||
*
|
||||
* `notification` is the payload the caller sent, which is exactly what the
|
||||
* socket adapter puts on `notif.message` — a subscriber reading this channel
|
||||
* sees no less than the desktop does, and no internal row either.
|
||||
* sees no less than the desktop does, and no internal row either. Dismissals
|
||||
* use a fresh dispatch id and omit the payload.
|
||||
*/
|
||||
export interface ProjectedNotifEvent extends ProjectedEventBase {
|
||||
op: 'post';
|
||||
op: 'post' | 'dismiss';
|
||||
uid: string;
|
||||
type: string;
|
||||
audience: NotificationAudience;
|
||||
/** App the notification is about; `null` for platform notifications. */
|
||||
appUid: string | null;
|
||||
notification: Record<string, unknown>;
|
||||
notification?: Record<string, unknown>;
|
||||
/**
|
||||
* Present when the notification replaced an earlier one under the same uid.
|
||||
* `ts` is then the replacement time, the same on live and fetched copies,
|
||||
@@ -200,6 +201,7 @@ export interface KvEventContext extends EventContextBase {
|
||||
* recipient's uuid because the anchor token is the mailbox.
|
||||
*/
|
||||
export interface NotifEventContext extends EventContextBase {
|
||||
dismissed?: true;
|
||||
userId: number;
|
||||
userUuid: string;
|
||||
uid: string;
|
||||
@@ -346,14 +348,14 @@ const notifMatchTarget = (event: NotifEventContext): string =>
|
||||
const notifMatchScope = (_anchorPath: string, slice: string): string => slice;
|
||||
|
||||
const notifProject = (delivery: NotifDeliveryContext): ProjectedNotifEvent => ({
|
||||
id: delivery.uid,
|
||||
id: delivery.dismissed ? delivery.id : delivery.uid,
|
||||
subject: `notif:${delivery.appUid ?? delivery.userUuid}:${delivery.audience}`,
|
||||
op: 'post',
|
||||
op: delivery.dismissed ? 'dismiss' : 'post',
|
||||
uid: delivery.uid,
|
||||
type: delivery.type,
|
||||
audience: delivery.audience,
|
||||
appUid: delivery.appUid,
|
||||
notification: delivery.notification,
|
||||
...(delivery.dismissed ? {} : { notification: delivery.notification }),
|
||||
...(delivery.replaced ? { replaced: true as const } : {}),
|
||||
self: delivery.self,
|
||||
ts: delivery.ts,
|
||||
@@ -464,7 +466,7 @@ export const PUBLIC_SUBJECTS = [
|
||||
{
|
||||
family: 'notif',
|
||||
subject: 'notif:*',
|
||||
internal: ['notif.created'],
|
||||
internal: ['notif.created', 'notif.acknowledged'],
|
||||
tokens: notifTokens,
|
||||
matchOn: notifMatchTarget,
|
||||
matchSeparator: NOTIF_MATCH_SEPARATOR,
|
||||
|
||||
@@ -24,6 +24,7 @@ import {
|
||||
HANDLER_NAME_MAX_LENGTH,
|
||||
hashContent,
|
||||
} from '../../stores/events/EventHandlerStore.js';
|
||||
import type { HandlerScopeKind } from '../../stores/events/types.js';
|
||||
|
||||
/** Request and response pieces the subscription and schedule verbs share. */
|
||||
|
||||
@@ -53,6 +54,27 @@ export const parseHandlerHash = (value: unknown): string | null => {
|
||||
return value;
|
||||
};
|
||||
|
||||
/** Source of an inline handler; the store enforces the size and content rules. */
|
||||
export const parseHandlerSource = (value: unknown): string | null => {
|
||||
if (value === undefined || value === null) return null;
|
||||
if (typeof value !== 'string' || value.trim().length === 0)
|
||||
throw badRequest(
|
||||
'handlerSource must be a non-empty string',
|
||||
'bad_request',
|
||||
);
|
||||
return value;
|
||||
};
|
||||
|
||||
/** Which handler set a request is about; absent means the app's. */
|
||||
export const parseHandlerScopeKind = (
|
||||
value: unknown,
|
||||
field: string,
|
||||
): HandlerScopeKind => {
|
||||
if (value === undefined || value === null || value === 'app') return 'app';
|
||||
if (value === 'user') return 'user';
|
||||
throw badRequest(`${field} must be 'app' or 'user'`, 'bad_request');
|
||||
};
|
||||
|
||||
/** Stored as JSON text; the byte cap is the store's to enforce. */
|
||||
export const parseContext = (value: unknown): string | null => {
|
||||
if (value === undefined || value === null) return null;
|
||||
|
||||
@@ -27,7 +27,15 @@
|
||||
* out of service so a revoked subscription stops costing anything at all.
|
||||
*/
|
||||
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
onTestFinished,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { EVENTS_COALESCE_WINDOW_MS } from '../../controllers/events/limits.js';
|
||||
import { SUSPENDED_ROW_TTL_DAYS } from '../../controllers/events/limits.js';
|
||||
@@ -43,6 +51,7 @@ import type { AclMode } from '../acl/ACLService.js';
|
||||
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
|
||||
import type { DeliveryEnvelope } from './EventsService.js';
|
||||
import { fsAnchorToken } from './subjects.js';
|
||||
import { appHandlerScope } from '../../stores/events/EventHandlerStore.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
const TABLE = 'event_subscriptions';
|
||||
@@ -206,7 +215,7 @@ const makeApp = async (path: string): Promise<Actor> => {
|
||||
EVENTS_BACKGROUND_PERMISSION,
|
||||
);
|
||||
await env.server.stores.eventHandler.publish({
|
||||
appUid: uid,
|
||||
scope: appHandlerScope(uid),
|
||||
name: 'onChange',
|
||||
source: 'async () => {}',
|
||||
});
|
||||
@@ -950,3 +959,165 @@ describe('what a revoked grant settles', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('a holder`s user scope when consent is withdrawn', () => {
|
||||
const scopeOf = (appUid: string, holder = owner.id) => ({
|
||||
appUid,
|
||||
holder,
|
||||
});
|
||||
const putUserHandler = async (appUid: string, holder = owner.id) => {
|
||||
await env.server.stores.eventHandler.publish({
|
||||
scope: scopeOf(appUid, holder),
|
||||
name: 'mine',
|
||||
source: 'async () => {}',
|
||||
broadcast: false,
|
||||
});
|
||||
};
|
||||
const held = (appUid: string, holder = owner.id) =>
|
||||
env.server.stores.eventHandler.listForApp(scopeOf(appUid, holder));
|
||||
|
||||
let destroys: Array<{ appUid: string; scope: string; userId: number }>;
|
||||
beforeAll(() => {
|
||||
env.server.clients.event.on('events.worker.destroy', (_key, event) => {
|
||||
destroys?.push({
|
||||
appUid: event.appUid,
|
||||
scope: event.scope,
|
||||
userId: event.actor.user.id,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it('drops the scope when background consent is revoked, announcing it as the holder', async () => {
|
||||
await clearRows();
|
||||
destroys = [];
|
||||
const path = await folder(`/${owner.username}/scope-consent`);
|
||||
const app = (await makeApp(path)).app!;
|
||||
await putUserHandler(app.uid);
|
||||
|
||||
await env.server.services.permission.revokeUserAppPermission(
|
||||
owner.actor,
|
||||
app.uid,
|
||||
EVENTS_BACKGROUND_PERMISSION,
|
||||
);
|
||||
|
||||
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
|
||||
timeout: 5_000,
|
||||
interval: 25,
|
||||
});
|
||||
expect(destroys.filter((event) => event.scope === 'user')).toEqual([
|
||||
{ appUid: app.uid, scope: 'user', userId: owner.id },
|
||||
]);
|
||||
});
|
||||
|
||||
it('drops the scope when the whole app is revoked', async () => {
|
||||
await clearRows();
|
||||
const path = await folder(`/${owner.username}/scope-whole-app`);
|
||||
const app = (await makeApp(path)).app!;
|
||||
await putUserHandler(app.uid);
|
||||
|
||||
await env.server.services.permission.revokeUserAppAll(
|
||||
owner.actor,
|
||||
app.uid,
|
||||
);
|
||||
|
||||
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
|
||||
timeout: 5_000,
|
||||
interval: 25,
|
||||
});
|
||||
});
|
||||
|
||||
it('leaves the scope when the consent still stands, as after a revoke then a grant', async () => {
|
||||
await clearRows();
|
||||
const path = await folder(`/${owner.username}/scope-regrant`);
|
||||
const app = (await makeApp(path)).app!;
|
||||
await putUserHandler(app.uid);
|
||||
|
||||
for (const permission of [EVENTS_BACKGROUND_PERMISSION, null])
|
||||
await events().settleRevokedGrant({
|
||||
holderUserId: owner.id,
|
||||
appUid: app.uid,
|
||||
permission,
|
||||
});
|
||||
|
||||
expect(await held(app.uid)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('leaves the scope when the consent check itself fails', async () => {
|
||||
await clearRows();
|
||||
const path = await folder(`/${owner.username}/scope-check-fails`);
|
||||
const app = (await makeApp(path)).app!;
|
||||
await putUserHandler(app.uid);
|
||||
const check = vi
|
||||
.spyOn(events(), 'hasBackgroundConsent')
|
||||
.mockResolvedValue(null);
|
||||
try {
|
||||
await events().settleRevokedGrant({
|
||||
holderUserId: owner.id,
|
||||
appUid: app.uid,
|
||||
permission: EVENTS_BACKGROUND_PERMISSION,
|
||||
});
|
||||
expect(check).toHaveBeenCalled();
|
||||
expect(await held(app.uid)).toHaveLength(1);
|
||||
} finally {
|
||||
check.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('leaves the scope for an unrelated grant revoked, and another holder`s scope for any', async () => {
|
||||
await clearRows();
|
||||
const path = await folder(`/${owner.username}/scope-unrelated`);
|
||||
const app = (await makeApp(path)).app!;
|
||||
await putUserHandler(app.uid);
|
||||
await putUserHandler(app.uid, guest.id);
|
||||
|
||||
await env.server.services.permission.revokeUserAppPermission(
|
||||
owner.actor,
|
||||
app.uid,
|
||||
`fs:${await uidOf(path)}:list`,
|
||||
);
|
||||
await env.server.services.permission.revokeUserAppPermission(
|
||||
owner.actor,
|
||||
app.uid,
|
||||
EVENTS_BACKGROUND_PERMISSION,
|
||||
);
|
||||
|
||||
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
|
||||
timeout: 5_000,
|
||||
interval: 25,
|
||||
});
|
||||
expect(await held(app.uid, guest.id)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('keeps the rows bound to a dropped scope, suspended rather than deleted', async () => {
|
||||
await clearRows();
|
||||
const path = await folder(`/${owner.username}/scope-rows`);
|
||||
const appActor = await makeApp(path);
|
||||
const app = appActor.app!;
|
||||
const config = (events() as unknown as { config: IConfig }).config;
|
||||
config.events!.userHandlers = true;
|
||||
onTestFinished(() => {
|
||||
config.events!.userHandlers = false;
|
||||
});
|
||||
const sub = (
|
||||
await events().subscribeDurable(appActor, {
|
||||
subject: `fs:${path}`,
|
||||
handlerSource: 'async () => {}',
|
||||
})
|
||||
).sub;
|
||||
|
||||
await env.server.services.permission.revokeUserAppPermission(
|
||||
owner.actor,
|
||||
app.uid,
|
||||
EVENTS_BACKGROUND_PERMISSION,
|
||||
);
|
||||
|
||||
await suspendedRow(sub.subId);
|
||||
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
|
||||
timeout: 5_000,
|
||||
interval: 25,
|
||||
});
|
||||
expect((await rowOf(sub.subId)).suspended_reason).toBe(
|
||||
'permission_revoked',
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -83,6 +83,7 @@ const recording = () => {
|
||||
createFor: vi.fn(),
|
||||
list: vi.fn(),
|
||||
get: vi.fn(),
|
||||
update: vi.fn(),
|
||||
remove: vi.fn(),
|
||||
receivePeerChanges: vi.fn(),
|
||||
prepareSendAt: vi.fn(),
|
||||
|
||||
@@ -48,8 +48,7 @@ export type SubjectFamily = 'fs' | 'kv' | 'notif';
|
||||
|
||||
export type FsOp = 'add' | 'write' | 'move' | 'remove' | 'meta';
|
||||
|
||||
/** A notification is only ever posted; the mailbox verbs are their own surface. */
|
||||
export type NotifOp = 'post';
|
||||
export type NotifOp = 'post' | 'dismiss';
|
||||
|
||||
export type SubjectOp = FsOp | KvOp | NotifOp;
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,386 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* A user-scope handler through the real local worker runtime: created inline by
|
||||
* an app session, deployed eagerly as its holder, fired by an event, and run
|
||||
* with the holder's identity. Also the rehydrate route's scope resolution.
|
||||
*/
|
||||
|
||||
import http from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import {
|
||||
appHandlerScope,
|
||||
hashContent,
|
||||
} from '../../stores/events/EventHandlerStore.js';
|
||||
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
|
||||
import type { IConfig } from '../../types.js';
|
||||
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
|
||||
import { userHandlerScript } from './workerRuntime.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 180_000;
|
||||
const TEST_TIMEOUT_MS = 60_000;
|
||||
const INTERNAL_SECRET = 'events-internal-secret';
|
||||
|
||||
const SOURCE = `async ({ event, ctx, user, fetch, ack }) => {
|
||||
await fetch(ctx.sink, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
label: ctx.label,
|
||||
path: event.path,
|
||||
token: user.authToken,
|
||||
}),
|
||||
});
|
||||
await ack();
|
||||
}`;
|
||||
|
||||
interface SinkPost {
|
||||
label?: string;
|
||||
path?: string;
|
||||
token?: string;
|
||||
}
|
||||
|
||||
let env: PuterTestEnv;
|
||||
let holders: Array<{ id: number; uuid: string; anchor: string; token: string }>;
|
||||
let appUid: string;
|
||||
let sink: http.Server;
|
||||
let sinkUrl: string;
|
||||
let sinkPosts: SinkPost[];
|
||||
|
||||
const localWorkers = () => env.server.services.localworkerservice;
|
||||
|
||||
const scriptFor = (source: string): string =>
|
||||
userHandlerScript(
|
||||
{ name: `@${hashContent(source)}`, sourceHash: hashContent(source) },
|
||||
'',
|
||||
);
|
||||
|
||||
const isResident = async (script: string): Promise<boolean> =>
|
||||
(await localWorkers().dispatchEventsWorker(
|
||||
new Request(`http://${script}.workers.puter.localhost/`, {
|
||||
method: 'GET',
|
||||
}),
|
||||
)) !== null;
|
||||
|
||||
const api = async (
|
||||
method: 'GET' | 'POST',
|
||||
path: string,
|
||||
token: string,
|
||||
body?: object,
|
||||
): Promise<{ status: number; body: Record<string, unknown> }> => {
|
||||
const response = await fetch(new URL(path, env.apiOrigin), {
|
||||
method,
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${token}`,
|
||||
},
|
||||
...(body ? { body: JSON.stringify(body) } : {}),
|
||||
});
|
||||
return {
|
||||
status: response.status,
|
||||
body: (await response.json()) as Record<string, unknown>,
|
||||
};
|
||||
};
|
||||
|
||||
const rehydrate = async (
|
||||
body: object,
|
||||
): Promise<{ status: number; body: Record<string, unknown> }> => {
|
||||
const response = await fetch(
|
||||
new URL('/events/worker/rehydrate', env.apiOrigin),
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'x-puter-internal-auth': INTERNAL_SECRET,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
},
|
||||
);
|
||||
return {
|
||||
status: response.status,
|
||||
body: (await response.json()) as Record<string, unknown>,
|
||||
};
|
||||
};
|
||||
|
||||
const createInline = async (
|
||||
holder: (typeof holders)[number],
|
||||
label: string,
|
||||
source = SOURCE,
|
||||
): Promise<string> => {
|
||||
const response = await api('POST', '/events/subscribe', holder.token, {
|
||||
subject: `fs:${holder.anchor}`,
|
||||
delivery: 'single',
|
||||
handlerSource: source,
|
||||
context: { sink: sinkUrl, label },
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
return String(response.body.subId);
|
||||
};
|
||||
|
||||
const delivered = (count = 1): Promise<void> =>
|
||||
vi.waitFor(() => expect(sinkPosts.length).toBeGreaterThanOrEqual(count), {
|
||||
timeout: 30_000,
|
||||
interval: 50,
|
||||
});
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv({
|
||||
events: {
|
||||
enabled: true,
|
||||
workerRuntime: true,
|
||||
userHandlers: true,
|
||||
internalSecret: INTERNAL_SECRET,
|
||||
},
|
||||
workers: { localServer: true },
|
||||
unlimitedMetering: true,
|
||||
} as IConfig);
|
||||
|
||||
const owner = (await env.server.stores.user.getByUsername(
|
||||
env.users.user.username,
|
||||
))!;
|
||||
appUid = `app-${uuidv4()}`;
|
||||
await env.server.clients.db.write(
|
||||
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
|
||||
[appUid, appUid, appUid, `https://${appUid}.example/`, owner.id],
|
||||
);
|
||||
|
||||
holders = [];
|
||||
for (const account of [env.users.user, env.users.other]) {
|
||||
const user = (await env.server.stores.user.getByUsername(
|
||||
account.username,
|
||||
))!;
|
||||
const anchor = `/${account.username}/uh-e2e`;
|
||||
await env.server.services.fs.mkdir(user.id, {
|
||||
path: anchor,
|
||||
createMissingParents: true,
|
||||
});
|
||||
const entry = await env.server.stores.fsEntry.getEntryByPath(anchor);
|
||||
const { actor } = await env.server.services.auth.authenticate(
|
||||
account.token,
|
||||
);
|
||||
for (const permission of [
|
||||
`fs:${entry!.uid}:list`,
|
||||
EVENTS_BACKGROUND_PERMISSION,
|
||||
])
|
||||
await env.server.services.permission.grantUserAppPermission(
|
||||
actor!,
|
||||
appUid,
|
||||
permission,
|
||||
);
|
||||
holders.push({
|
||||
id: user.id,
|
||||
uuid: user.uuid,
|
||||
anchor,
|
||||
token: await env.server.services.auth.getUserAppToken(
|
||||
actor!,
|
||||
appUid,
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
sinkPosts = [];
|
||||
sink = http.createServer((req, res) => {
|
||||
const chunks: Buffer[] = [];
|
||||
req.on('data', (chunk: Buffer) => chunks.push(chunk));
|
||||
req.on('end', () => {
|
||||
sinkPosts.push(
|
||||
JSON.parse(Buffer.concat(chunks).toString('utf8') || '{}'),
|
||||
);
|
||||
res.writeHead(200).end();
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
sink.listen(0, '127.0.0.1', () => resolve()),
|
||||
);
|
||||
sinkUrl = `http://127.0.0.1:${(sink.address() as AddressInfo).port}`;
|
||||
}, BOOT_TIMEOUT_MS);
|
||||
|
||||
afterAll(async () => {
|
||||
await new Promise<void>((resolve) => {
|
||||
if (!sink) return resolve();
|
||||
sink.closeAllConnections();
|
||||
sink.close(() => resolve());
|
||||
});
|
||||
await env?.shutdown();
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
sinkPosts = [];
|
||||
});
|
||||
|
||||
describe('an inline handler in a user scope, end to end', () => {
|
||||
it(
|
||||
'is deployed eagerly, fires on an event and runs as its holder',
|
||||
{ timeout: TEST_TIMEOUT_MS },
|
||||
async () => {
|
||||
const [holder] = holders;
|
||||
const source = `${SOURCE}\n// run-as-holder`;
|
||||
const script = scriptFor(source);
|
||||
expect(await isResident(script)).toBe(false);
|
||||
|
||||
await createInline(holder, 'inline', source);
|
||||
|
||||
// The deploy started with the publish, before any event.
|
||||
await vi.waitFor(async () =>
|
||||
expect(await isResident(script)).toBe(true),
|
||||
);
|
||||
|
||||
await env.server.services.fs.touch(holder.id, {
|
||||
path: `${holder.anchor}/note.txt`,
|
||||
});
|
||||
await delivered();
|
||||
|
||||
expect(sinkPosts[0]).toMatchObject({
|
||||
label: 'inline',
|
||||
path: `${holder.anchor}/note.txt`,
|
||||
});
|
||||
const { actor } = await env.server.services.auth.authenticate(
|
||||
sinkPosts[0].token!,
|
||||
);
|
||||
expect(actor!.user?.id).toBe(holder.id);
|
||||
expect(actor!.effectiveApp?.uid).toBe(appUid);
|
||||
},
|
||||
);
|
||||
|
||||
it(
|
||||
'shares one deployed script between holders of the same code, each run as its own holder',
|
||||
{ timeout: TEST_TIMEOUT_MS },
|
||||
async () => {
|
||||
const source = `${SOURCE}\n// shared-code`;
|
||||
const script = scriptFor(source);
|
||||
const create = vi.spyOn(env.server.drivers.workers, 'create');
|
||||
|
||||
await createInline(holders[0], 'first', source);
|
||||
await vi.waitFor(async () =>
|
||||
expect(await isResident(script)).toBe(true),
|
||||
);
|
||||
await createInline(holders[1], 'second', source);
|
||||
// Let the second holder's eager deploy settle before counting.
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
|
||||
// The marker made the second holder's eager deploy a no-op.
|
||||
expect(
|
||||
create.mock.calls.filter(
|
||||
([args]) =>
|
||||
(args as { workerName: string }).workerName === script,
|
||||
),
|
||||
).toHaveLength(1);
|
||||
|
||||
for (const [i, holder] of holders.entries())
|
||||
await env.server.services.fs.touch(holder.id, {
|
||||
path: `${holder.anchor}/shared-${i}.txt`,
|
||||
});
|
||||
await delivered(2);
|
||||
const runAs = new Set<number>();
|
||||
for (const post of sinkPosts)
|
||||
runAs.add(
|
||||
(await env.server.services.auth.authenticate(post.token!))
|
||||
.actor!.user!.id,
|
||||
);
|
||||
expect(runAs).toEqual(new Set(holders.map((h) => h.id)));
|
||||
create.mockRestore();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe('the rehydrate route', () => {
|
||||
const source = `${SOURCE}\n// rehydrate`;
|
||||
const script = scriptFor(source);
|
||||
|
||||
beforeAll(async () => {
|
||||
// Rows only: the handler is published, nothing is deployed here.
|
||||
const eventHandler = env.server.stores.eventHandler;
|
||||
await eventHandler.publish({
|
||||
scope: { appUid, holder: holders[0].id },
|
||||
name: `@${hashContent(source)}`,
|
||||
source,
|
||||
broadcast: false,
|
||||
});
|
||||
await eventHandler.publish({
|
||||
scope: { appUid, holder: holders[1].id },
|
||||
name: 'other',
|
||||
source: SOURCE,
|
||||
broadcast: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('resolves the holder`s user scope when the body names one', async () => {
|
||||
const response = await rehydrate({
|
||||
script,
|
||||
appUid,
|
||||
holder: holders[0].uuid,
|
||||
});
|
||||
expect(response).toMatchObject({
|
||||
status: 200,
|
||||
body: { deployed: true },
|
||||
});
|
||||
expect(await isResident(script)).toBe(true);
|
||||
});
|
||||
|
||||
it('reads an absent or empty holder as the app scope, which has no such handler', async () => {
|
||||
for (const body of [
|
||||
{ script, appUid },
|
||||
{ script, appUid, holder: '' },
|
||||
])
|
||||
expect(await rehydrate(body)).toMatchObject({
|
||||
status: 404,
|
||||
body: { deployed: false, reason: 'no-handlers' },
|
||||
});
|
||||
expect(
|
||||
await env.server.stores.eventHandler.listForApp(
|
||||
appHandlerScope(appUid),
|
||||
),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it('answers stale for a holder whose scope does not hold that handler', async () => {
|
||||
expect(
|
||||
await rehydrate({ script, appUid, holder: holders[1].uuid }),
|
||||
).toMatchObject({
|
||||
status: 404,
|
||||
body: { deployed: false, reason: 'stale' },
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses a holder that does not exist, and a malformed one', async () => {
|
||||
expect(
|
||||
await rehydrate({ script, appUid, holder: uuidv4() }),
|
||||
).toMatchObject({
|
||||
status: 404,
|
||||
body: { deployed: false, reason: 'no-holder' },
|
||||
});
|
||||
expect(
|
||||
(await rehydrate({ script, appUid, holder: 'a b' })).status,
|
||||
).toBe(400);
|
||||
expect((await rehydrate({ script, appUid, holder: 7 })).status).toBe(
|
||||
400,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -34,6 +34,7 @@ import type { AddressInfo } from 'node:net';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
@@ -58,8 +59,14 @@ import {
|
||||
EVENTS_WORKER_SESSION_NAME,
|
||||
eventsInvokeKey,
|
||||
eventsWorkerScript,
|
||||
userHandlerScript,
|
||||
} from './workerRuntime.js';
|
||||
import { handlerSetHash } from './workerSource.js';
|
||||
import {
|
||||
appHandlerScope,
|
||||
hashContent,
|
||||
} from '../../stores/events/EventHandlerStore.js';
|
||||
import type { WorkerInvocation } from './workerSeam.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 120_000;
|
||||
/** Short enough that a hung handler is a test case rather than a stall. */
|
||||
@@ -129,6 +136,7 @@ const startStub = async (): Promise<string> => {
|
||||
script: req.headers['x-puter-events-script'] as string,
|
||||
app: req.headers['x-puter-events-app'] as string,
|
||||
key: req.headers['x-puter-events-key'] as string,
|
||||
holder: req.headers['x-puter-events-holder'] as string,
|
||||
deployed,
|
||||
},
|
||||
body: JSON.parse(raw || '{}') as StubCall['body'],
|
||||
@@ -233,7 +241,7 @@ const makeWorkerApp = async (): Promise<{
|
||||
);
|
||||
const token = await env.server.services.auth.getUserAppToken(actor!, uid);
|
||||
await env.server.stores.eventHandler.publish({
|
||||
appUid: uid,
|
||||
scope: appHandlerScope(uid),
|
||||
name: HANDLER,
|
||||
source: SOURCE,
|
||||
});
|
||||
@@ -403,7 +411,7 @@ beforeAll(async () => {
|
||||
);
|
||||
appToken = await env.server.services.auth.getUserAppToken(actor!, appUid);
|
||||
await env.server.stores.eventHandler.publish({
|
||||
appUid,
|
||||
scope: appHandlerScope(appUid),
|
||||
name: HANDLER,
|
||||
source: SOURCE,
|
||||
});
|
||||
@@ -471,7 +479,9 @@ describe('the call an owed delivery makes', () => {
|
||||
|
||||
const script = eventsWorkerScript(
|
||||
handlerSetHash(
|
||||
await env.server.stores.eventHandler.setForApp(appUid),
|
||||
await env.server.stores.eventHandler.setForApp(
|
||||
appHandlerScope(appUid),
|
||||
),
|
||||
),
|
||||
// No `workers.internetExposedUrl` in this test's config.
|
||||
'',
|
||||
@@ -1014,7 +1024,10 @@ describe('with no events worker to address', () => {
|
||||
const subId = await subscribe();
|
||||
// Unpublished after the fact: the app's set now hashes to nothing, so
|
||||
// there is no script to name and nowhere to send the delivery.
|
||||
await env.server.stores.eventHandler.remove(appUid, HANDLER);
|
||||
await env.server.stores.eventHandler.remove(
|
||||
appHandlerScope(appUid),
|
||||
HANDLER,
|
||||
);
|
||||
|
||||
try {
|
||||
await touch('unresolved.txt');
|
||||
@@ -1037,7 +1050,7 @@ describe('with no events worker to address', () => {
|
||||
expect(calls).toEqual([]);
|
||||
} finally {
|
||||
await env.server.stores.eventHandler.publish({
|
||||
appUid,
|
||||
scope: appHandlerScope(appUid),
|
||||
name: HANDLER,
|
||||
source: SOURCE,
|
||||
});
|
||||
@@ -1416,3 +1429,349 @@ describe('handlers whose writes run handlers', () => {
|
||||
).resolves.toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('a handler in a user scope', () => {
|
||||
const USER_SOURCE =
|
||||
'async ({ event, ctx }) => { console.log("user", event.path, ctx.label); }';
|
||||
const userName = `@${hashContent(USER_SOURCE)}`;
|
||||
let holderUuid: string;
|
||||
let createSpy: ReturnType<typeof vi.spyOn>;
|
||||
let meterSpy: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
const setFlag = (on: boolean): void => {
|
||||
(
|
||||
events() as unknown as { config: IConfig }
|
||||
).config.events!.userHandlers = on;
|
||||
};
|
||||
const userScope = () => ({ appUid, holder: userId });
|
||||
const userScript = () =>
|
||||
userHandlerScript(
|
||||
{ name: userName, sourceHash: hashContent(USER_SOURCE) },
|
||||
'',
|
||||
);
|
||||
|
||||
const post = async (path: string, body: object) => {
|
||||
const response = await fetch(new URL(path, env.apiOrigin), {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
authorization: `Bearer ${appToken}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
return {
|
||||
status: response.status,
|
||||
body: (await response.json()) as Record<string, unknown>,
|
||||
};
|
||||
};
|
||||
/** A single-delivery row bound to a handler published by its own source. */
|
||||
const inlineRow = async (source = USER_SOURCE): Promise<string> => {
|
||||
const created = await post('/events/subscribe', {
|
||||
subject: `fs:${anchor}`,
|
||||
delivery: 'single',
|
||||
handlerSource: source,
|
||||
context: { label: 'user' },
|
||||
});
|
||||
expect(created.status).toBe(200);
|
||||
return String(created.body.subId);
|
||||
};
|
||||
const invocationFor = (subId: string): WorkerInvocation => ({
|
||||
subId,
|
||||
holderUserId: userId,
|
||||
appUid,
|
||||
handlerName: userName,
|
||||
handlerScope: 'user',
|
||||
event: {
|
||||
subject: 'schedule:s1',
|
||||
op: 'tick',
|
||||
id: 's1:1',
|
||||
scheduleId: 's1',
|
||||
scheduledFor: 1,
|
||||
missed: 0,
|
||||
attempt: 1,
|
||||
},
|
||||
context: null,
|
||||
handlerDepth: 0,
|
||||
});
|
||||
|
||||
beforeAll(async () => {
|
||||
holderUuid = (await env.server.stores.user.getById(userId))!.uuid;
|
||||
});
|
||||
beforeEach(async () => {
|
||||
setFlag(true);
|
||||
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
|
||||
// Eager and on-miss deploys have no deploy target in this suite.
|
||||
createSpy = vi
|
||||
.spyOn(env.server.drivers.workers, 'create')
|
||||
.mockResolvedValue({ success: true, errors: [] });
|
||||
meterSpy = vi.spyOn(
|
||||
env.server.services.metering,
|
||||
'bufferIncrementUsages',
|
||||
);
|
||||
});
|
||||
afterEach(async () => {
|
||||
createSpy.mockRestore();
|
||||
meterSpy.mockRestore();
|
||||
setFlag(false);
|
||||
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
|
||||
});
|
||||
|
||||
it('addresses the handler`s own script and names its holder', async () => {
|
||||
const subId = await inlineRow();
|
||||
await touch('user-scope.txt');
|
||||
await invoked(1);
|
||||
|
||||
const call = calls[0];
|
||||
expect(call.headers.script).toBe(userScript());
|
||||
expect(call.headers.holder).toBe(holderUuid);
|
||||
expect(call.headers.key).toBe(
|
||||
eventsInvokeKey(INTERNAL_SECRET, userScript()),
|
||||
);
|
||||
expect(call.body.handler).toBe(userName);
|
||||
await waitUntil(async () =>
|
||||
expect(await pending().depth(subId)).toBe(0),
|
||||
);
|
||||
});
|
||||
|
||||
it('sends no holder for an app-scope row', async () => {
|
||||
await subscribe();
|
||||
await touch('app-scope.txt');
|
||||
await invoked(1);
|
||||
expect(calls[0].headers.holder).toBeUndefined();
|
||||
});
|
||||
|
||||
it('keeps a user row and an app row of the same name on their own scope`s handler', async () => {
|
||||
await env.server.stores.eventHandler.publish({
|
||||
scope: userScope(),
|
||||
name: HANDLER,
|
||||
source: USER_SOURCE,
|
||||
});
|
||||
const created = await post('/events/subscribe', {
|
||||
subject: `fs:${anchor}`,
|
||||
delivery: 'single',
|
||||
handlerName: HANDLER,
|
||||
handlerScope: 'user',
|
||||
});
|
||||
expect(created.status).toBe(200);
|
||||
await touch('same-name-user.txt');
|
||||
await invoked(1);
|
||||
// One at a time: a second row firing alongside would go as a batch.
|
||||
expect(
|
||||
(
|
||||
await post('/events/unsubscribe', {
|
||||
subId: String(created.body.subId),
|
||||
})
|
||||
).status,
|
||||
).toBe(200);
|
||||
await subscribe();
|
||||
await touch('same-name-app.txt');
|
||||
await invoked(2);
|
||||
|
||||
const appScript = eventsWorkerScript(
|
||||
handlerSetHash(
|
||||
await env.server.stores.eventHandler.setForApp(
|
||||
appHandlerScope(appUid),
|
||||
),
|
||||
),
|
||||
'',
|
||||
);
|
||||
const mine = userHandlerScript(
|
||||
{ name: HANDLER, sourceHash: hashContent(USER_SOURCE) },
|
||||
'',
|
||||
);
|
||||
expect(mine).not.toBe(appScript);
|
||||
expect(calls[0].headers).toMatchObject({
|
||||
script: mine,
|
||||
holder: holderUuid,
|
||||
});
|
||||
expect(calls[1].headers.script).toBe(appScript);
|
||||
expect(calls[1].headers.holder).toBeUndefined();
|
||||
});
|
||||
|
||||
it('suspends a row whose handler left its holder`s set, and tells the holder only', async () => {
|
||||
const notify = vi.spyOn(env.server.services.notification, 'notify');
|
||||
const subId = await inlineRow();
|
||||
// Gone without the removal path that would suspend its rows.
|
||||
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
|
||||
|
||||
await touch('missing.txt');
|
||||
await waitUntil(async () =>
|
||||
expect((await durable().getBySubId(subId))?.suspendedReason).toBe(
|
||||
'handler_not_found',
|
||||
),
|
||||
);
|
||||
expect(calls).toEqual([]);
|
||||
expect(notify).toHaveBeenCalledWith(
|
||||
[userId],
|
||||
expect.objectContaining({ handler: userName }),
|
||||
expect.objectContaining({
|
||||
type: 'app.events.handler_suspended',
|
||||
appUid,
|
||||
}),
|
||||
);
|
||||
expect(notify).not.toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
expect.objectContaining({ type: 'app.events.suspended' }),
|
||||
);
|
||||
notify.mockRestore();
|
||||
});
|
||||
|
||||
it('does not suspend a schedule run whose replica read lags the publish', async () => {
|
||||
const subId = await inlineRow();
|
||||
const store = env.server.stores.eventHandler;
|
||||
const real = store.setForApp.bind(store);
|
||||
// The replica has not seen the insert; the primary has.
|
||||
const lagging = vi
|
||||
.spyOn(store, 'setForApp')
|
||||
.mockImplementation(async (scope, options) =>
|
||||
options?.replica
|
||||
? [{ name: 'other', sourceHash: 'a'.repeat(64) }]
|
||||
: real(scope, options),
|
||||
);
|
||||
try {
|
||||
const result = await events().invokeForSchedule(
|
||||
invocationFor(subId),
|
||||
);
|
||||
expect(result).toMatchObject({ outcome: 'settled' });
|
||||
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
|
||||
expect(calls[0].headers.script).toBe(userScript());
|
||||
} finally {
|
||||
lagging.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('reads a cold user scope from the replica only, once, for a delivery', async () => {
|
||||
await inlineRow();
|
||||
// Eager deploy at publish has finished with the primary by now.
|
||||
await new Promise((resolve) => setTimeout(resolve, 300));
|
||||
const redis = env.server.clients.redis;
|
||||
const key = `eventHandlers:uset:${appUid}:${userId}`;
|
||||
await redis.del(key, `${key}:r`);
|
||||
const db = env.server.clients.db;
|
||||
const primary = vi.spyOn(db, 'pread');
|
||||
const replica = vi.spyOn(db, 'read');
|
||||
const handlerReads = (spy: typeof primary) =>
|
||||
spy.mock.calls.filter(([sql]) =>
|
||||
String(sql).includes('event_user_handlers'),
|
||||
).length;
|
||||
|
||||
await touch('cold-user-scope.txt');
|
||||
await invoked(1);
|
||||
|
||||
expect(handlerReads(primary)).toBe(0);
|
||||
expect(handlerReads(replica)).toBe(1);
|
||||
primary.mockRestore();
|
||||
replica.mockRestore();
|
||||
});
|
||||
|
||||
it('resumes a suspended row when the name is published again', async () => {
|
||||
const subId = await inlineRow();
|
||||
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
|
||||
await touch('gone.txt');
|
||||
await waitUntil(async () =>
|
||||
expect((await durable().getBySubId(subId))?.suspendedReason).toBe(
|
||||
'handler_not_found',
|
||||
),
|
||||
);
|
||||
|
||||
expect(
|
||||
(
|
||||
await post('/events/handlers/publish', {
|
||||
scope: 'user',
|
||||
name: userName,
|
||||
source: USER_SOURCE,
|
||||
})
|
||||
).status,
|
||||
).toBe(200);
|
||||
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
|
||||
});
|
||||
|
||||
it('does not address a holder that is suspended, and leaves the row owed', async () => {
|
||||
const subId = await inlineRow();
|
||||
await env.server.clients.db.write(
|
||||
'UPDATE user SET suspended = 1 WHERE id = ?',
|
||||
[userId],
|
||||
);
|
||||
await env.server.stores.user.invalidateById(userId);
|
||||
try {
|
||||
await touch('suspended-holder.txt');
|
||||
await failuresRecorded(subId, 1);
|
||||
expect(calls).toEqual([]);
|
||||
expect(await pending().depth(subId)).toBe(1);
|
||||
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
|
||||
} finally {
|
||||
await env.server.clients.db.write(
|
||||
'UPDATE user SET suspended = 0 WHERE id = ?',
|
||||
[userId],
|
||||
);
|
||||
await env.server.stores.user.invalidateById(userId);
|
||||
}
|
||||
});
|
||||
|
||||
it('deploys the holder`s script on a dispatcher miss, as that holder', async () => {
|
||||
dispatchMissing = true;
|
||||
await inlineRow();
|
||||
await touch('miss.txt');
|
||||
await invoked(2);
|
||||
|
||||
expect(calls[0].headers.deployed).toBeUndefined();
|
||||
expect(calls[1].headers.deployed).toBe('1');
|
||||
// Once for the eager deploy at publish, once for the miss.
|
||||
const deployed = createSpy.mock.calls.filter(
|
||||
([args]) =>
|
||||
(args as { workerName: string }).workerName === userScript(),
|
||||
);
|
||||
expect(deployed.length).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
describe('with the flag off', () => {
|
||||
it('leaves the delivery owed: nothing is called, suspended or billed', async () => {
|
||||
const subId = await inlineRow();
|
||||
setFlag(false);
|
||||
meterSpy.mockClear();
|
||||
|
||||
await touch('flag-off.txt');
|
||||
await new Promise((resolve) => setTimeout(resolve, 400));
|
||||
|
||||
expect(calls).toEqual([]);
|
||||
expect(await pending().depth(subId)).toBe(1);
|
||||
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
|
||||
expect(meterSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('answers unavailable from the invoker and for a schedule run', async () => {
|
||||
const subId = await inlineRow();
|
||||
setFlag(false);
|
||||
const invocation = invocationFor(subId);
|
||||
|
||||
const worker = events().worker as unknown as {
|
||||
run: (i: WorkerInvocation) => Promise<unknown>;
|
||||
};
|
||||
expect(await worker.run(invocation)).toEqual({
|
||||
outcome: 'unavailable',
|
||||
});
|
||||
expect(await events().invokeForSchedule(invocation)).toEqual({
|
||||
outcome: 'unavailable',
|
||||
});
|
||||
expect(calls).toEqual([]);
|
||||
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
|
||||
});
|
||||
|
||||
it('leaves an app-scope row alone', async () => {
|
||||
setFlag(false);
|
||||
await subscribe();
|
||||
await touch('app-row-flag-off.txt');
|
||||
await invoked(1);
|
||||
expect(calls[0].headers.script).toMatch(/^evw-/);
|
||||
});
|
||||
});
|
||||
|
||||
it('runs a schedule invocation of a user scope with its holder', async () => {
|
||||
const subId = await inlineRow();
|
||||
const result = await events().invokeForSchedule(invocationFor(subId));
|
||||
expect(result).toMatchObject({ outcome: 'settled' });
|
||||
expect(calls[0].headers.script).toBe(userScript());
|
||||
expect(calls[0].headers.holder).toBe(holderUuid);
|
||||
});
|
||||
});
|
||||
@@ -61,6 +61,7 @@ import {
|
||||
eventsWorkerScript,
|
||||
} from './workerRuntime.js';
|
||||
import { handlerSetHash } from './workerSource.js';
|
||||
import { appHandlerScope } from '../../stores/events/EventHandlerStore.js';
|
||||
|
||||
const BOOT_TIMEOUT_MS = 180_000;
|
||||
const TEST_TIMEOUT_MS = 60_000;
|
||||
@@ -171,7 +172,7 @@ const localWorkers = () => env.server.services.localworkerservice;
|
||||
/** The script the app's published set currently deploys as. */
|
||||
const scriptName = async (): Promise<string> =>
|
||||
eventsWorkerScript(
|
||||
handlerSetHash(await env.server.stores.eventHandler.setForApp(appUid)),
|
||||
handlerSetHash(await env.server.stores.eventHandler.setForApp(appHandlerScope(appUid))),
|
||||
// No `workers.internetExposedUrl` in this test's config, so the scope
|
||||
// this backend binds is empty.
|
||||
'',
|
||||
@@ -886,7 +887,7 @@ describe('the worker follows the handler set', () => {
|
||||
// An empty set names no script, so nothing can be deployed for
|
||||
// this app again until something is published.
|
||||
expect(
|
||||
await env.server.stores.eventHandler.setForApp(appUid),
|
||||
await env.server.stores.eventHandler.setForApp(appHandlerScope(appUid)),
|
||||
).toEqual([]);
|
||||
expect((await durable().getBySubId(subId))?.suspendedReason).toBe(
|
||||
'handler_not_found',
|
||||
|
||||
@@ -23,7 +23,9 @@ import {
|
||||
eventsInvokeKey,
|
||||
eventsWorkerScope,
|
||||
eventsWorkerScript,
|
||||
userHandlerScript,
|
||||
} from './workerRuntime.js';
|
||||
import { handlerSetHash } from './workerSource.js';
|
||||
|
||||
const SET_HASH = 'a1'.repeat(32);
|
||||
const OTHER_HASH = 'b2'.repeat(32);
|
||||
@@ -50,6 +52,37 @@ describe('eventsWorkerScript', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('userHandlerScript', () => {
|
||||
const handler = { name: '@abc', sourceHash: SET_HASH };
|
||||
|
||||
it('is the script of the handler as a set of one', () => {
|
||||
expect(userHandlerScript(handler, '')).toBe(
|
||||
eventsWorkerScript(handlerSetHash([handler]), ''),
|
||||
);
|
||||
});
|
||||
|
||||
it('is the same for identical code whoever holds it, and differs by code and name', () => {
|
||||
// Nothing but name and source hash goes in: two holders (or two apps)
|
||||
// of the same handler deploy one script.
|
||||
const again = { ...handler };
|
||||
expect(userHandlerScript(again, 's')).toBe(
|
||||
userHandlerScript(handler, 's'),
|
||||
);
|
||||
expect(
|
||||
userHandlerScript({ ...handler, sourceHash: OTHER_HASH }, 's'),
|
||||
).not.toBe(userHandlerScript(handler, 's'));
|
||||
expect(userHandlerScript({ ...handler, name: '@abd' }, 's')).not.toBe(
|
||||
userHandlerScript(handler, 's'),
|
||||
);
|
||||
});
|
||||
|
||||
it('stays scoped to the backend', () => {
|
||||
expect(userHandlerScript(handler, 'https://a.example')).not.toBe(
|
||||
userHandlerScript(handler, 'https://b.example'),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('eventsWorkerScope', () => {
|
||||
it('reads the same field `#cfDeploy` binds as `puter_endpoint`, and falls back to empty', () => {
|
||||
expect(eventsWorkerScope({})).toBe('');
|
||||
|
||||
@@ -18,6 +18,8 @@
|
||||
*/
|
||||
|
||||
import { createHash, createHmac } from 'node:crypto';
|
||||
import type { EventHandler } from '../../stores/events/EventHandlerStore.js';
|
||||
import { handlerSetHash } from './workerSource.js';
|
||||
|
||||
/**
|
||||
* How an app's events worker is named and how it recognizes the platform.
|
||||
@@ -71,6 +73,17 @@ export const eventsWorkerScript = (setHash: string, scope: string): string =>
|
||||
.digest('hex')
|
||||
.slice(0, 32)}`;
|
||||
|
||||
/**
|
||||
* The script one user-scope handler deploys as: the set-of-one name, built from
|
||||
* the handler's name and source hash alone. The holder and the app are not part
|
||||
* of it, so identical code under one name is one script however many users and
|
||||
* apps publish it.
|
||||
*/
|
||||
export const userHandlerScript = (
|
||||
handler: Pick<EventHandler, 'name' | 'sourceHash'>,
|
||||
scope: string,
|
||||
): string => eventsWorkerScript(handlerSetHash([handler]), scope);
|
||||
|
||||
/** Version prefix on a derived key, so a rotation is visible in a log line. */
|
||||
export const EVENTS_INVOKE_KEY_VERSION = 'k1';
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@ import type {
|
||||
WorkerInvokeOptions,
|
||||
} from '../../clients/events/EventsWorkerInvokerClient.js';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import type { HandlerScopeKind } from '../../stores/events/types.js';
|
||||
import type { DeliverableEvent, ScheduleEvent } from './registry.js';
|
||||
|
||||
export type { ScheduleEvent } from './registry.js';
|
||||
@@ -42,6 +43,8 @@ export interface WorkerInvocation {
|
||||
/** The app whose handler runs, or null for an account-owned row. */
|
||||
appUid: string | null;
|
||||
handlerName: string | null;
|
||||
/** Whose handler set `handlerName` is in: absent means the app's own. */
|
||||
handlerScope?: HandlerScopeKind;
|
||||
event: DeliverableEvent | ScheduleEvent;
|
||||
/** The subscription's stored context, delivered to the handler as `ctx`. */
|
||||
context: string | null;
|
||||
@@ -58,11 +61,7 @@ export interface WorkerInvocation {
|
||||
* toward the run that suspends a subscription.
|
||||
*/
|
||||
export type WorkerInvocationOutcome =
|
||||
| 'settled'
|
||||
| 'terminal'
|
||||
| 'retriable'
|
||||
| 'unavailable'
|
||||
| 'deferred';
|
||||
'settled' | 'terminal' | 'retriable' | 'unavailable' | 'deferred';
|
||||
|
||||
export interface WorkerInvocationResult {
|
||||
outcome: WorkerInvocationOutcome;
|
||||
@@ -96,21 +95,25 @@ export interface AddressOptions {
|
||||
missRead?: 'primary' | 'replica';
|
||||
/** A handler the set must have; a set without it is re-read on the primary. */
|
||||
handlerName?: string;
|
||||
/** Absent means the app's own set. */
|
||||
handlerScope?: HandlerScopeKind;
|
||||
/** The user whose scope it is, for `handlerScope: 'user'`. */
|
||||
holderUserId?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* The script an app's handlers deploy as and its invocation key; null when the
|
||||
* app has no handlers or no events secret is configured.
|
||||
* The script a handler deploys as and its invocation key; null when it has none
|
||||
* to run, no events secret is configured, or (in a user scope) the holder is
|
||||
* gone or suspended. `holder` is the holder's uuid, for a user scope.
|
||||
*/
|
||||
export type EventsWorkerAddresser = (
|
||||
appUid: string,
|
||||
options?: AddressOptions,
|
||||
) => Promise<{ script: string; key: string } | null>;
|
||||
) => Promise<{ script: string; key: string; holder?: string } | null>;
|
||||
|
||||
/** What one run came to, including why nothing was invoked. */
|
||||
export type WorkerRunResult =
|
||||
| WorkerInvocationResult
|
||||
| { outcome: MintRefusal | 'deploy-busy' };
|
||||
WorkerInvocationResult | { outcome: MintRefusal | 'deploy-busy' };
|
||||
|
||||
/** How {@link EventsWorkerInvoker.run} reaches the worker. */
|
||||
export type WorkerRunOptions = WorkerInvokeOptions & AddressOptions;
|
||||
@@ -131,6 +134,7 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
|
||||
readonly #mintToken: SubscriberTokenMinter;
|
||||
readonly #address: EventsWorkerAddresser;
|
||||
readonly #available: () => boolean;
|
||||
readonly #userHandlersEnabled: () => boolean;
|
||||
/**
|
||||
* Script -> when it was last logged, so a failing script floods once a
|
||||
* minute.
|
||||
@@ -142,11 +146,13 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
|
||||
mintToken: SubscriberTokenMinter,
|
||||
address: EventsWorkerAddresser,
|
||||
available: () => boolean,
|
||||
userHandlersEnabled: () => boolean = () => true,
|
||||
) {
|
||||
this.#client = client;
|
||||
this.#mintToken = mintToken;
|
||||
this.#address = address;
|
||||
this.#available = available;
|
||||
this.#userHandlersEnabled = userHandlersEnabled;
|
||||
}
|
||||
|
||||
available(): boolean {
|
||||
@@ -178,6 +184,10 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
|
||||
const { appUid, handlerName } = invocation;
|
||||
if (!appUid || !handlerName) return { outcome: 'deferred' };
|
||||
if (!this.#available()) return { outcome: 'unavailable' };
|
||||
// Owed, not failed: nothing resolves a user scope while it is off.
|
||||
const { handlerScope } = invocation;
|
||||
if (handlerScope === 'user' && !this.#userHandlersEnabled())
|
||||
return { outcome: 'unavailable' };
|
||||
|
||||
const minted = await this.#mintToken(invocation);
|
||||
if ('refused' in minted) return { outcome: minted.refused };
|
||||
@@ -188,6 +198,9 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
|
||||
const address = await this.#address(appUid, {
|
||||
missRead: options.missRead,
|
||||
handlerName,
|
||||
...(handlerScope === 'user'
|
||||
? { handlerScope, holderUserId: invocation.holderUserId }
|
||||
: {}),
|
||||
});
|
||||
if (address === null) return { outcome: 'retriable' };
|
||||
|
||||
@@ -196,6 +209,7 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
|
||||
script: address.script,
|
||||
key: address.key,
|
||||
appUid,
|
||||
...(address.holder ? { holder: address.holder } : {}),
|
||||
handler: handlerName,
|
||||
token: minted.token,
|
||||
event: invocation.event,
|
||||
@@ -204,6 +218,7 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
|
||||
{
|
||||
lane: options.lane,
|
||||
stampEvent: options.stampEvent,
|
||||
beforeSend: options.beforeSend,
|
||||
},
|
||||
);
|
||||
if (result.deployBusy) return { outcome: 'deploy-busy' };
|
||||
|
||||
@@ -175,7 +175,7 @@ let destroys: Array<{ appUid: string; ownerId: number | undefined }>;
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv({
|
||||
events: { enabled: true },
|
||||
events: { enabled: true, userHandlers: true },
|
||||
unlimitedMetering: true,
|
||||
} as IConfig);
|
||||
|
||||
@@ -204,6 +204,7 @@ afterAll(async () => {
|
||||
|
||||
beforeEach(async () => {
|
||||
await env.server.clients.db.write('DELETE FROM `event_handlers`', []);
|
||||
await env.server.clients.db.write('DELETE FROM `event_user_handlers`', []);
|
||||
await env.server.clients.db.write('DELETE FROM `event_subscriptions`', []);
|
||||
await env.server.clients.db.write('DELETE FROM `apps`', []);
|
||||
creates.length = 0;
|
||||
@@ -539,6 +540,8 @@ describe('POST /events/workers/destroy', () => {
|
||||
|
||||
const suspended = await env.server.stores.durableSubscription.listByHandler(
|
||||
app.uid,
|
||||
'app',
|
||||
0,
|
||||
'ingestUpload',
|
||||
{ suspendedReason: 'handler_not_found' },
|
||||
);
|
||||
@@ -554,6 +557,8 @@ describe('POST /events/workers/destroy', () => {
|
||||
const stillSuspended =
|
||||
await env.server.stores.durableSubscription.listByHandler(
|
||||
app.uid,
|
||||
'app',
|
||||
0,
|
||||
'ingestUpload',
|
||||
{ suspendedReason: 'handler_not_found' },
|
||||
);
|
||||
@@ -621,3 +626,319 @@ describe('POST /events/workers/destroy', () => {
|
||||
expect(destroys).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
// -- User scopes -----------------------------------------------------
|
||||
|
||||
/** `userId`'s own session of `app`, wherever the app's owner is. */
|
||||
const sessionIn = async (app: { uid: string }, asUser: 'user' | 'other') => {
|
||||
const { actor } = await env.server.services.auth.authenticate(
|
||||
asUser === 'user' ? env.users.user.token : env.users.other.token,
|
||||
);
|
||||
return env.server.services.auth.getUserAppToken(actor!, app.uid);
|
||||
};
|
||||
|
||||
const publishUserScope = (token: string, name: string, source: string) =>
|
||||
call('POST', '/events/handlers/publish', token, {
|
||||
scope: 'user',
|
||||
name,
|
||||
source,
|
||||
});
|
||||
|
||||
describe('GET /events/workers with user scopes', () => {
|
||||
it('adds the caller`s user scopes to their app workers outside an app', async () => {
|
||||
const mine = await makeApp(userId);
|
||||
await publish(mine.token, {
|
||||
appUid: mine.uid,
|
||||
name: 'a',
|
||||
source: SOURCE,
|
||||
});
|
||||
const theirs = await makeApp(otherUserId);
|
||||
const inTheirs = await sessionIn(theirs, 'user');
|
||||
await publishUserScope(inTheirs, 'x', SOURCE);
|
||||
await publishUserScope(inTheirs, 'y', OTHER_SOURCE);
|
||||
// Another holder's scope in the same app is not the caller's.
|
||||
await publishUserScope(await sessionIn(theirs, 'other'), 'z', SOURCE);
|
||||
|
||||
const listed = await call(
|
||||
'GET',
|
||||
'/events/workers',
|
||||
env.users.user.token,
|
||||
);
|
||||
|
||||
expect(listed.status).toBe(200);
|
||||
expect(listed.body.items).toEqual([
|
||||
expect.objectContaining({
|
||||
scope: 'app',
|
||||
appUid: mine.uid,
|
||||
handlerCount: 1,
|
||||
}),
|
||||
expect.objectContaining({
|
||||
scope: 'user',
|
||||
appUid: theirs.uid,
|
||||
appName: theirs.uid,
|
||||
handlerCount: 2,
|
||||
}),
|
||||
]);
|
||||
expect(listed.body.cursor).toBeUndefined();
|
||||
});
|
||||
|
||||
it('pages through the app workers and then the user scopes', async () => {
|
||||
const mine = await makeApp(userId);
|
||||
await publish(mine.token, {
|
||||
appUid: mine.uid,
|
||||
name: 'a',
|
||||
source: SOURCE,
|
||||
});
|
||||
const first = await makeApp(otherUserId);
|
||||
const second = await makeApp(otherUserId);
|
||||
await publishUserScope(await sessionIn(first, 'user'), 'x', SOURCE);
|
||||
await publishUserScope(await sessionIn(second, 'user'), 'x', SOURCE);
|
||||
|
||||
const seen: Array<Record<string, unknown>> = [];
|
||||
let cursor: string | undefined;
|
||||
let pages = 0;
|
||||
do {
|
||||
const page = await call(
|
||||
'GET',
|
||||
`/events/workers?limit=1${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ''}`,
|
||||
env.users.user.token,
|
||||
);
|
||||
expect(page.status).toBe(200);
|
||||
seen.push(...(page.body.items as Array<Record<string, unknown>>));
|
||||
cursor = page.body.cursor as string | undefined;
|
||||
pages += 1;
|
||||
} while (cursor && pages < 10);
|
||||
|
||||
expect(seen.map((item) => [item.scope, item.appUid])).toEqual([
|
||||
['app', mine.uid],
|
||||
...[first.uid, second.uid].sort().map((uid) => ['user', uid]),
|
||||
]);
|
||||
});
|
||||
|
||||
it('lists only that app`s scope for the caller inside an app', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
const otherApp = await makeApp(otherUserId);
|
||||
const inTheirs = await sessionIn(theirs, 'user');
|
||||
await publishUserScope(inTheirs, 'x', SOURCE);
|
||||
await publishUserScope(await sessionIn(otherApp, 'user'), 'x', SOURCE);
|
||||
await publishUserScope(await sessionIn(theirs, 'other'), 'x', SOURCE);
|
||||
// The owner's own worker in that app is not the caller's to see.
|
||||
await publish(theirs.token, {
|
||||
appUid: theirs.uid,
|
||||
name: 'owner',
|
||||
source: SOURCE,
|
||||
});
|
||||
|
||||
const listed = await call('GET', '/events/workers', inTheirs);
|
||||
|
||||
expect(listed.status).toBe(200);
|
||||
expect(listed.body.items).toEqual([
|
||||
expect.objectContaining({
|
||||
scope: 'user',
|
||||
appUid: theirs.uid,
|
||||
handlerCount: 1,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it('still lists the owner`s worker beside their own user scope inside the app', async () => {
|
||||
const mine = await makeApp(userId);
|
||||
await publish(mine.token, {
|
||||
appUid: mine.uid,
|
||||
name: 'a',
|
||||
source: SOURCE,
|
||||
});
|
||||
await publishUserScope(mine.token, 'x', SOURCE);
|
||||
|
||||
const listed = await call('GET', '/events/workers', mine.token);
|
||||
|
||||
expect(
|
||||
(listed.body.items as Array<Record<string, unknown>>).map(
|
||||
(item) => item.scope,
|
||||
),
|
||||
).toEqual(['app', 'user']);
|
||||
});
|
||||
|
||||
it('refuses a scoped token, which has no scope of its own', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
await publishUserScope(await sessionIn(theirs, 'user'), 'x', SOURCE);
|
||||
const scoped = await appIssuedScopedToken({
|
||||
uid: theirs.uid,
|
||||
token: await sessionIn(theirs, 'user'),
|
||||
});
|
||||
|
||||
const listed = await call('GET', '/events/workers', scoped);
|
||||
expect(listed.status).toBe(403);
|
||||
expect(listed.body.code).toBe('events_worker_owner_only');
|
||||
});
|
||||
});
|
||||
|
||||
describe('POST /events/workers/destroy with a user scope', () => {
|
||||
it('removes the caller`s scope only, suspends what was bound, and announces it', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
const mineIn = await sessionIn(theirs, 'user');
|
||||
const otherIn = await sessionIn(theirs, 'other');
|
||||
await publish(theirs.token, {
|
||||
appUid: theirs.uid,
|
||||
name: 'owner',
|
||||
source: SOURCE,
|
||||
});
|
||||
await publishUserScope(mineIn, 'x', SOURCE);
|
||||
await publishUserScope(mineIn, 'y', OTHER_SOURCE);
|
||||
await publishUserScope(otherIn, 'x', SOURCE);
|
||||
destroys.length = 0;
|
||||
|
||||
const destroyed = await call(
|
||||
'POST',
|
||||
'/events/workers/destroy',
|
||||
mineIn,
|
||||
{
|
||||
scope: 'user',
|
||||
},
|
||||
);
|
||||
|
||||
expect(destroyed.status).toBe(200);
|
||||
expect(destroyed.body).toMatchObject({
|
||||
appUid: theirs.uid,
|
||||
removed: 2,
|
||||
suspended: 0,
|
||||
scope: 'user',
|
||||
});
|
||||
const store = env.server.stores.eventHandler;
|
||||
expect(
|
||||
await store.countForApp({ appUid: theirs.uid, holder: userId }),
|
||||
).toBe(0);
|
||||
expect(
|
||||
await store.countForApp({
|
||||
appUid: theirs.uid,
|
||||
holder: otherUserId,
|
||||
}),
|
||||
).toBe(1);
|
||||
expect(await store.countForApp({ appUid: theirs.uid, holder: 0 })).toBe(
|
||||
1,
|
||||
);
|
||||
expect(destroys).toEqual([{ appUid: theirs.uid, ownerId: userId }]);
|
||||
|
||||
const again = await call('POST', '/events/workers/destroy', mineIn, {
|
||||
scope: 'user',
|
||||
});
|
||||
expect(again.status).toBe(404);
|
||||
});
|
||||
|
||||
it('refuses a scoped token and an app session naming another app', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
const mineIn = await sessionIn(theirs, 'user');
|
||||
await publishUserScope(mineIn, 'x', SOURCE);
|
||||
const scoped = await appIssuedScopedToken({
|
||||
uid: theirs.uid,
|
||||
token: mineIn,
|
||||
});
|
||||
const elsewhere = await makeApp(otherUserId);
|
||||
|
||||
for (const [token, body] of [
|
||||
[scoped, { scope: 'user' }],
|
||||
[mineIn, { scope: 'user', appUid: elsewhere.uid }],
|
||||
] as const) {
|
||||
const refused = await call(
|
||||
'POST',
|
||||
'/events/workers/destroy',
|
||||
token,
|
||||
body,
|
||||
);
|
||||
expect(refused.status).toBe(403);
|
||||
expect(refused.body.code).toBe('events_handler_forbidden');
|
||||
}
|
||||
expect(
|
||||
await env.server.stores.eventHandler.countForApp({
|
||||
appUid: theirs.uid,
|
||||
holder: userId,
|
||||
}),
|
||||
).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('destroying a user scope from an account session', () => {
|
||||
it('removes the caller`s own scope of any app, and nobody else`s', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
await publishUserScope(await sessionIn(theirs, 'user'), 'x', SOURCE);
|
||||
await publishUserScope(await sessionIn(theirs, 'other'), 'x', SOURCE);
|
||||
const store = env.server.stores.eventHandler;
|
||||
destroys.length = 0;
|
||||
|
||||
const destroyed = await call(
|
||||
'POST',
|
||||
'/events/workers/destroy',
|
||||
env.users.user.token,
|
||||
{ scope: 'user', appUid: theirs.uid },
|
||||
);
|
||||
|
||||
expect(destroyed.status).toBe(200);
|
||||
expect(destroyed.body).toMatchObject({ removed: 1, scope: 'user' });
|
||||
expect(
|
||||
await store.countForApp({ appUid: theirs.uid, holder: userId }),
|
||||
).toBe(0);
|
||||
expect(
|
||||
await store.countForApp({ appUid: theirs.uid, holder: otherUserId }),
|
||||
).toBe(1);
|
||||
expect(destroys).toEqual([{ appUid: theirs.uid, ownerId: userId }]);
|
||||
|
||||
// The same call as the other user removes only theirs; the first user
|
||||
// has no scope left to touch.
|
||||
const none = await call(
|
||||
'POST',
|
||||
'/events/workers/destroy',
|
||||
env.users.user.token,
|
||||
{ scope: 'user', appUid: theirs.uid },
|
||||
);
|
||||
expect(none.status).toBe(404);
|
||||
expect(
|
||||
await store.countForApp({ appUid: theirs.uid, holder: otherUserId }),
|
||||
).toBe(1);
|
||||
});
|
||||
|
||||
it('needs an appUid, and still refuses a scoped token', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
const mineIn = await sessionIn(theirs, 'user');
|
||||
await publishUserScope(mineIn, 'x', SOURCE);
|
||||
|
||||
const noApp = await call(
|
||||
'POST',
|
||||
'/events/workers/destroy',
|
||||
env.users.user.token,
|
||||
{ scope: 'user' },
|
||||
);
|
||||
expect(noApp.status).toBe(403);
|
||||
|
||||
const scoped = await appIssuedScopedToken({
|
||||
uid: theirs.uid,
|
||||
token: mineIn,
|
||||
});
|
||||
const refused = await call('POST', '/events/workers/destroy', scoped, {
|
||||
scope: 'user',
|
||||
appUid: theirs.uid,
|
||||
});
|
||||
expect(refused.status).toBe(403);
|
||||
});
|
||||
|
||||
it('works with user handlers switched off, so a user can stop paying', async () => {
|
||||
const theirs = await makeApp(otherUserId);
|
||||
await publishUserScope(await sessionIn(theirs, 'user'), 'x', SOURCE);
|
||||
const config = (
|
||||
env.server.services.events as unknown as {
|
||||
config: { events: { userHandlers?: boolean } };
|
||||
}
|
||||
).config;
|
||||
config.events.userHandlers = false;
|
||||
try {
|
||||
const destroyed = await call(
|
||||
'POST',
|
||||
'/events/workers/destroy',
|
||||
env.users.user.token,
|
||||
{ scope: 'user', appUid: theirs.uid },
|
||||
);
|
||||
expect(destroyed.status).toBe(200);
|
||||
} finally {
|
||||
config.events.userHandlers = true;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -70,6 +70,12 @@ const EXPECTED_JOBS = {
|
||||
nearPrimary: true,
|
||||
leaseMs: 15 * MINUTE,
|
||||
},
|
||||
'events.userHandlers.gc': {
|
||||
every: HOUR,
|
||||
scope: 'fleet',
|
||||
nearPrimary: true,
|
||||
leaseMs: 15 * MINUTE,
|
||||
},
|
||||
'events.subscriptions.creditSweep': {
|
||||
every: 15 * MINUTE,
|
||||
scope: 'fleet',
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from 'vitest';
|
||||
import type { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
|
||||
const { disposals } = vi.hoisted(() => ({
|
||||
disposals: [] as Array<ReturnType<typeof vi.fn>>,
|
||||
}));
|
||||
|
||||
vi.mock('miniflare', () => ({
|
||||
Miniflare: class {
|
||||
dispose = vi.fn(async () => {});
|
||||
constructor() {
|
||||
disposals.push(this.dispose);
|
||||
}
|
||||
},
|
||||
}));
|
||||
|
||||
describe('LocalWorkerService shutdown disposal', () => {
|
||||
let server: PuterServer;
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer({
|
||||
api_base_url: 'http://api.puter.localhost',
|
||||
} as never);
|
||||
});
|
||||
afterAll(async () => {
|
||||
await server.shutdown();
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
disposals.length = 0;
|
||||
});
|
||||
|
||||
it('awaits every disposal even if one fails', async () => {
|
||||
const service = server.services.localworkerservice;
|
||||
await service.cfDeployLocal('first', 'auth', 'export default {}');
|
||||
await service.cfDeployLocal('second', 'auth', 'export default {}');
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
disposals[0].mockRejectedValueOnce(new Error('dispose failed'));
|
||||
disposals[1].mockImplementationOnce(async () => {
|
||||
await held;
|
||||
});
|
||||
let finished = false;
|
||||
const draining = Promise.resolve(service.onServerShutdown()).then(
|
||||
() => {
|
||||
finished = true;
|
||||
},
|
||||
);
|
||||
try {
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(
|
||||
disposals.every((dispose) => dispose.mock.calls.length === 1),
|
||||
).toBe(true);
|
||||
expect(finished).toBe(false);
|
||||
} finally {
|
||||
release();
|
||||
await draining;
|
||||
}
|
||||
});
|
||||
|
||||
it('bounds disposal of a hung worker to five seconds', async () => {
|
||||
const service = server.services.localworkerservice;
|
||||
await service.cfDeployLocal('hung', 'auth', 'export default {}');
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
disposals[0].mockImplementationOnce(async () => {
|
||||
await held;
|
||||
});
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
let finished = false;
|
||||
const draining = Promise.resolve(service.onServerShutdown()).then(
|
||||
() => {
|
||||
finished = true;
|
||||
},
|
||||
);
|
||||
await vi.advanceTimersByTimeAsync(4_999);
|
||||
expect(finished).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
await draining;
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
release();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -10,6 +10,8 @@ import {
|
||||
import { puterStores } from '../../stores';
|
||||
import type { SubdomainRow } from '../../stores/subdomain/SubdomainStore';
|
||||
import { LayerInstances } from '../../types';
|
||||
import { settleWithin } from '../../util/concurrency.js';
|
||||
import { FINAL_FLUSH_TIMEOUT_MS } from '../../util/gracefulShutdown.js';
|
||||
import { PuterService } from '../types';
|
||||
|
||||
const MAX_SOURCE_SIZE = 10 * 1024 * 1024; // 10 MB
|
||||
@@ -240,13 +242,16 @@ export class LocalWorkerService extends PuterService {
|
||||
idleSweepTimer.unref?.();
|
||||
}
|
||||
|
||||
override onServerShutdown(): void {
|
||||
override async onServerShutdown(): Promise<void> {
|
||||
if (idleSweepTimer) {
|
||||
clearInterval(idleSweepTimer);
|
||||
idleSweepTimer = null;
|
||||
}
|
||||
for (const name of [...activeWorkers.keys()]) {
|
||||
void this.#disposeWorker(name);
|
||||
const work = Promise.allSettled(
|
||||
[...activeWorkers.keys()].map((name) => this.#disposeWorker(name)),
|
||||
);
|
||||
if (!(await settleWithin(work, FINAL_FLUSH_TIMEOUT_MS))) {
|
||||
console.warn('[localworker] final flush timed out');
|
||||
}
|
||||
}
|
||||
async reconstructDeployArgs(workerName: string, row: SubdomainRow) {
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import { expect, it } from 'vitest';
|
||||
import { EventClient } from '../../clients/event/EventClient.js';
|
||||
import type { EventMap } from '../../clients/event/types.js';
|
||||
import { NotificationService } from './NotificationService.js';
|
||||
|
||||
it.each([undefined, 0, 1_700_000_000_000])(
|
||||
'emits replacement metadata for replacedAt %s',
|
||||
async (replacedAt) => {
|
||||
const event = new EventClient({} as never);
|
||||
const written: EventMap['notif.written'][] = [];
|
||||
event.on('notif.written', (_key, value) => {
|
||||
written.push(value);
|
||||
});
|
||||
const service = new NotificationService(
|
||||
{} as never,
|
||||
{ event } as never,
|
||||
{} as never,
|
||||
);
|
||||
|
||||
await service.emitWritten(
|
||||
{
|
||||
userId: 1,
|
||||
uid: 'notification-1',
|
||||
type: 'app.message',
|
||||
audience: 'app-user',
|
||||
appUid: 'app-1',
|
||||
value: { title: 'A message' },
|
||||
updated: replacedAt !== undefined,
|
||||
replacedAt,
|
||||
},
|
||||
{ sendId: 'send-1' },
|
||||
);
|
||||
|
||||
expect(written).toHaveLength(1);
|
||||
if (replacedAt === undefined)
|
||||
expect(written[0]).not.toHaveProperty('replaced');
|
||||
else expect(written[0].replaced).toBe(true);
|
||||
},
|
||||
);
|
||||
@@ -354,6 +354,7 @@ describe('NotificationService.notifyUpdate', () => {
|
||||
appUid,
|
||||
value: { title: 'two messages', type: 'app.message' },
|
||||
updated: true,
|
||||
replaced: true,
|
||||
});
|
||||
expect(desktop.seen).toEqual([]);
|
||||
expect(
|
||||
|
||||
@@ -310,6 +310,9 @@ export class NotificationService extends PuterService {
|
||||
...scope,
|
||||
value: structuredClone(payload),
|
||||
...(updated ? { updated: true as const } : {}),
|
||||
...(written.replacedAt !== undefined
|
||||
? { replaced: true as const }
|
||||
: {}),
|
||||
sendId: opts.sendId,
|
||||
...(opts.replayable ? { replayable: true as const } : {}),
|
||||
},
|
||||
|
||||
@@ -172,13 +172,23 @@ describe('parseAppMessage', () => {
|
||||
rejects({ title: 't', sensitive: 'yes' }, 'sensitive');
|
||||
});
|
||||
|
||||
it('refuses high priority to apps', () => {
|
||||
expect(() => parseAppMessage({ title: 't', priority: 'high' })).toThrow(
|
||||
"'high' is not available to apps",
|
||||
);
|
||||
rejects({ title: 't', priority: 'urgent' }, 'priority');
|
||||
it('accepts normal and high priority, defaulting to normal', () => {
|
||||
expect(parseAppMessage({ title: 't' }).priority).toBe('normal');
|
||||
expect(
|
||||
parseAppMessage({ title: 't', priority: 'normal' }).priority,
|
||||
).toBe('normal');
|
||||
expect(
|
||||
parseAppMessage({ title: 't', priority: 'high' }).priority,
|
||||
).toBe('high');
|
||||
});
|
||||
|
||||
it.each(['urgent', 'low', '', null, 1, true])(
|
||||
'rejects priority %s',
|
||||
(priority) => {
|
||||
rejects({ title: 't', priority }, 'priority');
|
||||
},
|
||||
);
|
||||
|
||||
it('bounds ttl to whole seconds within a day', () => {
|
||||
expect(parseAppMessage({ title: 't', ttl: 1 }).ttl).toBe(1);
|
||||
rejects({ title: 't', ttl: 0 }, 'ttl');
|
||||
@@ -243,4 +253,61 @@ describe('appMessageSender', () => {
|
||||
'a'.repeat(120),
|
||||
);
|
||||
});
|
||||
|
||||
describe('an app made for an origin', () => {
|
||||
const uid = 'app-65d98228-cfa5-5d18-a442-f6eeaf3dcfc7';
|
||||
const originApp = (fields: Record<string, unknown> = {}) => ({
|
||||
uid,
|
||||
name: uid,
|
||||
title: uid,
|
||||
description: 'App created from origin https://chat.example.com',
|
||||
index_url: 'https://chat.example.com',
|
||||
...fields,
|
||||
});
|
||||
|
||||
it('is named by the hostname of its index_url, without a port', () => {
|
||||
expect(appMessageSender(uid, originApp()).name).toBe(
|
||||
'chat.example.com',
|
||||
);
|
||||
expect(
|
||||
appMessageSender(
|
||||
uid,
|
||||
originApp({
|
||||
index_url: 'https://chat.example.com:8443/app/',
|
||||
}),
|
||||
).name,
|
||||
).toBe('chat.example.com');
|
||||
});
|
||||
|
||||
it('keeps its stored name when its index_url does not parse', () => {
|
||||
expect(
|
||||
appMessageSender(uid, originApp({ index_url: 'not a url' }))
|
||||
.name,
|
||||
).toBe(uid);
|
||||
expect(
|
||||
appMessageSender(uid, originApp({ index_url: undefined })).name,
|
||||
).toBe(uid);
|
||||
});
|
||||
|
||||
it('keeps a title its developer set', () => {
|
||||
expect(
|
||||
appMessageSender(uid, originApp({ title: 'Chat' })).name,
|
||||
).toBe('Chat');
|
||||
});
|
||||
|
||||
it('leaves apps that were not made for an origin alone', () => {
|
||||
expect(
|
||||
appMessageSender(uid, originApp({ description: 'My chat app' }))
|
||||
.name,
|
||||
).toBe(uid);
|
||||
expect(
|
||||
appMessageSender('app-1', {
|
||||
uid: 'app-1',
|
||||
name: 'chat',
|
||||
title: 'Chat',
|
||||
index_url: 'https://chat.example.com',
|
||||
}).name,
|
||||
).toBe('Chat');
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -18,6 +18,7 @@
|
||||
*/
|
||||
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import { isOriginBootstrapApp } from '../../util/appView.js';
|
||||
|
||||
/** The one notification type an app may create. */
|
||||
export const APP_MESSAGE_TYPE = 'app.message';
|
||||
@@ -38,7 +39,7 @@ export const APP_MESSAGE_RATE_LIMITS = {
|
||||
perDay: 1000,
|
||||
} as const;
|
||||
|
||||
export type AppMessagePriority = 'normal';
|
||||
export type AppMessagePriority = 'normal' | 'high';
|
||||
|
||||
/** An `app.message` value as stored, after normalization. */
|
||||
export type AppMessage = {
|
||||
@@ -60,14 +61,35 @@ export type AppMessage = {
|
||||
|
||||
export type AppMessageSender = { uid: string; name?: string; icon?: string };
|
||||
|
||||
/** The hostname an app made for an origin is named by. */
|
||||
const originAppHost = (row: unknown): string | undefined => {
|
||||
if (!isOriginBootstrapApp(row) || typeof row.index_url !== 'string') {
|
||||
return undefined;
|
||||
}
|
||||
try {
|
||||
return new URL(row.index_url).hostname || undefined;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
|
||||
/** What a message says about its app, read off the app's row. */
|
||||
export const appMessageSender = (
|
||||
uid: string,
|
||||
row: { name?: unknown; title?: unknown; icon?: unknown } | null | undefined,
|
||||
row:
|
||||
| {
|
||||
name?: unknown;
|
||||
title?: unknown;
|
||||
icon?: unknown;
|
||||
description?: unknown;
|
||||
index_url?: unknown;
|
||||
}
|
||||
| null
|
||||
| undefined,
|
||||
): AppMessageSender => {
|
||||
const sender: AppMessageSender = { uid };
|
||||
// Cleaned before the empty check, so a title of only invisible characters still leaves a visible sender.
|
||||
const name = [row?.title, row?.name]
|
||||
const name = [originAppHost(row), row?.title, row?.name]
|
||||
.map((v) =>
|
||||
typeof v === 'string'
|
||||
? [...cleanText(v)]
|
||||
@@ -300,13 +322,11 @@ export const parseAppMessage = (input: Record<string, unknown>): AppMessage => {
|
||||
message.sensitive = input.sensitive;
|
||||
}
|
||||
|
||||
if (input.priority !== undefined && input.priority !== 'normal') {
|
||||
throw invalid(
|
||||
'priority',
|
||||
input.priority === 'high'
|
||||
? "'high' is not available to apps; use 'normal'"
|
||||
: "must be 'normal'",
|
||||
);
|
||||
if (input.priority !== undefined) {
|
||||
if (input.priority !== 'normal' && input.priority !== 'high') {
|
||||
throw invalid('priority', "must be 'normal' or 'high'");
|
||||
}
|
||||
message.priority = input.priority;
|
||||
}
|
||||
|
||||
return message;
|
||||
|
||||
@@ -3,18 +3,19 @@
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
* Puter is free software: you can redistribute it and/or modify it under the
|
||||
* terms of the GNU Affero General Public License as published by the Free
|
||||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||||
* later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||||
* details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
* along with this program. If not, see
|
||||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
@@ -53,6 +54,9 @@ describe('NOTIFICATION_TYPES', () => {
|
||||
audience: 'developer',
|
||||
appScoped: true,
|
||||
});
|
||||
expect(
|
||||
findNotificationType('app.events.handler_suspended'),
|
||||
).toMatchObject({ audience: 'app-user', appScoped: true });
|
||||
expect(findNotificationType('app.events.ended')).toMatchObject({
|
||||
audience: 'app-user',
|
||||
appScoped: false,
|
||||
@@ -61,9 +65,9 @@ describe('NOTIFICATION_TYPES', () => {
|
||||
|
||||
it('projects a subject naming the app, or the recipient when there is none', () => {
|
||||
const account = findNotificationType('share.received')!;
|
||||
expect(
|
||||
account.subject({ userUuid: 'user-uuid', appUid: null }),
|
||||
).toBe('notif:user-uuid:account');
|
||||
expect(account.subject({ userUuid: 'user-uuid', appUid: null })).toBe(
|
||||
'notif:user-uuid:account',
|
||||
);
|
||||
|
||||
const developer = findNotificationType('app.events.suspended')!;
|
||||
expect(
|
||||
@@ -130,18 +134,20 @@ describe('resolveNotificationWrite', () => {
|
||||
});
|
||||
|
||||
it('requires an app uid for an app message', () => {
|
||||
expect(resolveNotificationWrite('app.message', 'app-1')).toMatchObject(
|
||||
{ audience: 'app-user', appScoped: true, groupable: true },
|
||||
);
|
||||
expect(resolveNotificationWrite('app.message', 'app-1')).toMatchObject({
|
||||
audience: 'app-user',
|
||||
appScoped: true,
|
||||
groupable: true,
|
||||
});
|
||||
expect(() => resolveNotificationWrite('app.message', null)).toThrow(
|
||||
'requires an app uid',
|
||||
);
|
||||
});
|
||||
|
||||
it('lets a subscription an account made end without naming an app', () => {
|
||||
expect(resolveNotificationWrite('app.events.ended', null).audience).toBe(
|
||||
'app-user',
|
||||
);
|
||||
expect(
|
||||
resolveNotificationWrite('app.events.ended', null).audience,
|
||||
).toBe('app-user');
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -17,6 +17,8 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
|
||||
/** Who the recipient is in relation to the notification. */
|
||||
export type NotificationAudience = 'account' | 'developer' | 'app-user';
|
||||
|
||||
@@ -115,6 +117,15 @@ export const NOTIFICATION_TYPES = [
|
||||
groupable: false,
|
||||
subject: subjectFor('developer'),
|
||||
},
|
||||
// A handler the user published into their own scope stopped taking
|
||||
// deliveries: the code is theirs, so the notice is too.
|
||||
{
|
||||
type: 'app.events.handler_suspended',
|
||||
audience: 'app-user',
|
||||
appScoped: true,
|
||||
groupable: false,
|
||||
subject: subjectFor('app-user'),
|
||||
},
|
||||
// The only type apps create; its kind goes in the value's `channel`. A
|
||||
// newer one replaces the newest with the same `group` unless that one was
|
||||
// shown or dismissed.
|
||||
@@ -144,6 +155,22 @@ export type NotificationTypeName = (typeof NOTIFICATION_TYPES)[number]['type'];
|
||||
/** Serialized `value` may not exceed this many bytes. */
|
||||
export const NOTIFICATION_VALUE_MAX_BYTES = 16 * 1024;
|
||||
|
||||
export const assertNotificationValueFits = (
|
||||
value: Record<string, unknown>,
|
||||
): void => {
|
||||
if (
|
||||
Buffer.byteLength(JSON.stringify(value), 'utf8') >
|
||||
NOTIFICATION_VALUE_MAX_BYTES
|
||||
) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
`\`value\` may not exceed ${NOTIFICATION_VALUE_MAX_BYTES} bytes`,
|
||||
{
|
||||
legacyCode: 'notification_value_too_large',
|
||||
},
|
||||
);
|
||||
}
|
||||
};
|
||||
const BY_NAME = new Map<string, NotificationType>(
|
||||
NOTIFICATION_TYPES.map((entry) => [entry.type, entry]),
|
||||
);
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { Actor } from '../../core/actor.js';
|
||||
import type { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import type { ResolvedShare } from './ShareService.js';
|
||||
|
||||
describe('ShareNotificationService shutdown drain', () => {
|
||||
let server: PuterServer;
|
||||
beforeEach(async () => {
|
||||
server = await setupTestServer({
|
||||
email: { from: '[email protected]', host: '127.0.0.1', port: 1 },
|
||||
share_notify_limits: { emailBatchSeconds: 0.01 },
|
||||
} as never);
|
||||
});
|
||||
afterEach(async () => {
|
||||
vi.useRealTimers();
|
||||
await server.services.shareNotification.onServerPrepareShutdown();
|
||||
await server.services.shareNotification.onServerShutdown();
|
||||
await server.shutdown();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const queueInvite = async (email: string, issuerId = 7) => {
|
||||
await server.services.shareNotification.notifyShared(
|
||||
{
|
||||
user: { id: issuerId, username: 'sender' },
|
||||
effectiveApp: null,
|
||||
} as Actor,
|
||||
[
|
||||
{
|
||||
pending: true,
|
||||
isNew: true,
|
||||
recipientEmail: email,
|
||||
name: 'shared.txt',
|
||||
} as ResolvedShare,
|
||||
],
|
||||
);
|
||||
};
|
||||
|
||||
it.each([false, true])(
|
||||
'awaits an active timer send and its failure requeue %s',
|
||||
async (failSend) => {
|
||||
const email = `active-${failSend}@puter.local`;
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
const send = vi
|
||||
.spyOn(server.clients.email, 'sendRaw')
|
||||
.mockImplementation(async () => {
|
||||
await held;
|
||||
if (failSend) throw new Error('transport unavailable');
|
||||
return null;
|
||||
});
|
||||
await queueInvite(email);
|
||||
await vi.waitFor(() => expect(send).toHaveBeenCalledTimes(1));
|
||||
const list = async () =>
|
||||
server.stores.kv.list({
|
||||
as: 'keys',
|
||||
pattern: `share:digest:invite:${email}:`,
|
||||
});
|
||||
expect((await list()).res).toEqual([]);
|
||||
let finished = false;
|
||||
const draining = server.services.shareNotification
|
||||
.onServerPrepareShutdown()
|
||||
.then(() => {
|
||||
finished = true;
|
||||
});
|
||||
try {
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(finished).toBe(false);
|
||||
} finally {
|
||||
release();
|
||||
await draining;
|
||||
}
|
||||
expect((await list()).res).toHaveLength(failSend ? 1 : 0);
|
||||
},
|
||||
);
|
||||
|
||||
it('waits for an active send before draining entries queued meanwhile', async () => {
|
||||
let releaseFirst!: () => void;
|
||||
let releaseTail!: () => void;
|
||||
const first = new Promise<void>((resolve) => {
|
||||
releaseFirst = resolve;
|
||||
});
|
||||
const tail = new Promise<void>((resolve) => {
|
||||
releaseTail = resolve;
|
||||
});
|
||||
const send = vi
|
||||
.spyOn(server.clients.email, 'sendRaw')
|
||||
.mockImplementationOnce(async () => {
|
||||
await first;
|
||||
return null;
|
||||
})
|
||||
.mockImplementationOnce(async () => {
|
||||
await tail;
|
||||
return null;
|
||||
});
|
||||
await queueInvite('[email protected]');
|
||||
await vi.waitFor(() => expect(send).toHaveBeenCalledTimes(1));
|
||||
(
|
||||
server.services.shareNotification.config.share_notify_limits as {
|
||||
emailBatchSeconds: number;
|
||||
}
|
||||
).emailBatchSeconds = 600;
|
||||
await queueInvite('[email protected]', 8);
|
||||
let finished = false;
|
||||
const draining = server.services.shareNotification
|
||||
.onServerPrepareShutdown()
|
||||
.then(() => {
|
||||
finished = true;
|
||||
});
|
||||
try {
|
||||
releaseFirst();
|
||||
await vi.waitFor(() => expect(send).toHaveBeenCalledTimes(2));
|
||||
expect(finished).toBe(false);
|
||||
} finally {
|
||||
releaseFirst();
|
||||
releaseTail();
|
||||
await draining;
|
||||
}
|
||||
expect(send).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
it.each(['onServerPrepareShutdown', 'onServerShutdown'] as const)(
|
||||
'bounds %s while an active send is hung',
|
||||
async (hook) => {
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
const send = vi
|
||||
.spyOn(server.clients.email, 'sendRaw')
|
||||
.mockImplementationOnce(async () => {
|
||||
await held;
|
||||
return null;
|
||||
});
|
||||
await queueInvite(`hung-${hook}@puter.local`);
|
||||
await vi.waitFor(() => expect(send).toHaveBeenCalledTimes(1));
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
let finished = false;
|
||||
const draining = server.services.shareNotification[hook]().then(
|
||||
() => {
|
||||
finished = true;
|
||||
},
|
||||
);
|
||||
await vi.advanceTimersByTimeAsync(4_999);
|
||||
expect(finished).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
await draining;
|
||||
} finally {
|
||||
release();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('drains a digest queued after prepare at shutdown', async () => {
|
||||
await server.services.shareNotification.onServerPrepareShutdown();
|
||||
const send = vi
|
||||
.spyOn(server.clients.email, 'sendRaw')
|
||||
.mockResolvedValue(null);
|
||||
(
|
||||
server.services.shareNotification.config.share_notify_limits as {
|
||||
emailBatchSeconds: number;
|
||||
}
|
||||
).emailBatchSeconds = 600;
|
||||
await queueInvite('[email protected]');
|
||||
await server.services.shareNotification.onServerShutdown();
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
expect(send.mock.calls[0][0].to).toBe('[email protected]');
|
||||
await new Promise((resolve) => setTimeout(resolve, 30));
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -20,7 +20,11 @@
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import type { Actor } from '../../core/actor';
|
||||
import { checkRateLimit } from '../../core/http/middleware/rateLimit.js';
|
||||
import { runWithConcurrencyLimitSettled } from '../../util/concurrency.js';
|
||||
import {
|
||||
runWithConcurrencyLimitSettled,
|
||||
settleWithin,
|
||||
} from '../../util/concurrency.js';
|
||||
import { FINAL_FLUSH_TIMEOUT_MS } from '../../util/gracefulShutdown.js';
|
||||
import { PuterService } from '../types';
|
||||
import {
|
||||
digestItemPaths,
|
||||
@@ -178,6 +182,7 @@ export class ShareNotificationService extends PuterService {
|
||||
* queued sends live in KV, where any node's flush can pick them up.
|
||||
*/
|
||||
#digestTimers = new Map<string, ReturnType<typeof setTimeout>>();
|
||||
#digestFlushes = new Map<string, Promise<void>>();
|
||||
|
||||
/** The recovery sweep; see `onServerStart`. */
|
||||
#digestSweep: ReturnType<typeof setInterval> | null = null;
|
||||
@@ -199,12 +204,46 @@ export class ShareNotificationService extends PuterService {
|
||||
|
||||
/** Send what's still waiting while the transport is alive to send it. */
|
||||
override async onServerPrepareShutdown(): Promise<void> {
|
||||
await this.#drainDigests();
|
||||
}
|
||||
|
||||
override async onServerShutdown(): Promise<void> {
|
||||
await this.#drainDigests();
|
||||
}
|
||||
|
||||
async #drainDigests(): Promise<void> {
|
||||
if (this.#digestSweep) clearInterval(this.#digestSweep);
|
||||
this.#digestSweep = null;
|
||||
const keys = [...this.#digestTimers.keys()];
|
||||
const keys = new Set([
|
||||
...this.#digestTimers.keys(),
|
||||
...this.#digestFlushes.keys(),
|
||||
]);
|
||||
for (const [, timer] of this.#digestTimers) clearTimeout(timer);
|
||||
this.#digestTimers.clear();
|
||||
await Promise.all(keys.map((key) => this.#flushDigest(key)));
|
||||
const active = [...this.#digestFlushes.values()];
|
||||
const deadline = Date.now() + FINAL_FLUSH_TIMEOUT_MS;
|
||||
const drain = async () => {
|
||||
try {
|
||||
await Promise.allSettled(active);
|
||||
if (Date.now() >= deadline) return;
|
||||
for (const [key, timer] of this.#digestTimers) {
|
||||
keys.add(key);
|
||||
clearTimeout(timer);
|
||||
}
|
||||
this.#digestTimers.clear();
|
||||
await Promise.allSettled(
|
||||
[...keys].map((key) => this.#flushDigest(key)),
|
||||
);
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
'[share-notify] shutdown digest flush failed:',
|
||||
err,
|
||||
);
|
||||
}
|
||||
};
|
||||
if (!(await settleWithin(drain(), FINAL_FLUSH_TIMEOUT_MS))) {
|
||||
console.warn('[share-notify] final flush timed out');
|
||||
}
|
||||
}
|
||||
|
||||
/** The recovery sweep, for tests that can't wait out an interval. */
|
||||
@@ -878,25 +917,37 @@ export class ShareNotificationService extends PuterService {
|
||||
* everything queued), and across regions the KV `take` is the real claim,
|
||||
* handing each entry to exactly one flusher.
|
||||
*/
|
||||
async #flushDigest(key: string): Promise<void> {
|
||||
await withRedisLock(
|
||||
this.clients.redis,
|
||||
`share:digest:lock:${key}`,
|
||||
() => this.#flushDigestLocked(key),
|
||||
{
|
||||
ttlMs: DIGEST_LOCK_SECONDS * 1000,
|
||||
// No lock beats no mail; `take` still prevents double sends.
|
||||
onUnavailable: 'run',
|
||||
onBusy: () => {
|
||||
console.log(
|
||||
'[share-notify] another flush holds the lock:',
|
||||
{
|
||||
key,
|
||||
},
|
||||
);
|
||||
#flushDigest(key: string): Promise<void> {
|
||||
const active = this.#digestFlushes.get(key);
|
||||
if (active) return active;
|
||||
const flushing = this.#sendDigest(key);
|
||||
this.#digestFlushes.set(key, flushing);
|
||||
return flushing;
|
||||
}
|
||||
|
||||
async #sendDigest(key: string): Promise<void> {
|
||||
try {
|
||||
await withRedisLock(
|
||||
this.clients.redis,
|
||||
`share:digest:lock:${key}`,
|
||||
() => this.#flushDigestLocked(key),
|
||||
{
|
||||
ttlMs: DIGEST_LOCK_SECONDS * 1000,
|
||||
// No lock beats no mail; `take` still prevents double sends.
|
||||
onUnavailable: 'run',
|
||||
onBusy: () => {
|
||||
console.log(
|
||||
'[share-notify] another flush holds the lock:',
|
||||
{ key },
|
||||
);
|
||||
},
|
||||
},
|
||||
},
|
||||
);
|
||||
);
|
||||
} catch (err) {
|
||||
console.warn('[share-notify] digest flush failed:', err);
|
||||
} finally {
|
||||
this.#digestFlushes.delete(key);
|
||||
}
|
||||
}
|
||||
|
||||
async #flushDigestLocked(key: string): Promise<void> {
|
||||
|
||||
@@ -122,6 +122,59 @@ describe('SignupService.runSignupGate', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('an OIDC-only policy refuses form and temp signups before the validate hook, and lets OIDC and team provisioning through', async () => {
|
||||
const oidcOnly = (_k: string, data: unknown) => {
|
||||
Object.assign(data as object, { oidcOnly: true });
|
||||
};
|
||||
server.clients.event.on('puter.signup.policy', oidcOnly);
|
||||
try {
|
||||
const gate = (input: {
|
||||
isTemp: boolean;
|
||||
source?: 'oidc' | 'team_provision';
|
||||
}) =>
|
||||
capture('puter.signup.validate', () =>
|
||||
server.services.signup.runSignupGate({
|
||||
req,
|
||||
data: {},
|
||||
email: `policy-${slug()}@example.com`,
|
||||
bonusCode: null,
|
||||
...input,
|
||||
}),
|
||||
);
|
||||
const form = await gate({ isTemp: false });
|
||||
const temp = await gate({ isTemp: true });
|
||||
const oidc = await gate({ isTemp: false, source: 'oidc' });
|
||||
const team = await gate({
|
||||
isTemp: false,
|
||||
source: 'team_provision',
|
||||
});
|
||||
|
||||
expect(form.result).toMatchObject({
|
||||
verdict: 'blocked',
|
||||
code: 'signup_oidc_only',
|
||||
});
|
||||
expect(temp.result).toEqual({
|
||||
verdict: 'no_temp_user',
|
||||
message: null,
|
||||
code: null,
|
||||
});
|
||||
expect(form.seen).toHaveLength(0);
|
||||
expect(temp.seen).toHaveLength(0);
|
||||
expect(oidc.result.verdict).toBe('allowed');
|
||||
expect(team.result.verdict).toBe('allowed');
|
||||
expect(await server.services.signup.signupPolicy()).toEqual({
|
||||
oidcOnly: true,
|
||||
message: null,
|
||||
});
|
||||
} finally {
|
||||
server.clients.event.off('puter.signup.policy', oidcOnly);
|
||||
}
|
||||
expect(await server.services.signup.signupPolicy()).toEqual({
|
||||
oidcOnly: false,
|
||||
message: null,
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses an unknown bonus code before the validate hook runs', async () => {
|
||||
const { result, seen } = await capture('puter.signup.validate', () =>
|
||||
server.services.signup.runSignupGate({
|
||||
|
||||
@@ -32,7 +32,6 @@ import {
|
||||
checkSignupBonus,
|
||||
validateSignupBonus,
|
||||
} from '../../util/signupBonus.js';
|
||||
import type { SignupUtm } from '../../util/signupUtm.js';
|
||||
import { isUsernameTaken } from '../../util/username.js';
|
||||
import { isEmailGateLast } from '../../util/verificationOrder.js';
|
||||
import {
|
||||
@@ -84,13 +83,15 @@ export interface SignupInput {
|
||||
isTemp: boolean;
|
||||
fingerprint: string | null;
|
||||
bonusCode: string | null;
|
||||
utm?: SignupUtm | null;
|
||||
}
|
||||
|
||||
/** The trusted client address; `req.ip` honors `trust proxy`. */
|
||||
export const signupClientIp = (req: Request): string | null =>
|
||||
req.ip || req.socket?.remoteAddress || null;
|
||||
|
||||
const OIDC_ONLY_MESSAGE =
|
||||
'New accounts can only be created with a sign-in provider right now.';
|
||||
|
||||
/**
|
||||
* Account creation for every signup route (form, temp, OIDC) and email
|
||||
* confirmation for every confirmation route. Callers parse requests and set
|
||||
@@ -149,6 +150,30 @@ export class SignupService extends PuterService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The signup methods extensions allow right now; a failing hook restricts
|
||||
* nothing.
|
||||
*/
|
||||
async signupPolicy(): Promise<{
|
||||
oidcOnly: boolean;
|
||||
message: string | null;
|
||||
}> {
|
||||
const policy = { oidcOnly: false, message: null as string | null };
|
||||
try {
|
||||
await this.clients.event?.emitAndWait(
|
||||
'puter.signup.policy',
|
||||
policy,
|
||||
{},
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[signup] policy hook failed:', e);
|
||||
}
|
||||
return {
|
||||
oidcOnly: policy.oidcOnly === true,
|
||||
message: typeof policy.message === 'string' ? policy.message : null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The extension-owned verdict on a signup: a dead bonus code is refused
|
||||
* first (before `puter.signup.validate` records the attempt), then the
|
||||
@@ -159,6 +184,23 @@ export class SignupService extends PuterService {
|
||||
const clientIp = signupClientIp(req);
|
||||
const fingerprint = input.fingerprint ?? null;
|
||||
|
||||
// Only form and temp signups carry no source; OIDC and team
|
||||
// provisioning are never restricted by the policy.
|
||||
if (!input.source) {
|
||||
const policy = await this.signupPolicy();
|
||||
if (policy.oidcOnly) {
|
||||
// A refused temp signup sends the GUI to its signup window,
|
||||
// which then offers only the providers.
|
||||
return isTemp
|
||||
? { verdict: 'no_temp_user', message: null, code: null }
|
||||
: {
|
||||
verdict: 'blocked',
|
||||
message: policy.message ?? OIDC_ONLY_MESSAGE,
|
||||
code: 'signup_oidc_only',
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
if (
|
||||
bonusCode &&
|
||||
!(
|
||||
@@ -280,7 +322,7 @@ export class SignupService extends PuterService {
|
||||
* refused signup answers with.
|
||||
*/
|
||||
async signup(input: SignupInput): Promise<UserRow> {
|
||||
const { req, body, isTemp, fingerprint, bonusCode, utm } = input;
|
||||
const { req, body, isTemp, fingerprint, bonusCode } = input;
|
||||
|
||||
// Runs before the duplicate checks so a disabled endpoint doesn't
|
||||
// reveal which usernames or emails exist. Claiming a pre-existing
|
||||
@@ -549,7 +591,6 @@ export class SignupService extends PuterService {
|
||||
// The created/claimed row: a claimed placeholder has credentials.
|
||||
isTemp: user.password === null && user.email === null,
|
||||
saveAccount: !isTemp,
|
||||
utm,
|
||||
});
|
||||
return user;
|
||||
}
|
||||
@@ -569,7 +610,6 @@ export class SignupService extends PuterService {
|
||||
/** The provider verified the address, so this is its confirmation. */
|
||||
emailConfirmed?: boolean;
|
||||
saveAccount: boolean;
|
||||
utm?: SignupUtm | null;
|
||||
},
|
||||
): void {
|
||||
if (opts.emailConfirmed) this.#emitEmailConfirmed(user);
|
||||
@@ -591,7 +631,6 @@ export class SignupService extends PuterService {
|
||||
// counters are written under one key and read under another.
|
||||
ip: opts.ip,
|
||||
...(opts.bonusCode ? { bonus_code: opts.bonusCode } : {}),
|
||||
...(opts.utm ? { utm: opts.utm } : {}),
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
@@ -801,7 +801,7 @@ export class AppStore extends PuterStore {
|
||||
* the row stays unclaimed until a developer registers an app for the same
|
||||
* origin and AppDriver's merge path takes ownership. Marker shape (`name
|
||||
* === uid && title === uid` + description prefix) is what
|
||||
* `#isOriginBootstrapApp` checks.
|
||||
* `isOriginBootstrapApp` (util/appView) checks.
|
||||
*/
|
||||
async createFromOrigin(
|
||||
uid,
|
||||
|
||||
@@ -265,6 +265,114 @@ describe('creating a subscription', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('handler scope', () => {
|
||||
const appUid = 'app-scoped';
|
||||
|
||||
it('round-trips through create and every select path', async () => {
|
||||
const { row: user } = await durable().create(
|
||||
input({ appUid, handlerName: 'h', handlerScope: 'user' }),
|
||||
);
|
||||
const { row: app } = await durable().create(
|
||||
input({ appUid, handlerName: 'h' }),
|
||||
);
|
||||
expect(user.handlerScope).toBe('user');
|
||||
expect(app).not.toHaveProperty('handlerScope');
|
||||
|
||||
await expect(durable().getBySubId(user.subId)).resolves.toMatchObject({
|
||||
handlerScope: 'user',
|
||||
});
|
||||
await expect(
|
||||
durable().getBySubId(app.subId),
|
||||
).resolves.not.toHaveProperty('handlerScope');
|
||||
const listed = await durable().listForHolder(userId, { appUid });
|
||||
expect(
|
||||
new Map(listed.items.map((r) => [r.subId, r.handlerScope])),
|
||||
).toEqual(
|
||||
new Map<string, string | undefined>([
|
||||
[user.subId, 'user'],
|
||||
[app.subId, undefined],
|
||||
]),
|
||||
);
|
||||
const stored = await env.server.clients.db.read(
|
||||
'SELECT `sub_id`, `handler_scope` FROM `event_subscriptions` WHERE `app_uid` = ?',
|
||||
[appUid],
|
||||
);
|
||||
expect(
|
||||
Object.fromEntries(stored.map((r) => [r.sub_id, r.handler_scope])),
|
||||
).toEqual({ [user.subId]: 1, [app.subId]: 0 });
|
||||
});
|
||||
|
||||
it('lists by scope and holder, never across scopes', async () => {
|
||||
const { row: appRow } = await durable().create(
|
||||
input({ appUid, handlerName: 'h' }),
|
||||
);
|
||||
const { row: mine } = await durable().create(
|
||||
input({ appUid, handlerName: 'h', handlerScope: 'user' }),
|
||||
);
|
||||
const { row: theirs } = await durable().create(
|
||||
input({
|
||||
appUid,
|
||||
handlerName: 'h',
|
||||
handlerScope: 'user',
|
||||
holderUserId: otherUserId,
|
||||
ownerUserId: otherUserId,
|
||||
token: `f#other-${uuidv4()}`,
|
||||
}),
|
||||
);
|
||||
const ids = async (scope: 'app' | 'user', holder: number, name = 'h') =>
|
||||
(await durable().listByHandler(appUid, scope, holder, name))
|
||||
.map((r) => r.subId)
|
||||
.sort();
|
||||
|
||||
expect(await ids('app', 0)).toEqual([appRow.subId]);
|
||||
// Holder is ignored for app scope.
|
||||
expect(await ids('app', userId)).toEqual([appRow.subId]);
|
||||
expect(await ids('user', userId)).toEqual([mine.subId]);
|
||||
expect(await ids('user', otherUserId)).toEqual([theirs.subId]);
|
||||
expect(await ids('user', userId, 'missing')).toEqual([]);
|
||||
});
|
||||
|
||||
it('lists the suspended half of a user scope', async () => {
|
||||
const { row } = await durable().create(
|
||||
input({ appUid, handlerName: 'h', handlerScope: 'user' }),
|
||||
);
|
||||
await env.server.clients.db.write(
|
||||
'UPDATE `event_subscriptions` SET `suspended_at` = 1, `suspended_reason` = ? WHERE `sub_id` = ?',
|
||||
['handler_not_found', row.subId],
|
||||
);
|
||||
await expect(
|
||||
durable().listByHandler(appUid, 'user', userId, 'h'),
|
||||
).resolves.toEqual([]);
|
||||
await expect(
|
||||
durable().listByHandler(appUid, 'user', userId, 'h', {
|
||||
suspendedReason: 'handler_not_found',
|
||||
}),
|
||||
).resolves.toEqual([expect.objectContaining({ subId: row.subId })]);
|
||||
});
|
||||
|
||||
it('reads a user scope through the holder index', async () => {
|
||||
const pread = vi.spyOn(env.server.clients.db, 'pread');
|
||||
try {
|
||||
await durable().listByHandler(appUid, 'user', userId, 'h');
|
||||
const [statement, params] = pread.mock.calls[0]!;
|
||||
const plan = await env.server.clients.db.read(
|
||||
`EXPLAIN QUERY PLAN ${statement}`,
|
||||
params,
|
||||
);
|
||||
const detail = plan.map((r) => String(r.detail)).join('\n');
|
||||
expect(detail).toContain('idx_event_subscriptions_holder');
|
||||
expect(detail).not.toContain('idx_event_subscriptions_app_handler');
|
||||
} finally {
|
||||
pread.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('carries the scope through the schedule join', async () => {
|
||||
const join = durable().scheduleSubscriptionJoin('s', 'x.`sub_id`');
|
||||
expect(join.columns).toContain('`sub_handler_scope`');
|
||||
});
|
||||
});
|
||||
|
||||
describe('validation at the row write', () => {
|
||||
it('refuses a context past the hard cap', async () => {
|
||||
await expect(
|
||||
@@ -1099,7 +1207,7 @@ describe('rows a schedule owns', () => {
|
||||
it('are found by handler and by app, like any other row', async () => {
|
||||
const { row } = await durable().create(scheduleInput());
|
||||
await expect(
|
||||
durable().listByHandler('app-scheduler', 'tick'),
|
||||
durable().listByHandler('app-scheduler', 'app', 0, 'tick'),
|
||||
).resolves.toEqual([expect.objectContaining({ subId: row.subId })]);
|
||||
await expect(
|
||||
durable().listHolderIdsForApp('app-scheduler'),
|
||||
|
||||
@@ -44,6 +44,7 @@ import {
|
||||
SCHEDULE_TOKEN_PREFIX,
|
||||
scheduleToken,
|
||||
type DurableSubscription,
|
||||
type HandlerScopeKind,
|
||||
type SubscriptionPermission,
|
||||
} from './types.js';
|
||||
|
||||
@@ -100,6 +101,8 @@ export interface DurableSubscriptionInput {
|
||||
op: FsOp | null;
|
||||
delivery: DeliveryClass;
|
||||
handlerName: string | null;
|
||||
/** Which set `handlerName` refers to. Omitted means the app's. */
|
||||
handlerScope?: HandlerScopeKind;
|
||||
context: string | null;
|
||||
permission: SubscriptionPermission;
|
||||
expiresAt: number | null;
|
||||
@@ -307,6 +310,9 @@ const toRow = (row: Record<string, unknown>): DurableSubscription => ({
|
||||
row.handler_name === null || row.handler_name === undefined
|
||||
? null
|
||||
: String(row.handler_name),
|
||||
...(Number(row.handler_scope) === 1
|
||||
? { handlerScope: 'user' as const }
|
||||
: {}),
|
||||
context:
|
||||
row.context === null || row.context === undefined
|
||||
? null
|
||||
@@ -324,7 +330,7 @@ const toRow = (row: Record<string, unknown>): DurableSubscription => ({
|
||||
const SELECT_COLUMNS =
|
||||
'`id`, `sub_id`, `token`, `owner_user_id`, `holder_user_id`, `app_uid`, ' +
|
||||
'`subject`, `anchor_uid`, `anchor_path`, `match`, `delivery`, `ops`, ' +
|
||||
'`handler_name`, `context`, `permission`, `include_value`, ' +
|
||||
'`handler_name`, `handler_scope`, `context`, `permission`, `include_value`, ' +
|
||||
'`suspended_at`, `suspended_reason`, `expires_at`, `created_at`';
|
||||
|
||||
export class DurableSubscriptionStore extends PuterStore {
|
||||
@@ -376,6 +382,9 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
permission: input.permission,
|
||||
delivery: input.delivery,
|
||||
handlerName: input.handlerName,
|
||||
...(input.handlerScope === 'user'
|
||||
? { handlerScope: 'user' as const }
|
||||
: {}),
|
||||
context: input.context,
|
||||
...(input.includeValue ? { includeValue: true } : {}),
|
||||
expiresAt: input.expiresAt,
|
||||
@@ -398,6 +407,7 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
delivery: row.delivery,
|
||||
ops: row.op,
|
||||
handler_name: row.handlerName,
|
||||
handler_scope: row.handlerScope === 'user' ? 1 : 0,
|
||||
targets: legacyTargets(row.appUid),
|
||||
context: row.context,
|
||||
permission: row.permission,
|
||||
@@ -456,7 +466,10 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
async suspend(
|
||||
rows: readonly DurableSubscription[],
|
||||
reason: SuspendedReason,
|
||||
opts: { override?: readonly SuspendedReason[] } = {},
|
||||
opts: {
|
||||
override?: readonly SuspendedReason[];
|
||||
guard?: SqlGuard;
|
||||
} = {},
|
||||
): Promise<{ suspended: DurableSubscription[]; bumps: GenerationBump[] }> {
|
||||
if (rows.length === 0) return { suspended: [], bumps: [] };
|
||||
|
||||
@@ -478,8 +491,17 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
const written = await this.clients.db.write(
|
||||
`UPDATE \`${TABLE}\` SET \`suspended_at\` = ?, ` +
|
||||
'`suspended_reason` = ? ' +
|
||||
`WHERE \`sub_id\` = ? AND ${condition}`,
|
||||
[at, reason, row.subId, ...override],
|
||||
`WHERE \`sub_id\` = ? AND ${condition}` +
|
||||
(opts.guard
|
||||
? ` AND EXISTS (SELECT 1 FROM \`${opts.guard.table}\` WHERE ${opts.guard.where})`
|
||||
: ''),
|
||||
[
|
||||
at,
|
||||
reason,
|
||||
row.subId,
|
||||
...override,
|
||||
...(opts.guard?.values ?? []),
|
||||
],
|
||||
);
|
||||
if (written.anyRowsAffected)
|
||||
suspended.push({
|
||||
@@ -811,8 +833,11 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* The rows bound to one of an app's handler names. What a removal has to
|
||||
* The rows bound to one of a scope's handler names. What a removal has to
|
||||
* suspend, and — asking for the suspended half — what a republish resumes.
|
||||
* `holder` is read for user scope only. App scope never returns user-scope
|
||||
* rows; user scope is read through the holder index, so it never walks the
|
||||
* other holders of a popular app.
|
||||
*
|
||||
* Bounded by the per-account quota times nothing: a widely-used handler can
|
||||
* carry more rows than one read should return, so this is a page and the
|
||||
@@ -820,11 +845,28 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
*/
|
||||
async listByHandler(
|
||||
appUid: string,
|
||||
scope: HandlerScopeKind,
|
||||
holder: number,
|
||||
handlerName: string,
|
||||
options: { suspendedReason?: SuspendedReason; limit?: number } = {},
|
||||
): Promise<DurableSubscription[]> {
|
||||
const where = ['`app_uid` = ?', '`handler_name` = ?'];
|
||||
const params: unknown[] = [appUid, handlerName];
|
||||
const where =
|
||||
scope === 'user'
|
||||
? [
|
||||
'`holder_user_id` = ?',
|
||||
'`app_uid` = ?',
|
||||
'`handler_scope` = 1',
|
||||
'`handler_name` = ?',
|
||||
]
|
||||
: [
|
||||
'`app_uid` = ?',
|
||||
'`handler_scope` = 0',
|
||||
'`handler_name` = ?',
|
||||
];
|
||||
const params: unknown[] =
|
||||
scope === 'user'
|
||||
? [holder, appUid, handlerName]
|
||||
: [appUid, handlerName];
|
||||
if (options.suspendedReason === undefined) {
|
||||
where.push('`suspended_at` IS NULL');
|
||||
} else {
|
||||
@@ -929,6 +971,20 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
return this.#reap(await this.#listForApp(appUid, batchSize));
|
||||
}
|
||||
|
||||
scheduleSubscriptionContextEntry(
|
||||
scheduleUid: string,
|
||||
context: string | null,
|
||||
guard: SqlGuard,
|
||||
): BatchEntry {
|
||||
assertDurableContext(context);
|
||||
return {
|
||||
statement:
|
||||
`UPDATE \`${TABLE}\` SET \`context\` = ? WHERE \`token\` = ? ` +
|
||||
`AND EXISTS (SELECT 1 FROM \`${guard.table}\` WHERE ${guard.where})`,
|
||||
values: [context, scheduleToken(scheduleUid), ...guard.values],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete the row a schedule owns, whether or not the schedule ever recorded
|
||||
* its id: a create that crashed between the two inserts leaves one it
|
||||
@@ -965,7 +1021,8 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
/**
|
||||
* A `LEFT JOIN` of the row a schedule owns, as `alias`, on `subIdColumn`,
|
||||
* and the columns it selects: `sub_row_id`, `sub_suspended_at`,
|
||||
* `sub_suspended_reason`, `sub_handler_name` and `sub_context`.
|
||||
* `sub_suspended_reason`, `sub_handler_name`, `sub_handler_scope` and
|
||||
* `sub_context`.
|
||||
*/
|
||||
scheduleSubscriptionJoin(
|
||||
alias: string,
|
||||
@@ -977,6 +1034,7 @@ export class DurableSubscriptionStore extends PuterStore {
|
||||
`${alias}.\`suspended_at\` AS \`sub_suspended_at\`, ` +
|
||||
`${alias}.\`suspended_reason\` AS \`sub_suspended_reason\`, ` +
|
||||
`${alias}.\`handler_name\` AS \`sub_handler_name\`, ` +
|
||||
`${alias}.\`handler_scope\` AS \`sub_handler_scope\`, ` +
|
||||
`${alias}.\`context\` AS \`sub_context\``,
|
||||
join: `LEFT JOIN \`${TABLE}\` ${alias} ON ${alias}.\`sub_id\` = ${subIdColumn}`,
|
||||
};
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,392 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/**
|
||||
* The user-scope handler schema and queries against MySQL 8.4. Runs only when
|
||||
* `PUTER_TEST_MYSQL_PORT` points at a server (root / `puter`).
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import mysql from 'mysql2/promise';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { splitMysqlStatements } from '../../clients/database/splitMysqlStatements.js';
|
||||
import type { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { appHandlerScope, hashContent } from './EventHandlerStore.js';
|
||||
|
||||
const connection = {
|
||||
host: '127.0.0.1',
|
||||
port: Number(process.env.PUTER_TEST_MYSQL_PORT ?? 3307),
|
||||
user: 'root',
|
||||
password: 'puter',
|
||||
};
|
||||
const database = `event_user_handlers_${Date.now().toString(36)}`;
|
||||
const migrationDir = fileURLToPath(
|
||||
new URL('../../clients/database/migrations/mysql', import.meta.url),
|
||||
);
|
||||
const SOURCE = 'async ({ event }) => { console.log(event.path); }';
|
||||
|
||||
describe.skipIf(!process.env.PUTER_TEST_MYSQL_PORT)(
|
||||
'user-scoped handlers (MySQL)',
|
||||
() => {
|
||||
let server: PuterServer;
|
||||
let conn: mysql.Connection;
|
||||
const userIds: number[] = [];
|
||||
|
||||
const handlers = () => server.stores.eventHandler;
|
||||
const rows = async (sql: string, params: unknown[] = []) =>
|
||||
(await conn.query(sql, params))[0] as Record<string, unknown>[];
|
||||
|
||||
const applyMig55 = async () => {
|
||||
const sql = readFileSync(
|
||||
`${migrationDir}/mysql_mig_55.sql`,
|
||||
'utf8',
|
||||
);
|
||||
for (const statement of splitMysqlStatements(sql))
|
||||
await conn.query(statement);
|
||||
};
|
||||
|
||||
const bind = async (
|
||||
holder: number,
|
||||
app: string,
|
||||
name: string,
|
||||
handlerScope: number,
|
||||
suspended = false,
|
||||
) => {
|
||||
await conn.query(
|
||||
'INSERT INTO `event_subscriptions` (`sub_id`, `token`, `owner_user_id`, `holder_user_id`, `app_uid`, `subject`, `anchor_uid`, `anchor_path`, `delivery`, `handler_name`, `handler_scope`, `targets`, `permission`, `suspended_at`, `suspended_reason`, `created_at`) ' +
|
||||
"VALUES (?, ?, ?, ?, ?, 'fs:/x', 'a', '/x', 'single', ?, ?, '[]', 'list', ?, ?, 1)",
|
||||
[
|
||||
uuidv4(),
|
||||
`f#${uuidv4()}`,
|
||||
holder,
|
||||
holder,
|
||||
app,
|
||||
name,
|
||||
handlerScope,
|
||||
suspended ? 1 : null,
|
||||
suspended ? 'handler_not_found' : null,
|
||||
],
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* The `key` column of every table in the plan of the last statement a
|
||||
* spy saw.
|
||||
*/
|
||||
const planKeys = async (spy: {
|
||||
mock: { calls: unknown[][] };
|
||||
}): Promise<Record<string, string | null>> => {
|
||||
const [statement, params] = spy.mock.calls.at(-1) as [
|
||||
string,
|
||||
unknown[],
|
||||
];
|
||||
const plan = await rows(`EXPLAIN ${statement}`, params);
|
||||
return Object.fromEntries(
|
||||
plan.map((row) => [
|
||||
String(row.table),
|
||||
(row.key as string | null) ?? null,
|
||||
]),
|
||||
);
|
||||
};
|
||||
|
||||
beforeAll(async () => {
|
||||
const admin = await mysql.createConnection(connection);
|
||||
await admin.query(`CREATE DATABASE \`${database}\``);
|
||||
await admin.end();
|
||||
server = await setupTestServer({
|
||||
database: {
|
||||
engine: 'mysql',
|
||||
...connection,
|
||||
database,
|
||||
migrationPaths: [migrationDir],
|
||||
},
|
||||
events: { enabled: true },
|
||||
} as never);
|
||||
conn = await mysql.createConnection({ ...connection, database });
|
||||
for (let i = 0; i < 12; i++) {
|
||||
const user = await server.stores.user.create({
|
||||
username: `ush-${uuidv4().slice(0, 8)}`,
|
||||
uuid: uuidv4(),
|
||||
password: null,
|
||||
email: null,
|
||||
});
|
||||
userIds.push(user.id);
|
||||
}
|
||||
}, 240_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await conn?.end();
|
||||
await server?.shutdown();
|
||||
const admin = await mysql.createConnection(connection);
|
||||
await admin.query(`DROP DATABASE IF EXISTS \`${database}\``);
|
||||
await admin.end();
|
||||
});
|
||||
|
||||
it('runs on MySQL 8.4', async () => {
|
||||
const [row] = await rows('SELECT VERSION() AS version');
|
||||
expect(String(row!.version)).toMatch(/^8\.4\./);
|
||||
});
|
||||
|
||||
it('applies over existing subscription rows, and a replay is a no-op', async () => {
|
||||
// Back to the state before the migration, with rows already in.
|
||||
await conn.query('DROP TABLE `event_user_handlers`');
|
||||
await conn.query(
|
||||
'ALTER TABLE `event_subscriptions` DROP COLUMN `handler_scope`',
|
||||
);
|
||||
await conn.query('DELETE FROM `event_subscriptions`');
|
||||
for (const name of ['a', 'b', 'c'])
|
||||
await conn.query(
|
||||
'INSERT INTO `event_subscriptions` (`sub_id`, `token`, `owner_user_id`, `holder_user_id`, `app_uid`, `subject`, `anchor_uid`, `anchor_path`, `delivery`, `handler_name`, `targets`, `permission`, `created_at`) ' +
|
||||
"VALUES (?, ?, ?, ?, 'app-old', 'fs:/x', 'a', '/x', 'single', ?, '[]', 'list', 1)",
|
||||
[uuidv4(), `f#${uuidv4()}`, userIds[0], userIds[0], name],
|
||||
);
|
||||
|
||||
await applyMig55();
|
||||
const scopes = async () =>
|
||||
(
|
||||
await rows(
|
||||
'SELECT `handler_scope` FROM `event_subscriptions`',
|
||||
)
|
||||
).map((row) => Number(row.handler_scope));
|
||||
expect(await scopes()).toEqual([0, 0, 0]);
|
||||
|
||||
await applyMig55();
|
||||
expect(await scopes()).toEqual([0, 0, 0]);
|
||||
|
||||
const [column] = await rows(
|
||||
"SELECT COLUMN_TYPE, IS_NULLABLE, COLUMN_DEFAULT FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'event_subscriptions' AND COLUMN_NAME = 'handler_scope'",
|
||||
);
|
||||
expect(column).toMatchObject({
|
||||
COLUMN_TYPE: 'tinyint unsigned',
|
||||
IS_NULLABLE: 'NO',
|
||||
COLUMN_DEFAULT: '0',
|
||||
});
|
||||
|
||||
const indexes = await rows(
|
||||
"SELECT INDEX_NAME, NON_UNIQUE, SEQ_IN_INDEX, COLUMN_NAME FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'event_user_handlers' ORDER BY INDEX_NAME, SEQ_IN_INDEX",
|
||||
);
|
||||
const byIndex = (name: string) =>
|
||||
indexes
|
||||
.filter((row) => row.INDEX_NAME === name)
|
||||
.map((row) => String(row.COLUMN_NAME));
|
||||
expect(byIndex('idx_event_user_handlers_app_holder_name')).toEqual([
|
||||
'app_uid',
|
||||
'holder_user_id',
|
||||
'name',
|
||||
]);
|
||||
expect(byIndex('idx_event_user_handlers_holder_app')).toEqual([
|
||||
'holder_user_id',
|
||||
'app_uid',
|
||||
]);
|
||||
expect(
|
||||
indexes.find(
|
||||
(row) =>
|
||||
row.INDEX_NAME ===
|
||||
'idx_event_user_handlers_app_holder_name',
|
||||
),
|
||||
).toMatchObject({ NON_UNIQUE: 0 });
|
||||
|
||||
const [appUidColumn] = await rows(
|
||||
"SELECT CHARACTER_SET_NAME, COLLATION_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'event_user_handlers' AND COLUMN_NAME = 'app_uid'",
|
||||
);
|
||||
expect(appUidColumn).toMatchObject({
|
||||
CHARACTER_SET_NAME: 'latin1',
|
||||
COLLATION_NAME: 'latin1_swedish_ci',
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps the same name once in the app scope and once per holder', async () => {
|
||||
const app = `app-${uuidv4()}`;
|
||||
const insert = (holder: number) =>
|
||||
conn.query(
|
||||
'INSERT INTO `event_user_handlers` (`app_uid`, `holder_user_id`, `name`, `source`, `source_hash`, `created_at`, `updated_at`, `scope_since`) VALUES (?, ?, ?, ?, ?, 1, 1, 1)',
|
||||
[app, holder, 'daily', SOURCE, hashContent(SOURCE)],
|
||||
);
|
||||
await handlers().publish({
|
||||
scope: appHandlerScope(app),
|
||||
name: 'daily',
|
||||
source: SOURCE,
|
||||
});
|
||||
await insert(userIds[1]!);
|
||||
await insert(userIds[2]!);
|
||||
await expect(insert(userIds[1]!)).rejects.toMatchObject({
|
||||
code: 'ER_DUP_ENTRY',
|
||||
});
|
||||
});
|
||||
|
||||
it('runs the store against both tables', async () => {
|
||||
const app = `app-${uuidv4()}`;
|
||||
const [a, b] = [userIds[3]!, userIds[4]!];
|
||||
const scope = (holder: number) => ({ appUid: app, holder });
|
||||
await handlers().publish({
|
||||
scope: appHandlerScope(app),
|
||||
name: 'daily',
|
||||
source: SOURCE,
|
||||
});
|
||||
for (const holder of [a, b])
|
||||
await handlers().publish({
|
||||
scope: scope(holder),
|
||||
name: 'daily',
|
||||
source: `${SOURCE} // ${holder}`,
|
||||
});
|
||||
await handlers().publish({
|
||||
scope: scope(a),
|
||||
name: `@${hashContent(SOURCE)}`,
|
||||
source: SOURCE,
|
||||
});
|
||||
|
||||
expect(await handlers().countForApp(scope(a))).toBe(2);
|
||||
expect(await handlers().countForApp(appHandlerScope(app))).toBe(1);
|
||||
expect(await handlers().totalSourceBytesForApp(scope(b))).toBe(
|
||||
Buffer.byteLength(`${SOURCE} // ${b}`),
|
||||
);
|
||||
expect(
|
||||
(await handlers().setForApp(scope(a), { replica: true })).map(
|
||||
(h) => h.name,
|
||||
),
|
||||
).toEqual([`@${hashContent(SOURCE)}`, 'daily']);
|
||||
expect(await handlers().countUserScopesWithHandlersForUser(a)).toBe(
|
||||
1,
|
||||
);
|
||||
expect(
|
||||
await handlers().countUserScopesWithHandlersForUser(a, {
|
||||
createdBefore: 1_000,
|
||||
}),
|
||||
).toBe(0);
|
||||
expect(
|
||||
await handlers().countUserScopesWithHandlersForUser(a, {
|
||||
createdBefore: Date.now() + 60_000,
|
||||
}),
|
||||
).toBe(1);
|
||||
const page = await handlers().listUserScopesForUser(a);
|
||||
expect(page.total).toBe(1);
|
||||
expect(page.items).toEqual([
|
||||
expect.objectContaining({ appUid: app, handlerCount: 2 }),
|
||||
]);
|
||||
expect(
|
||||
(await handlers().listUserScopes(app)).items.map(
|
||||
(s) => s.holder,
|
||||
),
|
||||
).toEqual([a, b].sort((x, y) => x - y));
|
||||
|
||||
await bind(a, app, 'daily', 1);
|
||||
await bind(b, app, 'daily', 1, true);
|
||||
await bind(a, app, 'daily', 0);
|
||||
expect(
|
||||
await handlers().countSubscriptionsByHandler(scope(a)),
|
||||
).toEqual(new Map([['daily', 1]]));
|
||||
expect(
|
||||
await handlers().countSubscriptionsByHandler(
|
||||
appHandlerScope(app),
|
||||
),
|
||||
).toEqual(new Map([['daily', 1]]));
|
||||
|
||||
await conn.query(
|
||||
'UPDATE `event_user_handlers` SET `updated_at` = 1 WHERE `app_uid` = ?',
|
||||
[app],
|
||||
);
|
||||
const found = await handlers().listUnboundUserHandlers(1_000);
|
||||
expect(
|
||||
found.items
|
||||
.filter((item) => item.appUid === app)
|
||||
.map((item) => [item.holder, item.name]),
|
||||
).toEqual([[a, `@${hashContent(SOURCE)}`]]);
|
||||
await expect(
|
||||
handlers().deleteUserHandlerIfUnbound(scope(a), 'daily', 1_000),
|
||||
).resolves.toBe(false);
|
||||
await expect(
|
||||
handlers().deleteUserHandlerIfUnbound(
|
||||
scope(a),
|
||||
`@${hashContent(SOURCE)}`,
|
||||
1_000,
|
||||
),
|
||||
).resolves.toBe(true);
|
||||
|
||||
await expect(handlers().deleteUserScopesForHolder(b)).resolves.toBe(
|
||||
1,
|
||||
);
|
||||
await expect(handlers().deleteForApp(app)).resolves.toBe(2);
|
||||
expect(await handlers().countForApp(scope(a))).toBe(0);
|
||||
});
|
||||
|
||||
it('serves the user-scope reads from the holder-leading indexes', async () => {
|
||||
const app = `app-${uuidv4()}`;
|
||||
// A popular handler: many holders on one (app, name), so the
|
||||
// (app_uid, handler_name) index is the wrong one to pick.
|
||||
for (const holder of userIds)
|
||||
for (let i = 0; i < 80; i++)
|
||||
await bind(holder, app, 'popular', 1);
|
||||
await handlers().publish({
|
||||
scope: { appUid: app, holder: userIds[0]! },
|
||||
name: 'popular',
|
||||
source: SOURCE,
|
||||
});
|
||||
await conn.query('ANALYZE TABLE `event_subscriptions`');
|
||||
await conn.query('ANALYZE TABLE `event_user_handlers`');
|
||||
|
||||
const pread = vi.spyOn(server.clients.db, 'pread');
|
||||
const read = vi.spyOn(server.clients.db, 'read');
|
||||
try {
|
||||
await server.stores.durableSubscription.listByHandler(
|
||||
app,
|
||||
'user',
|
||||
userIds[0]!,
|
||||
'popular',
|
||||
);
|
||||
expect(await planKeys(pread)).toEqual({
|
||||
event_subscriptions: 'idx_event_subscriptions_holder',
|
||||
});
|
||||
|
||||
await handlers().countSubscriptionsByHandler({
|
||||
appUid: app,
|
||||
holder: userIds[0]!,
|
||||
});
|
||||
expect(await planKeys(read)).toEqual({
|
||||
event_subscriptions: 'idx_event_subscriptions_holder',
|
||||
});
|
||||
|
||||
await handlers().countUserScopesWithHandlersForUser(
|
||||
userIds[0]!,
|
||||
{
|
||||
createdBefore: Date.now(),
|
||||
},
|
||||
);
|
||||
expect((await planKeys(pread)).event_user_handlers).toBe(
|
||||
'idx_event_user_handlers_holder_app',
|
||||
);
|
||||
|
||||
await handlers().listUserScopesForUser(userIds[0]!);
|
||||
expect((await planKeys(read)).event_user_handlers).toBe(
|
||||
'idx_event_user_handlers_holder_app',
|
||||
);
|
||||
|
||||
await handlers().listUnboundUserHandlers(1_000);
|
||||
expect((await planKeys(read)).s).toBe(
|
||||
'idx_event_subscriptions_holder',
|
||||
);
|
||||
} finally {
|
||||
pread.mockRestore();
|
||||
read.mockRestore();
|
||||
}
|
||||
}, 120_000);
|
||||
},
|
||||
);
|
||||
@@ -21,6 +21,7 @@ import { createHash } from 'node:crypto';
|
||||
import {
|
||||
EVENTS_HANDLERS_PER_APP,
|
||||
EVENTS_HANDLER_SOURCE_MAX_BYTES,
|
||||
EVENTS_USER_HANDLERS_PER_SCOPE,
|
||||
} from '../../controllers/events/limits.js';
|
||||
import { HttpError } from '../../core/http/HttpError.js';
|
||||
import {
|
||||
@@ -47,6 +48,7 @@ import { PuterStore } from '../types.js';
|
||||
*/
|
||||
|
||||
const TABLE = 'event_handlers';
|
||||
const USER_TABLE = 'event_user_handlers';
|
||||
const SUBSCRIPTION_TABLE = 'event_subscriptions';
|
||||
const APP_TABLE = 'apps';
|
||||
|
||||
@@ -57,12 +59,24 @@ const APP_TABLE = 'apps';
|
||||
export const EVENTS_WORKERS_LIST_DEFAULT_LIMIT = 50;
|
||||
export const EVENTS_WORKERS_LIST_LIMIT_CAP = 200;
|
||||
|
||||
/** Default page of `listUserScopes`, and of one GC read. */
|
||||
const USER_SCOPES_PAGE_DEFAULT = 500;
|
||||
const USER_SCOPES_PAGE_CAP = 1000;
|
||||
const UNBOUND_HANDLERS_BATCH = 200;
|
||||
|
||||
/** Longest a handler name may be, matching the column that holds it. */
|
||||
export const HANDLER_NAME_MAX_LENGTH = 128;
|
||||
|
||||
/** Names are addressable identifiers, not free text. */
|
||||
const NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.:-]*$/;
|
||||
|
||||
/**
|
||||
* A server-minted name for an inline handler: `@` and the source hash. `@` is
|
||||
* outside {@link NAME_PATTERN}, so a minted name never collides with a chosen
|
||||
* one, and only a user scope accepts it.
|
||||
*/
|
||||
const INLINE_NAME_PATTERN = /^@[0-9a-f]{64}$/;
|
||||
|
||||
/**
|
||||
* Longest a delivery may address an app's handler set from cache. Every write
|
||||
* here drops the entry on every node, so this only bounds a read that raced a
|
||||
@@ -70,15 +84,66 @@ const NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.:-]*$/;
|
||||
*/
|
||||
export const HANDLER_SET_CACHE_TTL_SECONDS = 30;
|
||||
|
||||
const handlerSetCacheKey = (appUid: string): string =>
|
||||
`eventHandlers:set:${appUid}`;
|
||||
// -- Scopes -----------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Whose handlers a call is about: an app's own set (`holder` 0), or the
|
||||
* handlers one user published into their scope of that app. The two live in
|
||||
* separate tables, so code that predates user scopes never reads one.
|
||||
*/
|
||||
export interface HandlerScope {
|
||||
appUid: string;
|
||||
/** `users.id` of the scope's holder, or 0 for the app's own set. */
|
||||
holder: number;
|
||||
}
|
||||
|
||||
/** The app's own scope. */
|
||||
export const appHandlerScope = (appUid: string): HandlerScope => ({
|
||||
appUid,
|
||||
holder: 0,
|
||||
});
|
||||
|
||||
const isUserScope = (scope: HandlerScope): boolean => scope.holder !== 0;
|
||||
|
||||
const assertScope = (scope: HandlerScope): void => {
|
||||
if (!Number.isInteger(scope.holder) || scope.holder < 0)
|
||||
throw new TypeError('handler scope holder must be a user id or 0');
|
||||
};
|
||||
|
||||
const tableOf = (scope: HandlerScope): string =>
|
||||
isUserScope(scope) ? USER_TABLE : TABLE;
|
||||
|
||||
const handlerCapOf = (scope: HandlerScope): number =>
|
||||
isUserScope(scope)
|
||||
? EVENTS_USER_HANDLERS_PER_SCOPE
|
||||
: EVENTS_HANDLERS_PER_APP;
|
||||
|
||||
/** The WHERE clause for a scope's rows and its parameters. */
|
||||
const scopeFilter = (
|
||||
scope: HandlerScope,
|
||||
): { where: string; params: unknown[] } => {
|
||||
assertScope(scope);
|
||||
return isUserScope(scope)
|
||||
? {
|
||||
where: '`app_uid` = ? AND `holder_user_id` = ?',
|
||||
params: [scope.appUid, scope.holder],
|
||||
}
|
||||
: { where: '`app_uid` = ?', params: [scope.appUid] };
|
||||
};
|
||||
|
||||
// App scope keeps the key it always had, so nodes on either side of a rolling
|
||||
// deploy share it.
|
||||
const handlerSetCacheKey = (scope: HandlerScope): string =>
|
||||
isUserScope(scope)
|
||||
? `eventHandlers:uset:${scope.appUid}:${scope.holder}`
|
||||
: `eventHandlers:set:${scope.appUid}`;
|
||||
|
||||
/**
|
||||
* The same set as read from the replica, kept apart so a lagging read can never
|
||||
* stand in for the primary's answer on the delivery path.
|
||||
*/
|
||||
const replicaSetCacheKey = (appUid: string): string =>
|
||||
`${handlerSetCacheKey(appUid)}:r`;
|
||||
const replicaSetCacheKey = (scope: HandlerScope): string =>
|
||||
`${handlerSetCacheKey(scope)}:r`;
|
||||
|
||||
// -- Shapes -----------------------------------------------------------
|
||||
|
||||
@@ -126,8 +191,45 @@ export interface ListEventsWorkersOptions {
|
||||
appUid?: string;
|
||||
}
|
||||
|
||||
export interface PublishHandlerInput {
|
||||
/** One user scope: the handlers a user keeps in one app. */
|
||||
export interface UserScopeSummary {
|
||||
appUid: string;
|
||||
/** Empty when the app row is gone. */
|
||||
appName: string;
|
||||
appTitle: string;
|
||||
handlerCount: number;
|
||||
/** Earliest handler `created_at` in the scope. */
|
||||
createdAt: number;
|
||||
/** Latest handler `updated_at` in the scope. */
|
||||
updatedAt: number;
|
||||
}
|
||||
|
||||
/** A user scope as an app-wide listing reports it. */
|
||||
export interface AppUserScope {
|
||||
holder: number;
|
||||
handlerCount: number;
|
||||
createdAt: number;
|
||||
updatedAt: number;
|
||||
}
|
||||
|
||||
/** One user-scope handler a GC pass may delete. */
|
||||
export interface UnboundUserHandler {
|
||||
id: number;
|
||||
appUid: string;
|
||||
holder: number;
|
||||
name: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* One entry of a scope's handler set. `updatedAt` (unix seconds) is set for a
|
||||
* user scope and absent from entries cached without it, meaning unknown.
|
||||
*/
|
||||
export type HandlerSetEntry = Pick<EventHandler, 'name' | 'sourceHash'> & {
|
||||
updatedAt?: number;
|
||||
};
|
||||
|
||||
export interface PublishHandlerInput {
|
||||
scope: HandlerScope;
|
||||
name: string;
|
||||
source: string;
|
||||
/**
|
||||
@@ -137,6 +239,11 @@ export interface PublishHandlerInput {
|
||||
ifHash?: string | null;
|
||||
/** Take the name whatever is published under it. */
|
||||
replace?: boolean;
|
||||
/**
|
||||
* A create leaves peer nodes' cached sets alone: one missing the new name
|
||||
* is re-read when asked for it. Updates always broadcast.
|
||||
*/
|
||||
broadcast?: boolean;
|
||||
}
|
||||
|
||||
/** What one publish did, which is what the caller reports back. */
|
||||
@@ -169,10 +276,12 @@ const invalidSource = (): HttpError =>
|
||||
legacyCode: 'events_handler_source_invalid',
|
||||
});
|
||||
|
||||
const tooManyHandlers = (): HttpError =>
|
||||
const tooManyHandlers = (scope: HandlerScope): HttpError =>
|
||||
new HttpError(
|
||||
429,
|
||||
`An app may publish ${EVENTS_HANDLERS_PER_APP} handlers`,
|
||||
isUserScope(scope)
|
||||
? `A user scope may hold ${EVENTS_USER_HANDLERS_PER_SCOPE} handlers`
|
||||
: `An app may publish ${EVENTS_HANDLERS_PER_APP} handlers`,
|
||||
{ legacyCode: 'events_handler_limit' },
|
||||
);
|
||||
|
||||
@@ -211,10 +320,11 @@ const SELECT_COLUMNS =
|
||||
export const hashContent = (content: string): string =>
|
||||
createHash('sha256').update(content, 'utf8').digest('hex');
|
||||
|
||||
export const isValidHandlerName = (name: string): boolean =>
|
||||
/** `userScope` also admits the `@<sha256>` names minted for inline handlers. */
|
||||
export const isValidHandlerName = (name: string, userScope = false): boolean =>
|
||||
name.length > 0 &&
|
||||
name.length <= HANDLER_NAME_MAX_LENGTH &&
|
||||
NAME_PATTERN.test(name);
|
||||
(NAME_PATTERN.test(name) || (userScope && INLINE_NAME_PATTERN.test(name)));
|
||||
|
||||
export class EventHandlerStore extends PuterStore {
|
||||
// -- Writes ------------------------------------------------------
|
||||
@@ -227,11 +337,12 @@ export class EventHandlerStore extends PuterStore {
|
||||
* with a row that no longer exists.
|
||||
*/
|
||||
async publish(input: PublishHandlerInput): Promise<PublishResult> {
|
||||
const name = this.#assertName(input.name);
|
||||
const { scope } = input;
|
||||
const name = this.#assertName(input.name, scope);
|
||||
const source = this.#assertSource(input.source);
|
||||
const sourceHash = hashContent(source);
|
||||
|
||||
const existing = await this.getByName(input.appUid, name);
|
||||
const existing = await this.getByName(scope, name);
|
||||
if (!existing) {
|
||||
// `ifHash` naming a source that is not there is the same race from
|
||||
// the other side: something removed the row this publish was
|
||||
@@ -239,10 +350,11 @@ export class EventHandlerStore extends PuterStore {
|
||||
if (input.ifHash && !input.replace) throw publishConflict(name);
|
||||
return {
|
||||
handler: await this.#insert(
|
||||
input.appUid,
|
||||
scope,
|
||||
name,
|
||||
source,
|
||||
sourceHash,
|
||||
input.broadcast !== false,
|
||||
),
|
||||
outcome: 'created',
|
||||
};
|
||||
@@ -260,34 +372,155 @@ export class EventHandlerStore extends PuterStore {
|
||||
if (!input.replace && !basedOnPublished) throw publishConflict(name);
|
||||
|
||||
return {
|
||||
handler: await this.#update(existing, source, sourceHash),
|
||||
handler: await this.#update(scope, existing, source, sourceHash),
|
||||
outcome: 'updated',
|
||||
};
|
||||
}
|
||||
|
||||
/** Drop one handler. Null when the app had nothing published by that name. */
|
||||
async remove(appUid: string, name: string): Promise<EventHandler | null> {
|
||||
const existing = await this.getByName(appUid, name);
|
||||
/** Drop one handler. Null when the scope had nothing published by that name. */
|
||||
async remove(
|
||||
scope: HandlerScope,
|
||||
name: string,
|
||||
): Promise<EventHandler | null> {
|
||||
const existing = await this.getByName(scope, name);
|
||||
if (!existing) return null;
|
||||
const { where, params } = scopeFilter(scope);
|
||||
await this.clients.db.write(
|
||||
`DELETE FROM \`${TABLE}\` WHERE \`app_uid\` = ? AND \`name\` = ?`,
|
||||
[appUid, name],
|
||||
`DELETE FROM \`${tableOf(scope)}\` WHERE ${where} AND \`name\` = ?`,
|
||||
[...params, name],
|
||||
);
|
||||
await this.#forgetSet(appUid);
|
||||
await this.#forgetSet(scope);
|
||||
return existing;
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop every handler an app has published — what a deleted app leaves
|
||||
* behind.
|
||||
* Drop every handler an app has published, in its own set and in every
|
||||
* user's scope — what a deleted app leaves behind. Returns the rows
|
||||
* removed. User scopes go a batch of holders at a time, each batch's cached
|
||||
* sets dropped here and on every peer.
|
||||
*/
|
||||
async deleteForApp(appUid: string): Promise<number> {
|
||||
const result = await this.clients.db.write(
|
||||
`DELETE FROM \`${TABLE}\` WHERE \`app_uid\` = ?`,
|
||||
[appUid],
|
||||
);
|
||||
await this.#forgetSet(appUid);
|
||||
return result.affectedRows;
|
||||
let removed = result.affectedRows ?? 0;
|
||||
for (;;) {
|
||||
const holders = (
|
||||
await this.clients.db.pread(
|
||||
`SELECT DISTINCT \`holder_user_id\` FROM \`${USER_TABLE}\` ` +
|
||||
'WHERE `app_uid` = ? ORDER BY `holder_user_id` LIMIT ?',
|
||||
[appUid, USER_SCOPES_PAGE_DEFAULT],
|
||||
)
|
||||
).map((row) => Number(row.holder_user_id));
|
||||
if (holders.length === 0) break;
|
||||
const batch = await this.clients.db.write(
|
||||
`DELETE FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? ` +
|
||||
`AND \`holder_user_id\` IN (${holders.map(() => '?').join(', ')})`,
|
||||
[appUid, ...holders],
|
||||
);
|
||||
await this.#forgetSets(
|
||||
holders.map((holder) => ({ appUid, holder })),
|
||||
);
|
||||
const affected = batch.affectedRows ?? 0;
|
||||
removed += affected;
|
||||
if (affected === 0) break;
|
||||
}
|
||||
await this.#forgetSet(appHandlerScope(appUid));
|
||||
return removed;
|
||||
}
|
||||
|
||||
/** Drop one user's scope of one app. Returns the handlers removed. */
|
||||
async deleteUserScope(appUid: string, holder: number): Promise<number> {
|
||||
const scope = { appUid, holder };
|
||||
if (!isUserScope(scope)) return 0;
|
||||
assertScope(scope);
|
||||
const result = await this.clients.db.write(
|
||||
`DELETE FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` = ?`,
|
||||
[appUid, holder],
|
||||
);
|
||||
await this.#forgetSet(scope);
|
||||
return result.affectedRows ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop every scope a user holds, across apps — what an account deletion
|
||||
* leaves behind. Returns the handlers removed.
|
||||
*/
|
||||
async deleteUserScopesForHolder(userId: number): Promise<number> {
|
||||
if (!Number.isInteger(userId) || userId <= 0) return 0;
|
||||
const apps = await this.clients.db.pread(
|
||||
`SELECT DISTINCT \`app_uid\` FROM \`${USER_TABLE}\` WHERE \`holder_user_id\` = ?`,
|
||||
[userId],
|
||||
);
|
||||
const result = await this.clients.db.write(
|
||||
`DELETE FROM \`${USER_TABLE}\` WHERE \`holder_user_id\` = ?`,
|
||||
[userId],
|
||||
);
|
||||
for (const row of apps)
|
||||
await this.#forgetSet({
|
||||
appUid: String(row.app_uid),
|
||||
holder: userId,
|
||||
});
|
||||
return result.affectedRows ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete one user-scope handler if no subscription row of any state is
|
||||
* bound to it and it was last touched before `olderThan` (unix seconds).
|
||||
* The check and the delete are one statement, so a row bound in between
|
||||
* keeps the handler. True when it deleted.
|
||||
*/
|
||||
async deleteUserHandlerIfUnbound(
|
||||
scope: HandlerScope,
|
||||
name: string,
|
||||
olderThan: number,
|
||||
): Promise<boolean> {
|
||||
if (!isUserScope(scope)) return false;
|
||||
assertScope(scope);
|
||||
const result = await this.clients.db.write(
|
||||
`DELETE FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` = ? ` +
|
||||
'AND `name` = ? AND `updated_at` < ? ' +
|
||||
`AND NOT EXISTS (SELECT 1 FROM \`${SUBSCRIPTION_TABLE}\` ` +
|
||||
'WHERE `holder_user_id` = ? AND `app_uid` = ? ' +
|
||||
'AND `handler_scope` = 1 AND `handler_name` = ?)',
|
||||
[
|
||||
scope.appUid,
|
||||
scope.holder,
|
||||
name,
|
||||
olderThan,
|
||||
scope.holder,
|
||||
scope.appUid,
|
||||
name,
|
||||
],
|
||||
);
|
||||
if (!result.anyRowsAffected) return false;
|
||||
await this.#forgetSet(scope);
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark one user-scope handler as just used, so the unbound sweep (which
|
||||
* only takes rows last touched before its cutoff) leaves it alone. False
|
||||
* when the handler is gone.
|
||||
*/
|
||||
async touchUserHandler(
|
||||
scope: HandlerScope,
|
||||
name: string,
|
||||
): Promise<boolean> {
|
||||
if (!isUserScope(scope)) return false;
|
||||
assertScope(scope);
|
||||
const result = await this.clients.db.write(
|
||||
`UPDATE \`${USER_TABLE}\` SET \`updated_at\` = ? ` +
|
||||
'WHERE `app_uid` = ? AND `holder_user_id` = ? AND `name` = ?',
|
||||
[nowSeconds(), scope.appUid, scope.holder, name],
|
||||
);
|
||||
return Boolean(result.anyRowsAffected);
|
||||
}
|
||||
|
||||
/** Drop this region's cached sets of a scope, without telling peers. */
|
||||
async forgetScopeCache(scope: HandlerScope): Promise<void> {
|
||||
await this.#forgetSet(scope, false);
|
||||
}
|
||||
|
||||
// -- Reads -------------------------------------------------------
|
||||
@@ -299,34 +532,36 @@ export class EventHandlerStore extends PuterStore {
|
||||
* absent.
|
||||
*/
|
||||
async getByName(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
name: string,
|
||||
): Promise<EventHandler | null> {
|
||||
const { where, params } = scopeFilter(scope);
|
||||
const rows = await this.clients.db.pread(
|
||||
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` ` +
|
||||
'WHERE `app_uid` = ? AND `name` = ?',
|
||||
[appUid, name],
|
||||
`SELECT ${SELECT_COLUMNS} FROM \`${tableOf(scope)}\` ` +
|
||||
`WHERE ${where} AND \`name\` = ?`,
|
||||
[...params, name],
|
||||
);
|
||||
return rows.length > 0 ? toRow(rows[0]) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* What an app has published, with how many subscriptions each name is
|
||||
* carrying. Bounded by the per-app cap, so the whole set is one page.
|
||||
* What a scope has published, with how many subscriptions each name is
|
||||
* carrying. Bounded by the scope's cap, so the whole set is one page.
|
||||
*
|
||||
* The counts come from one grouped read rather than a join, because the
|
||||
* subscription side has to count names an app never published — a handler
|
||||
* subscription side has to count names a scope never published — a handler
|
||||
* removed while subscriptions still pointed at it leaves exactly that.
|
||||
*/
|
||||
async listForApp(appUid: string): Promise<EventHandlerSummary[]> {
|
||||
async listForApp(scope: HandlerScope): Promise<EventHandlerSummary[]> {
|
||||
const { where, params } = scopeFilter(scope);
|
||||
const rows = await this.clients.db.read(
|
||||
'SELECT `name`, `source_hash`, `updated_at` ' +
|
||||
`FROM \`${TABLE}\` WHERE \`app_uid\` = ? ` +
|
||||
`FROM \`${tableOf(scope)}\` WHERE ${where} ` +
|
||||
'ORDER BY `name` LIMIT ?',
|
||||
[appUid, EVENTS_HANDLERS_PER_APP],
|
||||
[...params, handlerCapOf(scope)],
|
||||
);
|
||||
|
||||
const counts = await this.countSubscriptionsByHandler(appUid);
|
||||
const counts = await this.countSubscriptionsByHandler(scope);
|
||||
return rows.map((row) => ({
|
||||
name: String(row.name),
|
||||
hash: String(row.source_hash),
|
||||
@@ -336,27 +571,30 @@ export class EventHandlerStore extends PuterStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Name and source hash for every handler an app has published: the identity
|
||||
* of its set, without reading a byte of source. What a deploy checks the
|
||||
* script it was asked for against, and what the delivery path's cache is
|
||||
* filled from. Primary, so neither reads a set a replica has not caught up
|
||||
* to.
|
||||
* Name and source hash for every handler a scope has published: the
|
||||
* identity of its set, without reading a byte of source. What a deploy
|
||||
* checks the script it was asked for against, and what the delivery path's
|
||||
* cache is filled from. Primary, so neither reads a set a replica has not
|
||||
* caught up to.
|
||||
*/
|
||||
async setForApp(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
options: { replica?: boolean } = {},
|
||||
): Promise<Pick<EventHandler, 'name' | 'sourceHash'>[]> {
|
||||
): Promise<HandlerSetEntry[]> {
|
||||
const { where, params: whereParams } = scopeFilter(scope);
|
||||
const withAge = isUserScope(scope);
|
||||
const sql =
|
||||
'SELECT `name`, `source_hash` ' +
|
||||
`FROM \`${TABLE}\` WHERE \`app_uid\` = ? ` +
|
||||
`SELECT \`name\`, \`source_hash\`${withAge ? ', `updated_at`' : ''} ` +
|
||||
`FROM \`${tableOf(scope)}\` WHERE ${where} ` +
|
||||
'ORDER BY `name` LIMIT ?';
|
||||
const params = [appUid, EVENTS_HANDLERS_PER_APP];
|
||||
const params = [...whereParams, handlerCapOf(scope)];
|
||||
const rows = options.replica
|
||||
? await this.clients.db.read(sql, params)
|
||||
: await this.clients.db.pread(sql, params);
|
||||
return rows.map((row) => ({
|
||||
name: String(row.name),
|
||||
sourceHash: String(row.source_hash),
|
||||
...(withAge ? { updatedAt: Number(row.updated_at) || 0 } : {}),
|
||||
}));
|
||||
}
|
||||
|
||||
@@ -371,12 +609,12 @@ export class EventHandlerStore extends PuterStore {
|
||||
* right after a publish would fail the run. Called by extensions.
|
||||
*/
|
||||
async cachedSetForApp(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
options: {
|
||||
missRead?: 'primary' | 'replica';
|
||||
handlerName?: string;
|
||||
} = {},
|
||||
): Promise<Pick<EventHandler, 'name' | 'sourceHash'>[]> {
|
||||
): Promise<HandlerSetEntry[]> {
|
||||
const replica = options.missRead === 'replica';
|
||||
// A set without the handler may be from before its publish.
|
||||
const usable = (set: Pick<EventHandler, 'name'>[]) =>
|
||||
@@ -384,8 +622,8 @@ export class EventHandlerStore extends PuterStore {
|
||||
(options.handlerName === undefined ||
|
||||
set.some((handler) => handler.name === options.handlerName));
|
||||
const key = replica
|
||||
? replicaSetCacheKey(appUid)
|
||||
: handlerSetCacheKey(appUid);
|
||||
? replicaSetCacheKey(scope)
|
||||
: handlerSetCacheKey(scope);
|
||||
try {
|
||||
const cached = JSON.parse(
|
||||
(await this.clients.redis.get(key)) ?? 'null',
|
||||
@@ -394,13 +632,13 @@ export class EventHandlerStore extends PuterStore {
|
||||
Array.isArray(cached) &&
|
||||
usable(cached as Pick<EventHandler, 'name'>[])
|
||||
)
|
||||
return cached as Pick<EventHandler, 'name' | 'sourceHash'>[];
|
||||
return cached as HandlerSetEntry[];
|
||||
} catch {
|
||||
// Read the table instead.
|
||||
}
|
||||
|
||||
let set = await this.setForApp(appUid, { replica });
|
||||
if (replica && !usable(set)) set = await this.setForApp(appUid);
|
||||
let set = await this.setForApp(scope, { replica });
|
||||
if (replica && !usable(set)) set = await this.setForApp(scope);
|
||||
if (set.length > 0) {
|
||||
try {
|
||||
await this.clients.redis.set(
|
||||
@@ -417,57 +655,70 @@ export class EventHandlerStore extends PuterStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Every handler the app has published, sources included — what the events
|
||||
* Every handler the scope has published, sources included — what the events
|
||||
* worker is generated from. Primary, for the same reason `getByName` is:
|
||||
* the bake follows the publish that triggered it.
|
||||
*/
|
||||
async allForApp(appUid: string): Promise<EventHandler[]> {
|
||||
async allForApp(scope: HandlerScope): Promise<EventHandler[]> {
|
||||
const { where, params } = scopeFilter(scope);
|
||||
const rows = await this.clients.db.pread(
|
||||
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` ` +
|
||||
'WHERE `app_uid` = ? ORDER BY `name` LIMIT ?',
|
||||
[appUid, EVENTS_HANDLERS_PER_APP],
|
||||
`SELECT ${SELECT_COLUMNS} FROM \`${tableOf(scope)}\` ` +
|
||||
`WHERE ${where} ORDER BY \`name\` LIMIT ?`,
|
||||
[...params, handlerCapOf(scope)],
|
||||
);
|
||||
return rows.map(toRow);
|
||||
}
|
||||
|
||||
async countForApp(appUid: string): Promise<number> {
|
||||
async countForApp(scope: HandlerScope): Promise<number> {
|
||||
const { where, params } = scopeFilter(scope);
|
||||
const [row] = await this.clients.db.pread(
|
||||
`SELECT COUNT(*) AS \`total\` FROM \`${TABLE}\` WHERE \`app_uid\` = ?`,
|
||||
[appUid],
|
||||
`SELECT COUNT(*) AS \`total\` FROM \`${tableOf(scope)}\` WHERE ${where}`,
|
||||
params,
|
||||
);
|
||||
return Number(row?.total ?? 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* Bytes of source an app has published, across every handler. What a
|
||||
* publish is checked against before it lands, so the app's generated events
|
||||
* Bytes of source a scope has published, across every handler. What a
|
||||
* publish is checked against before it lands, so the generated events
|
||||
* worker never grows past what the runtime may load.
|
||||
*
|
||||
* `OCTET_LENGTH`, not `LENGTH`: only MySQL measures the latter in bytes,
|
||||
* and the caller compares this against `Buffer.byteLength`.
|
||||
*/
|
||||
async totalSourceBytesForApp(appUid: string): Promise<number> {
|
||||
async totalSourceBytesForApp(scope: HandlerScope): Promise<number> {
|
||||
const { where, params } = scopeFilter(scope);
|
||||
const [row] = await this.clients.db.pread(
|
||||
`SELECT SUM(OCTET_LENGTH(\`source\`)) AS \`total\` FROM \`${TABLE}\` WHERE \`app_uid\` = ?`,
|
||||
[appUid],
|
||||
`SELECT SUM(OCTET_LENGTH(\`source\`)) AS \`total\` FROM \`${tableOf(scope)}\` WHERE ${where}`,
|
||||
params,
|
||||
);
|
||||
return Number(row?.total ?? 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* How many subscriptions each of an app's handler names is carrying,
|
||||
* How many subscriptions each of a scope's handler names is carrying,
|
||||
* suspended rows included — a suspended subscription is still a dependent,
|
||||
* and it is the reason a removal is not a delete.
|
||||
*
|
||||
* A user scope reads by holder, so it never scans the other holders of a
|
||||
* popular app.
|
||||
*/
|
||||
async countSubscriptionsByHandler(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
): Promise<Map<string, number>> {
|
||||
assertScope(scope);
|
||||
const filter = isUserScope(scope)
|
||||
? '`holder_user_id` = ? AND `app_uid` = ? AND `handler_scope` = 1'
|
||||
: '`app_uid` = ? AND `handler_scope` = 0';
|
||||
const params = isUserScope(scope)
|
||||
? [scope.holder, scope.appUid]
|
||||
: [scope.appUid];
|
||||
const rows = await this.clients.db.read(
|
||||
'SELECT `handler_name`, COUNT(*) AS `total` ' +
|
||||
`FROM \`${SUBSCRIPTION_TABLE}\` ` +
|
||||
'WHERE `app_uid` = ? AND `handler_name` IS NOT NULL ' +
|
||||
`WHERE ${filter} AND \`handler_name\` IS NOT NULL ` +
|
||||
'GROUP BY `handler_name`',
|
||||
[appUid],
|
||||
params,
|
||||
);
|
||||
return new Map(
|
||||
rows.map((row) => [
|
||||
@@ -480,7 +731,8 @@ export class EventHandlerStore extends PuterStore {
|
||||
/**
|
||||
* How many of an owner's apps have at least one published handler — what a
|
||||
* rent listener counts to tell a first publish (0→1) from a redeploy, and a
|
||||
* last removal (1→0) from one of several.
|
||||
* last removal (1→0) from one of several. Reads the apps' own sets only:
|
||||
* user scopes are never the owner's.
|
||||
*
|
||||
* `createdBefore` (epoch ms) restricts to apps whose events worker existed
|
||||
* before that time, by its earliest handler; used to bill only what was
|
||||
@@ -512,10 +764,36 @@ export class EventHandlerStore extends PuterStore {
|
||||
return Number(rows[0]?.total ?? 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* How many apps a user holds a scope in — the rent count for user scopes,
|
||||
* with `createdBefore` (epoch ms) meaning what it does for
|
||||
* {@link countAppsWithHandlersForOwner}.
|
||||
*/
|
||||
async countUserScopesWithHandlersForUser(
|
||||
userId: number,
|
||||
opts: { createdBefore?: number } = {},
|
||||
): Promise<number> {
|
||||
const params: unknown[] = [userId];
|
||||
let having = '';
|
||||
if (opts.createdBefore !== undefined) {
|
||||
having = ' HAVING MIN(`scope_since`) < ?';
|
||||
params.push(Math.ceil(opts.createdBefore / 1000));
|
||||
}
|
||||
const rows = await this.clients.db.pread(
|
||||
'SELECT COUNT(*) AS `total` FROM (' +
|
||||
`SELECT \`app_uid\` FROM \`${USER_TABLE}\` ` +
|
||||
`WHERE \`holder_user_id\` = ? GROUP BY \`app_uid\`${having}` +
|
||||
') AS `counted`',
|
||||
params,
|
||||
);
|
||||
return Number(rows[0]?.total ?? 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* The events workers an owner is running, one row per app with at least one
|
||||
* published handler. Keyset-paginated on `app_uid`, since the grouping key
|
||||
* is the only stable order a set this shape can offer.
|
||||
* is the only stable order a set this shape can offer. Reads the apps' own
|
||||
* sets only.
|
||||
*/
|
||||
async listEventsWorkersForOwner(
|
||||
ownerUserId: number,
|
||||
@@ -574,21 +852,184 @@ export class EventHandlerStore extends PuterStore {
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* The scopes one user holds, one row per app, keyset-paginated on
|
||||
* `app_uid`. `total` counts every scope the user holds (or the one app's,
|
||||
* when `appUid` narrows it).
|
||||
*/
|
||||
async listUserScopesForUser(
|
||||
userId: number,
|
||||
opts: ListEventsWorkersOptions = {},
|
||||
): Promise<PageResult<UserScopeSummary>> {
|
||||
const limit = Math.min(
|
||||
Math.max(
|
||||
1,
|
||||
Math.floor(opts.limit ?? EVENTS_WORKERS_LIST_DEFAULT_LIMIT),
|
||||
),
|
||||
EVENTS_WORKERS_LIST_LIMIT_CAP,
|
||||
);
|
||||
const after = decodeCursor(opts.cursor)?.appUid;
|
||||
|
||||
const where = [`\`${USER_TABLE}\`.\`holder_user_id\` = ?`];
|
||||
const params: unknown[] = [userId];
|
||||
if (typeof opts.appUid === 'string') {
|
||||
where.push(`\`${USER_TABLE}\`.\`app_uid\` = ?`);
|
||||
params.push(opts.appUid);
|
||||
}
|
||||
const totalRows = await this.clients.db.read(
|
||||
'SELECT COUNT(*) AS `total` FROM (' +
|
||||
`SELECT \`${USER_TABLE}\`.\`app_uid\` FROM \`${USER_TABLE}\` ` +
|
||||
`WHERE ${where.join(' AND ')} GROUP BY \`${USER_TABLE}\`.\`app_uid\`` +
|
||||
') AS `counted`',
|
||||
params,
|
||||
);
|
||||
if (typeof after === 'string') {
|
||||
where.push(`\`${USER_TABLE}\`.\`app_uid\` > ?`);
|
||||
params.push(after);
|
||||
}
|
||||
|
||||
const rows = await this.clients.db.read(
|
||||
`SELECT \`${USER_TABLE}\`.\`app_uid\` AS \`app_uid\`, ` +
|
||||
`\`${APP_TABLE}\`.\`name\` AS \`app_name\`, ` +
|
||||
`\`${APP_TABLE}\`.\`title\` AS \`app_title\`, ` +
|
||||
'COUNT(*) AS `handler_count`, ' +
|
||||
`MIN(\`${USER_TABLE}\`.\`created_at\`) AS \`created_at\`, ` +
|
||||
`MAX(\`${USER_TABLE}\`.\`updated_at\`) AS \`updated_at\` ` +
|
||||
`FROM \`${USER_TABLE}\` ` +
|
||||
`LEFT JOIN \`${APP_TABLE}\` ON \`${APP_TABLE}\`.\`uid\` = \`${USER_TABLE}\`.\`app_uid\` ` +
|
||||
`WHERE ${where.join(' AND ')} ` +
|
||||
`GROUP BY \`${USER_TABLE}\`.\`app_uid\`, \`${APP_TABLE}\`.\`name\`, \`${APP_TABLE}\`.\`title\` ` +
|
||||
`ORDER BY \`${USER_TABLE}\`.\`app_uid\` LIMIT ?`,
|
||||
[...params, limit + 1],
|
||||
);
|
||||
|
||||
const page = keysetPage(rows, limit, undefined, (last) => ({
|
||||
appUid: last.app_uid,
|
||||
}));
|
||||
const result: PageResult<UserScopeSummary> = {
|
||||
items: page.rows.map((row) => ({
|
||||
appUid: String(row.app_uid),
|
||||
appName: row.app_name == null ? '' : String(row.app_name),
|
||||
appTitle: row.app_title == null ? '' : String(row.app_title),
|
||||
handlerCount: Number(row.handler_count) || 0,
|
||||
createdAt: Number(row.created_at) || 0,
|
||||
updatedAt: Number(row.updated_at) || 0,
|
||||
})),
|
||||
total: Number(totalRows[0]?.total ?? 0),
|
||||
};
|
||||
if (page.cursor) result.cursor = page.cursor;
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* The user scopes one app holds, ordered by holder. A batch scan: it
|
||||
* carries no `total`, to spare a COUNT per page. `cursor` is null at the
|
||||
* end.
|
||||
*/
|
||||
async listUserScopes(
|
||||
appUid: string,
|
||||
opts: { limit?: number; cursor?: string } = {},
|
||||
): Promise<{ items: AppUserScope[]; cursor: string | null }> {
|
||||
const limit = Math.min(
|
||||
Math.max(1, Math.floor(opts.limit ?? USER_SCOPES_PAGE_DEFAULT)),
|
||||
USER_SCOPES_PAGE_CAP,
|
||||
);
|
||||
const after = Number(decodeCursor(opts.cursor)?.holder);
|
||||
const rows = await this.clients.db.read(
|
||||
'SELECT `holder_user_id`, COUNT(*) AS `handler_count`, ' +
|
||||
'MIN(`created_at`) AS `created_at`, ' +
|
||||
'MAX(`updated_at`) AS `updated_at` ' +
|
||||
`FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` > ? ` +
|
||||
'GROUP BY `holder_user_id` ORDER BY `holder_user_id` LIMIT ?',
|
||||
[appUid, Number.isFinite(after) ? after : 0, limit + 1],
|
||||
);
|
||||
const page = keysetPage(rows, limit, undefined, (last) => ({
|
||||
holder: Number(last.holder_user_id),
|
||||
}));
|
||||
return {
|
||||
items: page.rows.map((row) => ({
|
||||
holder: Number(row.holder_user_id),
|
||||
handlerCount: Number(row.handler_count) || 0,
|
||||
createdAt: Number(row.created_at) || 0,
|
||||
updatedAt: Number(row.updated_at) || 0,
|
||||
})),
|
||||
cursor: page.cursor ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* One keyset page of user-scope handlers last touched before `olderThan`
|
||||
* (unix seconds) that no subscription row of any state is bound to. A batch
|
||||
* scan: it carries no `total`, to spare a COUNT per page. `cursor` is null
|
||||
* at the end. A candidate may be bound before it is deleted, so delete
|
||||
* through {@link deleteUserHandlerIfUnbound}.
|
||||
*/
|
||||
async listUnboundUserHandlers(
|
||||
olderThan: number,
|
||||
opts: { limit?: number; cursor?: string } = {},
|
||||
): Promise<{ items: UnboundUserHandler[]; cursor: string | null }> {
|
||||
const limit = Math.max(
|
||||
1,
|
||||
Math.floor(opts.limit ?? UNBOUND_HANDLERS_BATCH),
|
||||
);
|
||||
const after = Number(decodeCursor(opts.cursor)?.id);
|
||||
const rows = await this.clients.db.read(
|
||||
'SELECT h.`id`, h.`app_uid`, h.`holder_user_id`, h.`name` ' +
|
||||
`FROM \`${USER_TABLE}\` h ` +
|
||||
'WHERE h.`id` > ? AND h.`updated_at` < ? ' +
|
||||
`AND NOT EXISTS (SELECT 1 FROM \`${SUBSCRIPTION_TABLE}\` s ` +
|
||||
'WHERE s.`holder_user_id` = h.`holder_user_id` ' +
|
||||
'AND s.`app_uid` = h.`app_uid` AND s.`handler_scope` = 1 ' +
|
||||
'AND s.`handler_name` = h.`name`) ' +
|
||||
'ORDER BY h.`id` LIMIT ?',
|
||||
[Number.isFinite(after) ? after : 0, olderThan, limit + 1],
|
||||
);
|
||||
const page = keysetPage(rows, limit, undefined);
|
||||
return {
|
||||
items: page.rows.map((row) => ({
|
||||
id: Number(row.id),
|
||||
appUid: String(row.app_uid),
|
||||
holder: Number(row.holder_user_id),
|
||||
name: String(row.name),
|
||||
})),
|
||||
cursor: page.cursor ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
// -- Internals ---------------------------------------------------
|
||||
|
||||
async #insert(
|
||||
appUid: string,
|
||||
scope: HandlerScope,
|
||||
name: string,
|
||||
source: string,
|
||||
sourceHash: string,
|
||||
broadcast: boolean,
|
||||
): Promise<EventHandler> {
|
||||
if ((await this.countForApp(appUid)) >= EVENTS_HANDLERS_PER_APP)
|
||||
throw tooManyHandlers();
|
||||
// A user scope's count read also yields when the scope began, so its
|
||||
// age survives the handlers that held it.
|
||||
let scopeSince: number | null = null;
|
||||
if (isUserScope(scope)) {
|
||||
const [state] = await this.clients.db.pread(
|
||||
'SELECT COUNT(*) AS `total`, MIN(`scope_since`) AS `since` ' +
|
||||
`FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` = ?`,
|
||||
[scope.appUid, scope.holder],
|
||||
);
|
||||
if (Number(state?.total ?? 0) >= handlerCapOf(scope))
|
||||
throw tooManyHandlers(scope);
|
||||
scopeSince = state?.since == null ? null : Number(state.since);
|
||||
} else if ((await this.countForApp(scope)) >= handlerCapOf(scope))
|
||||
throw tooManyHandlers(scope);
|
||||
|
||||
const at = nowSeconds();
|
||||
try {
|
||||
await this.clients.db.insert(TABLE, {
|
||||
app_uid: appUid,
|
||||
await this.clients.db.insert(tableOf(scope), {
|
||||
app_uid: scope.appUid,
|
||||
...(isUserScope(scope)
|
||||
? {
|
||||
holder_user_id: scope.holder,
|
||||
scope_since: scopeSince ?? at,
|
||||
}
|
||||
: {}),
|
||||
name,
|
||||
source,
|
||||
source_hash: sourceHash,
|
||||
@@ -603,9 +1044,25 @@ export class EventHandlerStore extends PuterStore {
|
||||
if (isUniqueViolation(err)) throw publishConflict(name);
|
||||
throw err;
|
||||
}
|
||||
await this.#forgetSet(appUid);
|
||||
// Concurrent creates can all pass the count above. Counted again with
|
||||
// this row in it, each one past the cap takes itself back out — at the
|
||||
// last slot that may refuse more than it had to, never fewer. User
|
||||
// scopes only: reads stop at the cap, so a row past it would be unseen.
|
||||
if (
|
||||
isUserScope(scope) &&
|
||||
(await this.countForApp(scope)) > handlerCapOf(scope)
|
||||
) {
|
||||
const { where, params } = scopeFilter(scope);
|
||||
await this.clients.db.write(
|
||||
`DELETE FROM \`${tableOf(scope)}\` WHERE ${where} AND \`name\` = ?`,
|
||||
[...params, name],
|
||||
);
|
||||
await this.#forgetSet(scope);
|
||||
throw tooManyHandlers(scope);
|
||||
}
|
||||
await this.#forgetSet(scope, broadcast);
|
||||
return {
|
||||
appUid,
|
||||
appUid: scope.appUid,
|
||||
name,
|
||||
source,
|
||||
sourceHash,
|
||||
@@ -615,23 +1072,25 @@ export class EventHandlerStore extends PuterStore {
|
||||
}
|
||||
|
||||
async #update(
|
||||
scope: HandlerScope,
|
||||
existing: EventHandler,
|
||||
source: string,
|
||||
sourceHash: string,
|
||||
): Promise<EventHandler> {
|
||||
const at = nowSeconds();
|
||||
const { where, params } = scopeFilter(scope);
|
||||
// The `source_hash` predicate is what makes the check-then-write a
|
||||
// compare-and-set: a publish that raced past the read above updates
|
||||
// nothing here, and its caller is told.
|
||||
const updated = await this.clients.db.write(
|
||||
`UPDATE \`${TABLE}\` SET \`source\` = ?, \`source_hash\` = ?, ` +
|
||||
'`updated_at` = ? WHERE `app_uid` = ? AND `name` = ? ' +
|
||||
`UPDATE \`${tableOf(scope)}\` SET \`source\` = ?, \`source_hash\` = ?, ` +
|
||||
`\`updated_at\` = ? WHERE ${where} AND \`name\` = ? ` +
|
||||
'AND `source_hash` = ?',
|
||||
[
|
||||
source,
|
||||
sourceHash,
|
||||
at,
|
||||
existing.appUid,
|
||||
...params,
|
||||
existing.name,
|
||||
existing.sourceHash,
|
||||
],
|
||||
@@ -639,20 +1098,37 @@ export class EventHandlerStore extends PuterStore {
|
||||
if (updated?.anyRowsAffected === false)
|
||||
throw publishConflict(existing.name);
|
||||
|
||||
await this.#forgetSet(existing.appUid);
|
||||
await this.#forgetSet(scope);
|
||||
return { ...existing, source, sourceHash, updatedAt: at };
|
||||
}
|
||||
|
||||
/** Drop the delivery path's cached set here and on every peer. */
|
||||
async #forgetSet(appUid: string): Promise<void> {
|
||||
/**
|
||||
* Drop the delivery path's cached set here, and on every peer if
|
||||
* `broadcast`.
|
||||
*/
|
||||
async #forgetSet(scope: HandlerScope, broadcast = true): Promise<void> {
|
||||
await this.publishCacheKeys({
|
||||
keys: [handlerSetCacheKey(appUid), replicaSetCacheKey(appUid)],
|
||||
keys: [handlerSetCacheKey(scope), replicaSetCacheKey(scope)],
|
||||
broadcast,
|
||||
});
|
||||
}
|
||||
|
||||
/** {@link #forgetSet} for many scopes, as one peer message. */
|
||||
async #forgetSets(scopes: HandlerScope[]): Promise<void> {
|
||||
await this.publishCacheKeys({
|
||||
keys: scopes.flatMap((scope) => [
|
||||
handlerSetCacheKey(scope),
|
||||
replicaSetCacheKey(scope),
|
||||
]),
|
||||
broadcast: true,
|
||||
});
|
||||
}
|
||||
|
||||
#assertName(name: unknown): string {
|
||||
if (typeof name !== 'string' || !isValidHandlerName(name))
|
||||
#assertName(name: unknown, scope: HandlerScope): string {
|
||||
if (
|
||||
typeof name !== 'string' ||
|
||||
!isValidHandlerName(name, isUserScope(scope))
|
||||
)
|
||||
throw invalidName();
|
||||
return name;
|
||||
}
|
||||
|
||||
@@ -155,6 +155,15 @@ const liveRegionsByToken = (
|
||||
return live;
|
||||
};
|
||||
|
||||
/**
|
||||
* Present on every durable row this code caches: the handler scope is stated (a
|
||||
* user row also carries `handlerScope`). A node without user scopes writes rows
|
||||
* without it, and its entries say nothing about scope.
|
||||
*/
|
||||
const SCOPE_MARKER = 'sc';
|
||||
|
||||
const SCOPE_LOOKUP_BATCH = 200;
|
||||
|
||||
/** A row we cannot read is a row we cannot deliver against. */
|
||||
const parseRow = (raw: string): DispatchSubscription | null => {
|
||||
try {
|
||||
@@ -1015,7 +1024,11 @@ export class EventSubscriptionStore extends PuterStore {
|
||||
const write = this.clients.redis.pipeline();
|
||||
for (const row of rows) {
|
||||
const key = tokenKey(ownerUserId, row.token);
|
||||
write.hset(key, row.subId, JSON.stringify(row));
|
||||
write.hset(
|
||||
key,
|
||||
row.subId,
|
||||
JSON.stringify({ ...row, [SCOPE_MARKER]: 1 }),
|
||||
);
|
||||
write.expire(key, DURABLE_CACHE_TTL_SECONDS);
|
||||
write.hset(durableMapKey(ownerUserId), row.subId, row.token);
|
||||
}
|
||||
@@ -1269,9 +1282,96 @@ export class EventSubscriptionStore extends PuterStore {
|
||||
subs.push(parsed);
|
||||
}
|
||||
}
|
||||
await this.#settleScopes(ownerUserId, subs);
|
||||
return subs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Rows a node without user scopes cached carry no scope marker, so a user
|
||||
* row among them reads as an app row. Those with a handler are resolved
|
||||
* from the table (replica, then primary for what it lacks) and rewritten
|
||||
* with the marker; marked rows cost nothing.
|
||||
*/
|
||||
async #settleScopes(
|
||||
ownerUserId: number,
|
||||
rows: DispatchSubscription[],
|
||||
): Promise<void> {
|
||||
const unmarked: DispatchSubscription[] = [];
|
||||
for (const row of rows) {
|
||||
const marked = (row as unknown as Record<string, unknown>)[
|
||||
SCOPE_MARKER
|
||||
];
|
||||
delete (row as unknown as Record<string, unknown>)[SCOPE_MARKER];
|
||||
if (marked === undefined && row.durable === true && row.handlerName)
|
||||
unmarked.push(row);
|
||||
}
|
||||
if (unmarked.length === 0) return;
|
||||
|
||||
// A failed lookup throws, as a table read that fails does for the
|
||||
// rest of dispatch: a row of unknown scope must never be returned,
|
||||
// since it would read as an app row.
|
||||
const scopes = new Map<string, number>();
|
||||
const lookup = async (
|
||||
read: 'read' | 'pread',
|
||||
ids: string[],
|
||||
): Promise<void> => {
|
||||
const found = await this.clients.db[read](
|
||||
'SELECT `sub_id`, `handler_scope` FROM `event_subscriptions` ' +
|
||||
`WHERE \`sub_id\` IN (${ids.map(() => '?').join(', ')})`,
|
||||
ids,
|
||||
);
|
||||
for (const row of found)
|
||||
scopes.set(String(row.sub_id), Number(row.handler_scope));
|
||||
};
|
||||
for (let i = 0; i < unmarked.length; i += SCOPE_LOOKUP_BATCH) {
|
||||
const ids = unmarked
|
||||
.slice(i, i + SCOPE_LOOKUP_BATCH)
|
||||
.map((row) => row.subId);
|
||||
await lookup('read', ids);
|
||||
const missing = ids.filter((id) => !scopes.has(id));
|
||||
if (missing.length > 0) await lookup('pread', missing);
|
||||
}
|
||||
|
||||
// In neither the replica nor the primary: a ghost entry. Dropped,
|
||||
// as the normal path treats a row that is gone, so it is not
|
||||
// resolved again on every read.
|
||||
const ghosts = unmarked.filter((row) => !scopes.has(row.subId));
|
||||
if (ghosts.length > 0) {
|
||||
const drop = this.clients.redis.pipeline();
|
||||
for (const row of ghosts)
|
||||
drop.hdel(tokenKey(ownerUserId, row.token), row.subId);
|
||||
await drop.exec();
|
||||
for (const row of ghosts) rows.splice(rows.indexOf(row), 1);
|
||||
}
|
||||
|
||||
const resolved = unmarked.filter((row) => scopes.has(row.subId));
|
||||
for (const row of resolved)
|
||||
if (scopes.get(row.subId) === 1)
|
||||
(row as DurableSubscription).handlerScope = 'user';
|
||||
|
||||
// Rewritten only where the entry still stands, so a row dropped
|
||||
// meanwhile is not brought back. Failing to rewrite costs a
|
||||
// repeat lookup, not a wrong answer.
|
||||
try {
|
||||
const check = this.clients.redis.pipeline();
|
||||
for (const row of resolved)
|
||||
check.hexists(tokenKey(ownerUserId, row.token), row.subId);
|
||||
const standing = (await check.exec()) ?? [];
|
||||
const write = this.clients.redis.pipeline();
|
||||
resolved.forEach((row, i) => {
|
||||
if (Number(standing[i]?.[1]) !== 1) return;
|
||||
write.hset(
|
||||
tokenKey(ownerUserId, row.token),
|
||||
row.subId,
|
||||
JSON.stringify({ ...row, [SCOPE_MARKER]: 1 }),
|
||||
);
|
||||
});
|
||||
await write.exec();
|
||||
} catch (err) {
|
||||
console.warn('[events] could not mark cached row scopes', err);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* One pipelined first page per token, which is the whole hash for any token
|
||||
* small enough to be stored compactly; only a larger one is scanned
|
||||
@@ -1316,6 +1416,7 @@ export class EventSubscriptionStore extends PuterStore {
|
||||
}
|
||||
subs.push(...taken.values());
|
||||
}
|
||||
await this.#settleScopes(ownerUserId, subs);
|
||||
return subs;
|
||||
}
|
||||
|
||||
|
||||
@@ -18,7 +18,10 @@
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { EVENTS_INVOKE_TIMEOUT_MS } from '../../clients/events/EventsWorkerInvokerClient.js';
|
||||
import {
|
||||
EVENTS_INVOKE_TIMEOUT_MS,
|
||||
EVENTS_WORKER_LEASE_FACTOR,
|
||||
} from '../../clients/events/EventsWorkerInvokerClient.js';
|
||||
import {
|
||||
EVENTS_FAILURE_COUNTER_TTL_MS,
|
||||
EVENTS_PENDING_DELIVERIES_PER_SUBSCRIPTION,
|
||||
@@ -316,7 +319,7 @@ export class PendingDeliveryStore extends PuterStore {
|
||||
typeof configured === 'number' && configured > 0
|
||||
? configured
|
||||
: EVENTS_INVOKE_TIMEOUT_MS;
|
||||
return invokeTimeoutMs * 2;
|
||||
return invokeTimeoutMs * EVENTS_WORKER_LEASE_FACTOR;
|
||||
}
|
||||
|
||||
// -- Writes ------------------------------------------------------
|
||||
|
||||
@@ -29,6 +29,12 @@ import type { FsOp } from '../../services/events/subjects.js';
|
||||
* is declared once here rather than per store.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Which handler set a subscription's `handlerName` refers to: the app's own
|
||||
* (`'app'`) or the holder's scope of that app (`'user'`). Stored as 0 and 1.
|
||||
*/
|
||||
export type HandlerScopeKind = 'app' | 'user';
|
||||
|
||||
/**
|
||||
* Whether a row's events also run the app's published handler in its events
|
||||
* worker. Only an app has a worker, and only a named handler can be run.
|
||||
@@ -97,6 +103,8 @@ export interface DispatchSubscription {
|
||||
durable?: true;
|
||||
delivery?: DeliveryClass;
|
||||
handlerName?: string | null;
|
||||
/** Durable rows only: absent means `'app'`. */
|
||||
handlerScope?: HandlerScopeKind;
|
||||
/** Durable rows only: handed to the handler, and read nowhere else. */
|
||||
context?: string | null;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { EventClient } from '../../clients/event/EventClient.js';
|
||||
import type { EventMap } from '../../clients/event/types.js';
|
||||
import { NotificationStore } from './NotificationStore.js';
|
||||
|
||||
const setup = (row: Record<string, unknown> | null, changed = true) => {
|
||||
const event = new EventClient({} as never);
|
||||
const emitted: EventMap['notif.acknowledged'][] = [];
|
||||
event.on('notif.acknowledged', (_key, data) => { emitted.push(data); });
|
||||
const db = {
|
||||
write: vi.fn(async () => ({ anyRowsAffected: changed })),
|
||||
pread: vi.fn(async () => row ? [row] : []),
|
||||
read: vi.fn(async () => []),
|
||||
};
|
||||
const store = new NotificationStore({}, { event, db } as never, {});
|
||||
return { store, emitted, db };
|
||||
};
|
||||
|
||||
const appRow = {
|
||||
uid: 'one', user_id: 7, type: 'app.message', app_uid: 'app-one',
|
||||
audience: 'app-user', value: { title: 'private', data: 'private' },
|
||||
};
|
||||
|
||||
describe('acknowledgement emission', () => {
|
||||
it('announces a committed app message dismissal without content', async () => {
|
||||
const { store, emitted, db } = setup(appRow);
|
||||
await expect(store.markAcknowledged('one', 7)).resolves.toBe(true);
|
||||
expect(db.write.mock.calls[0][0]).toContain('`acknowledged` IS NULL');
|
||||
expect(db.pread).toHaveBeenCalledWith(expect.stringContaining('`user_id` = ?'), ['one', 7]);
|
||||
expect(emitted).toEqual([{ userId: 7, uid: 'one', type: 'app.message', audience: 'app-user', appUid: 'app-one' }]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ ...appRow, type: 'share.received' },
|
||||
{ ...appRow, app_uid: null },
|
||||
null,
|
||||
])('does not announce a row outside an app message scope: %s', async (row) => {
|
||||
const { store, emitted } = setup(row);
|
||||
await store.markAcknowledged('one', 7);
|
||||
expect(emitted).toEqual([]);
|
||||
});
|
||||
|
||||
it('does not announce or read back an unchanged row', async () => {
|
||||
const { store, emitted, db } = setup(appRow, false);
|
||||
await expect(store.markAcknowledged('one', 7)).resolves.toBe(false);
|
||||
expect(db.pread).not.toHaveBeenCalled();
|
||||
expect(emitted).toEqual([]);
|
||||
});
|
||||
|
||||
it('preserves a committed acknowledgement if the announcement read fails', async () => {
|
||||
const { store, db, emitted } = setup(appRow);
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
db.pread.mockRejectedValueOnce(new Error('unavailable'));
|
||||
try {
|
||||
await expect(store.markAcknowledged('one', 7)).resolves.toBe(true);
|
||||
expect(emitted).toEqual([]);
|
||||
} finally {
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('does not propagate markShown', async () => {
|
||||
const { store, emitted } = setup(appRow);
|
||||
await store.markShown('one', 7);
|
||||
expect(emitted).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('displayed notification state query', () => {
|
||||
it('reads all named ids in one scoped query', async () => {
|
||||
const { store, db } = setup(appRow);
|
||||
await store.listByUserId(7, {
|
||||
limit: 50, uids: ['one', 'two'],
|
||||
scope: { appUid: 'app-one', audiences: ['app-user'] },
|
||||
});
|
||||
expect(db.pread).not.toHaveBeenCalled();
|
||||
expect(db.read).toHaveBeenCalledExactlyOnceWith(
|
||||
expect.stringContaining('`uid` IN (?, ?)'),
|
||||
[7, 'one', 'two', 'app-user', 'app-one', 50],
|
||||
);
|
||||
expect(db.read.mock.calls[0][0]).toContain('`user_id` = ?');
|
||||
expect(db.read.mock.calls[0][0]).toContain('`app_uid` = ?');
|
||||
});
|
||||
|
||||
it('does not query an empty id set', async () => {
|
||||
const { store, db } = setup(appRow);
|
||||
expect(await store.listByUserId(7, { uids: [] })).toEqual([]);
|
||||
expect(db.read).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -156,11 +156,12 @@ export class NotificationStore extends PuterStore {
|
||||
* @param {number} userId
|
||||
* @param {{
|
||||
* limit?: number;
|
||||
* uids?: readonly string[];
|
||||
* onlyUnacknowledged?: boolean;
|
||||
* filter?: string;
|
||||
* scope?: {
|
||||
* audiences: readonly string[];
|
||||
* appUid: string | null;
|
||||
* appUid?: string | null;
|
||||
* } | null;
|
||||
* excludeTypes?: readonly string[];
|
||||
* }} [opts]
|
||||
@@ -169,6 +170,7 @@ export class NotificationStore extends PuterStore {
|
||||
userId,
|
||||
{
|
||||
limit = 200,
|
||||
uids,
|
||||
onlyUnacknowledged = false,
|
||||
filter = undefined,
|
||||
scope = null,
|
||||
@@ -178,6 +180,12 @@ export class NotificationStore extends PuterStore {
|
||||
const where = ['`user_id` = ?'];
|
||||
const params = [userId];
|
||||
|
||||
if (uids) {
|
||||
if (uids.length === 0) return [];
|
||||
where.push(`\`uid\` IN (${uids.map(() => '?').join(', ')})`);
|
||||
params.push(...uids);
|
||||
}
|
||||
|
||||
if (excludeTypes.length > 0) {
|
||||
where.push(
|
||||
`\`type\` NOT IN (${excludeTypes.map(() => '?').join(', ')})`,
|
||||
@@ -315,6 +323,57 @@ export class NotificationStore extends PuterStore {
|
||||
return rows.map((r) => this.#normalizeRow(r));
|
||||
}
|
||||
|
||||
/**
|
||||
* The newest row id in a mailbox slice, or null when it has none, from the
|
||||
* primary. Scoped like `listScoped`; rows of every type count.
|
||||
*
|
||||
* @param {number} userId
|
||||
* @param {{ audience: string; appUid?: string | null }} scope
|
||||
* @returns {Promise<number | null>}
|
||||
*/
|
||||
async latestScopedId(userId, { audience, appUid }) {
|
||||
const { where, params } = sliceWhere(userId, {
|
||||
audience,
|
||||
appUid,
|
||||
excludeTypes: [],
|
||||
});
|
||||
const [row] = await this.clients.db.pread(
|
||||
`SELECT \`id\` FROM \`notification\`
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY \`id\` DESC
|
||||
LIMIT 1`,
|
||||
params,
|
||||
);
|
||||
return row ? Number(row.id) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The latest replacement in a mailbox slice by (`replaced_at`, `id`), or
|
||||
* null when none was replaced, from the primary. Scoped like `listScoped`.
|
||||
*
|
||||
* @param {number} userId
|
||||
* @param {{ audience: string; appUid?: string | null }} scope
|
||||
* @returns {Promise<{ replacedAt: number; id: number } | null>}
|
||||
*/
|
||||
async latestScopedReplacement(userId, { audience, appUid }) {
|
||||
const { where, params } = sliceWhere(userId, {
|
||||
audience,
|
||||
appUid,
|
||||
excludeTypes: [],
|
||||
});
|
||||
where.push('`replaced_at` IS NOT NULL');
|
||||
const [row] = await this.clients.db.pread(
|
||||
`SELECT \`replaced_at\`, \`id\` FROM \`notification\`
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY \`replaced_at\` DESC, \`id\` DESC
|
||||
LIMIT 1`,
|
||||
params,
|
||||
);
|
||||
return row
|
||||
? { replacedAt: Number(row.replaced_at), id: Number(row.id) }
|
||||
: null;
|
||||
}
|
||||
|
||||
// -- Writes -------------------------------------------------------
|
||||
|
||||
/**
|
||||
@@ -522,6 +581,32 @@ export class NotificationStore extends PuterStore {
|
||||
'UPDATE `notification` SET `acknowledged` = ? WHERE `uid` = ? AND `user_id` = ? AND `acknowledged` IS NULL',
|
||||
[now, uid, userId],
|
||||
);
|
||||
if (result.anyRowsAffected) {
|
||||
try {
|
||||
const [row] = await this.clients.db.pread(
|
||||
'SELECT `type`, `app_uid`, `audience` FROM `notification` WHERE `uid` = ? AND `user_id` = ? LIMIT 1',
|
||||
[uid, userId],
|
||||
);
|
||||
if (row?.type === 'app.message' && row.app_uid) {
|
||||
this.clients.event.emit(
|
||||
'notif.acknowledged',
|
||||
{
|
||||
userId,
|
||||
uid,
|
||||
type: row.type,
|
||||
audience: row.audience,
|
||||
appUid: row.app_uid,
|
||||
},
|
||||
{},
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
'[notification] dismissal announcement failed',
|
||||
error,
|
||||
);
|
||||
}
|
||||
}
|
||||
return result.anyRowsAffected;
|
||||
}
|
||||
|
||||
|
||||
@@ -410,6 +410,61 @@ describe('NotificationStore', () => {
|
||||
expect(rest.map((r) => r.uid)).toEqual([c.uid]);
|
||||
});
|
||||
|
||||
it('finds the newest id of a slice, or null for an empty one', async () => {
|
||||
const u = await makeUser();
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
const write = (audience, inApp) =>
|
||||
store.create({ userId: u.id, value: {}, audience, appUid: inApp });
|
||||
const latest = (audience, inApp) =>
|
||||
store.latestScopedId(u.id, { audience, appUid: inApp });
|
||||
|
||||
expect(await latest('app-user', appUid)).toBeNull();
|
||||
|
||||
await write('app-user', appUid);
|
||||
const newest = await write('app-user', appUid);
|
||||
await write('app-user', `app-${uuidv4()}`);
|
||||
const unattributed = await write('developer', null);
|
||||
const developer = await write('developer', appUid);
|
||||
await store.create({
|
||||
userId: other.id,
|
||||
value: {},
|
||||
audience: 'app-user',
|
||||
appUid,
|
||||
});
|
||||
|
||||
expect(await latest('app-user', appUid)).toBe(Number(newest.id));
|
||||
expect(await latest('developer', null)).toBe(Number(unattributed.id));
|
||||
expect(await latest('developer', undefined)).toBe(Number(developer.id));
|
||||
expect(await latest('account', undefined)).toBeNull();
|
||||
});
|
||||
|
||||
it('finds the latest replacement of a slice, or null when none', async () => {
|
||||
const u = await makeUser();
|
||||
const appUid = `app-${uuidv4()}`;
|
||||
const write = (inApp = appUid) =>
|
||||
store.create({
|
||||
userId: u.id,
|
||||
value: {},
|
||||
audience: 'app-user',
|
||||
appUid: inApp,
|
||||
});
|
||||
const latest = () =>
|
||||
store.latestScopedReplacement(u.id, {
|
||||
audience: 'app-user',
|
||||
appUid,
|
||||
});
|
||||
|
||||
const a = await write();
|
||||
const b = await write();
|
||||
const otherApp = await write(`app-${uuidv4()}`);
|
||||
expect(await latest()).toBeNull();
|
||||
|
||||
await store.updateValue(a.uid, u.id, {}, { replacedAt: 2_000 });
|
||||
await store.updateValue(b.uid, u.id, {}, { replacedAt: 2_000 });
|
||||
await store.updateValue(otherApp.uid, u.id, {}, { replacedAt: 9_000 });
|
||||
expect(await latest()).toEqual({ replacedAt: 2_000, id: Number(b.id) });
|
||||
});
|
||||
|
||||
// -- mutations -----------------------------------------------------
|
||||
|
||||
it('acknowledges only once and only for the owning user', async () => {
|
||||
|
||||
+15
-1
@@ -1270,6 +1270,11 @@ interface IConfigOptional {
|
||||
* through a handle both reject with `events_kv_handles_disabled`, and a
|
||||
* handle row already made stops delivering. Nothing on the write path
|
||||
* reads it.
|
||||
* - `userHandlers` — whether an app may publish handlers into a signed-in
|
||||
* user's own scope, including inline handlers created with a
|
||||
* subscription. Absent means off: user-scope publishing and inline
|
||||
* creates reject with `events_user_handlers_disabled`; app-scope handlers
|
||||
* are unaffected.
|
||||
* - `workerRuntime` — whether publishing handlers deploys a per-app events
|
||||
* worker and delivery invokes it. Absent means off: publish stores rows
|
||||
* and nothing is deployed, and the invoker keeps its null resolver, so
|
||||
@@ -1286,6 +1291,7 @@ interface IConfigOptional {
|
||||
crossAppKv?: boolean;
|
||||
notificationsFoldIn?: boolean;
|
||||
kvHandles?: boolean;
|
||||
userHandlers?: boolean;
|
||||
workerRuntime?: boolean;
|
||||
forwardSession?: boolean;
|
||||
/** How long a handler has to answer an invocation. Default 30 s. */
|
||||
@@ -1307,6 +1313,12 @@ interface IConfigOptional {
|
||||
* default: deploys refuse rather than land somewhere public.
|
||||
*/
|
||||
workerNamespace?: string;
|
||||
/**
|
||||
* Per-invocation CPU limit, in milliseconds, set on every events worker
|
||||
* at deploy. Unset means no limit. Scripts already deployed pick it up
|
||||
* on their next deploy.
|
||||
*/
|
||||
workerCpuMs?: number;
|
||||
/**
|
||||
* Where the events dispatcher answers `POST /invoke` — its own
|
||||
* hostname, since it carries no zone route. Unset means invocations
|
||||
@@ -1368,7 +1380,9 @@ export interface WithLifecycle extends Object {
|
||||
/**
|
||||
* When the drain starts, before `onServerPrepareShutdown`, which waits for
|
||||
* every connection (upgraded sockets included) to close or the drain to
|
||||
* end.
|
||||
* end. Hooks run concurrently, and the server keeps accepting connections
|
||||
* until all of them settle, so a hook can hold the listener open (until
|
||||
* traffic stops arriving, say) by returning a promise.
|
||||
*/
|
||||
onServerDrainStart?: (info?: ServerDrainInfo) => Promise<void> | void;
|
||||
}
|
||||
|
||||
@@ -18,7 +18,12 @@
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { toAppReadView, toAppSummary, toAppView } from './appView.js';
|
||||
import {
|
||||
isOriginBootstrapApp,
|
||||
toAppReadView,
|
||||
toAppSummary,
|
||||
toAppView,
|
||||
} from './appView.js';
|
||||
|
||||
const row = (overrides: Record<string, unknown> = {}) => ({
|
||||
id: 42,
|
||||
@@ -192,3 +197,27 @@ describe('toAppReadView', () => {
|
||||
).toBe(64);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isOriginBootstrapApp', () => {
|
||||
const originRow = (overrides: Record<string, unknown> = {}) =>
|
||||
row({
|
||||
name: 'app-1234',
|
||||
title: 'app-1234',
|
||||
description: 'App created from origin https://chat.example.com',
|
||||
...overrides,
|
||||
});
|
||||
|
||||
it('matches a row created for an origin', () => {
|
||||
expect(isOriginBootstrapApp(originRow())).toBe(true);
|
||||
});
|
||||
|
||||
it('does not match once the app has its own name, title or description', () => {
|
||||
expect(isOriginBootstrapApp(originRow({ name: 'chat' }))).toBe(false);
|
||||
expect(isOriginBootstrapApp(originRow({ title: 'Chat' }))).toBe(false);
|
||||
expect(isOriginBootstrapApp(originRow({ description: 'Chat' }))).toBe(
|
||||
false,
|
||||
);
|
||||
expect(isOriginBootstrapApp(row())).toBe(false);
|
||||
expect(isOriginBootstrapApp(null)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -25,6 +25,18 @@ import type { PrivateLaunchDecision } from './privateLaunchAccess.js';
|
||||
/** A raw `apps` store row. */
|
||||
export type AppRow = Record<string, unknown>;
|
||||
|
||||
/** A row created for an origin, whose name and title are still its uid. */
|
||||
export function isOriginBootstrapApp(app: unknown): app is AppRow {
|
||||
if (!app || typeof app !== 'object') return false;
|
||||
const { uid, name, title, description } = app as AppRow;
|
||||
if (typeof uid !== 'string' || !uid) return false;
|
||||
if (name !== uid || title !== uid) return false;
|
||||
return (
|
||||
typeof description === 'string' &&
|
||||
description.startsWith('App created from origin ')
|
||||
);
|
||||
}
|
||||
|
||||
/** Client shape of an app. Never carries internal ids or the owner's id. */
|
||||
export interface AppView {
|
||||
uid: unknown;
|
||||
|
||||
Loaded 100 of 242 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user