Compare commits

...
Author SHA1 Message Date
Nariman Jelveh 08a1a24dfe fix(homepage): serve the shell for /@<user>/<path>
Only /@<user> was routed, so a link to something inside a user's Public
folder fell through to the not-found redirect and landed on / without
opening anything. The GUI already opens the folder in Files, or the
file in its app, once the shell is served.
2026-11-04 11:40:47 -07:00
Nariman Jelveh 620a94c7fc refactor(gui): remove the desktop; every boot opens the dashboard
The desktop (UIDesktop, the taskbar, toolbar, start menu, search, task
manager, welcome, QR and wallpaper windows) is gone. Every boot outside an
auth popup opens the dashboard, including iframe embeds and URLs that used
to load the desktop:

- `/desktop`, `/desktop/app/<name>` redirect to `/` and `/app/<name>`,
  keeping the query; the GUI also drops the prefix if one reaches it.
- `/@<user>/<path>` and `/show/<path>` open the folder in Files, or the
  file in its app. `/show` no longer carries a boot hook naming a GUI
  global that no longer exists, which had stopped the GUI booting.
- `/settings` opens Home; `?app=explorer[&path=]` opens Files.
- `/?download=<url>` keeps its confirm-and-save-to-Desktop flow.

The window system stays: the dashboard, its apps and the file pickers
run on it. Snapping, the taskbar-height layout math and the folder
layout/sort endpoints are untouched. The backend drops the
`/set-desktop-bg` and `/update-taskbar-items` endpoints and the
`taskbar_items` and `desktop_bg_*` user fields (columns kept).
2026-11-04 10:57:52 -07:00
Juan Fernando Castro 93798cba93 fix(puter-js): saved push handlers stay while any tab uses them, and the worker can skip them (#4481)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment

* test(puter-js): keep the push boot test comment to one line

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.

* fix(static): push-sw.js refreshes at the CDN within five minutes

* feat(puter-js): the push worker can skip saved handlers

* fix(puter-js): off() keeps a saved push handler another tab still uses

* docs(notifications): self-hosting the push worker

* docs(notifications): name the pinned push worker version as a placeholder

* fix(puter-js): pages with push handlers can enter the back/forward cache

* test(puter-js): a restored page saves its push handler again

* docs(notifications): pin the push worker only with a pinned puter.js
2026-11-04 10:15:36 -05:00
Juan Fernando Castro 848e336455 fix(puter-js): page loads check push status instead of registering again (#4480)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment

* test(puter-js): keep the push boot test comment to one line

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.

* fix(puter-js): a page load checks push status instead of registering again

* fix(puter-js): subscriptions made with an earlier Puter key are replaced

* fix(puter-js): a replaced browser subscription is picked up on the next visit

* fix(puter-js): a page load that only checks push status still looks for a newer worker

* fix(puter-js): the worker subscribes again only where a Puter page used it

* docs(puter-js): soften the note on subscriptions Puter made earlier

* fix(puter-js): a status answer on load leaves a subscription another tab just turned on
2026-11-04 10:15:25 -05:00
Juan Fernando Castro a9a8df6409 feat(events): fetch can start at the newest notification, and the feed uses it (#4479)
* feat(events): fetch can start from the newest event

GET /events/fetch takes from=end: with no cursor it answers an empty page
whose cursor sits at the newest row of the slice, so later reads return
only rows created or replaced after it. puter.events.fetch() passes
`from` through. Adds a (user_id, audience, app_uid, id) index on
notification so the newest row of a slice is one seek.

* fix(puter-js): onPush starts its feed from the newest notification

A notifications listener started without catch-up asks for from: 'end'
and keeps the cursor it gets back instead of reading every stored
notification to mark them as seen. A server that ignores `from` answers
with the oldest page, and the listener reads through it as before.

* fix(events): read the newest notification id from the primary

* fix(events): mark the end cursor at the slice's latest replacement

The end cursor's replacement mark is the newest (replaced_at, id) in the
slice rather than the serving node's clock, since replaced_at is stamped
by whichever node wrote it.

* docs(events): describe the end cursor's replacement mark
2026-11-04 10:15:14 -05:00
Juan Fernando Castro 0a8b37ae56 fix(notifications): website apps are named by their hostname (#4468)
An app created for an origin keeps its uid as name and title, so its
notifications showed that uid as the sender. appMessageSender now names
such an app by the hostname of its index_url, and keeps the stored name
when index_url doesn't parse. The check AppDriver used for these rows
moves to util/appView so both share it.

The Settings app list shows a generic label for an app whose title is
still its uid.
2026-11-04 10:15:03 -05:00
Daniel Salazar 23782dec12 feat(events): inline handlers in the user's own scope for onSchedule and onPersistent (#4487)
* feat(events): schema for user-scope handlers

Add event_user_handlers and event_subscriptions.handler_scope in all three
dialects. event_handlers is unchanged.

* feat(events): scope the handler and durable subscription stores

Handler store methods take a scope (app, or a user's scope of an app) and read
the matching table. App-scope behavior, SQL and cache keys are unchanged; user
scopes get their own cache keys, a 20-handler cap, and the inline @<sha256>
name. Adds user-scope listing, counting, deletion and unbound-handler reads.
Durable subscriptions store and return handlerScope, and listByHandler
filters by scope and holder.

* fix(events): hold the user-scope handler cap under concurrency and drop all scope caches on app delete

A create that lands past the cap removes itself and is refused. deleteForApp
drops each holder's cached set in batches, here and on peers. Batch scans over
user scopes and unbound handlers take limit and cursor.

* feat(events): set a CPU limit on events worker deploys

* test(events): cover redeploy and resend of an evicted events script

* test(events): name the deploy-on-miss test for what it checks

* fix(events): keep a deploy-on-miss call inside the delivery lease

* feat(workers): cap rented units per plan and refuse only new ones

* feat(events): publish handlers into a user scope, with inline create

* test(events): cover user-scope publish, inline create and listing

* test(workers): name the credentials gate without a vendor

* feat(events): run user-scope handlers as the holder

A user scope deploys one script per handler under the holder's own
identity, throttled per scope, with an eager deploy after publish that
skips an upload another holder already made. Delivery and schedule runs
address the holder's handler, carry the holder on the invoke headers,
batch calls, the miss handler and the rehydrate route, and stay owed
while the flag is off. Suspensions of a user scope notify the holder,
teardown (app delete, consent revoke, account delete) drops the scope,
and a job deletes unbound handlers after a day.

* test(events): cover user-scope deploy, address, notices, teardown and the sweep

Runs an inline user-scope handler through the local worker runtime as its
holder, and covers the holder on the wire, the flag-off path, revoke and
account-delete teardown, and the unbound-handler sweep.

* test(events): a lagging replica read does not suspend a user-scope schedule run

* feat(puter-js): inline user-scope handlers and scope option for handlers and workers

* docs(events): document inline and user-scope handlers, with playground examples

Cover inline handlers on onSchedule and onPersistent, handlerScope, the
scope option on handlers and workers, the user-scope limits and rented
units, and the new error codes. Add six playground examples and pin the
new limit constants in limits.test.ts.

* feat(schedules): carry the handler scope on the schedule view

* docs(events): describe user-handler cleanup, schedule handlerScope and suspension notices

* fix(events): gate the rented-unit cap on user handlers and let an account session destroy its own user scope

* fix(events): keep user-scope rows off app handlers across mixed versions and bound their deploys and reads

Cached durable rows carry an explicit scope marker; an unmarked entry with a
handler is resolved from the table and rewritten. User-scope rows are parked
before credit checks, billing, client delivery and suspension while the flag
is off. Delivery-time handler checks and addressing share the replica cache
entry. Concurrent eager deploys of one user-scope script upload once.

* fix(events): keep a user scope's age across handler churn for rent

event_user_handlers.scope_since records when the scope first held a handler.
A new row takes the scope's existing minimum, so replacing code does not make
the scope look newer to the monthly sweep. The count uses it instead of the
oldest remaining handler's created_at.

* fix(events): drop cached entries no row backs; document account-session scope destroy

* test(jobs): list the user-handler sweep among the system jobs

* fix(events): let the user-handlers migration apply on its own

* fix(events): fail a cached-row read whose scope lookup fails instead of returning app-scope rows

* fix(events): touch a reused user-scope handler before binding so the sweep cannot take it

* fix(events): bound the deploy-on-miss wait by the call's deadline
2026-11-03 23:26:04 -07:00
Daniel Salazar 6a6cf78c03 fix(drivers): stream results with pipe so responses stay under the close-listener limit (#4485)
stream.pipeline adds seven close listeners to the response; with the ones
metering, compression and tracing add, every streamed /drivers/call passed
Node's ten-listener limit and logged MaxListenersExceededWarning. pipe plus
explicit error and close handling adds two, and keeps the same behavior: a
source failing mid-body aborts the response and logs once, and a client
hanging up stops the source.
2026-11-03 23:25:55 -07:00
188 changed files with 12324 additions and 9129 deletions

No files matched your search

+3
View File
@@ -63,6 +63,9 @@ store:
sealed cursor: the last `id` it returned, and the last `(replaced_at, id)`
among rows up to that id. Each is its own indexed read, merged into one
page, so a row replaced after a client read it comes back from that cursor.
`from=end` with no cursor answers an empty page whose cursor holds the
slice's newest `id` and the slice's latest `(replaced_at, id)` as the
replacement mark.
Cursors that carry a sort also pin it: a request that passes a cursor plus a
conflicting sort is rejected with 400.
+5 -10
View File
@@ -257,8 +257,6 @@ describe('whoami extension — handleWhoami', () => {
expect(body.created_ts).toBeGreaterThan(0);
// `directories` is only sent to user actors — confirm it’s present.
expect(body.directories).toBeDefined();
// taskbar_items is only sent to user actors.
expect(body).toHaveProperty('taskbar_items');
});
it('reports the SMS-to-card fallback only once a send has opened it', async () => {
@@ -322,7 +320,7 @@ describe('whoami extension — handleWhoami', () => {
expect(flags.payment_bypass).toBeUndefined();
});
it('strips desktop_bg_*, created_ts and human_readable_age fields for app actors', async () => {
it('strips created_ts and human_readable_age fields for app actors', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
@@ -340,9 +338,6 @@ describe('whoami extension — handleWhoami', () => {
expect(body.app_name).toBe('app-test-actor');
// is_user_token is stripped for app actors.
expect(body.is_user_token).toBeUndefined();
expect(body.desktop_bg_url).toBeUndefined();
expect(body.desktop_bg_color).toBeUndefined();
expect(body.desktop_bg_fit).toBeUndefined();
expect(body.human_readable_age).toBeUndefined();
// Account age, in either form, is not exposed to apps.
expect(body.created_ts).toBeUndefined();
@@ -432,7 +427,7 @@ describe('whoami extension — handleWhoami', () => {
});
});
it('cannot add taskbar_items for an app actor, which core omits', async () => {
it('cannot add directories for an app actor, which core omits', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
@@ -440,11 +435,11 @@ describe('whoami extension — handleWhoami', () => {
app: { uid: 'app-test-actor' },
}),
(_key, event) => {
event.details.taskbar_items = [{ name: 'injected' }];
event.details.directories = { '/injected': 'x' };
},
);
expect(body).not.toHaveProperty('taskbar_items');
expect(body).not.toHaveProperty('directories');
});
});
@@ -525,7 +520,7 @@ describe('whoami extension — handleWhoami', () => {
expect(body.otp_secret).toBeUndefined();
expect(body.signup_ip).toBeUndefined();
// Not just absent as a key — the values must not appear anywhere
// in the payload (nested under metadata, taskbar items, …).
// in the payload (nested under metadata, …).
const serialized = JSON.stringify(body);
expect(serialized).not.toContain('5551234567');
expect(serialized).not.toContain('fp_ABC123');
+3 -17
View File
@@ -1,6 +1,5 @@
import { Context } from '@heyputer/backend/src/core';
import { extension } from '@heyputer/backend/src/extensions';
import { APP_ICON_SIZES } from '@heyputer/backend/src/util/appIcon.js';
import {
buildUserDetails,
scrubSensitive,
@@ -26,7 +25,7 @@ const APP_VISIBLE_LISTENER_KEYS: ReadonlySet<string> = new Set([
]);
export const handleWhoami = async (
req: Request,
_req: Request,
res: Response,
): Promise<void> => {
const actor = Context.get('actor');
@@ -54,19 +53,10 @@ export const handleWhoami = async (
return;
}
const rawIconSize =
typeof req.query?.icon_size === 'string'
? Number(req.query.icon_size)
: undefined;
const iconSize =
rawIconSize !== undefined && APP_ICON_SIZES.includes(rawIconSize)
? rawIconSize
: undefined;
const details = await buildUserDetails(
user,
{ config: extension.config, clients, stores, services },
{ isUser, iconSize, noIcons: !iconSize },
{ isUser },
);
details.human_readable_age = user.timestamp
? timeago.format(new Date(user.timestamp as string))
@@ -81,9 +71,6 @@ export const handleWhoami = async (
delete details.email;
delete details.unconfirmed_email;
}
delete details.desktop_bg_url;
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.created_ts;
delete details.is_user_token;
@@ -121,8 +108,7 @@ export const handleWhoami = async (
}
const subscription = details.subscription as
| { offering?: Record<string, unknown> }
| undefined;
{ offering?: Record<string, unknown> } | undefined;
if (subscription?.offering) {
delete subscription.offering.group;
delete subscription.offering.benefits;
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 96;
const CURRENT_SCHEMA_VERSION = 98;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -735,6 +735,74 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
expect(columns.map((c) => String(c.name))).toEqual(['send_key']);
});
it('creates event_user_handlers with its unique and holder indexes', async () => {
const columns = await client.read(
'PRAGMA table_info(`event_user_handlers`)',
);
expect(columns.map((c) => String(c.name))).toEqual([
'id',
'app_uid',
'holder_user_id',
'name',
'source',
'source_hash',
'created_at',
'updated_at',
'scope_since',
]);
const indexes = await client.read(
'PRAGMA index_list(`event_user_handlers`)',
);
expect(
indexes.find(
(i) => i.name === 'idx_event_user_handlers_app_holder_name',
),
).toMatchObject({ unique: 1 });
expect(
indexes.find((i) => i.name === 'idx_event_user_handlers_holder_app'),
).toMatchObject({ unique: 0 });
const columnsOf = async (index: string) =>
(await client.read(`PRAGMA index_info(\`${index}\`)`)).map((c) =>
String(c.name),
);
expect(await columnsOf('idx_event_user_handlers_app_holder_name')).toEqual(
['app_uid', 'holder_user_id', 'name'],
);
expect(await columnsOf('idx_event_user_handlers_holder_app')).toEqual([
'holder_user_id',
'app_uid',
]);
});
it('leaves event_handlers as it was', async () => {
const columns = await client.read('PRAGMA table_info(`event_handlers`)');
expect(columns.map((c) => String(c.name))).toEqual([
'id',
'app_uid',
'name',
'source',
'source_hash',
'created_at',
'updated_at',
]);
});
it('adds event_subscriptions.handler_scope, app scope for existing rows', async () => {
const columns = await client.read(
'PRAGMA table_info(`event_subscriptions`)',
);
expect(columns.find((c) => c.name === 'handler_scope')).toMatchObject({
notnull: 1,
dflt_value: '0',
});
const indexes = await client.read(
'PRAGMA index_list(`event_subscriptions`)',
);
expect(indexes.map((i) => String(i.name))).toContain(
'idx_event_subscriptions_holder',
);
});
it('leaves an already-migrated database untouched on a second boot', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
const path = join(dir, 'nested', 'puter.sqlite');
@@ -135,6 +135,8 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[93, ['0098_notification-groups.sql']],
[94, ['0099_kv-share-handle-notify.sql']],
[95, ['0100_notification-send-key.sql']],
[96, ['0101_user-scoped-handlers.sql']],
[97, ['0102_notification-scope-id-index.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,66 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Handlers an app's user published into their own scope, kept apart from
-- `event_handlers` so code that predates them never reads them. See
-- sqlite/0101_user-scoped-handlers.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- and a guarded column add. There is no per-file applied-state tracking, so a
-- replay has to be a no-op.
CREATE TABLE IF NOT EXISTS `event_user_handlers` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`holder_user_id` int unsigned NOT NULL,
`name` varchar(128) NOT NULL,
`source` mediumtext NOT NULL,
`source_hash` char(64) NOT NULL,
`created_at` bigint NOT NULL,
`updated_at` bigint NOT NULL,
-- When the scope first held a handler; kept across handler churn, so the
-- rent sweep sees the scope's age, not its oldest remaining handler's.
`scope_since` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_user_handlers_app_holder_name` (`app_uid`, `holder_user_id`, `name`),
KEY `idx_event_user_handlers_holder_app` (`holder_user_id`, `app_uid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Guarded here rather than through `_puter_add_col`, so the file also applies
-- on its own to a database that never ran the earlier migrations.
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_handler_scope;
DELIMITER //
CREATE PROCEDURE _puter_add_event_subscriptions_handler_scope()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'handler_scope'
) THEN
ALTER TABLE `event_subscriptions`
ADD COLUMN `handler_scope` tinyint unsigned NOT NULL DEFAULT 0,
ALGORITHM=INSTANT;
END IF;
END//
DELIMITER ;
CALL _puter_add_event_subscriptions_handler_scope();
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_handler_scope;
@@ -0,0 +1,41 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `notification` (`user_id`, `audience`, `app_uid`, `id`). See
-- sqlite/0102_notification-scope-id-index.sql for the rationale. No per-file
-- applied-state tracking, so the index is added only when missing and a
-- replay does nothing.
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_id_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_scope_id_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_scope_id'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_scope_id`
(`user_id`, `audience`, `app_uid`, `id`), ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_notification_scope_id_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_id_index;
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Handlers an app's user published into their own scope. See
-- sqlite/0101_user-scoped-handlers.sql for the column rationale.
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS event_user_handlers (
id BIGSERIAL PRIMARY KEY,
app_uid VARCHAR(40) NOT NULL,
holder_user_id INTEGER NOT NULL,
name VARCHAR(128) NOT NULL,
source TEXT NOT NULL,
source_hash VARCHAR(64) NOT NULL,
created_at BIGINT NOT NULL,
updated_at BIGINT NOT NULL,
scope_since BIGINT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_event_user_handlers_app_holder_name
ON event_user_handlers (app_uid, holder_user_id, name);
CREATE INDEX IF NOT EXISTS idx_event_user_handlers_holder_app
ON event_user_handlers (holder_user_id, app_uid);
ALTER TABLE event_subscriptions ADD COLUMN IF NOT EXISTS handler_scope smallint NOT NULL DEFAULT 0;
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- notification (user_id, audience, app_uid, id). See
-- sqlite/0102_notification-scope-id-index.sql for the rationale. Idempotent via
-- IF NOT EXISTS.
CREATE INDEX IF NOT EXISTS idx_notification_scope_id
ON notification (user_id, audience, app_uid, id);
@@ -0,0 +1,48 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Handlers an app's user published into their own scope, in a table of their
-- own so `event_handlers` stays the app owner's set. Same columns as
-- `event_handlers` (see 0076_event-handlers.sql) plus the holder.
--
-- - `holder_user_id` : the user whose scope this is. A name is unique per
-- (app, holder), so the same name can live in the app's
-- set and in each holder's.
-- - `(holder_user_id, app_uid)` : a holder's scopes, for listing and for the
-- rent count.
--
-- `event_subscriptions.handler_scope` says which table a row's `handler_name`
-- refers to: 0 the app's set, 1 the row holder's scope.
CREATE TABLE IF NOT EXISTS `event_user_handlers` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"app_uid" TEXT NOT NULL,
"holder_user_id" INTEGER NOT NULL,
"name" TEXT NOT NULL,
"source" TEXT NOT NULL,
"source_hash" TEXT NOT NULL,
"created_at" INTEGER NOT NULL, -- unix seconds
"updated_at" INTEGER NOT NULL,
"scope_since" INTEGER NOT NULL -- unix seconds; the scope's first handler, kept across churn
);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_event_user_handlers_app_holder_name`
ON `event_user_handlers` (`app_uid`, `holder_user_id`, `name`);
CREATE INDEX IF NOT EXISTS `idx_event_user_handlers_holder_app`
ON `event_user_handlers` (`holder_user_id`, `app_uid`);
ALTER TABLE `event_subscriptions` ADD COLUMN `handler_scope` INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index a mailbox slice by row id, so a fetch finds the newest row of a slice,
-- and pages forward through one, with a seek rather than a scan of the slice.
CREATE INDEX IF NOT EXISTS `idx_notification_scope_id`
ON `notification` (`user_id`, `audience`, `app_uid`, `id`);
+17 -12
View File
@@ -666,9 +666,18 @@ export type EventMap = {
// and going away (1→0, whether by removing the last handler or by the
// destroy route). `actor.user` is the app's owner, not the caller — a
// developer session publishing for an app it owns is the common case, but
// billing follows ownership.
'events.worker.create': { actor: Actor; appUid: string };
'events.worker.destroy': { actor: Actor; appUid: string };
// billing follows ownership. For `scope: 'user'` the unit is one user's
// handlers in one app instead, and `actor.user` is that user.
'events.worker.create': {
actor: Actor;
appUid: string;
scope: 'app' | 'user';
};
'events.worker.destroy': {
actor: Actor;
appUid: string;
scope: 'app' | 'user';
};
// ---- Outer / GUI broadcast ----
'outer.cacheUpdate': {
@@ -1032,11 +1041,10 @@ export type EventKey = keyof EventMap & string;
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
? | `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
@@ -1047,10 +1055,7 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
/** Acked, not acked, no socket anywhere, or sockets only in a peer region. */
export type NotifDeliveryOutcome =
| 'received'
| 'unreceived'
| 'no-socket'
| 'remote-only';
'received' | 'unreceived' | 'no-socket' | 'remote-only';
/** One socket's receipt for a live notification. */
export interface NotifDeliveryResponder {
@@ -23,16 +23,20 @@
*/
import type { fetch as undiciFetch } from 'undici';
import { describe, expect, it, vi } from 'vitest';
import { afterEach, describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types.js';
import {
DispatcherInvokeTransport,
EVENTS_DEPLOYED_HEADER,
EVENTS_DEADLINE_MARGIN_MS,
EVENTS_DISPATCH_BATCH_PATH,
EVENTS_DISPATCH_ERROR_HEADER,
EVENTS_DISPATCH_PATH,
EVENTS_ERROR_HEADER,
EVENTS_HANDLED_HEADER,
EVENTS_HOLDER_HEADER,
EVENTS_INVOKE_TIMEOUT_MS,
EVENTS_WORKER_LEASE_FACTOR,
EventsWorkerInvokerClient,
INVOKE_CONNECTIONS,
SCHEDULE_INVOKE_CONNECTIONS,
@@ -1302,3 +1306,217 @@ describe('outcomeForStatus', () => {
},
);
});
describe('EventsWorkerInvokerClient overall deadline', () => {
afterEach(() => {
vi.useRealTimers();
});
const LEASE_MS = EVENTS_INVOKE_TIMEOUT_MS * EVENTS_WORKER_LEASE_FACTOR;
/**
* A transport on a fake clock: each send costs what `cost` says, and
* answers a miss until the deployed header, then a handled 200.
*/
const timedTransport = (cost: (deployed: boolean) => number) => {
const sends: Array<{ deployed: boolean; timeoutMs: number }> = [];
const transport = {
send: async (call: {
deployed?: boolean;
timeoutMs: number;
}): Promise<unknown> => {
const deployed = call.deployed === true;
sends.push({ deployed, timeoutMs: call.timeoutMs });
// A send never outlasts its own timeout.
vi.advanceTimersByTime(
Math.min(cost(deployed), call.timeoutMs * 1.5),
);
return deployed
? { status: 200, handled: true }
: {
status: null,
dispatchReason: 'missing',
error: 'dispatcher: missing (404)',
};
},
};
return { transport, sends };
};
const run = async (
cost: (deployed: boolean) => number,
deployMs: number,
) => {
vi.useFakeTimers({ toFake: ['Date'] });
const { transport, sends } = timedTransport(cost);
const client = makeClient();
client.setTransport(transport as never);
client.setMissHandler(async () => {
vi.advanceTimersByTime(deployMs);
return 'deployed';
});
const started = Date.now();
const result = await client.invoke(REQUEST);
return { result, sends, elapsed: Date.now() - started };
};
it('leaves a fast miss as it was: a full-timeout resend', async () => {
const { result, sends } = await run(() => 100, 2_000);
expect(result).toEqual({ outcome: 'settled', status: 200 });
expect(sends).toEqual([
{ deployed: false, timeoutMs: EVENTS_INVOKE_TIMEOUT_MS },
{ deployed: true, timeoutMs: EVENTS_INVOKE_TIMEOUT_MS },
]);
});
it('shortens the resend after a slow deploy, so the call ends before the lease', async () => {
// A miss that took 12 s, a deploy that took 30 s, and a resend that
// would use everything it is given.
const { result, sends, elapsed } = await run(
(deployed) => (deployed ? 1e9 : 12_000),
30_000,
);
expect(result.outcome).toBe('settled');
expect(sends[1].deployed).toBe(true);
expect(sends[1].timeoutMs).toBeLessThan(EVENTS_INVOKE_TIMEOUT_MS);
expect(elapsed).toBeLessThanOrEqual(
LEASE_MS - EVENTS_DEADLINE_MARGIN_MS,
);
});
it('gives up on a deploy that never settles before the lease, leaving it running', async () => {
vi.useFakeTimers();
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
const { transport, sends } = timedTransport(() => 0);
const client = makeClient();
client.setTransport(transport as never);
let rejectDeploy!: (err: Error) => void;
client.setMissHandler(
() =>
new Promise<string>((_resolve, reject) => {
rejectDeploy = reject;
}),
);
let finished: unknown = null;
const pending = client.invoke(REQUEST).then((r) => (finished = r));
// Up to where the call must have ended, and no further.
await vi.advanceTimersByTimeAsync(LEASE_MS - EVENTS_DEADLINE_MARGIN_MS);
expect(finished).toMatchObject({
outcome: 'unavailable',
status: null,
});
await pending;
expect(sends).toHaveLength(1);
// The deploy was not cancelled; its late failure is logged, not thrown.
rejectDeploy(new Error('stalled'));
await vi.advanceTimersByTimeAsync(0);
expect(warn).toHaveBeenCalledWith(
'[events] deploy failed late',
expect.any(Error),
);
warn.mockRestore();
});
it('does not resend when the deploy leaves less than the margin', async () => {
const { result, sends } = await run(
() => 100,
LEASE_MS - EVENTS_DEADLINE_MARGIN_MS * 2 + 1_000,
);
expect(sends).toHaveLength(1);
expect(result).toMatchObject({ outcome: 'unavailable', status: null });
});
});
describe('the holder of a user scope on the wire', () => {
const HOLDER = 'user-uuid-1';
it('sends the holder header on a single invoke, and none for an app scope', async () => {
const seen: Array<Record<string, string>> = [];
const fetchImpl = (async (_url: string, init?: RequestInit) => {
seen.push((init?.headers ?? {}) as Record<string, string>);
return new Response(null, {
status: 200,
headers: { [EVENTS_HANDLED_HEADER]: '1' },
});
}) as unknown as FetchImpl;
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl,
batchSize: 1,
}),
);
await client.invoke({ ...REQUEST, holder: HOLDER });
await client.invoke(REQUEST);
expect(seen[0][EVENTS_HOLDER_HEADER]).toBe(HOLDER);
expect(EVENTS_HOLDER_HEADER in seen[1]).toBe(false);
});
it('puts a holder field on a batch call of a user scope only', async () => {
const bodies: Array<{ calls: Array<Record<string, unknown>> }> = [];
const fetchImpl = (async (_url: string, init?: RequestInit) => {
bodies.push(JSON.parse(String(init?.body)));
return new Response(
'{"i":0,"status":200,"handled":true}\n{"i":1,"status":200,"handled":true}\n',
{ status: 200 },
);
}) as unknown as FetchImpl;
const transport = new DispatcherInvokeTransport(
'http://dispatcher',
's',
{ fetchImpl },
);
await Promise.all([
transport.send({ ...CALL, holder: HOLDER }),
transport.send(CALL),
]);
expect(bodies).toHaveLength(1);
expect(bodies[0].calls[0].holder).toBe(HOLDER);
expect('holder' in bodies[0].calls[1]).toBe(false);
});
it('hands the holder to the miss handler, and nothing for an app scope', async () => {
const seen: Array<{ holder?: string }> = [];
const client = makeClient();
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl: missUntilDeployedFetch([]),
batchSize: 1,
}),
);
client.setMissHandler(async (_appUid, _script, options) => {
seen.push(options);
return 'deployed';
});
await client.invoke({ ...REQUEST, holder: HOLDER });
await client.invoke(REQUEST);
expect(seen[0].holder).toBe(HOLDER);
expect('holder' in seen[1]).toBe(false);
});
it('keeps the holder out of the body the script sees', async () => {
const bodies: string[] = [];
const client = makeClient();
client.setTransport({
send: async (call) => {
bodies.push(call.body);
expect(call.holder).toBe(HOLDER);
return { status: 200, handled: true };
},
});
await client.invoke({ ...REQUEST, holder: HOLDER });
expect(Object.keys(JSON.parse(bodies[0])).sort()).toEqual([
'ctx',
'event',
'handler',
'token',
]);
});
});
@@ -35,6 +35,7 @@ import { PuterClient } from '../types.js';
* x-puter-internal-auth: <internal secret>
* x-puter-events-script: <script the handler set deploys as>
* x-puter-events-app: <app uid, for the deploy-on-miss callback>
* x-puter-events-holder: <holder uuid of a user scope; absent for app scope>
* x-puter-events-key: <key derived for that script>
* { handler, event, ctx, token }
*
@@ -48,7 +49,7 @@ import { PuterClient } from '../types.js';
*
* POST <events dispatcher>/invoke-batch
* x-puter-internal-auth: <internal secret>
* { calls: [{ script, appUid, key, deployed?, body, waitMs }] }
* { calls: [{ script, appUid, holder?, key, deployed?, body, waitMs }] }
*
* The dispatcher runs a few at a time, in order, and streams NDJSON: `{ i,
* started: true }` when a call goes to its worker, then `{ i, status, handled?,
@@ -127,6 +128,12 @@ const PLATFORM_ERROR_CODES = new Set([
*/
export const EVENTS_DEPLOYED_HEADER = 'x-puter-events-deployed';
/**
* Names the user whose scope a script belongs to, so a miss deploys it from
* their handlers. Absent for an app's own scope.
*/
export const EVENTS_HOLDER_HEADER = 'x-puter-events-holder';
/**
* Dispatch-error reasons this backend can resolve itself by deploying, rather
* than wait on the rehydrate callback finding a backend behind the dispatcher's
@@ -150,6 +157,11 @@ const ALREADY_DEPLOYED_MISS_REASONS = new Set(['deploy-timeout']);
/** Whose call asked for the deploy, for the share of deploys it may take. */
export interface WorkerMissOptions {
lane?: InvokeLane;
/**
* Holder uuid of the user scope the script belongs to; absent for app
* scope.
*/
holder?: string;
}
/** What a self-deploy attempt did, and to which app/script it applies. */
@@ -159,9 +171,55 @@ export type WorkerMissHandler = (
options: WorkerMissOptions,
) => Promise<'deployed' | string>;
/**
* `work`'s result, or null once `ms` has passed. The work keeps running, and a
* failure after the cutoff is logged since nothing else would see it.
*/
const withinTime = async <T>(
work: Promise<T>,
ms: number,
): Promise<T | null> => {
let timedOut = false;
let timer: ReturnType<typeof setTimeout> | undefined;
const limit = new Promise<null>((resolve) => {
timer = setTimeout(
() => {
timedOut = true;
resolve(null);
},
Math.max(0, ms),
);
});
void (async () => {
try {
await work;
} catch (err) {
if (timedOut) console.warn('[events] deploy failed late', err);
}
})();
try {
return await Promise.race([work, limit]);
} finally {
clearTimeout(timer);
}
};
/** How long a handler has to answer before the attempt is abandoned. */
export const EVENTS_INVOKE_TIMEOUT_MS = 30_000;
/**
* How many invoke timeouts a delivery's worker lease lasts. The lease store
* reads this, and `invoke()` keeps a whole call, deploy and resend included,
* inside it.
*/
export const EVENTS_WORKER_LEASE_FACTOR = 2;
/**
* Left unspent at the end of the lease, and the least a resend is worth
* starting with: below it the call is given up as `unavailable` and retried.
*/
export const EVENTS_DEADLINE_MARGIN_MS = 5_000;
/**
* Connections held open to the dispatcher, so a busy fleet is not
* re-handshaking. Also how many calls one transport makes at once: a call past
@@ -194,6 +252,8 @@ export interface WorkerInvokeRequest {
/** Script the app's current handler set deploys as. */
script: string;
appUid: string;
/** Holder uuid, when the handler lives in a user scope. */
holder?: string;
handler: string;
/** The subscriber's app session token. */
token: string;
@@ -207,6 +267,8 @@ export interface WorkerInvokeRequest {
export interface EventsInvokeCall {
script: string;
appUid: string;
/** Holder uuid of the user scope the script belongs to. */
holder?: string;
/** Value the script compares against its own invoke-key binding. */
key: string;
/** Serialized `{ handler, event, ctx, token }`. */
@@ -255,10 +317,7 @@ export interface EventsInvokeTransport {
* handler); retried like `retriable` but never the handler's fault.
*/
export type WorkerInvokeOutcome =
| 'settled'
| 'terminal'
| 'retriable'
| 'unavailable';
'settled' | 'terminal' | 'retriable' | 'unavailable';
export interface WorkerInvokeResult {
outcome: WorkerInvokeOutcome;
@@ -329,25 +388,62 @@ export class EventsWorkerInvokerClient extends PuterClient {
ctx: request.ctx,
token: request.token,
});
const send = (deployed: boolean): Promise<EventsInvokeAnswer> =>
const timeoutMs = this.#timeoutMs();
// The whole call, deploy and resend included, ends before the lease a
// delivery holds would let it be claimed again mid-run.
// A short configured timeout gets a proportionally short margin.
const margin = Math.min(EVENTS_DEADLINE_MARGIN_MS, timeoutMs / 2);
const deadline =
Date.now() + timeoutMs * EVENTS_WORKER_LEASE_FACTOR - margin;
const send = (
deployed: boolean,
sendTimeoutMs = timeoutMs,
): Promise<EventsInvokeAnswer> =>
transport.send({
script: request.script,
appUid: request.appUid,
...(request.holder ? { holder: request.holder } : {}),
key: request.key,
body: render(),
timeoutMs: this.#timeoutMs(),
timeoutMs: sendTimeoutMs,
...(deployed ? { deployed: true } : {}),
...(options.lane ? { lane: options.lane } : {}),
...(stampEvent ? { render } : {}),
});
// A send may wait for a connection for half its timeout before the
// run's own timeout starts, so it is given what is left divided by that.
const resend = (): Promise<EventsInvokeAnswer> | null => {
const remaining = deadline - Date.now();
if (remaining < margin) return null;
return send(
true,
Math.floor(
Math.min(
timeoutMs,
remaining / (1 + CONNECTION_WAIT_SHARE),
),
),
);
};
const outOfTime = (): WorkerInvokeResult => ({
outcome: 'unavailable',
status: null,
error: 'out of time to resend after deploy',
});
let result = await send(false);
if (this.#missHandler && selfDeployable(result)) {
const outcome = await this.#missHandler(
request.appUid,
request.script,
{ lane: options.lane },
// The deploy may be shared with other callers, so it is left
// running when this call runs out of time for it.
const outcome = await withinTime(
this.#missHandler(request.appUid, request.script, {
lane: options.lane,
...(request.holder ? { holder: request.holder } : {}),
}),
deadline - Date.now() - margin,
);
if (outcome === null) return outOfTime();
if (outcome !== 'deployed')
return {
outcome: 'unavailable',
@@ -355,7 +451,9 @@ export class EventsWorkerInvokerClient extends PuterClient {
error: `deploy: ${outcome}`,
...(outcome === DEPLOY_BUSY ? { deployBusy: true } : {}),
};
result = await send(true);
const resent = await resend();
if (!resent) return outOfTime();
result = resent;
} else if (
result.status === null &&
result.dispatchReason &&
@@ -363,7 +461,9 @@ export class EventsWorkerInvokerClient extends PuterClient {
) {
// No deploy to trigger — just give propagation the one retry
// it usually needs, without spending an upload on it.
result = await send(true);
const resent = await resend();
if (!resent) return outOfTime();
result = resent;
}
const { status, handled, error, timedOut, retryAfterMs } = result;
@@ -546,6 +646,9 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
'x-puter-internal-auth': this.#secret,
'x-puter-events-script': call.script,
'x-puter-events-app': call.appUid,
...(call.holder
? { [EVENTS_HOLDER_HEADER]: call.holder }
: {}),
'x-puter-events-key': call.key,
...(call.deployed
? { [EVENTS_DEPLOYED_HEADER]: '1' }
@@ -610,6 +713,7 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
calls: batch.map(({ call }, i) => ({
script: call.script,
appUid: call.appUid,
...(call.holder ? { holder: call.holder } : {}),
key: call.key,
...(call.deployed ? { deployed: true } : {}),
body: call.render ? call.render() : call.body,
@@ -1416,7 +1416,6 @@ describe('AuthController.handleLogin', () => {
is_user_token: true,
});
expect(body.user.directories).toBeTypeOf('object');
expect(Array.isArray(body.user.taskbar_items)).toBe(true);
});
// A deleted row still answering from cache reaches the session INSERT.
@@ -25,18 +25,13 @@ const ALLOWED_SORT_BY = ['name', 'size', 'modified', 'type'];
const ALLOWED_SORT_ORDER = ['asc', 'desc'];
/**
* Desktop/UI preference routes.
*
* Two categories:
*
* - User-level: desktop background, taskbar items (UserStore)
* - Folder-level: layout, sort_by/sort_order (fsentries table)
* Folder view preference routes: an explorer window's layout and sort order,
* kept on the folder's fsentry.
*/
/**
* Desktop preference writes — background, taskbar, layout, sort order. All four
* persist to the user row on every call, and the GUI fires them on direct user
* action, so a per-minute ceiling well above human speed is enough to catch a
* stuck client.
* Folder preference writes — layout, sort order. The GUI fires them on direct
* user action, so a per-minute ceiling well above human speed is enough to
* catch a stuck client.
*/
const PREFERENCE_WRITE_LIMIT = {
scope: 'desktop-preference',
@@ -47,89 +42,6 @@ const PREFERENCE_WRITE_LIMIT = {
export class DesktopController extends PuterController {
registerRoutes(router) {
// -- Desktop background --------------------------------------
router.post(
'/set-desktop-bg',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { url, color, fit } = req.body ?? {};
const patch = {};
if (url !== undefined) {
if (url !== null && typeof url !== 'string') {
throw new HttpError(
400,
'`url` must be a string or null',
{ legacyCode: 'bad_request' },
);
}
patch.desktop_bg_url = url;
}
if (color !== undefined) {
if (color !== null && typeof color !== 'string') {
throw new HttpError(
400,
'`color` must be a string or null',
{ legacyCode: 'bad_request' },
);
}
patch.desktop_bg_color = color;
}
if (fit !== undefined) {
if (fit !== null && typeof fit !== 'string') {
throw new HttpError(
400,
'`fit` must be a string or null',
{ legacyCode: 'bad_request' },
);
}
patch.desktop_bg_fit = fit;
}
if (Object.keys(patch).length === 0) {
throw new HttpError(400, 'No fields provided', {
legacyCode: 'bad_request',
});
}
await this.stores.user.update(req.actor.user.id, patch);
res.json({});
},
);
// -- Taskbar items -------------------------------------------
router.post(
'/update-taskbar-items',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { items } = req.body ?? {};
if (!Array.isArray(items)) {
throw new HttpError(
400,
'Missing or invalid `items` array',
{ legacyCode: 'bad_request' },
);
}
await this.stores.user.update(req.actor.user.id, {
taskbar_items: JSON.stringify(items),
});
res.json({});
},
);
// -- Folder layout -------------------------------------------
router.post(
@@ -130,150 +130,6 @@ const callRoute = async (
});
};
// ── /set-desktop-bg ─────────────────────────────────────────────────
describe('DesktopController POST /set-desktop-bg', () => {
it('persists url/color/fit on the user row', async () => {
const { actor, userId } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({
body: {
url: 'https://cdn.test/wallpaper.png',
color: '#000000',
fit: 'cover',
},
actor,
}),
res,
);
expect(captured.body).toEqual({});
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.desktop_bg_url).toBe(
'https://cdn.test/wallpaper.png',
);
expect(refreshed?.desktop_bg_color).toBe('#000000');
expect(refreshed?.desktop_bg_fit).toBe('cover');
});
it('persists only the supplied fields (partial update)', async () => {
const { actor, userId } = await makeUser();
const { res } = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { color: '#ffffff' }, actor }),
res,
);
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.desktop_bg_color).toBe('#ffffff');
// Untouched fields remain at their column default.
expect(refreshed?.desktop_bg_url).toBeFalsy();
expect(refreshed?.desktop_bg_fit).toBeFalsy();
});
it('passes through `null` to clear a field', async () => {
const { actor, userId } = await makeUser();
// First populate, then clear.
const populate = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({
body: { url: 'https://cdn.test/x.png' },
actor,
}),
populate.res,
);
const before = await server.stores.user.getById(userId);
expect(before?.desktop_bg_url).toBe('https://cdn.test/x.png');
const clear = makeRes();
await callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { url: null }, actor }),
clear.res,
);
const after = await server.stores.user.getById(userId);
expect(after?.desktop_bg_url).toBeNull();
});
it('throws 400 when url is not a string or null', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: { url: 123 }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when no fields are provided', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/set-desktop-bg',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── /update-taskbar-items ───────────────────────────────────────────
describe('DesktopController POST /update-taskbar-items', () => {
it('persists items as a JSON-encoded string', async () => {
const { actor, userId } = await makeUser();
const items = [{ name: 'editor' }, { name: 'browser' }];
const { res } = makeRes();
await callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: { items }, actor }),
res,
);
const refreshed = await server.stores.user.getById(userId);
expect(refreshed?.taskbar_items).toBe(JSON.stringify(items));
});
it('throws 400 when items is missing', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: {}, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when items is not an array', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/update-taskbar-items',
makeReq({ body: { items: 'oops' }, actor }),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── /set_layout ─────────────────────────────────────────────────────
describe('DesktopController POST /set_layout', () => {
@@ -64,6 +64,8 @@ const EVENTS_SCRIPT_REGEX = new RegExp(`^${EVENTS_WORKER_PREFIX}[a-f0-9]{32}$`);
/** Matches the edge dispatcher's own validation of the app uid it forwards. */
const APP_UID_REGEX = /^app-[A-Za-z0-9_-]{1,64}$/;
/** A user uuid. */
const HOLDER_REGEX = /^[A-Za-z0-9-]{1,64}$/;
@Controller('/events')
export class EventsController extends PuterController {
@@ -132,7 +134,8 @@ export class EventsController extends PuterController {
*
* Stateless: no subscription, no stored position, nothing written. The
* cursor comes back to the caller and goes out again as `cursor` (or
* `after`, the name it first shipped under).
* `after`, the name it first shipped under). `from=end` with no cursor
* starts at the newest row instead of the oldest.
*/
@Get('/fetch', {
subdomain: 'api',
@@ -143,9 +146,16 @@ export class EventsController extends PuterController {
async fetch(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const from = query.from as 'start' | 'end' | undefined;
if (from !== undefined && from !== 'start' && from !== 'end') {
throw new HttpError(400, '`from` must be `start` or `end`', {
legacyCode: 'bad_request',
});
}
const page = await this.services.events.fetchMissed(actor, {
subject: typeof query.subject === 'string' ? query.subject : '',
from,
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
after: typeof query.after === 'string' ? query.after : undefined,
limit: normalizeLimit(query.limit, {
@@ -402,6 +412,7 @@ export class EventsController extends PuterController {
handlers: await this.services.events.listHandlers(actor, {
appUid:
typeof query.appUid === 'string' ? query.appUid : undefined,
scope: query.scope,
}),
});
}
@@ -500,6 +511,15 @@ export class EventsController extends PuterController {
throw new HttpError(400, 'Missing or invalid `script`/`appUid`', {
legacyCode: 'bad_request',
});
// Absent or empty is the app's own scope.
const holder = body.holder === undefined ? '' : body.holder;
if (
typeof holder !== 'string' ||
!(holder === '' || HOLDER_REGEX.test(holder))
)
throw new HttpError(400, 'Invalid `holder`', {
legacyCode: 'bad_request',
});
const deployer = this.#eventsWorkerDeployer();
if (!deployer.enabled) {
@@ -507,7 +527,13 @@ export class EventsController extends PuterController {
return;
}
const outcome = await deployer.ensure(appUid, script);
const scope = await deployer.scopeFor(appUid, holder);
if (!scope) {
res.status(404).json({ deployed: false, reason: 'no-holder' });
return;
}
const outcome = await deployer.ensure(scope, script);
if (outcome === 'deployed') {
res.json({ deployed: true });
return;
@@ -540,14 +566,22 @@ export class EventsController extends PuterController {
override onServerStart(): void {
const deployer = this.#eventsWorkerDeployer();
if (!deployer.enabled) return;
this.services.events.useEagerDeploy((scope, script) =>
deployer.deployEagerly(scope, script),
);
if (this.config.workers?.localServer) {
this.services.events.useWorkerTransport(
new LocalEventsInvokeTransport(this.services, deployer),
);
return;
}
this.services.events.useWorkerMissHandler((appUid, script, options) =>
deployer.ensure(appUid, script, options),
this.services.events.useWorkerMissHandler(
async (appUid, script, options) => {
const scope = await deployer.scopeFor(appUid, options.holder);
return scope
? deployer.ensure(scope, script, options)
: 'no-holder';
},
);
}
@@ -34,12 +34,21 @@ import {
EVENTS_DURABLE_SUBSCRIPTIONS_MAX,
EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP,
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER,
EVENTS_HANDLER_PUBLISH_LIMIT,
EVENTS_HANDLER_SOURCE_MAX_BYTES,
EVENTS_HANDLERS_PER_APP,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_USER,
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT,
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT,
EVENTS_SINGLE_DELIVERY_LIMIT,
EVENTS_USER_HANDLERS_PER_SCOPE,
EVENTS_USER_SCOPE_SOURCE_MAX_BYTES,
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_INVOCATION_LIMIT,
EVENTS_WORKER_SOURCE_MAX_BYTES,
RENTED_UNITS_PER_USER,
limitFor,
tiered,
type TieredLimit,
@@ -183,3 +192,42 @@ describe('the delivery budgets', () => {
}
});
});
describe('the handler limits', () => {
it('pins the app-scope numbers the quotas page lists', () => {
expect(EVENTS_HANDLERS_PER_APP).toBe(100);
expect(EVENTS_HANDLER_SOURCE_MAX_BYTES).toBe(64 * 1024);
expect(EVENTS_WORKER_SOURCE_MAX_BYTES).toBe(5 * 1024 * 1024);
expect(EVENTS_WORKER_DEPLOYS_PER_HOUR).toBe(30);
expect(EVENTS_HANDLER_PUBLISH_LIMIT.limit).toBe(60);
expect(EVENTS_HANDLER_PUBLISH_LIMIT.window).toBe(60_000);
});
it('pins the user-scope numbers the quotas page lists', () => {
expect(EVENTS_USER_HANDLERS_PER_SCOPE).toBe(20);
expect(EVENTS_USER_SCOPE_SOURCE_MAX_BYTES).toBe(1024 * 1024);
expect(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR).toBe(10);
});
it('keeps a user scope tighter than an app scope', () => {
expect(EVENTS_USER_HANDLERS_PER_SCOPE).toBeLessThan(
EVENTS_HANDLERS_PER_APP,
);
expect(EVENTS_USER_SCOPE_SOURCE_MAX_BYTES).toBeLessThan(
EVENTS_WORKER_SOURCE_MAX_BYTES,
);
expect(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR).toBeLessThan(
EVENTS_WORKER_DEPLOYS_PER_HOUR,
);
});
it('pins the rented-unit cap by plan', () => {
expect(RENTED_UNITS_PER_USER.limit).toBe(100);
expect(limitFor(RENTED_UNITS_PER_USER, DEFAULT_FREE_SUBSCRIPTION)).toBe(
16,
);
expect(limitFor(RENTED_UNITS_PER_USER, DEFAULT_TEMP_SUBSCRIPTION)).toBe(
16,
);
});
});
+19
View File
@@ -269,6 +269,12 @@ export const KV_SHARE_NOTIFY_LIMITS = [
*/
export const EVENTS_HANDLERS_PER_APP = 100;
/**
* Handlers one user may hold in their own scope of one app. Each is a script of
* its own, so the cap is tighter than the app's.
*/
export const EVENTS_USER_HANDLERS_PER_SCOPE = 20;
/** Longest a serialized handler may be. */
export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
@@ -280,6 +286,9 @@ export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
*/
export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
/** Source one user scope may hold, all its handlers combined. */
export const EVENTS_USER_SCOPE_SOURCE_MAX_BYTES = 1024 * 1024;
/**
* Deploys one app's events worker may trigger per hour.
*
@@ -289,6 +298,16 @@ export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
*/
export const EVENTS_WORKER_DEPLOYS_PER_HOUR = 30;
/** Deploys one user scope may trigger per hour. */
export const EVENTS_USER_WORKER_DEPLOYS_PER_HOUR = 10;
/**
* Units one user may rent at once: ordinary workers, user scopes, and the
* events workers of apps they own. Only creating a unit is refused; holdings
* over the cap are kept.
*/
export const RENTED_UNITS_PER_USER = tiered(100, 16, 16);
/**
* Handlers one `publishAll` may carry. A build step publishes its whole set in
* one call, and the set is capped by what an app may hold anyway.
@@ -25,14 +25,35 @@
import MockRedis from 'ioredis-mock';
import { describe, expect, it } from 'vitest';
import type { EventsInvokeCall } from '../../clients/events/EventsWorkerInvokerClient.js';
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
import type { fetch as undiciFetch } from 'undici';
import {
DispatcherInvokeTransport,
EVENTS_DEPLOYED_HEADER,
EVENTS_DISPATCH_ERROR_HEADER,
EVENTS_HANDLED_HEADER,
EventsWorkerInvokerClient,
type EventsInvokeCall,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import { INTERNAL_DEPLOY_TARGET } from '../../drivers/workers/WorkerDriver.js';
import {
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_DEPLOYS_PER_HOUR,
} from './limits.js';
import {
EventsWorkerDeployer,
LocalEventsInvokeTransport,
type EventsDeployOptions,
type EventsDeployOutcome,
} from './workerDeploy.js';
import { eventsWorkerScript } from '../../services/events/workerRuntime.js';
import {
eventsWorkerScript,
userHandlerScript,
} from '../../services/events/workerRuntime.js';
import {
appHandlerScope,
hashContent,
type HandlerScope,
} from '../../stores/events/EventHandlerStore.js';
import { handlerSetHash } from '../../services/events/workerSource.js';
import type { IConfig } from '../../types.js';
@@ -51,14 +72,22 @@ interface Owner {
type Layers = ConstructorParameters<typeof EventsWorkerDeployer>[0];
/** Stub layers whose only real dependency is a fresh redis mock. */
const makeLayers = (owners: Record<string, Owner | null>) => {
const makeLayers = (
owners: Record<string, Owner | null>,
events: Record<string, unknown> = {},
) => {
const redis = new MockRedis.Cluster(['redis://localhost:7001']);
const created: string[] = [];
const targets: Array<Record<string, unknown>> = [];
/** Apps whose handler sources were read, once per read. */
const sourceReads: string[] = [];
const layers = {
config: {
events: { workerRuntime: true, internalSecret: 'sekret' },
events: {
workerRuntime: true,
internalSecret: 'sekret',
...events,
},
// `#namespace()` needs a local-worker shape to resolve without
// throwing — this stub never reaches a real Cloudflare or
// Miniflare deploy either way.
@@ -68,7 +97,7 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
stores: {
eventHandler: {
setForApp: async () => [{ name: 'h', sourceHash: SOURCE_HASH }],
allForApp: async (appUid: string) => {
allForApp: async ({ appUid }: { appUid: string }) => {
sourceReads.push(appUid);
return [
{
@@ -91,6 +120,18 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
},
},
user: {
getByUuid: async (uuid: string) => {
const owner = Object.values(owners).find(
(o) => `uuid-${o?.ownerUserId}` === uuid,
);
if (!owner) return null;
return {
id: owner.ownerUserId,
uuid,
username: `user-${owner.ownerUserId}`,
suspended: owner.suspended ?? false,
};
},
getById: async (id: number) => {
const owner = Object.values(owners).find(
(o) => o?.ownerUserId === id,
@@ -108,8 +149,12 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
services: {},
drivers: {
workers: {
create: async (args: { workerName: string }) => {
create: async (args: {
workerName: string;
[INTERNAL_DEPLOY_TARGET]?: Record<string, unknown>;
}) => {
created.push(args.workerName);
targets.push(args[INTERNAL_DEPLOY_TARGET] ?? {});
return { success: true, errors: [] };
},
},
@@ -118,6 +163,7 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
return {
layers: layers as unknown as Layers,
created,
targets,
redis,
sourceReads,
};
@@ -136,8 +182,8 @@ describe('EventsWorkerDeployer.ensure', () => {
const deployer = new EventsWorkerDeployer(layers);
const [a, b] = await Promise.all([
deployer.ensure('app-a', SCRIPT),
deployer.ensure('app-b', SCRIPT),
deployer.ensure(appHandlerScope('app-a'), SCRIPT),
deployer.ensure(appHandlerScope('app-b'), SCRIPT),
]);
expect(a).toBe('deployed');
expect(b).toBe('no-owner');
@@ -154,7 +200,9 @@ describe('EventsWorkerDeployer.ensure', () => {
String(EVENTS_WORKER_DEPLOYS_PER_HOUR),
);
expect(await deployer.ensure('app-c', SCRIPT)).toBe('throttled');
expect(await deployer.ensure(appHandlerScope('app-c'), SCRIPT)).toBe(
'throttled',
);
expect(sourceReads).toEqual([]);
expect(created).toEqual([]);
});
@@ -169,16 +217,41 @@ describe('EventsWorkerDeployer.ensure', () => {
'',
);
expect(await deployer.ensure('app-e', stale)).toBe('stale');
expect(await deployer.ensure(appHandlerScope('app-e'), stale)).toBe(
'stale',
);
expect(sourceReads).toEqual([]);
const hour = new Date().toISOString().slice(0, 13);
expect(await redis.get(`ev:deploys:{app-e}:${hour}`)).toBeNull();
});
it('passes events.workerCpuMs to the deploy target', async () => {
const { layers, targets } = makeLayers(
{ 'app-f': { ownerUserId: 6 } },
{ workerCpuMs: 50 },
);
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure(appHandlerScope('app-f'), SCRIPT)).toBe(
'deployed',
);
expect(targets[0]!.cpuMs).toBe(50);
});
it('passes no cpuMs when events.workerCpuMs is unset', async () => {
const { layers, targets } = makeLayers({ 'app-g': { ownerUserId: 7 } });
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure(appHandlerScope('app-g'), SCRIPT)).toBe(
'deployed',
);
expect(targets[0]!.cpuMs).toBeUndefined();
});
it('deploys normally under the cap', async () => {
const { layers } = makeLayers({ 'app-d': { ownerUserId: 4 } });
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure('app-d', SCRIPT)).toBe('deployed');
expect(await deployer.ensure(appHandlerScope('app-d'), SCRIPT)).toBe(
'deployed',
);
});
});
@@ -209,16 +282,22 @@ describe('the schedules` deploy gate', () => {
it('is skipped without schedules, or with schedules that have no gate', async () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
await expect(
new EventsWorkerDeployer(layers).ensure('app-x', SCRIPT, {
lane: 'schedule',
}),
new EventsWorkerDeployer(layers).ensure(
appHandlerScope('app-x'),
SCRIPT,
{
lane: 'schedule',
},
),
).resolves.toBe('deployed');
const gateless = new EventsWorkerDeployer({
...layers,
services: { ...layers.services, schedules: {} },
} as unknown as Layers);
await expect(
gateless.ensure('app-x', SCRIPT, { lane: 'schedule' }),
gateless.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
}),
).resolves.toBe('deployed');
expect(created).toEqual([SCRIPT, SCRIPT]);
});
@@ -227,11 +306,13 @@ describe('the schedules` deploy gate', () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer, gated } = withGate(layers);
await expect(deployer.ensure('app-x', SCRIPT)).resolves.toBe(
'deployed',
);
await expect(
deployer.ensure('app-x', SCRIPT, { lane: 'schedule' }),
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
).resolves.toBe('deployed');
await expect(
deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
}),
).resolves.toBe('deployed');
expect(gated).toEqual([
{ script: SCRIPT, capped: false },
@@ -245,11 +326,13 @@ describe('the schedules` deploy gate', () => {
const { deployer } = withGate(layers, (capped) => capped);
await expect(
deployer.ensure('app-x', SCRIPT, { lane: 'schedule' }),
deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
}),
).resolves.toBe('busy');
await expect(deployer.ensure('app-x', SCRIPT)).resolves.toBe(
'deployed',
);
await expect(
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
).resolves.toBe('deployed');
expect(created).toEqual([SCRIPT]);
});
@@ -257,12 +340,12 @@ describe('the schedules` deploy gate', () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer, gated } = withGate(layers, (capped) => capped);
const scheduled = deployer.ensure('app-x', SCRIPT, {
const scheduled = deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
});
const deliveries = [
deployer.ensure('app-x', SCRIPT),
deployer.ensure('app-x', SCRIPT),
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
];
await expect(scheduled).resolves.toBe('busy');
await expect(Promise.all(deliveries)).resolves.toEqual([
@@ -277,8 +360,8 @@ describe('the schedules` deploy gate', () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer, gated } = withGate(layers, (capped) => capped);
const delivery = deployer.ensure('app-x', SCRIPT);
const scheduled = deployer.ensure('app-x', SCRIPT, {
const delivery = deployer.ensure(appHandlerScope('app-x'), SCRIPT);
const scheduled = deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
});
await expect(Promise.all([delivery, scheduled])).resolves.toEqual([
@@ -312,6 +395,7 @@ describe('LocalEventsInvokeTransport', () => {
resident = true;
return 'deployed';
},
scopeFor: async (appUid: string) => appHandlerScope(appUid),
} as unknown as EventsWorkerDeployer,
);
let renders = 0;
@@ -329,3 +413,484 @@ describe('LocalEventsInvokeTransport', () => {
expect(bodies).toEqual(['{"n":1}', '{"n":2}']);
});
});
describe('deploy-on-miss of an evicted script', () => {
it('redeploys the evicted script, then resends the run with the deployed marker', async () => {
const { layers, created } = makeLayers({ 'app-m': { ownerUserId: 8 } });
// The namespace only has the script once the deploy lands: before
// that the dispatcher answers a miss.
let resident = false;
layers.drivers.workers.create = (async (args: {
workerName: string;
}) => {
created.push(args.workerName);
resident = true;
return { success: true, errors: [] };
}) as never;
const deployer = new EventsWorkerDeployer(layers);
const sent: boolean[] = [];
const fetchImpl = (async (_url: string, init?: RequestInit) => {
const headers = (init?.headers ?? {}) as Record<string, string>;
sent.push(headers[EVENTS_DEPLOYED_HEADER] === '1');
if (resident)
return new Response(null, {
status: 200,
headers: { [EVENTS_HANDLED_HEADER]: '1' },
});
return new Response(null, {
status: 404,
headers: { [EVENTS_DISPATCH_ERROR_HEADER]: 'missing' },
});
}) as unknown as typeof undiciFetch;
const client = new EventsWorkerInvokerClient({
events: { workerRuntime: true },
} as IConfig);
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl,
}),
);
client.setMissHandler((appUid, script, options) =>
deployer.ensure(appHandlerScope(appUid), script, options),
);
const result = await client.invoke({
script: SCRIPT,
appUid: 'app-m',
handler: 'h',
token: 't',
key: 'k',
event: {},
ctx: {},
});
expect(result).toEqual({ outcome: 'settled', status: 200 });
expect(created).toEqual([SCRIPT]);
// One miss, then the resend carrying the deployed marker.
expect(sent).toEqual([false, true]);
});
});
// The redis mock is shared across tests, so each one uses its own app and code.
let fresh = 0;
const nextApp = () => `app-u${++fresh}`;
const userScope = (holder: number, appUid: string) => ({ appUid, holder });
const sourceN = (label: string) =>
`async () => { await ${label}${++fresh}(); }`;
const scriptOf = (name: string, source: string) =>
userHandlerScript({ name, sourceHash: hashContent(source) }, '');
type Sets = Record<number, Array<{ name: string; source: string }>>;
/** Layers whose handler sets are per holder (0 is the app's own). */
const withSets = (layers: Layers, sets: Sets) => {
const handlersOf = (scope: { appUid: string; holder: number }) =>
(sets[scope.holder] ?? []).map((h) => ({
appUid: scope.appUid,
name: h.name,
source: h.source,
sourceHash: hashContent(h.source),
createdAt: 0,
updatedAt: 0,
}));
const store = layers.stores.eventHandler as unknown as Record<
string,
unknown
>;
store.setForApp = async (scope: { appUid: string; holder: number }) =>
handlersOf(scope).map(({ name, sourceHash }) => ({ name, sourceHash }));
store.allForApp = async (scope: { appUid: string; holder: number }) =>
handlersOf(scope);
};
const holders = (...ids: number[]) =>
Object.fromEntries(
ids.map((id) => [`holder-${id}`, { ownerUserId: id }]),
) as Record<string, Owner>;
describe('EventsWorkerDeployer in a user scope', () => {
it('deploys the one handler whose script was asked for, as the holder', async () => {
const app = nextApp();
const a = sourceN('a');
const b = sourceN('b');
const { layers, created, targets } = makeLayers(holders(10));
withSets(layers, {
10: [
{ name: 'a', source: a },
{ name: 'b', source: b },
],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('b', b);
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script]);
const target = targets[0]!;
expect(target.tags).toEqual([`events-app:${app}`, 'user:uuid-10']);
// One handler's script: the other one's code is not in it.
expect(target.source).toContain(b);
expect(target.source).not.toContain(a);
});
it('answers a script no handler of the scope matches as stale, before the throttle', async () => {
const app = nextApp();
const { layers, redis, created } = makeLayers(holders(10));
withSets(layers, { 10: [{ name: 'a', source: sourceN('a') }] });
const deployer = new EventsWorkerDeployer(layers);
expect(
await deployer.ensure(
userScope(10, app),
scriptOf('b', sourceN('b')),
),
).toBe('stale');
expect(created).toEqual([]);
const hour = new Date().toISOString().slice(0, 13);
expect(await redis.get(`ev:deploys:{${app}}:10:${hour}`)).toBeNull();
});
it('answers a scope with no handlers as no-handlers', async () => {
const { layers } = makeLayers(holders(10));
withSets(layers, {});
const deployer = new EventsWorkerDeployer(layers);
expect(
await deployer.ensure(
userScope(10, nextApp()),
scriptOf('a', sourceN('a')),
),
).toBe('no-handlers');
});
it('does not deploy for a holder that is suspended or gone', async () => {
const source = sourceN('a');
const { layers, created } = makeLayers({
'holder-10': { ownerUserId: 10, suspended: true },
});
withSets(layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
const app = nextApp();
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'no-owner',
);
expect(await deployer.ensure(userScope(11, app), script)).toBe(
'no-owner',
);
expect(created).toEqual([]);
});
it('gives each scope its own hourly cap, apart from the app scope', async () => {
const app = nextApp();
const source = sourceN('a');
const { layers, created } = makeLayers({
...holders(10, 11),
[app]: { ownerUserId: 12 },
});
withSets(layers, {
0: [{ name: 'a', source }],
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
for (let i = 0; i < EVENTS_USER_WORKER_DEPLOYS_PER_HOUR; i++)
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'deployed',
);
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'throttled',
);
expect(created).toHaveLength(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR);
// Another holder of the same app, and the app's own scope, still go.
expect(await deployer.ensure(userScope(11, app), script)).toBe(
'deployed',
);
expect(await deployer.ensure(appHandlerScope(app), script)).toBe(
'deployed',
);
});
it('collapses concurrent deploys of one script across holders into one upload', async () => {
const app = nextApp();
const source = sourceN('a');
const { layers, created } = makeLayers(holders(10, 11));
withSets(layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
await Promise.all([
deployer.ensure(userScope(10, app), script),
deployer.ensure(userScope(10, app), script),
deployer.ensure(userScope(11, app), script),
deployer.ensure(userScope(11, nextApp()), script),
]);
expect(created).toHaveLength(1);
});
it('names a script for identical code the same under any holder and app', async () => {
const source = sourceN('a');
const { layers, created } = makeLayers(holders(10, 11));
withSets(layers, {
10: [{ name: '@x', source }],
11: [{ name: '@x', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('@x', source);
expect(await deployer.ensure(userScope(10, nextApp()), script)).toBe(
'deployed',
);
expect(await deployer.ensure(userScope(11, nextApp()), script)).toBe(
'deployed',
);
expect(created).toEqual([script, script]);
});
});
describe('the deployed marker', () => {
const setup = () => {
const source = sourceN('a');
const made = makeLayers(holders(10, 11));
withSets(made.layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
return {
...made,
deployer: new EventsWorkerDeployer(made.layers),
script: scriptOf('a', source),
app: nextApp(),
};
};
it('is left by a user-scope deploy with a ttl under a day', async () => {
const { deployer, redis, script, app } = setup();
await deployer.ensure(userScope(10, app), script);
const ttl = await redis.ttl(`ev:deployed:${script}`);
expect(ttl).toBeGreaterThan(0);
expect(ttl).toBeLessThan(24 * 60 * 60);
});
it('lets an eager deploy skip the upload another holder already made', async () => {
const { deployer, created, script, app } = setup();
expect(await deployer.deployEagerly(userScope(10, app), script)).toBe(
'deployed',
);
expect(await deployer.deployEagerly(userScope(11, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script]);
});
it('spends no throttle on a skipped eager deploy', async () => {
const { deployer, redis, script, app } = setup();
await deployer.deployEagerly(userScope(10, app), script);
await deployer.deployEagerly(userScope(11, app), script);
const hour = new Date().toISOString().slice(0, 13);
expect(await redis.get(`ev:deploys:{${app}}:11:${hour}`)).toBeNull();
});
it('never skips a deploy-on-miss', async () => {
const { deployer, created, script, app } = setup();
await deployer.deployEagerly(userScope(10, app), script);
expect(await deployer.ensure(userScope(11, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script, script]);
});
it('is not left by an app-scope deploy', async () => {
const app = nextApp();
const source = sourceN('a');
const { layers, redis, created } = makeLayers({
[app]: { ownerUserId: 4 },
});
withSets(layers, { 0: [{ name: 'a', source }] });
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
await deployer.ensure(appHandlerScope(app), script);
expect(created).toEqual([script]);
expect(await redis.get(`ev:deployed:${script}`)).toBeNull();
});
it('still deploys eagerly when the marker cannot be read', async () => {
const { deployer, redis, created, script, app } = setup();
await deployer.deployEagerly(userScope(10, app), script);
const get = redis.get.bind(redis);
redis.get = (async (key: string) => {
if (key.startsWith('ev:deployed:')) throw new Error('down');
return get(key);
}) as never;
expect(await deployer.deployEagerly(userScope(11, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script, script]);
});
});
describe('scopeFor', () => {
it('reads no holder as the app scope, and resolves a uuid to the user scope', async () => {
const { layers } = makeLayers(holders(10));
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.scopeFor('app-u')).toEqual({
appUid: 'app-u',
holder: 0,
});
expect(await deployer.scopeFor('app-u', '')).toEqual({
appUid: 'app-u',
holder: 0,
});
expect(await deployer.scopeFor('app-u', 'uuid-10')).toEqual({
appUid: 'app-u',
holder: 10,
});
expect(await deployer.scopeFor('app-u', 'uuid-99')).toBeNull();
});
});
describe('LocalEventsInvokeTransport in a user scope', () => {
it('deploys the holder`s script on a miss and refuses an unknown holder', async () => {
const source = sourceN('a');
const { layers, created } = makeLayers(holders(10));
withSets(layers, { 10: [{ name: 'a', source }] });
const deployer = new EventsWorkerDeployer(layers);
let resident = false;
const transport = new LocalEventsInvokeTransport(
{
localworkerservice: {
dispatchEventsWorker: async () => {
if (!resident) return null;
return new Response(null, {
status: 200,
headers: { 'x-puter-events-handled': '1' },
});
},
},
} as never,
{
ensure: async (
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
) => {
const outcome = await deployer.ensure(
scope,
script,
options,
);
resident = outcome === 'deployed';
return outcome;
},
scopeFor: (appUid: string, holder?: string) =>
deployer.scopeFor(appUid, holder),
} as unknown as EventsWorkerDeployer,
);
const script = scriptOf('a', source);
const call: EventsInvokeCall = {
script,
appUid: nextApp(),
holder: 'uuid-10',
key: 'k',
body: '{}',
timeoutMs: 5000,
};
await expect(transport.send(call)).resolves.toMatchObject({
status: 200,
});
expect(created).toEqual([script]);
resident = false;
await expect(
transport.send({ ...call, holder: 'uuid-99' }),
).resolves.toMatchObject({ status: null, error: 'deploy: no-holder' });
expect(created).toEqual([script]);
});
});
describe('concurrent eager deploys of one script', () => {
const setup = () => {
const source = sourceN('a');
const made = makeLayers(holders(10, 11));
withSets(made.layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
return {
...made,
script: scriptOf('a', source),
app: nextApp(),
};
};
it('upload once when two holders publish the same new code together', async () => {
const { layers, created, script, app } = setup();
const deployer = new EventsWorkerDeployer(layers);
const outcomes = await Promise.all([
deployer.deployEagerly(userScope(10, app), script),
deployer.deployEagerly(userScope(11, app), script),
]);
expect(outcomes).toEqual(['deployed', 'deployed']);
expect(created).toEqual([script]);
});
it('skip the upload when the marker appears while they wait on the gate', async () => {
const { layers, redis, created, script, app } = setup();
// Another node's deploy of the script finishes before the gate opens.
const gated = new EventsWorkerDeployer({
...layers,
services: {
schedules: {
gateDeploy: async (
_script: string,
_options: { capped: boolean },
deploy: () => Promise<EventsDeployOutcome>,
) => {
await redis.set(`ev:deployed:${script}`, '1');
return deploy();
},
},
},
} as unknown as Layers);
expect(
await gated.ensure(userScope(10, app), script, { eager: true }),
).toBe('deployed');
expect(created).toEqual([]);
// A miss still uploads, whatever the marker says.
expect(await gated.ensure(userScope(11, app), script)).toBe('deployed');
expect(created).toEqual([script]);
});
it('make a miss that joins an eager deploy deploy after it', async () => {
const { layers, redis, created, script, app } = setup();
const deployer = new EventsWorkerDeployer(layers);
await redis.set(`ev:deployed:${script}`, '1');
const eager = deployer.ensure(userScope(10, app), script, {
eager: true,
});
const miss = deployer.ensure(userScope(11, app), script);
expect(await eager).toBe('deployed');
expect(await miss).toBe('deployed');
// The eager one skipped on the marker; the miss uploaded.
expect(created).toEqual([script]);
});
});
+202 -55
View File
@@ -42,23 +42,33 @@ import {
eventsInvokeKey,
eventsWorkerScope,
eventsWorkerScript,
userHandlerScript,
} from '../../services/events/workerRuntime.js';
import {
generateEventsWorkerSource,
handlerSetHash,
} from '../../services/events/workerSource.js';
import {
appHandlerScope,
type HandlerScope,
} from '../../stores/events/EventHandlerStore.js';
import type { puterStores } from '../../stores/index.js';
import type { IConfig, LayerInstances } from '../../types.js';
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
import {
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_DEPLOYS_PER_HOUR,
} from './limits.js';
/**
* Deploys an app's events worker from its published handlers.
*
* Deployed the first time a delivery needs it and again on eviction — a publish
* only writes rows. Scripts carry no worker token and no `subdomains` row: not
* one of the owner's workers, not addressable from the internet, not listed or
* deleted as if it were. A republished set leaves the old script behind rather
* than deleting it; redeploying a still-live one is always safe.
* only writes rows (a user-scope publish also deploys eagerly). An app's scope
* is one script for the whole set; a user scope is one script per handler,
* deployed as the holder. Scripts carry no worker token and no `subdomains`
* row: not one of the owner's workers, not addressable from the internet, not
* listed or deleted as if it were. A republished set leaves the old script
* behind rather than deleting it; redeploying a still-live one is always safe.
*/
/** Why a deploy could not happen, for the callers that answer differently. */
@@ -74,6 +84,11 @@ export type EventsDeployOutcome =
/** Whose call asked for the deploy: a schedule run's is held to a share. */
export interface EventsDeployOptions {
lane?: InvokeLane;
/**
* A user-scope publish's deploy, which skips the upload when the script's
* deployed marker is already set. A deploy-on-miss is never eager.
*/
eager?: boolean;
}
interface DeployLayers {
@@ -86,14 +101,22 @@ interface DeployLayers {
}
/**
* `ev:deploys:{<appUid>}:<hour>` bucket TTL — comfortably past the hour it
* counts.
* `ev:deploys:{<appUid>}[:<holder>]:<hour>` bucket TTL — comfortably past the
* hour it counts.
*/
const DEPLOY_THROTTLE_TTL_SECONDS = 2 * 60 * 60;
/** How long `ev:deployed:<script>` says a script was uploaded. Under a day. */
const DEPLOYED_MARKER_TTL_SECONDS = 20 * 60 * 60;
const deployedMarkerKey = (script: string): string => `ev:deployed:${script}`;
const isUserScope = (scope: HandlerScope): boolean => scope.holder !== 0;
interface InFlightDeploy {
/** A schedule run's, which the region's deploy slots may refuse. */
capped: boolean;
eager: boolean;
promise: Promise<EventsDeployOutcome>;
}
@@ -120,38 +143,76 @@ export class EventsWorkerDeployer {
}
/**
* Deploy `script`, which must be what the app's handlers currently hash to.
* A request for anything else is `stale` — the set moved on, and the next
* delivery resolves the new script rather than this one being revived.
* Deploy `script`, which must be what the scope's handlers currently hash
* to: the whole set for an app, one handler for a user scope. A request for
* anything else is `stale` — the handlers moved on, and the next delivery
* resolves the new script rather than this one being revived. Always
* uploads; a deploy-on-miss means whatever said it was there is wrong.
*/
ensure(
appUid: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions = {},
): Promise<EventsDeployOutcome> {
const key = `${appUid}|${script}`;
// A user scope's script is the same bytes whoever holds it, so
// concurrent deploys of it from any holder or app are one.
const key = isUserScope(scope)
? `user|${script}`
: `${scope.appUid}|${scope.holder}|${script}`;
const capped = options.lane === 'schedule';
const existing = this.#inFlight.get(key);
if (!existing) return this.#start(key, appUid, script, options);
if (!existing) return this.#start(key, scope, script, options);
// An eager deploy may have skipped the upload, which a miss must not
// take for done.
if (existing.eager && !options.eager)
return this.#afterEager(
existing.promise,
key,
scope,
script,
options,
);
if (capped || !existing.capped) return existing.promise;
return this.#afterCapped(
existing.promise,
key,
appUid,
script,
options,
);
return this.#afterCapped(existing.promise, key, scope, script, options);
}
/**
* {@link ensure} for the deploy a user-scope publish starts, which skips the
* upload when another holder's deploy of the same script left its marker.
* Only this path reads the marker.
*/
async deployEagerly(
scope: HandlerScope,
script: string,
): Promise<EventsDeployOutcome> {
if (await this.#markedDeployed(script)) return 'deployed';
return this.ensure(scope, script, { eager: true });
}
/**
* The scope a script request names: the app's own, or the user scope of
* `holder` (a user uuid). Null for a holder that does not exist.
*/
async scopeFor(
appUid: string,
holder?: string | null,
): Promise<HandlerScope | null> {
if (!holder) return appHandlerScope(appUid);
const user = await this.#layers.stores.user.getByUuid(holder);
if (!user?.id) return null;
return { appUid, holder: Number(user.id) };
}
#start(
key: string,
appUid: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
const entry: InFlightDeploy = {
capped: options.lane === 'schedule',
promise: this.#gated(appUid, script, options).finally(() => {
eager: options.eager === true,
promise: this.#gated(scope, script, options).finally(() => {
if (this.#inFlight.get(key) === entry)
this.#inFlight.delete(key);
}),
@@ -160,6 +221,20 @@ export class EventsWorkerDeployer {
return entry.promise;
}
/** A miss that found an eager deploy in flight deploys after it. */
async #afterEager(
eager: Promise<EventsDeployOutcome>,
key: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
await eager;
const current = this.#inFlight.get(key);
if (current && !current.eager) return current.promise;
return this.#start(key, scope, script, options);
}
/**
* A delivery that joined a schedule run's deploy: `busy` means no deploy
* started, and a delivery is never refused a slot, so it asks again.
@@ -167,7 +242,7 @@ export class EventsWorkerDeployer {
async #afterCapped(
capped: Promise<EventsDeployOutcome>,
key: string,
appUid: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
@@ -175,7 +250,7 @@ export class EventsWorkerDeployer {
if (outcome !== 'busy') return outcome;
const current = this.#inFlight.get(key);
if (current && !current.capped) return current.promise;
return this.#start(key, appUid, script, options);
return this.#start(key, scope, script, options);
}
/**
@@ -185,11 +260,12 @@ export class EventsWorkerDeployer {
* may be refused with `busy`. A delivery's deploy is never refused.
*/
#gated(
appUid: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
const deploy = () => this.#deploy(appUid, script);
const deploy = () =>
this.#deploy(scope, script, options.eager === true);
return (
this.#layers.services.schedules?.gateDeploy?.(
script,
@@ -200,28 +276,55 @@ export class EventsWorkerDeployer {
}
async #deploy(
appUid: string,
handlerScope: HandlerScope,
script: string,
eager: boolean,
): Promise<EventsDeployOutcome> {
const { appUid } = handlerScope;
const { stores, drivers } = this.#layers;
const scope = eventsWorkerScope(this.#layers.config);
const backend = eventsWorkerScope(this.#layers.config);
const userScope = isUserScope(handlerScope);
// Names and hashes are enough to tell a stale request, so the set's
// sources are read and built only once the throttle has let it through.
const set = await stores.eventHandler.setForApp(appUid);
const set = await stores.eventHandler.setForApp(handlerScope);
if (set.length === 0) return 'no-handlers';
// Two apps that published byte-identical sets share one script, which
// is sound: the source is the same, and everything per-app arrives with
// the invocation. Whoever deploys first owns it.
if (eventsWorkerScript(handlerSetHash(set), scope) !== script)
// the invocation. Whoever deploys first owns it. A user scope's script
// is one handler's, so it is shared the same way across holders.
const wanted = userScope
? set.find((entry) => userHandlerScript(entry, backend) === script)
: undefined;
if (
userScope
? !wanted
: eventsWorkerScript(handlerSetHash(set), backend) !== script
)
return 'stale';
if (await this.#tooManyDeploysThisHour(appUid)) return 'throttled';
// Asked again inside the gate, where another deploy of this script may
// have just finished.
if (eager && userScope && (await this.#markedDeployed(script)))
return 'deployed';
const handlers = await stores.eventHandler.allForApp(appUid);
if (await this.#tooManyDeploysThisHour(handlerScope))
return 'throttled';
const all = await stores.eventHandler.allForApp(handlerScope);
const handlers = wanted
? all.filter(
(handler) =>
handler.name === wanted.name &&
handler.sourceHash === wanted.sourceHash,
)
: all;
const generated = generateEventsWorkerSource(handlers);
// The set can move between the two reads.
if (eventsWorkerScript(generated.setHash, scope) !== script)
if (
handlers.length === 0 ||
eventsWorkerScript(generated.setHash, backend) !== script
)
return 'stale';
if (generated.tooLarge) {
console.error(
@@ -235,7 +338,7 @@ export class EventsWorkerDeployer {
generated.broken.join(', '),
);
const owner = await this.#owner(appUid);
const owner = await this.#owner(handlerScope);
if (!owner) return 'no-owner';
const secret = this.#layers.config.events?.internalSecret;
@@ -264,6 +367,7 @@ export class EventsWorkerDeployer {
runtime: 'events',
source: generated.source,
omitOwnerToken: true,
cpuMs: this.#layers.config.events?.workerCpuMs,
// What the script compares an invocation against.
// Derived from the script name, so a redeploy of
// the same set binds the same key.
@@ -294,34 +398,74 @@ export class EventsWorkerDeployer {
);
return 'failed';
}
if (userScope) await this.#markDeployed(script);
return 'deployed';
}
/**
* Whether this app has already deployed `EVENTS_WORKER_DEPLOYS_PER_HOUR`
* times this hour. A flapping set (publish, remove, republish) provisions
* upstream on every miss, and this is the ceiling on that loop.
*/
async #tooManyDeploysThisHour(appUid: string): Promise<boolean> {
const redis = this.#layers.clients.redis;
const hour = new Date().toISOString().slice(0, 13);
const key = `ev:deploys:{${appUid}}:${hour}`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
return count > EVENTS_WORKER_DEPLOYS_PER_HOUR;
async #markedDeployed(script: string): Promise<boolean> {
try {
return !!(await this.#layers.clients.redis.get(
deployedMarkerKey(script),
));
} catch (err) {
console.warn('[events] deployed marker unreadable', err);
return false;
}
}
/** The app owner, as an actor the deploy can run as. */
async #markDeployed(script: string): Promise<void> {
try {
await this.#layers.clients.redis.set(
deployedMarkerKey(script),
'1',
'EX',
DEPLOYED_MARKER_TTL_SECONDS,
);
} catch (err) {
console.warn('[events] could not mark a script deployed', err);
}
}
/**
* Whether this scope has already deployed its hourly cap this hour. A
* flapping set (publish, remove, republish) provisions upstream on every
* miss, and this is the ceiling on that loop. An app's counter is per app;
* a user scope has its own, so one user cannot spend another's.
*/
async #tooManyDeploysThisHour(scope: HandlerScope): Promise<boolean> {
const redis = this.#layers.clients.redis;
const hour = new Date().toISOString().slice(0, 13);
const userScope = isUserScope(scope);
const key = userScope
? `ev:deploys:{${scope.appUid}}:${scope.holder}:${hour}`
: `ev:deploys:{${scope.appUid}}:${hour}`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
return (
count >
(userScope
? EVENTS_USER_WORKER_DEPLOYS_PER_HOUR
: EVENTS_WORKER_DEPLOYS_PER_HOUR)
);
}
/**
* The user a scope's deploy runs as, as an actor: the app's owner, or the
* holder of a user scope.
*/
async #owner(
appUid: string,
scope: HandlerScope,
): Promise<
| (Actor & { user: { id: number; uuid: string; username: string } })
| null
> {
const { stores } = this.#layers;
const app = await stores.app.getByUid(appUid);
const ownerUserId = Number(app?.owner_user_id);
if (!app || !Number.isFinite(ownerUserId)) return null;
let ownerUserId = scope.holder;
if (!isUserScope(scope)) {
const app = await stores.app.getByUid(scope.appUid);
ownerUserId = Number(app?.owner_user_id);
if (!app || !Number.isFinite(ownerUserId)) return null;
}
const user = await stores.user.getById(ownerUserId);
if (!user?.id || !user.uuid || !user.username || user.suspended)
@@ -402,11 +546,14 @@ export class LocalEventsInvokeTransport {
const resident = await this.#dispatch(request(), call.timeoutMs);
if (resident !== null) return resident;
const outcome = await this.#deployer.ensure(
const scope = await this.#deployer.scopeFor(
call.appUid,
call.script,
{ lane: call.lane },
call.holder,
);
if (!scope) return { status: null, error: 'deploy: no-holder' };
const outcome = await this.#deployer.ensure(scope, call.script, {
lane: call.lane,
});
if (outcome !== 'deployed')
return {
status: null,
@@ -78,11 +78,13 @@ interface CapturedResponse {
statusCode: number;
body: unknown;
contentType?: string;
location?: string;
}
const makeReq = (init: {
params?: Record<string, unknown>;
path?: string;
originalUrl?: string;
actor?: Actor;
hostname?: string;
protocol?: string;
@@ -93,6 +95,7 @@ const makeReq = (init: {
headers: {},
params: init.params ?? {},
path: init.path ?? '/',
originalUrl: init.originalUrl ?? init.path ?? '/',
hostname: init.hostname ?? 'test.local',
protocol: init.protocol ?? 'http',
actor: init.actor,
@@ -130,6 +133,11 @@ const makeRes = () => {
captured.contentType = value;
return res;
}),
redirect: vi.fn((code: number, location: string) => {
captured.statusCode = code;
captured.location = location;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
@@ -213,14 +221,12 @@ describe('HomepageController shell routes', () => {
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
it('serves the shell on /settings, /dashboard, /desktop, /action, /@:username', async () => {
it('serves the shell on /settings, /dashboard, /action, /@:username', async () => {
for (const path of [
'/settings',
'/settings/*splat',
'/dashboard',
'/dashboard/',
'/desktop',
'/desktop/',
'/action/*splat',
'/@:username',
]) {
@@ -230,6 +236,18 @@ describe('HomepageController shell routes', () => {
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
}
});
it('serves the shell on a path inside a public folder, /@:username/*splat', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/@:username/*splat',
makeReq({ path: '/@alice/photos/cat.png' }),
res,
);
expect(captured.statusCode).toBe(200);
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
});
// ── /app/:name ──────────────────────────────────────────────────────
@@ -359,49 +377,6 @@ describe('HomepageController GET /app/:name', () => {
expect(html).not.toContain('owner_user_id');
});
it('serves the same app shell under /desktop/app/:name', async () => {
const { userId } = await makeUser();
const name = `desk-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.app.create(
{
name,
title: 'Desktop App',
description: 'opens on the desktop instead of the dashboard',
index_url: `https://example.com/${name}/`,
approved_for_listing: 1,
},
{ ownerUserId: userId },
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/desktop/app/:name',
makeReq({ params: { name }, path: `/desktop/app/${name}` }),
res,
);
expect(captured.statusCode).toBe(200);
const html = String(captured.body);
expect(html).toMatch(/<!DOCTYPE html>/i);
expect(html).toContain('Desktop App');
});
it('returns 404 under /desktop/app/:name when the app is unknown', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/desktop/app/:name',
makeReq({
params: { name: 'no-such-app' },
path: '/desktop/app/no-such-app',
}),
res,
);
expect(captured.statusCode).toBe(404);
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
it('omits index_url even for a public app — the shell is not the launch authority', async () => {
const { userId } = await makeUser();
const name = `pub-${Math.random().toString(36).slice(2, 10)}`;
@@ -433,8 +408,8 @@ describe('HomepageController GET /app/:name', () => {
// -- app landing rate limit --
describe('HomepageController app landing rate limit', () => {
it('rate-limits /app/:name and /desktop/app/:name per network', () => {
for (const path of ['/app/:name', '/desktop/app/:name']) {
it('rate-limits /app/:name per network', () => {
for (const path of ['/app/:name']) {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === path,
);
@@ -452,7 +427,7 @@ describe('HomepageController app landing rate limit', () => {
// ── /show/* ─────────────────────────────────────────────────────────
describe('HomepageController GET /show/*splat', () => {
it('emits a launch_app explorer call with the post-/show path', async () => {
it('serves the plain shell; the GUI opens the path itself', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
@@ -460,13 +435,52 @@ describe('HomepageController GET /show/*splat', () => {
makeReq({ path: '/show/alice/Documents' }),
res,
);
// The launch payload is JSON-serialized into the rendered HTML.
// We assert the rendered shell included the explorer launch hint
// pointing at the expected (slashed) path.
const html = String(captured.body);
expect(html).toContain('launch_app');
expect(html).toContain('explorer');
expect(html).toContain('/alice/Documents');
expect(html).toMatch(/<!DOCTYPE html>/i);
// No boot-time launch hook: the one it used to carry named a GUI
// global that no longer exists, which stopped the GUI booting.
expect(html).not.toContain('launch_app');
});
});
// ── /desktop ────────────────────────────────────────────────────────
describe('HomepageController GET /desktop', () => {
it('redirects /desktop to the root, keeping the query', async () => {
for (const path of ['/desktop', '/desktop/']) {
const { res, captured } = makeRes();
await callRoute('get', path, makeReq({ path }), res);
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe('/');
}
const { res, captured } = makeRes();
await callRoute(
'get',
'/desktop',
makeReq({
path: '/desktop',
originalUrl: '/desktop?shared=%2Falice%2Fx%2Fy&download=a',
}),
res,
);
expect(captured.location).toBe('/?shared=%2Falice%2Fx%2Fy&download=a');
});
it('redirects /desktop/app/:name to the app landing', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/desktop/app/:name',
makeReq({
params: { name: 'editor' },
path: '/desktop/app/editor',
originalUrl: '/desktop/app/editor?file=%2Falice%2Fa.txt',
}),
res,
);
expect(captured.statusCode).toBe(302);
expect(captured.location).toBe('/app/editor?file=%2Falice%2Fa.txt');
});
});
@@ -25,7 +25,6 @@ import type { PuterRouter } from '../../core/http/PuterRouter';
import type {
PuterHomepageService,
PageMeta,
LaunchOptions,
} from '../../services/homepage/PuterHomepageService';
/** Longest description/og:description/twitter:description on `/app/<name>`. */
@@ -43,9 +42,9 @@ const APP_LANDING_LIMIT = {
};
/**
* App descriptions can run long; search snippets and social cards want a
* short blurb. Collapses whitespace, then cuts at a word boundary with an
* ellipsis so the result never exceeds `APP_META_DESCRIPTION_MAX`.
* App descriptions can run long; search snippets and social cards want a short
* blurb. Collapses whitespace, then cuts at a word boundary with an ellipsis so
* the result never exceeds `APP_META_DESCRIPTION_MAX`.
*/
export function appMetaDescription(text: string): string {
const clean = text.replace(/\s+/g, ' ').trim();
@@ -89,7 +88,6 @@ export class HomepageController extends PuterController {
req: express.Request,
res: express.Response,
metaOverrides: Partial<PageMeta> = {},
launch: LaunchOptions = {},
) => {
const meta = { ...defaultMeta(req), ...metaOverrides };
const actor =
@@ -98,7 +96,7 @@ export class HomepageController extends PuterController {
actor?: Parameters<typeof homepage.send>[0]['actor'];
}
).actor ?? null;
await homepage.send({ req, res, actor }, meta, launch);
await homepage.send({ req, res, actor }, meta);
};
// -- Root + path-aliased shell routes ------------------------
@@ -111,17 +109,41 @@ export class HomepageController extends PuterController {
router.get('/dashboard', {}, (req, res) => sendShell(req, res));
router.get('/dashboard/', {}, (req, res) => sendShell(req, res));
router.get('/desktop', {}, (req, res) => sendShell(req, res));
router.get('/desktop/', {}, (req, res) => sendShell(req, res));
// `/desktop` used to pick the desktop interface. What follows it is
// the same route at the root, so it redirects there, keeping the query
// (`?shared=`, `?download=`); the browser carries the hash over.
const redirectWithoutDesktop = (
req: express.Request,
res: express.Response,
target: string,
) => {
const url = req.originalUrl ?? req.url ?? '';
const query = url.includes('?') ? url.slice(url.indexOf('?')) : '';
res.redirect(302, `${target}${query}`);
};
router.get('/desktop', {}, (req, res) =>
redirectWithoutDesktop(req, res, '/'),
);
router.get('/desktop/', {}, (req, res) =>
redirectWithoutDesktop(req, res, '/'),
);
router.get('/desktop/app/:name', {}, (req, res) =>
redirectWithoutDesktop(
req,
res,
`/app/${encodeURIComponent(String(req.params.name))}`,
),
);
router.get('/action/*splat', {}, (req, res) => sendShell(req, res));
// `/@<user>/<path>` names something in that user's Public folder; the
// GUI opens it.
router.get('/@:username', {}, (req, res) => sendShell(req, res));
router.get('/@:username/*splat', {}, (req, res) => sendShell(req, res));
// -- /app/:name - app metadata baked into the shell ----------
// `/desktop/app/:name` is the same landing booted on the desktop
// instead of the dashboard; the GUI strips the prefix and treats the
// rest of the path identically, so both render the same shell.
const sendAppShell = async (
req: express.Request,
@@ -176,27 +198,10 @@ export class HomepageController extends PuterController {
{ rateLimit: APP_LANDING_LIMIT },
sendAppShell,
);
router.get(
'/desktop/app/:name',
{ rateLimit: APP_LANDING_LIMIT },
sendAppShell,
);
// -- /show/* - launch explorer with the requested file path --
// -- /show/* - the GUI opens the path that follows ----------
router.get('/show/*splat', {}, (req, res) => {
const filePath = req.path.slice('/show'.length);
const launch: LaunchOptions = {
on_initialized: [
{
$: 'window-call',
fn_name: 'launch_app',
args: [{ name: 'explorer', path: filePath }],
},
],
};
return sendShell(req, res, {}, launch);
});
router.get('/show/*splat', {}, (req, res) => sendShell(req, res));
// -- /sw.js - the service worker, scoped to the whole origin --
@@ -19,8 +19,9 @@
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import type { AddressInfo } from 'node:net';
import path from 'node:path';
import type { Request, Response } from 'express';
import express, { type Request, type Response } from 'express';
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import type { RouteDescriptor } from '../../core/http/types.js';
@@ -111,6 +112,7 @@ const callGetHandler = async (
let captured: { filename?: string; root?: string } = {};
const req = {} as Request;
const res = {
setHeader: vi.fn(),
sendFile: vi.fn((filename: string, opts: { root: string }) => {
captured = { filename, root: opts.root };
}),
@@ -238,6 +240,48 @@ describe('StaticAssetsController puterjs_root', () => {
});
});
// ── push-sw.js caching ──────────────────────────────────────────────
/** Serves the route's handler over real HTTP and returns the response. */
const fetchThroughExpress = async (router: PuterRouter, routePath: string) => {
const route = findRoute(router, 'get', routePath);
if (!route) throw new Error(`No GET ${routePath} registered`);
const app = express();
app.get(routePath, (req, res, next) => route.handler(req, res, next));
const listener = app.listen(0);
await new Promise((resolve) => listener.once('listening', resolve));
try {
const { port } = listener.address() as AddressInfo;
return await fetch(`http://127.0.0.1:${port}${routePath}`);
} finally {
listener.close();
}
};
describe('StaticAssetsController push-sw.js', () => {
it.each([
['client_libs_root', 'puter.js/push-sw.js'],
['puterjs_root', 'push-sw.js'],
] as const)(
'lets the CDN keep it for five minutes from %s',
async (configKey, file) => {
const root = makeTempDir();
mkdirSync(path.dirname(path.join(root, file)), { recursive: true });
writeFileSync(path.join(root, file), '/* worker */');
const router = buildController({ [configKey]: root });
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
const response = await fetchThroughExpress(router, route);
expect(response.status).toBe(200);
expect(response.headers.get('cache-control')).toBe(
'public, max-age=0, s-maxage=300',
);
expect(await response.text()).toBe('/* worker */');
}
},
);
});
// ── gui_assets_root ─────────────────────────────────────────────────
describe('StaticAssetsController gui_assets_root', () => {
@@ -23,6 +23,9 @@ import path from 'node:path';
import { PuterController } from '../types.js';
import type { PuterRouter } from '../../core/http/PuterRouter';
// Sites import push-sw.js by URL, so the CDN refetches it within five minutes.
const PUSH_WORKER_CACHE_CONTROL = 'public, max-age=0, s-maxage=300';
/**
* Static asset routes.
*
@@ -58,6 +61,7 @@ export class StaticAssetsController extends PuterController {
// `/v2/push-sw.js` is the path older docs gave.
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
router.get(route, { subdomain: 'js' }, (_req, res) => {
res.setHeader('Cache-Control', PUSH_WORKER_CACHE_CONTROL);
res.sendFile('puter.js/push-sw.js', { root });
});
}
@@ -108,6 +112,10 @@ export class StaticAssetsController extends PuterController {
});
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
router.get(route, { subdomain: 'js' }, (_req, res) => {
res.setHeader(
'Cache-Control',
PUSH_WORKER_CACHE_CONTROL,
);
res.sendFile('push-sw.js', { root: puterjsRoot });
});
}
+2
View File
@@ -69,6 +69,8 @@ export interface ScheduleView {
appUid: string;
/** `null` once ended: the handler binding goes with the schedule. */
handlerName: string | null;
/** Set only when the handler lives in the holder's own scope. */
handlerScope?: 'user';
contextKeys: string[] | null;
contextHash: string | null;
/** Interval of an `every` schedule. */
+2 -18
View File
@@ -31,7 +31,7 @@ import {
normalizeRawBase64ImageString,
validateIconDataUrl,
} from '../../util/appIcon.js';
import { toAppReadView } from '../../util/appView.js';
import { isOriginBootstrapApp, toAppReadView } from '../../util/appView.js';
import { awaitLater } from '../../util/concurrency.js';
import { isUniqueViolation } from '../../util/dbError.js';
import {
@@ -1088,22 +1088,6 @@ export class AppDriver extends PuterDriver {
}
}
/**
* Origin-bootstrap detection: rows auto-created when an unknown origin
* first needed an app row (no human-supplied metadata). Marker is `name ===
* uid && title === uid` and a description starting with "App created from
* origin ". Only these rows are eligible for same-owner merging — refusing
* to merge arbitrary same-owner apps prevents accidental data loss.
*/
#isOriginBootstrapApp(app) {
if (!app || typeof app !== 'object') return false;
if (typeof app.uid !== 'string' || !app.uid) return false;
if (app.name !== app.uid) return false;
if (app.title !== app.uid) return false;
if (typeof app.description !== 'string') return false;
return app.description.startsWith('App created from origin ');
}
// -- Canonical-uid alias kvstore pair -----------------------------
//
// After a merge, the source app uid → canonical uid mapping is
@@ -1266,7 +1250,7 @@ export class AppDriver extends PuterDriver {
if (
Number.isInteger(conflictOwnerUserId) &&
conflictOwnerUserId === user.id &&
!this.#isOriginBootstrapApp(appToJoin)
!isOriginBootstrapApp(appToJoin)
) {
// Prevent merging arbitrary same-owner apps; only allow the
// auto-created origin bootstrap row to be absorbed.
@@ -129,6 +129,27 @@ describe('pipeStreamResult', () => {
expect(warn).not.toHaveBeenCalled();
});
it('adds at most two close listeners to the response', async () => {
// The response already carries close listeners from metering,
// compression and tracing; Node warns past ten.
let added = -1;
server = createServer((_req, res) => {
const before = res.listenerCount('close');
pipeStreamResult(Readable.from(['a', 'b']), res, 'test.run');
added = res.listenerCount('close') - before;
});
await new Promise<void>((resolve) => server!.listen(0, resolve));
const { port } = server.address() as AddressInfo;
await new Promise<void>((resolve) =>
get(`http://127.0.0.1:${port}/`, (res) => {
res.resume();
res.on('end', () => resolve());
}),
);
expect(added).toBeGreaterThan(0);
expect(added).toBeLessThanOrEqual(2);
});
it('drops the source, without throwing, when the client left before the result was ready', async () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
const source = Readable.from(['late']);
+16 -8
View File
@@ -17,7 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { pipeline, type Readable } from 'node:stream';
import type { Readable } from 'node:stream';
// Only what's used here, so callers on another @types/node (or express's
// Response) still fit.
@@ -32,24 +32,32 @@ type ResponseSink = NodeJS.WritableStream & {
* mid-body destroys the response, so the client sees an aborted transfer rather
* than a clean end, and the failure is logged once instead of surfacing as an
* unhandled stream error. A client that hangs up stops the source.
*
* `pipe` rather than `pipeline`: pipeline puts seven close listeners on the
* response, and with the ones metering, compression and tracing add, every
* streamed response tripped Node's ten-listener warning.
*/
export function pipeStreamResult(
stream: Readable,
res: ResponseSink,
label: string,
): void {
// The caller hung up while the driver ran; `pipeline` throws on a closed
// destination, so drop the source instead.
// The caller hung up while the driver ran: nothing will read the source.
if (res.closed || res.destroyed) {
stream.destroy();
return;
}
pipeline(stream, res, (err) => {
if (!err) return;
stream.once('error', (err) => {
stream.unpipe(res);
if (!res.destroyed) res.destroy();
// The client hanging up is not a failure of ours.
if ((err as { code?: string }).code === 'ERR_STREAM_PREMATURE_CLOSE')
return;
console.warn(`[drivers] ${label} stream failed mid-response:`, err);
});
// A response that closes before the source ended is a client hanging up.
res.once('close', () => {
if (stream.readableEnded) return;
stream.unpipe(res);
stream.destroy();
});
res.once('error', () => stream.destroy());
stream.pipe(res);
}
@@ -754,6 +754,50 @@ describe('WorkerDriver.create with a configured deploy backend', () => {
expect(putCalls()).toHaveLength(1);
});
const uploadedMetadata = () => {
const [, init] = putCalls().at(-1)!;
const form = (init as RequestInit).body as FormData;
return JSON.parse(form.get('metadata') as string);
};
it('sends limits.cpu_ms in the upload metadata when cpuMs is set', async () => {
const { user, actor } = await makeUser();
const path = `/${user.username}/cpu.worker.js`;
await writeSource(actor, user.id, path, 'export default {}');
const scriptName = `dt-cpu-${user.username}`;
await inCtx(actor, () =>
target.create({
appId: '',
workerName: scriptName,
filePath: path,
[INTERNAL_DEPLOY_TARGET]: { scriptName, cpuMs: 50 },
}),
);
expect(uploadedMetadata().limits).toEqual({ cpu_ms: 50 });
});
it('sends no limits key without cpuMs, and none on an ordinary create', async () => {
const { user, actor } = await makeUser();
const path = `/${user.username}/nocpu.worker.js`;
await writeSource(actor, user.id, path, 'export default {}');
const scriptName = `dt-nocpu-${user.username}`;
await deployTo(actor, path, scriptName);
expect(uploadedMetadata()).not.toHaveProperty('limits');
const name = `plain-${user.username}`;
await inCtx(actor, () =>
target.create({ appId: '', workerName: name, filePath: path }),
);
const metadata = uploadedMetadata();
expect(metadata).not.toHaveProperty('limits');
expect(Object.keys(metadata)).toEqual([
'body_part',
'compatibility_flags',
'compatibility_date',
'bindings',
]);
});
it('refuses a file another user owns, even with manage on it', async () => {
const owner = await makeUser();
const deployer = await makeUser();
@@ -3,6 +3,7 @@
// however the interactions with the Puter API are
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
@@ -13,6 +14,7 @@ import {
import { Actor, makeActor as resolveActor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { DEFAULT_FREE_SUBSCRIPTION } from '../../services/metering/consts.js';
import {
setupPuterTestEnv,
setupTestServer,
@@ -797,6 +799,174 @@ describe('WorkerDriver', () => {
});
});
describe('rented-unit cap', () => {
const FREE_CAP = 16;
const PAID_CAP = 100;
// The cap sits after the deploy-credentials gate, so the driver needs
// credentials to reach it; a create under the cap then fails on the
// missing source file, which is what tells it past the cap check.
let originalWorkersConfig: unknown;
let originalEvents: unknown;
beforeAll(() => {
const driverConfig = (
target as unknown as { config: Record<string, unknown> }
).config;
originalEvents = driverConfig.events;
driverConfig.events = { userHandlers: true };
originalWorkersConfig = driverConfig.workers;
driverConfig.workers = {
XAUTHKEY: 'test-key',
ACCOUNTID: 'test-account',
};
});
afterAll(() => {
const driverConfig = (
target as unknown as { config: Record<string, unknown> }
).config;
driverConfig.workers = originalWorkersConfig;
driverConfig.events = originalEvents;
});
const holding = (
plan: string,
held: { workers?: number; scopes?: number; owned?: number },
) => {
vi.spyOn(
server.services.metering,
'getActorSubscription',
).mockResolvedValue({ id: plan } as never);
vi.spyOn(
server.stores.subdomain,
'countByUserIdAndPrefix',
).mockResolvedValue(held.workers ?? 0);
vi.spyOn(
server.stores.eventHandler,
'countUserScopesWithHandlersForUser',
).mockResolvedValue(held.scopes ?? 0);
vi.spyOn(
server.stores.eventHandler,
'countAppsWithHandlersForOwner',
).mockResolvedValue(held.owned ?? 0);
};
afterEach(() => {
vi.restoreAllMocks();
});
const createWorker = () =>
inCtx(() =>
target.create({
appId: 'test-app',
workerName: 'capped',
filePath: '/test.js',
}),
);
it('keeps the flat 100-worker cap over ordinary workers with user scopes off', async () => {
const driverConfig = (
target as unknown as { config: Record<string, unknown> }
).config;
driverConfig.events = { userHandlers: false };
try {
// 16 workers is fine here, and scopes and owned apps don't count.
holding(DEFAULT_FREE_SUBSCRIPTION, {
workers: 16,
scopes: 50,
owned: 50,
});
const list = vi
.spyOn(server.stores.subdomain, 'listByUserIdAndPrefix')
.mockResolvedValue(
Array.from({ length: 16 }, () => ({})) as never,
);
await expect(createWorker()).rejects.not.toMatchObject({
statusCode: 403,
});
list.mockResolvedValue(
Array.from({ length: 100 }, () => ({})) as never,
);
await expect(createWorker()).rejects.toMatchObject({
statusCode: 403,
message: 'Worker limit reached (max 100)',
legacyCode: 'forbidden',
});
} finally {
driverConfig.events = { userHandlers: true };
}
});
it('refuses a free user one unit past the cap, across all three sources', async () => {
for (const held of [
{ workers: FREE_CAP },
{ scopes: FREE_CAP },
{ owned: FREE_CAP },
{ workers: 10, scopes: 4, owned: 2 },
]) {
holding(DEFAULT_FREE_SUBSCRIPTION, held);
await expect(createWorker()).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
}
});
it('lets a free user create up to the cap', async () => {
holding(DEFAULT_FREE_SUBSCRIPTION, {
workers: 10,
scopes: 3,
owned: 2,
});
await expect(createWorker()).rejects.not.toMatchObject({
statusCode: 403,
});
});
it('refuses a paid user the unit after 100', async () => {
holding('paid-plan', { workers: 90, scopes: 5, owned: 5 });
await expect(createWorker()).rejects.toMatchObject({
statusCode: 403,
});
holding('paid-plan', { workers: 90, scopes: 5, owned: 4 });
await expect(createWorker()).rejects.not.toMatchObject({
statusCode: 403,
});
expect(PAID_CAP).toBe(100);
});
it('keeps a free user over the cap, redeploys what they hold, and refuses a new one', async () => {
holding(DEFAULT_FREE_SUBSCRIPTION, { workers: 20 });
const user = actor.user;
const exists = vi
.spyOn(server.stores.subdomain, 'getBySubdomain')
.mockImplementation(
async (name: string) =>
(name === 'workers.puter.capped'
? { user_id: user.id, uuid: 'existing-uuid' }
: null) as never,
);
const noApp = makeActor({ app: undefined });
const create = (workerName: string) =>
inCtx(
() =>
target.create({
appId: '',
workerName,
filePath: '/test.js',
}),
noApp,
);
await expect(create('capped')).rejects.not.toMatchObject({
statusCode: 403,
});
await expect(create('another')).rejects.toMatchObject({
statusCode: 403,
});
exists.mockRestore();
});
});
describe('admission bypass', () => {
const unverified = (): Actor =>
makeActor({
+38 -22
View File
@@ -43,6 +43,7 @@ import { PuterDriver } from '../types.js';
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
import { awaitLater } from '../../util/concurrency.js';
import { isUniqueViolation } from '../../util/dbError.js';
import { assertRentedUnitRoom } from '../../util/rentedUnits.js';
import { loadFileInput } from '../util/fileInput.js';
import {
normalizeLimit,
@@ -57,6 +58,7 @@ const CF_BASE_URL = 'https://api.cloudflare.com/client/v4/accounts';
export const WORKER_COMPATIBILITY_DATE = '2025-07-15';
export const WORKER_COMPATIBILITY_FLAGS = ['global_fetch_strictly_public'];
const WORKER_NAME_REGEX = /^[a-zA-Z0-9_-]+$/;
/** Cap on ordinary workers while user-scope handlers are off. */
const MAX_WORKERS_PER_USER = 100;
/**
@@ -179,12 +181,17 @@ export interface InternalDeployTarget {
* scripts belong to what.
*/
tags?: string[];
/**
* Per-invocation CPU limit in milliseconds, sent with the upload. Unset
* sends no limit. Ignored by the local runtime.
*/
cpuMs?: number;
}
const MAX_SOURCE_SIZE = 10 * 1024 * 1024; // 10 MB
// How far to scan an app's child apps when resolving which workers it may
// see. A user is capped at MAX_WORKERS_PER_USER workers, so only that many
// child apps can actually own one; the ceiling is generous slack over that.
// see. A user is capped at their plan's rented units, so only that many
// child apps can actually own a worker; the ceiling is generous slack over that.
const CHILD_APP_SCAN_LIMIT = 1000;
// -- Preamble --------------------------------------------------------
@@ -452,26 +459,34 @@ export class WorkerDriver extends PuterDriver {
this.#requireCfConfig();
// Quota check — count existing workers.puter.* subdomains owned by user
//
// A deploy target creates no such row, so this count can neither see
// its scripts nor be grown by them; running it would only make an
// unrelated worker quota decide whether these deploy. Whatever bounds
// them has to be counted where they actually live, and belongs to the
// caller that owns their lifecycle.
const existingWorkers =
skipAdmission || deployTarget
? []
: await this.stores.subdomain.listByUserIdAndPrefix(
actor.user.id,
WORKER_SUBDOMAIN_PREFIX,
);
if (existingWorkers.length >= MAX_WORKERS_PER_USER) {
throw new HttpError(
403,
`Worker limit reached (max ${MAX_WORKERS_PER_USER})`,
{ legacyCode: 'forbidden' },
);
// With user scopes on, a new worker is one more rented unit under the
// plan's cap, and redeploying an existing worker is never refused.
// Deploy targets create no worker row and are bounded by their caller.
if (this.config.events?.userHandlers === true) {
if (!skipAdmission && !deployTarget && !existingSub) {
await assertRentedUnitRoom(
{
stores: this.stores,
metering: this.services.metering,
},
actor,
);
}
} else {
const existingWorkers =
skipAdmission || deployTarget
? []
: await this.stores.subdomain.listByUserIdAndPrefix(
actor.user.id,
WORKER_SUBDOMAIN_PREFIX,
);
if (existingWorkers.length >= MAX_WORKERS_PER_USER) {
throw new HttpError(
403,
`Worker limit reached (max ${MAX_WORKERS_PER_USER})`,
{ legacyCode: 'forbidden' },
);
}
}
// Read source file. loadFileInput runs the read-ACL check internally
@@ -940,6 +955,7 @@ export class WorkerDriver extends PuterDriver {
compatibility_flags: WORKER_COMPATIBILITY_FLAGS,
compatibility_date: WORKER_COMPATIBILITY_DATE,
...(target?.tags ? { tags: target.tags } : {}),
...(target?.cpuMs ? { limits: { cpu_ms: target.cpuMs } } : {}),
bindings: [
// A script deployed without a token gets no binding at all,
// rather than an empty one for its code to find.
+1 -1
View File
@@ -2864,7 +2864,7 @@ export class AuthService extends PuterService {
#actorUserFromRow(user: UserRow) {
// Strip the password hash; pass everything else through so callers
// can read metadata, desktop_bg_*, otp_enabled, etc. without
// can read metadata, otp_enabled, etc. without
// re-fetching the user. Mirrors what /whoami exposes off of UserRow.
const { password: _password, ...rest } = user;
return {
@@ -185,7 +185,11 @@ const aclService = () => ({
!denied.has(resource.path) && !deniedHolders.has(actor.user?.id ?? -1),
getSafeAclError: async (_actor: Actor, resource: { path: string }) =>
denied.get(resource.path) === 'forbidden'
? { status: 403, message: 'Forbidden', fields: { code: 'forbidden' } }
? {
status: 403,
message: 'Forbidden',
fields: { code: 'forbidden' },
}
: {
status: 404,
message: 'Subject does not exist',
@@ -281,8 +285,8 @@ interface ForwardedCall {
}
/**
* Each service gets its own outbox. A delivery still in flight when a test
* ends must land in that test's record, not in the next one's.
* Each service gets its own outbox. A delivery still in flight when a test ends
* must land in that test's record, not in the next one's.
*/
const buildService = (
config: IConfig,
@@ -350,10 +354,7 @@ const buildService = (
acl: aclService(),
permission: permissionService(),
metering: {
bufferIncrementUsages: (
actor: Actor,
usages: UsageInput[],
) => {
bufferIncrementUsages: (actor: Actor, usages: UsageInput[]) => {
for (const usage of usages)
lines.push({
userUuid: actor.user?.uuid,
@@ -597,6 +598,13 @@ describe('the feature switch', () => {
nearPrimary: true,
leaseMs: 15 * 60_000,
},
{
name: 'events.userHandlers.gc',
every: 60 * 60_000,
scope: 'fleet',
nearPrimary: true,
leaseMs: 15 * 60_000,
},
{
name: 'events.subscriptions.creditSweep',
every: 15 * 60_000,
@@ -1256,7 +1264,9 @@ describe('matching', () => {
it('stops delivering to a durable row the moment it expires', async () => {
const { documents, file } = seedTree();
const now = Math.floor(Date.now() / 1000);
const row = (over: Partial<DurableSubscription>): DurableSubscription => ({
const row = (
over: Partial<DurableSubscription>,
): DurableSubscription => ({
subId: `durable-${seq}-${over.expiresAt}`,
token: `f#${documents.uid}`,
ownerUserId: userId,
@@ -1405,9 +1415,9 @@ describe('matching', () => {
await dispatch(closed);
await flush();
expect(sent.map((s) => (s.envelope.event as { uid: string }).uid)).toEqual([
reachable.uid,
]);
expect(
sent.map((s) => (s.envelope.event as { uid: string }).uid),
).toEqual([reachable.uid]);
});
it('publishes nothing for an event with no registry entry', async () => {
@@ -1449,9 +1459,9 @@ describe('matching', () => {
await flush();
expect(sent).toHaveLength(1);
expect(
(sent[0].envelope.event as { path: string }).path,
).toBe(inside.path);
expect((sent[0].envelope.event as { path: string }).path).toBe(
inside.path,
);
});
it('does not reach a sibling that only shares the missing name as a prefix', async () => {
@@ -1523,8 +1533,9 @@ describe('coalescing', () => {
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
expect(sent).toHaveLength(2);
expect(sent.map((s) => (s.envelope.event as { uid: string }).uid).sort())
.toEqual([file.uid, second.uid].sort());
expect(
sent.map((s) => (s.envelope.event as { uid: string }).uid).sort(),
).toEqual([file.uid, second.uid].sort());
});
it('counts exactly what it delivered', async () => {
@@ -1670,9 +1681,9 @@ describe('limits', () => {
}
}
await expect(subscribe(`fs:${documents.uid}`)).resolves.toMatchObject(
{ subId: expect.any(String) },
);
await expect(subscribe(`fs:${documents.uid}`)).resolves.toMatchObject({
subId: expect.any(String),
});
});
it('refuses unsubscribes past their own per-minute budget', async () => {
@@ -1741,9 +1752,7 @@ describe('limits', () => {
),
);
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(
1,
);
expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
expect(results.filter((r) => r.status === 'rejected')).toHaveLength(
attempts - 1,
);
@@ -1940,9 +1949,9 @@ describe('limits', () => {
).toHaveLength(EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT);
// 200 evaluated, 20 never reached: everything past the delivery cap is
// told it missed something, itself capped.
expect(
sent.filter((s) => s.envelope.event.op === 'gap'),
).toHaveLength(EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT);
expect(sent.filter((s) => s.envelope.event.op === 'gap')).toHaveLength(
EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT,
);
expect(
sent.find((s) => s.envelope.event.op === 'gap')?.envelope.event,
).toMatchObject({ reason: 'filter_evaluation_limit' });
@@ -1993,8 +2002,7 @@ describe('the socket surface', () => {
fire: (event, ...args) =>
(
handlers.get(event) as
| ((...a: unknown[]) => void)
| undefined
((...a: unknown[]) => void) | undefined
)?.(...args),
};
};
@@ -2175,10 +2183,14 @@ it('names the delivery channel the clients listen on', () => {
describe('asking a kv subscription for the value', () => {
it('is recorded on the row and reported in the view', async () => {
const { sub } = await service.subscribe(appActorFor(OWN_APP), socketId, {
subject: `kv:${OWN_APP}:cart`,
includeValue: true,
});
const { sub } = await service.subscribe(
appActorFor(OWN_APP),
socketId,
{
subject: `kv:${OWN_APP}:cart`,
includeValue: true,
},
);
expect(sub.includeValue).toBe(true);
const plain = await subscribeKv(`kv:${OWN_APP}:cart`);
@@ -2261,10 +2273,7 @@ describe('resolving a kv subject', () => {
});
it('refuses a pattern the grammar cannot enumerate from a key', async () => {
for (const subject of [
`kv:${OWN_APP}:ca*rt`,
`kv:${OWN_APP}:cart?`,
])
for (const subject of [`kv:${OWN_APP}:ca*rt`, `kv:${OWN_APP}:cart?`])
await expect(subscribeKv(subject)).rejects.toSatisfy(
(err: unknown) =>
isHttpError(err) && err.legacyCode === 'invalid_kv_pattern',
@@ -2379,8 +2388,7 @@ describe('delivering a kv change', () => {
const eventFor = (subId: string) =>
sent.find((one) => one.envelope.subId === subId)?.envelope.event as
| Record<string, unknown>
| undefined;
Record<string, unknown> | undefined;
it('hands the value to the row that asked for it and to no other', async () => {
vi.useFakeTimers();
@@ -2495,7 +2503,9 @@ describe('delivering a kv change', () => {
await dispatchKv(['cart'], { values: ['kept'] });
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
const deliveries = sent.filter((one) => one.envelope.event.op === 'set');
const deliveries = sent.filter(
(one) => one.envelope.event.op === 'set',
);
const gaps = sent.filter((one) => one.envelope.event.op === 'gap');
expect(deliveries).toHaveLength(128);
expect(gaps).toHaveLength(1);
@@ -2542,12 +2552,12 @@ describe('delivering a kv change', () => {
await dispatchKv(['cart']);
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
expect(sent.filter((one) => one.envelope.event.op === 'set')).toHaveLength(
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT.limit,
);
expect(sent.filter((one) => one.envelope.event.op === 'gap')).toHaveLength(
1,
);
expect(
sent.filter((one) => one.envelope.event.op === 'set'),
).toHaveLength(EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT.limit);
expect(
sent.filter((one) => one.envelope.event.op === 'gap'),
).toHaveLength(1);
});
it('uses the free KV limit when owner policy lookup fails', async () => {
@@ -2558,7 +2568,9 @@ describe('delivering a kv change', () => {
await dispatchKv(['cart']);
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
expect(sent.filter((one) => one.envelope.event.op === 'set')).toHaveLength(
expect(
sent.filter((one) => one.envelope.event.op === 'set'),
).toHaveLength(
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT.bySubscription[
DEFAULT_FREE_SUBSCRIPTION
],
@@ -2571,7 +2583,9 @@ describe('delivering a kv change', () => {
await seedKvSubscriptions(
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT.bySubscription[
DEFAULT_FREE_SUBSCRIPTION
] - 128 + 1,
] -
128 +
1,
{ includeValue: true, match: 'elsewhere', tag: 'filtered-out' },
);
@@ -2593,7 +2607,9 @@ describe('delivering a kv change', () => {
});
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
const deliveries = sent.filter((one) => one.envelope.event.op === 'set');
const deliveries = sent.filter(
(one) => one.envelope.event.op === 'set',
);
expect(deliveries).toHaveLength(128);
expect(
deliveries.every(
@@ -2819,8 +2835,7 @@ describe('what a kv dispatch costs', () => {
});
describe('the cross-app kv gate', () => {
const grantRead = (app: string) =>
grants.add(`app-data:${app}:kv:read`);
const grantRead = (app: string) => grants.add(`app-data:${app}:kv:read`);
const crossAppService = () =>
buildService({
@@ -2884,7 +2899,8 @@ describe('the cross-app kv gate', () => {
({ service } = crossAppService());
await expect(subscribeKv(`kv:${OTHER_APP}:cart`)).rejects.toSatisfy(
(err: unknown) => isHttpError(err) && err.legacyCode === 'forbidden',
(err: unknown) =>
isHttpError(err) && err.legacyCode === 'forbidden',
);
});
@@ -2895,8 +2911,7 @@ describe('the cross-app kv gate', () => {
await expect(subscribeKv('kv:app-gone:cart')).rejects.toSatisfy(
(err: unknown) =>
isHttpError(err) &&
err.legacyCode === 'subject_does_not_exist',
isHttpError(err) && err.legacyCode === 'subject_does_not_exist',
);
});
@@ -2910,7 +2925,8 @@ describe('the cross-app kv gate', () => {
});
await expect(subscribeKv(`kv:${OTHER_APP}:cart`)).rejects.toSatisfy(
(err: unknown) => isHttpError(err) && err.legacyCode === 'forbidden',
(err: unknown) =>
isHttpError(err) && err.legacyCode === 'forbidden',
);
});
@@ -3185,9 +3201,7 @@ describe('cross-user kv handles', () => {
expect(off.kvHandlesEnabled).toBe(false);
expect(service.kvHandlesEnabled).toBe(true);
await expect(
subscribeAsGuest(`kv:${handle}:*`, off),
).rejects.toSatisfy(
await expect(subscribeAsGuest(`kv:${handle}:*`, off)).rejects.toSatisfy(
(err: unknown) =>
isHttpError(err) &&
err.legacyCode === 'events_kv_handles_disabled',
@@ -3395,10 +3409,13 @@ describe('cross-user kv handles', () => {
it.each([
['an unknown handle', () => undefined],
['a revoked handle', () => mintHandle({ revokedAt: 1 })],
['a handle whose grant the caller does not hold', () => {
mintHandle();
grants.delete(permission);
}],
[
'a handle whose grant the caller does not hold',
() => {
mintHandle();
grants.delete(permission);
},
],
])('answers %s as absent', async (_case, setUp) => {
setUp();
await expect(subscribeAsGuest(`kv:${handle}:*`)).rejects.toSatisfy(
File diff suppressed because it is too large. Load diff
@@ -19,20 +19,29 @@
/**
* What deleting an app leaves behind on the events side: its durable rows,
* their backlog and its published handlers can never deliver again — the
* grant identity they are re-checked under no longer resolves — so
* `settleDeletedApp` tears them down on the `app.changed` bus key rather than
* leaving them to cost their holder an anchor slot forever. Also the hourly
* sweep that catches whatever neither this nor `workers.destroy` reached.
* their backlog and its published handlers can never deliver again — the grant
* identity they are re-checked under no longer resolves — so `settleDeletedApp`
* tears them down on the `app.changed` bus key rather than leaving them to cost
* their holder an anchor slot forever. Also the hourly sweep that catches
* whatever neither this nor `workers.destroy` reached.
*/
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import { testJobContext } from '../jobs/jobsTestUtil.js';
import type { IConfig } from '../../types.js';
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
import { EVENTS_WORKER_SESSION_NAME } from './workerRuntime.js';
import { appHandlerScope } from '../../stores/events/EventHandlerStore.js';
const BOOT_TIMEOUT_MS = 120_000;
@@ -126,10 +135,13 @@ const emitDeleted = async (appUid: string): Promise<void> => {
const publish = (
token: string,
body: { appUid: string; name: string; source: string },
): Promise<ApiResponse> => call('POST', '/events/handlers/publish', token, body);
): Promise<ApiResponse> =>
call('POST', '/events/handlers/publish', token, body);
/** A durable `single` subscription bound to a published handler, with the
* grants a background delivery needs. Returns the subscribed subId. */
/**
* A durable `single` subscription bound to a published handler, with the grants
* a background delivery needs. Returns the subscribed subId.
*/
const subscribeBackgroundHandler = async (
app: { uid: string; token: string },
handlerName: string,
@@ -163,6 +175,87 @@ const subscribeBackgroundHandler = async (
return subscribed.body.subId as string;
};
describe('user scopes of a deleted app or account', () => {
const publishUserHandler = (
appUid: string,
holder: number,
name = 'mine',
) =>
env.server.stores.eventHandler.publish({
scope: { appUid, holder },
name,
source: SOURCE,
broadcast: false,
});
const userHandlers = (appUid: string, holder: number) =>
env.server.stores.eventHandler.listForApp({ appUid, holder });
let destroyed: Array<{ appUid: string; scope: string; userId: number }>;
beforeAll(() => {
env.server.clients.event.on('events.worker.destroy', (_key, event) => {
destroyed?.push({
appUid: event.appUid,
scope: event.scope,
userId: event.actor.user.id,
});
});
});
beforeEach(async () => {
destroyed = [];
await env.server.clients.db.write(
'DELETE FROM `event_user_handlers`',
[],
);
});
it('removes every holder`s scope with the app, announcing each as the holder', async () => {
const app = await makeApp(userId);
await publishUserHandler(app.uid, userId);
await publishUserHandler(app.uid, otherUserId);
await publishUserHandler(app.uid, otherUserId, 'second');
const untouched = await makeApp(userId);
await publishUserHandler(untouched.uid, userId);
await emitDeleted(app.uid);
expect(await userHandlers(app.uid, userId)).toEqual([]);
expect(await userHandlers(app.uid, otherUserId)).toEqual([]);
expect(
destroyed
.filter((event) => event.scope === 'user')
.map((event) => [event.appUid, event.userId])
.sort(),
).toEqual(
[
[app.uid, userId],
[app.uid, otherUserId],
].sort(),
);
// Another app's scope is not touched.
expect(await userHandlers(untouched.uid, userId)).toHaveLength(1);
});
it('removes a deleted account`s scopes in every app, and no one else`s', async () => {
const first = await makeApp(userId);
const second = await makeApp(userId);
for (const app of [first, second]) {
await publishUserHandler(app.uid, otherUserId);
await publishUserHandler(app.uid, userId);
}
await env.server.clients.event.emitAndWait(
'user.delete',
{ user_id: otherUserId, appUids: [] },
{},
);
for (const app of [first, second]) {
expect(await userHandlers(app.uid, otherUserId)).toEqual([]);
expect(await userHandlers(app.uid, userId)).toHaveLength(1);
}
});
});
describe('an app deleted while it has events state', () => {
it('clears the rows, the backlog and the handlers when the app is deleted', async () => {
const app = await makeApp(userId);
@@ -187,15 +280,17 @@ describe('an app deleted while it has events state', () => {
await emitDeleted(app.uid);
const listed = await env.server.stores.durableSubscription.listForHolder(
userId,
{ appUid: app.uid },
);
const listed =
await env.server.stores.durableSubscription.listForHolder(userId, {
appUid: app.uid,
});
expect(listed.items).toEqual([]);
expect(await env.server.stores.pendingDelivery.depth(subId)).toBe(0);
expect(await env.server.stores.eventHandler.listForApp(app.uid)).toEqual(
[],
);
expect(
await env.server.stores.eventHandler.listForApp(
appHandlerScope(app.uid),
),
).toEqual([]);
});
it("clears the rows and handlers of a deleted account's apps, and the account's subscription keys", async () => {
@@ -218,14 +313,16 @@ describe('an app deleted while it has events state', () => {
{},
);
const listed = await env.server.stores.durableSubscription.listForHolder(
userId,
{ appUid: app.uid },
);
const listed =
await env.server.stores.durableSubscription.listForHolder(userId, {
appUid: app.uid,
});
expect(listed.items).toEqual([]);
expect(await env.server.stores.eventHandler.listForApp(app.uid)).toEqual(
[],
);
expect(
await env.server.stores.eventHandler.listForApp(
appHandlerScope(app.uid),
),
).toEqual([]);
expect(
await env.server.stores.eventSubscription.isRegionWarm(goneUserId),
).toBe(false);
@@ -260,12 +357,10 @@ describe('an app deleted while it has events state', () => {
expiresAt: null,
});
const { actor: ownerActor } = await env.server.services.auth.authenticate(
env.users.user.token,
);
const { actor: otherActor } = await env.server.services.auth.authenticate(
env.users.other.token,
);
const { actor: ownerActor } =
await env.server.services.auth.authenticate(env.users.user.token);
const { actor: otherActor } =
await env.server.services.auth.authenticate(env.users.other.token);
const ownerToken = await env.server.services.auth.createWorkerAppToken(
ownerActor!,
app.uid,
@@ -616,6 +616,210 @@ describe('reading again from a saved cursor', () => {
});
});
describe('starting from the end', () => {
const SUBJECT = 'notif:app-user';
const message = async (title: string, inApp = appUid): Promise<string> =>
(
await env.server.stores.notification.create({
userId,
value: { type: 'app.message', title },
type: 'app.message',
audience: 'app-user',
appUid: inApp,
group: `group-${uuidv4()}`,
})
).uid as string;
const fromEnd = async (token = appToken): Promise<string> => {
const end = await fetchPage(token, { subject: SUBJECT, from: 'end' });
expect(end.status).toBe(200);
expect(end.body.items).toEqual([]);
expect(end.body.cursor).toBeTruthy();
return end.body.cursor!;
};
it('answers a cursor on an empty slice that returns what comes after', async () => {
await clearMailboxes();
const cursor = await fromEnd();
const later = await message('later');
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
expect(uidsOf(next)).toEqual([later]);
});
it('skips what is there and returns only newer rows', async () => {
await clearMailboxes();
for (let i = 0; i < 3; i++) await message(`before ${i}`);
const cursor = await fromEnd();
const quiet = await fetchPage(appToken, { subject: SUBJECT, cursor });
expect(quiet.body.items).toEqual([]);
expect(quiet.body.cursor).toBeUndefined();
const later = [await message('after 0'), await message('after 1')];
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
expect(uidsOf(next)).toEqual(later);
});
it('returns a row replaced after the cursor was taken', async () => {
await clearMailboxes();
const old = await message('old');
await message('older still unchanged');
const cursor = await fromEnd();
await env.server.services.notification.notifyUpdate(
old,
userId,
{ title: 'replaced' },
{ type: 'app.message', replace: true },
);
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
expect(uidsOf(next)).toEqual([old]);
expect(next.body.items![0]).toMatchObject({
replaced: true,
notification: { title: 'replaced' },
});
});
it('returns a replacement stamped by a clock behind the server`s', async () => {
await clearMailboxes();
const behind = Date.now() - 60_000;
const [early, late] = [await message('early'), await message('late')];
const replace = (uid: string, title: string, replacedAt: number) =>
env.server.stores.notification.updateValue(
uid,
userId,
{ type: 'app.message', title },
{ onlyUnseen: true, replacedAt },
);
await replace(early, 'replaced before', behind - 1_000);
const cursor = await fromEnd();
await replace(late, 'replaced after', behind);
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
expect(uidsOf(next)).toEqual([late]);
expect(next.body.items![0]).toMatchObject({
notification: { title: 'replaced after' },
});
});
it('keeps to the slice the subject names', async () => {
await clearMailboxes();
await message('own before');
const cursor = await fromEnd();
await message('other app', otherAppUid);
await seed(userId, { title: 'account row' });
const own = await message('own after');
const next = await fetchPage(appToken, { subject: SUBJECT, cursor });
expect(uidsOf(next)).toEqual([own]);
});
it('answers no cursor for a slice the caller cannot see', async () => {
await clearMailboxes();
await message('other app', otherAppUid);
for (const [token, subject] of [
[appToken, `notif:${otherAppUid}:app-user`],
[appToken, `notif:${userUuid}:account`],
[appReadUrlToken, SUBJECT],
] as const) {
const page = await fetchPage(token, { subject, from: 'end' });
expect(page.status).toBe(200);
expect(page.body.items).toEqual([]);
expect(page.body.cursor).toBeUndefined();
}
});
it('reads from the cursor when one is given', async () => {
await clearMailboxes();
const uids = [await message('m0'), await message('m1')];
const first = await fetchPage(appToken, { subject: SUBJECT, limit: 1 });
const page = await fetchPage(appToken, {
subject: SUBJECT,
cursor: first.body.cursor,
from: 'end',
});
expect(uidsOf(page)).toEqual([uids[1]]);
});
it('rejects a `from` that is neither start nor end', async () => {
const page = await fetchPage(appToken, {
subject: SUBJECT,
from: 'middle',
});
expect(page.status).toBe(400);
const start = await fetchPage(appToken, {
subject: SUBJECT,
from: 'start',
});
expect(start.status).toBe(200);
});
it('starts the SDK feed without catch-up in one read', async () => {
await clearMailboxes();
for (let i = 0; i < 3; i++) await message(`before ${i}`);
const appActor = (await env.server.services.auth.authenticate(appToken))
.actor!;
const reads: Array<Record<string, unknown>> = [];
let receive: (delivery: { event: unknown }) => void = () => {};
const titles: string[] = [];
const feed = new NotificationFeed(
{
authToken: appToken,
events: {
onLocal: async (
_subject: string,
handler: typeof receive,
) => {
receive = handler;
return { off: async () => {} };
},
fetch: async (options: Record<string, unknown>) => {
reads.push(options);
return env.server.services.events.fetchMissed(
appActor,
options,
);
},
},
} as never,
(notification: { title: string | null }) => {
titles.push(String(notification.title));
},
{ catchUp: false },
);
try {
await feed.start();
expect(reads).toEqual([
{ subject: SUBJECT, limit: 200, from: 'end' },
]);
await message('after start');
receive({ event: { op: 'gap', reason: 'reconnect' } });
await vi.waitFor(() => expect(reads).toHaveLength(2));
await new Promise((resolve) => setTimeout(resolve, 0));
} finally {
await feed.off();
}
expect(reads[1]).toEqual({
subject: SUBJECT,
limit: 200,
cursor: expect.any(String),
});
expect(titles).toEqual(['after start']);
});
});
describe('cross-user isolation', () => {
it('returns nothing of another account, whatever the subject names', async () => {
await clearMailboxes();
@@ -45,7 +45,10 @@ import {
EVENTS_WORKER_SOURCE_MAX_BYTES,
} from '../../controllers/events/limits.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import { hashContent } from '../../stores/events/EventHandlerStore.js';
import {
appHandlerScope,
hashContent,
} from '../../stores/events/EventHandlerStore.js';
import type { IConfig } from '../../types.js';
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
import type { DurableSubscriptionView } from './EventsService.js';
@@ -532,7 +535,7 @@ describe('what a publish reads', () => {
if (sql.includes(BY_NAME_SQL) && !landed) {
landed = true;
await env.server.stores.eventHandler.publish({
appUid,
scope: appHandlerScope(appUid),
name: 'ingestUpload',
source: NEXT_SOURCE,
replace: true,
@@ -590,7 +593,7 @@ describe('what a publish reads', () => {
pread.mockRestore();
}
const standing = await env.server.stores.eventHandler.getByName(
appUid,
appHandlerScope(appUid),
'ingestUpload',
);
expect(standing?.sourceHash).toBe(hashContent(THIRD_SOURCE));
@@ -603,7 +606,7 @@ describe('what a publish reads', () => {
if (sql.includes(BY_NAME_SQL) && !beaten) {
beaten = true;
await env.server.stores.eventHandler.publish({
appUid,
scope: appHandlerScope(appUid),
name: 'ingestUpload',
source: THIRD_SOURCE,
});
@@ -659,7 +662,7 @@ describe('what a publish reads', () => {
pread.mockRestore();
}
await expect(
env.server.stores.eventHandler.countForApp(appUid),
env.server.stores.eventHandler.countForApp(appHandlerScope(appUid)),
).resolves.toBe(0);
});
@@ -24,6 +24,7 @@ import {
HANDLER_NAME_MAX_LENGTH,
hashContent,
} from '../../stores/events/EventHandlerStore.js';
import type { HandlerScopeKind } from '../../stores/events/types.js';
/** Request and response pieces the subscription and schedule verbs share. */
@@ -53,6 +54,27 @@ export const parseHandlerHash = (value: unknown): string | null => {
return value;
};
/** Source of an inline handler; the store enforces the size and content rules. */
export const parseHandlerSource = (value: unknown): string | null => {
if (value === undefined || value === null) return null;
if (typeof value !== 'string' || value.trim().length === 0)
throw badRequest(
'handlerSource must be a non-empty string',
'bad_request',
);
return value;
};
/** Which handler set a request is about; absent means the app's. */
export const parseHandlerScopeKind = (
value: unknown,
field: string,
): HandlerScopeKind => {
if (value === undefined || value === null || value === 'app') return 'app';
if (value === 'user') return 'user';
throw badRequest(`${field} must be 'app' or 'user'`, 'bad_request');
};
/** Stored as JSON text; the byte cap is the store's to enforce. */
export const parseContext = (value: unknown): string | null => {
if (value === undefined || value === null) return null;
@@ -27,7 +27,15 @@
* out of service so a revoked subscription stops costing anything at all.
*/
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import {
afterAll,
beforeAll,
describe,
expect,
it,
onTestFinished,
vi,
} from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { EVENTS_COALESCE_WINDOW_MS } from '../../controllers/events/limits.js';
import { SUSPENDED_ROW_TTL_DAYS } from '../../controllers/events/limits.js';
@@ -43,6 +51,7 @@ import type { AclMode } from '../acl/ACLService.js';
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
import type { DeliveryEnvelope } from './EventsService.js';
import { fsAnchorToken } from './subjects.js';
import { appHandlerScope } from '../../stores/events/EventHandlerStore.js';
const BOOT_TIMEOUT_MS = 120_000;
const TABLE = 'event_subscriptions';
@@ -206,7 +215,7 @@ const makeApp = async (path: string): Promise<Actor> => {
EVENTS_BACKGROUND_PERMISSION,
);
await env.server.stores.eventHandler.publish({
appUid: uid,
scope: appHandlerScope(uid),
name: 'onChange',
source: 'async () => {}',
});
@@ -950,3 +959,165 @@ describe('what a revoked grant settles', () => {
}
});
});
describe('a holder`s user scope when consent is withdrawn', () => {
const scopeOf = (appUid: string, holder = owner.id) => ({
appUid,
holder,
});
const putUserHandler = async (appUid: string, holder = owner.id) => {
await env.server.stores.eventHandler.publish({
scope: scopeOf(appUid, holder),
name: 'mine',
source: 'async () => {}',
broadcast: false,
});
};
const held = (appUid: string, holder = owner.id) =>
env.server.stores.eventHandler.listForApp(scopeOf(appUid, holder));
let destroys: Array<{ appUid: string; scope: string; userId: number }>;
beforeAll(() => {
env.server.clients.event.on('events.worker.destroy', (_key, event) => {
destroys?.push({
appUid: event.appUid,
scope: event.scope,
userId: event.actor.user.id,
});
});
});
it('drops the scope when background consent is revoked, announcing it as the holder', async () => {
await clearRows();
destroys = [];
const path = await folder(`/${owner.username}/scope-consent`);
const app = (await makeApp(path)).app!;
await putUserHandler(app.uid);
await env.server.services.permission.revokeUserAppPermission(
owner.actor,
app.uid,
EVENTS_BACKGROUND_PERMISSION,
);
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
timeout: 5_000,
interval: 25,
});
expect(destroys.filter((event) => event.scope === 'user')).toEqual([
{ appUid: app.uid, scope: 'user', userId: owner.id },
]);
});
it('drops the scope when the whole app is revoked', async () => {
await clearRows();
const path = await folder(`/${owner.username}/scope-whole-app`);
const app = (await makeApp(path)).app!;
await putUserHandler(app.uid);
await env.server.services.permission.revokeUserAppAll(
owner.actor,
app.uid,
);
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
timeout: 5_000,
interval: 25,
});
});
it('leaves the scope when the consent still stands, as after a revoke then a grant', async () => {
await clearRows();
const path = await folder(`/${owner.username}/scope-regrant`);
const app = (await makeApp(path)).app!;
await putUserHandler(app.uid);
for (const permission of [EVENTS_BACKGROUND_PERMISSION, null])
await events().settleRevokedGrant({
holderUserId: owner.id,
appUid: app.uid,
permission,
});
expect(await held(app.uid)).toHaveLength(1);
});
it('leaves the scope when the consent check itself fails', async () => {
await clearRows();
const path = await folder(`/${owner.username}/scope-check-fails`);
const app = (await makeApp(path)).app!;
await putUserHandler(app.uid);
const check = vi
.spyOn(events(), 'hasBackgroundConsent')
.mockResolvedValue(null);
try {
await events().settleRevokedGrant({
holderUserId: owner.id,
appUid: app.uid,
permission: EVENTS_BACKGROUND_PERMISSION,
});
expect(check).toHaveBeenCalled();
expect(await held(app.uid)).toHaveLength(1);
} finally {
check.mockRestore();
}
});
it('leaves the scope for an unrelated grant revoked, and another holder`s scope for any', async () => {
await clearRows();
const path = await folder(`/${owner.username}/scope-unrelated`);
const app = (await makeApp(path)).app!;
await putUserHandler(app.uid);
await putUserHandler(app.uid, guest.id);
await env.server.services.permission.revokeUserAppPermission(
owner.actor,
app.uid,
`fs:${await uidOf(path)}:list`,
);
await env.server.services.permission.revokeUserAppPermission(
owner.actor,
app.uid,
EVENTS_BACKGROUND_PERMISSION,
);
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
timeout: 5_000,
interval: 25,
});
expect(await held(app.uid, guest.id)).toHaveLength(1);
});
it('keeps the rows bound to a dropped scope, suspended rather than deleted', async () => {
await clearRows();
const path = await folder(`/${owner.username}/scope-rows`);
const appActor = await makeApp(path);
const app = appActor.app!;
const config = (events() as unknown as { config: IConfig }).config;
config.events!.userHandlers = true;
onTestFinished(() => {
config.events!.userHandlers = false;
});
const sub = (
await events().subscribeDurable(appActor, {
subject: `fs:${path}`,
handlerSource: 'async () => {}',
})
).sub;
await env.server.services.permission.revokeUserAppPermission(
owner.actor,
app.uid,
EVENTS_BACKGROUND_PERMISSION,
);
await suspendedRow(sub.subId);
await vi.waitFor(async () => expect(await held(app.uid)).toEqual([]), {
timeout: 5_000,
interval: 25,
});
expect((await rowOf(sub.subId)).suspended_reason).toBe(
'permission_revoked',
);
});
});
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,386 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* A user-scope handler through the real local worker runtime: created inline by
* an app session, deployed eagerly as its holder, fired by an event, and run
* with the holder's identity. Also the rehydrate route's scope resolution.
*/
import http from 'node:http';
import type { AddressInfo } from 'node:net';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import {
appHandlerScope,
hashContent,
} from '../../stores/events/EventHandlerStore.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import { EVENTS_BACKGROUND_PERMISSION } from './authorization.js';
import { userHandlerScript } from './workerRuntime.js';
const BOOT_TIMEOUT_MS = 180_000;
const TEST_TIMEOUT_MS = 60_000;
const INTERNAL_SECRET = 'events-internal-secret';
const SOURCE = `async ({ event, ctx, user, fetch, ack }) => {
await fetch(ctx.sink, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
label: ctx.label,
path: event.path,
token: user.authToken,
}),
});
await ack();
}`;
interface SinkPost {
label?: string;
path?: string;
token?: string;
}
let env: PuterTestEnv;
let holders: Array<{ id: number; uuid: string; anchor: string; token: string }>;
let appUid: string;
let sink: http.Server;
let sinkUrl: string;
let sinkPosts: SinkPost[];
const localWorkers = () => env.server.services.localworkerservice;
const scriptFor = (source: string): string =>
userHandlerScript(
{ name: `@${hashContent(source)}`, sourceHash: hashContent(source) },
'',
);
const isResident = async (script: string): Promise<boolean> =>
(await localWorkers().dispatchEventsWorker(
new Request(`http://${script}.workers.puter.localhost/`, {
method: 'GET',
}),
)) !== null;
const api = async (
method: 'GET' | 'POST',
path: string,
token: string,
body?: object,
): Promise<{ status: number; body: Record<string, unknown> }> => {
const response = await fetch(new URL(path, env.apiOrigin), {
method,
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
...(body ? { body: JSON.stringify(body) } : {}),
});
return {
status: response.status,
body: (await response.json()) as Record<string, unknown>,
};
};
const rehydrate = async (
body: object,
): Promise<{ status: number; body: Record<string, unknown> }> => {
const response = await fetch(
new URL('/events/worker/rehydrate', env.apiOrigin),
{
method: 'POST',
headers: {
'content-type': 'application/json',
'x-puter-internal-auth': INTERNAL_SECRET,
},
body: JSON.stringify(body),
},
);
return {
status: response.status,
body: (await response.json()) as Record<string, unknown>,
};
};
const createInline = async (
holder: (typeof holders)[number],
label: string,
source = SOURCE,
): Promise<string> => {
const response = await api('POST', '/events/subscribe', holder.token, {
subject: `fs:${holder.anchor}`,
delivery: 'single',
handlerSource: source,
context: { sink: sinkUrl, label },
});
expect(response.status).toBe(200);
return String(response.body.subId);
};
const delivered = (count = 1): Promise<void> =>
vi.waitFor(() => expect(sinkPosts.length).toBeGreaterThanOrEqual(count), {
timeout: 30_000,
interval: 50,
});
beforeAll(async () => {
env = await setupPuterTestEnv({
events: {
enabled: true,
workerRuntime: true,
userHandlers: true,
internalSecret: INTERNAL_SECRET,
},
workers: { localServer: true },
unlimitedMetering: true,
} as IConfig);
const owner = (await env.server.stores.user.getByUsername(
env.users.user.username,
))!;
appUid = `app-${uuidv4()}`;
await env.server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[appUid, appUid, appUid, `https://${appUid}.example/`, owner.id],
);
holders = [];
for (const account of [env.users.user, env.users.other]) {
const user = (await env.server.stores.user.getByUsername(
account.username,
))!;
const anchor = `/${account.username}/uh-e2e`;
await env.server.services.fs.mkdir(user.id, {
path: anchor,
createMissingParents: true,
});
const entry = await env.server.stores.fsEntry.getEntryByPath(anchor);
const { actor } = await env.server.services.auth.authenticate(
account.token,
);
for (const permission of [
`fs:${entry!.uid}:list`,
EVENTS_BACKGROUND_PERMISSION,
])
await env.server.services.permission.grantUserAppPermission(
actor!,
appUid,
permission,
);
holders.push({
id: user.id,
uuid: user.uuid,
anchor,
token: await env.server.services.auth.getUserAppToken(
actor!,
appUid,
),
});
}
sinkPosts = [];
sink = http.createServer((req, res) => {
const chunks: Buffer[] = [];
req.on('data', (chunk: Buffer) => chunks.push(chunk));
req.on('end', () => {
sinkPosts.push(
JSON.parse(Buffer.concat(chunks).toString('utf8') || '{}'),
);
res.writeHead(200).end();
});
});
await new Promise<void>((resolve) =>
sink.listen(0, '127.0.0.1', () => resolve()),
);
sinkUrl = `http://127.0.0.1:${(sink.address() as AddressInfo).port}`;
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await new Promise<void>((resolve) => {
if (!sink) return resolve();
sink.closeAllConnections();
sink.close(() => resolve());
});
await env?.shutdown();
});
beforeEach(() => {
sinkPosts = [];
});
describe('an inline handler in a user scope, end to end', () => {
it(
'is deployed eagerly, fires on an event and runs as its holder',
{ timeout: TEST_TIMEOUT_MS },
async () => {
const [holder] = holders;
const source = `${SOURCE}\n// run-as-holder`;
const script = scriptFor(source);
expect(await isResident(script)).toBe(false);
await createInline(holder, 'inline', source);
// The deploy started with the publish, before any event.
await vi.waitFor(async () =>
expect(await isResident(script)).toBe(true),
);
await env.server.services.fs.touch(holder.id, {
path: `${holder.anchor}/note.txt`,
});
await delivered();
expect(sinkPosts[0]).toMatchObject({
label: 'inline',
path: `${holder.anchor}/note.txt`,
});
const { actor } = await env.server.services.auth.authenticate(
sinkPosts[0].token!,
);
expect(actor!.user?.id).toBe(holder.id);
expect(actor!.effectiveApp?.uid).toBe(appUid);
},
);
it(
'shares one deployed script between holders of the same code, each run as its own holder',
{ timeout: TEST_TIMEOUT_MS },
async () => {
const source = `${SOURCE}\n// shared-code`;
const script = scriptFor(source);
const create = vi.spyOn(env.server.drivers.workers, 'create');
await createInline(holders[0], 'first', source);
await vi.waitFor(async () =>
expect(await isResident(script)).toBe(true),
);
await createInline(holders[1], 'second', source);
// Let the second holder's eager deploy settle before counting.
await new Promise((resolve) => setTimeout(resolve, 500));
// The marker made the second holder's eager deploy a no-op.
expect(
create.mock.calls.filter(
([args]) =>
(args as { workerName: string }).workerName === script,
),
).toHaveLength(1);
for (const [i, holder] of holders.entries())
await env.server.services.fs.touch(holder.id, {
path: `${holder.anchor}/shared-${i}.txt`,
});
await delivered(2);
const runAs = new Set<number>();
for (const post of sinkPosts)
runAs.add(
(await env.server.services.auth.authenticate(post.token!))
.actor!.user!.id,
);
expect(runAs).toEqual(new Set(holders.map((h) => h.id)));
create.mockRestore();
},
);
});
describe('the rehydrate route', () => {
const source = `${SOURCE}\n// rehydrate`;
const script = scriptFor(source);
beforeAll(async () => {
// Rows only: the handler is published, nothing is deployed here.
const eventHandler = env.server.stores.eventHandler;
await eventHandler.publish({
scope: { appUid, holder: holders[0].id },
name: `@${hashContent(source)}`,
source,
broadcast: false,
});
await eventHandler.publish({
scope: { appUid, holder: holders[1].id },
name: 'other',
source: SOURCE,
broadcast: false,
});
});
it('resolves the holder`s user scope when the body names one', async () => {
const response = await rehydrate({
script,
appUid,
holder: holders[0].uuid,
});
expect(response).toMatchObject({
status: 200,
body: { deployed: true },
});
expect(await isResident(script)).toBe(true);
});
it('reads an absent or empty holder as the app scope, which has no such handler', async () => {
for (const body of [
{ script, appUid },
{ script, appUid, holder: '' },
])
expect(await rehydrate(body)).toMatchObject({
status: 404,
body: { deployed: false, reason: 'no-handlers' },
});
expect(
await env.server.stores.eventHandler.listForApp(
appHandlerScope(appUid),
),
).toEqual([]);
});
it('answers stale for a holder whose scope does not hold that handler', async () => {
expect(
await rehydrate({ script, appUid, holder: holders[1].uuid }),
).toMatchObject({
status: 404,
body: { deployed: false, reason: 'stale' },
});
});
it('refuses a holder that does not exist, and a malformed one', async () => {
expect(
await rehydrate({ script, appUid, holder: uuidv4() }),
).toMatchObject({
status: 404,
body: { deployed: false, reason: 'no-holder' },
});
expect(
(await rehydrate({ script, appUid, holder: 'a b' })).status,
).toBe(400);
expect((await rehydrate({ script, appUid, holder: 7 })).status).toBe(
400,
);
});
});
@@ -34,6 +34,7 @@ import type { AddressInfo } from 'node:net';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
@@ -58,8 +59,14 @@ import {
EVENTS_WORKER_SESSION_NAME,
eventsInvokeKey,
eventsWorkerScript,
userHandlerScript,
} from './workerRuntime.js';
import { handlerSetHash } from './workerSource.js';
import {
appHandlerScope,
hashContent,
} from '../../stores/events/EventHandlerStore.js';
import type { WorkerInvocation } from './workerSeam.js';
const BOOT_TIMEOUT_MS = 120_000;
/** Short enough that a hung handler is a test case rather than a stall. */
@@ -129,6 +136,7 @@ const startStub = async (): Promise<string> => {
script: req.headers['x-puter-events-script'] as string,
app: req.headers['x-puter-events-app'] as string,
key: req.headers['x-puter-events-key'] as string,
holder: req.headers['x-puter-events-holder'] as string,
deployed,
},
body: JSON.parse(raw || '{}') as StubCall['body'],
@@ -233,7 +241,7 @@ const makeWorkerApp = async (): Promise<{
);
const token = await env.server.services.auth.getUserAppToken(actor!, uid);
await env.server.stores.eventHandler.publish({
appUid: uid,
scope: appHandlerScope(uid),
name: HANDLER,
source: SOURCE,
});
@@ -403,7 +411,7 @@ beforeAll(async () => {
);
appToken = await env.server.services.auth.getUserAppToken(actor!, appUid);
await env.server.stores.eventHandler.publish({
appUid,
scope: appHandlerScope(appUid),
name: HANDLER,
source: SOURCE,
});
@@ -471,7 +479,9 @@ describe('the call an owed delivery makes', () => {
const script = eventsWorkerScript(
handlerSetHash(
await env.server.stores.eventHandler.setForApp(appUid),
await env.server.stores.eventHandler.setForApp(
appHandlerScope(appUid),
),
),
// No `workers.internetExposedUrl` in this test's config.
'',
@@ -1014,7 +1024,10 @@ describe('with no events worker to address', () => {
const subId = await subscribe();
// Unpublished after the fact: the app's set now hashes to nothing, so
// there is no script to name and nowhere to send the delivery.
await env.server.stores.eventHandler.remove(appUid, HANDLER);
await env.server.stores.eventHandler.remove(
appHandlerScope(appUid),
HANDLER,
);
try {
await touch('unresolved.txt');
@@ -1037,7 +1050,7 @@ describe('with no events worker to address', () => {
expect(calls).toEqual([]);
} finally {
await env.server.stores.eventHandler.publish({
appUid,
scope: appHandlerScope(appUid),
name: HANDLER,
source: SOURCE,
});
@@ -1416,3 +1429,349 @@ describe('handlers whose writes run handlers', () => {
).resolves.toBeNull();
});
});
describe('a handler in a user scope', () => {
const USER_SOURCE =
'async ({ event, ctx }) => { console.log("user", event.path, ctx.label); }';
const userName = `@${hashContent(USER_SOURCE)}`;
let holderUuid: string;
let createSpy: ReturnType<typeof vi.spyOn>;
let meterSpy: ReturnType<typeof vi.spyOn>;
const setFlag = (on: boolean): void => {
(
events() as unknown as { config: IConfig }
).config.events!.userHandlers = on;
};
const userScope = () => ({ appUid, holder: userId });
const userScript = () =>
userHandlerScript(
{ name: userName, sourceHash: hashContent(USER_SOURCE) },
'',
);
const post = async (path: string, body: object) => {
const response = await fetch(new URL(path, env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${appToken}`,
},
body: JSON.stringify(body),
});
return {
status: response.status,
body: (await response.json()) as Record<string, unknown>,
};
};
/** A single-delivery row bound to a handler published by its own source. */
const inlineRow = async (source = USER_SOURCE): Promise<string> => {
const created = await post('/events/subscribe', {
subject: `fs:${anchor}`,
delivery: 'single',
handlerSource: source,
context: { label: 'user' },
});
expect(created.status).toBe(200);
return String(created.body.subId);
};
const invocationFor = (subId: string): WorkerInvocation => ({
subId,
holderUserId: userId,
appUid,
handlerName: userName,
handlerScope: 'user',
event: {
subject: 'schedule:s1',
op: 'tick',
id: 's1:1',
scheduleId: 's1',
scheduledFor: 1,
missed: 0,
attempt: 1,
},
context: null,
handlerDepth: 0,
});
beforeAll(async () => {
holderUuid = (await env.server.stores.user.getById(userId))!.uuid;
});
beforeEach(async () => {
setFlag(true);
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
// Eager and on-miss deploys have no deploy target in this suite.
createSpy = vi
.spyOn(env.server.drivers.workers, 'create')
.mockResolvedValue({ success: true, errors: [] });
meterSpy = vi.spyOn(
env.server.services.metering,
'bufferIncrementUsages',
);
});
afterEach(async () => {
createSpy.mockRestore();
meterSpy.mockRestore();
setFlag(false);
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
});
it('addresses the handler`s own script and names its holder', async () => {
const subId = await inlineRow();
await touch('user-scope.txt');
await invoked(1);
const call = calls[0];
expect(call.headers.script).toBe(userScript());
expect(call.headers.holder).toBe(holderUuid);
expect(call.headers.key).toBe(
eventsInvokeKey(INTERNAL_SECRET, userScript()),
);
expect(call.body.handler).toBe(userName);
await waitUntil(async () =>
expect(await pending().depth(subId)).toBe(0),
);
});
it('sends no holder for an app-scope row', async () => {
await subscribe();
await touch('app-scope.txt');
await invoked(1);
expect(calls[0].headers.holder).toBeUndefined();
});
it('keeps a user row and an app row of the same name on their own scope`s handler', async () => {
await env.server.stores.eventHandler.publish({
scope: userScope(),
name: HANDLER,
source: USER_SOURCE,
});
const created = await post('/events/subscribe', {
subject: `fs:${anchor}`,
delivery: 'single',
handlerName: HANDLER,
handlerScope: 'user',
});
expect(created.status).toBe(200);
await touch('same-name-user.txt');
await invoked(1);
// One at a time: a second row firing alongside would go as a batch.
expect(
(
await post('/events/unsubscribe', {
subId: String(created.body.subId),
})
).status,
).toBe(200);
await subscribe();
await touch('same-name-app.txt');
await invoked(2);
const appScript = eventsWorkerScript(
handlerSetHash(
await env.server.stores.eventHandler.setForApp(
appHandlerScope(appUid),
),
),
'',
);
const mine = userHandlerScript(
{ name: HANDLER, sourceHash: hashContent(USER_SOURCE) },
'',
);
expect(mine).not.toBe(appScript);
expect(calls[0].headers).toMatchObject({
script: mine,
holder: holderUuid,
});
expect(calls[1].headers.script).toBe(appScript);
expect(calls[1].headers.holder).toBeUndefined();
});
it('suspends a row whose handler left its holder`s set, and tells the holder only', async () => {
const notify = vi.spyOn(env.server.services.notification, 'notify');
const subId = await inlineRow();
// Gone without the removal path that would suspend its rows.
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
await touch('missing.txt');
await waitUntil(async () =>
expect((await durable().getBySubId(subId))?.suspendedReason).toBe(
'handler_not_found',
),
);
expect(calls).toEqual([]);
expect(notify).toHaveBeenCalledWith(
[userId],
expect.objectContaining({ handler: userName }),
expect.objectContaining({
type: 'app.events.handler_suspended',
appUid,
}),
);
expect(notify).not.toHaveBeenCalledWith(
expect.anything(),
expect.anything(),
expect.objectContaining({ type: 'app.events.suspended' }),
);
notify.mockRestore();
});
it('does not suspend a schedule run whose replica read lags the publish', async () => {
const subId = await inlineRow();
const store = env.server.stores.eventHandler;
const real = store.setForApp.bind(store);
// The replica has not seen the insert; the primary has.
const lagging = vi
.spyOn(store, 'setForApp')
.mockImplementation(async (scope, options) =>
options?.replica
? [{ name: 'other', sourceHash: 'a'.repeat(64) }]
: real(scope, options),
);
try {
const result = await events().invokeForSchedule(
invocationFor(subId),
);
expect(result).toMatchObject({ outcome: 'settled' });
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
expect(calls[0].headers.script).toBe(userScript());
} finally {
lagging.mockRestore();
}
});
it('reads a cold user scope from the replica only, once, for a delivery', async () => {
await inlineRow();
// Eager deploy at publish has finished with the primary by now.
await new Promise((resolve) => setTimeout(resolve, 300));
const redis = env.server.clients.redis;
const key = `eventHandlers:uset:${appUid}:${userId}`;
await redis.del(key, `${key}:r`);
const db = env.server.clients.db;
const primary = vi.spyOn(db, 'pread');
const replica = vi.spyOn(db, 'read');
const handlerReads = (spy: typeof primary) =>
spy.mock.calls.filter(([sql]) =>
String(sql).includes('event_user_handlers'),
).length;
await touch('cold-user-scope.txt');
await invoked(1);
expect(handlerReads(primary)).toBe(0);
expect(handlerReads(replica)).toBe(1);
primary.mockRestore();
replica.mockRestore();
});
it('resumes a suspended row when the name is published again', async () => {
const subId = await inlineRow();
await env.server.stores.eventHandler.deleteUserScope(appUid, userId);
await touch('gone.txt');
await waitUntil(async () =>
expect((await durable().getBySubId(subId))?.suspendedReason).toBe(
'handler_not_found',
),
);
expect(
(
await post('/events/handlers/publish', {
scope: 'user',
name: userName,
source: USER_SOURCE,
})
).status,
).toBe(200);
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
});
it('does not address a holder that is suspended, and leaves the row owed', async () => {
const subId = await inlineRow();
await env.server.clients.db.write(
'UPDATE user SET suspended = 1 WHERE id = ?',
[userId],
);
await env.server.stores.user.invalidateById(userId);
try {
await touch('suspended-holder.txt');
await failuresRecorded(subId, 1);
expect(calls).toEqual([]);
expect(await pending().depth(subId)).toBe(1);
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
} finally {
await env.server.clients.db.write(
'UPDATE user SET suspended = 0 WHERE id = ?',
[userId],
);
await env.server.stores.user.invalidateById(userId);
}
});
it('deploys the holder`s script on a dispatcher miss, as that holder', async () => {
dispatchMissing = true;
await inlineRow();
await touch('miss.txt');
await invoked(2);
expect(calls[0].headers.deployed).toBeUndefined();
expect(calls[1].headers.deployed).toBe('1');
// Once for the eager deploy at publish, once for the miss.
const deployed = createSpy.mock.calls.filter(
([args]) =>
(args as { workerName: string }).workerName === userScript(),
);
expect(deployed.length).toBeGreaterThanOrEqual(1);
});
describe('with the flag off', () => {
it('leaves the delivery owed: nothing is called, suspended or billed', async () => {
const subId = await inlineRow();
setFlag(false);
meterSpy.mockClear();
await touch('flag-off.txt');
await new Promise((resolve) => setTimeout(resolve, 400));
expect(calls).toEqual([]);
expect(await pending().depth(subId)).toBe(1);
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
expect(meterSpy).not.toHaveBeenCalled();
});
it('answers unavailable from the invoker and for a schedule run', async () => {
const subId = await inlineRow();
setFlag(false);
const invocation = invocationFor(subId);
const worker = events().worker as unknown as {
run: (i: WorkerInvocation) => Promise<unknown>;
};
expect(await worker.run(invocation)).toEqual({
outcome: 'unavailable',
});
expect(await events().invokeForSchedule(invocation)).toEqual({
outcome: 'unavailable',
});
expect(calls).toEqual([]);
expect((await durable().getBySubId(subId))?.suspendedAt).toBeNull();
});
it('leaves an app-scope row alone', async () => {
setFlag(false);
await subscribe();
await touch('app-row-flag-off.txt');
await invoked(1);
expect(calls[0].headers.script).toMatch(/^evw-/);
});
});
it('runs a schedule invocation of a user scope with its holder', async () => {
const subId = await inlineRow();
const result = await events().invokeForSchedule(invocationFor(subId));
expect(result).toMatchObject({ outcome: 'settled' });
expect(calls[0].headers.script).toBe(userScript());
expect(calls[0].headers.holder).toBe(holderUuid);
});
});
@@ -61,6 +61,7 @@ import {
eventsWorkerScript,
} from './workerRuntime.js';
import { handlerSetHash } from './workerSource.js';
import { appHandlerScope } from '../../stores/events/EventHandlerStore.js';
const BOOT_TIMEOUT_MS = 180_000;
const TEST_TIMEOUT_MS = 60_000;
@@ -171,7 +172,7 @@ const localWorkers = () => env.server.services.localworkerservice;
/** The script the app's published set currently deploys as. */
const scriptName = async (): Promise<string> =>
eventsWorkerScript(
handlerSetHash(await env.server.stores.eventHandler.setForApp(appUid)),
handlerSetHash(await env.server.stores.eventHandler.setForApp(appHandlerScope(appUid))),
// No `workers.internetExposedUrl` in this test's config, so the scope
// this backend binds is empty.
'',
@@ -886,7 +887,7 @@ describe('the worker follows the handler set', () => {
// An empty set names no script, so nothing can be deployed for
// this app again until something is published.
expect(
await env.server.stores.eventHandler.setForApp(appUid),
await env.server.stores.eventHandler.setForApp(appHandlerScope(appUid)),
).toEqual([]);
expect((await durable().getBySubId(subId))?.suspendedReason).toBe(
'handler_not_found',
@@ -23,7 +23,9 @@ import {
eventsInvokeKey,
eventsWorkerScope,
eventsWorkerScript,
userHandlerScript,
} from './workerRuntime.js';
import { handlerSetHash } from './workerSource.js';
const SET_HASH = 'a1'.repeat(32);
const OTHER_HASH = 'b2'.repeat(32);
@@ -50,6 +52,37 @@ describe('eventsWorkerScript', () => {
});
});
describe('userHandlerScript', () => {
const handler = { name: '@abc', sourceHash: SET_HASH };
it('is the script of the handler as a set of one', () => {
expect(userHandlerScript(handler, '')).toBe(
eventsWorkerScript(handlerSetHash([handler]), ''),
);
});
it('is the same for identical code whoever holds it, and differs by code and name', () => {
// Nothing but name and source hash goes in: two holders (or two apps)
// of the same handler deploy one script.
const again = { ...handler };
expect(userHandlerScript(again, 's')).toBe(
userHandlerScript(handler, 's'),
);
expect(
userHandlerScript({ ...handler, sourceHash: OTHER_HASH }, 's'),
).not.toBe(userHandlerScript(handler, 's'));
expect(userHandlerScript({ ...handler, name: '@abd' }, 's')).not.toBe(
userHandlerScript(handler, 's'),
);
});
it('stays scoped to the backend', () => {
expect(userHandlerScript(handler, 'https://a.example')).not.toBe(
userHandlerScript(handler, 'https://b.example'),
);
});
});
describe('eventsWorkerScope', () => {
it('reads the same field `#cfDeploy` binds as `puter_endpoint`, and falls back to empty', () => {
expect(eventsWorkerScope({})).toBe('');
@@ -18,6 +18,8 @@
*/
import { createHash, createHmac } from 'node:crypto';
import type { EventHandler } from '../../stores/events/EventHandlerStore.js';
import { handlerSetHash } from './workerSource.js';
/**
* How an app's events worker is named and how it recognizes the platform.
@@ -71,6 +73,17 @@ export const eventsWorkerScript = (setHash: string, scope: string): string =>
.digest('hex')
.slice(0, 32)}`;
/**
* The script one user-scope handler deploys as: the set-of-one name, built from
* the handler's name and source hash alone. The holder and the app are not part
* of it, so identical code under one name is one script however many users and
* apps publish it.
*/
export const userHandlerScript = (
handler: Pick<EventHandler, 'name' | 'sourceHash'>,
scope: string,
): string => eventsWorkerScript(handlerSetHash([handler]), scope);
/** Version prefix on a derived key, so a rotation is visible in a log line. */
export const EVENTS_INVOKE_KEY_VERSION = 'k1';
+24 -10
View File
@@ -22,6 +22,7 @@ import type {
WorkerInvokeOptions,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import type { Actor } from '../../core/actor.js';
import type { HandlerScopeKind } from '../../stores/events/types.js';
import type { DeliverableEvent, ScheduleEvent } from './registry.js';
export type { ScheduleEvent } from './registry.js';
@@ -42,6 +43,8 @@ export interface WorkerInvocation {
/** The app whose handler runs, or null for an account-owned row. */
appUid: string | null;
handlerName: string | null;
/** Whose handler set `handlerName` is in: absent means the app's own. */
handlerScope?: HandlerScopeKind;
event: DeliverableEvent | ScheduleEvent;
/** The subscription's stored context, delivered to the handler as `ctx`. */
context: string | null;
@@ -58,11 +61,7 @@ export interface WorkerInvocation {
* toward the run that suspends a subscription.
*/
export type WorkerInvocationOutcome =
| 'settled'
| 'terminal'
| 'retriable'
| 'unavailable'
| 'deferred';
'settled' | 'terminal' | 'retriable' | 'unavailable' | 'deferred';
export interface WorkerInvocationResult {
outcome: WorkerInvocationOutcome;
@@ -96,21 +95,25 @@ export interface AddressOptions {
missRead?: 'primary' | 'replica';
/** A handler the set must have; a set without it is re-read on the primary. */
handlerName?: string;
/** Absent means the app's own set. */
handlerScope?: HandlerScopeKind;
/** The user whose scope it is, for `handlerScope: 'user'`. */
holderUserId?: number;
}
/**
* The script an app's handlers deploy as and its invocation key; null when the
* app has no handlers or no events secret is configured.
* The script a handler deploys as and its invocation key; null when it has none
* to run, no events secret is configured, or (in a user scope) the holder is
* gone or suspended. `holder` is the holder's uuid, for a user scope.
*/
export type EventsWorkerAddresser = (
appUid: string,
options?: AddressOptions,
) => Promise<{ script: string; key: string } | null>;
) => Promise<{ script: string; key: string; holder?: string } | null>;
/** What one run came to, including why nothing was invoked. */
export type WorkerRunResult =
| WorkerInvocationResult
| { outcome: MintRefusal | 'deploy-busy' };
WorkerInvocationResult | { outcome: MintRefusal | 'deploy-busy' };
/** How {@link EventsWorkerInvoker.run} reaches the worker. */
export type WorkerRunOptions = WorkerInvokeOptions & AddressOptions;
@@ -131,6 +134,7 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
readonly #mintToken: SubscriberTokenMinter;
readonly #address: EventsWorkerAddresser;
readonly #available: () => boolean;
readonly #userHandlersEnabled: () => boolean;
/**
* Script -> when it was last logged, so a failing script floods once a
* minute.
@@ -142,11 +146,13 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
mintToken: SubscriberTokenMinter,
address: EventsWorkerAddresser,
available: () => boolean,
userHandlersEnabled: () => boolean = () => true,
) {
this.#client = client;
this.#mintToken = mintToken;
this.#address = address;
this.#available = available;
this.#userHandlersEnabled = userHandlersEnabled;
}
available(): boolean {
@@ -178,6 +184,10 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
const { appUid, handlerName } = invocation;
if (!appUid || !handlerName) return { outcome: 'deferred' };
if (!this.#available()) return { outcome: 'unavailable' };
// Owed, not failed: nothing resolves a user scope while it is off.
const { handlerScope } = invocation;
if (handlerScope === 'user' && !this.#userHandlersEnabled())
return { outcome: 'unavailable' };
const minted = await this.#mintToken(invocation);
if ('refused' in minted) return { outcome: minted.refused };
@@ -188,6 +198,9 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
const address = await this.#address(appUid, {
missRead: options.missRead,
handlerName,
...(handlerScope === 'user'
? { handlerScope, holderUserId: invocation.holderUserId }
: {}),
});
if (address === null) return { outcome: 'retriable' };
@@ -196,6 +209,7 @@ export class EventsWorkerInvoker implements WorkerInvokerSeam {
script: address.script,
key: address.key,
appUid,
...(address.holder ? { holder: address.holder } : {}),
handler: handlerName,
token: minted.token,
event: invocation.event,
@@ -175,7 +175,7 @@ let destroys: Array<{ appUid: string; ownerId: number | undefined }>;
beforeAll(async () => {
env = await setupPuterTestEnv({
events: { enabled: true },
events: { enabled: true, userHandlers: true },
unlimitedMetering: true,
} as IConfig);
@@ -204,6 +204,7 @@ afterAll(async () => {
beforeEach(async () => {
await env.server.clients.db.write('DELETE FROM `event_handlers`', []);
await env.server.clients.db.write('DELETE FROM `event_user_handlers`', []);
await env.server.clients.db.write('DELETE FROM `event_subscriptions`', []);
await env.server.clients.db.write('DELETE FROM `apps`', []);
creates.length = 0;
@@ -539,6 +540,8 @@ describe('POST /events/workers/destroy', () => {
const suspended = await env.server.stores.durableSubscription.listByHandler(
app.uid,
'app',
0,
'ingestUpload',
{ suspendedReason: 'handler_not_found' },
);
@@ -554,6 +557,8 @@ describe('POST /events/workers/destroy', () => {
const stillSuspended =
await env.server.stores.durableSubscription.listByHandler(
app.uid,
'app',
0,
'ingestUpload',
{ suspendedReason: 'handler_not_found' },
);
@@ -621,3 +626,319 @@ describe('POST /events/workers/destroy', () => {
expect(destroys).toEqual([]);
});
});
// -- User scopes -----------------------------------------------------
/** `userId`'s own session of `app`, wherever the app's owner is. */
const sessionIn = async (app: { uid: string }, asUser: 'user' | 'other') => {
const { actor } = await env.server.services.auth.authenticate(
asUser === 'user' ? env.users.user.token : env.users.other.token,
);
return env.server.services.auth.getUserAppToken(actor!, app.uid);
};
const publishUserScope = (token: string, name: string, source: string) =>
call('POST', '/events/handlers/publish', token, {
scope: 'user',
name,
source,
});
describe('GET /events/workers with user scopes', () => {
it('adds the caller`s user scopes to their app workers outside an app', async () => {
const mine = await makeApp(userId);
await publish(mine.token, {
appUid: mine.uid,
name: 'a',
source: SOURCE,
});
const theirs = await makeApp(otherUserId);
const inTheirs = await sessionIn(theirs, 'user');
await publishUserScope(inTheirs, 'x', SOURCE);
await publishUserScope(inTheirs, 'y', OTHER_SOURCE);
// Another holder's scope in the same app is not the caller's.
await publishUserScope(await sessionIn(theirs, 'other'), 'z', SOURCE);
const listed = await call(
'GET',
'/events/workers',
env.users.user.token,
);
expect(listed.status).toBe(200);
expect(listed.body.items).toEqual([
expect.objectContaining({
scope: 'app',
appUid: mine.uid,
handlerCount: 1,
}),
expect.objectContaining({
scope: 'user',
appUid: theirs.uid,
appName: theirs.uid,
handlerCount: 2,
}),
]);
expect(listed.body.cursor).toBeUndefined();
});
it('pages through the app workers and then the user scopes', async () => {
const mine = await makeApp(userId);
await publish(mine.token, {
appUid: mine.uid,
name: 'a',
source: SOURCE,
});
const first = await makeApp(otherUserId);
const second = await makeApp(otherUserId);
await publishUserScope(await sessionIn(first, 'user'), 'x', SOURCE);
await publishUserScope(await sessionIn(second, 'user'), 'x', SOURCE);
const seen: Array<Record<string, unknown>> = [];
let cursor: string | undefined;
let pages = 0;
do {
const page = await call(
'GET',
`/events/workers?limit=1${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ''}`,
env.users.user.token,
);
expect(page.status).toBe(200);
seen.push(...(page.body.items as Array<Record<string, unknown>>));
cursor = page.body.cursor as string | undefined;
pages += 1;
} while (cursor && pages < 10);
expect(seen.map((item) => [item.scope, item.appUid])).toEqual([
['app', mine.uid],
...[first.uid, second.uid].sort().map((uid) => ['user', uid]),
]);
});
it('lists only that app`s scope for the caller inside an app', async () => {
const theirs = await makeApp(otherUserId);
const otherApp = await makeApp(otherUserId);
const inTheirs = await sessionIn(theirs, 'user');
await publishUserScope(inTheirs, 'x', SOURCE);
await publishUserScope(await sessionIn(otherApp, 'user'), 'x', SOURCE);
await publishUserScope(await sessionIn(theirs, 'other'), 'x', SOURCE);
// The owner's own worker in that app is not the caller's to see.
await publish(theirs.token, {
appUid: theirs.uid,
name: 'owner',
source: SOURCE,
});
const listed = await call('GET', '/events/workers', inTheirs);
expect(listed.status).toBe(200);
expect(listed.body.items).toEqual([
expect.objectContaining({
scope: 'user',
appUid: theirs.uid,
handlerCount: 1,
}),
]);
});
it('still lists the owner`s worker beside their own user scope inside the app', async () => {
const mine = await makeApp(userId);
await publish(mine.token, {
appUid: mine.uid,
name: 'a',
source: SOURCE,
});
await publishUserScope(mine.token, 'x', SOURCE);
const listed = await call('GET', '/events/workers', mine.token);
expect(
(listed.body.items as Array<Record<string, unknown>>).map(
(item) => item.scope,
),
).toEqual(['app', 'user']);
});
it('refuses a scoped token, which has no scope of its own', async () => {
const theirs = await makeApp(otherUserId);
await publishUserScope(await sessionIn(theirs, 'user'), 'x', SOURCE);
const scoped = await appIssuedScopedToken({
uid: theirs.uid,
token: await sessionIn(theirs, 'user'),
});
const listed = await call('GET', '/events/workers', scoped);
expect(listed.status).toBe(403);
expect(listed.body.code).toBe('events_worker_owner_only');
});
});
describe('POST /events/workers/destroy with a user scope', () => {
it('removes the caller`s scope only, suspends what was bound, and announces it', async () => {
const theirs = await makeApp(otherUserId);
const mineIn = await sessionIn(theirs, 'user');
const otherIn = await sessionIn(theirs, 'other');
await publish(theirs.token, {
appUid: theirs.uid,
name: 'owner',
source: SOURCE,
});
await publishUserScope(mineIn, 'x', SOURCE);
await publishUserScope(mineIn, 'y', OTHER_SOURCE);
await publishUserScope(otherIn, 'x', SOURCE);
destroys.length = 0;
const destroyed = await call(
'POST',
'/events/workers/destroy',
mineIn,
{
scope: 'user',
},
);
expect(destroyed.status).toBe(200);
expect(destroyed.body).toMatchObject({
appUid: theirs.uid,
removed: 2,
suspended: 0,
scope: 'user',
});
const store = env.server.stores.eventHandler;
expect(
await store.countForApp({ appUid: theirs.uid, holder: userId }),
).toBe(0);
expect(
await store.countForApp({
appUid: theirs.uid,
holder: otherUserId,
}),
).toBe(1);
expect(await store.countForApp({ appUid: theirs.uid, holder: 0 })).toBe(
1,
);
expect(destroys).toEqual([{ appUid: theirs.uid, ownerId: userId }]);
const again = await call('POST', '/events/workers/destroy', mineIn, {
scope: 'user',
});
expect(again.status).toBe(404);
});
it('refuses a scoped token and an app session naming another app', async () => {
const theirs = await makeApp(otherUserId);
const mineIn = await sessionIn(theirs, 'user');
await publishUserScope(mineIn, 'x', SOURCE);
const scoped = await appIssuedScopedToken({
uid: theirs.uid,
token: mineIn,
});
const elsewhere = await makeApp(otherUserId);
for (const [token, body] of [
[scoped, { scope: 'user' }],
[mineIn, { scope: 'user', appUid: elsewhere.uid }],
] as const) {
const refused = await call(
'POST',
'/events/workers/destroy',
token,
body,
);
expect(refused.status).toBe(403);
expect(refused.body.code).toBe('events_handler_forbidden');
}
expect(
await env.server.stores.eventHandler.countForApp({
appUid: theirs.uid,
holder: userId,
}),
).toBe(1);
});
});
describe('destroying a user scope from an account session', () => {
it('removes the caller`s own scope of any app, and nobody else`s', async () => {
const theirs = await makeApp(otherUserId);
await publishUserScope(await sessionIn(theirs, 'user'), 'x', SOURCE);
await publishUserScope(await sessionIn(theirs, 'other'), 'x', SOURCE);
const store = env.server.stores.eventHandler;
destroys.length = 0;
const destroyed = await call(
'POST',
'/events/workers/destroy',
env.users.user.token,
{ scope: 'user', appUid: theirs.uid },
);
expect(destroyed.status).toBe(200);
expect(destroyed.body).toMatchObject({ removed: 1, scope: 'user' });
expect(
await store.countForApp({ appUid: theirs.uid, holder: userId }),
).toBe(0);
expect(
await store.countForApp({ appUid: theirs.uid, holder: otherUserId }),
).toBe(1);
expect(destroys).toEqual([{ appUid: theirs.uid, ownerId: userId }]);
// The same call as the other user removes only theirs; the first user
// has no scope left to touch.
const none = await call(
'POST',
'/events/workers/destroy',
env.users.user.token,
{ scope: 'user', appUid: theirs.uid },
);
expect(none.status).toBe(404);
expect(
await store.countForApp({ appUid: theirs.uid, holder: otherUserId }),
).toBe(1);
});
it('needs an appUid, and still refuses a scoped token', async () => {
const theirs = await makeApp(otherUserId);
const mineIn = await sessionIn(theirs, 'user');
await publishUserScope(mineIn, 'x', SOURCE);
const noApp = await call(
'POST',
'/events/workers/destroy',
env.users.user.token,
{ scope: 'user' },
);
expect(noApp.status).toBe(403);
const scoped = await appIssuedScopedToken({
uid: theirs.uid,
token: mineIn,
});
const refused = await call('POST', '/events/workers/destroy', scoped, {
scope: 'user',
appUid: theirs.uid,
});
expect(refused.status).toBe(403);
});
it('works with user handlers switched off, so a user can stop paying', async () => {
const theirs = await makeApp(otherUserId);
await publishUserScope(await sessionIn(theirs, 'user'), 'x', SOURCE);
const config = (
env.server.services.events as unknown as {
config: { events: { userHandlers?: boolean } };
}
).config;
config.events.userHandlers = false;
try {
const destroyed = await call(
'POST',
'/events/workers/destroy',
env.users.user.token,
{ scope: 'user', appUid: theirs.uid },
);
expect(destroyed.status).toBe(200);
} finally {
config.events.userHandlers = true;
}
});
});
@@ -70,6 +70,12 @@ const EXPECTED_JOBS = {
nearPrimary: true,
leaseMs: 15 * MINUTE,
},
'events.userHandlers.gc': {
every: HOUR,
scope: 'fleet',
nearPrimary: true,
leaseMs: 15 * MINUTE,
},
'events.subscriptions.creditSweep': {
every: 15 * MINUTE,
scope: 'fleet',
@@ -243,4 +243,61 @@ describe('appMessageSender', () => {
'a'.repeat(120),
);
});
describe('an app made for an origin', () => {
const uid = 'app-65d98228-cfa5-5d18-a442-f6eeaf3dcfc7';
const originApp = (fields: Record<string, unknown> = {}) => ({
uid,
name: uid,
title: uid,
description: 'App created from origin https://chat.example.com',
index_url: 'https://chat.example.com',
...fields,
});
it('is named by the hostname of its index_url, without a port', () => {
expect(appMessageSender(uid, originApp()).name).toBe(
'chat.example.com',
);
expect(
appMessageSender(
uid,
originApp({
index_url: 'https://chat.example.com:8443/app/',
}),
).name,
).toBe('chat.example.com');
});
it('keeps its stored name when its index_url does not parse', () => {
expect(
appMessageSender(uid, originApp({ index_url: 'not a url' }))
.name,
).toBe(uid);
expect(
appMessageSender(uid, originApp({ index_url: undefined })).name,
).toBe(uid);
});
it('keeps a title its developer set', () => {
expect(
appMessageSender(uid, originApp({ title: 'Chat' })).name,
).toBe('Chat');
});
it('leaves apps that were not made for an origin alone', () => {
expect(
appMessageSender(uid, originApp({ description: 'My chat app' }))
.name,
).toBe(uid);
expect(
appMessageSender('app-1', {
uid: 'app-1',
name: 'chat',
title: 'Chat',
index_url: 'https://chat.example.com',
}).name,
).toBe('Chat');
});
});
});
@@ -18,6 +18,7 @@
*/
import { HttpError } from '../../core/http/HttpError.js';
import { isOriginBootstrapApp } from '../../util/appView.js';
/** The one notification type an app may create. */
export const APP_MESSAGE_TYPE = 'app.message';
@@ -60,14 +61,35 @@ export type AppMessage = {
export type AppMessageSender = { uid: string; name?: string; icon?: string };
/** The hostname an app made for an origin is named by. */
const originAppHost = (row: unknown): string | undefined => {
if (!isOriginBootstrapApp(row) || typeof row.index_url !== 'string') {
return undefined;
}
try {
return new URL(row.index_url).hostname || undefined;
} catch {
return undefined;
}
};
/** What a message says about its app, read off the app's row. */
export const appMessageSender = (
uid: string,
row: { name?: unknown; title?: unknown; icon?: unknown } | null | undefined,
row:
| {
name?: unknown;
title?: unknown;
icon?: unknown;
description?: unknown;
index_url?: unknown;
}
| null
| undefined,
): AppMessageSender => {
const sender: AppMessageSender = { uid };
// Cleaned before the empty check, so a title of only invisible characters still leaves a visible sender.
const name = [row?.title, row?.name]
const name = [originAppHost(row), row?.title, row?.name]
.map((v) =>
typeof v === 'string'
? [...cleanText(v)]
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { describe, expect, it } from 'vitest';
@@ -53,6 +54,9 @@ describe('NOTIFICATION_TYPES', () => {
audience: 'developer',
appScoped: true,
});
expect(
findNotificationType('app.events.handler_suspended'),
).toMatchObject({ audience: 'app-user', appScoped: true });
expect(findNotificationType('app.events.ended')).toMatchObject({
audience: 'app-user',
appScoped: false,
@@ -61,9 +65,9 @@ describe('NOTIFICATION_TYPES', () => {
it('projects a subject naming the app, or the recipient when there is none', () => {
const account = findNotificationType('share.received')!;
expect(
account.subject({ userUuid: 'user-uuid', appUid: null }),
).toBe('notif:user-uuid:account');
expect(account.subject({ userUuid: 'user-uuid', appUid: null })).toBe(
'notif:user-uuid:account',
);
const developer = findNotificationType('app.events.suspended')!;
expect(
@@ -130,18 +134,20 @@ describe('resolveNotificationWrite', () => {
});
it('requires an app uid for an app message', () => {
expect(resolveNotificationWrite('app.message', 'app-1')).toMatchObject(
{ audience: 'app-user', appScoped: true, groupable: true },
);
expect(resolveNotificationWrite('app.message', 'app-1')).toMatchObject({
audience: 'app-user',
appScoped: true,
groupable: true,
});
expect(() => resolveNotificationWrite('app.message', null)).toThrow(
'requires an app uid',
);
});
it('lets a subscription an account made end without naming an app', () => {
expect(resolveNotificationWrite('app.events.ended', null).audience).toBe(
'app-user',
);
expect(
resolveNotificationWrite('app.events.ended', null).audience,
).toBe('app-user');
});
});
@@ -115,6 +115,15 @@ export const NOTIFICATION_TYPES = [
groupable: false,
subject: subjectFor('developer'),
},
// A handler the user published into their own scope stopped taking
// deliveries: the code is theirs, so the notice is too.
{
type: 'app.events.handler_suspended',
audience: 'app-user',
appScoped: true,
groupable: false,
subject: subjectFor('app-user'),
},
// The only type apps create; its kind goes in the value's `channel`. A
// newer one replaces the newest with the same `group` unless that one was
// shown or dismissed.
+1 -1
View File
@@ -801,7 +801,7 @@ export class AppStore extends PuterStore {
* the row stays unclaimed until a developer registers an app for the same
* origin and AppDriver's merge path takes ownership. Marker shape (`name
* === uid && title === uid` + description prefix) is what
* `#isOriginBootstrapApp` checks.
* `isOriginBootstrapApp` (util/appView) checks.
*/
async createFromOrigin(
uid,
@@ -265,6 +265,114 @@ describe('creating a subscription', () => {
});
});
describe('handler scope', () => {
const appUid = 'app-scoped';
it('round-trips through create and every select path', async () => {
const { row: user } = await durable().create(
input({ appUid, handlerName: 'h', handlerScope: 'user' }),
);
const { row: app } = await durable().create(
input({ appUid, handlerName: 'h' }),
);
expect(user.handlerScope).toBe('user');
expect(app).not.toHaveProperty('handlerScope');
await expect(durable().getBySubId(user.subId)).resolves.toMatchObject({
handlerScope: 'user',
});
await expect(
durable().getBySubId(app.subId),
).resolves.not.toHaveProperty('handlerScope');
const listed = await durable().listForHolder(userId, { appUid });
expect(
new Map(listed.items.map((r) => [r.subId, r.handlerScope])),
).toEqual(
new Map<string, string | undefined>([
[user.subId, 'user'],
[app.subId, undefined],
]),
);
const stored = await env.server.clients.db.read(
'SELECT `sub_id`, `handler_scope` FROM `event_subscriptions` WHERE `app_uid` = ?',
[appUid],
);
expect(
Object.fromEntries(stored.map((r) => [r.sub_id, r.handler_scope])),
).toEqual({ [user.subId]: 1, [app.subId]: 0 });
});
it('lists by scope and holder, never across scopes', async () => {
const { row: appRow } = await durable().create(
input({ appUid, handlerName: 'h' }),
);
const { row: mine } = await durable().create(
input({ appUid, handlerName: 'h', handlerScope: 'user' }),
);
const { row: theirs } = await durable().create(
input({
appUid,
handlerName: 'h',
handlerScope: 'user',
holderUserId: otherUserId,
ownerUserId: otherUserId,
token: `f#other-${uuidv4()}`,
}),
);
const ids = async (scope: 'app' | 'user', holder: number, name = 'h') =>
(await durable().listByHandler(appUid, scope, holder, name))
.map((r) => r.subId)
.sort();
expect(await ids('app', 0)).toEqual([appRow.subId]);
// Holder is ignored for app scope.
expect(await ids('app', userId)).toEqual([appRow.subId]);
expect(await ids('user', userId)).toEqual([mine.subId]);
expect(await ids('user', otherUserId)).toEqual([theirs.subId]);
expect(await ids('user', userId, 'missing')).toEqual([]);
});
it('lists the suspended half of a user scope', async () => {
const { row } = await durable().create(
input({ appUid, handlerName: 'h', handlerScope: 'user' }),
);
await env.server.clients.db.write(
'UPDATE `event_subscriptions` SET `suspended_at` = 1, `suspended_reason` = ? WHERE `sub_id` = ?',
['handler_not_found', row.subId],
);
await expect(
durable().listByHandler(appUid, 'user', userId, 'h'),
).resolves.toEqual([]);
await expect(
durable().listByHandler(appUid, 'user', userId, 'h', {
suspendedReason: 'handler_not_found',
}),
).resolves.toEqual([expect.objectContaining({ subId: row.subId })]);
});
it('reads a user scope through the holder index', async () => {
const pread = vi.spyOn(env.server.clients.db, 'pread');
try {
await durable().listByHandler(appUid, 'user', userId, 'h');
const [statement, params] = pread.mock.calls[0]!;
const plan = await env.server.clients.db.read(
`EXPLAIN QUERY PLAN ${statement}`,
params,
);
const detail = plan.map((r) => String(r.detail)).join('\n');
expect(detail).toContain('idx_event_subscriptions_holder');
expect(detail).not.toContain('idx_event_subscriptions_app_handler');
} finally {
pread.mockRestore();
}
});
it('carries the scope through the schedule join', async () => {
const join = durable().scheduleSubscriptionJoin('s', 'x.`sub_id`');
expect(join.columns).toContain('`sub_handler_scope`');
});
});
describe('validation at the row write', () => {
it('refuses a context past the hard cap', async () => {
await expect(
@@ -1099,7 +1207,7 @@ describe('rows a schedule owns', () => {
it('are found by handler and by app, like any other row', async () => {
const { row } = await durable().create(scheduleInput());
await expect(
durable().listByHandler('app-scheduler', 'tick'),
durable().listByHandler('app-scheduler', 'app', 0, 'tick'),
).resolves.toEqual([expect.objectContaining({ subId: row.subId })]);
await expect(
durable().listHolderIdsForApp('app-scheduler'),
@@ -44,6 +44,7 @@ import {
SCHEDULE_TOKEN_PREFIX,
scheduleToken,
type DurableSubscription,
type HandlerScopeKind,
type SubscriptionPermission,
} from './types.js';
@@ -100,6 +101,8 @@ export interface DurableSubscriptionInput {
op: FsOp | null;
delivery: DeliveryClass;
handlerName: string | null;
/** Which set `handlerName` refers to. Omitted means the app's. */
handlerScope?: HandlerScopeKind;
context: string | null;
permission: SubscriptionPermission;
expiresAt: number | null;
@@ -307,6 +310,9 @@ const toRow = (row: Record<string, unknown>): DurableSubscription => ({
row.handler_name === null || row.handler_name === undefined
? null
: String(row.handler_name),
...(Number(row.handler_scope) === 1
? { handlerScope: 'user' as const }
: {}),
context:
row.context === null || row.context === undefined
? null
@@ -324,7 +330,7 @@ const toRow = (row: Record<string, unknown>): DurableSubscription => ({
const SELECT_COLUMNS =
'`id`, `sub_id`, `token`, `owner_user_id`, `holder_user_id`, `app_uid`, ' +
'`subject`, `anchor_uid`, `anchor_path`, `match`, `delivery`, `ops`, ' +
'`handler_name`, `context`, `permission`, `include_value`, ' +
'`handler_name`, `handler_scope`, `context`, `permission`, `include_value`, ' +
'`suspended_at`, `suspended_reason`, `expires_at`, `created_at`';
export class DurableSubscriptionStore extends PuterStore {
@@ -376,6 +382,9 @@ export class DurableSubscriptionStore extends PuterStore {
permission: input.permission,
delivery: input.delivery,
handlerName: input.handlerName,
...(input.handlerScope === 'user'
? { handlerScope: 'user' as const }
: {}),
context: input.context,
...(input.includeValue ? { includeValue: true } : {}),
expiresAt: input.expiresAt,
@@ -398,6 +407,7 @@ export class DurableSubscriptionStore extends PuterStore {
delivery: row.delivery,
ops: row.op,
handler_name: row.handlerName,
handler_scope: row.handlerScope === 'user' ? 1 : 0,
targets: legacyTargets(row.appUid),
context: row.context,
permission: row.permission,
@@ -811,8 +821,11 @@ export class DurableSubscriptionStore extends PuterStore {
}
/**
* The rows bound to one of an app's handler names. What a removal has to
* The rows bound to one of a scope's handler names. What a removal has to
* suspend, and — asking for the suspended half — what a republish resumes.
* `holder` is read for user scope only. App scope never returns user-scope
* rows; user scope is read through the holder index, so it never walks the
* other holders of a popular app.
*
* Bounded by the per-account quota times nothing: a widely-used handler can
* carry more rows than one read should return, so this is a page and the
@@ -820,11 +833,28 @@ export class DurableSubscriptionStore extends PuterStore {
*/
async listByHandler(
appUid: string,
scope: HandlerScopeKind,
holder: number,
handlerName: string,
options: { suspendedReason?: SuspendedReason; limit?: number } = {},
): Promise<DurableSubscription[]> {
const where = ['`app_uid` = ?', '`handler_name` = ?'];
const params: unknown[] = [appUid, handlerName];
const where =
scope === 'user'
? [
'`holder_user_id` = ?',
'`app_uid` = ?',
'`handler_scope` = 1',
'`handler_name` = ?',
]
: [
'`app_uid` = ?',
'`handler_scope` = 0',
'`handler_name` = ?',
];
const params: unknown[] =
scope === 'user'
? [holder, appUid, handlerName]
: [appUid, handlerName];
if (options.suspendedReason === undefined) {
where.push('`suspended_at` IS NULL');
} else {
@@ -965,7 +995,8 @@ export class DurableSubscriptionStore extends PuterStore {
/**
* A `LEFT JOIN` of the row a schedule owns, as `alias`, on `subIdColumn`,
* and the columns it selects: `sub_row_id`, `sub_suspended_at`,
* `sub_suspended_reason`, `sub_handler_name` and `sub_context`.
* `sub_suspended_reason`, `sub_handler_name`, `sub_handler_scope` and
* `sub_context`.
*/
scheduleSubscriptionJoin(
alias: string,
@@ -977,6 +1008,7 @@ export class DurableSubscriptionStore extends PuterStore {
`${alias}.\`suspended_at\` AS \`sub_suspended_at\`, ` +
`${alias}.\`suspended_reason\` AS \`sub_suspended_reason\`, ` +
`${alias}.\`handler_name\` AS \`sub_handler_name\`, ` +
`${alias}.\`handler_scope\` AS \`sub_handler_scope\`, ` +
`${alias}.\`context\` AS \`sub_context\``,
join: `LEFT JOIN \`${TABLE}\` ${alias} ON ${alias}.\`sub_id\` = ${subIdColumn}`,
};
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,392 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* The user-scope handler schema and queries against MySQL 8.4. Runs only when
* `PUTER_TEST_MYSQL_PORT` points at a server (root / `puter`).
*/
import { readFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import mysql from 'mysql2/promise';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { splitMysqlStatements } from '../../clients/database/splitMysqlStatements.js';
import type { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { appHandlerScope, hashContent } from './EventHandlerStore.js';
const connection = {
host: '127.0.0.1',
port: Number(process.env.PUTER_TEST_MYSQL_PORT ?? 3307),
user: 'root',
password: 'puter',
};
const database = `event_user_handlers_${Date.now().toString(36)}`;
const migrationDir = fileURLToPath(
new URL('../../clients/database/migrations/mysql', import.meta.url),
);
const SOURCE = 'async ({ event }) => { console.log(event.path); }';
describe.skipIf(!process.env.PUTER_TEST_MYSQL_PORT)(
'user-scoped handlers (MySQL)',
() => {
let server: PuterServer;
let conn: mysql.Connection;
const userIds: number[] = [];
const handlers = () => server.stores.eventHandler;
const rows = async (sql: string, params: unknown[] = []) =>
(await conn.query(sql, params))[0] as Record<string, unknown>[];
const applyMig55 = async () => {
const sql = readFileSync(
`${migrationDir}/mysql_mig_55.sql`,
'utf8',
);
for (const statement of splitMysqlStatements(sql))
await conn.query(statement);
};
const bind = async (
holder: number,
app: string,
name: string,
handlerScope: number,
suspended = false,
) => {
await conn.query(
'INSERT INTO `event_subscriptions` (`sub_id`, `token`, `owner_user_id`, `holder_user_id`, `app_uid`, `subject`, `anchor_uid`, `anchor_path`, `delivery`, `handler_name`, `handler_scope`, `targets`, `permission`, `suspended_at`, `suspended_reason`, `created_at`) ' +
"VALUES (?, ?, ?, ?, ?, 'fs:/x', 'a', '/x', 'single', ?, ?, '[]', 'list', ?, ?, 1)",
[
uuidv4(),
`f#${uuidv4()}`,
holder,
holder,
app,
name,
handlerScope,
suspended ? 1 : null,
suspended ? 'handler_not_found' : null,
],
);
};
/**
* The `key` column of every table in the plan of the last statement a
* spy saw.
*/
const planKeys = async (spy: {
mock: { calls: unknown[][] };
}): Promise<Record<string, string | null>> => {
const [statement, params] = spy.mock.calls.at(-1) as [
string,
unknown[],
];
const plan = await rows(`EXPLAIN ${statement}`, params);
return Object.fromEntries(
plan.map((row) => [
String(row.table),
(row.key as string | null) ?? null,
]),
);
};
beforeAll(async () => {
const admin = await mysql.createConnection(connection);
await admin.query(`CREATE DATABASE \`${database}\``);
await admin.end();
server = await setupTestServer({
database: {
engine: 'mysql',
...connection,
database,
migrationPaths: [migrationDir],
},
events: { enabled: true },
} as never);
conn = await mysql.createConnection({ ...connection, database });
for (let i = 0; i < 12; i++) {
const user = await server.stores.user.create({
username: `ush-${uuidv4().slice(0, 8)}`,
uuid: uuidv4(),
password: null,
email: null,
});
userIds.push(user.id);
}
}, 240_000);
afterAll(async () => {
await conn?.end();
await server?.shutdown();
const admin = await mysql.createConnection(connection);
await admin.query(`DROP DATABASE IF EXISTS \`${database}\``);
await admin.end();
});
it('runs on MySQL 8.4', async () => {
const [row] = await rows('SELECT VERSION() AS version');
expect(String(row!.version)).toMatch(/^8\.4\./);
});
it('applies over existing subscription rows, and a replay is a no-op', async () => {
// Back to the state before the migration, with rows already in.
await conn.query('DROP TABLE `event_user_handlers`');
await conn.query(
'ALTER TABLE `event_subscriptions` DROP COLUMN `handler_scope`',
);
await conn.query('DELETE FROM `event_subscriptions`');
for (const name of ['a', 'b', 'c'])
await conn.query(
'INSERT INTO `event_subscriptions` (`sub_id`, `token`, `owner_user_id`, `holder_user_id`, `app_uid`, `subject`, `anchor_uid`, `anchor_path`, `delivery`, `handler_name`, `targets`, `permission`, `created_at`) ' +
"VALUES (?, ?, ?, ?, 'app-old', 'fs:/x', 'a', '/x', 'single', ?, '[]', 'list', 1)",
[uuidv4(), `f#${uuidv4()}`, userIds[0], userIds[0], name],
);
await applyMig55();
const scopes = async () =>
(
await rows(
'SELECT `handler_scope` FROM `event_subscriptions`',
)
).map((row) => Number(row.handler_scope));
expect(await scopes()).toEqual([0, 0, 0]);
await applyMig55();
expect(await scopes()).toEqual([0, 0, 0]);
const [column] = await rows(
"SELECT COLUMN_TYPE, IS_NULLABLE, COLUMN_DEFAULT FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'event_subscriptions' AND COLUMN_NAME = 'handler_scope'",
);
expect(column).toMatchObject({
COLUMN_TYPE: 'tinyint unsigned',
IS_NULLABLE: 'NO',
COLUMN_DEFAULT: '0',
});
const indexes = await rows(
"SELECT INDEX_NAME, NON_UNIQUE, SEQ_IN_INDEX, COLUMN_NAME FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'event_user_handlers' ORDER BY INDEX_NAME, SEQ_IN_INDEX",
);
const byIndex = (name: string) =>
indexes
.filter((row) => row.INDEX_NAME === name)
.map((row) => String(row.COLUMN_NAME));
expect(byIndex('idx_event_user_handlers_app_holder_name')).toEqual([
'app_uid',
'holder_user_id',
'name',
]);
expect(byIndex('idx_event_user_handlers_holder_app')).toEqual([
'holder_user_id',
'app_uid',
]);
expect(
indexes.find(
(row) =>
row.INDEX_NAME ===
'idx_event_user_handlers_app_holder_name',
),
).toMatchObject({ NON_UNIQUE: 0 });
const [appUidColumn] = await rows(
"SELECT CHARACTER_SET_NAME, COLLATION_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'event_user_handlers' AND COLUMN_NAME = 'app_uid'",
);
expect(appUidColumn).toMatchObject({
CHARACTER_SET_NAME: 'latin1',
COLLATION_NAME: 'latin1_swedish_ci',
});
});
it('keeps the same name once in the app scope and once per holder', async () => {
const app = `app-${uuidv4()}`;
const insert = (holder: number) =>
conn.query(
'INSERT INTO `event_user_handlers` (`app_uid`, `holder_user_id`, `name`, `source`, `source_hash`, `created_at`, `updated_at`, `scope_since`) VALUES (?, ?, ?, ?, ?, 1, 1, 1)',
[app, holder, 'daily', SOURCE, hashContent(SOURCE)],
);
await handlers().publish({
scope: appHandlerScope(app),
name: 'daily',
source: SOURCE,
});
await insert(userIds[1]!);
await insert(userIds[2]!);
await expect(insert(userIds[1]!)).rejects.toMatchObject({
code: 'ER_DUP_ENTRY',
});
});
it('runs the store against both tables', async () => {
const app = `app-${uuidv4()}`;
const [a, b] = [userIds[3]!, userIds[4]!];
const scope = (holder: number) => ({ appUid: app, holder });
await handlers().publish({
scope: appHandlerScope(app),
name: 'daily',
source: SOURCE,
});
for (const holder of [a, b])
await handlers().publish({
scope: scope(holder),
name: 'daily',
source: `${SOURCE} // ${holder}`,
});
await handlers().publish({
scope: scope(a),
name: `@${hashContent(SOURCE)}`,
source: SOURCE,
});
expect(await handlers().countForApp(scope(a))).toBe(2);
expect(await handlers().countForApp(appHandlerScope(app))).toBe(1);
expect(await handlers().totalSourceBytesForApp(scope(b))).toBe(
Buffer.byteLength(`${SOURCE} // ${b}`),
);
expect(
(await handlers().setForApp(scope(a), { replica: true })).map(
(h) => h.name,
),
).toEqual([`@${hashContent(SOURCE)}`, 'daily']);
expect(await handlers().countUserScopesWithHandlersForUser(a)).toBe(
1,
);
expect(
await handlers().countUserScopesWithHandlersForUser(a, {
createdBefore: 1_000,
}),
).toBe(0);
expect(
await handlers().countUserScopesWithHandlersForUser(a, {
createdBefore: Date.now() + 60_000,
}),
).toBe(1);
const page = await handlers().listUserScopesForUser(a);
expect(page.total).toBe(1);
expect(page.items).toEqual([
expect.objectContaining({ appUid: app, handlerCount: 2 }),
]);
expect(
(await handlers().listUserScopes(app)).items.map(
(s) => s.holder,
),
).toEqual([a, b].sort((x, y) => x - y));
await bind(a, app, 'daily', 1);
await bind(b, app, 'daily', 1, true);
await bind(a, app, 'daily', 0);
expect(
await handlers().countSubscriptionsByHandler(scope(a)),
).toEqual(new Map([['daily', 1]]));
expect(
await handlers().countSubscriptionsByHandler(
appHandlerScope(app),
),
).toEqual(new Map([['daily', 1]]));
await conn.query(
'UPDATE `event_user_handlers` SET `updated_at` = 1 WHERE `app_uid` = ?',
[app],
);
const found = await handlers().listUnboundUserHandlers(1_000);
expect(
found.items
.filter((item) => item.appUid === app)
.map((item) => [item.holder, item.name]),
).toEqual([[a, `@${hashContent(SOURCE)}`]]);
await expect(
handlers().deleteUserHandlerIfUnbound(scope(a), 'daily', 1_000),
).resolves.toBe(false);
await expect(
handlers().deleteUserHandlerIfUnbound(
scope(a),
`@${hashContent(SOURCE)}`,
1_000,
),
).resolves.toBe(true);
await expect(handlers().deleteUserScopesForHolder(b)).resolves.toBe(
1,
);
await expect(handlers().deleteForApp(app)).resolves.toBe(2);
expect(await handlers().countForApp(scope(a))).toBe(0);
});
it('serves the user-scope reads from the holder-leading indexes', async () => {
const app = `app-${uuidv4()}`;
// A popular handler: many holders on one (app, name), so the
// (app_uid, handler_name) index is the wrong one to pick.
for (const holder of userIds)
for (let i = 0; i < 80; i++)
await bind(holder, app, 'popular', 1);
await handlers().publish({
scope: { appUid: app, holder: userIds[0]! },
name: 'popular',
source: SOURCE,
});
await conn.query('ANALYZE TABLE `event_subscriptions`');
await conn.query('ANALYZE TABLE `event_user_handlers`');
const pread = vi.spyOn(server.clients.db, 'pread');
const read = vi.spyOn(server.clients.db, 'read');
try {
await server.stores.durableSubscription.listByHandler(
app,
'user',
userIds[0]!,
'popular',
);
expect(await planKeys(pread)).toEqual({
event_subscriptions: 'idx_event_subscriptions_holder',
});
await handlers().countSubscriptionsByHandler({
appUid: app,
holder: userIds[0]!,
});
expect(await planKeys(read)).toEqual({
event_subscriptions: 'idx_event_subscriptions_holder',
});
await handlers().countUserScopesWithHandlersForUser(
userIds[0]!,
{
createdBefore: Date.now(),
},
);
expect((await planKeys(pread)).event_user_handlers).toBe(
'idx_event_user_handlers_holder_app',
);
await handlers().listUserScopesForUser(userIds[0]!);
expect((await planKeys(read)).event_user_handlers).toBe(
'idx_event_user_handlers_holder_app',
);
await handlers().listUnboundUserHandlers(1_000);
expect((await planKeys(read)).s).toBe(
'idx_event_subscriptions_holder',
);
} finally {
pread.mockRestore();
read.mockRestore();
}
}, 120_000);
},
);
+562 -86
View File
@@ -21,6 +21,7 @@ import { createHash } from 'node:crypto';
import {
EVENTS_HANDLERS_PER_APP,
EVENTS_HANDLER_SOURCE_MAX_BYTES,
EVENTS_USER_HANDLERS_PER_SCOPE,
} from '../../controllers/events/limits.js';
import { HttpError } from '../../core/http/HttpError.js';
import {
@@ -47,6 +48,7 @@ import { PuterStore } from '../types.js';
*/
const TABLE = 'event_handlers';
const USER_TABLE = 'event_user_handlers';
const SUBSCRIPTION_TABLE = 'event_subscriptions';
const APP_TABLE = 'apps';
@@ -57,12 +59,24 @@ const APP_TABLE = 'apps';
export const EVENTS_WORKERS_LIST_DEFAULT_LIMIT = 50;
export const EVENTS_WORKERS_LIST_LIMIT_CAP = 200;
/** Default page of `listUserScopes`, and of one GC read. */
const USER_SCOPES_PAGE_DEFAULT = 500;
const USER_SCOPES_PAGE_CAP = 1000;
const UNBOUND_HANDLERS_BATCH = 200;
/** Longest a handler name may be, matching the column that holds it. */
export const HANDLER_NAME_MAX_LENGTH = 128;
/** Names are addressable identifiers, not free text. */
const NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.:-]*$/;
/**
* A server-minted name for an inline handler: `@` and the source hash. `@` is
* outside {@link NAME_PATTERN}, so a minted name never collides with a chosen
* one, and only a user scope accepts it.
*/
const INLINE_NAME_PATTERN = /^@[0-9a-f]{64}$/;
/**
* Longest a delivery may address an app's handler set from cache. Every write
* here drops the entry on every node, so this only bounds a read that raced a
@@ -70,15 +84,66 @@ const NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.:-]*$/;
*/
export const HANDLER_SET_CACHE_TTL_SECONDS = 30;
const handlerSetCacheKey = (appUid: string): string =>
`eventHandlers:set:${appUid}`;
// -- Scopes -----------------------------------------------------------
/**
* Whose handlers a call is about: an app's own set (`holder` 0), or the
* handlers one user published into their scope of that app. The two live in
* separate tables, so code that predates user scopes never reads one.
*/
export interface HandlerScope {
appUid: string;
/** `users.id` of the scope's holder, or 0 for the app's own set. */
holder: number;
}
/** The app's own scope. */
export const appHandlerScope = (appUid: string): HandlerScope => ({
appUid,
holder: 0,
});
const isUserScope = (scope: HandlerScope): boolean => scope.holder !== 0;
const assertScope = (scope: HandlerScope): void => {
if (!Number.isInteger(scope.holder) || scope.holder < 0)
throw new TypeError('handler scope holder must be a user id or 0');
};
const tableOf = (scope: HandlerScope): string =>
isUserScope(scope) ? USER_TABLE : TABLE;
const handlerCapOf = (scope: HandlerScope): number =>
isUserScope(scope)
? EVENTS_USER_HANDLERS_PER_SCOPE
: EVENTS_HANDLERS_PER_APP;
/** The WHERE clause for a scope's rows and its parameters. */
const scopeFilter = (
scope: HandlerScope,
): { where: string; params: unknown[] } => {
assertScope(scope);
return isUserScope(scope)
? {
where: '`app_uid` = ? AND `holder_user_id` = ?',
params: [scope.appUid, scope.holder],
}
: { where: '`app_uid` = ?', params: [scope.appUid] };
};
// App scope keeps the key it always had, so nodes on either side of a rolling
// deploy share it.
const handlerSetCacheKey = (scope: HandlerScope): string =>
isUserScope(scope)
? `eventHandlers:uset:${scope.appUid}:${scope.holder}`
: `eventHandlers:set:${scope.appUid}`;
/**
* The same set as read from the replica, kept apart so a lagging read can never
* stand in for the primary's answer on the delivery path.
*/
const replicaSetCacheKey = (appUid: string): string =>
`${handlerSetCacheKey(appUid)}:r`;
const replicaSetCacheKey = (scope: HandlerScope): string =>
`${handlerSetCacheKey(scope)}:r`;
// -- Shapes -----------------------------------------------------------
@@ -126,8 +191,45 @@ export interface ListEventsWorkersOptions {
appUid?: string;
}
export interface PublishHandlerInput {
/** One user scope: the handlers a user keeps in one app. */
export interface UserScopeSummary {
appUid: string;
/** Empty when the app row is gone. */
appName: string;
appTitle: string;
handlerCount: number;
/** Earliest handler `created_at` in the scope. */
createdAt: number;
/** Latest handler `updated_at` in the scope. */
updatedAt: number;
}
/** A user scope as an app-wide listing reports it. */
export interface AppUserScope {
holder: number;
handlerCount: number;
createdAt: number;
updatedAt: number;
}
/** One user-scope handler a GC pass may delete. */
export interface UnboundUserHandler {
id: number;
appUid: string;
holder: number;
name: string;
}
/**
* One entry of a scope's handler set. `updatedAt` (unix seconds) is set for a
* user scope and absent from entries cached without it, meaning unknown.
*/
export type HandlerSetEntry = Pick<EventHandler, 'name' | 'sourceHash'> & {
updatedAt?: number;
};
export interface PublishHandlerInput {
scope: HandlerScope;
name: string;
source: string;
/**
@@ -137,6 +239,11 @@ export interface PublishHandlerInput {
ifHash?: string | null;
/** Take the name whatever is published under it. */
replace?: boolean;
/**
* A create leaves peer nodes' cached sets alone: one missing the new name
* is re-read when asked for it. Updates always broadcast.
*/
broadcast?: boolean;
}
/** What one publish did, which is what the caller reports back. */
@@ -169,10 +276,12 @@ const invalidSource = (): HttpError =>
legacyCode: 'events_handler_source_invalid',
});
const tooManyHandlers = (): HttpError =>
const tooManyHandlers = (scope: HandlerScope): HttpError =>
new HttpError(
429,
`An app may publish ${EVENTS_HANDLERS_PER_APP} handlers`,
isUserScope(scope)
? `A user scope may hold ${EVENTS_USER_HANDLERS_PER_SCOPE} handlers`
: `An app may publish ${EVENTS_HANDLERS_PER_APP} handlers`,
{ legacyCode: 'events_handler_limit' },
);
@@ -211,10 +320,11 @@ const SELECT_COLUMNS =
export const hashContent = (content: string): string =>
createHash('sha256').update(content, 'utf8').digest('hex');
export const isValidHandlerName = (name: string): boolean =>
/** `userScope` also admits the `@<sha256>` names minted for inline handlers. */
export const isValidHandlerName = (name: string, userScope = false): boolean =>
name.length > 0 &&
name.length <= HANDLER_NAME_MAX_LENGTH &&
NAME_PATTERN.test(name);
(NAME_PATTERN.test(name) || (userScope && INLINE_NAME_PATTERN.test(name)));
export class EventHandlerStore extends PuterStore {
// -- Writes ------------------------------------------------------
@@ -227,11 +337,12 @@ export class EventHandlerStore extends PuterStore {
* with a row that no longer exists.
*/
async publish(input: PublishHandlerInput): Promise<PublishResult> {
const name = this.#assertName(input.name);
const { scope } = input;
const name = this.#assertName(input.name, scope);
const source = this.#assertSource(input.source);
const sourceHash = hashContent(source);
const existing = await this.getByName(input.appUid, name);
const existing = await this.getByName(scope, name);
if (!existing) {
// `ifHash` naming a source that is not there is the same race from
// the other side: something removed the row this publish was
@@ -239,10 +350,11 @@ export class EventHandlerStore extends PuterStore {
if (input.ifHash && !input.replace) throw publishConflict(name);
return {
handler: await this.#insert(
input.appUid,
scope,
name,
source,
sourceHash,
input.broadcast !== false,
),
outcome: 'created',
};
@@ -260,34 +372,155 @@ export class EventHandlerStore extends PuterStore {
if (!input.replace && !basedOnPublished) throw publishConflict(name);
return {
handler: await this.#update(existing, source, sourceHash),
handler: await this.#update(scope, existing, source, sourceHash),
outcome: 'updated',
};
}
/** Drop one handler. Null when the app had nothing published by that name. */
async remove(appUid: string, name: string): Promise<EventHandler | null> {
const existing = await this.getByName(appUid, name);
/** Drop one handler. Null when the scope had nothing published by that name. */
async remove(
scope: HandlerScope,
name: string,
): Promise<EventHandler | null> {
const existing = await this.getByName(scope, name);
if (!existing) return null;
const { where, params } = scopeFilter(scope);
await this.clients.db.write(
`DELETE FROM \`${TABLE}\` WHERE \`app_uid\` = ? AND \`name\` = ?`,
[appUid, name],
`DELETE FROM \`${tableOf(scope)}\` WHERE ${where} AND \`name\` = ?`,
[...params, name],
);
await this.#forgetSet(appUid);
await this.#forgetSet(scope);
return existing;
}
/**
* Drop every handler an app has published — what a deleted app leaves
* behind.
* Drop every handler an app has published, in its own set and in every
* user's scope — what a deleted app leaves behind. Returns the rows
* removed. User scopes go a batch of holders at a time, each batch's cached
* sets dropped here and on every peer.
*/
async deleteForApp(appUid: string): Promise<number> {
const result = await this.clients.db.write(
`DELETE FROM \`${TABLE}\` WHERE \`app_uid\` = ?`,
[appUid],
);
await this.#forgetSet(appUid);
return result.affectedRows;
let removed = result.affectedRows ?? 0;
for (;;) {
const holders = (
await this.clients.db.pread(
`SELECT DISTINCT \`holder_user_id\` FROM \`${USER_TABLE}\` ` +
'WHERE `app_uid` = ? ORDER BY `holder_user_id` LIMIT ?',
[appUid, USER_SCOPES_PAGE_DEFAULT],
)
).map((row) => Number(row.holder_user_id));
if (holders.length === 0) break;
const batch = await this.clients.db.write(
`DELETE FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? ` +
`AND \`holder_user_id\` IN (${holders.map(() => '?').join(', ')})`,
[appUid, ...holders],
);
await this.#forgetSets(
holders.map((holder) => ({ appUid, holder })),
);
const affected = batch.affectedRows ?? 0;
removed += affected;
if (affected === 0) break;
}
await this.#forgetSet(appHandlerScope(appUid));
return removed;
}
/** Drop one user's scope of one app. Returns the handlers removed. */
async deleteUserScope(appUid: string, holder: number): Promise<number> {
const scope = { appUid, holder };
if (!isUserScope(scope)) return 0;
assertScope(scope);
const result = await this.clients.db.write(
`DELETE FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` = ?`,
[appUid, holder],
);
await this.#forgetSet(scope);
return result.affectedRows ?? 0;
}
/**
* Drop every scope a user holds, across apps — what an account deletion
* leaves behind. Returns the handlers removed.
*/
async deleteUserScopesForHolder(userId: number): Promise<number> {
if (!Number.isInteger(userId) || userId <= 0) return 0;
const apps = await this.clients.db.pread(
`SELECT DISTINCT \`app_uid\` FROM \`${USER_TABLE}\` WHERE \`holder_user_id\` = ?`,
[userId],
);
const result = await this.clients.db.write(
`DELETE FROM \`${USER_TABLE}\` WHERE \`holder_user_id\` = ?`,
[userId],
);
for (const row of apps)
await this.#forgetSet({
appUid: String(row.app_uid),
holder: userId,
});
return result.affectedRows ?? 0;
}
/**
* Delete one user-scope handler if no subscription row of any state is
* bound to it and it was last touched before `olderThan` (unix seconds).
* The check and the delete are one statement, so a row bound in between
* keeps the handler. True when it deleted.
*/
async deleteUserHandlerIfUnbound(
scope: HandlerScope,
name: string,
olderThan: number,
): Promise<boolean> {
if (!isUserScope(scope)) return false;
assertScope(scope);
const result = await this.clients.db.write(
`DELETE FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` = ? ` +
'AND `name` = ? AND `updated_at` < ? ' +
`AND NOT EXISTS (SELECT 1 FROM \`${SUBSCRIPTION_TABLE}\` ` +
'WHERE `holder_user_id` = ? AND `app_uid` = ? ' +
'AND `handler_scope` = 1 AND `handler_name` = ?)',
[
scope.appUid,
scope.holder,
name,
olderThan,
scope.holder,
scope.appUid,
name,
],
);
if (!result.anyRowsAffected) return false;
await this.#forgetSet(scope);
return true;
}
/**
* Mark one user-scope handler as just used, so the unbound sweep (which
* only takes rows last touched before its cutoff) leaves it alone. False
* when the handler is gone.
*/
async touchUserHandler(
scope: HandlerScope,
name: string,
): Promise<boolean> {
if (!isUserScope(scope)) return false;
assertScope(scope);
const result = await this.clients.db.write(
`UPDATE \`${USER_TABLE}\` SET \`updated_at\` = ? ` +
'WHERE `app_uid` = ? AND `holder_user_id` = ? AND `name` = ?',
[nowSeconds(), scope.appUid, scope.holder, name],
);
return Boolean(result.anyRowsAffected);
}
/** Drop this region's cached sets of a scope, without telling peers. */
async forgetScopeCache(scope: HandlerScope): Promise<void> {
await this.#forgetSet(scope, false);
}
// -- Reads -------------------------------------------------------
@@ -299,34 +532,36 @@ export class EventHandlerStore extends PuterStore {
* absent.
*/
async getByName(
appUid: string,
scope: HandlerScope,
name: string,
): Promise<EventHandler | null> {
const { where, params } = scopeFilter(scope);
const rows = await this.clients.db.pread(
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` ` +
'WHERE `app_uid` = ? AND `name` = ?',
[appUid, name],
`SELECT ${SELECT_COLUMNS} FROM \`${tableOf(scope)}\` ` +
`WHERE ${where} AND \`name\` = ?`,
[...params, name],
);
return rows.length > 0 ? toRow(rows[0]) : null;
}
/**
* What an app has published, with how many subscriptions each name is
* carrying. Bounded by the per-app cap, so the whole set is one page.
* What a scope has published, with how many subscriptions each name is
* carrying. Bounded by the scope's cap, so the whole set is one page.
*
* The counts come from one grouped read rather than a join, because the
* subscription side has to count names an app never published — a handler
* subscription side has to count names a scope never published — a handler
* removed while subscriptions still pointed at it leaves exactly that.
*/
async listForApp(appUid: string): Promise<EventHandlerSummary[]> {
async listForApp(scope: HandlerScope): Promise<EventHandlerSummary[]> {
const { where, params } = scopeFilter(scope);
const rows = await this.clients.db.read(
'SELECT `name`, `source_hash`, `updated_at` ' +
`FROM \`${TABLE}\` WHERE \`app_uid\` = ? ` +
`FROM \`${tableOf(scope)}\` WHERE ${where} ` +
'ORDER BY `name` LIMIT ?',
[appUid, EVENTS_HANDLERS_PER_APP],
[...params, handlerCapOf(scope)],
);
const counts = await this.countSubscriptionsByHandler(appUid);
const counts = await this.countSubscriptionsByHandler(scope);
return rows.map((row) => ({
name: String(row.name),
hash: String(row.source_hash),
@@ -336,27 +571,30 @@ export class EventHandlerStore extends PuterStore {
}
/**
* Name and source hash for every handler an app has published: the identity
* of its set, without reading a byte of source. What a deploy checks the
* script it was asked for against, and what the delivery path's cache is
* filled from. Primary, so neither reads a set a replica has not caught up
* to.
* Name and source hash for every handler a scope has published: the
* identity of its set, without reading a byte of source. What a deploy
* checks the script it was asked for against, and what the delivery path's
* cache is filled from. Primary, so neither reads a set a replica has not
* caught up to.
*/
async setForApp(
appUid: string,
scope: HandlerScope,
options: { replica?: boolean } = {},
): Promise<Pick<EventHandler, 'name' | 'sourceHash'>[]> {
): Promise<HandlerSetEntry[]> {
const { where, params: whereParams } = scopeFilter(scope);
const withAge = isUserScope(scope);
const sql =
'SELECT `name`, `source_hash` ' +
`FROM \`${TABLE}\` WHERE \`app_uid\` = ? ` +
`SELECT \`name\`, \`source_hash\`${withAge ? ', `updated_at`' : ''} ` +
`FROM \`${tableOf(scope)}\` WHERE ${where} ` +
'ORDER BY `name` LIMIT ?';
const params = [appUid, EVENTS_HANDLERS_PER_APP];
const params = [...whereParams, handlerCapOf(scope)];
const rows = options.replica
? await this.clients.db.read(sql, params)
: await this.clients.db.pread(sql, params);
return rows.map((row) => ({
name: String(row.name),
sourceHash: String(row.source_hash),
...(withAge ? { updatedAt: Number(row.updated_at) || 0 } : {}),
}));
}
@@ -371,12 +609,12 @@ export class EventHandlerStore extends PuterStore {
* right after a publish would fail the run. Called by extensions.
*/
async cachedSetForApp(
appUid: string,
scope: HandlerScope,
options: {
missRead?: 'primary' | 'replica';
handlerName?: string;
} = {},
): Promise<Pick<EventHandler, 'name' | 'sourceHash'>[]> {
): Promise<HandlerSetEntry[]> {
const replica = options.missRead === 'replica';
// A set without the handler may be from before its publish.
const usable = (set: Pick<EventHandler, 'name'>[]) =>
@@ -384,8 +622,8 @@ export class EventHandlerStore extends PuterStore {
(options.handlerName === undefined ||
set.some((handler) => handler.name === options.handlerName));
const key = replica
? replicaSetCacheKey(appUid)
: handlerSetCacheKey(appUid);
? replicaSetCacheKey(scope)
: handlerSetCacheKey(scope);
try {
const cached = JSON.parse(
(await this.clients.redis.get(key)) ?? 'null',
@@ -394,13 +632,13 @@ export class EventHandlerStore extends PuterStore {
Array.isArray(cached) &&
usable(cached as Pick<EventHandler, 'name'>[])
)
return cached as Pick<EventHandler, 'name' | 'sourceHash'>[];
return cached as HandlerSetEntry[];
} catch {
// Read the table instead.
}
let set = await this.setForApp(appUid, { replica });
if (replica && !usable(set)) set = await this.setForApp(appUid);
let set = await this.setForApp(scope, { replica });
if (replica && !usable(set)) set = await this.setForApp(scope);
if (set.length > 0) {
try {
await this.clients.redis.set(
@@ -417,57 +655,70 @@ export class EventHandlerStore extends PuterStore {
}
/**
* Every handler the app has published, sources included — what the events
* Every handler the scope has published, sources included — what the events
* worker is generated from. Primary, for the same reason `getByName` is:
* the bake follows the publish that triggered it.
*/
async allForApp(appUid: string): Promise<EventHandler[]> {
async allForApp(scope: HandlerScope): Promise<EventHandler[]> {
const { where, params } = scopeFilter(scope);
const rows = await this.clients.db.pread(
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` ` +
'WHERE `app_uid` = ? ORDER BY `name` LIMIT ?',
[appUid, EVENTS_HANDLERS_PER_APP],
`SELECT ${SELECT_COLUMNS} FROM \`${tableOf(scope)}\` ` +
`WHERE ${where} ORDER BY \`name\` LIMIT ?`,
[...params, handlerCapOf(scope)],
);
return rows.map(toRow);
}
async countForApp(appUid: string): Promise<number> {
async countForApp(scope: HandlerScope): Promise<number> {
const { where, params } = scopeFilter(scope);
const [row] = await this.clients.db.pread(
`SELECT COUNT(*) AS \`total\` FROM \`${TABLE}\` WHERE \`app_uid\` = ?`,
[appUid],
`SELECT COUNT(*) AS \`total\` FROM \`${tableOf(scope)}\` WHERE ${where}`,
params,
);
return Number(row?.total ?? 0);
}
/**
* Bytes of source an app has published, across every handler. What a
* publish is checked against before it lands, so the app's generated events
* Bytes of source a scope has published, across every handler. What a
* publish is checked against before it lands, so the generated events
* worker never grows past what the runtime may load.
*
* `OCTET_LENGTH`, not `LENGTH`: only MySQL measures the latter in bytes,
* and the caller compares this against `Buffer.byteLength`.
*/
async totalSourceBytesForApp(appUid: string): Promise<number> {
async totalSourceBytesForApp(scope: HandlerScope): Promise<number> {
const { where, params } = scopeFilter(scope);
const [row] = await this.clients.db.pread(
`SELECT SUM(OCTET_LENGTH(\`source\`)) AS \`total\` FROM \`${TABLE}\` WHERE \`app_uid\` = ?`,
[appUid],
`SELECT SUM(OCTET_LENGTH(\`source\`)) AS \`total\` FROM \`${tableOf(scope)}\` WHERE ${where}`,
params,
);
return Number(row?.total ?? 0);
}
/**
* How many subscriptions each of an app's handler names is carrying,
* How many subscriptions each of a scope's handler names is carrying,
* suspended rows included — a suspended subscription is still a dependent,
* and it is the reason a removal is not a delete.
*
* A user scope reads by holder, so it never scans the other holders of a
* popular app.
*/
async countSubscriptionsByHandler(
appUid: string,
scope: HandlerScope,
): Promise<Map<string, number>> {
assertScope(scope);
const filter = isUserScope(scope)
? '`holder_user_id` = ? AND `app_uid` = ? AND `handler_scope` = 1'
: '`app_uid` = ? AND `handler_scope` = 0';
const params = isUserScope(scope)
? [scope.holder, scope.appUid]
: [scope.appUid];
const rows = await this.clients.db.read(
'SELECT `handler_name`, COUNT(*) AS `total` ' +
`FROM \`${SUBSCRIPTION_TABLE}\` ` +
'WHERE `app_uid` = ? AND `handler_name` IS NOT NULL ' +
`WHERE ${filter} AND \`handler_name\` IS NOT NULL ` +
'GROUP BY `handler_name`',
[appUid],
params,
);
return new Map(
rows.map((row) => [
@@ -480,7 +731,8 @@ export class EventHandlerStore extends PuterStore {
/**
* How many of an owner's apps have at least one published handler — what a
* rent listener counts to tell a first publish (0→1) from a redeploy, and a
* last removal (1→0) from one of several.
* last removal (1→0) from one of several. Reads the apps' own sets only:
* user scopes are never the owner's.
*
* `createdBefore` (epoch ms) restricts to apps whose events worker existed
* before that time, by its earliest handler; used to bill only what was
@@ -512,10 +764,36 @@ export class EventHandlerStore extends PuterStore {
return Number(rows[0]?.total ?? 0);
}
/**
* How many apps a user holds a scope in — the rent count for user scopes,
* with `createdBefore` (epoch ms) meaning what it does for
* {@link countAppsWithHandlersForOwner}.
*/
async countUserScopesWithHandlersForUser(
userId: number,
opts: { createdBefore?: number } = {},
): Promise<number> {
const params: unknown[] = [userId];
let having = '';
if (opts.createdBefore !== undefined) {
having = ' HAVING MIN(`scope_since`) < ?';
params.push(Math.ceil(opts.createdBefore / 1000));
}
const rows = await this.clients.db.pread(
'SELECT COUNT(*) AS `total` FROM (' +
`SELECT \`app_uid\` FROM \`${USER_TABLE}\` ` +
`WHERE \`holder_user_id\` = ? GROUP BY \`app_uid\`${having}` +
') AS `counted`',
params,
);
return Number(rows[0]?.total ?? 0);
}
/**
* The events workers an owner is running, one row per app with at least one
* published handler. Keyset-paginated on `app_uid`, since the grouping key
* is the only stable order a set this shape can offer.
* is the only stable order a set this shape can offer. Reads the apps' own
* sets only.
*/
async listEventsWorkersForOwner(
ownerUserId: number,
@@ -574,21 +852,184 @@ export class EventHandlerStore extends PuterStore {
return result;
}
/**
* The scopes one user holds, one row per app, keyset-paginated on
* `app_uid`. `total` counts every scope the user holds (or the one app's,
* when `appUid` narrows it).
*/
async listUserScopesForUser(
userId: number,
opts: ListEventsWorkersOptions = {},
): Promise<PageResult<UserScopeSummary>> {
const limit = Math.min(
Math.max(
1,
Math.floor(opts.limit ?? EVENTS_WORKERS_LIST_DEFAULT_LIMIT),
),
EVENTS_WORKERS_LIST_LIMIT_CAP,
);
const after = decodeCursor(opts.cursor)?.appUid;
const where = [`\`${USER_TABLE}\`.\`holder_user_id\` = ?`];
const params: unknown[] = [userId];
if (typeof opts.appUid === 'string') {
where.push(`\`${USER_TABLE}\`.\`app_uid\` = ?`);
params.push(opts.appUid);
}
const totalRows = await this.clients.db.read(
'SELECT COUNT(*) AS `total` FROM (' +
`SELECT \`${USER_TABLE}\`.\`app_uid\` FROM \`${USER_TABLE}\` ` +
`WHERE ${where.join(' AND ')} GROUP BY \`${USER_TABLE}\`.\`app_uid\`` +
') AS `counted`',
params,
);
if (typeof after === 'string') {
where.push(`\`${USER_TABLE}\`.\`app_uid\` > ?`);
params.push(after);
}
const rows = await this.clients.db.read(
`SELECT \`${USER_TABLE}\`.\`app_uid\` AS \`app_uid\`, ` +
`\`${APP_TABLE}\`.\`name\` AS \`app_name\`, ` +
`\`${APP_TABLE}\`.\`title\` AS \`app_title\`, ` +
'COUNT(*) AS `handler_count`, ' +
`MIN(\`${USER_TABLE}\`.\`created_at\`) AS \`created_at\`, ` +
`MAX(\`${USER_TABLE}\`.\`updated_at\`) AS \`updated_at\` ` +
`FROM \`${USER_TABLE}\` ` +
`LEFT JOIN \`${APP_TABLE}\` ON \`${APP_TABLE}\`.\`uid\` = \`${USER_TABLE}\`.\`app_uid\` ` +
`WHERE ${where.join(' AND ')} ` +
`GROUP BY \`${USER_TABLE}\`.\`app_uid\`, \`${APP_TABLE}\`.\`name\`, \`${APP_TABLE}\`.\`title\` ` +
`ORDER BY \`${USER_TABLE}\`.\`app_uid\` LIMIT ?`,
[...params, limit + 1],
);
const page = keysetPage(rows, limit, undefined, (last) => ({
appUid: last.app_uid,
}));
const result: PageResult<UserScopeSummary> = {
items: page.rows.map((row) => ({
appUid: String(row.app_uid),
appName: row.app_name == null ? '' : String(row.app_name),
appTitle: row.app_title == null ? '' : String(row.app_title),
handlerCount: Number(row.handler_count) || 0,
createdAt: Number(row.created_at) || 0,
updatedAt: Number(row.updated_at) || 0,
})),
total: Number(totalRows[0]?.total ?? 0),
};
if (page.cursor) result.cursor = page.cursor;
return result;
}
/**
* The user scopes one app holds, ordered by holder. A batch scan: it
* carries no `total`, to spare a COUNT per page. `cursor` is null at the
* end.
*/
async listUserScopes(
appUid: string,
opts: { limit?: number; cursor?: string } = {},
): Promise<{ items: AppUserScope[]; cursor: string | null }> {
const limit = Math.min(
Math.max(1, Math.floor(opts.limit ?? USER_SCOPES_PAGE_DEFAULT)),
USER_SCOPES_PAGE_CAP,
);
const after = Number(decodeCursor(opts.cursor)?.holder);
const rows = await this.clients.db.read(
'SELECT `holder_user_id`, COUNT(*) AS `handler_count`, ' +
'MIN(`created_at`) AS `created_at`, ' +
'MAX(`updated_at`) AS `updated_at` ' +
`FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` > ? ` +
'GROUP BY `holder_user_id` ORDER BY `holder_user_id` LIMIT ?',
[appUid, Number.isFinite(after) ? after : 0, limit + 1],
);
const page = keysetPage(rows, limit, undefined, (last) => ({
holder: Number(last.holder_user_id),
}));
return {
items: page.rows.map((row) => ({
holder: Number(row.holder_user_id),
handlerCount: Number(row.handler_count) || 0,
createdAt: Number(row.created_at) || 0,
updatedAt: Number(row.updated_at) || 0,
})),
cursor: page.cursor ?? null,
};
}
/**
* One keyset page of user-scope handlers last touched before `olderThan`
* (unix seconds) that no subscription row of any state is bound to. A batch
* scan: it carries no `total`, to spare a COUNT per page. `cursor` is null
* at the end. A candidate may be bound before it is deleted, so delete
* through {@link deleteUserHandlerIfUnbound}.
*/
async listUnboundUserHandlers(
olderThan: number,
opts: { limit?: number; cursor?: string } = {},
): Promise<{ items: UnboundUserHandler[]; cursor: string | null }> {
const limit = Math.max(
1,
Math.floor(opts.limit ?? UNBOUND_HANDLERS_BATCH),
);
const after = Number(decodeCursor(opts.cursor)?.id);
const rows = await this.clients.db.read(
'SELECT h.`id`, h.`app_uid`, h.`holder_user_id`, h.`name` ' +
`FROM \`${USER_TABLE}\` h ` +
'WHERE h.`id` > ? AND h.`updated_at` < ? ' +
`AND NOT EXISTS (SELECT 1 FROM \`${SUBSCRIPTION_TABLE}\` s ` +
'WHERE s.`holder_user_id` = h.`holder_user_id` ' +
'AND s.`app_uid` = h.`app_uid` AND s.`handler_scope` = 1 ' +
'AND s.`handler_name` = h.`name`) ' +
'ORDER BY h.`id` LIMIT ?',
[Number.isFinite(after) ? after : 0, olderThan, limit + 1],
);
const page = keysetPage(rows, limit, undefined);
return {
items: page.rows.map((row) => ({
id: Number(row.id),
appUid: String(row.app_uid),
holder: Number(row.holder_user_id),
name: String(row.name),
})),
cursor: page.cursor ?? null,
};
}
// -- Internals ---------------------------------------------------
async #insert(
appUid: string,
scope: HandlerScope,
name: string,
source: string,
sourceHash: string,
broadcast: boolean,
): Promise<EventHandler> {
if ((await this.countForApp(appUid)) >= EVENTS_HANDLERS_PER_APP)
throw tooManyHandlers();
// A user scope's count read also yields when the scope began, so its
// age survives the handlers that held it.
let scopeSince: number | null = null;
if (isUserScope(scope)) {
const [state] = await this.clients.db.pread(
'SELECT COUNT(*) AS `total`, MIN(`scope_since`) AS `since` ' +
`FROM \`${USER_TABLE}\` WHERE \`app_uid\` = ? AND \`holder_user_id\` = ?`,
[scope.appUid, scope.holder],
);
if (Number(state?.total ?? 0) >= handlerCapOf(scope))
throw tooManyHandlers(scope);
scopeSince = state?.since == null ? null : Number(state.since);
} else if ((await this.countForApp(scope)) >= handlerCapOf(scope))
throw tooManyHandlers(scope);
const at = nowSeconds();
try {
await this.clients.db.insert(TABLE, {
app_uid: appUid,
await this.clients.db.insert(tableOf(scope), {
app_uid: scope.appUid,
...(isUserScope(scope)
? {
holder_user_id: scope.holder,
scope_since: scopeSince ?? at,
}
: {}),
name,
source,
source_hash: sourceHash,
@@ -603,9 +1044,25 @@ export class EventHandlerStore extends PuterStore {
if (isUniqueViolation(err)) throw publishConflict(name);
throw err;
}
await this.#forgetSet(appUid);
// Concurrent creates can all pass the count above. Counted again with
// this row in it, each one past the cap takes itself back out — at the
// last slot that may refuse more than it had to, never fewer. User
// scopes only: reads stop at the cap, so a row past it would be unseen.
if (
isUserScope(scope) &&
(await this.countForApp(scope)) > handlerCapOf(scope)
) {
const { where, params } = scopeFilter(scope);
await this.clients.db.write(
`DELETE FROM \`${tableOf(scope)}\` WHERE ${where} AND \`name\` = ?`,
[...params, name],
);
await this.#forgetSet(scope);
throw tooManyHandlers(scope);
}
await this.#forgetSet(scope, broadcast);
return {
appUid,
appUid: scope.appUid,
name,
source,
sourceHash,
@@ -615,23 +1072,25 @@ export class EventHandlerStore extends PuterStore {
}
async #update(
scope: HandlerScope,
existing: EventHandler,
source: string,
sourceHash: string,
): Promise<EventHandler> {
const at = nowSeconds();
const { where, params } = scopeFilter(scope);
// The `source_hash` predicate is what makes the check-then-write a
// compare-and-set: a publish that raced past the read above updates
// nothing here, and its caller is told.
const updated = await this.clients.db.write(
`UPDATE \`${TABLE}\` SET \`source\` = ?, \`source_hash\` = ?, ` +
'`updated_at` = ? WHERE `app_uid` = ? AND `name` = ? ' +
`UPDATE \`${tableOf(scope)}\` SET \`source\` = ?, \`source_hash\` = ?, ` +
`\`updated_at\` = ? WHERE ${where} AND \`name\` = ? ` +
'AND `source_hash` = ?',
[
source,
sourceHash,
at,
existing.appUid,
...params,
existing.name,
existing.sourceHash,
],
@@ -639,20 +1098,37 @@ export class EventHandlerStore extends PuterStore {
if (updated?.anyRowsAffected === false)
throw publishConflict(existing.name);
await this.#forgetSet(existing.appUid);
await this.#forgetSet(scope);
return { ...existing, source, sourceHash, updatedAt: at };
}
/** Drop the delivery path's cached set here and on every peer. */
async #forgetSet(appUid: string): Promise<void> {
/**
* Drop the delivery path's cached set here, and on every peer if
* `broadcast`.
*/
async #forgetSet(scope: HandlerScope, broadcast = true): Promise<void> {
await this.publishCacheKeys({
keys: [handlerSetCacheKey(appUid), replicaSetCacheKey(appUid)],
keys: [handlerSetCacheKey(scope), replicaSetCacheKey(scope)],
broadcast,
});
}
/** {@link #forgetSet} for many scopes, as one peer message. */
async #forgetSets(scopes: HandlerScope[]): Promise<void> {
await this.publishCacheKeys({
keys: scopes.flatMap((scope) => [
handlerSetCacheKey(scope),
replicaSetCacheKey(scope),
]),
broadcast: true,
});
}
#assertName(name: unknown): string {
if (typeof name !== 'string' || !isValidHandlerName(name))
#assertName(name: unknown, scope: HandlerScope): string {
if (
typeof name !== 'string' ||
!isValidHandlerName(name, isUserScope(scope))
)
throw invalidName();
return name;
}
@@ -155,6 +155,15 @@ const liveRegionsByToken = (
return live;
};
/**
* Present on every durable row this code caches: the handler scope is stated (a
* user row also carries `handlerScope`). A node without user scopes writes rows
* without it, and its entries say nothing about scope.
*/
const SCOPE_MARKER = 'sc';
const SCOPE_LOOKUP_BATCH = 200;
/** A row we cannot read is a row we cannot deliver against. */
const parseRow = (raw: string): DispatchSubscription | null => {
try {
@@ -1015,7 +1024,11 @@ export class EventSubscriptionStore extends PuterStore {
const write = this.clients.redis.pipeline();
for (const row of rows) {
const key = tokenKey(ownerUserId, row.token);
write.hset(key, row.subId, JSON.stringify(row));
write.hset(
key,
row.subId,
JSON.stringify({ ...row, [SCOPE_MARKER]: 1 }),
);
write.expire(key, DURABLE_CACHE_TTL_SECONDS);
write.hset(durableMapKey(ownerUserId), row.subId, row.token);
}
@@ -1269,9 +1282,96 @@ export class EventSubscriptionStore extends PuterStore {
subs.push(parsed);
}
}
await this.#settleScopes(ownerUserId, subs);
return subs;
}
/**
* Rows a node without user scopes cached carry no scope marker, so a user
* row among them reads as an app row. Those with a handler are resolved
* from the table (replica, then primary for what it lacks) and rewritten
* with the marker; marked rows cost nothing.
*/
async #settleScopes(
ownerUserId: number,
rows: DispatchSubscription[],
): Promise<void> {
const unmarked: DispatchSubscription[] = [];
for (const row of rows) {
const marked = (row as unknown as Record<string, unknown>)[
SCOPE_MARKER
];
delete (row as unknown as Record<string, unknown>)[SCOPE_MARKER];
if (marked === undefined && row.durable === true && row.handlerName)
unmarked.push(row);
}
if (unmarked.length === 0) return;
// A failed lookup throws, as a table read that fails does for the
// rest of dispatch: a row of unknown scope must never be returned,
// since it would read as an app row.
const scopes = new Map<string, number>();
const lookup = async (
read: 'read' | 'pread',
ids: string[],
): Promise<void> => {
const found = await this.clients.db[read](
'SELECT `sub_id`, `handler_scope` FROM `event_subscriptions` ' +
`WHERE \`sub_id\` IN (${ids.map(() => '?').join(', ')})`,
ids,
);
for (const row of found)
scopes.set(String(row.sub_id), Number(row.handler_scope));
};
for (let i = 0; i < unmarked.length; i += SCOPE_LOOKUP_BATCH) {
const ids = unmarked
.slice(i, i + SCOPE_LOOKUP_BATCH)
.map((row) => row.subId);
await lookup('read', ids);
const missing = ids.filter((id) => !scopes.has(id));
if (missing.length > 0) await lookup('pread', missing);
}
// In neither the replica nor the primary: a ghost entry. Dropped,
// as the normal path treats a row that is gone, so it is not
// resolved again on every read.
const ghosts = unmarked.filter((row) => !scopes.has(row.subId));
if (ghosts.length > 0) {
const drop = this.clients.redis.pipeline();
for (const row of ghosts)
drop.hdel(tokenKey(ownerUserId, row.token), row.subId);
await drop.exec();
for (const row of ghosts) rows.splice(rows.indexOf(row), 1);
}
const resolved = unmarked.filter((row) => scopes.has(row.subId));
for (const row of resolved)
if (scopes.get(row.subId) === 1)
(row as DurableSubscription).handlerScope = 'user';
// Rewritten only where the entry still stands, so a row dropped
// meanwhile is not brought back. Failing to rewrite costs a
// repeat lookup, not a wrong answer.
try {
const check = this.clients.redis.pipeline();
for (const row of resolved)
check.hexists(tokenKey(ownerUserId, row.token), row.subId);
const standing = (await check.exec()) ?? [];
const write = this.clients.redis.pipeline();
resolved.forEach((row, i) => {
if (Number(standing[i]?.[1]) !== 1) return;
write.hset(
tokenKey(ownerUserId, row.token),
row.subId,
JSON.stringify({ ...row, [SCOPE_MARKER]: 1 }),
);
});
await write.exec();
} catch (err) {
console.warn('[events] could not mark cached row scopes', err);
}
}
/**
* One pipelined first page per token, which is the whole hash for any token
* small enough to be stored compactly; only a larger one is scanned
@@ -1316,6 +1416,7 @@ export class EventSubscriptionStore extends PuterStore {
}
subs.push(...taken.values());
}
await this.#settleScopes(ownerUserId, subs);
return subs;
}
@@ -18,7 +18,10 @@
*/
import { randomUUID } from 'node:crypto';
import { EVENTS_INVOKE_TIMEOUT_MS } from '../../clients/events/EventsWorkerInvokerClient.js';
import {
EVENTS_INVOKE_TIMEOUT_MS,
EVENTS_WORKER_LEASE_FACTOR,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import {
EVENTS_FAILURE_COUNTER_TTL_MS,
EVENTS_PENDING_DELIVERIES_PER_SUBSCRIPTION,
@@ -316,7 +319,7 @@ export class PendingDeliveryStore extends PuterStore {
typeof configured === 'number' && configured > 0
? configured
: EVENTS_INVOKE_TIMEOUT_MS;
return invokeTimeoutMs * 2;
return invokeTimeoutMs * EVENTS_WORKER_LEASE_FACTOR;
}
// -- Writes ------------------------------------------------------
+8
View File
@@ -29,6 +29,12 @@ import type { FsOp } from '../../services/events/subjects.js';
* is declared once here rather than per store.
*/
/**
* Which handler set a subscription's `handlerName` refers to: the app's own
* (`'app'`) or the holder's scope of that app (`'user'`). Stored as 0 and 1.
*/
export type HandlerScopeKind = 'app' | 'user';
/**
* Whether a row's events also run the app's published handler in its events
* worker. Only an app has a worker, and only a named handler can be run.
@@ -97,6 +103,8 @@ export interface DispatchSubscription {
durable?: true;
delivery?: DeliveryClass;
handlerName?: string | null;
/** Durable rows only: absent means `'app'`. */
handlerScope?: HandlerScopeKind;
/** Durable rows only: handed to the handler, and read nowhere else. */
context?: string | null;
}
@@ -315,6 +315,57 @@ export class NotificationStore extends PuterStore {
return rows.map((r) => this.#normalizeRow(r));
}
/**
* The newest row id in a mailbox slice, or null when it has none, from the
* primary. Scoped like `listScoped`; rows of every type count.
*
* @param {number} userId
* @param {{ audience: string; appUid?: string | null }} scope
* @returns {Promise<number | null>}
*/
async latestScopedId(userId, { audience, appUid }) {
const { where, params } = sliceWhere(userId, {
audience,
appUid,
excludeTypes: [],
});
const [row] = await this.clients.db.pread(
`SELECT \`id\` FROM \`notification\`
WHERE ${where.join(' AND ')}
ORDER BY \`id\` DESC
LIMIT 1`,
params,
);
return row ? Number(row.id) : null;
}
/**
* The latest replacement in a mailbox slice by (`replaced_at`, `id`), or
* null when none was replaced, from the primary. Scoped like `listScoped`.
*
* @param {number} userId
* @param {{ audience: string; appUid?: string | null }} scope
* @returns {Promise<{ replacedAt: number; id: number } | null>}
*/
async latestScopedReplacement(userId, { audience, appUid }) {
const { where, params } = sliceWhere(userId, {
audience,
appUid,
excludeTypes: [],
});
where.push('`replaced_at` IS NOT NULL');
const [row] = await this.clients.db.pread(
`SELECT \`replaced_at\`, \`id\` FROM \`notification\`
WHERE ${where.join(' AND ')}
ORDER BY \`replaced_at\` DESC, \`id\` DESC
LIMIT 1`,
params,
);
return row
? { replacedAt: Number(row.replaced_at), id: Number(row.id) }
: null;
}
// -- Writes -------------------------------------------------------
/**
@@ -410,6 +410,61 @@ describe('NotificationStore', () => {
expect(rest.map((r) => r.uid)).toEqual([c.uid]);
});
it('finds the newest id of a slice, or null for an empty one', async () => {
const u = await makeUser();
const appUid = `app-${uuidv4()}`;
const write = (audience, inApp) =>
store.create({ userId: u.id, value: {}, audience, appUid: inApp });
const latest = (audience, inApp) =>
store.latestScopedId(u.id, { audience, appUid: inApp });
expect(await latest('app-user', appUid)).toBeNull();
await write('app-user', appUid);
const newest = await write('app-user', appUid);
await write('app-user', `app-${uuidv4()}`);
const unattributed = await write('developer', null);
const developer = await write('developer', appUid);
await store.create({
userId: other.id,
value: {},
audience: 'app-user',
appUid,
});
expect(await latest('app-user', appUid)).toBe(Number(newest.id));
expect(await latest('developer', null)).toBe(Number(unattributed.id));
expect(await latest('developer', undefined)).toBe(Number(developer.id));
expect(await latest('account', undefined)).toBeNull();
});
it('finds the latest replacement of a slice, or null when none', async () => {
const u = await makeUser();
const appUid = `app-${uuidv4()}`;
const write = (inApp = appUid) =>
store.create({
userId: u.id,
value: {},
audience: 'app-user',
appUid: inApp,
});
const latest = () =>
store.latestScopedReplacement(u.id, {
audience: 'app-user',
appUid,
});
const a = await write();
const b = await write();
const otherApp = await write(`app-${uuidv4()}`);
expect(await latest()).toBeNull();
await store.updateValue(a.uid, u.id, {}, { replacedAt: 2_000 });
await store.updateValue(b.uid, u.id, {}, { replacedAt: 2_000 });
await store.updateValue(otherApp.uid, u.id, {}, { replacedAt: 9_000 });
expect(await latest()).toEqual({ replacedAt: 2_000, id: Number(b.id) });
});
// -- mutations -----------------------------------------------------
it('acknowledges only once and only for the owning user', async () => {
+15 -7
View File
@@ -256,12 +256,7 @@ export interface IPreludeConfig {
* an RCS agent provisioned in the Prelude account to actually use RCS.
*/
preferredChannel?:
| 'sms'
| 'rcs'
| 'whatsapp'
| 'viber'
| 'zalo'
| 'telegram';
'sms' | 'rcs' | 'whatsapp' | 'viber' | 'zalo' | 'telegram';
}
/**
@@ -1270,6 +1265,11 @@ interface IConfigOptional {
* through a handle both reject with `events_kv_handles_disabled`, and a
* handle row already made stops delivering. Nothing on the write path
* reads it.
* - `userHandlers` — whether an app may publish handlers into a signed-in
* user's own scope, including inline handlers created with a
* subscription. Absent means off: user-scope publishing and inline
* creates reject with `events_user_handlers_disabled`; app-scope handlers
* are unaffected.
* - `workerRuntime` — whether publishing handlers deploys a per-app events
* worker and delivery invokes it. Absent means off: publish stores rows
* and nothing is deployed, and the invoker keeps its null resolver, so
@@ -1286,6 +1286,7 @@ interface IConfigOptional {
crossAppKv?: boolean;
notificationsFoldIn?: boolean;
kvHandles?: boolean;
userHandlers?: boolean;
workerRuntime?: boolean;
forwardSession?: boolean;
/** How long a handler has to answer an invocation. Default 30 s. */
@@ -1307,6 +1308,12 @@ interface IConfigOptional {
* default: deploys refuse rather than land somewhere public.
*/
workerNamespace?: string;
/**
* Per-invocation CPU limit, in milliseconds, set on every events worker
* at deploy. Unset means no limit. Scripts already deployed pick it up
* on their next deploy.
*/
workerCpuMs?: number;
/**
* Where the events dispatcher answers `POST /invoke` — its own
* hostname, since it carries no zone route. Unset means invocations
@@ -1379,7 +1386,8 @@ export interface ServerDrainInfo {
}
export interface WithCostsReporting extends WithLifecycle {
getReportedCosts?: () => // eslint-disable-next-line @typescript-eslint/no-explicit-any
getReportedCosts?: () =>
// eslint-disable-next-line @typescript-eslint/no-explicit-any
| Promise<Record<string, any>[]>
// eslint-disable-next-line @typescript-eslint/no-explicit-any
| Record<string, any>[];
+30 -1
View File
@@ -18,7 +18,12 @@
*/
import { describe, expect, it } from 'vitest';
import { toAppReadView, toAppSummary, toAppView } from './appView.js';
import {
isOriginBootstrapApp,
toAppReadView,
toAppSummary,
toAppView,
} from './appView.js';
const row = (overrides: Record<string, unknown> = {}) => ({
id: 42,
@@ -192,3 +197,27 @@ describe('toAppReadView', () => {
).toBe(64);
});
});
describe('isOriginBootstrapApp', () => {
const originRow = (overrides: Record<string, unknown> = {}) =>
row({
name: 'app-1234',
title: 'app-1234',
description: 'App created from origin https://chat.example.com',
...overrides,
});
it('matches a row created for an origin', () => {
expect(isOriginBootstrapApp(originRow())).toBe(true);
});
it('does not match once the app has its own name, title or description', () => {
expect(isOriginBootstrapApp(originRow({ name: 'chat' }))).toBe(false);
expect(isOriginBootstrapApp(originRow({ title: 'Chat' }))).toBe(false);
expect(isOriginBootstrapApp(originRow({ description: 'Chat' }))).toBe(
false,
);
expect(isOriginBootstrapApp(row())).toBe(false);
expect(isOriginBootstrapApp(null)).toBe(false);
});
});
+12
View File
@@ -25,6 +25,18 @@ import type { PrivateLaunchDecision } from './privateLaunchAccess.js';
/** A raw `apps` store row. */
export type AppRow = Record<string, unknown>;
/** A row created for an origin, whose name and title are still its uid. */
export function isOriginBootstrapApp(app: unknown): app is AppRow {
if (!app || typeof app !== 'object') return false;
const { uid, name, title, description } = app as AppRow;
if (typeof uid !== 'string' || !uid) return false;
if (name !== uid || title !== uid) return false;
return (
typeof description === 'string' &&
description.startsWith('App created from origin ')
);
}
/** Client shape of an app. Never carries internal ids or the owner's id. */
export interface AppView {
uid: unknown;
+75
View File
@@ -0,0 +1,75 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Actor } from '../core/actor.js';
import { HttpError } from '../core/http/HttpError.js';
import {
RENTED_UNITS_PER_USER,
limitFor,
} from '../controllers/events/limits.js';
import {
actorPlanId,
type SubscriptionMetering,
} from '../services/metering/enforcement.js';
import type { EventHandlerStore } from '../stores/events/EventHandlerStore.js';
import {
WORKER_SUBDOMAIN_PREFIX,
type SubdomainStore,
} from '../stores/subdomain/SubdomainStore.js';
export interface RentedUnitDeps {
stores: {
subdomain: Pick<SubdomainStore, 'countByUserIdAndPrefix'>;
eventHandler: Pick<
EventHandlerStore,
| 'countUserScopesWithHandlersForUser'
| 'countAppsWithHandlersForOwner'
>;
};
metering: SubscriptionMetering | undefined;
}
/**
* Refuse creating one more rented unit for a user already holding their plan's
* cap of them. Call only where a unit is created: a redeploy or a publish into
* an existing unit must never reach it.
*/
export const assertRentedUnitRoom = async (
deps: RentedUnitDeps,
actor: Actor,
): Promise<void> => {
const userId = actor.user?.id;
if (userId === undefined) return;
const cap = limitFor(
RENTED_UNITS_PER_USER,
await actorPlanId(deps.metering, actor),
);
const [workers, scopes, owned] = await Promise.all([
deps.stores.subdomain.countByUserIdAndPrefix(
userId,
WORKER_SUBDOMAIN_PREFIX,
),
deps.stores.eventHandler.countUserScopesWithHandlersForUser(userId),
deps.stores.eventHandler.countAppsWithHandlersForOwner(userId),
]);
if (Number(workers) + scopes + owned >= cap)
throw new HttpError(403, `Worker limit reached (max ${cap})`, {
legacyCode: 'forbidden',
});
};
-271
View File
@@ -1,271 +0,0 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { PuterServer } from '../server.ts';
import { setupTestServer } from '../testUtil.ts';
import { getTaskbarItems } from './taskbarItems.ts';
describe('getTaskbarItems', () => {
let server: PuterServer;
let deps: Parameters<typeof getTaskbarItems>[1];
beforeAll(async () => {
server = await setupTestServer();
deps = {
clients: server.clients,
stores: server.stores,
apiBaseUrl: 'https://api.puter.com',
config: { static_hosting_domain: 'puter.site' },
} as unknown as Parameters<typeof getTaskbarItems>[1];
});
afterAll(async () => {
await server?.shutdown();
});
const makeUser = async (taskbarItems?: unknown) => {
const username = `tb-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
} as never);
if (taskbarItems !== undefined) {
await server.stores.user.update(created.id, {
taskbar_items:
typeof taskbarItems === 'string'
? taskbarItems
: JSON.stringify(taskbarItems),
});
}
return (await server.stores.user.getById(created.id))!;
};
const makeApp = async (ownerUserId: number, overrides = {}) => {
const name = `tb-app-${Math.random().toString(36).slice(2, 10)}`;
return server.stores.app.create(
{
name,
title: `Title ${name}`,
index_url: `https://${name}.example.com/`,
description: 'a test app',
...overrides,
},
{ ownerUserId },
);
};
it('seeds the default taskbar for a user that has none, and persists it', async () => {
const user = await makeUser();
expect(user.taskbar_items).toBeFalsy();
await getTaskbarItems(user as never, deps);
const [row] = await server.clients.db.read(
'SELECT `taskbar_items` FROM `user` WHERE `id` = ?',
[user.id],
);
const stored = JSON.parse(String(row.taskbar_items));
expect(stored.map((e: { name: string }) => e.name)).toContain(
'app-center',
);
// The cached row must not keep serving the pre-seed value.
expect(
(await server.stores.user.getById(user.id))?.taskbar_items,
).toBeTruthy();
});
it('resolves entries by name, uid and id', async () => {
const owner = await makeUser([]);
const byName = await makeApp(owner.id);
const byUid = await makeApp(owner.id);
const byId = await makeApp(owner.id);
const user = await makeUser([
{ name: byName.name, type: 'app' },
{ uid: byUid.uid, type: 'app' },
{ id: byId.id, type: 'app' },
]);
const items = await getTaskbarItems(user as never, deps);
expect(items.map((i) => i.uid)).toEqual([
byName.uid,
byUid.uid,
byId.uid,
]);
expect(items[0]).toMatchObject({
uuid: byName.uid,
name: byName.name,
title: byName.title,
godmode: false,
maximize_on_start: false,
description: 'a test app',
});
});
it('resolves the whole taskbar with one batched lookup per key, not one per entry', async () => {
const owner = await makeUser([]);
const apps = await Promise.all([0, 1, 2].map(() => makeApp(owner.id)));
const byUid = await makeApp(owner.id);
const user = await makeUser([
...apps.map((app) => ({ name: app.name, type: 'app' })),
{ uid: byUid.uid, type: 'app' },
]);
const appStore = server.stores.app;
const spies = {
getByName: vi.spyOn(appStore, 'getByName'),
getByUid: vi.spyOn(appStore, 'getByUid'),
getById: vi.spyOn(appStore, 'getById'),
getByNames: vi.spyOn(appStore, 'getByNames'),
getByUids: vi.spyOn(appStore, 'getByUids'),
};
try {
const items = await getTaskbarItems(user as never, deps);
expect(items.map((i) => i.uid)).toEqual([
...apps.map((app) => app.uid),
byUid.uid,
]);
expect(spies.getByName).not.toHaveBeenCalled();
expect(spies.getByUid).not.toHaveBeenCalled();
expect(spies.getById).not.toHaveBeenCalled();
expect(spies.getByNames).toHaveBeenCalledTimes(1);
expect(spies.getByUids).toHaveBeenCalledTimes(1);
} finally {
for (const spy of Object.values(spies)) spy.mockRestore();
}
});
it('skips non-app entries, the explorer pin, and unresolvable apps', async () => {
const owner = await makeUser([]);
const app = await makeApp(owner.id);
const user = await makeUser([
{ name: 'explorer', type: 'app' },
{ name: 'nothing-here', type: 'app' },
{ name: app.name, type: 'separator' },
{ type: 'app' },
{ name: app.name, type: 'app' },
]);
const items = await getTaskbarItems(user as never, deps);
expect(items).toHaveLength(1);
expect(items[0].name).toBe(app.name);
});
it('falls back to an empty taskbar when the stored JSON is corrupt', async () => {
const user = await makeUser('{not json');
expect(await getTaskbarItems(user as never, deps)).toEqual([]);
});
it('accepts an already-parsed taskbar array on the user row', async () => {
const owner = await makeUser([]);
const app = await makeApp(owner.id);
const user = await makeUser([]);
const items = await getTaskbarItems(
{ ...user, taskbar_items: [{ name: app.name, type: 'app' }] },
deps,
);
expect(items.map((i) => i.name)).toEqual([app.name]);
});
it('builds a sized icon URL by default and drops the icon on request', async () => {
const owner = await makeUser([]);
const app = await makeApp(owner.id);
const user = await makeUser([{ name: app.name, type: 'app' }]);
const withIcon = await getTaskbarItems(user as never, deps, {
iconSize: 64,
});
expect(withIcon[0].icon).toBe(
`https://api.puter.com/app-icon/${app.uid}/64`,
);
const withoutIcon = await getTaskbarItems(user as never, deps, {
noIcons: true,
});
expect('icon' in withoutIcon[0]).toBe(false);
});
it('pairs the icon URL with the direct subdomain URL when one exists', async () => {
const owner = await makeUser([]);
// An http(s) icon column is what the resize pipeline leaves behind, so
// the sized PNG is on the subdomain.
const app = await makeApp(owner.id, {
icon: 'https://api.puter.com/app-icon/x',
});
const user = await makeUser([{ name: app.name, type: 'app' }]);
const items = await getTaskbarItems(user as never, deps, {
iconSize: 64,
});
expect(items[0].icon).toBe(
`https://api.puter.com/app-icon/${app.uid}/64`,
);
expect(items[0].iconCdnUrl).toBe(
`https://puter-app-icons.puter.site/${app.uid}-64.png`,
);
});
it('reports no subdomain URL for an app whose icon was never generated', async () => {
const owner = await makeUser([]);
const app = await makeApp(owner.id);
const user = await makeUser([{ name: app.name, type: 'app' }]);
const items = await getTaskbarItems(user as never, deps, {
iconSize: 64,
});
expect(items[0].iconCdnUrl).toBeNull();
});
it('falls back to the raw icon column when no API base URL is configured', async () => {
const owner = await makeUser([]);
const app = await makeApp(owner.id, {
icon: 'https://cdn.example.com/i.png',
});
const user = await makeUser([{ name: app.name, type: 'app' }]);
const items = await getTaskbarItems(
user as never,
{
...deps,
apiBaseUrl: undefined,
} as never,
);
expect(items[0].icon).toBe('https://cdn.example.com/i.png');
});
it('reports null for an app with no icon and no API base URL', async () => {
const owner = await makeUser([]);
const app = await makeApp(owner.id);
const user = await makeUser([{ name: app.name, type: 'app' }]);
const items = await getTaskbarItems(
user as never,
{
...deps,
apiBaseUrl: undefined,
} as never,
);
expect(items[0].icon).toBeNull();
});
});
-134
View File
@@ -1,134 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { AppIconHostConfig } from './appIcon.js';
import { toAppSummary, type AppRow } from './appView.js';
interface TaskbarEntry {
name?: string;
id?: number;
uid?: string;
type?: string;
}
interface TaskbarOptions {
iconSize?: number;
noIcons?: boolean;
}
interface TaskbarDeps {
apiBaseUrl?: string;
config?: AppIconHostConfig;
stores: {
app: {
getByNames: (names: string[]) => Promise<Map<string, AppRow>>;
getByUids: (uids: string[]) => Promise<Map<string, AppRow>>;
getByIds: (ids: number[]) => Promise<Map<number, AppRow>>;
};
user: {
update: (
id: number,
patch: Record<string, unknown>,
) => Promise<unknown>;
};
};
}
const DEFAULT_TASKBAR_ITEMS: TaskbarEntry[] = [
{ name: 'app-center', type: 'app' },
{ name: 'builder', type: 'app' },
{ name: 'dev-center', type: 'app' },
{ name: 'editor', type: 'app' },
{ name: 'code', type: 'app' },
{ name: 'camera', type: 'app' },
{ name: 'recorder', type: 'app' },
{ name: 'browser', type: 'app' },
];
export async function getTaskbarItems(
user: Record<string, unknown>,
deps: TaskbarDeps,
options: TaskbarOptions = {},
): Promise<Array<Record<string, unknown>>> {
let raw: TaskbarEntry[];
if (!user.taskbar_items) {
raw = DEFAULT_TASKBAR_ITEMS;
await deps.stores.user.update(user.id as number, {
taskbar_items: JSON.stringify(raw),
});
} else {
try {
raw =
typeof user.taskbar_items === 'string'
? JSON.parse(user.taskbar_items as string)
: (user.taskbar_items as TaskbarEntry[]);
} catch {
raw = [];
}
}
const entries = raw.filter(
(entry) => entry.type === 'app' && entry.name !== 'explorer',
);
const [byName, byUid, byId] = await Promise.all([
deps.stores.app.getByNames(
entries.flatMap((entry) => (entry.name ? [entry.name] : [])),
),
deps.stores.app.getByUids(
entries.flatMap((entry) =>
!entry.name && entry.uid ? [entry.uid] : [],
),
),
deps.stores.app.getByIds(
entries.flatMap((entry) =>
!entry.name && !entry.uid && entry.id ? [entry.id] : [],
),
),
]);
const items: Array<Record<string, unknown>> = [];
for (const entry of entries) {
const app = entry.name
? byName.get(entry.name)
: entry.uid
? byUid.get(entry.uid)
: entry.id
? byId.get(entry.id)
: undefined;
if (!app) continue;
const item: Record<string, unknown> = {
uid: app.uid,
...toAppSummary(app, {
apiBaseUrl: deps.apiBaseUrl,
config: deps.config,
iconSize: options.iconSize,
}),
description: app.description,
};
if (options.noIcons) {
delete item.icon;
delete item.iconCdnUrl;
}
items.push(item);
}
return items;
}
+9 -23
View File
@@ -64,7 +64,7 @@ const makeUser = async () => {
};
describe('buildUserDetails', () => {
it('builds the desktop payload for a user actor', async () => {
it('builds the GUI payload for a user actor', async () => {
const user = await makeUser();
const details = await buildUserDetails(user, deps(), {
isUser: true,
@@ -85,15 +85,18 @@ describe('buildUserDetails', () => {
[`/${user.username}/Desktop`]: user.desktop_uuid,
[`/${user.username}/Trash`]: user.trash_uuid,
});
expect(Array.isArray(details.taskbar_items)).toBe(true);
// The desktop's taskbar and wallpaper are gone with it.
expect(details).not.toHaveProperty('taskbar_items');
expect(details).not.toHaveProperty('desktop_bg_url');
expect(details).not.toHaveProperty('desktop_bg_color');
expect(details).not.toHaveProperty('desktop_bg_fit');
});
it('leaves the desktop-only parts out for an app actor', async () => {
it('leaves the GUI-only parts out for an app actor', async () => {
const user = await makeUser();
const details = await buildUserDetails(user, deps(), {
isUser: false,
});
expect(details).not.toHaveProperty('taskbar_items');
expect(details).not.toHaveProperty('directories');
expect(details.card_fallback_available).toBe(false);
});
@@ -124,7 +127,7 @@ describe('buildUserDetails lookups', () => {
return keys.slice(keys.indexOf('hasDevAccountAccess') + 1);
};
it('reads the linked provider, taskbar and seat together, listed in a fixed order', async () => {
it('reads the linked provider and seat together, listed in a fixed order', async () => {
const user = {
...(await makeUser()),
password: null,
@@ -162,7 +165,6 @@ describe('buildUserDetails lookups', () => {
);
expect(tailKeys(details)).toEqual([
'oidc_revalidate_url',
'taskbar_items',
'directories',
'team',
'last_activity_ts',
@@ -177,7 +179,6 @@ describe('buildUserDetails lookups', () => {
const user = {
...(await makeUser()),
password: null,
taskbar_items: JSON.stringify([{ type: 'app', name: 'editor' }]),
last_activity_ts: '2026-01-02 03:04:05',
};
const events: string[] = [];
@@ -199,15 +200,6 @@ describe('buildUserDetails lookups', () => {
},
},
stores: Object.assign(Object.create(d.stores), {
app: {
getByNames: () =>
timed(events, 'taskbar', 20, () => {
throw new Error('taskbar down');
}),
getByUids: async () => new Map(),
getByIds: async () => new Map(),
},
// Fails first, but is reported after the taskbar.
team: {
getOrgSeat: () =>
timed(events, 'seat', 5, () => {
@@ -219,18 +211,12 @@ describe('buildUserDetails lookups', () => {
{ isUser: true },
);
expect(warn.mock.calls).toEqual([
['[user-details] taskbar_items failed:', expect.any(Error)],
['[user-details] team lookup failed:', expect.any(Error)],
]);
} finally {
warn.mockRestore();
}
expect(tailKeys(details)).toEqual([
'taskbar_items',
'directories',
'last_activity_ts',
]);
expect(details.taskbar_items).toEqual([]);
expect(tailKeys(details)).toEqual(['directories', 'last_activity_ts']);
});
});
+8 -50
View File
@@ -27,18 +27,12 @@ import {
isCardFallbackEligible,
type CardFallbackClients,
} from './cardFallback.js';
import { getTaskbarItems } from './taskbarItems.js';
import { isEmailGateLast } from './verificationOrder.js';
type TaskbarDeps = Parameters<typeof getTaskbarItems>[1];
export interface UserDetailsDeps {
config: IConfig;
// `db.write` backs the taskbar's first-read default row.
clients: CardFallbackClients & {
db: { write: (query: string, params?: unknown[]) => Promise<unknown> };
};
stores: TaskbarDeps['stores'] & {
clients: CardFallbackClients;
stores: {
kv: Pick<SystemKVStore, 'get'>;
team?: Pick<TeamStoreContract, 'getOrgSeat'>;
};
@@ -78,8 +72,8 @@ const SENSITIVE_KEYS: ReadonlySet<string> = new Set([
]);
/**
* Delete every sensitive key at any depth (`metadata` and `taskbar_items` are
* nested). Depth-limited and cycle-safe. Runs last over a whole payload.
* Delete every sensitive key at any depth (`metadata` is nested). Depth-limited
* and cycle-safe. Runs last over a whole payload.
*/
export const scrubSensitive = (
value: unknown,
@@ -118,13 +112,13 @@ export const toUnixSeconds = (value: unknown): number | undefined => {
/**
* The account as the client keeps it in `window.user`: what `/whoami` answers
* and what a sign-in hands back. `isUser` false (an app acting for the user)
* leaves out the desktop-only parts; trimming fields an app may not see is the
* leaves out the GUI-only parts; trimming fields an app may not see is the
* caller's job. Not scrubbed: run `scrubSensitive` on the final payload.
*/
export async function buildUserDetails(
user: UserRow,
deps: UserDetailsDeps,
opts: { isUser: boolean; iconSize?: number; noIcons?: boolean },
opts: { isUser: boolean },
): Promise<Record<string, unknown>> {
const { config, clients, stores, services } = deps;
const { isUser } = opts;
@@ -170,9 +164,6 @@ export async function buildUserDetails(
cardFallbackDepsFrom(clients),
)
: false,
desktop_bg_url: user.desktop_bg_url,
desktop_bg_color: user.desktop_bg_color,
desktop_bg_fit: user.desktop_bg_fit,
is_temp: user.password === null && user.email === null,
is_user_token: true,
// Null until the account asks for a code; never minted here.
@@ -185,14 +176,6 @@ export async function buildUserDetails(
hasDevAccountAccess: !!user.metadata?.hasDevAccountAccess,
};
// Not an inline literal, so a `TaskbarDeps` without `clients` still
// accepts it.
const taskbarDeps = {
clients,
stores,
apiBaseUrl: String(config.api_base_url ?? ''),
config,
};
const linkedProvider = async () => {
if (!oidcOnly) return null;
try {
@@ -202,19 +185,6 @@ export async function buildUserDetails(
return null;
}
};
const taskbarItems = async () => {
if (!isUser) return null;
try {
return {
items: await getTaskbarItems(user, taskbarDeps, {
iconSize: opts.iconSize,
noIcons: opts.noIcons,
}),
};
} catch (error) {
return { error };
}
};
// The team that pays for the account; every seat restriction keys on it.
const orgSeat = async () => {
if (!isUser || config.teams_enabled !== true) return null;
@@ -226,26 +196,14 @@ export async function buildUserDetails(
};
// Independent and each best-effort, so read together; the results land
// below in the order the payload has always listed them.
const [provider, taskbar, team] = await Promise.all([
linkedProvider(),
taskbarItems(),
orgSeat(),
]);
const [provider, team] = await Promise.all([linkedProvider(), orgSeat()]);
if (provider) {
const origin = (config.origin ?? '').replace(/\/$/, '');
details.oidc_revalidate_url = `${origin}/auth/oidc/${provider}/start?flow=revalidate&user_uuid=${encodeURIComponent(user.uuid)}`;
}
if (taskbar) {
// Best-effort: the account still works without them.
if ('error' in taskbar) {
console.warn('[user-details] taskbar_items failed:', taskbar.error);
details.taskbar_items = [];
} else {
details.taskbar_items = taskbar.items;
}
if (isUser) {
const directories: Record<string, unknown> = {};
const nameToProp: Record<string, string> = {
desktop_uuid: `/${user.username}/Desktop`,
+24 -14
View File
@@ -11,7 +11,7 @@ The Events API tells your app when a user's data changes. Subscribe to a file, a
There are two kinds of subscription:
- **Session subscriptions** ([`onLocal()`](/Events/onLocal/)) belong to the open page and end when it closes. Use them to keep a UI in sync.
- **Persistent subscriptions** ([`onPersistent()`](/Events/onPersistent/)) are stored on the account and keep running while your app is closed. Each change runs a [handler](/Events/handlers/) your app published. Use them for background work, such as processing uploads as they arrive.
- **Persistent subscriptions** ([`onPersistent()`](/Events/onPersistent/)) are stored on the account and keep running while your app is closed. Each change runs a [handler](/Events/handlers/), either inline code you pass in the call or one your app published. Use them for background work, such as processing uploads as they arrive.
What you watch is written as a subject string:
@@ -25,16 +25,21 @@ What you watch is written as a subject string:
[Subjects and events](/Events/subjects/) has the full subject syntax and the shape of each event.
To run a handler on a timer instead of when data changes, create a schedule with [`onSchedule()`](/Events/onSchedule/). It runs your handler at an interval, on a cron expression in the user's time zone, or once at a set time, even while your app is closed:
To run a handler on a timer instead of when data changes, create a schedule with [`onSchedule()`](/Events/onSchedule/). It runs your handler at an interval, on a cron expression in the user's time zone, or once at a set time, even while your app is closed. Pass the handler inline, and Puter stores it in the signed-in user's own [user scope](/Events/handlers/#user-scope) of your app:
```js
await puter.events.onSchedule({
cron: '0 9 * * 1-5',
tz: 'America/New_York',
handlerName: 'sendDigest',
handler: async ({ ctx, user }) => {
await user.notifications.notify({ title: `Good morning, ${ctx.name}` });
},
context: { name: 'Ana' },
});
```
To ship one handler to every user, the app's owner can publish it under a name and each schedule or subscription binds to it with `handlerName`.
With the [User-Pays Model](/user-pays-model/), deliveries and schedule runs are billed to the user holding the subscription or schedule, not to you. See [Rate Limits and Quotas](/rate-limits-and-quotas/#events) for limits and costs.
## Features
@@ -161,26 +166,25 @@ With the [User-Pays Model](/user-pays-model/), deliveries and schedule runs are
#### Keep watching with a handler after the page closes
Publishing a handler needs an app you own, so this runs in your own app rather than in the playground.
The handler goes in with the subscription and runs in the events worker. It's stored in the signed-in user's own [user scope](/Events/handlers/#user-scope) of your app. The playground examples below run this form.
```js
// Once, as the app's developer
await puter.events.handlers.publish('ingestUpload', async ({ event, ctx }) => {
await fetch(ctx.endpoint, { method: 'POST', body: event.path });
});
// In your app, for each user
await puter.perms.request('backgroundEvents');
const sub = await puter.events.onPersistent({
subject: 'fs:inbox',
handlerName: 'ingestUpload',
handler: async ({ event, ctx }) => {
await fetch(ctx.endpoint, { method: 'POST', body: event.path });
},
context: { endpoint: 'https://example.com/ingest' },
});
// It outlives the page. End it when it's no longer needed.
// It outlives the page. End it, and remove its handler, when it's no longer needed.
await puter.events.unsubscribe(sub.subId);
await puter.events.handlers.remove(sub.handlerName, { scope: 'user' });
```
To share one handler between all your users, the app's owner publishes it once under a name and each subscription binds to it with `handlerName`. See [`puter.events.handlers`](/Events/handlers/).
</div>
<div class="example-content" data-section="list">
@@ -262,8 +266,8 @@ These Events features are supported out of the box when using Puter.js:
- **[`puter.events.getSchedule()`](/Events/getSchedule/)** - Read one schedule
- **[`puter.events.removeSchedule()`](/Events/removeSchedule/)** - End a schedule
- **[`puter.events.fetch()`](/Events/fetch/)** - Read the notifications you missed while nothing was listening
- **[`puter.events.handlers`](/Events/handlers/)** - Publish, list and remove the named handlers persistent subscriptions run
- **[`puter.events.workers`](/Events/workers/)** - List and destroy an app's events worker
- **[`puter.events.handlers`](/Events/handlers/)** - Publish, list and remove the named handlers persistent subscriptions and schedules run, for the app or for one user
- **[`puter.events.workers`](/Events/workers/)** - List and destroy an app's events worker, or a user's own
- **[Subjects and events](/Events/subjects/)** - The subject syntax and the shape of every event
## Examples
@@ -280,10 +284,16 @@ You can see various Puter.js Events features in action from the following exampl
- [Find Suspended Subscriptions](/playground/events-list-suspended/)
- [Change a Subscription's Context](/playground/events-update/)
- [End a Persistent Subscription](/playground/events-unsubscribe/)
- [Run an Inline Handler on a Folder](/playground/events-persistent-inline/)
- [Queue Events for an Inline Handler](/playground/events-persistent-queue-inline/)
- Schedules
- [List Schedules](/playground/events-list-schedules/)
- [Read a Schedule](/playground/events-get-schedule/)
- [Remove a Schedule](/playground/events-remove-schedule/)
- [Run Code Once at a Set Time](/playground/events-schedule-inline/)
- [Run Code on Weekday Mornings](/playground/events-schedule-cron-tz/)
- [Run Code Every 15 Minutes](/playground/events-schedule-every/)
- [Share a Handler Across Schedules](/playground/events-handlers-user-scope/)
- Catching up
- [Catch Up on Notifications](/playground/events-fetch/)
- [Catch Up, Then Keep Listening](/playground/events-fetch-then-listen/)
+4 -2
View File
@@ -28,6 +28,7 @@ puter.events.fetch(options)
- `subject` (String) (required): What to read. Use `notif:app-user` for your app's notifications for the user, and `notif:developer` for notifications about your app, for its owner. The full form is `notif:<appId>:<audience>`, where the audience is `account`, `developer`, or `app-user`. Apps and websites never see `account` notifications, so `notif:account` returns an empty page for them.
- `cursor` (String): The `cursor` from the previous page. Leave it off to start from the oldest notification still kept. A read from a cursor returns the notifications created after it, plus any notification [replaced](/Notifications/notify/#groups) since that page was read, even one from an earlier page.
- `after` (String): The older name for `cursor`, still accepted. If both are given, `cursor` is used.
- `from` (String): Where a read without a cursor starts. `'start'`, the default, starts from the oldest notification still kept. `'end'` skips what's already there: it returns no items and a `cursor` at the newest notification, and reads from that cursor return only notifications created or replaced after it. Ignored when a cursor is given.
- `limit` (Number): Events per page. Defaults to 50, max 200.
## Return value
@@ -35,7 +36,7 @@ puter.events.fetch(options)
A `Promise` for `{ items, cursor }`:
- `items`: The events, oldest first, in the same shape as live deliveries.
- `cursor`: Pass it back as `cursor` to read the next page. It's absent on the last page. Treat it as opaque.
- `cursor`: Pass it back as `cursor` to read the next page. It's absent on the last page, but always present on a `from: 'end'` read of a part of the mailbox you can see. Treat it as opaque.
Each item is a notification event:
@@ -60,6 +61,7 @@ Besides `type` and `title`, `notification` carries these fields for each type:
- `share.claimed` (`account`): `fields.count`.
- `app.worker.deployed`, `app.worker.deployFailed` (`developer`): nothing else.
- `app.events.suspended` (`developer`): `handler` and `subscriptions`, when removing a handler or repeated failures [suspended](/Events/onPersistent/#suspended-subscriptions) subscriptions bound to it.
- `app.events.handler_suspended` (`app-user`): `handler` and `subscriptions`, when a handler's removal, repeated failures, or a delivery that found it missing [suspended](/Events/onPersistent/#suspended-subscriptions) subscriptions or schedules bound to a handler in the user's own [user scope](/Events/handlers/#user-scope). It's the holder's counterpart to `app.events.suspended`, and arrives on `notif:app-user` of the app.
- `app.events.ended` (`app-user`): `reason` (`anchor_deleted`, `permission_revoked`, or `no_credit`), sent to the holder when persistent subscriptions stopped without being unsubscribed. With `anchor_deleted` or `permission_revoked`, it also has `subject`, `subjects` (up to 20), and `count`. It arrives on `notif:app-user` of the app that made the subscriptions.
- `app.message` (`app-user`): A notification your app sent with [`notify()`](/Notifications/notify/). It carries `priority`, `ttl`, and `app` (`{ uid, name, icon }`, the app that sent it), plus `text`, `icon`, `target`, `data`, `group`, `channel`, and `sensitive` when they were set.
- `kv.share.received` (`app-user`): The `handle`, `access`, `label`, and `createdAt` (in unix seconds) of a key-value share another user made with you with `notify` on, or turned `notify` on for (see [`puter.kv.share()`](/KV/share/#what-the-holder-sees)).
@@ -78,7 +80,7 @@ The promise rejects with `{ message, code }`:
| --- | --- |
| `invalid_subject` | `subject` is missing or doesn't parse. |
| `invalid_subject_audience` | A `notif:` subject names an audience other than `account`, `developer`, or `app-user`. |
| `invalid_request` | `limit` isn't a number, or `cursor` or `after` isn't a string. |
| `invalid_request` | `limit` isn't a number, `cursor` or `after` isn't a string, or `from` isn't `'start'` or `'end'`. |
| `fetch_unsupported_subject` | The subject is `fs:` or `kv:`, which store nothing to read. |
| `auth_canceled` | Nobody was signed in, and the user closed the sign-in without finishing it. |
| `too_many_requests` | Over the `fetch()` rate limit. |
+74 -14
View File
@@ -1,14 +1,19 @@
---
title: puter.events.handlers
description: Publish, list and remove the named handlers that persistent subscriptions and schedules run.
description: Publish, list and remove the named handlers that persistent subscriptions and schedules run, for an app or for one user.
platforms: [websites, apps, nodejs, workers]
---
<div class="info">The Events API is in beta. Event shapes, limits, and behavior may change between releases.</div>
A **handler** is a function your app publishes once, under a name, for [persistent subscriptions](/Events/onPersistent/) and [schedules](/Events/onSchedule/) to run. A handler runs only when a subscription bound to its name has a delivery, or a schedule bound to it is due. Nothing else can trigger it.
A **handler** is a function that [persistent subscriptions](/Events/onPersistent/) and [schedules](/Events/onSchedule/) run in the background. A handler runs only when a subscription bound to it has a delivery, or a schedule bound to it is due. Nothing else can trigger it.
Only the app's owner can publish, list, and remove its handlers. Inside an app, including a website using Puter.js, these calls act on that app, and work only while the signed-in user owns it. Outside an app, such as in Node.js with your account token, name the app with `appUid`.
A handler lives in one of two scopes:
- **App scope.** The handler belongs to the app, and every user's subscriptions and schedules can bind to it by name. Only the app's owner can publish, list, and remove these. Inside an app, including a website using Puter.js, these calls act on that app, and work only while the signed-in user owns it. Outside an app, such as in Node.js with your account token, name the app with `appUid`.
- **User scope.** The handler belongs to one user in one app, and only that user's subscriptions and schedules can bind to it. Any signed-in user's session of the app can publish, list, and remove these, with `scope: 'user'`. See [User scope](#user-scope).
You don't have to publish a handler to use one. Pass it inline as `handler` to [`onSchedule()`](/Events/onSchedule/) or [`onPersistent()`](/Events/onPersistent/) without a `handlerName`, and it's published into the signed-in user's scope for you.
On a website with nobody signed in, each method asks the user to sign in first, as other Puter.js calls do. An app running on Puter is always signed in.
@@ -21,6 +26,38 @@ await puter.events.handlers.list({ appUid }); // [{ name, hash, updatedAt, s
await puter.events.handlers.remove('ingestUpload', { appUid });
```
## User scope
A user scope holds the handlers one user has published in one app. Nobody else can publish into it, list it, or remove from it, including the app's owner.
An inline `handler` without a `handlerName` is published into the scope for you, under a name Puter chooses that starts with `@`, which is reserved for these. The name is derived from the code, so the same code is one handler, and changing the code makes a new one. Subscriptions and schedules keep running the code they were created with.
To ship a new version to existing subscriptions and schedules, publish a named handler into the scope and bind to it with `handlerName` and `handlerScope: 'user'`. Publishing the name again moves every one of them to the new code:
```js
await puter.events.handlers.publish('digest', async ({ ctx, user }) => {
await user.notifications.notify({ title: ctx.title });
}, { scope: 'user' });
await puter.events.onSchedule({
cron: '0 8 * * 1-5',
handlerName: 'digest',
handlerScope: 'user',
context: { title: 'Good morning' },
});
```
Handlers in a user scope:
- Are published only from an app acting for a signed-in user, never from a scoped access token or a temporary (guest) account, and only into that app's scope.
- Are billed to the user like their own workers. A hosted Puter server may bill each user scope as a monthly cost, and publishing into one needs credit. Listing and removing don't.
- Count toward the user's [worker units](/rate-limits-and-quotas/#worker-units), as one unit per app. A user over their plan's limit keeps what they hold, and can't create a new scope.
- Are limited to 20 handlers and 1 MB of source combined per scope, and 10 deploys an hour. See [Rate Limits and Quotas](/rate-limits-and-quotas/#handlers).
- Clean themselves up. Once no subscription or schedule, in any state, is bound to a handler and it's more than a day old, Puter removes it. This covers inline handlers and named ones. A scope with no handlers left stops counting as a worker unit.
- Suspend like any other handler. The user gets an `app.events.handler_suspended` notification, and the developer isn't told, when a subscription or schedule bound to one is suspended because the handler was removed (including by the user, with `remove()` or `workers.destroy()`), because it failed 5 times in a row, or because a delivery found the handler missing. Publishing the name again resumes them.
The server may turn user-scope handlers off, and then publishing into one fails with `events_user_handlers_disabled`. To remove a whole scope, see [`workers.destroy()`](/Events/workers/).
## Handlers can't use outside variables
A handler is serialized with `Function.prototype.toString()` and run later, somewhere else, so nothing around the function comes with it.
@@ -61,7 +98,8 @@ puter.events.handlers.publish(name, handler, options)
- `name` (String) (required): The name subscriptions bind to. Letters, digits, and `_ . : -`, starting with a letter or digit, up to 128 characters. Unique within the app.
- `handler` (Function | String | Object) (required): A function (serialized with `toString()`), a source string, or `{ file: '~/AppData/…/handler.js' }`. A file is read at this call, so later edits to it take effect only when you publish again.
- `options.replace` (Boolean): Overwrite whatever is published under the name.
- `options.appUid` (String): The app to publish into. Required outside an app. Inside an app, it can only name that app.
- `options.scope` (String): `'app'` or `'user'`. Defaults to `'app'`. `'user'` publishes into the signed-in user's own [user scope](#user-scope) of the app.
- `options.appUid` (String): The app to publish into. Required outside an app. Inside an app, it can only name that app. With `scope: 'user'` it can only name the app making the call, and needs no value. Only [`workers.destroy()`](/Events/workers/) can name another app for a user scope, from an account session.
Resolves to `{ name, hash, updatedAt, outcome, resumed }`. Its `outcome` is `'created'`, `'updated'`, or `'unchanged'` (the same source was already published), and `resumed` counts the suspended subscriptions this publish brought back.
@@ -78,7 +116,7 @@ puter.events.handlers.publishAll(handlers)
puter.events.handlers.publishAll(handlers, options)
```
Publishes a set in one call, for build steps. Pass `handlers` as an array of up to 50 `{ name, handler, replace? }`, published in order. The `options.appUid` option works as it does for `publish()`. If one is refused, publishing stops there. Earlier items stay published, and the error says where it stopped.
Publishes a set in one call, for build steps. Pass `handlers` as an array of up to 50 `{ name, handler, replace? }`, published in order. The `options.appUid` and `options.scope` options work as they do for `publish()`. If one is refused, publishing stops there. Earlier items stay published, and the error says where it stopped.
Resolves to an array of `publish()` results.
@@ -89,7 +127,7 @@ puter.events.handlers.list()
puter.events.handlers.list(options)
```
Resolves to `[{ name, hash, updatedAt, subscriptions }]` for every handler the app has published, sorted by name. Each `subscriptions` count includes the subscriptions and schedules bound to that name, suspended ones included. The `options.appUid` option works as it does for `publish()`.
Resolves to `[{ name, hash, updatedAt, subscriptions }]` for every handler the app, or the user's scope with `scope: 'user'`, has published, sorted by name. Each `subscriptions` count includes the subscriptions and schedules bound to that name, suspended ones included. The `options.appUid` and `options.scope` options work as they do for `publish()`.
The source is never returned.
@@ -100,18 +138,18 @@ puter.events.handlers.remove(name)
puter.events.handlers.remove(name, options)
```
Resolves to `{ name, removed, suspended }`, where `removed` is `false` when nothing was published under the name. The `options.appUid` option works as it does for `publish()`.
Resolves to `{ name, removed, suspended }`, where `removed` is `false` when nothing was published under the name. The `options.appUid` and `options.scope` options work as they do for `publish()`.
| Situation | What happens |
| --- | --- |
| No subscriptions or schedules use the name | The handler is deleted. |
| Subscriptions or schedules use it | The handler is deleted, and its subscriptions and schedules are suspended with `suspendedReason: 'handler_not_found'`, not deleted. The app's developer gets an `app.events.suspended` notification. |
| Subscriptions or schedules use it | The handler is deleted, and its subscriptions and schedules are suspended with `suspendedReason: 'handler_not_found'`, not deleted. The app's developer gets an `app.events.suspended` notification. For a handler in a user scope, the user gets an `app.events.handler_suspended` notification instead. |
Publishing the name again resumes them, with the same ids and context. This is how you recover from a bad deploy. A resumed schedule runs at its next slot, and doesn't make up the runs it missed.
To rename a handler, publish the new name and remove the old one. Subscriptions and schedules don't follow the rename. Users have to subscribe again, so their subscriptions never switch to different code without them knowing.
An app's first published handler creates its [events worker](/Events/workers/), and removing the last one takes it down.
An app's first published handler creates its [events worker](/Events/workers/), and removing the last one takes it down. A user scope works the same way: its first handler creates it, and removing its last one removes the scope.
## Failed runs
@@ -126,14 +164,17 @@ All four methods reject with `{ message, code }`:
| `auth_canceled` | Nobody was signed in, and the user closed the sign-in without finishing it. |
| `events_handler_free_variable` | The handler uses an outside variable. The message names it. |
| `events_handler_invalid` | `handler` isn't a function, a source string, or `{ file }`. |
| `events_handler_name_invalid` | The name is empty, too long, or has characters that aren't allowed. |
| `events_handler_name_invalid` | The name is empty, too long, or has characters that aren't allowed. Names starting with `@` are reserved for inline handlers. |
| `events_handler_conflict` | Someone else published different source under this name since you last saw it. Pass `replace: true` to overwrite. |
| `events_handler_app_required` | The call was made outside an app without `appUid`. |
| `events_handler_forbidden` | The signed-in user doesn't own the app, the app doesn't exist, `appUid` names a different app than the one making the call, or the caller is a scoped access token. |
| `events_handler_forbidden` | For app scope, the signed-in user doesn't own the app, the app doesn't exist, `appUid` names a different app than the one making the call, or the caller is a scoped access token. For `scope: 'user'`, publishing, listing and removing need an app acting for a signed-in user, so the caller was a scoped access token, a temporary account, or a different app. [`workers.destroy()`](/Events/workers/) also accepts an account session. |
| `events_user_handlers_disabled` | `scope: 'user'` was used to publish, and this server doesn't have user-scope handlers turned on. |
| `insufficient_funds` | Publishing into a user scope needs credit on a hosted Puter server, and the user has none. Arrives as HTTP 402. |
| `forbidden` | The user already holds as many [worker units](/rate-limits-and-quotas/#worker-units) as their plan allows, so a new user scope can't be created. |
| `events_handler_too_large` | The handler is over 64 KB. |
| `events_worker_too_large` | The app's handlers would exceed 5 MB combined. |
| `events_worker_too_large` | The app's handlers would exceed 5 MB combined, or a user scope's would exceed 1 MB. |
| `events_handler_source_invalid` | The handler source is empty. |
| `events_handler_limit` | The app already has the [maximum number of handlers](/rate-limits-and-quotas/#handlers). |
| `events_handler_limit` | The app, or the user scope, already has the [maximum number of handlers](/rate-limits-and-quotas/#handlers). |
| `invalid_request` | `publishAll()` got something other than a non-empty array. |
| `too_many_requests` | Over the publish and remove rate limit. |
| `events_disabled` | Events aren't enabled on this server. |
@@ -141,7 +182,26 @@ All four methods reject with `{ message, code }`:
## Examples
These examples run in Node.js signed in with your account token, and `appUid` names an app you own. Inside your own app, leave out `appUid`.
The first example publishes into a user scope from inside an app, for the signed-in user. The others run in Node.js signed in with your account token, and `appUid` names an app you own. Inside your own app, leave out `appUid`.
<strong class="example-title">Publish into the user's scope, list it, then remove it</strong>
```js
const published = await puter.events.handlers.publish(
'digest',
async ({ ctx, user }) => {
await user.notifications.notify({ title: ctx.title });
},
{ scope: 'user' },
);
console.log(published.outcome);
for (const handler of await puter.events.handlers.list({ scope: 'user' })) {
console.log(`${handler.name}: ${handler.subscriptions} bound`);
}
await puter.events.handlers.remove('digest', { scope: 'user' });
```
<strong class="example-title">Publish a handler, list it, then remove it</strong>
+48 -19
View File
@@ -6,7 +6,7 @@ platforms: [websites, apps, nodejs, workers]
<div class="info">The Events API is in beta. Event shapes, limits, and behavior may change between releases.</div>
Creates a subscription that outlives this connection. It's stored on the account, keeps matching while your app is closed, and runs a handler your app published with [`puter.events.handlers.publish()`](/Events/handlers/). Each event also reaches every connected client. Compare [`puter.events.onLocal()`](/Events/onLocal/), which ends with the page.
Creates a subscription that outlives this connection. It's stored on the account, keeps matching while your app is closed, and runs a handler in an [events worker](/Events/workers/). The handler is either inline code you pass as `handler`, which is published into the signed-in user's own [user scope](/Events/handlers/#user-scope) of your app, or a handler published under a name with [`puter.events.handlers.publish()`](/Events/handlers/). Each event also reaches every connected client. Compare [`puter.events.onLocal()`](/Events/onLocal/), which ends with the page.
The subscription is live right away in the region where it was created. Right after this call resolves, changes made in other regions can take a moment longer to reach it, usually well under a second.
@@ -24,9 +24,10 @@ puter.events.onPersistent(options)
#### `options` (Object) (required)
- `subject` (String) (required): What to watch, in the same form [`onLocal()`](/Events/onLocal/) takes, such as `fs:~/Documents` or `kv:<handle>:*`.
- `handlerName` (String): The published handler to run in your app's [events worker](/Events/workers/). Up to 128 characters. A subscription made outside an app has no events worker, so its handler runs only in connected clients.
- `queue` (Boolean): Handles events like a work queue. Each event goes to one consumer at a time, and it's offered again if a run never finishes or hits a temporary error. Requires `handlerName`. Defaults to `false`. See [Choosing a mode](#choosing-a-mode).
- `handler` (Function | String | Object): The handler source you wrote this subscription against, as a function, a source string, or `{ file: '~/AppData/…/handler.js' }`. Only its hash is sent, and the subscription is created only if it matches what's published under `handlerName`, which is required with it. A function also runs in this client for each delivery it receives (see [Where the handler runs](#where-the-handler-runs)).
- `handlerName` (String): The published handler to run in an [events worker](/Events/workers/). Up to 128 characters. A subscription made outside an app has no events worker, so its handler runs only in connected clients.
- `handlerScope` (String): Where `handlerName` is looked up, `'app'` or `'user'`. Defaults to `'app'`, the handlers the app's owner published. `'user'` binds to a handler in the signed-in user's own scope, published with `handlers.publish(name, handler, { scope: 'user' })`.
- `queue` (Boolean): Handles events like a work queue. Each event goes to one consumer at a time, and it's offered again if a run never finishes or hits a temporary error. Requires `handlerName` or an inline `handler`. Defaults to `false`. See [Choosing a mode](#choosing-a-mode).
- `handler` (Function | String | Object): The handler code, as a function, a source string, or `{ file: '~/AppData/…/handler.js' }`. Without `handlerName`, the source is sent and published into the signed-in user's own user scope, and it runs for this user's events only. With `handlerName`, only its hash is sent, and the subscription is created only if it matches what's published under that name. A function also runs in this client for each delivery it receives (see [Where the handler runs](#where-the-handler-runs)).
- `context` (Object): Values the handler needs, passed to it as a frozen `ctx`. Up to 4 KB serialized. See [Context is captured once](#context-is-captured-once-up-to-4-kb).
- `expiresAt` (Number | String): When the subscription ends on its own, as unix seconds or an ISO-8601 string. Must be in the future.
- `includeValue` (Boolean): For a `kv:` subject, include the key's new value as `event.value`. See [Getting the new value](/Events/subjects/#getting-the-new-value) for when it's left out. Other subjects are refused with `invalid_include_value`.
@@ -36,19 +37,19 @@ puter.events.onPersistent(options)
## Background delivery needs consent
Running your handler while the user isn't there needs its own per-app permission, `events:background`. When an app subscribes with a `handlerName` without it, the call fails with `events_background_consent_required`. To ask for it, call [`puter.perms.request()`](/Perms/request/) with the `'backgroundEvents'` resource:
Running your handler while the user isn't there needs its own per-app permission, `events:background`. When an app subscribes with a `handlerName` or an inline `handler` without it, the call fails with `events_background_consent_required`. To ask for it, call [`puter.perms.request()`](/Perms/request/) with the `'backgroundEvents'` resource:
```js
await puter.perms.request('backgroundEvents');
```
A subscription without a `handlerName` runs nothing in the background and needs no consent.
A subscription with neither a `handlerName` nor an inline `handler` runs nothing in the background and needs no consent.
Background handlers run as a worker session for your app, which appears in the user's sessions list. Revoking that session invalidates the `user` token a run holds, and the next delivery starts a new session. Withdrawing `events:background`, uninstalling the app, destroying its events worker, or deleting the app also stops background deliveries. What happens to the subscriptions depends on which one it was:
- **Withdrawing `events:background`** suspends every subscription with a `handlerName` the app holds for that user, with `permission_revoked`.
- **Withdrawing `events:background`** suspends every subscription with a handler the app holds for that user, with `permission_revoked`.
- **Uninstalling the app** suspends every subscription the app holds for that user, with `permission_revoked`.
- **Destroying the events worker** suspends its subscriptions with `handler_not_found`. Publishing the handlers again resumes them.
- **Destroying the events worker** suspends its subscriptions with `handler_not_found`. Publishing the handlers again resumes them. Destroying a user scope does the same for the handlers in it, including inline ones, which resume when you publish the same code again.
- **Deleting the app** deletes its subscriptions and handlers.
A subscription suspended with `permission_revoked` doesn't resume when access is granted again. Subscribe again instead.
@@ -71,12 +72,12 @@ A queued event comes back only on a retry or a lost answer, with the same `event
Without `queue`, every event runs the handler in each place that's listening:
- The app's [events worker](/Events/workers/) runs the handler named by `handlerName`, whether or not a client is open. It skips a run when the server has no worker runtime, or when the [handler-run limit](/rate-limits-and-quotas/#events) for this user and app is used up for the minute.
- An [events worker](/Events/workers/) runs the handler named by `handlerName`, or the inline `handler`, whether or not a client is open. It skips a run when the server has no worker runtime, or when the [handler-run limit](/rate-limits-and-quotas/#events) for this user and app is used up for the minute.
- Every connected client that passed a function `handler` runs it too. Every [`onLocal()`](/Events/onLocal/) subscription that matches the change gets the event as well.
With `queue: true`, a connected client that passed a function `handler` gets the first try. If no client has one, or its run fails, the events worker takes the event right away. A client that doesn't answer within 15 seconds loses the event to the events worker, so keep client handlers short. If the user has your app open in several tabs, each tab that passed a function `handler` gets the event, so keep side effects that must happen once in the worker handler.
Only a function `handler` runs in this client. A source string or `{ file }` is only used for its hash. The connection reconnects on its own when it drops, so this client keeps taking deliveries unless `onError` is called.
Only a function `handler` runs in this client. A source string or `{ file }` is only published, or hashed to check it against `handlerName`. The connection reconnects on its own when it drops, so this client keeps taking deliveries unless `onError` is called.
Wherever it runs, the handler is the same code, called with:
@@ -116,7 +117,7 @@ throw Object.assign(new Error('Service busy, try later'), { retry: true });
In the events worker, a run that fails because of Puter doesn't count as a failure. That covers a run Puter couldn't start or finish, and a handler that throws after a call it made through `user` got a `5xx`, a `429`, or a network error from Puter. It's retried on the same schedule, and waits longer if Puter's answer asked for a longer wait, up to 5 minutes. To drop the event in that case anyway, throw an error with `terminal: true`.
Five failures in a row suspend the subscription with `failures`. The developer is notified, and publishing the handler again resumes it.
Five failures in a row suspend the subscription with `failures`, and publishing the handler again resumes it.
Without `queue`, a failed run is never retried.
@@ -149,7 +150,7 @@ A persistent subscription can stop without being unsubscribed. It's then suspend
A suspended subscription stops delivering and isn't billed. Its backlog is cut to 100 deliveries and kept for 24 hours (`handler_not_found`, `failures`) or 1 hour (`no_credit`), then replaced by one gap marker. A `permission_revoked` backlog is dropped right away. Suspended subscriptions are deleted after 30 days.
The app's developer gets an `app.events.suspended` notification for `handler_not_found` and `failures`. The holder gets an `app.events.ended` notification on `notif:app-user` for `no_credit` and `permission_revoked`. Both can be read with [`fetch()`](/Events/fetch/).
The app's developer gets an `app.events.suspended` notification for `handler_not_found` and `failures` on a handler in the app's scope. The holder gets an `app.events.handler_suspended` notification instead when a subscription bound to a handler in their own user scope is suspended, and the developer isn't told. That covers the handler being removed, including by the holder with `handlers.remove()` or `workers.destroy()`, five failures in a row, and a delivery that finds the handler missing. The holder also gets an `app.events.ended` notification on `notif:app-user` for `no_credit` and `permission_revoked`. Both can be read with [`fetch()`](/Events/fetch/).
A subscription suspended for a reason that can resume still counts toward your [persistent subscription limit](/rate-limits-and-quotas/#what-an-account-can-hold) until you unsubscribe it. One suspended with `permission_revoked` doesn't count. An account keeps at most 500 of those, and past that, subscribing again deletes the oldest ones.
@@ -169,7 +170,8 @@ A `Promise` that resolves to the subscription:
- `subId` (String): Its id, which [`puter.events.update()`](/Events/update/) and [`puter.events.unsubscribe()`](/Events/unsubscribe/) take. It never changes.
- `subject` (String): For `kv:` and `notif:`, the subject in full form, so `kv:cart` comes back as `kv:<appId>:cart`. For `fs:`, as you passed it, except a relative path comes back resolved the same way [`onLocal()`](/Events/onLocal/) returns it.
- `anchor`, `match`, `op`: As [`onLocal()`](/Events/onLocal/#return-value) returns them.
- `queue` (Boolean), `handlerName` (String | null), `includeValue` (Boolean).
- `queue` (Boolean), `handlerName` (String | null), `includeValue` (Boolean). An inline handler comes back as its server-assigned `handlerName`, which starts with `@`.
- `handlerScope` (String): `'user'` when the subscription is bound to a handler in the user's own scope. Absent otherwise.
- `appUid` (String | null): The app that created it, or `null` if it was created outside an app.
- `contextKeys` (Array | null), `contextHash` (String | null): The key names and a hash of the stored context, never its values.
- `createdAt`, `expiresAt` (Number | null): Unix seconds.
@@ -182,14 +184,21 @@ The promise rejects with `{ message, code }`:
| --- | --- |
| `invalid_subject` | The subject is empty, not a string, or can't be parsed, or it's a `notif:` subject and this server has notification events turned off. |
| `auth_canceled` | Nobody was signed in, and the user closed the sign-in without finishing it. |
| `events_handler_name_required` | `handler` was given without `handlerName`. |
| `events_user_handlers_disabled` | Publishing into a user scope isn't turned on for this server. |
| `events_handler_forbidden` | The call can't publish into a user scope: it came from a scoped access token, a temporary account, or on behalf of a different app. |
| `insufficient_funds` | Publishing into a user scope needs credit on a hosted Puter server, and the user has none. Arrives as HTTP 402. |
| `forbidden` | The user already holds as many [worker units](/rate-limits-and-quotas/#worker-units) as their plan allows, so a new user scope can't be created. |
| `events_handler_limit` | The user scope already holds the [maximum number of handlers](/rate-limits-and-quotas/#handlers). |
| `events_handler_too_large` | The handler is over 64 KB. |
| `events_worker_too_large` | The user scope's handlers would exceed 1 MB combined. |
| `events_handler_name_invalid` | `handlerName` has characters that aren't allowed, or starts with `@`, which is reserved for inline handlers. |
| `events_handler_free_variable` | The handler uses an outside variable. The message names it. |
| `events_handler_invalid` | `handler` isn't a function, a source string, or `{ file }`. |
| `events_handler_hash_unavailable` | This environment has no `crypto.subtle`, so `handler` can't be hashed. Pass `handlerName` alone. |
| `events_handler_not_found` | Nothing is published under `handlerName` in this app. |
| `events_handler_not_found` | Nothing is published under `handlerName` in that scope of this app. |
| `events_handler_hash_mismatch` | The published handler isn't the source you passed as `handler`. |
| `events_handler_required` | `queue: true` without a `handlerName`. |
| `events_background_consent_required` | An app subscribed with a `handlerName`, and the user hasn't granted `events:background`. |
| `events_handler_required` | `queue: true` without a `handlerName` or an inline `handler`. |
| `events_background_consent_required` | An app subscribed with a handler, and the user hasn't granted `events:background`. |
| `events_context_too_large` | The serialized `context` is over 4 KB. |
| `events_context_invalid` | `context` isn't JSON-serializable. |
| `bad_request` | `queue` isn't a boolean, `delivery` isn't `'broadcast'` or `'single'`, the two disagree, or `handlerName` isn't a string or is over 128 characters. |
@@ -206,9 +215,29 @@ The promise rejects with `{ message, code }`:
## Examples
Publishing a handler needs an app you own, so these examples run in your own app or from Node.js signed in as its developer, not in the playground.
The first example is the inline form, which any app can use for the signed-in user. The others bind to a handler the app's owner published, so they run in your own app or from Node.js signed in as its developer.
<strong class="example-title">Watch a folder with a handler that keeps running</strong>
<strong class="example-title">Watch a folder with an inline handler</strong>
```js
await puter.perms.request('backgroundEvents');
const sub = await puter.events.onPersistent({
subject: 'fs:~/inbox',
handler: async ({ event, ctx }) => {
await fetch(ctx.webhook, { method: 'POST', body: event.path });
},
context: { webhook: 'https://example.com/ingest' },
});
// It outlives the page. End it when it's no longer needed.
await puter.events.unsubscribe(sub.subId);
```
The handler is stored in the user's scope under `sub.handlerName`, and keeps running for this user until the subscription ends. After the subscription ends, Puter removes the handler about a day later, once nothing is bound to it. You can remove it sooner, as [Clean Up Background Work](/recipes/clean-up-background-work/#remove-a-user-scope) shows.
<strong class="example-title">Watch a folder with a handler the app's owner published</strong>
Publish once as the developer to ship one shared handler to every user.
```js
// Once, as the app's developer
+20 -9
View File
@@ -8,7 +8,9 @@ platforms: [websites, apps, nodejs, workers]
<div class="info">Schedules are available on puter.com. A self-hosted Puter server answers <code>events_disabled</code>.</div>
Runs a handler your app published with [`puter.events.handlers.publish()`](/Events/handlers/) on a timer. A schedule can run at an interval, on a cron expression in a time zone, or once at a set time. It's stored on the user's account and runs in your app's [events worker](/Events/workers/), whether or not your app is open. It never runs in the client that created it.
Runs a handler on a timer. A schedule can run at an interval, on a cron expression in a time zone, or once at a set time. It's stored on the user's account and runs in an [events worker](/Events/workers/), whether or not your app is open. It never runs in the client that created it.
The handler is either inline code you pass as `handler`, or a handler published under a name with [`puter.events.handlers.publish()`](/Events/handlers/). Inline code is published into the signed-in user's own [user scope](/Events/handlers/#user-scope) of your app and runs for that user only. A named handler published by the app's owner is shared by every user of the app.
To run a handler when the user's data changes instead, use [`puter.events.onPersistent()`](/Events/onPersistent/).
@@ -34,9 +36,10 @@ Give exactly one of `every`, `cron` or `at`.
- `startAt` (Number | String | Date): For `every`, the first run. Without it, the first run is one interval after this call. A `startAt` in the past sets where the intervals line up, and the first run is the next one after this call. For example, `every: '1d'` with `startAt: '2026-01-01'` runs at midnight UTC every day. For `cron`, the earliest time the schedule may run. At most 366 days ahead.
- `expiresAt` (Number | String | Date): The last time the schedule may run. A run due at exactly this time still runs.
- `maxRuns` (Number): How many slots the schedule has, from 1 to 100,000. It ends at its `maxRuns`-th slot counted from the first. It counts slots, not runs that happened, so slots skipped while the schedule was behind or suspended (see [When a schedule runs](#when-a-schedule-runs)) still count, and the retries of one slot count once. With `expiresAt` too, the one that comes first ends the schedule.
- `handlerName` (String) (required): The published handler each run calls.
- `handler` (Function | String | Object): The handler source you wrote this schedule against, as a function, a source string, or `{ file: '~/AppData/…/handler.js' }`. Only its hash is sent. The schedule is created only if it matches what's published under `handlerName`. It never runs in this client.
- `handlerHash` (String): The SHA-256 hex digest of the handler source, in place of `handler`.
- `handler` (Function | String | Object): The handler code, as a function, a source string, or `{ file: '~/AppData/…/handler.js' }`. Required unless you pass `handlerName`. Without `handlerName`, the source is sent and published into the signed-in user's own user scope, and every run calls it. With `handlerName`, only its hash is sent, and the schedule is created only if it matches what's published under that name. It never runs in this client.
- `handlerName` (String): The published handler each run calls. Required unless you pass `handler`.
- `handlerScope` (String): Where `handlerName` is looked up, `'app'` or `'user'`. Defaults to `'app'`, the handlers the app's owner published. `'user'` binds to a handler the signed-in user's scope holds, published with `handlers.publish(name, handler, { scope: 'user' })`.
- `handlerHash` (String): The SHA-256 hex digest of the handler source, in place of `handler`, to check it against `handlerName`.
- `context` (Object): Values the handler needs, passed to every run as a frozen `ctx`. Up to 4 KB serialized. It's read once, at this call, and can't be changed later. To change it, remove the schedule and create a new one.
### Times
@@ -164,7 +167,7 @@ The `user` binding is valid for 15 minutes, so use it only during the run. Write
A run that fails because of Puter doesn't count as a failure. It's retried on the same schedule, and waits longer if Puter's answer asked for a longer wait, up to 5 minutes. To drop the run in that case anyway, throw an error with `terminal: true`. The rules are the same as for a [`queue` subscription](/Events/onPersistent/#what-happens-when-a-handler-fails), except that the retries stop once the schedule's next run is due, or after 24 hours for an `at` schedule or a last run. The run is then dropped without billing.
Five failures in a row suspend the schedule with `failures`. The developer is notified, and publishing the handler again resumes it.
Five failures in a row suspend the schedule with `failures`, and publishing the handler again resumes it. For an app handler the developer gets an `app.events.suspended` notification. For a handler in a user scope the user gets an `app.events.handler_suspended` notification instead, and the developer isn't told. The user gets it too when the handler is removed, including by the user, or a run finds it missing, which suspends the schedule with `handler_not_found`.
## Background runs need consent
@@ -210,7 +213,8 @@ A `Promise` that resolves to the schedule. All times are in milliseconds since t
- `scheduleId` (String): Its id, which [`getSchedule()`](/Events/getSchedule/) and [`removeSchedule()`](/Events/removeSchedule/) take.
- `kind` (String): `'recurring'` for `every` and `cron`, `'once'` for `at`, and `'deferred'` for a notification sent later with [`notify()`](/Notifications/notify/#send-it-later).
- `appUid` (String): The app that created it.
- `handlerName` (String | null): The handler each run calls, or `null` once the schedule has ended.
- `handlerName` (String | null): The handler each run calls, or `null` once the schedule has ended. For a schedule created with an inline `handler`, it's the name Puter gave the handler, which starts with `@`.
- `handlerScope` (String): `'user'` when the handler is in the user's own [user scope](/Events/handlers/#user-scope). Absent for an app handler.
- `contextKeys` (Array | null), `contextHash` (String | null): The key names and a hash of the stored context, never its values. The `contextKeys` field is `null` when there's no context, and once the schedule has ended.
- `every` (Number | null): The interval, in milliseconds, of an `every` schedule.
- `cron` (String | null): The expression of a `cron` schedule.
@@ -229,8 +233,15 @@ The promise rejects with `{ message, code }`:
| `code` | Meaning |
| --- | --- |
| `invalid_request` | `options` isn't an object. |
| `events_handler_required` | There's no `handlerName`. |
| `events_handler_name_required` | `handler` was given without `handlerName`. |
| `events_handler_required` | There's neither a `handler` nor a `handlerName`. |
| `events_user_handlers_disabled` | Publishing into a user scope isn't turned on for this server. |
| `events_handler_forbidden` | The call can't publish into a user scope: it came from a scoped access token, a temporary account, or on behalf of a different app. |
| `insufficient_funds` | Publishing into a user scope needs credit on a hosted Puter server, and the user has none. Arrives as HTTP 402. |
| `forbidden` | The user already holds as many [worker units](/rate-limits-and-quotas/#worker-units) as their plan allows, so a new user scope can't be created. |
| `events_handler_limit` | The user scope already holds the [maximum number of handlers](/rate-limits-and-quotas/#handlers). |
| `events_handler_too_large` | The handler is over 64 KB. |
| `events_worker_too_large` | The user scope's handlers would exceed 1 MB combined. |
| `events_handler_name_invalid` | `handlerName` is empty, too long, has characters that aren't allowed, or starts with `@`, which is reserved for inline handlers. |
| `events_handler_free_variable` | The handler uses an outside variable. The message names it. |
| `events_handler_invalid` | `handler` isn't a function, a source string, or `{ file }`. |
| `events_handler_hash_unavailable` | This environment has no `crypto.subtle`, so `handler` can't be hashed. Pass `handlerName` alone. |
@@ -246,7 +257,7 @@ The promise rejects with `{ message, code }`:
| `events_schedule_too_frequent` | It would run more often than your plan's [minimum interval](#how-often-a-schedule-can-run). |
| `events_schedule_too_deep` | The call was made from a handler at the end of a [handler chain](/Events/onPersistent/#handlers-that-trigger-handlers), so the schedule could never run. |
| `bad_request` | `handlerName` is empty or over 128 characters, or `handlerHash` isn't a SHA-256 hex digest. |
| `events_handler_not_found` | Nothing is published under `handlerName`. The schedule isn't created. |
| `events_handler_not_found` | Nothing is published under `handlerName` in that scope. The schedule isn't created. |
| `events_handler_hash_mismatch` | The published handler isn't the source you passed as `handler`. |
| `events_schedule_limit` | The user holds as many schedules as their plan allows, for your app or in total. |
| `events_schedule_budget` | The schedule would make your app's schedules, or all of the user's schedules, run more times an hour than their plan allows. |
+21 -9
View File
@@ -1,21 +1,21 @@
---
title: puter.events.workers
description: List and destroy the events worker a published handler set stands up.
description: List and destroy the events worker a published handler set stands up, for an app or for one user.
platforms: [websites, apps, nodejs, workers]
---
<div class="info">The Events API is in beta. Event shapes, limits, and behavior may change between releases.</div>
An **events worker** runs an app's published [handlers](/Events/handlers/). It runs them for every event on a subscription with a `handlerName`, whether or not a client is connected, for a [`queue`](/Events/onPersistent/#choosing-a-mode) delivery once no connected client takes it, and for every run of a [schedule](/Events/onSchedule/). Each app has at most one, created when it publishes its first handler, however many handlers it publishes.
An **events worker** runs published [handlers](/Events/handlers/), the app's own or one user's in a [user scope](/Events/handlers/#user-scope). It runs them for every event on a subscription with a `handlerName`, whether or not a client is connected, for a [`queue`](/Events/onPersistent/#choosing-a-mode) delivery once no connected client takes it, and for every run of a [schedule](/Events/onSchedule/). Each app has at most one for its own handlers, created when it publishes its first handler. Each user scope is a unit of its own, created when the user's first handler in an app is published, whether by publishing it or by passing an inline `handler`.
A hosted Puter server may bill each app's events worker as a monthly cost, even if nothing ever delivers to it. Use this API to see which apps have one and remove the ones you don't need.
A hosted Puter server may bill each app's events worker as a monthly cost, even if nothing ever delivers to it, and bills a user scope to the user the same way. Use this API to see which apps and scopes have one and remove the ones you don't need.
```js
const { items } = await puter.events.workers.list(); // [{ appUid, appName, handlerCount, ... }]
await puter.events.workers.destroy(items[0].appUid); // removes every handler that app published
```
Outside an app, such as in Node.js with your account token, you see the workers of every app you own. Inside an app, you see only that app's own worker (0 or 1 item), and only if the signed-in user owns the app.
Outside an app, such as in Node.js with your account token, you see the workers of every app you own, and the user scopes you hold in any app. Inside an app, you see only that app's own worker, if the signed-in user owns the app, and the signed-in user's scope of that app.
On a website with nobody signed in, each method asks the user to sign in first, as other Puter.js calls do. An app running on Puter is always signed in.
@@ -33,7 +33,7 @@ puter.events.workers.list(options)
Resolves to `{ items, cursor, deployable }`:
- `items`: One `{ appUid, appName, appTitle, handlerCount, createdAt, updatedAt, script }` per app. In each item, `createdAt` is when the app's first handler was published, `updatedAt` is its latest publish, and `script` names the deployed script, which is useful when reporting an issue.
- `items`: One `{ scope, appUid, appName, appTitle, handlerCount, createdAt, updatedAt, script }` per app worker, then one per user scope you hold. `scope` is `'app'` for an app's own worker and `'user'` for your scope of that app. In each item, `createdAt` is when the first handler was published, `updatedAt` is the latest publish, and `script` names the deployed script, which is useful when reporting an issue. A `'user'` item has no `script`.
- `cursor`: Present while more pages exist.
- `deployable`: `false` when this server doesn't run events workers, such as a self-hosted install without the runtime turned on. Handlers can still be published there, but background deliveries never run.
@@ -41,13 +41,16 @@ Resolves to `{ items, cursor, deployable }`:
```js
puter.events.workers.destroy(appUid)
puter.events.workers.destroy(appUid, options)
```
Removes every handler the app has published, the same as calling [`puter.events.handlers.remove()`](/Events/handlers/) on each. Their subscriptions and schedules are suspended with `handler_not_found`, not deleted, and publishing the handlers again resumes them.
- `options.scope` (String): `'app'` or `'user'`. Defaults to `'app'`. `'user'` removes the signed-in user's own scope of the app. Inside an app, `appUid` can only name that app. With an account session, outside an app such as the desktop or Node.js with your account token, `appUid` can name any app you hold a user scope in, and it works even when the server has user-scope handlers turned off.
It also ends the worker session background deliveries ran under, for every user. The next delivery after a new publish starts a new one.
Removes every handler the app has published, or every handler in the user's scope with `scope: 'user'`, the same as calling [`puter.events.handlers.remove()`](/Events/handlers/) on each. Their subscriptions and schedules are suspended with `handler_not_found`, not deleted, and publishing the handlers again resumes them.
Resolves to `{ appUid, removed, suspended }`, the number of handlers deleted and the number of subscriptions and schedules suspended. An app with nothing published rejects with `events_handler_not_found`.
It also ends the worker session background deliveries ran under, for every user, or for the one user with `scope: 'user'`. The next delivery after a new publish starts a new one.
Resolves to `{ appUid, removed, suspended }`, plus `scope: 'user'` for a user scope, the number of handlers deleted and the number of subscriptions and schedules suspended. An app with nothing published rejects with `events_handler_not_found`.
## Errors
@@ -58,7 +61,7 @@ Both methods reject with `{ message, code }`:
| `auth_canceled` | Nobody was signed in, and the user closed the sign-in without finishing it. |
| `events_worker_owner_only` | `list()` was called with a scoped access token, such as the one in a [`getReadURL()`](/FS/getReadURL/) URL. |
| `events_handler_not_found` | `destroy()` named an app with no published handlers. |
| `events_handler_forbidden` | The signed-in user doesn't own the app, the app doesn't exist, the call came from a different app, or the caller is a scoped access token. |
| `events_handler_forbidden` | The signed-in user doesn't own the app, the app doesn't exist, the call came from a different app, or the caller is a scoped access token. For `scope: 'user'`, the caller is a scoped access token, a temporary account, or an app naming a different app than its own. |
| `too_many_requests` | Over the publish and remove rate limit, or the listing rate limit. |
| `events_disabled` | Events aren't enabled on this server. |
| `events_failed` | The server sent a response the SDK couldn't read. |
@@ -79,3 +82,12 @@ for (const worker of items) {
const destroyed = await puter.events.workers.destroy(appUid); // an app you own
console.log(`removed ${destroyed.removed}, suspended ${destroyed.suspended}`);
```
<strong class="example-title">Remove the signed-in user's scope of your app</strong>
The user scope belongs to the user, so it needs no owner step. In your app, name the app itself. With an account session outside an app, name any app you hold a scope in.
```js
const destroyed = await puter.events.workers.destroy(puter.appID, { scope: 'user' });
console.log(`removed ${destroyed.removed}, suspended ${destroyed.suspended}`);
```
+1 -1
View File
@@ -44,7 +44,7 @@ It rejects with an error carrying `message` and `code`:
| `push_revoked` | The user turned your app off in Puter's settings. Call [`requestPermission()`](/Notifications/requestPermission/) from a click, then call [`enable()`](/Notifications/enable/) again. |
| `push_worker_unavailable` | The service worker couldn't be registered or didn't start, for example because nothing is served at its path. |
| `push_worker_conflict` | Another service worker controls the scope. Add the `importScripts()` line to that worker and pass its path as `serviceWorker`. |
| `push_subscription_conflict` | The worker already has a push subscription from another service, such as Firebase or OneSignal. Give Puter its own worker in a folder, such as `/puter/puter-sw.js`, and pass that path as `serviceWorker`. |
| `push_subscription_conflict` | The worker already has a push subscription from another service, such as Firebase or OneSignal. Give Puter its own worker in a folder, such as `/puter/puter-sw.js`, and pass that path as `serviceWorker`. A subscription Puter made earlier, even with a key Puter has since changed, is replaced instead. |
| `push_subscribe_failed` | The browser couldn't create a push subscription, usually because it couldn't reach its push service. |
| `push_subscription_invalid` | The browser's push service refused the subscription, even after a fresh one. |
| `push_endpoint_revoked` | Another Puter account turned your app off on this browser's subscription, and a fresh subscription couldn't replace it. |
+13 -3
View File
@@ -20,7 +20,7 @@ On a website, pushes arrive on your own origin through a service worker that loa
importScripts('https://js.puter.com/push-sw.js');
```
To use another worker, such as one your site already has, pass its path as `serviceWorker`, and pass the same path to [`enable()`](/Notifications/enable/), [`disable()`](/Notifications/disable/) and [`status()`](/Notifications/status/). [Push notifications for your app](/Notifications/push/#1-serve-the-service-worker) has the setup and the rules the worker follows. Inside the Puter desktop, pushes go through Puter and no worker is needed.
To use another worker, such as one your site already has, pass its path as `serviceWorker`, and pass the same path to [`enable()`](/Notifications/enable/), [`disable()`](/Notifications/disable/) and [`status()`](/Notifications/status/). [Push notifications for your app](/Notifications/push/#1-serve-the-service-worker) has the setup and the rules the worker follows, and how to [serve your own copy](/Notifications/push/#serving-your-own-copy) of `push-sw.js` from npm. Inside the Puter desktop, pushes go through Puter and no worker is needed.
## Syntax
```js
@@ -57,9 +57,11 @@ On a website, the handler is saved as source in this browser and run again in yo
- **Fast.** It has 5 seconds. After that, or if it throws, the push shows the notification's own title and text.
- **Something always shows.** Browsers show a notification for every push, so a handler can change what shows but can't hide it.
- **Small input.** A push carries about 4 KB, with `data` at most 1 KB. Put IDs in `data` and fetch the rest.
- **`eval`.** It's re-created with `new Function`, so a Content-Security-Policy on your worker has to allow `'unsafe-eval'`.
- **`eval`.** It's re-created with `new Function`, so a Content-Security-Policy on your worker has to allow `'unsafe-eval'`, unless the worker turns saved handlers off ([below](#handlers-in-the-worker)).
- **Errors.** An error it throws in the worker shows only in the service worker's console, not the page's.
### Handlers in the worker
A handler that needs more, such as imported code, can be registered in the worker itself, where it takes priority over the saved one:
```js
@@ -67,8 +69,16 @@ importScripts('https://js.puter.com/push-sw.js', '/my-formatting.js');
puterPush.onPush('chat', (n) => ({ title: formatTitle(n) }));
```
To run only the handlers registered in the worker, call `puterPush.savedHandlers(false)` after `importScripts()`. The worker then never re-creates a handler from the source your pages saved, and a push for a key with no handler in the worker shows the notification's own title and text. Saved handlers are on by default, and `puterPush.savedHandlers(true)` turns them back on.
```js
importScripts('https://js.puter.com/push-sw.js', '/my-formatting.js');
puterPush.savedHandlers(false);
puterPush.onPush('chat', (n) => ({ title: formatTitle(n) }));
```
## Return value
A `Promise` that resolves to `{ off }`. Calling `off()` stops the handler in this page. When no other handler for `key` is left in the page, it also removes the saved one.
A `Promise` that resolves to `{ off }`. Calling `off()` stops the handler in this page. The saved handler is removed when no tab on this browser still uses a handler for `key`. Where the browser has no Web Locks, such as in some sandboxed frames, it stays.
It rejects with an error carrying `message` and `code`:
+16 -1
View File
@@ -56,13 +56,27 @@ importScripts('https://js.puter.com/push-sw.js');
The worker follows the browser's service worker rules:
- **One worker per scope.** If your site already has a service worker for that scope, add the `importScripts()` line to it and pass its path as `serviceWorker` to [`onPush()`](/Notifications/onPush/), [`enable()`](/Notifications/enable/), [`disable()`](/Notifications/disable/) and [`status()`](/Notifications/status/). Otherwise [`enable()`](/Notifications/enable/) rejects with `push_worker_conflict` and leaves your worker in place. Puter's script only adds event listeners and only handles pushes sent by Puter, so your worker's own install, fetch and push handling keep working.
- **One push subscription per worker.** If that worker already gets pushes from another service, such as Firebase or OneSignal, give Puter its own worker in a folder, such as `/puter/puter-sw.js`, and pass that path as `serviceWorker`. Otherwise [`enable()`](/Notifications/enable/) rejects with `push_subscription_conflict`.
- **One push subscription per worker.** If that worker already gets pushes from another service, such as Firebase or OneSignal, give Puter its own worker in a folder, such as `/puter/puter-sw.js`, and pass that path as `serviceWorker`. Otherwise [`enable()`](/Notifications/enable/) rejects with `push_subscription_conflict`. A subscription Puter made earlier, even with a key Puter has since changed, is replaced instead.
- **Content-Security-Policy.** If the worker is served with a Content-Security-Policy, its `script-src` has to allow `https://js.puter.com` and `'unsafe-eval'`. The worker re-creates your handler from its source with `new Function`.
- **Updates.** A changed worker only runs once it activates. Call `self.skipWaiting()` in it, or close every tab of your site.
- **Errors.** An error your handler throws in the worker shows only in the service worker's console, not the page's. In Chrome, open it from DevTools under **Application** > **Service workers**.
To check the worker, open `https://<your site>/puter-sw.js` in a browser tab. It should show the `importScripts` line as plain text.
#### Serving your own copy
The worker at `https://js.puter.com/push-sw.js` always serves the current version, and Puter's CDN picks up a new one within five minutes. It matches `https://js.puter.com/v2/`, which always serves the newest Puter.js, so a site that loads Puter.js from there should keep the default worker.
A site that pins Puter.js to one version, from npm or a CDN such as jsdelivr, can pin the worker to the same version. The file ships in the [`@heyputer/puter.js`](https://www.npmjs.com/package/@heyputer/puter.js) npm package as `src/push-sw.js`. Copy it to your site, or import it from a CDN that serves npm packages, with the version in the URL:
```js
importScripts('https://cdn.jsdelivr.net/npm/@heyputer/puter.js@<version>/src/push-sw.js');
```
Replace `<version>` with the version of the Puter.js your site loads, and change both together. The worker and your pages share what [`onPush()`](/Notifications/onPush/) and [`enable()`](/Notifications/enable/) save in the browser. A Content-Security-Policy on the worker has to allow the copy's origin instead of `https://js.puter.com`.
A worker can also run only the handlers registered in it, and never the source your pages saved. [`onPush()`](/Notifications/onPush/#handler-limits) shows how.
### 2. Say what a push shows
On every page load, register a handler with [`onPush()`](/Notifications/onPush/) for each kind of notification you send. The key is the `channel` you pass to [`notify()`](/Notifications/notify/).
@@ -302,6 +316,7 @@ The browser can be allowed to show notifications while the operating system hide
| It rejects with `push_revoked` | The user turned your app off in Puter's settings. Call [`requestPermission()`](/Notifications/requestPermission/) from a click first. |
| The `onError` of [`onPush()`](/Notifications/onPush/) gets `push_verification_timeout` | The worker doesn't import `push-sw.js`, or the confirmation push didn't arrive within 30 seconds. The next page load tries again. |
| A push shows the plain title and text, not your handler's result | This browser hasn't run [`onPush()`](/Notifications/onPush/) for that `channel` yet, or the handler threw, took over 5 seconds, or used an outside variable. Check the worker's console. |
| Pushes stop until the user opens your site again | The browser replaced its push subscription for your site, as browsers sometimes do. Your service worker keeps the new one, and the next page load that calls [`onPush()`](/Notifications/onPush/) registers it with Puter. Pushes sent before that don't arrive. |
| No push while your site is open in a tab | The open page received the notification live. It shows one only when it's hidden or its window isn't focused. |
| No push at all | The browser isn't running, the operating system hides its notifications ([above](#operating-system-settings)), the user reached the [push limit](/rate-limits-and-quotas/#notifications) for your app, or the user has no usage credit left. |
+1 -2
View File
@@ -23,7 +23,7 @@ A set of key/value pairs that configure the window.
* `has_head` (Boolean): if set to `true`, window will have a head which contains the icon and close, minimize, and maximize buttons.
* `height` (Float): height of window in pixels.
* `is_resizable` (Boolean): if set to `true`, user will be able to resize the window.
* `show_in_taskbar` (Boolean): if set to `true`, window will be represented in the taskbar.
* `show_in_taskbar` (Boolean): deprecated, and has no effect. Puter no longer has a taskbar.
* `title` (String): title of the window.
* `width` (Float): width of window in pixels.
@@ -46,7 +46,6 @@ A `Promise` that resolves to a window handle object with an `id` (String) proper
is_resizable: false,
has_head: true,
center: true,
show_in_taskbar: false,
})
</script>
</body>
+6 -6
View File
@@ -46,12 +46,12 @@ the window before the app's own code runs, so a service app cannot avoid briefly
appearing on screen.
The instance stays private to your app for as long as it is hidden: it has no
taskbar item and no running mark on its icon, and opening the app from the
taskbar or from Puter's app list starts a separate, ordinary instance for the
user rather than handing them the one you are talking to. It can show itself at
any time with [`puter.ui.showWindow()`](/UI/showWindow), and from that moment it
is an ordinary window — it takes its place in the taskbar, and the user can
return to it, hide it, or close it like any other. Defaults to `false`.
running mark on its icon, and opening the app from Puter's app list starts a
separate, ordinary instance for the user rather than handing them the one you
are talking to. It can show itself at any time with
[`puter.ui.showWindow()`](/UI/showWindow), and from that moment it is an
ordinary window — the user can return to it, hide it, or close it like any
other. Defaults to `false`.
A background app closes when the app that launched it closes: it was launched to
serve that app, and the user never saw it. Once it has shown itself it keeps
+36
View File
@@ -676,6 +676,18 @@ const examples = [
slug: 'events-unsubscribe',
source: '/playground/examples/events-unsubscribe.html',
},
{
title: 'Run an Inline Handler on a Folder',
description: 'Watch a folder in the background with a handler you pass inline, using the Puter.js Events API. Run it in the playground.',
slug: 'events-persistent-inline',
source: '/playground/examples/events-persistent-inline.html',
},
{
title: 'Queue Events for an Inline Handler',
description: 'Send each change to one inline handler that acknowledges it, using the Puter.js Events API. Run it in the playground.',
slug: 'events-persistent-queue-inline',
source: '/playground/examples/events-persistent-queue-inline.html',
},
{
title: 'List Schedules',
description: 'List the schedules and stored notifications an app holds for a user with the Puter.js Events API. Run and experiment with this example in the playground.',
@@ -694,6 +706,30 @@ const examples = [
slug: 'events-remove-schedule',
source: '/playground/examples/events-remove-schedule.html',
},
{
title: 'Run Code Once at a Set Time',
description: 'Schedule an inline handler to run once in two minutes with the Puter.js Events API, then remove it. Run it in the playground.',
slug: 'events-schedule-inline',
source: '/playground/examples/events-schedule-inline.html',
},
{
title: 'Run Code on Weekday Mornings',
description: 'Schedule an inline handler on a cron expression in the browser time zone with the Puter.js Events API. Run it in the playground.',
slug: 'events-schedule-cron-tz',
source: '/playground/examples/events-schedule-cron-tz.html',
},
{
title: 'Run Code Every 15 Minutes',
description: 'Schedule an inline handler at an interval with a run limit, and read what it wrote, using the Puter.js Events API. Run it in the playground.',
slug: 'events-schedule-every',
source: '/playground/examples/events-schedule-every.html',
},
{
title: 'Share a Handler Across Schedules',
description: 'Publish a named handler into your own user scope, bind two schedules to it, and update it, using the Puter.js Events API. Run it in the playground.',
slug: 'events-handlers-user-scope',
source: '/playground/examples/events-handlers-user-scope.html',
},
{
title: 'Catch Up on Notifications',
description: 'Page through notifications missed while your app was closed with the Puter.js Events API. Run it in the playground.',
@@ -0,0 +1,61 @@
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
if (!await puter.perms.request('backgroundEvents')) {
puter.print('background events were not allowed<br>');
return;
}
// (1) Publish a named handler into this user's own scope of the app
const name = `reminder-${Date.now()}`;
const v1 = await puter.events.handlers.publish(
name,
async ({ ctx, user }) => {
await user.notifications.notify({ title: `${ctx.label} (v1)` });
},
{ scope: 'user' },
);
puter.print(`published ${v1.name}: ${v1.outcome}<br>`);
// (2) Bind two schedules to it
const hour = 60 * 60 * 1000;
const schedules = [
await puter.events.onSchedule({
every: '1h',
handlerName: name,
handlerScope: 'user',
context: { label: 'Stretch' },
}),
await puter.events.onSchedule({
at: Date.now() + 2 * hour,
handlerName: name,
handlerScope: 'user',
context: { label: 'Lunch' },
}),
];
// (3) Publishing again moves both schedules to the new code
const v2 = await puter.events.handlers.publish(
name,
async ({ ctx, user }) => {
await user.notifications.notify({ title: `${ctx.label} (v2)` });
},
{ scope: 'user' },
);
puter.print(`republished: ${v2.outcome}<br>`);
// (4) The scope's handlers, and how many schedules each one carries
for (const handler of await puter.events.handlers.list({ scope: 'user' })) {
puter.print(`${handler.name}: ${handler.subscriptions} bound<br>`);
}
// (5) Cleanup: end the schedules, then remove the handler
for (const schedule of schedules) await schedule.off();
await puter.events.handlers.remove(name, { scope: 'user' });
puter.print('removed<br>');
})();
</script>
</body>
</html>
@@ -0,0 +1,36 @@
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
if (!await puter.perms.request('backgroundEvents')) {
puter.print('background events were not allowed<br>');
return;
}
// (1) A folder to watch
const dir = puter.randName();
await puter.fs.mkdir(dir);
// (2) The handler runs in the events worker for every change, even with this page closed
const sub = await puter.events.onPersistent({
subject: `fs:${dir}`,
handler: async ({ event, ctx, fetch }) => {
await fetch(ctx.webhook, { method: 'POST', body: event.path });
},
context: { webhook: 'https://example.com/ingest' },
});
puter.print(`watching ${dir} with ${sub.handlerName}<br>`);
// (3) A change to deliver
await puter.fs.write(`${dir}/hello.txt`, 'Hello!');
// (4) Cleanup: end the subscription, remove its handler, delete the folder
await puter.events.unsubscribe(sub.subId);
await puter.events.handlers.remove(sub.handlerName, { scope: 'user' });
await puter.fs.delete(dir);
puter.print('removed<br>');
})();
</script>
</body>
</html>
@@ -0,0 +1,36 @@
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
if (!await puter.perms.request('backgroundEvents')) {
puter.print('background events were not allowed<br>');
return;
}
const dir = puter.randName();
await puter.fs.mkdir(dir);
// (1) A queue: each event goes to one consumer and is offered again until it is acknowledged
const sub = await puter.events.onPersistent({
subject: `fs:${dir}`,
queue: true,
handler: async ({ event, ctx, fetch, ack }) => {
await ack();
await fetch(ctx.webhook, { method: 'POST', body: event.path });
},
context: { webhook: 'https://example.com/ingest' },
});
puter.print(`queue ${sub.queue}, handler ${sub.handlerName}<br>`);
await puter.fs.write(`${dir}/hello.txt`, 'Hello!');
// (2) Cleanup: end the subscription, remove its handler, delete the folder
await puter.events.unsubscribe(sub.subId);
await puter.events.handlers.remove(sub.handlerName, { scope: 'user' });
await puter.fs.delete(dir);
puter.print('removed<br>');
})();
</script>
</body>
</html>
@@ -0,0 +1,34 @@
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
if (!await puter.perms.request('backgroundEvents')) {
puter.print('background events were not allowed<br>');
return;
}
// (1) Weekdays at 08:00 in this browser's time zone
const schedule = await puter.events.onSchedule({
cron: '0 8 * * 1-5',
tz: Intl.DateTimeFormat().resolvedOptions().timeZone,
handler: async ({ ctx, user, fetch }) => {
const response = await fetch(ctx.url);
await user.notifications.notify({
title: ctx.title,
text: await response.text(),
});
},
context: { url: 'https://api.github.com/zen', title: 'Good morning' },
});
puter.print(`${schedule.cron} in ${schedule.tz}<br>`);
puter.print(`next run: ${new Date(schedule.nextRunAt).toLocaleString()}<br>`);
// (2) Cleanup: end the schedule, then remove the handler it created
await schedule.off();
await puter.events.handlers.remove(schedule.handlerName, { scope: 'user' });
puter.print('removed<br>');
})();
</script>
</body>
</html>
@@ -0,0 +1,35 @@
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
if (!await puter.perms.request('backgroundEvents')) {
puter.print('background events were not allowed<br>');
return;
}
// (1) Every 15 minutes, 3 times. Each run adds one to a counter in the key-value store
const key = `runs-${puter.randName()}`;
const schedule = await puter.events.onSchedule({
every: '15m',
maxRuns: 3,
handler: async ({ ctx, user }) => {
const runs = (await user.kv.get(ctx.key)) ?? 0;
await user.kv.set(ctx.key, runs + 1);
},
context: { key },
});
puter.print(`first run at ${new Date(schedule.nextRunAt).toLocaleTimeString()}<br>`);
// (2) The page reads what the handler wrote. Nothing has run yet, so this is 0
puter.print(`runs so far: ${(await puter.kv.get(key)) ?? 0}<br>`);
// (3) Cleanup: end the schedule, remove its handler, delete the counter
await schedule.off();
await puter.events.handlers.remove(schedule.handlerName, { scope: 'user' });
await puter.kv.del(key);
puter.print('removed<br>');
})();
</script>
</body>
</html>
@@ -0,0 +1,29 @@
<html>
<body>
<script src="https://js.puter.com/v2/"></script>
<script>
(async () => {
if (!await puter.perms.request('backgroundEvents')) {
puter.print('background events were not allowed<br>');
return;
}
// (1) The code goes in with the schedule, and runs in the events worker
const schedule = await puter.events.onSchedule({
at: Date.now() + 2 * 60 * 1000,
handler: async ({ ctx, user }) => {
await user.notifications.notify({ title: ctx.message });
},
context: { message: 'Two minutes have passed' },
});
puter.print(`runs at ${new Date(schedule.nextRunAt).toLocaleTimeString()}<br>`);
puter.print(`handler stored as ${schedule.handlerName}<br>`);
// (2) Cleanup: end the schedule, then remove the handler it created
await schedule.off();
await puter.events.handlers.remove(schedule.handlerName, { scope: 'user' });
puter.print('removed<br>');
})();
</script>
</body>
</html>
+20 -3
View File
@@ -23,7 +23,7 @@ Under the [User-Pays Model](/user-pays-model), limits are counted against each u
| ---------------------- | ---------------------- |
| **Per user, per app** | One user in one app. **This is the default**: a limit that names no scope uses it. The same user in another app has a separate budget. The Puter desktop and API tokens count as one more app, and each [worker](/Workers/) gets its own budget too. |
| **Per user, all apps** | One user, summed across every app they use. Used for quotas on things an account keeps (subscriptions, share handles, shares per day, uploads in progress) and a few call budgets. |
| **Per app, all users** | One app, summed across all of its users. Used for things a developer publishes, such as event handlers. |
| **Per app, all users** | One app, summed across all of its users. Used for things a developer publishes, such as event handlers. A user's own handlers in an app are counted separately, per user and per app. |
| **Per network** | One IP address or device. Used by routes with no signed-in user, such as signed URLs and WebDAV. |
When a quota has both a per-app and an all-apps number, both apply: one app can use up to its per-app share, and all apps together can use up to the all-apps total.
@@ -398,7 +398,7 @@ For long-lived mounts, sign in with a `-token` username and an API token as the
| ------------------------------------------------------------------ | ------------------------------- | ----------- |
| `nameAvailable` | 60/min | |
| App icon | 12,000/min | Per network |
| App landing page (`/app/<name>`, `/desktop/app/<name>`) | 600/min | Per network |
| App landing page (`/app/<name>`) | 600/min | Per network |
| Names per batch app lookup (`GET /apps/:names`, `POST /query/app`) | 200, up to 200 characters each | |
| `metadata` | 16 KiB, measured as JSON | |
| `filetypeAssociations` | 200 entries, 60 characters each | |
@@ -579,7 +579,7 @@ Counted per event, per region:
| New recipients of `kv.share()` | 300/min, 2,000/hour | Per user, all apps |
| Recipients named in `kv.share()` | 1,200/min | Per user, all apps |
| `kv.share.received` notifications | 5/hour | Per owner and holder |
| Handler publish / remove | 60/min | Per user, all apps |
| Handler publish / remove, app and user scope | 60/min | Per user, all apps |
| Subscription and share-handle listings (combined) | 120/min | Per user, per app |
| Handler listings | 120/min | Per user, per app |
| Events worker listings | 120/min | Per user, per app |
@@ -617,6 +617,9 @@ Unsubscribing counts toward its own `unsubscribe` limit, not the `subscribe` one
| Source for all handlers combined | 5 MB | Per app, all users |
| Handlers per `publishAll()` call | 50 | Per call |
| Events worker deploys | 30/hour | Per app, all users |
| Published handlers in a user scope | 20 | Per user, per app |
| Source for all handlers in a user scope combined | 1 MB | Per user, per app |
| Events worker deploys for a user scope | 10/hour | Per user, per app |
| Run timeout | 30 seconds | Per run |
| Retry delay (`queue`, temporary errors) | 2 seconds, doubling, up to 5 minutes | Per delivery |
| Failures in a row before suspension | 5 | Per subscription |
@@ -628,6 +631,20 @@ Only the handler's own failures count, including a run past the 30-second timeou
Over the deploy limit, deliveries stay queued and retry after the hour rolls over.
A [user scope](/Events/handlers/#user-scope) holds the handlers one user published in one app, including every inline `handler` passed to [`onSchedule()`](/Events/onSchedule/) or [`onPersistent()`](/Events/onPersistent/). Each one is its own script. The scope-specific rows above are separate from the app's: they don't count toward the app's 100 handlers or 5 MB.
- A publish over the handler cap fails with `events_handler_limit`, and one over the combined source cap with `events_worker_too_large`. A single handler over 64 KB fails with `events_handler_too_large`.
- A user-scope handler that no subscription or schedule is bound to is removed once it's more than a day old.
- Publishing into a user scope needs credit on a hosted Puter server, and fails with `insufficient_funds` (402) without it. A server that hasn't turned user scopes on answers `events_user_handlers_disabled`. Listing and removing a scope's handlers need neither.
#### Worker units
| Limit | Paid | Free | Scope |
| ---------------------------------------- | ---- | ---- | ------------------ |
| Worker units held at once | 100 | 16 | Per user, all apps |
The units are the user's [workers](/Workers/), their user scopes (one per app that holds a handler of theirs), and the [events workers](/Events/workers/) of apps they own. Only creating a new unit is refused, with a `403` and code `forbidden`. A user already over the limit keeps what they hold, and publishing into a scope they already have isn't refused.
#### Handler chains
| Limit | Scope | Paid | Free |
@@ -41,9 +41,11 @@ await puter.kv.set(`${prefix}score`, { home: 2, away: 1, final: true });
Publish a handler that alerts on the final score, and on every goal for fans
who asked for that, with
[`puter.events.handlers.publish()`](/Events/handlers/). Only the account that
owns the app can publish, so run this once yourself, signed in to your site as
the owner. The browser console is enough:
[`puter.events.handlers.publish()`](/Events/handlers/). Publishing is a
developer step, so run this once yourself, signed in to your site as the app's
owner. The browser console is enough. To skip it, pass the same function as
`handler` to each fan's `onPersistent()` call instead, which stores it in the
fan's own [user scope](/Events/handlers/#user-scope):
```js
await puter.events.handlers.publish('alertScore', async ({ event, user, ctx }) => {
@@ -14,10 +14,10 @@ snoozed task comes back to the list, a poll closes, or a shared link stops
working. On other platforms you'd store the time in a database and run a job
that checks it every minute.
In Puter, you publish a function once, and create a one-time schedule that
runs it at that time with [`puter.events.onSchedule()`](/Events/onSchedule/)
and `at`. Puter runs it in your app's [events worker](/Events/workers/), a
serverless function that belongs to your app, even while your app is closed.
In Puter, you create a one-time schedule that runs a function at that time with
[`puter.events.onSchedule()`](/Events/onSchedule/) and `at`. Puter runs it in an
[events worker](/Events/workers/), a serverless function, even while your app is
closed.
When all that's due is a notification,
[Remind Users Later](/recipes/remind-users-later/) sends one at a set time
@@ -30,10 +30,12 @@ gets a notification about it.
## Publish the Handler
The function a schedule runs is called a handler. Publish it with
[`puter.events.handlers.publish()`](/Events/handlers/). This is a developer
step, like a deploy, and it only works for the account that owns the app. Run
it from a deploy script or your browser console:
The function a schedule runs is called a handler. This recipe publishes it once
under a name with [`puter.events.handlers.publish()`](/Events/handlers/), so one
copy serves every user and you can update it for all of them. Publishing is a
developer step, like a deploy, and it only works for the account that owns the
app. Run it from a deploy script or your browser console. To skip the deploy,
[pass the handler inline](#pass-the-handler-inline) instead.
```js
const app = await puter.apps.get('task-list'); // your app's name
@@ -62,6 +64,31 @@ task as no longer snoozed with [`user.kv.update()`](/KV/update/).
[Run Code on a Schedule](/recipes/run-code-on-a-schedule/#publish-the-handler)
explains what else a handler can use.
### Pass the Handler Inline
For a one-time schedule, the handler can go straight into `onSchedule()`. Leave
out `handlerName`, and Puter stores the code in the signed-in user's own
[user scope](/Events/handlers/#user-scope) of your app:
```js
await puter.events.onSchedule({
at: Date.now() + 60 * 60 * 1000,
handler: async ({ ctx, user }) => {
const key = `task:${ctx.taskId}`;
const task = await user.kv.get(key);
if (!task) return;
await user.notifications.notify({ title: 'Back on your list', text: task.title });
await user.kv.update(key, { snoozedUntil: null });
},
context: { taskId: 'task-42' },
});
```
Puter removes the code from the user's scope about a day after the schedule
ends. To remove it sooner, see
[Clean Up Background Work](/recipes/clean-up-background-work/#remove-a-user-scope).
## Ask for Permission
A schedule runs code while the user isn't there, so it needs the user's
@@ -244,7 +271,7 @@ the schedule ends on its own after its last run.
Each run of a repeating schedule has its own `event.id`, so the handler keeps
the id of the last run it handled and skips a retry of that run. Publish it as
the app's developer, like `unsnooze`:
the app's developer, like `unsnooze`, or pass it inline as `handler`:
```js
async function reopenTask ({ event, ctx, user }) {
@@ -9,15 +9,17 @@ order: 70
An app that runs code in the background, as in
[Run Code in the Background](/recipes/run-code-in-the-background/), leaves
two kinds of things behind. Each user has persistent subscriptions in their
account, and you have the handlers you published and the events worker that
runs them. When a feature changes or goes away, the ones you no longer need can
three kinds of things behind. Each user has persistent subscriptions and
schedules in their account, and handlers they hold in their own
[user scope](/Events/handlers/#user-scope) when your app passed code inline. You
have the handlers you published and the events worker that runs them. When a feature changes or goes away, the ones you no longer need can
keep running. A subscription with a handler runs it for every event it matches,
and each run is billed to the user. An [events worker](/Events/workers/) can be
billed to you even when nothing runs on it.
This recipe removes both. Your app cleans up each user's subscriptions, and you
remove the handlers and events workers you no longer use.
This recipe removes all of them. Your app cleans up each user's subscriptions
and user scope, and you remove the handlers and events workers you no longer
use.
## Remove Stale Subscriptions
@@ -50,7 +52,9 @@ uses, and ends the rest:
- **A handler you stopped using.** Put only the handler names the current
version of your app subscribes with in `handlersInUse`. A subscription made
without a handler has `handlerName: null`, so add `null` if your app makes
those.
those. A subscription created with an inline `handler` has a `handlerName`
that starts with `@`, so match those with `sub.handlerName?.startsWith('@')`
if your app makes them.
- **Paused with `permission_revoked`.** The user took back the permission, or
lost access to what the subscription watched. It never resumes. It stays in
the list for up to 30 days unless you remove it.
@@ -61,10 +65,46 @@ Call `cleanUpSubscriptions()` when the app opens, before `startBackgroundWork()`
from [Run Code in the Background](/recipes/run-code-in-the-background/#subscribe-once-per-user),
so that function finds the subscription that's left.
## Remove a User Scope
An inline `handler` passed to [`onSchedule()`](/Events/onSchedule/) or
[`onPersistent()`](/Events/onPersistent/) is stored in the signed-in user's own
user scope of your app. Puter removes it about a day after the last
subscription or schedule bound to it is gone, so you don't have to. Until then it
counts toward the scope's [limits](/rate-limits-and-quotas/#handlers), and the
scope counts as one of the user's [worker units](/rate-limits-and-quotas/#worker-units).
To free it sooner, the
user scope is the user's, so your app removes it from the user's session, with
no owner step. [`handlers.list()`](/Events/handlers/) with `scope: 'user'` shows
what the scope holds, and
[`handlers.remove()`](/Events/handlers/) removes one:
```js
for (const handler of await puter.events.handlers.list({ scope: 'user' })) {
console.log(`${handler.name}: ${handler.subscriptions} subscriptions`);
}
await puter.events.handlers.remove('syncFeeds', { scope: 'user' });
```
Subscriptions and schedules bound to a removed handler are paused with
`handler_not_found`, as for an app handler. To remove everything the user's scope
holds and end the scope, pass the app's uid to
[`puter.events.workers.destroy()`](/Events/workers/) with `scope: 'user'`:
```js
await puter.events.workers.destroy(puter.appID, { scope: 'user' });
```
Do this after you've ended the subscriptions and schedules that use it, with
`unsubscribe()` and `removeSchedule()`. Listing and removing don't need credit,
and [`workers.list()`](/Events/workers/) shows the user's scopes with
`scope: 'user'`.
## Remove Handlers You No Longer Use
Removing a handler is a developer step, so run it from a deploy script or your
browser console, signed in as the app's owner.
Removing a published handler is a developer step, so run it from a deploy script
or your browser console, signed in as the app's owner.
[`puter.events.handlers.list()`](/Events/handlers/) shows how many
subscriptions still use each handler, and
[`puter.events.handlers.remove()`](/Events/handlers/) removes one:
@@ -93,7 +133,8 @@ gets `events_handler_not_found` from
## Remove an Events Worker
Your app's events worker is created when you publish its first handler, and
removed with its last one. To see which of your apps have one, call
removed with its last one. A user's scope is a unit of its own, and isn't part of
yours. To see which of your apps have one, call
[`puter.events.workers.list()`](/Events/workers/) outside an app, for example
in your browser console on Puter. It returns a page at a time:
@@ -199,7 +199,10 @@ closed. The handler calls [`notify()`](/Notifications/notify/) for its own
member, so it needs no share. Use this in place of `notifyRoom()`.
Publish the handler once, signed in to your site as the app's owner, with
[`puter.events.handlers.publish()`](/Events/handlers/).
[`puter.events.handlers.publish()`](/Events/handlers/). To skip the publish,
pass the same function as `handler` to each member's `onPersistent()` call
instead, which stores it in the member's own
[user scope](/Events/handlers/#user-scope).
[Run Code in the Background](/recipes/run-code-in-the-background/#publish-the-handler)
explains handlers in more detail:
@@ -38,7 +38,9 @@ timestamp.
The handler keeps its cursor in the user's key-value store, so each run picks
up where the last one stopped. Publish it as the app's developer, from a
deploy script or your browser console:
deploy script or your browser console, so one copy serves every user. To skip
the deploy, pass the same function as `handler` to `onSchedule()` instead, which
stores it in the user's own [user scope](/Events/handlers/#user-scope):
```js
const app = await puter.apps.get('order-sync'); // your app's name
+2 -1
View File
@@ -136,7 +136,8 @@ done, run a handler at that time instead with
checks the task and sends the notification itself.
Publish the handler once, as the app's developer, from a deploy script or
your browser console. [Run Code on a Schedule](/recipes/run-code-on-a-schedule/)
your browser console, or pass the same function as `handler` to `onSchedule()`
to store it in the user's own [user scope](/Events/handlers/#user-scope). [Run Code on a Schedule](/recipes/run-code-on-a-schedule/)
explains handlers and the `user` object they get:
```js
@@ -12,12 +12,12 @@ each file a user adds to a folder, or posting to a chat when a report is saved.
On other platforms you'd use a webhook, a background job, or a cloud function
that runs on upload.
In Puter, you publish a function once, and each user's persistent subscription
In Puter, you give each user's persistent subscription a function, and it
runs it when their files change. A persistent subscription, created with
[`puter.events.onPersistent()`](/Events/onPersistent/), is saved to the user's
account and keeps running after your app closes. Puter runs the function in
your app's [events worker](/Events/workers/), a serverless function that
belongs to your app, so you don't run a server. To update a page while it's
an [events worker](/Events/workers/), a serverless function, so you don't run
a server. To update a page while it's
open, use the subscriptions in
[Watch for Changes](/recipes/watch-for-changes/) instead.
@@ -27,10 +27,15 @@ so the app can list them later.
## Publish the Handler
A handler is the function a subscription runs. Publish it with
[`puter.events.handlers.publish()`](/Events/handlers/). This is a developer
A handler is the function a subscription runs. There are two ways to give one
to a subscription. The simplest is to [pass it inline](#pass-the-handler-inline),
which any signed-in user's session can do from your app. The other is to publish
it once under a name with [`puter.events.handlers.publish()`](/Events/handlers/),
and have every user's subscription bind to that name. Publishing is a developer
step, like a deploy, and only the account that owns the app can do it, so run it
from a deploy script or your browser console:
from a deploy script or your browser console. Use it when you want one copy of
the code that you can update for every user at once. This section and the ones
after it use a published handler:
```js
const app = await puter.apps.get('drop-box'); // your app's name
@@ -78,6 +83,42 @@ the key-value store. A file it writes into `Inbox` is a new change that runs the
handler again, and Puter stops that loop after a few runs without an error.
[Handler chains](/rate-limits-and-quotas/#handler-chains) has the numbers.
## Pass the Handler Inline
To skip the deploy step, pass the function as `handler` and leave out
`handlerName`. Puter stores the code in the signed-in user's own
[user scope](/Events/handlers/#user-scope) of your app, and runs it for that
user's events only:
```js
const inbox = `~/AppData/${puter.appID}/Inbox`;
const sub = await puter.events.onPersistent({
subject: `fs:${inbox}`,
queue: true,
handler: async ({ event, user }) => {
const arrived = event.op === 'add' || (event.op === 'move' && !event.from);
if (!arrived) return;
await user.kv.set(`uploads:${event.uid}`, {
name: event.path.split('/').pop(),
addedAt: event.ts,
});
},
});
```
The handler follows the same rules as a published one, and gets its settings
through `context` in the same way. The handler is private to that user, so one
user's copy can't be changed by anyone else, including you. It also counts
toward that user's [limits](/rate-limits-and-quotas/#handlers) and bills to the
user like their own workers, and publishing needs credit on a hosted Puter
server. Changing the code later makes a new handler, so the old subscription
keeps running the old code. To roll out new code, unsubscribe and subscribe
again, or [publish a named handler into the user scope](/Events/handlers/#user-scope).
[Clean Up Background Work](/recipes/clean-up-background-work/#remove-a-user-scope)
covers removing what a user's scope holds.
## Ask for Permission
Running code while your app is closed needs the user's permission. Ask from
+45 -8
View File
@@ -14,10 +14,9 @@ did. For example, sending a summary every weekday morning, clearing out old
data every night, or checking a feed every hour. On other platforms you'd use
a cron job or a scheduled cloud function.
In Puter, you write a function, publish it once, and give each user a
schedule that runs it. Puter runs it in your app's
[events worker](/Events/workers/), a serverless function that belongs to your
app, at the times the schedule says. You don't need to run a server.
In Puter, you write a function and give each user a schedule that runs it.
Puter runs it in an [events worker](/Events/workers/), a serverless function,
at the times the schedule says. You don't need to run a server.
[Run Code in the Background](/recipes/run-code-in-the-background/) runs a
function when a user's files change. This recipe runs one at set times
@@ -27,10 +26,14 @@ weekday morning, in their own time zone, with how many tasks they have open.
## Publish the Handler
The function a schedule runs is called a handler. Publish it with
[`puter.events.handlers.publish()`](/Events/handlers/). This is a developer
step, like a deploy, and it only works for the account that owns the app. Run
it from a deploy script or your browser console:
The function a schedule runs is called a handler. You can
[pass it inline](#pass-the-handler-inline) to `onSchedule()`, or publish it once
under a name with [`puter.events.handlers.publish()`](/Events/handlers/) and
have every user's schedule bind to that name. Publishing is a developer step,
like a deploy, and it only works for the account that owns the app, so run it
from a deploy script or your browser console. Use it when you want one copy of
the code that you can update for every user at once. This section and the ones
after it use a published handler:
```js
const app = await puter.apps.get('task-list'); // your app's name
@@ -68,6 +71,40 @@ when none is open and the user turned pushes on.
to set pushes up. The `group` makes each morning's summary replace the one
before it.
## Pass the Handler Inline
Without a deploy step, pass the function as `handler` and leave out
`handlerName`. Puter stores the code in the signed-in user's own
[user scope](/Events/handlers/#user-scope) of your app, and runs it for that
user's schedule only:
```js
await puter.events.onSchedule({
cron: '0 8 * * 1-5',
tz: Intl.DateTimeFormat().resolvedOptions().timeZone,
handler: async ({ user }) => {
const tasks = (await user.kv.get('tasks')) ?? [];
const open = tasks.filter((task) => !task.done);
if (open.length === 0) return;
await user.notifications.notify({
title: 'Good morning',
text: `You have ${open.length} open tasks today.`,
group: 'morning-summary',
});
},
});
```
A schedule keeps running the code it was created with. To change it, remove the
schedule and create a new one, or publish a named handler into the user scope
with `scope: 'user'` and bind schedules to it with `handlerScope: 'user'`. The
handler counts toward the user's [limits](/rate-limits-and-quotas/#handlers) and
bills to the user like their own workers, and creating the schedule needs credit
on a hosted Puter server.
[Clean Up Background Work](/recipes/clean-up-background-work/#remove-a-user-scope)
covers removing what a user's scope holds.
## Ask for Permission
A schedule runs code while the user isn't there, so it needs the user's
Loaded 100 of 188 files, more files were not shown because too many files have changed in this diff. Show more