Compare commits

..
Author SHA1 Message Date
velzie 57fb5ec606 fix(peer): grant forgery bug and stale comment 2026-11-10 22:13:53 -05:00
Nariman Jelveh d392b6857d fix(dashboard): drop the repeated add-on credits remainder from the Usage tab 2026-11-10 17:38:03 -07:00
Daniel Salazar d8c3e01dd9 fix(http): parse large JSON bodies in slices; store WebDAV uploads labeled as JSON (#4563)
JSON request bodies over 32 KiB are parsed in slices, with the event loop
running between them, instead of in one JSON.parse call. Reading, limits,
decompression, req.rawBody and the errors stay as express.json has them.
Large bodies parse one at a time.

Gemini TTS and video replies use the same parser, and their base64 is
checked and decoded 1 MiB at a time.

A profile picture is sized before its pattern runs, so one of several
megabytes gets a 413.

Body parsers skip the dav subdomain: a PUT labeled as JSON or form data
was read by them and stored empty, or refused.

puter.js sends auth_token as the first field of a driver call. A chat body
over 50 MiB that opens with a valid token is accepted up to 128 MiB, as
one with a header credential is.

JSON in a charset other than UTF-8 is now limited to 1 MiB, and a body
over 32 KiB nested deeper than 10,000 levels is refused.
2026-11-10 16:36:52 -07:00
Nariman Jelveh 4bc679956f fix(dashboard): label the Usage tab's credit bar "Monthly credits" 2026-11-10 16:28:49 -07:00
Daniel Salazar b751f75467 fix(fs): keep a file readable when two overwrites create the same path at once (#4579)
Two writers that both start before the row exists each upload under their
own object key. The second one found the first writer's row and updated its
bucket and size while keeping the first writer's uuid, so the row could name
an object that was never written under that key.

The store now leaves a row untouched when an overwrite arrives with a
different object key, and the service moves the uploaded bytes onto the
row's key before updating it. The last writer wins and the uuid is stable.
2026-11-10 16:15:42 -07:00
Daniel Salazar 224b2f2367 fix(backend): null driver args, non-string chat model, and long app-stats uid lists no longer answer 500 (#4576)
- drivers: a null `args` in a driver call is treated as no arguments
- ai-chat: a `model` that is not a string is answered as model not found (400)
- apps: app stats split a long uid list across ClickHouse queries, which cap the size of one query parameter
2026-11-10 16:08:55 -07:00
Nariman Jelveh 80bcc4ba14 feat(dashboard): show add-on credit as its own bar on the Usage tab (#4578)
The Usage tab told an account how much add-on credit it had left in a
line of text under the monthly meter. It now gets a meter of its own
there: a heading, how much of the credit bought has been spent, and a
bar, with the line saying what is left and when it is used kept under
it. An account that never had add-on credit sees the tab as before.
2026-11-10 15:11:21 -07:00
Nariman Jelveh f26201649a feat(gui): let a first-party page frame the deployment's top-up dialog (#4570)
* feat(gui): let a first-party page frame the deployment's top-up dialog

`/embed/top-up` joins the framed popups. Like the upgrade, it covers the
framing page with a transparent GUI page, serves only the page's own
user, and reports `unavailable` unless the deployment defines
`window.UITopUp` with `embeddable === true` and no account check is
waiting. It runs `new UITopUp({ onClose }).open()` and posts
`closed { toppedUp }` once the dialog closes. The frame's lifecycle is the
upgrade's, now shared by both.

`<puter-user-menu>` gets an Add credits item under Usage, shown only when
whoami reports `subscribed: true`. It frames the top-up on a page that can
frame the GUI, and otherwise opens the dashboard with `?topup=` in a new
tab, within the click. Hidden items are skipped by the menu's arrow keys.

`puter.ui.showTopUp_()` is internal; no public SDK method is added.

* fix(puter-js): show no usage-limit dialog over an open top-up

A call refused for funds while the framed top-up was open put the
usage-limit dialog in the top layer over it, covering the payment the
user was making to clear that refusal. The prompt now stays out of the
way while a top-up shows, as it does for the framed upgrade.

* fix(puter-js): leave Add credits out of the user menu for a team seat

whoami reports a seat as subscribed through its team's plan, so the menu
offered Add credits to an account that can't buy any: the dialog only
answers that the team pays. The item now stays hidden when whoami names a
team paying for the account, as the desktop's own menu already does.

* fix(puter-js): open the top-up tab on the page's own deployment

The tab the Add credits item falls back to went to puter.com whatever
GUI origin the page belongs to, so on another deployment it opened a
different one, signed in as whoever is signed in there. It now opens the
dashboard of the GUI origin, like the menu's Usage link beside it.

* fix(puter-js): return keyboard focus to the account button after Add credits

Choosing Add credits closed the menu with focus on the item it had just
hidden, and the frame restored focus to the menu's shadow host, which
can't take it, so a keyboard user was left on the page body once the
dialog closed. The menu now moves focus to its button first, and a
framed popup restores focus to the element inside a shadow root.

* docs(embed): say when the top-up dialog's open() resolves

It resolves after the account has loaded, not as soon as the dialog
shows.

* fix(puter-js): offer Add credits after an upgrade made on the page

The user menu showed Add credits by the whoami it loaded with the page.
A user who subscribed in the framed upgrade therefore found no Add
credits in the menu until a reload. A framed upgrade that reports a plan
now marks the page's cached whoami as subscribed, as a confirmed email
already is, and the menu reads it again each time it opens.

* fix(puter-js): let a top-up that failed to start be asked for again

showTopUp_ cleared its in-flight marker in a finally inside the async
function it stored. A failure before that function's first await, such
as building the dashboard address from an origin with no scheme, ran the
finally before the marker was set, so the marker kept the rejected
promise for good: every later call got the same rejection, and
topUpShowing_ went on suppressing the usage-limit dialog. The marker is
now cleared once the stored promise settles.

* fix(puter-js): keep requestUpgrade from framing the upgrade over an open top-up

The usage-limit dialog already stays away while the top-up shows, but a
page's own requestUpgrade() call, such as one in the handler of a call
refused mid-payment, still mounted the upgrade frame. Frames are single
per path, so it covered the top-up and the payment being typed in it,
and the two dialogs' own guard can't see across documents. The call now
waits for the top-up and resolves { upgraded: false } once it closes.

* fix(puter-js): offer Add credits after an upgrade when whoami was read again

The user menu keeps the whoami object it loaded and ignores a token set
again unchanged, but the SDK reads whoami into a new object each time a
token is set. A framed upgrade then marked the new object as subscribed
and the menu went on reading the old one, so Add credits stayed hidden
until a reload. The menu now reads the SDK's cached whoami when it is
the same account's.

* docs(embed): list the top-up tests among the checks to run
2026-11-10 14:53:34 -07:00
Nariman Jelveh 1550be358d fix(explorer): navigation, keyboard, drag-and-drop, realtime and layout fixes (#4569)
* fix(explorer): record typed address-bar paths in navigation history and canonicalize them

Pressing Enter in the path input now pushes the destination onto the
window's Back/Forward history like every other navigation, and resolves
the entry first (trim, ~ and home-relative expansion, trailing or doubled
slashes) so the window's path matches the form the rest of the GUI compares
against.

* fix(explorer): keep the address-bar input in step with the Shared view and the root

The path input was only updated after a successful stat, so in views that
are never statted it kept the previous folder's path and Enter navigated
back there.

* fix(explorer): drop the previous folder's identity when a folder fails to load

A rejected stat was never handled, so the window kept the previous folder's
uid, title and item count while showing the load error, and layout or sort
changes were saved to the wrong folder. The window now clears them, and the
failed request no longer surfaces as an unhandled rejection.

* fix(explorer): release a window's selection listeners and history when it closes

Each folder window left its rubber-band selection instance, a document
mouseup handler and its navigation history behind after closing, so every
later mouse press ran once per window ever opened. Windows that are not
folders no longer bind the handler at all.

* fix(explorer): anchor a Ctrl/Cmd rubber-band selection where the drag started

The rectangle's origin was only recorded for unmodified drags, so adding to
a selection drew it from the previous drag's start point or the top-left
corner.

* fix(explorer): keep arrow-key navigation on visible items and a valid anchor

Arrow keys could land on a hidden dotfile and then stop moving, threw when
the remembered anchor belonged to another window, and left the footer's
selected count stale.

* fix(explorer): ignore non-character keys in type-to-select

Shift, arrows and Enter were appended to the search term by name, so typing
a capital letter or typing right after navigating matched nothing. Hidden
files are no longer matched, and the footer count follows the container the
selection changed in.

* fix(explorer): make the keyboard permanent-delete confirmation work in every language

The confirmation compared the dialog's answer with the English button label,
so in any other language confirming did nothing.

* fix(explorer): paste once per Ctrl+V press

The shortcut ran on both keydown and keyup, so releasing V before Ctrl
pasted the clipboard twice.

* fix(explorer): keep selection shortcuts working after navigating, and out of text fields

Select All, Copy, Cut and Delete looked up the folder from the element last
pressed, which is a removed item or a sidebar entry after navigating, so the
shortcuts did nothing. Select All also skips hidden files, and Delete keys
typed in a text field are left to the field.

* fix(explorer): count only visible items in the footer and drop hidden files from the selection

The item count included dotfiles that were not shown, and hiding hidden
files left them selected, so a later Delete or Cut acted on files the user
could not see.

* fix(explorer): paste cut items after their folder is no longer displayed

Paste resolved the cut paths to rows on screen, so cutting a file and then
navigating the same window to the destination moved nothing and silently
emptied the clipboard. Items without a row are now looked up and moved the
same way.

* fix(explorer): show a moved item once in its destination

The tab that moves an item renders the row and also receives the move
event, so the destination showed the item twice whenever the event arrived
first. Both paths now replace any row the other already rendered.

* fix(explorer): sort names the way the dashboard does, with a stable tie-break

Folder windows and the desktop compared raw lowercase strings, so file10
came before file2 and accented names landed after z, and equal sizes, dates
or types kept whatever order they arrived in. They now use the dashboard's
comparator, so the same folder reads the same in both.

* fix(explorer): keep the Trash window's states when Trash is emptied elsewhere

Emptying Trash from another tab cleared the whole container, including its
empty, error and loading elements, and left the item count stale.

* fix(explorer): remove rows deleted elsewhere instead of hiding them

A deletion from another tab or an app only faded the row out, so it still
counted in the footer, in Select All and in keyboard actions. The row is now
removed, and windows showing the deleted folder close even when its own row
is not displayed anywhere.

* fix(explorer): refresh size and date when a file is overwritten elsewhere

An overwrite arrives as an update event, which only applied the name, icon
and path, so the size and modified columns and sorting kept old values.

* fix(explorer): download files whose names contain &, #, % or +

The path went into the download URL unencoded, so the server received a
truncated path and the download failed.

* fix(explorer): allow moving into folders whose name starts with Trash

The deleted-folder check matched any sibling of Trash sharing its prefix,
so moving into a folder such as Trash-old was refused.

* fix(explorer): end the extract operation when an archive is corrupt or has no files

A corrupt zip threw before its error could be shown, and an archive with
nothing to write or a failed upload never closed the progress window. A
progress update arriving before the window existed also threw.

* fix(explorer): follow renamed and moved folders in Back/Forward and on undo

A window's navigation history kept the old path after a folder was renamed
or moved, so Back led to a load error. Undoing a folder rename also left
windows and items inside it on the renamed path, because the undo retargeted
from the path it was restoring.

* fix(explorer): keep one progress window per copy or move

The progress timer is re-armed after every conflict dialog, and it opened a
second window when one was already showing, leaving the first on screen for
good.

* fix(explorer): show file names once-escaped in move alerts

The name was escaped before being passed to an alert that escapes its whole
message, so names with &, quotes or angle brackets showed entity text.

* fix(explorer): don't flash the empty-folder message while a folder refreshes

Clearing the rows for a refresh showed the message until the listing came
back, even for a folder full of items.

* fix(explorer): accept drops into a window that covers a desktop folder

A desktop folder hidden under a window still reacted to the drag and
disabled the window as a drop target, so the drop was ignored. A drop that
bailed out early also left every folder view disabled as a target until the
next hover; targets are now restored when the drag ends.

* fix(explorer): release each row's document click handler when the row is removed

Every file row bound a click handler on document that was never unbound, so
browsing folders grew the number of handlers run on every click and kept the
removed rows alive.

* fix(explorer): let Delete work on a file after its rename was cancelled

Cancelling a rename with Escape, or leaving the editor without a change,
left the item marked as being renamed, and the Delete shortcut skips such
items.

* fix(explorer): offer a download when no app can open a file

The prompt called a translation import that is not a function, so opening
a file with no suitable app threw and did nothing.

* fix(explorer): use an item's current path and name in its context menu

Cut, Copy, Download, Create Shortcut and the rename editor read the path and
name captured when the row was created. After a rename made from another
window or tab they acted on the old path, and closing the editor repainted
the old name.

* fix(explorer): offer Cut only where the item can be moved

The item menu on the desktop and in folder windows offered Cut on trashed
items and on shared items the user may not move, unlike the same menu built
for the dashboard.

* fix(explorer): give a folder named *.app a folder icon

The app-shortcut branch ran before the directory branch, so such a folder
was read as a file on every listing, failed, and showed the app icon.

* fix(explorer): make New > JPEG Image create an actual JPEG

The blank image was encoded with the canvas default, so the .jpg file held
transparent PNG data.

* fix(explorer): handle a drop onto a folder that is part of the dragged selection

Dragging a selection onto one of its own selected folders also tried to move
that folder into itself. On the desktop the drop also cleared the saved
position of the target folder instead of the items that left the desktop.

* fix(explorer): don't highlight a window as a drop target for its own items

The check meant to skip the folder an item is already in compared the wrong
path, so dragging inside a window outlined it although dropping there does
nothing.

* fix(explorer): keep desktop rearranging to the desktop itself

Dropping an item from a window showing the Desktop folder onto the desktop
background positioned the window's row absolutely and saved that position.

* fix(explorer): empty Trash from its window in one request

The Trash window's own Empty Trash sent one delete request per item and
reported Trash as empty before any of them finished. It now runs the same
single operation as the taskbar and desktop.

* fix(explorer): label the root and Shared in the Back/Forward menus

The long-press history menus showed an empty row for the root folder and a
lowercase path segment for Shared, and the Forward menu's Home entry was not
translated.

* fix(explorer): keep sidebar labels translated and titled after reordering

A reordered sidebar rendered the labels saved at the time of the drag, so
they stayed in the old language, and every item's tooltip was empty.

* fix(explorer): enable the Open dialog's button only for a selected file

The button stayed enabled after navigating to another folder, and a
rubber-band selection of folders alone enabled it, although clicking it then
did nothing.

* fix(explorer): show the slow-load message only for a load that is slow

Once any folder took over a second to load, the spinner text switched to
the slow-load message three seconds later even after the load finished, and
every later load in that window opened with it. The text is also translated
now.

* fix(explorer): keep the current folder visible in a long breadcrumb

A path too long for the bar was cut at its end, hiding the folder being
viewed and its nearest parents. It is now cut at the root end.

* fix(explorer): grey out the whole selected row in an unfocused details view

Only the name lost its highlight when the window lost focus, leaving a grey
pill inside a blue row.

* fix(explorer): align the details header with its rows and mark it clickable

The header ended 10px before the rows and the selection highlight, and its
sortable column titles kept the default cursor.

* fix(explorer): give the layout button a tooltip and a pointer cursor

It was the only navbar control without either, so it did not read as
clickable.

* fix(explorer): remove the stray tooltip from the loading spinner

The spinner's SVG carried an internal title that showed as a tooltip and
was read out by screen readers.

* fix(explorer): refuse uploads, copies and shortcuts into the Shared view

Dropping files on Shared in the sidebar, or Ctrl/Alt-dropping items onto a
window showing it, sent requests against a path that is not a directory and
ended in an error or a stuck progress window.

* fix(explorer): open an item's context menu where it was requested

The menu is built after permission and app lookups, and it opened wherever
the pointer had moved to by the time they finished.

* fix(explorer): close a folder's window when the folder is deleted, however it was opened

Windows opened by path, such as with Ctrl-click on a breadcrumb or sidebar
entry, never received the marker used to find a deleted folder's windows,
so they stayed open on a folder that no longer existed.
2026-11-10 11:23:30 -07:00
55 changed files with 3095 additions and 326 deletions

No files matched your search

+31 -1
View File
@@ -259,6 +259,36 @@ no user to name), puter.js opens the dashboard's upgrade page in a new tab with
what was refused, through a consent dialog when the click has expired or the
tab is blocked, and `requestUpgrade()` resolves `{ upgraded: false }`.
## Top-up
`GET /embed/top-up` is the deployment's top-up dialog, where a user with a plan
buys add-on credit, served and framed like the upgrade: a transparent GUI page
covering the host with no box or close button, allowing `payment` and
`otp-credentials`, with Escape left to the dialog. Its fragment names `user`.
puter.js frames it for the **Add credits** item of `<puter-user-menu>`, which
shows only when whoami reports `subscribed: true` and no `team` paying for the
account. On a page that can't frame the GUI, the item opens the dashboard with
`?topup=` in a new tab instead.
The GUI reports `unavailable` unless its session belongs to `user`, the account
has no verification or password change waiting, and the deployment defines
`window.UITopUp` with `UITopUp.embeddable === true`. Otherwise it runs:
```js
new UITopUp({ onClose }).open();
```
`open()` resolves once the dialog has loaded the account and shows what it can
buy, and `onClose({ toppedUp })` must run exactly once, when the dialog closes
or when it declines to open for the account. The GUI then posts `{ msg: 'puterPopupFrame', state: 'closed', toppedUp }`
and the page removes the frame. When the frame can't show the top-up, puter.js
opens the dashboard's top-up in a new tab, as it does the upgrade.
While the top-up shows, a call refused for funds still rejects but opens no
usage limit dialog over it, and `requestUpgrade()` frames nothing over it: it
resolves `{ upgraded: false }` once the top-up closes.
## Email confirmation
`GET /embed/request-email-confirmation` walks a first-party page's user through
@@ -299,7 +329,7 @@ Chromium installed), and the backend route tests:
```sh
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser src/feedbackEmbed
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/modules/UI.showTopUp.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
npx vitest run --config src/gui/vitest.config.js src/gui/src/helpers/popupFrame.test.js
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
```
@@ -501,6 +501,29 @@ describe('DriverController.#handleCall (via captured router)', () => {
expect(body.service.name).toBe('puter-kvstore');
});
it('treats null `args` as no arguments', async () => {
const actor = await makeUserActor();
await server.stores.permission.setFlatUserPerm(
actor.user!.id!,
'service:puter-subdomains:ii:puter-subdomains',
{
permission: 'service:puter-subdomains:ii:puter-subdomains',
deleted: false,
issuer_user_id: actor.user!.id!,
} as never,
);
const res = makeRes();
const req = makeReq(
{ interface: 'puter-subdomains', method: 'select', args: null },
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](req, res as unknown as Response, () => {}),
);
expect((res.body as { success: boolean }).success).toBe(true);
});
it('pipes a stream-shaped result instead of JSON', async () => {
// Use an ephemeral driver registered onto the controller's bag.
const streamPayload = Buffer.from('hello-world');
@@ -276,8 +276,10 @@ export class DriverController extends PuterController {
interface: ifaceName,
method,
driver: driverName,
args = {},
args: rawArgs,
} = (req.body ?? {}) as Record<string, unknown>;
// Drivers read fields off `args`, so a null one counts as absent.
const args = rawArgs ?? {};
if (!ifaceName || typeof ifaceName !== 'string') {
throw new HttpError(400, 'Missing or invalid `interface`', {
@@ -36,9 +36,9 @@ import type {
/**
* GUI popups a first-party page can show in a frame at `/embed/<action>`,
* instead of opening `/action/<action>` in a window, plus the upgrade dialog,
* which it would otherwise open in a dashboard tab, and the email confirmation,
* which it otherwise can't show.
* instead of opening `/action/<action>` in a window, plus the upgrade and
* top-up dialogs, which it would otherwise open in a dashboard tab, and the
* email confirmation, which it otherwise can't show.
*/
export const FRAMED_POPUP_ACTIONS = [
'sign-in',
@@ -46,6 +46,7 @@ export const FRAMED_POPUP_ACTIONS = [
'show-save-file-picker',
'show-directory-picker',
'upgrade',
'top-up',
'request-email-confirmation',
] as const;
+20 -13
View File
@@ -24,11 +24,7 @@ import { computeNetworkFingerprint } from '../../core/http/middleware/rateLimit.
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterController } from '../types.js';
import { PEER_COSTS } from './costs.js';
import {
readClaimedGrantIdentifier,
signGuestGrant,
verifyGuestGrant,
} from './guestGrant.js';
import { signGuestGrant, verifyGuestGrant } from './guestGrant.js';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
@@ -191,19 +187,30 @@ export class PeerController extends PuterController {
// guests share one bucket and no host can be relayed for by
// more guests per minute than this — the only ceiling on guest
// spend we can apply before the bytes are already spent.
// A grant that doesn't parse can't name a bucket, so those
// requests fall back to the caller's own network.
// The limiter counts a request when it admits it, so only a
// grant that would redeem names a host; otherwise anyone
// holding the host's identifier could drain the bucket and
// lock out every real guest. Anything else - forged, expired,
// malformed, or no secret configured - is charged to the
// caller's own network.
rateLimit: {
scope: 'peer-guest-turn',
limit: 60,
window: 60_000,
key: (req: Request) => {
const claimed = readClaimedGrantIdentifier(
(req.body as { grant?: unknown } | undefined)
?.grant,
);
return claimed
? `host:${claimed}`
const secret =
this.config.peers?.guest_turn?.grant_secret;
const verified = secret
? verifyGuestGrant({
grant: (
req.body as
{ grant?: unknown } | undefined
)?.grant,
secret,
})
: null;
return verified?.status === 'ok'
? `host:${verified.customIdentifier}`
: `net:${computeNetworkFingerprint(req)}`;
},
},
+1 -1
View File
@@ -19,7 +19,7 @@
import { toMicroCents } from '../../services/metering/utils';
// Microcents per byte of TURN egress ($0.05/GB).
// Microcents per byte of TURN egress ($0.10/GB).
export const PEER_COSTS = {
'turn:egress-bytes': toMicroCents(0.1 / 1000 ** 3),
} as const;
@@ -1,7 +1,6 @@
import { createHmac } from 'node:crypto';
import { describe, expect, it } from 'vitest';
import {
readClaimedGrantIdentifier,
signGuestGrant,
verifyGuestGrant,
} from './guestGrant.js';
@@ -243,59 +242,3 @@ describe('signGuestGrant / verifyGuestGrant', () => {
});
});
});
describe('readClaimedGrantIdentifier', () => {
it('reads the identifier from a valid grant', () => {
const { grant } = signGuestGrant({
customIdentifier: USER_ID,
ttlSeconds: 60,
secret: SECRET,
});
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
});
it('reads the claimed identifier without checking the signature', () => {
// Bucketing runs before verification, so this is expected: a forged
// grant still names a bucket, and the handler still rejects it.
const grant = forgeGrant(
{ id: USER_ID, exp: 9_999_999_999, n: 'x' },
{ secret: 'wrong-secret' },
);
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
});
it('reads an expired grant, which the handler then rejects', () => {
const { grant } = signGuestGrant({
customIdentifier: USER_ID,
ttlSeconds: 60,
secret: SECRET,
now: 1_000_000_000_000,
});
expect(readClaimedGrantIdentifier(grant)).toBe(USER_ID);
expect(verifyGuestGrant({ grant, secret: SECRET })).toEqual({
status: 'expired',
});
});
it.each([
['a non-string', 42],
['undefined', undefined],
['a wrong-shaped string', 'not-a-grant'],
['an unknown version', 'pg2.abc.def'],
['an over-long string', `pg1.${'a'.repeat(600)}.sig`],
[
'a non-JSON payload',
`pg1.${Buffer.from('x').toString('base64url')}.sig`,
],
])('returns null for %s', (_label, grant) => {
expect(readClaimedGrantIdentifier(grant)).toBeNull();
});
it('returns null for an identifier of the wrong shape', () => {
const grant = forgeGrant({ id: 'nope', exp: 9_999_999_999 });
expect(readClaimedGrantIdentifier(grant)).toBeNull();
});
});
@@ -171,37 +171,3 @@ export const verifyGuestGrant = ({
expiresAt: payload.exp,
};
};
/**
* Read the claimed identifier out of a grant _without_ verifying it, for
* rate-limit bucketing only.
*
* Bucketing has to happen before the handler runs, and a forged grant is
* rejected there without reaching the upstream API — so an unverified read is
* enough to put a host's guests in one bucket, and the worst a forger achieves
* is choosing which bucket their own rejections land in. Never use this to
* decide attribution.
*
* @returns The claimed identifier, or null if the grant doesn't parse.
*/
export const readClaimedGrantIdentifier = (grant: unknown): string | null => {
if (typeof grant !== 'string' || grant.length > MAX_GRANT_LENGTH) {
return null;
}
const parts = grant.split('.');
if (parts.length !== 3 || parts[0] !== GRANT_VERSION) return null;
try {
const payload = JSON.parse(
Buffer.from(parts[1]!, 'base64url').toString('utf8'),
) as { id?: unknown };
if (
typeof payload?.id !== 'string' ||
!IDENTIFIER_RE.test(payload.id)
) {
return null;
}
return payload.id;
} catch {
return null;
}
};
@@ -0,0 +1,89 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import {
setupPuterTestEnv,
TEST_USER_CREDENTIALS,
type PuterTestEnv,
} from '../../testUtil.js';
/** A PUT body is the file, whatever content type the client labels it with. */
describe('PUT stores the body as sent', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 60_000);
afterAll(async () => {
await env?.shutdown();
});
const dav = (path: string, init: RequestInit) => {
const port = new URL(env.apiOrigin).port;
const auth = Buffer.from(
`${TEST_USER_CREDENTIALS.username}:${TEST_USER_CREDENTIALS.password}`,
).toString('base64');
return fetch(`http://dav.puter.localhost:${port}${encodeURI(path)}`, {
...init,
headers: { Authorization: `Basic ${auth}`, ...init.headers },
});
};
it.each([
['application/json', '{"name":"report","rows":[1,2,3]}'],
// Not JSON at all: the label alone must not get the body rejected.
['application/json', 'not json'],
['text/plain;actually=json', '{"a":1}'],
['application/x-www-form-urlencoded', 'a=1&b=two'],
['application/octet-stream', 'plain bytes'],
])(
'with Content-Type %s',
async (contentType, content) => {
const path = `/${env.users.user.username}/Documents/put-${randomUUID().slice(0, 8)}.bin`;
const put = await dav(path, {
method: 'PUT',
headers: { 'Content-Type': contentType },
body: content,
});
expect(put.status).toBe(201);
const get = await dav(path, { method: 'GET' });
expect(get.status).toBe(200);
expect(await get.text()).toBe(content);
},
30_000,
);
it('with a form body over the form parser limit', async () => {
const path = `/${env.users.user.username}/Documents/put-${randomUUID().slice(0, 8)}.bin`;
const content = `a=${'x'.repeat(200 * 1024)}`;
const put = await dav(path, {
method: 'PUT',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: content,
});
expect(put.status).toBe(201);
const get = await dav(path, { method: 'GET' });
expect(await get.text()).toBe(content);
}, 30_000);
});
@@ -0,0 +1,263 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { brotliCompressSync, deflateSync, gzipSync } from 'node:zlib';
import express, {
type ErrorRequestHandler,
type RequestHandler,
} from 'express';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { SLICE_BYTES } from '../../../util/slicedJson';
import { createJsonBodyParser } from './jsonBody';
const LIMIT = 4 * 1024 * 1024;
const TEXT_AS_JSON = 'text/plain;actually=json';
type Verify = NonNullable<Parameters<typeof express.json>[0]>['verify'];
const captureRawBody: Verify = (req, _res, buf) => {
(req as { rawBody?: Buffer }).rawBody = buf;
};
/** An app that answers with what its parsers left on the request. */
const listen = (parsers: RequestHandler[]): Promise<Server> => {
const app = express();
app.use(...parsers);
app.post('/', (req, res) => {
const rawBody = (req as { rawBody?: Buffer }).rawBody;
res.json({
body: req.body,
hasBody: req.body !== undefined,
rawBody: rawBody?.toString('base64'),
});
});
const onError: ErrorRequestHandler = (err, _req, res, _next) => {
res.status(err.statusCode ?? 500).json({ type: err.type });
};
app.use(onError);
return new Promise((resolve) => {
const server = app.listen(0, '127.0.0.1', () => resolve(server));
});
};
describe('createJsonBodyParser', () => {
let sliced: Server;
let framework: Server;
beforeAll(async () => {
sliced = await listen([
createJsonBodyParser({ limit: LIMIT, verify: captureRawBody }),
]);
// What the server ran before: one parser per content type.
framework = await listen([
express.json({ limit: LIMIT, verify: captureRawBody }),
express.json({
limit: LIMIT,
type: (req) => req.headers['content-type'] === TEXT_AS_JSON,
verify: captureRawBody,
}),
]);
});
afterAll(() => {
sliced?.close();
framework?.close();
});
const post = async (
server: Server,
body: string | Buffer | undefined,
headers: Record<string, string>,
) => {
const { port } = server.address() as AddressInfo;
const response = await fetch(`http://127.0.0.1:${port}/`, {
method: 'POST',
headers,
body,
});
return { status: response.status, answer: await response.json() };
};
const large = `${JSON.stringify({
text: 'é😀 "quoted" \\ line\n'.repeat(SLICE_BYTES / 8),
list: Array.from({ length: 20_000 }, (_, i) => ({
i,
on: i % 2 === 0,
})),
}).slice(0, -1)},"__proto__":{"nested":true}}`;
it.each<[string, string | Buffer | undefined, Record<string, string>]>([
['an object', '{"a":1}', { 'Content-Type': 'application/json' }],
['an array', '[1,2]', { 'Content-Type': 'application/json' }],
[
'a body sent as text',
'{"interface":"x","auth_token":"t"}',
{ 'Content-Type': TEXT_AS_JSON },
],
['a large body', large, { 'Content-Type': 'application/json' }],
['a large body sent as text', large, { 'Content-Type': TEXT_AS_JSON }],
[
'a charset parameter',
large,
{ 'Content-Type': 'application/json; charset=UTF-8' },
],
[
'a quoted charset parameter',
'{"a":"é"}',
{ 'Content-Type': 'application/json;charset="utf-8"' },
],
[
'another parameter',
'{"a":1}',
{ 'Content-Type': 'application/json; profile=x' },
],
['an empty body', '', { 'Content-Type': 'application/json' }],
['no body', undefined, { 'Content-Type': 'application/json' }],
[
'a byte order mark',
`\ufeff${large}`,
{ 'Content-Type': 'application/json' },
],
[
'only a byte order mark',
'\ufeff',
{ 'Content-Type': 'application/json' },
],
['only whitespace', ' \n', { 'Content-Type': 'application/json' }],
['a bare string', '"text"', { 'Content-Type': 'application/json' }],
['a bare number', ' 12', { 'Content-Type': 'application/json' }],
['null', 'null', { 'Content-Type': 'application/json' }],
['malformed JSON', '{"a":', { 'Content-Type': 'application/json' }],
[
'malformed large JSON',
`${large.slice(0, -1)},]`,
{ 'Content-Type': 'application/json' },
],
[
'a large bare string',
JSON.stringify('x'.repeat(4 * SLICE_BYTES)),
{ 'Content-Type': 'application/json' },
],
[
'invalid UTF-8',
Buffer.concat([
Buffer.from('{"a":"'),
Buffer.from([0xff, 0xe2, 0x82]),
Buffer.from('"}'),
]),
{ 'Content-Type': 'application/json' },
],
[
'a body over the limit',
`["${'x'.repeat(LIMIT)}"]`,
{ 'Content-Type': 'application/json' },
],
[
'a gzipped body',
gzipSync(large),
{ 'Content-Type': 'application/json', 'Content-Encoding': 'gzip' },
],
[
'a deflated body',
deflateSync(large),
{
'Content-Type': 'application/json',
'Content-Encoding': 'deflate',
},
],
[
'a brotli body',
brotliCompressSync(large),
{ 'Content-Type': 'application/json', 'Content-Encoding': 'br' },
],
[
'a body that inflates past the limit',
gzipSync(`["${'x'.repeat(LIMIT)}"]`),
{ 'Content-Type': 'application/json', 'Content-Encoding': 'gzip' },
],
[
'an unknown content encoding',
'{"a":1}',
{ 'Content-Type': 'application/json', 'Content-Encoding': 'lzw' },
],
['plain text', '{"a":1}', { 'Content-Type': 'text/plain' }],
[
'text with the marker and a charset',
'{"a":1}',
{ 'Content-Type': `${TEXT_AS_JSON}; charset=utf-8` },
],
[
'a JSON-suffixed type',
'{"a":1}',
{ 'Content-Type': 'application/vnd.api+json' },
],
[
'form data',
'a=1',
{ 'Content-Type': 'application/x-www-form-urlencoded' },
],
['no content type', '{"a":1}', { 'Content-Type': '' }],
])('answers %s the way express.json does', async (_name, body, headers) => {
const got = await post(sliced, body, headers);
const want = await post(framework, body, headers);
expect(got).toEqual(want);
});
it('leaves a body in another charset unread', async () => {
for (const charset of ['utf-16', 'utf8', 'latin1']) {
const { status, answer } = await post(
sliced,
Buffer.from('{"a":1}', 'utf16le'),
{ 'Content-Type': `application/json; charset=${charset}` },
);
expect(status).toBe(200);
expect(answer.hasBody).toBe(false);
}
// The last charset named is the one that counts.
const { answer } = await post(sliced, '{"a":1}', {
'Content-Type': 'application/json; charset=utf-8; charset=utf-16',
});
expect(answer.hasBody).toBe(false);
});
it('parses bodies that arrive together, each to its own result', async () => {
const bodies = Array.from({ length: 6 }, (_, n) =>
JSON.stringify({ n, pad: 'x'.repeat((n + 1) * SLICE_BYTES) }),
);
const answers = await Promise.all(
bodies.map((body) =>
post(sliced, body, { 'Content-Type': 'application/json' }),
),
);
answers.forEach(({ status, answer }, n) => {
expect(status).toBe(200);
expect(answer.body).toEqual(JSON.parse(bodies[n]!));
});
// A failed parse hands its turn on like a finished one.
const mixed = await Promise.all([
post(sliced, `${bodies[5]!},`, {
'Content-Type': 'application/json',
}),
post(sliced, bodies[4], { 'Content-Type': 'application/json' }),
post(sliced, bodies[3], { 'Content-Type': 'application/json' }),
]);
expect(mixed.map(({ status }) => status)).toEqual([400, 200, 200]);
});
});
@@ -0,0 +1,127 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import express, { type Request, type RequestHandler } from 'express';
import { parseJsonInSlices, SLICE_BYTES } from '../../../util/slicedJson';
/** What puter.js labels a JSON driver call, to keep it a simple CORS request. */
const TEXT_AS_JSON = 'text/plain;actually=json';
const UTF8_CHARSET = /;\s*charset=("?)utf-8\1\s*(?:;|$)/i;
/** No charset, or exactly UTF-8. Anything else is left to `express.json`. */
const declaresUtf8 = (contentType: string): boolean => {
const mentions = contentType.match(/charset/gi)?.length ?? 0;
return mentions === 0 || (mentions === 1 && UTF8_CHARSET.test(contentType));
};
const isUtf8Json = (req: Request): boolean => {
const contentType = req.headers['content-type'];
if (contentType === TEXT_AS_JSON) return true;
return (
typeof contentType === 'string' &&
Boolean(req.is('application/json')) &&
declaresUtf8(contentType)
);
};
/** The error `express.json` raises, which the error handler answers with a 400. */
const parseFailed = (message: string): Error =>
Object.assign(new SyntaxError(message), {
status: 400,
statusCode: 400,
expose: true,
type: 'entity.parse.failed',
});
const waiting: Array<() => void> = [];
let parsing = false;
/**
* Bodies too large for one `JSON.parse` call parse one at a time, so a single
* tree is under construction however many arrive together.
*/
const parseInTurn = async (body: Buffer): Promise<unknown> => {
if (body.length <= SLICE_BYTES) return parseJsonInSlices(body);
if (parsing) await new Promise<void>((resolve) => waiting.push(resolve));
parsing = true;
try {
return await parseJsonInSlices(body);
} finally {
// Handing over keeps `parsing` set, so a new arrival can't cut in.
const next = waiting.shift();
if (next) next();
else parsing = false;
}
};
interface JsonBodyOptions {
limit: number;
verify?: NonNullable<Parameters<typeof express.raw>[0]>['verify'];
}
/**
* `express.json` for UTF-8 bodies, with the parse done in slices so a large
* body can't hold the event loop. Reading, limits, decompression and `verify`
* are the framework's; the result and the errors match `express.json`.
*/
export const createJsonBodyParser = ({
limit,
verify,
}: JsonBodyOptions): RequestHandler => {
const read = express.raw({
limit,
verify,
type: (req) => isUtf8Json(req as Request),
});
return (req, res, next) => {
read(req, res, async (err?: unknown) => {
// Not a Buffer: the reader skipped this request.
if (err || !Buffer.isBuffer(req.body)) {
next(err);
return;
}
let body: Buffer = req.body;
req.body = undefined;
// The framework's decoder drops a byte order mark.
if (body[0] === 0xef && body[1] === 0xbb && body[2] === 0xbf) {
body = body.subarray(3);
}
if (body.length === 0) {
req.body = {};
next();
return;
}
let parsed: unknown;
try {
parsed = await parseInTurn(body);
} catch (e) {
next(parseFailed((e as Error).message));
return;
}
if (typeof parsed !== 'object' || parsed === null) {
next(parseFailed('JSON body must be an object or an array'));
return;
}
req.body = parsed;
next();
});
};
};
@@ -0,0 +1,184 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { request, type Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { setImmediate as nextTurn } from 'node:timers/promises';
import express, {
type ErrorRequestHandler,
type RequestHandler,
} from 'express';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createJsonBodyParser } from './jsonBody';
import {
createLargeJsonBodyParser,
DEFAULT_JSON_LIMIT_BYTES,
} from './largeJsonBody';
const MiB = 1024 * 1024;
const TOKEN = 'valid-token';
const PAD = 'x'.repeat(DEFAULT_JSON_LIMIT_BYTES);
/** Stands in for the auth probe: one token is valid, wherever it is sent. */
const authProbe: RequestHandler = (req, _res, next) => {
const token =
(req.body as { auth_token?: string } | undefined)?.auth_token ??
req.headers.authorization;
if (!req.actor && token === TOKEN) {
req.actor = { user: { uuid: 'u1' } } as typeof req.actor;
}
next();
};
describe('a JSON body over the default limit', () => {
let server: Server;
beforeAll(async () => {
const app = express();
app.post(
'/call',
createLargeJsonBodyParser({
authProbe,
limitBytes: 128 * MiB,
budgetBytes: 256 * MiB,
perUserBytes: 128 * MiB,
}),
);
app.use(createJsonBodyParser({ limit: DEFAULT_JSON_LIMIT_BYTES }));
app.use(authProbe);
app.post('/call', (req, res) => {
const body = req.body as { auth_token?: string; pad: string };
res.json({
user: req.actor?.user.uuid ?? null,
token: body.auth_token ?? null,
padLength: body.pad.length,
});
});
const onError: ErrorRequestHandler = (err, _req, res, _next) => {
res.status(err.statusCode ?? 500).json({});
};
app.use(onError);
await new Promise<void>((resolve) => {
server = app.listen(0, '127.0.0.1', resolve);
});
});
afterAll(() => {
server?.close();
});
/** Sends `chunks` one write at a time, so the server sees them apart. */
const post = (chunks: string[], headers: Record<string, string> = {}) =>
new Promise<{ status: number; answer: Record<string, unknown> }>(
(resolve, reject) => {
const { port } = server.address() as AddressInfo;
const req = request(
{
host: '127.0.0.1',
port,
path: '/call',
method: 'POST',
headers: {
'Content-Type': 'text/plain;actually=json',
'Content-Length': chunks.reduce(
(total, chunk) =>
total + Buffer.byteLength(chunk),
0,
),
...headers,
},
},
(res) => {
const parts: Buffer[] = [];
res.on('data', (part: Buffer) => parts.push(part));
res.on('end', () =>
resolve({
status: res.statusCode!,
answer: JSON.parse(
Buffer.concat(parts).toString(),
),
}),
);
},
);
req.on('error', reject);
void (async () => {
for (const chunk of chunks) {
req.write(chunk);
await nextTurn();
}
req.end();
})();
},
);
it('is accepted when it opens with a valid auth_token', async () => {
const { status, answer } = await post([
`{"auth_token":"${TOKEN}","pad":"${PAD}"}`,
]);
expect(status).toBe(200);
expect(answer).toEqual({
user: 'u1',
token: TOKEN,
padLength: PAD.length,
});
}, 60_000);
it('is accepted when the token arrives in pieces, with spacing', async () => {
const opening = ` {\n "auth_token" : "${TOKEN}" ,`;
const { status, answer } = await post([...opening, `"pad":"${PAD}"}`]);
expect(status).toBe(200);
expect(answer.user).toBe('u1');
}, 60_000);
it('is accepted with a header credential, as before', async () => {
const { status, answer } = await post([`{"pad":"${PAD}"}`], {
Authorization: TOKEN,
});
expect(status).toBe(200);
expect(answer).toEqual({
user: 'u1',
token: null,
padLength: PAD.length,
});
}, 60_000);
it.each([
[
'its token comes after other fields',
`{"pad":"${PAD}","auth_token":"${TOKEN}"}`,
],
['its token is not valid', `{"auth_token":"expired","pad":"${PAD}"}`],
['it has no token', `{"pad":"${PAD}"}`],
[
'a later field replaces the token it opens with',
`{"auth_token":"${TOKEN}","pad":"${PAD}","auth_token":"other"}`,
],
[
'the token it opens with is nested',
`{"args":{"auth_token":"${TOKEN}"},"pad":"${PAD}"}`,
],
])(
'is refused with 413 when %s',
async (_name, body) => {
expect((await post([body])).status).toBe(413);
},
60_000,
);
});
@@ -18,6 +18,7 @@
*/
import { EventEmitter } from 'node:events';
import { Readable } from 'node:stream';
import type { NextFunction, Request, RequestHandler, Response } from 'express';
import { describe, expect, it, vi } from 'vitest';
import { isHttpError } from '../HttpError';
@@ -35,7 +36,7 @@ const makeReq = (
user: string | null = 'u1',
body?: unknown,
) =>
({
Object.assign(Readable.from([]), {
complete: true,
body,
headers: {
+115 -65
View File
@@ -17,13 +17,54 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import express, { type Request, type RequestHandler } from 'express';
import type { Request, RequestHandler, Response } from 'express';
import { actorUid } from '../../actor';
import { HttpError } from '../HttpError';
import { createJsonBodyParser } from './jsonBody';
/** The global JSON parser's limit, which applies before auth. */
export const DEFAULT_JSON_LIMIT_BYTES = 50 * 1024 * 1024;
/** How much of a body is read to find a token it opens with. */
const LEADING_TOKEN_BYTES = 8 * 1024;
const LEADING_TOKEN =
/^[ \t\n\r]*\{[ \t\n\r]*"auth_token"[ \t\n\r]*:[ \t\n\r]*"([^"\\]+)"/;
/**
* The first `bytes` of the body, put back for the parser to read. Empty when
* the body ends or fails before that many arrive.
*/
const peekBody = (req: Request, bytes: number): Promise<Buffer> =>
new Promise((resolve) => {
const settle = (head: Buffer | null) => {
req.off('readable', onReadable);
req.off('end', onGone);
req.off('close', onGone);
req.off('error', onGone);
if (head) req.unshift(head);
resolve(head ?? Buffer.alloc(0));
};
const onReadable = () => {
// Null until `bytes` are buffered.
const head = req.read(bytes) as Buffer | null;
if (head) settle(head);
};
const onGone = () => settle(null);
req.on('readable', onReadable);
req.on('end', onGone);
req.on('close', onGone);
req.on('error', onGone);
});
/**
* The `auth_token` a JSON body opens with, which puter.js sends first so it can
* be read ahead of the body.
*/
const leadingBodyToken = async (req: Request): Promise<string | undefined> => {
const head = await peekBody(req, LEADING_TOKEN_BYTES);
return LEADING_TOKEN.exec(head.toString('latin1'))?.[1];
};
interface LargeJsonBodyOptions {
/** The global auth probe; it skips requests that already carry an actor. */
authProbe: RequestHandler;
@@ -38,9 +79,9 @@ interface LargeJsonBodyOptions {
/**
* Parses JSON bodies over the default limit, up to `limitBytes`, once the
* caller's header or cookie credential resolves. Unauthenticated requests and
* requests without a Content-Length fall through to the global parser and its
* limit.
* caller's credential resolves: a header, a cookie, or an `auth_token` the body
* opens with. Unauthenticated requests and requests without a Content-Length
* fall through to the global parser and its limit.
*
* The parsed body lives as long as the request, so a body's bytes count against
* both budgets until its response closes. Over the user's budget gets a 429,
@@ -53,82 +94,91 @@ export const createLargeJsonBodyParser = ({
perUserBytes,
accepts,
}: LargeJsonBodyOptions): RequestHandler => {
const json = express.json({ limit: limitBytes });
const textAsJson = express.json({
limit: limitBytes,
type: (req) =>
req.headers['content-type'] === 'text/plain;actually=json',
});
const json = createJsonBodyParser({ limit: limitBytes });
let inFlightBytes = 0;
const inFlightByUser = new Map<string, number>();
return (req, res, next) => {
const probe = (req: Request, res: Response) =>
new Promise<void>((resolve) => {
void authProbe(req, res, () => resolve());
});
return async (req, res, next) => {
const declared = Number(req.headers['content-length']);
if (!(declared > DEFAULT_JSON_LIMIT_BYTES && declared <= limitBytes)) {
next();
return;
}
void authProbe(req, res, () => {
// `res.closed`: the client left during the probe, so `close` has
// already fired and would never release a charge.
if (!req.actor || res.closed) {
next();
await probe(req, res);
let bodyToken: string | undefined;
if (!req.actor && !res.closed) {
bodyToken = await leadingBodyToken(req);
if (bodyToken) {
// Where the probe looks for a body token.
req.body = { auth_token: bodyToken };
await probe(req, res);
req.body = undefined;
}
}
// `res.closed`: the client left during the probe, so `close` has
// already fired and would never release a charge.
if (!req.actor || res.closed) {
next();
return;
}
const user = req.actor.user.uuid ?? actorUid(req.actor);
const userBytes = inFlightByUser.get(user) ?? 0;
if (userBytes + declared > perUserBytes) {
res.setHeader('Retry-After', '5');
next(
new HttpError(
429,
'Too many large requests in progress for this account',
{ legacyCode: 'too_many_requests' },
),
);
return;
}
if (inFlightBytes + declared > budgetBytes) {
res.setHeader('Retry-After', '5');
next(
new HttpError(503, 'Too many large requests in progress', {
code: 'large_body_capacity',
// Load shedding, not a fault: record it, don't page.
alarmSeverity: 'info',
}),
);
return;
}
inFlightBytes += declared;
inFlightByUser.set(user, userBytes + declared);
res.once('close', () => {
inFlightBytes -= declared;
const left = (inFlightByUser.get(user) ?? 0) - declared;
if (left > 0) inFlightByUser.set(user, left);
else inFlightByUser.delete(user);
});
json(req, res, (err?: unknown) => {
if (err) {
next(err);
return;
}
const user = req.actor.user.uuid ?? actorUid(req.actor);
const userBytes = inFlightByUser.get(user) ?? 0;
if (userBytes + declared > perUserBytes) {
res.setHeader('Retry-After', '5');
// The caller was resolved from the token the body opens with,
// so the parsed body has to name the same one.
const tokenMoved =
bodyToken !== undefined &&
(req.body as { auth_token?: unknown } | undefined)
?.auth_token !== bodyToken;
if (tokenMoved || (accepts && !accepts(req))) {
// The same answer the default limit gives.
next(
new HttpError(
429,
'Too many large requests in progress for this account',
{ legacyCode: 'too_many_requests' },
),
);
return;
}
if (inFlightBytes + declared > budgetBytes) {
res.setHeader('Retry-After', '5');
next(
new HttpError(503, 'Too many large requests in progress', {
code: 'large_body_capacity',
// Load shedding, not a fault: record it, don't page.
alarmSeverity: 'info',
new HttpError(413, 'request entity too large', {
legacyCode: 'client_bad_request',
}),
);
return;
}
inFlightBytes += declared;
inFlightByUser.set(user, userBytes + declared);
res.once('close', () => {
inFlightBytes -= declared;
const left = (inFlightByUser.get(user) ?? 0) - declared;
if (left > 0) inFlightByUser.set(user, left);
else inFlightByUser.delete(user);
});
json(req, res, (err?: unknown) => {
if (err) {
next(err);
return;
}
textAsJson(req, res, (err?: unknown) => {
if (err) {
next(err);
return;
}
if (accepts && !accepts(req)) {
// The same answer the default limit gives.
next(
new HttpError(413, 'request entity too large', {
legacyCode: 'client_bad_request',
}),
);
return;
}
next();
});
});
next();
});
};
};
@@ -355,6 +355,17 @@ describe('ChatCompletionDriver.complete auth and model resolution', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a model that is not a string with 400', async () => {
await expect(
withTestActor(() =>
driver.complete({
messages: [{ role: 'user', content: 'hi' }],
model: { id: 'gpt' },
} as unknown as ICompleteArguments),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('routes by alias when a provider exposes one (puterId is auto-aliased on the bucket)', async () => {
// Inject a model that carries a `puterId` so we can verify the
// alias bucket resolves back to the canonical id. Spy on
@@ -2130,7 +2130,9 @@ export class ChatCompletionDriver extends PuterDriver {
}
#resolveModel(modelId: string, provider?: string): IChatModel | null {
const models = this.#modelIdMap[normalizeModelKey(modelId ?? '')];
// `model` comes straight from the caller and may not be a string.
if (typeof modelId !== 'string') return null;
const models = this.#modelIdMap[normalizeModelKey(modelId)];
if (!models || models.length === 0) return null;
// An explicitly requested provider is honoured even if its route is
// marked — the caller asked for that one, not for the cheapest hop.
@@ -37,7 +37,7 @@ import {
postGeminiJson,
geminiTokenCount as tokenCount,
} from '../../../util/geminiRest.js';
import { decodeStrictBase64 } from '../../../../util/mediaSniff.js';
import { decodeStrictBase64InSlices } from '../../../../util/mediaSniff.js';
import { isUpstreamTimeoutError } from '../../../util/upstreamErrors.js';
import { SAMPLE_AUDIO_URL } from '../../../util/testMode.js';
import type {
@@ -456,7 +456,8 @@ export class GeminiTTSProvider implements ITTSProvider {
},
);
}
const { body, contentType } = this.#decodeAudio(audio);
const { body, contentType } =
await this.#decodeAudio(audio);
outputTokens = usageTokens(
usage?.output_tokens_by_modality,
@@ -474,7 +475,7 @@ export class GeminiTTSProvider implements ITTSProvider {
dataType: 'stream',
content_type: contentType,
chunked: true,
stream: Readable.from(body),
stream: Readable.from(body, { objectMode: false }),
};
} finally {
// Every outcome pays once for the counts it reports.
@@ -519,13 +520,15 @@ export class GeminiTTSProvider implements ITTSProvider {
}
/** WAV passes through byte for byte; raw PCM is wrapped in a WAV header. */
#decodeAudio(audio: InteractionContent): {
body: Buffer;
async #decodeAudio(audio: InteractionContent): Promise<{
body: Buffer[];
contentType: string;
} {
}> {
const { data, mime_type: mimeType } = audio;
const bytes =
typeof data === 'string' ? decodeStrictBase64(data) : null;
typeof data === 'string'
? await decodeStrictBase64InSlices(data)
: null;
if (!bytes || typeof mimeType !== 'string') {
throw upstreamFailure('Gemini TTS returned invalid audio data');
}
@@ -541,7 +544,7 @@ export class GeminiTTSProvider implements ITTSProvider {
) {
throw upstreamFailure('Gemini TTS returned invalid audio data');
}
return { body: bytes, contentType: 'audio/wav' };
return { body: [bytes], contentType: 'audio/wav' };
}
if (PCM_MIME_TYPES.has(essence) && bytes.length > 0) {
const rate = Number(
@@ -551,8 +554,9 @@ export class GeminiTTSProvider implements ITTSProvider {
Number.isSafeInteger(rate) && rate > 0
? rate
: DEFAULT_PCM_SAMPLE_RATE;
// Sent ahead of the samples, which are never copied behind it.
return {
body: this.#wrapPcmInWav(bytes, sampleRate, 1, 16),
body: [this.#wavHeader(bytes.length, sampleRate, 1, 16), bytes],
contentType: 'audio/wav',
};
}
@@ -560,17 +564,15 @@ export class GeminiTTSProvider implements ITTSProvider {
}
/** Wrap raw PCM samples in a WAV container so browsers can play it. */
#wrapPcmInWav(
pcm: Buffer,
#wavHeader(
dataSize: number,
sampleRate: number,
channels: number,
bitsPerSample: number,
): Buffer {
const byteRate = (sampleRate * channels * bitsPerSample) / 8;
const blockAlign = (channels * bitsPerSample) / 8;
const dataSize = pcm.length;
const headerSize = 44;
const buffer = Buffer.alloc(headerSize + dataSize);
const buffer = Buffer.alloc(44);
// RIFF header
buffer.write('RIFF', 0);
@@ -590,7 +592,6 @@ export class GeminiTTSProvider implements ITTSProvider {
// data sub-chunk
buffer.write('data', 36);
buffer.writeUInt32LE(dataSize, 40);
pcm.copy(buffer, headerSize);
return buffer;
}
@@ -25,7 +25,7 @@ import { insufficientCreditsError } from '../../../../services/metering/enforcem
import type { UsageInput } from '../../../../services/metering/types.js';
import { normalizeRawBase64ImageString } from '../../../../util/appIcon.js';
import {
decodeStrictBase64,
decodeStrictBase64InSlices,
sniffImageMime,
sniffVideoMime,
} from '../../../../util/mediaSniff.js';
@@ -405,7 +405,7 @@ export class GeminiVideoProvider extends VideoProvider {
'Gemini video generation did not complete the request',
);
}
const video = this.#readVideo(videoBlock);
const video = await this.#readVideo(videoBlock);
if (!billed) {
throw geminiUpstreamFailure(
'Gemini video generation returned unusable usage',
@@ -722,7 +722,7 @@ export class GeminiVideoProvider extends VideoProvider {
}
/** The base64 MP4 held by the video block. */
#readVideo(video: InteractionContent | undefined): string {
async #readVideo(video: InteractionContent | undefined): Promise<string> {
if (!video) {
throw new HttpError(400, 'Gemini did not return a video', {
legacyCode: 'upstream_bad_request',
@@ -731,7 +731,7 @@ export class GeminiVideoProvider extends VideoProvider {
}
const bytes =
video.mime_type === 'video/mp4' && typeof video.data === 'string'
? decodeStrictBase64(video.data)
? await decodeStrictBase64InSlices(video.data)
: null;
if (!bytes || sniffVideoMime(bytes) !== 'video/mp4') {
throw geminiUpstreamFailure(
+5 -3
View File
@@ -18,6 +18,7 @@
*/
import { HttpError } from '../../core/http/HttpError.js';
import { parseJsonInSlices } from '../../util/slicedJson.js';
import { readBoundedBody } from './fileInput.js';
import { upstreamFetch } from './upstreamErrors.js';
@@ -39,8 +40,8 @@ export const geminiUpstreamFailure = (message: string): HttpError =>
/**
* POSTs `body` to a Gemini REST `path` and parses the JSON object it answers
* with. `timeoutMs` covers the body read, and a reply over `maxBytes` fails as
* an upstream error unless `tooLarge` supplies another. `T` is the documented reply shape; callers check every
* field before use.
* an upstream error unless `tooLarge` supplies another. `T` is the documented
* reply shape; callers check every field before use.
*/
export async function postGeminiJson<T extends object>(
apiKey: string,
@@ -87,7 +88,8 @@ export async function postGeminiJson<T extends object>(
const raw = await readBoundedBody(response, maxBytes, tooLarge);
let parsed: unknown = null;
try {
parsed = JSON.parse(raw.toString('utf8'));
// Media replies run to tens of megabytes of base64.
parsed = await parseJsonInSlices(raw);
} catch {
// Handled with the non-object case below.
}
+22 -1
View File
@@ -414,6 +414,7 @@ describe('PuterServer host header validation', () => {
'show-save-file-picker',
'show-directory-picker',
'upgrade',
'top-up',
'request-email-confirmation',
])('lets a deployment subdomain frame the %s popup and names it to the GUI', async (action) => {
const parent = `http://spreadsheet.puter.localhost:${port}`;
@@ -447,7 +448,7 @@ describe('PuterServer host header validation', () => {
expect(redirected.headers['x-frame-options']).toBe('SAMEORIGIN');
});
it.each(['/embed/SIGN-IN', '/embed/Show-Open-File-Picker', '/embed/sign-in/', '/embed/Upgrade', '/embed/upgrade/', '/embed/Request-Email-Confirmation'])(
it.each(['/embed/SIGN-IN', '/embed/Show-Open-File-Picker', '/embed/sign-in/', '/embed/Upgrade', '/embed/upgrade/', '/embed/Top-Up', '/embed/top-up/', '/embed/Request-Email-Confirmation'])(
'lets a first-party page frame only the exact spelling of %s',
async (spelling) => {
const res = await request(spelling, {
@@ -792,6 +793,26 @@ describe('PuterServer request parsing', () => {
}
});
it('parses a chat call above 50 MiB whose body opens with its token', async () => {
const fields = `"interface":"puter-chat-completion","method":"no-such-method","pad":"${'x'.repeat(51 * MiB)}"`;
const asText = { 'content-type': 'text/plain;actually=json' };
const first = await postJson(
'/drivers/call',
`{"auth_token":"${token}",${fields}}`,
asText,
);
expect(first.status).toBe(404);
expect(first.body).toContain('no-such-method');
// A token after other fields can't be read ahead of the body.
const last = await postJson(
'/drivers/call',
`{${fields},"auth_token":"${token}"}`,
asText,
);
expect(last.status).toBe(413);
});
it('keeps the default JSON limit for other drivers', async () => {
const payload = JSON.stringify({
interface: 'puter-kvstore',
+30 -11
View File
@@ -36,6 +36,7 @@ import { createAuthProbe } from './core/http/middleware/authProbe';
import { createRequestContextMiddleware } from './core/http/middleware/requestContext';
import { createFingerprintMiddleware } from './core/http/middleware/fingerprint';
import { createErrorHandler } from './core/http/middleware/errorHandler';
import { createJsonBodyParser } from './core/http/middleware/jsonBody';
import {
createLargeJsonBodyParser,
DEFAULT_JSON_LIMIT_BYTES,
@@ -144,6 +145,8 @@ const LARGE_JSON_ROUTES = [
const LARGE_JSON_LIMIT_BYTES = 128 * 1024 * 1024;
/** Two max-size bodies per process. */
const LARGE_JSON_BUDGET_BYTES = 2 * LARGE_JSON_LIMIT_BYTES;
/** These parse in one go, so they stay small enough for that to be quick. */
const NON_UTF8_JSON_LIMIT_BYTES = 1024 * 1024;
/**
* Through `/drivers/call`, only chat takes a large body; other drivers keep the
@@ -575,26 +578,42 @@ export class PuterServer {
}),
);
}
// A WebDAV body is the file being stored, whatever content type the
// client labels it with, so the controller reads it off the stream.
const exceptDav =
(parser: RequestHandler): RequestHandler =>
(req, res, next) => {
if (activeSubdomain(req) === 'dav') {
next();
return;
}
return parser(req, res, next);
};
this.#app.use(
express.json({
limit: DEFAULT_JSON_LIMIT_BYTES,
verify: captureRawBody,
}),
exceptDav(
createJsonBodyParser({
limit: DEFAULT_JSON_LIMIT_BYTES,
verify: captureRawBody,
}),
),
);
// JSON in another charset, which the parser above leaves alone.
this.#app.use(
express.json({
limit: DEFAULT_JSON_LIMIT_BYTES,
type: (req) =>
req.headers['content-type'] === 'text/plain;actually=json',
verify: captureRawBody,
}),
exceptDav(
express.json({
limit: NON_UTF8_JSON_LIMIT_BYTES,
verify: captureRawBody,
}),
),
);
// Form-encoded bodies (e.g. `/down` from the GUI's iframe-triggered
// download form). Needs to run before the auth probe so
// `req.body.auth_token` is populated for urlencoded POSTs the same
// way it is for JSON POSTs. Small cap — this parser is only here to
// cover the auth_token / anti_csrf field shape, not file uploads.
this.#app.use(express.urlencoded({ extended: true, limit: '100kb' }));
this.#app.use(
exceptDav(express.urlencoded({ extended: true, limit: '100kb' })),
);
// -- Auth probe ----------------------------------------------
if (authProbe) {
+16
View File
@@ -647,6 +647,22 @@ describe('FSService overwrite and dedupe resolution', () => {
expect(await readBack(second)).toBe('second');
});
it('leaves one readable file when two overwrites create the same path at once', async () => {
const path = `${user.home}/Documents/raced.txt`;
const written = await Promise.all([
writeFile(user, path, 'short', { overwrite: true }),
writeFile(user, path, 'a much longer body', { overwrite: true }),
]);
// Both started before the row existed, so each uploaded under its own
// key; the row has to end up describing bytes that are really there.
expect(written[0].uuid).toBe(written[1].uuid);
const stored = (await entryAt(user, '/Documents/raced.txt'))!;
const body = await readBack(stored);
expect(['short', 'a much longer body']).toContain(body);
expect(stored.size).toBe(Buffer.byteLength(body));
});
it('dedupes into an unused " (n)" name, skipping names already taken', async () => {
await writeFile(user, `${user.home}/Documents/d.txt`, 'a');
await writeFile(user, `${user.home}/Documents/d (1).txt`, 'a');
+96 -31
View File
@@ -2073,10 +2073,7 @@ export class FSService extends PuterService {
);
});
const fsEntries = await this.stores.fsEntry.batchCreateEntries(
createInputs,
true,
);
const fsEntries = await this.#createUploadedEntries(createInputs);
return preparedBatch.items.map((item, index) => {
const fsEntry = fsEntries[index];
if (!fsEntry) {
@@ -3001,30 +2998,97 @@ export class FSService extends PuterService {
legacyCode: 'conflict',
});
}
await this.#moveObjectOntoKey(
session.objectKey,
liveKey,
{ bucket, region, size },
session.contentType,
);
}
async #moveObjectOntoKey(
fromKey: string,
toKey: string,
at: { bucket: string; region: string; size: number },
contentType?: string | null,
): Promise<void> {
await this.stores.s3Object.copyObjectOfAnySize(
{
sourceBucket: bucket,
sourceKey: session.objectKey,
destinationBucket: bucket,
destinationKey: liveKey,
contentType: session.contentType,
sourceBucket: at.bucket,
sourceKey: fromKey,
destinationBucket: at.bucket,
destinationKey: toKey,
...(contentType ? { contentType } : {}),
metadataDirective: 'REPLACE',
},
region,
size,
at.region,
at.size,
);
try {
await this.stores.s3Object.deleteObject(
bucket,
session.objectKey,
region,
at.bucket,
fromKey,
at.region,
);
} catch (e) {
// The live object is already right; the staged one is only litter.
console.warn('[fs] staged overwrite left behind:', e);
// The destination is already right; the source is only litter.
console.warn('[fs] moved object left behind:', e);
}
}
/**
* Creates the rows for objects already uploaded. A row another writer
* created first keeps its uuid, so those bytes move onto that key.
*/
async #createUploadedEntries(
inputs: FSEntryCreateInput[],
): Promise<FSEntry[]> {
const entries = await this.stores.fsEntry.batchCreateEntries(
inputs,
true,
);
const raced = inputs.flatMap((input, index) => {
const liveKey = entries[index]?.uuid;
return liveKey && liveKey !== input.uuid
? [{ input, index, liveKey }]
: [];
});
if (raced.length === 0) return entries;
await Promise.all(
raced.map(({ input, liveKey }) =>
this.#moveObjectOntoKey(
input.uuid,
liveKey,
{
bucket: this.stores.s3Object.resolveBucket(
input.bucket,
),
region: this.stores.s3Object.resolveRegion(
input.bucketRegion,
),
size: input.size,
},
input.contentType,
),
),
);
const adopted = await this.stores.fsEntry.batchCreateEntries(
raced.map(({ input, liveKey }) => ({ ...input, uuid: liveKey })),
true,
);
raced.forEach(({ index, liveKey }, i) => {
const entry = adopted[i];
if (entry?.uuid !== liveKey) {
throw new HttpError(409, 'Write target changed mid-write', {
legacyCode: 'conflict',
});
}
entries[index] = entry;
});
return entries;
}
async completeUrlWrite(
userId: number,
completeWriteRequest: CompleteWriteRequest,
@@ -3581,10 +3645,10 @@ export class FSService extends PuterService {
: uploadBody.contentHashSha256;
const createInput = this.#toCreateInput(normalizedInput, objectKey);
const fsEntry = await this.stores.fsEntry.createEntry(
createInput,
normalizedInput.createMissingParents,
);
const [fsEntry] = await this.#createUploadedEntries([createInput]);
if (!fsEntry) {
throw new Error('Failed to create entry');
}
this.#emitFsEvent(
existingEntry ? 'fs.write.file' : 'fs.create.file',
@@ -3682,17 +3746,18 @@ export class FSService extends PuterService {
updates.filter(isValid),
);
let next = 0;
return updates.map((update): PromiseSettledResult<FSEntry> =>
isValid(update)
? stored[next++]
: {
status: 'rejected',
reason: new HttpError(
400,
'Invalid file entry identifier for thumbnail update',
{ legacyCode: 'bad_request' },
),
},
return updates.map(
(update): PromiseSettledResult<FSEntry> =>
isValid(update)
? stored[next++]
: {
status: 'rejected',
reason: new HttpError(
400,
'Invalid file entry identifier for thumbnail update',
{ legacyCode: 'bad_request' },
),
},
);
}
@@ -18,6 +18,7 @@
*/
import { Writable } from 'node:stream';
import v8 from 'node:v8';
import type { Request, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
@@ -229,6 +230,37 @@ describe('ProfileService', () => {
).toBeNull();
});
// V8 stops optimizing regexps once a process holds enough compiled code,
// as long-running servers do, and an unoptimized one overflows on this.
it('answers a picture of several megabytes with 413', () => {
const picture = `data:image/png;base64,${'A'.repeat(6 * 1024 * 1024)}`;
v8.setFlagsFromString('--no-regexp-optimization');
try {
expect(() => service.validatePatch({ picture })).toThrow(
expect.objectContaining({
statusCode: 413,
code: 'profile_picture_too_large',
}),
);
} finally {
v8.setFlagsFromString('--regexp-optimization');
}
});
it('accepts a picture of the largest size, with or without padding', () => {
const prefix = 'data:image/png;base64,';
const full = `${prefix}${'A'.repeat(PROFILE_PICTURE_MAX_BYTES - prefix.length)}`;
expect(service.validatePatch({ picture: full })).toEqual({
picture: full,
});
expect(service.validatePatch({ picture: `${prefix}QQ` })).toEqual({
picture: `${prefix}QQ`,
});
expect(() => service.validatePatch({ picture: `${full}A` })).toThrow(
expect.objectContaining({ code: 'profile_picture_too_large' }),
);
});
it('keeps what validates from a legacy profile and names the rest', () => {
expect(
service.normalizeLegacyProfile({
@@ -83,6 +83,14 @@ const validateField = (field: ProfileField, value: unknown): string | null => {
);
}
if (field === 'picture') {
// Sized before the pattern runs, which fails outright on megabytes of
// text. Only ASCII passes the pattern, so the length is the byte count.
if (value.length > PROFILE_PICTURE_MAX_BYTES) {
throw new HttpError(413, 'picture is too large', {
code: 'profile_picture_too_large',
fields: { field, maxBytes: PROFILE_PICTURE_MAX_BYTES },
});
}
if (!PICTURE_DATA_URL.test(value)) {
throw invalid(
field,
@@ -90,12 +98,6 @@ const validateField = (field: ProfileField, value: unknown): string | null => {
'profile_picture_invalid',
);
}
if (Buffer.byteLength(value) > PROFILE_PICTURE_MAX_BYTES) {
throw new HttpError(413, 'picture is too large', {
code: 'profile_picture_too_large',
fields: { field, maxBytes: PROFILE_PICTURE_MAX_BYTES },
});
}
return value;
}
const maxLength =
+27 -14
View File
@@ -26,6 +26,7 @@ import {
PuterStore,
} from '../types';
import { HttpError } from '../../core/http/HttpError.js';
import { runWithConcurrencyLimit } from '../../util/concurrency.js';
import { isUniqueViolation } from '../../util/dbError.js';
import {
validateUrl,
@@ -89,6 +90,8 @@ const STATS_CACHE_TTL_SECONDS = 30 * 60;
// `app-` + a UUID. ClickHouse's `app_opens.app_uid` is FixedString(40) and
// fails the whole query on a longer value.
const APP_UID_MAX_BYTES = 40;
// ClickHouse caps the size of one query parameter, and the uid list is one.
const CLICKHOUSE_UIDS_PER_QUERY = 1000;
// Period helpers for detailed/grouped stats. Ported from v1
// AppInformationService — queries go straight to ClickHouse/MySQL on demand
@@ -1727,20 +1730,30 @@ export class AppStore extends PuterStore {
const clickhouse = this.clients.clickhouse;
if (clickhouse) {
const res = await clickhouse.query({
query: `
SELECT app_uid,
uniqExact((user_id, intDiv(ts, ${APP_OPEN_COUNT_PERIOD_SECONDS}))) AS open_count,
count(DISTINCT user_id) AS user_count
FROM app_opens
WHERE app_uid IN {uids:Array(String)}
GROUP BY app_uid
`,
query_params: { uids: uids.map((u) => String(u)) },
format: 'JSONEachRow',
});
const rows = await res.json();
for (const row of rows) {
const chunks = [];
for (let i = 0; i < uids.length; i += CLICKHOUSE_UIDS_PER_QUERY) {
chunks.push(uids.slice(i, i + CLICKHOUSE_UIDS_PER_QUERY));
}
const results = await runWithConcurrencyLimit(
chunks,
4,
async (chunk) => {
const res = await clickhouse.query({
query: `
SELECT app_uid,
uniqExact((user_id, intDiv(ts, ${APP_OPEN_COUNT_PERIOD_SECONDS}))) AS open_count,
count(DISTINCT user_id) AS user_count
FROM app_opens
WHERE app_uid IN {uids:Array(String)}
GROUP BY app_uid
`,
query_params: { uids: chunk.map((u) => String(u)) },
format: 'JSONEachRow',
});
return res.json();
},
);
for (const row of results.flat()) {
out.set(row.app_uid, {
open_count: parseInt(row.open_count, 10) || 0,
user_count: parseInt(row.user_count, 10) || 0,
+31
View File
@@ -200,6 +200,37 @@ describe('AppStore app stats (cache-on-read)', () => {
}
});
it('splits a long uid list across ClickHouse queries', async () => {
const uids = Array.from({ length: 2500 }, () => freshUid());
const queried = [];
server.clients.clickhouse = {
query: async ({ query_params }) => {
queried.push(query_params.uids.length);
return {
json: async () =>
query_params.uids.map((u) => ({
app_uid: u,
open_count: '2',
user_count: '1',
})),
};
},
};
try {
const stats = await appStore.getAppsStats(uids);
expect(queried.sort((a, b) => b - a)).toEqual([1000, 1000, 500]);
expect(stats.size).toBe(2500);
expect(stats.get(uids[2499])).toMatchObject({
open_count: 2,
user_count: 1,
});
} finally {
delete server.clients.clickhouse;
}
});
it('keeps ids longer than an app uid out of the ClickHouse query', async () => {
const uid = freshUid();
const longId = `iap-${'a'.repeat(36)}-extra`;
+40 -8
View File
@@ -444,7 +444,7 @@ describe('FSEntryStore cache invalidation', () => {
store.batchCreateEntries([
{
userId: user.userId,
uuid: uuidv4(),
uuid: file.uuid,
path: file.path,
size: 7,
overwrite: true,
@@ -1185,7 +1185,7 @@ describe('FSEntryStore entry creation', () => {
const [updated] = await store.batchCreateEntries([
{
userId: user.userId,
uuid: uuidv4(),
uuid: original.uuid,
path: `${user.home}/Documents/ow.txt`,
size: 42,
overwrite: true,
@@ -1204,6 +1204,33 @@ describe('FSEntryStore entry creation', () => {
expect(updated?.thumbnail).toBe('data:image/png;base64,AA');
});
it('leaves the row alone when an overwrite brings a different object key', async () => {
const user = await makeUser();
const original = await createFile(
user,
`${user.home}/Documents/otherkey.txt`,
{ size: 1 },
);
// The bytes for this write sit under a key the row doesn't name, so
// repointing bucket or size would describe an object that isn't there.
const returned = await createFile(user, original.path, {
size: 42,
overwrite: true,
bucket: 'elsewhere',
bucketRegion: 'eu-central-1',
});
expect(returned.uuid).toBe(original.uuid);
await expect(
store.getEntryByPath(original.path),
).resolves.toMatchObject({
uuid: original.uuid,
size: 1,
bucket: 'puter-local',
});
});
it('refuses to clobber an existing entry without overwrite, or a directory with it', async () => {
const user = await makeUser();
await createFile(user, `${user.home}/Documents/keep.txt`);
@@ -1227,7 +1254,10 @@ describe('FSEntryStore entry creation', () => {
it('propagates the underlying failure when an overwrite update fails', async () => {
const user = await makeUser();
await createFile(user, `${user.home}/Documents/upfail.txt`);
const existing = await createFile(
user,
`${user.home}/Documents/upfail.txt`,
);
const db = server.clients.db;
const originalWrite = (Object.getPrototypeOf(db) as typeof db).write;
@@ -1243,6 +1273,7 @@ describe('FSEntryStore entry creation', () => {
await expect(
createFile(user, `${user.home}/Documents/upfail.txt`, {
overwrite: true,
uuid: existing.uuid,
}),
).rejects.toThrow('update rejected');
@@ -3801,7 +3832,7 @@ describe('FSEntryStore lost-insert recovery', () => {
spy.mockRestore();
});
it('updates the race winner in place when overwrite was requested', async () => {
it('hands back the race winner untouched when overwrite was requested', async () => {
const user = await makeUser();
const uuid = uuidv4();
const spy = loseInsertRace(uuid, () =>
@@ -3827,13 +3858,14 @@ describe('FSEntryStore lost-insert recovery', () => {
]);
spy.mockRestore();
// The winner's row survived; our payload was written onto it.
// Our object is under `uuid`, not the winner's key, so the winner's
// row is returned as it stands for the caller to move the bytes onto.
expect(entry?.uuid).not.toBe(uuid);
expect(entry?.size).toBe(77);
expect(entry?.thumbnail).toBe('thumb');
expect(entry?.size).not.toBe(77);
expect(entry?.thumbnail).not.toBe('thumb');
await expect(
store.getEntryByPath(`${user.home}/Documents/recovered.txt`),
).resolves.toMatchObject({ size: 77, uuid: entry?.uuid });
).resolves.toMatchObject({ size: entry?.size, uuid: entry?.uuid });
});
});
+13
View File
@@ -1941,6 +1941,11 @@ export class FSEntryStore extends PuterStore {
return entry;
}
/**
* An overwrite whose `uuid` is not the existing row's leaves that row
* untouched and returns it: the caller's object is under another key, so
* the row can't describe it until the caller moves the bytes.
*/
async batchCreateEntries(
entries: FSEntryCreateInput[],
createPaths = true,
@@ -2074,6 +2079,10 @@ export class FSEntryStore extends PuterStore {
{ legacyCode: 'cannot_overwrite_a_directory' },
);
}
if (existingEntry.uuid !== entry.input.uuid) {
updatedResultsByIndex.set(entry.index, existingEntry);
continue;
}
const updatedEntry = {
...existingEntry,
@@ -2398,6 +2407,10 @@ export class FSEntryStore extends PuterStore {
);
continue;
}
if (existing.uuid !== entry.input.uuid) {
updatedResultsByIndex.set(entry.index, existing);
continue;
}
const updatedEntry: FSEntry = {
...existing,
+53 -1
View File
@@ -18,7 +18,10 @@
*/
import { describe, expect, it } from 'vitest';
import { decodeStrictBase64 } from './mediaSniff.ts';
import {
decodeStrictBase64,
decodeStrictBase64InSlices,
} from './mediaSniff.ts';
describe('decodeStrictBase64', () => {
it.each([
@@ -50,3 +53,52 @@ describe('decodeStrictBase64', () => {
}
});
});
describe('decodeStrictBase64InSlices', () => {
// Several slices, with a last one that is partial and padded.
const bytes = Buffer.alloc(5 * 1024 * 1024 + 2);
for (let i = 0; i < bytes.length; i += 4099) bytes[i] = i % 251;
const valid = bytes.toString('base64');
it('decodes a multi-MB payload to the same bytes', async () => {
expect(valid.endsWith('==')).toBe(true);
expect((await decodeStrictBase64InSlices(valid))?.equals(bytes)).toBe(
true,
);
const short = Buffer.from('abcd').toString('base64');
expect(await decodeStrictBase64InSlices(short)).toEqual(
Buffer.from('abcd'),
);
});
it.each([
['empty input', ''],
['unpadded input', valid.slice(0, -2)],
[
'a character outside the alphabet in the first slice',
`*${valid.slice(1)}`,
],
[
'a character outside the alphabet in the last slice',
`${valid.slice(0, -5)}*${valid.slice(-4)}`,
],
[
'the URL-safe alphabet',
`${valid.slice(0, 3_000_000)}-${valid.slice(3_000_001)}`,
],
[
'a non-ASCII character',
`${valid.slice(0, 3_000_000)}ī${valid.slice(3_000_001)}`,
],
[
'padding where a slice ends',
`${valid.slice(0, 1024 * 1024 - 1)}=${valid.slice(1024 * 1024)}`,
],
[
'padding inside a slice',
`${valid.slice(0, 2_000_002)}==${valid.slice(2_000_004)}`,
],
])('rejects %s', async (_label, value) => {
expect(await decodeStrictBase64InSlices(value)).toBeNull();
});
});
+29
View File
@@ -17,6 +17,8 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { setImmediate as nextTurn } from 'node:timers/promises';
// Identify uploaded media by its bytes. A caller-supplied MIME type or file
// extension is never trusted.
@@ -69,6 +71,33 @@ export function decodeStrictBase64(value: string): Buffer | null {
return bytes.length === (value.length / 4) * 3 - padding ? bytes : null;
}
/** Base64 characters decoded per turn; a multiple of four. */
const BASE64_SLICE_CHARS = 1024 * 1024;
/**
* `decodeStrictBase64` for payloads of many megabytes: decodes a slice per turn
* of the event loop, so the whole string never holds it.
*/
export async function decodeStrictBase64InSlices(
value: string,
): Promise<Buffer | null> {
if (value.length <= BASE64_SLICE_CHARS) return decodeStrictBase64(value);
if (value.length % 4 !== 0) return null;
const padding = value.endsWith('==') ? 2 : value.endsWith('=') ? 1 : 0;
const bytes = Buffer.allocUnsafe((value.length / 4) * 3 - padding);
let written = 0;
for (let at = 0; at < value.length; at += BASE64_SLICE_CHARS) {
const slice = decodeStrictBase64(
value.slice(at, at + BASE64_SLICE_CHARS),
);
// Padding inside the string leaves a slice short, and the total with it.
if (!slice || written + slice.length > bytes.length) return null;
written += slice.copy(bytes, written);
await nextTurn();
}
return written === bytes.length ? bytes : null;
}
function looksLikeSvg(bytes: Buffer): boolean {
let head = bytes.subarray(0, SVG_SNIFF_WINDOW).toString('utf8');
if (head.charCodeAt(0) === 0xfeff) head = head.slice(1);
+374
View File
@@ -0,0 +1,374 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterEach, describe, expect, it, vi } from 'vitest';
import {
MAX_DEPTH,
parseJsonInSlices,
SLICE_BYTES,
STRING_PIECE_BYTES,
} from './slicedJson.js';
type Outcome = { value: unknown } | { error: string };
const native = (body: Buffer): Outcome => {
try {
return { value: JSON.parse(body.toString('utf8')) };
} catch (e) {
return { error: (e as Error).name };
}
};
const sliced = async (body: Buffer): Promise<Outcome> => {
try {
return { value: await parseJsonInSlices(body) };
} catch (e) {
return { error: (e as Error).name };
}
};
/** Key order, own `__proto__` keys and lone surrogates all show in the text. */
const shown = (outcome: Outcome): string =>
'error' in outcome ? outcome.error : JSON.stringify(outcome.value);
const expectSameAsNative = async (body: Buffer | string): Promise<void> => {
const buffer = typeof body === 'string' ? Buffer.from(body) : body;
expect(shown(await sliced(buffer))).toBe(shown(native(buffer)));
};
/** Whitespace that pushes `text` onto the sliced path without changing it. */
const padded = (text: string): string => `${' '.repeat(SLICE_BYTES)}${text}`;
// mulberry32: the cases must be the same on every run.
const seeded = (seed: number) => () => {
seed = (seed + 0x6d2b79f5) | 0;
let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
};
const STRING_PARTS = [
'a',
'word ',
'"',
'\\',
'\\\\"',
'\n',
'\u0001',
'é',
'日本',
'😀',
'\ud83d',
'\u2028',
'/',
'\\u00e9',
];
const KEYS = ['id', 'name', '__proto__', '7', '3', 'constructor', 'a b', ''];
// No key holds a backslash: with one in a key seen earlier, some V8 versions
// misread a later escaped key, in `JSON.parse` itself.
const KEY_PARTS = STRING_PARTS.filter((part) => !part.includes('\\'));
const SCALARS = [
'0',
'-0',
'1.5e300',
'-12.25',
'1e-7',
'9007199254740993',
'true',
'false',
'null',
];
const GAPS = ['', '', '', ' ', '\n\t', '\r\n '];
/** A random JSON document, written as text so spacing and escapes vary. */
const randomDocument = (
random: () => number,
targetBytes: number,
{ plainKeys = false } = {},
): string => {
const pick = <T>(items: readonly T[]): T =>
items[Math.floor(random() * items.length)]!;
const gap = () => pick(GAPS);
const string = (maxParts: number, from = STRING_PARTS): string => {
let text = '';
const parts = Math.floor(random() * maxParts);
for (let i = 0; i < parts; i++) text += pick(from);
return JSON.stringify(text);
};
let written = 0;
const value = (depth: number): string => {
const roll = random();
let text: string;
if (depth > 0 && roll < 0.25) {
const count = Math.floor(random() * 12);
const items: string[] = [];
for (let i = 0; i < count; i++) items.push(value(depth - 1));
text = `[${gap()}${items.join(`${gap()},${gap()}`)}${gap()}]`;
} else if (depth > 0 && roll < 0.5) {
const count = Math.floor(random() * 12);
const members: string[] = [];
for (let i = 0; i < count; i++) {
const key =
plainKeys || random() < 0.7
? JSON.stringify(pick(KEYS))
: string(6, KEY_PARTS);
members.push(`${key}${gap()}:${gap()}${value(depth - 1)}`);
}
text = `{${gap()}${members.join(`${gap()},${gap()}`)}${gap()}}`;
} else if (roll < 0.75) {
// Now and then one long enough to be read in pieces.
text = string(random() < 0.002 ? 3 * SLICE_BYTES : 20);
} else {
text = pick(SCALARS);
}
written += text.length;
return text;
};
const items: string[] = [];
while (written < targetBytes) items.push(value(6));
return random() < 0.5
? `[${items.join(',')}]`
: `{${items.map((item, i) => `"k${i % 50}":${item}`).join(',')}}`;
};
describe('parseJsonInSlices', () => {
afterEach(() => {
vi.restoreAllMocks();
});
it('parses what JSON.parse parses, the same way', async () => {
const random = seeded(1);
for (let i = 0; i < 60; i++) {
const size = SLICE_BYTES * (1 + Math.floor(random() * 12));
await expectSameAsNative(randomDocument(random, size));
}
}, 60_000);
it('parses the same when every clock read ends the turn', async () => {
let now = 0;
vi.spyOn(performance, 'now').mockImplementation(() => (now += 1000));
const random = seeded(3);
for (let i = 0; i < 30; i++) {
await expectSameAsNative(randomDocument(random, 6 * SLICE_BYTES));
}
// Strings long enough for a turn to end inside them.
await expectSameAsNative(
`["${'ab\\n'.repeat(SLICE_BYTES)}",{"k":"${'\\\\'.repeat(SLICE_BYTES)}"},${'1234567890'.repeat(SLICE_BYTES / 2)}]`,
);
}, 60_000);
it('rejects what JSON.parse rejects', async () => {
const random = seeded(2);
const replacements = Buffer.from('[]{},:"\\ 0a\néx-.e');
for (let i = 0; i < 40; i++) {
// Plain keys, so no edit can leave a backslash in one.
const valid = Buffer.from(
randomDocument(random, 2 * SLICE_BYTES, { plainKeys: true }),
);
for (let edit = 0; edit < 12; edit++) {
const body = Buffer.from(valid);
const at = Math.floor(random() * body.length);
const roll = random();
if (roll < 0.4) {
body[at] =
replacements[
Math.floor(random() * replacements.length)
]!;
await expectSameAsNative(body);
} else if (roll < 0.7) {
await expectSameAsNative(
Buffer.concat([
body.subarray(0, at),
body.subarray(at + 1),
]),
);
} else {
await expectSameAsNative(body.subarray(0, at));
}
}
}
}, 120_000);
it.each([
['an empty body', ''],
['only whitespace', ' \n '],
['two values', '[1] [2]'],
['a trailing comma in an array', '[1,2,]'],
['a trailing comma in an object', '{"a":1,}'],
['a leading comma', '[,1]'],
['a doubled comma', '[1,,2]'],
['a missing colon', '{"a" 1}'],
['a missing value', '{"a":}'],
['a bare key', '{a:1}'],
['a colon in an array', '[1:2]'],
['mismatched brackets', '[1}'],
['an extra close', '[1]]'],
['an unterminated string', '["abc'],
['an unterminated array', '[1,2'],
['a byte order mark', '\ufeff[1]'],
['a comment', '[1/* x */]'],
['a bad literal', '[tru]'],
['a bad number', '[01]'],
['a bad escape', '["\\x"]'],
['a raw control character in a string', '["a\tb"]'],
['a wide space between values', '[1,\u00a02]'],
])('rejects %s', async (_name, text) => {
await expectSameAsNative(text);
// Short input takes the native path, so check the sliced one too.
await expectSameAsNative(padded(text));
await expectSameAsNative(
`[${'1,'.repeat(SLICE_BYTES)}${text},${'2,'.repeat(SLICE_BYTES)}3]`,
);
});
it('reads a string longer than one piece, wherever a piece ends', async () => {
const units = [
'\\u00e9',
'\\n',
'\\\\',
'\\"',
'é',
'日',
'😀',
'\\ud83d\\ude00',
];
for (const unit of units) {
for (let shift = 0; shift < 9; shift++) {
const lead = 'a'.repeat(STRING_PIECE_BYTES - 12 + shift);
const text = `{"text":"${lead}${unit.repeat(8)}${lead}","n":1}`;
await expectSameAsNative(text);
}
}
}, 60_000);
it('reads long runs of escapes and stray UTF-8 bytes', async () => {
const length = STRING_PIECE_BYTES + SLICE_BYTES;
await expectSameAsNative(`["${'\\\\'.repeat(length)}"]`);
await expectSameAsNative(`["${'\\"'.repeat(length)}"]`);
await expectSameAsNative(`["${'\\'.repeat(2 * length + 1)}"]`);
await expectSameAsNative(
Buffer.concat([
Buffer.from('["'),
Buffer.alloc(2 * length, 0x80),
Buffer.from([0xe2, 0x82]),
Buffer.from('"]'),
]),
);
}, 60_000);
it('reads a long key', async () => {
const key = 'k'.repeat(2 * SLICE_BYTES);
await expectSameAsNative(`{"a":1,"${key}":{"b":[2]},"${key}x":"v"}`);
await expectSameAsNative(`{"${key}\\n":2,"z":3}`);
});
it('reads long gaps of whitespace', async () => {
const gap = ' '.repeat(2 * SLICE_BYTES);
await expectSameAsNative(
`${gap}{${gap}"a"${gap}:${gap}1${gap},${gap}"b":[${gap}2${gap},3${gap}]${gap}}${gap}`,
);
await expectSameAsNative(`[1,${gap}2${gap}]`);
});
it('reads a scalar or a string at the top level', async () => {
await expectSameAsNative(padded('12.5'));
await expectSameAsNative(padded('null'));
await expectSameAsNative(`"${'x'.repeat(4 * SLICE_BYTES)}"`);
});
it('keeps the last of a repeated key, in its first position', async () => {
const filler = `"f":"${'x'.repeat(SLICE_BYTES)}"`;
await expectSameAsNative(`{"a":1,"b":2,${filler},"a":3,"7":4,"3":5}`);
const value = (await parseJsonInSlices(
Buffer.from(`{"a":1,"b":2,${filler},"a":3}`),
)) as Record<string, unknown>;
expect(Object.keys(value)).toEqual(['a', 'b', 'f']);
expect(value.a).toBe(3);
});
it('keeps __proto__ as a plain own property', async () => {
const filler = `"f":"${'x'.repeat(SLICE_BYTES)}"`;
const bodies = [
// In a batch of members, as a container of its own, and alone.
`{${filler},"__proto__":{"polluted":true},"z":1}`,
`{"__proto__":{"polluted":true,${filler}}}`,
`{"__proto__":"${'y'.repeat(2 * SLICE_BYTES)}"}`,
];
for (const body of bodies) {
const value = (await parseJsonInSlices(
Buffer.from(body),
)) as Record<string, unknown>;
expect(Object.getPrototypeOf(value)).toBe(Object.prototype);
expect(Object.hasOwn(value, '__proto__')).toBe(true);
expect(({} as { polluted?: boolean }).polluted).toBeUndefined();
}
});
it('keeps the sign of negative zero', async () => {
const value = (await parseJsonInSlices(
Buffer.from(`[${'0,'.repeat(SLICE_BYTES)}-0]`),
)) as number[];
expect(Object.is(value.at(-1), -0)).toBe(true);
});
it('reads nesting up to the depth limit and refuses more', async () => {
const nested = (depth: number) =>
padded(`${'['.repeat(depth)}${']'.repeat(depth)}`);
// The root is not a level of nesting.
let value = await parseJsonInSlices(Buffer.from(nested(MAX_DEPTH)));
let depth = 0;
while (Array.isArray(value)) {
depth++;
value = value[0];
}
expect(depth).toBe(MAX_DEPTH);
await expect(
parseJsonInSlices(Buffer.from(nested(MAX_DEPTH + 1))),
).rejects.toThrow(SyntaxError);
});
it('reads members spread thinly across many levels', async () => {
// No level holds a slice of members, but together they hold many.
const members = '1,'.repeat(SLICE_BYTES / 8);
const depth = 200;
await expectSameAsNative(
`${`[${members}`.repeat(depth)}0${']'.repeat(depth)}`,
);
await expectSameAsNative(
`${`{"a":[${members}0],"b":`.repeat(depth)}0${'}'.repeat(depth)}`,
);
});
it('lets the event loop run while it parses', async () => {
// Dense input: the slowest kind to parse for its size.
const body = Buffer.from(`[${'[],'.repeat(1_000_000)}[]]`);
let turns = 0;
let running = true;
const tick = () => {
turns++;
if (running) setImmediate(tick);
};
setImmediate(tick);
const value = (await parseJsonInSlices(body)) as unknown[];
running = false;
expect(value).toHaveLength(1_000_001);
expect(turns).toBeGreaterThan(5);
});
});
+581
View File
@@ -0,0 +1,581 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { setImmediate as nextTurn } from 'node:timers/promises';
/**
* Text handed to one native `JSON.parse` call stays within a few multiples of
* this, which bounds how long the call can run. Shorter input parses whole.
*/
export const SLICE_BYTES = 32 * 1024;
/** Bytes of a long string decoded per turn. */
export const STRING_PIECE_BYTES = 512 * 1024;
/** Deeper nesting is refused. No mainstream serializer writes this deep. */
export const MAX_DEPTH = 10_000;
/** How long a parse runs before the event loop gets a turn. */
const TURN_BUDGET_MS = 3;
/** Bytes scanned between clock reads. */
const CLOCK_INTERVAL_BYTES = 16 * 1024;
/** Bytes of a string scanned one at a time before a native search takes over. */
const BYTEWISE_BYTES = 64;
/** Rounds of a string scan between clock reads. */
const CLOCK_INTERVAL_ROUNDS = 64;
const QUOTE = 0x22;
const BACKSLASH = 0x5c;
// Byte classes. Anything unlisted is part of a number or literal.
const C_SCALAR = 0;
const C_SPACE = 1;
const C_QUOTE = 2;
const C_OPEN_ARRAY = 3;
const C_OPEN_OBJECT = 4;
const C_CLOSE_ARRAY = 5;
const C_CLOSE_OBJECT = 6;
const C_COMMA = 7;
const C_COLON = 8;
const BYTE_CLASS = new Uint8Array(256);
for (const byte of [0x20, 0x09, 0x0a, 0x0d]) BYTE_CLASS[byte] = C_SPACE;
BYTE_CLASS[QUOTE] = C_QUOTE;
BYTE_CLASS[0x5b] = C_OPEN_ARRAY;
BYTE_CLASS[0x7b] = C_OPEN_OBJECT;
BYTE_CLASS[0x5d] = C_CLOSE_ARRAY;
BYTE_CLASS[0x7d] = C_CLOSE_OBJECT;
BYTE_CLASS[0x2c] = C_COMMA;
BYTE_CLASS[0x3a] = C_COLON;
const ROOT = 0;
const ARRAY = 1;
const OBJECT = 2;
/** Just opened: a member or the close. */
const S_FIRST = 0;
/** After a comma: a member. */
const S_MEMBER = 1;
/** After an object key. */
const S_COLON = 2;
/** After a colon, and the root before its value. */
const S_VALUE = 3;
/** After a value: a comma or the close. The root is done. */
const S_AFTER = 4;
const DONE = 0;
const TURN = 1;
const LONG_STRING = 2;
/**
* One open container, or the root. Members are scanned, not parsed: `pending`
* marks text handed to `JSON.parse` once enough of it has built up, and a
* container too large for one call gets its `out` filled a batch at a time.
*/
class Frame {
kind = ROOT;
state = S_VALUE;
/** Offset of the opening bracket. */
start = 0;
/** The container under construction, once it is known to be large. */
out: unknown[] | Record<string, unknown> | null = null;
/** Where the members not yet parsed start, or -1. */
pending = -1;
/** Where the last complete member ended. */
lastEnd = -1;
/** Bounds of the member being scanned. */
memberStart = -1;
keyStart = -1;
keyEnd = -1;
valueStart = -1;
/** The current member's key, when the member is parsed apart from `pending`. */
ownKey: string | null = null;
/** Bytes of the parent's `pending` waiting behind this container. */
backlog = 0;
open(kind: number, start: number): void {
this.kind = kind;
this.state = kind === ROOT ? S_VALUE : S_FIRST;
this.start = start;
this.out = null;
this.pending = -1;
this.lastEnd = -1;
this.memberStart = -1;
this.keyStart = -1;
this.keyEnd = -1;
this.valueStart = -1;
this.ownKey = null;
this.backlog = 0;
}
}
const unexpected = (at: number): SyntaxError =>
new SyntaxError(`Unexpected token in JSON at position ${at}`);
/** `JSON.parse` keeps `__proto__` as an own property; assignment would not. */
const setOwn = (
target: Record<string, unknown>,
key: string,
value: unknown,
): void => {
if (key === '__proto__') {
Object.defineProperty(target, key, {
value,
writable: true,
enumerable: true,
configurable: true,
});
} else {
target[key] = value;
}
};
class SlicedJsonParser {
result: unknown;
/** Bounds of the string `run` stopped at, quotes included. */
longStart = -1;
longEnd = -1;
readonly #body: Buffer;
readonly #end: number;
#pos = 0;
#frames = [new Frame()];
#depth = 0;
/** Frames below this depth have an `out`; the root counts as one. */
#filling = 1;
/** Bytes of `pending` in frames below the top one. */
#backlog = 0;
#inScalar = false;
/** The string being scanned, and where its scan resumes. */
#stringStart = -1;
#stringPos = 0;
/** Next quote and backslash at or after the last search origin. */
#quoteAt = -1;
#backslashAt = -1;
#longIsKey = false;
constructor(body: Buffer) {
this.#body = body;
this.#end = body.length;
}
/** Scans until the input ends, `deadline` passes, or a long string starts. */
run(deadline: number): number {
const body = this.#body;
const end = this.#end;
let i = this.#pos;
let frame = this.#frames[this.#depth]!;
let clockAt = i + CLOCK_INTERVAL_BYTES;
for (;;) {
if (this.#stringStart !== -1) {
const close = this.#stringEnd(deadline);
if (close === -1) return TURN;
const start = this.#stringStart;
this.#stringStart = -1;
i = close + 1;
if (this.#string(frame, start, close)) {
this.#pos = i;
return LONG_STRING;
}
continue;
}
if (i >= clockAt) {
clockAt = i + CLOCK_INTERVAL_BYTES;
if (performance.now() >= deadline) {
this.#pos = i;
return TURN;
}
}
if (i >= end) break;
const byteClass = BYTE_CLASS[body[i]!]!;
if (byteClass === C_SCALAR) {
if (!this.#inScalar) {
this.#valueStart(frame, i);
this.#inScalar = true;
}
const stop = Math.min(end, clockAt);
i++;
while (i < stop && BYTE_CLASS[body[i]!] === C_SCALAR) i++;
continue;
}
if (this.#inScalar) {
this.#inScalar = false;
this.#valueEnd(frame, i);
}
switch (byteClass) {
case C_SPACE: {
const stop = Math.min(end, clockAt);
i++;
while (i < stop && BYTE_CLASS[body[i]!] === C_SPACE) i++;
break;
}
case C_QUOTE:
this.#stringStart = i;
this.#stringPos = i + 1;
break;
case C_OPEN_ARRAY:
case C_OPEN_OBJECT: {
this.#valueStart(frame, i);
if (this.#depth === MAX_DEPTH) {
throw new SyntaxError('JSON is nested too deeply');
}
const parent = frame;
// Members before this one wait until it closes or grows.
const before =
parent.pending !== -1 &&
parent.pending !== parent.memberStart
? parent.lastEnd - parent.pending
: 0;
this.#depth++;
frame = this.#frames[this.#depth] ??= new Frame();
frame.open(byteClass === C_OPEN_ARRAY ? ARRAY : OBJECT, i);
frame.backlog = before;
this.#backlog += before;
// Parsed now, so deep nesting can't save it all up.
if (this.#backlog >= SLICE_BYTES) this.#fill();
i++;
break;
}
case C_CLOSE_ARRAY:
case C_CLOSE_OBJECT: {
if (
frame.kind !==
(byteClass === C_CLOSE_ARRAY ? ARRAY : OBJECT) ||
(frame.state !== S_FIRST && frame.state !== S_AFTER)
) {
throw unexpected(i);
}
let filled = this.#depth < this.#filling;
if (!filled && i + 1 - frame.start > SLICE_BYTES) {
this.#fill();
filled = true;
}
if (filled) {
if (frame.pending !== -1) {
this.#parsePending(frame);
}
frame.out = null;
this.#filling = this.#depth;
}
this.#backlog -= frame.backlog;
this.#depth--;
frame = this.#frames[this.#depth]!;
i++;
if (filled) {
frame.state = S_AFTER;
frame.lastEnd = i;
} else {
this.#valueEnd(frame, i);
}
break;
}
case C_COMMA:
if (frame.kind === ROOT || frame.state !== S_AFTER) {
throw unexpected(i);
}
frame.state = S_MEMBER;
i++;
break;
default:
if (frame.kind !== OBJECT || frame.state !== S_COLON) {
throw unexpected(i);
}
frame.state = S_VALUE;
i++;
}
}
if (this.#inScalar) {
this.#inScalar = false;
this.#valueEnd(frame, end);
}
if (this.#depth !== 0 || frame.state !== S_AFTER) {
throw new SyntaxError('Unexpected end of JSON input');
}
if (frame.pending !== -1) this.#parsePending(frame);
return DONE;
}
/** Supplies the string `run` stopped at. */
resolveLongString(value: string): void {
const frame = this.#frames[this.#depth]!;
if (this.#longIsKey) {
frame.ownKey = value;
return;
}
this.#attach(frame, frame.ownKey, value);
frame.ownKey = null;
frame.lastEnd = this.longEnd + 1;
frame.state = S_AFTER;
}
/**
* Offset of the quote closing the string being scanned, or -1 when
* `deadline` passed first.
*/
#stringEnd(deadline: number): number {
const body = this.#body;
const end = this.#end;
let at = this.#stringPos;
for (let round = 1; ; round++) {
// Bytewise through a short string or a stretch of escapes.
const near = Math.min(at + BYTEWISE_BYTES, end);
while (at < near) {
const byte = body[at]!;
if (byte === QUOTE) return at;
at += byte === BACKSLASH ? 2 : 1;
}
if (at >= end) throw new SyntaxError('Unterminated string in JSON');
// Then a native search across text that holds neither.
if (this.#quoteAt < at) {
this.#quoteAt = body.indexOf(QUOTE, at);
if (this.#quoteAt === -1) {
throw new SyntaxError('Unterminated string in JSON');
}
}
if (this.#backslashAt < at) {
const found = body.indexOf(BACKSLASH, at);
this.#backslashAt = found === -1 ? end : found;
}
if (this.#backslashAt > this.#quoteAt) return this.#quoteAt;
at = this.#backslashAt;
if (round % CLOCK_INTERVAL_ROUNDS === 0) {
if (performance.now() >= deadline) {
this.#stringPos = at;
return -1;
}
}
}
}
/** Handles a scanned string; true when `run` must stop for a long one. */
#string(frame: Frame, start: number, close: number): boolean {
const long = close - start > SLICE_BYTES;
if (
frame.kind === OBJECT &&
(frame.state === S_FIRST || frame.state === S_MEMBER)
) {
this.#memberStart(frame, start);
frame.keyStart = start;
frame.keyEnd = close;
frame.state = S_COLON;
if (!long) return false;
this.#detachMember(frame);
this.#longIsKey = true;
} else {
this.#valueStart(frame, start);
if (!long) {
this.#valueEnd(frame, close + 1);
return false;
}
if (frame.ownKey === null) {
this.#detachMember(frame);
if (frame.kind === OBJECT) frame.ownKey = this.#key(frame);
}
this.#longIsKey = false;
}
this.longStart = start;
this.longEnd = close;
return true;
}
#memberStart(frame: Frame, at: number): void {
if (frame.pending !== -1 && at - frame.pending >= SLICE_BYTES) {
this.#fill();
this.#parsePending(frame);
}
frame.memberStart = at;
if (frame.pending === -1) frame.pending = at;
}
#valueStart(frame: Frame, at: number): void {
if (frame.kind === OBJECT) {
if (frame.state !== S_VALUE) throw unexpected(at);
// A long gap after the key would reach `JSON.parse` with it.
if (frame.ownKey === null && at - frame.memberStart > SLICE_BYTES) {
this.#detachMember(frame);
frame.ownKey = this.#key(frame);
}
} else {
if (
frame.kind === ROOT
? frame.state !== S_VALUE
: frame.state > S_MEMBER
) {
throw unexpected(at);
}
this.#memberStart(frame, at);
}
frame.valueStart = at;
}
#valueEnd(frame: Frame, at: number): void {
frame.lastEnd = at;
frame.state = S_AFTER;
if (frame.ownKey !== null) {
this.#attach(
frame,
frame.ownKey,
JSON.parse(this.#text(frame.valueStart, at)),
);
frame.ownKey = null;
}
}
/** Takes the current member out of `pending`; its value is attached alone. */
#detachMember(frame: Frame): void {
this.#fill();
if (frame.pending !== -1 && frame.pending !== frame.memberStart) {
this.#parsePending(frame);
}
frame.pending = -1;
}
/**
* Gives every open container an `out`, attached to its parent after the
* members that precede it.
*/
#fill(): void {
const frames = this.#frames;
for (let level = this.#filling; level <= this.#depth; level++) {
const parent = frames[level - 1]!;
const child = frames[level]!;
if (
parent.pending !== -1 &&
parent.pending !== parent.memberStart
) {
this.#parsePending(parent);
}
parent.pending = -1;
const key =
parent.kind === OBJECT
? (parent.ownKey ?? this.#key(parent))
: null;
parent.ownKey = null;
child.out = child.kind === ARRAY ? [] : {};
child.backlog = 0;
this.#attach(parent, key, child.out);
}
this.#filling = this.#depth + 1;
this.#backlog = 0;
}
/** Parses the complete members in `pending` into the frame's container. */
#parsePending(frame: Frame): void {
const text = this.#text(frame.pending, frame.lastEnd);
frame.pending = -1;
if (frame.kind === ROOT) {
this.result = JSON.parse(text);
} else if (frame.kind === ARRAY) {
const out = frame.out as unknown[];
const items = JSON.parse(`[${text}]`) as unknown[];
for (let k = 0; k < items.length; k++) out.push(items[k]);
} else {
const out = frame.out as Record<string, unknown>;
const members = JSON.parse(`{${text}}`) as Record<string, unknown>;
for (const key of Object.keys(members)) {
setOwn(out, key, members[key]);
}
}
}
#attach(frame: Frame, key: string | null, value: unknown): void {
if (frame.kind === ROOT) this.result = value;
else if (frame.kind === ARRAY) (frame.out as unknown[]).push(value);
else setOwn(frame.out as Record<string, unknown>, key!, value);
}
#key(frame: Frame): string {
return JSON.parse(
this.#text(frame.keyStart, frame.keyEnd + 1),
) as string;
}
#text(from: number, to: number): string {
return this.#body.toString('utf8', from, to);
}
}
/**
* Where a string's content can be cut at or before `target` without splitting
* an escape or a UTF-8 sequence. `from` starts a character or an escape.
*/
const pieceEnd = (body: Buffer, from: number, target: number): number => {
// Far enough back that a `\uXXXX` reaching `target` starts after it.
let at = Math.max(from, target - 8);
// An odd run of backslashes before `at` means `at` is an escaped byte.
let run = 0;
while (at - run > from && body[at - run - 1] === BACKSLASH) run++;
at -= run % 2;
for (;;) {
const length =
body[at] !== BACKSLASH ? 1 : body[at + 1] === 0x75 ? 6 : 2;
if (at + length > target) break;
at += length;
}
// A sequence has at most three continuation bytes; more are stray.
for (let back = 0; back < 3 && (body[at]! & 0xc0) === 0x80; back++) at--;
return at;
};
/** The string literal at `[start, close]`, decoded a piece per turn. */
const readLongString = async (
body: Buffer,
start: number,
close: number,
): Promise<string> => {
const pieces: string[] = [];
let at = start + 1;
while (at < close) {
const end =
close - at <= STRING_PIECE_BYTES
? close
: pieceEnd(body, at, at + STRING_PIECE_BYTES);
pieces.push(
JSON.parse(`"${body.toString('utf8', at, end)}"`) as string,
);
at = end;
await nextTurn();
}
return pieces.join('');
};
/**
* `JSON.parse` of UTF-8 `body`, done in slices with the event loop running
* between them, so no input holds it for long. Throws a `SyntaxError` for
* malformed JSON, and for nesting deeper than `MAX_DEPTH`.
*/
export async function parseJsonInSlices(body: Buffer): Promise<unknown> {
if (body.length <= SLICE_BYTES) return JSON.parse(body.toString('utf8'));
const parser = new SlicedJsonParser(body);
for (;;) {
const signal = parser.run(performance.now() + TURN_BUDGET_MS);
if (signal === DONE) return parser.result;
if (signal === LONG_STRING) {
parser.resolveLongString(
await readLongString(body, parser.longStart, parser.longEnd),
);
} else {
await nextTurn();
}
}
}
+1 -1
View File
@@ -159,7 +159,7 @@ The [`puter.ai.realtime()`](/AI/realtime/) SDK also applies these limits before
`/puterai/anthropic/v1/messages/count_tokens` has its own budget of 120 requests per minute per user, with no concurrency limit, and is not charged.
Chat requests accept JSON bodies up to 128 MiB (134,217,728 bytes), including `text/plain;actually=json` bodies. That covers `puter-chat-completion` calls through `/drivers/call`, which `puter.ai.chat()` uses, and POSTs to `/puterai/openai/v1/*`, `/puterai/anthropic/v1/messages` and `/puterai/anthropic/v1/messages/count_tokens`. A body over 50 MiB needs a `Content-Length` header and a token in the `Authorization` or `x-api-key` header (or a session cookie); an `auth_token` inside the body isn't read until the body is, so it only works up to 50 MiB. Larger bodies, and bodies over 50 MiB for other drivers, return HTTP 413. One account can have up to 128 MiB of these large bodies in flight at once; past that, requests get `429` with `Retry-After`. A server handling too many large bodies at once returns `503` with `Retry-After`. Retry either after the given number of seconds. Other routes accept JSON bodies up to 50 MiB.
Chat requests accept JSON bodies up to 128 MiB (134,217,728 bytes), including `text/plain;actually=json` bodies. That covers `puter-chat-completion` calls through `/drivers/call`, which `puter.ai.chat()` uses, and POSTs to `/puterai/openai/v1/*`, `/puterai/anthropic/v1/messages` and `/puterai/anthropic/v1/messages/count_tokens`. A body over 50 MiB needs a `Content-Length` header and a credential that can be read before the rest of the body. That is a token in the `Authorization` or `x-api-key` header, a session cookie, or an `auth_token` that is the first field of the body, which is where Puter.js sends it. An `auth_token` after other fields isn't read until the body is, so it only works up to 50 MiB. Larger bodies, and bodies over 50 MiB for other drivers, return HTTP 413. One account can have up to 128 MiB of these large bodies in flight at once; past that, requests get `429` with `Retry-After`. A server handling too many large bodies at once returns `503` with `Retry-After`. Retry either after the given number of seconds. Other routes accept JSON bodies up to 50 MiB. A JSON body in a charset other than UTF-8 is accepted up to 1 MiB. A JSON body over 32 KiB can nest arrays and objects up to 10,000 levels deep, and deeper nesting returns HTTP 400.
Claude's server-executed tools are clamped so a request can't reserve an unbounded amount of credit:
@@ -69,6 +69,7 @@ beforeAll(async () => {
[sessionToken]: { uuid: sessionUser, username: 'session-user' },
'app-token-session-user': { uuid: sessionUser, username: 'session-user' },
'app-token-other-user': { uuid: otherUser, username: 'other-user' },
'app-token-subscriber': { uuid: sessionUser, username: 'session-user', subscribed: true },
};
// The SDK on a sibling subdomain sends credentialed cross-origin requests.
app.use(['/whoami', '/profile', '/rao'], (req, res, next) => {
@@ -513,6 +514,81 @@ describe('embedded app browser', () => {
}
});
it('offers Add credits to an account with a plan, and frames the top-up over the page', async () => {
const context = await menuContext('app-token-subscriber');
try {
const page = await context.newPage();
page.setDefaultTimeout(6000);
await page.goto(`http://spreadsheet.puter.localhost:${port}/menu-host`);
await page.locator('puter-user-menu .avatar').click();
const item = page.locator('puter-user-menu .add-credits');
await item.waitFor();
expect(await item.textContent()).toBe('Add credits');
// Under Usage, in the same row style as the links around it.
expect(await page.locator('puter-user-menu .usage + .add-credits').count()).toBe(1);
expect(await item.evaluate(el => getComputedStyle(el).height))
.toBe(await page.locator('puter-user-menu .feedback').evaluate(el => getComputedStyle(el).height));
await item.click();
const frame = page.locator('dialog.puter-popup-frame iframe');
await frame.waitFor({ state: 'attached' });
expect(await frame.getAttribute('src')).toBe(`${origin}/embed/top-up#user=${sessionUser}`);
expect(await frame.getAttribute('allow')).toBe('payment; otp-credentials');
expect(await page.locator('puter-user-menu .menu').isHidden()).toBe(true);
} finally {
guestApps = [];
await context.close();
}
});
it('returns keyboard focus to the account button when Add credits is chosen', async () => {
const context = await menuContext('app-token-subscriber');
try {
const page = await context.newPage();
page.setDefaultTimeout(6000);
await page.goto(`http://spreadsheet.puter.localhost:${port}/menu-host`);
await page.locator('puter-user-menu .avatar').focus();
await page.keyboard.press('Enter');
await page.locator('puter-user-menu .add-credits').waitFor();
await page.locator('puter-user-menu .add-credits').focus();
await page.keyboard.press('Enter');
await page.locator('dialog.puter-popup-frame iframe').waitFor({ state: 'attached' });
const focused = () => page.evaluate(() => document.activeElement.shadowRoot?.activeElement?.className ?? document.activeElement.tagName);
expect(await focused()).toBe('avatar');
// The dialog hands the focus back when it closes.
const fromFrame = (state) => page.evaluate(([guiOrigin, frameState]) => {
const source = document.querySelector('dialog.puter-popup-frame iframe').contentWindow;
window.dispatchEvent(new MessageEvent('message', {
data: { msg: 'puterPopupFrame', state: frameState }, origin: guiOrigin, source,
}));
}, [origin, state]);
await fromFrame('ready');
expect(await page.locator('dialog.puter-popup-frame').evaluate(el => el.open)).toBe(true);
await fromFrame('closed');
await page.locator('dialog.puter-popup-frame').waitFor({ state: 'detached' });
expect(await focused()).toBe('avatar');
} finally {
guestApps = [];
await context.close();
}
});
it('leaves Add credits out of the menu of an account without a plan', async () => {
const context = await menuContext('app-token-session-user');
try {
const page = await context.newPage();
page.setDefaultTimeout(6000);
await page.goto(`http://spreadsheet.puter.localhost:${port}/menu-host`);
await page.locator('puter-user-menu .avatar').click();
await page.locator('puter-user-menu .usage').waitFor();
expect(await page.locator('puter-user-menu .add-credits').isHidden()).toBe(true);
} finally {
guestApps = [];
await context.close();
}
});
it('closes the account menu when Tab leaves it', async () => {
const context = await menuContext('app-token-session-user');
try {
+32 -9
View File
@@ -19,7 +19,7 @@
import { formatCount, formatCredits, formatDollarsFromMicrocents, usageIsCredits } from './credits.js';
import { isOrgSeat } from './orgSeat.js';
import { addonCreditsRemaining, meterPercent, monthlyUsageOrThrow, usageBudget } from './usageBudget.js';
import { addonCreditsBudget, addonCreditsRemaining, meterPercent, monthlyUsageOrThrow, usageBudget } from './usageBudget.js';
import { getMonthlyUsage, getStorageSpace } from './usageRequests.js';
// Whether the server reported credits — decides whether the usage surfaces
@@ -67,7 +67,7 @@ const TabUsage = {
</div>
<div class="driver-usage-container" style="margin-top: 30px;">
<div class="driver-usage-header">
<h3 style="margin:0; font-size: 14px; flex-grow: 1; font-weight: 500;">${i18n('Resources')}</h3>
<h3 style="margin:0; font-size: 14px; flex-grow: 1; font-weight: 500;">${i18n('monthly_credits_title')}</h3>
<div style="font-size: 13px; margin-bottom: 3px; opacity:0.85;">
<span class="usage-progbar-percent">--%</span>
<span> &middot; </span>
@@ -77,10 +77,22 @@ const TabUsage = {
<div class="usage-progbar-wrapper">
<div class="usage-progbar" style="width: 0;"></div>
</div>
<p class="usage-addon-credits" style="display: none;">
<span class="usage-addon-left"></span>
<span class="usage-addon-note">&middot; ${i18n('addon_credits_note')}</span>
</p>
<div class="usage-addon" style="display: none;">
<div class="driver-usage-header">
<h3 id="usage-addon-title" style="margin:0; font-size: 14px; flex-grow: 1; font-weight: 500;">${i18n('addon_credits')}</h3>
<div style="font-size: 13px; margin-bottom: 3px; opacity:0.85;">
<span class="usage-addon-percent">--%</span>
<span> &middot; </span>
${i18n('used_of', { used: '<span class="usage-addon-used">--</span>', available: '<span class="usage-addon-capacity">--</span>' }, false)}
</div>
</div>
<div class="usage-progbar-wrapper" role="progressbar" aria-labelledby="usage-addon-title" aria-valuemin="0" aria-valuemax="100">
<div class="usage-addon-bar" style="width: 0;"></div>
</div>
<p class="usage-addon-credits">
<span class="usage-addon-note">${i18n('addon_credits_note')}</span>
</p>
</div>
<h3 style="margin:15px 0 10px 0; font-size: 14px; font-weight: 500;">${i18n('usage_details')}</h3>
<div class="driver-usage-details-content visible">
</div>
@@ -330,9 +342,20 @@ async function update_usage_details ($el_window) {
'background-color': window.usage_bar_color(budget.barPercent),
});
$('.usage-addon-credits').toggle(addonLeft !== null);
if ( addonLeft !== null ) {
$('.usage-addon-left').html(i18n('addon_credits_available', { amount: amount(addonLeft) }));
// Its own meter under the monthly one, for an account that has ever had any.
const addon = addonCreditsBudget(res.allowanceInfo);
$('.usage-addon').toggle(addon !== null);
if ( addon !== null ) {
$('.usage-addon-used').html(amount(addon.used));
$('.usage-addon-capacity').html(amount(addon.capacity));
$('.usage-addon-percent').html(`${addon.percent}%`);
$('.usage-addon-bar').css({
width: `${addon.barPercent}%`,
'background-color': window.usage_bar_color(addon.barPercent),
}).parent().attr({
'aria-valuenow': addon.percent,
'aria-valuetext': `${addon.percent}%`,
});
}
// Store raw data for sorting
+24 -2
View File
@@ -30,8 +30,8 @@ export const meterPercent = (share) => (share < 100 ? Math.min(99, Math.round(sh
/**
* @typedef {Object} UsageBudget
* @property {number} used - Allowance-charged spend, in the server's units.
* @property {number} capacity - The monthly plan allowance.
* @property {number} used - Spend charged to it, in the server's units.
* @property {number} capacity - What the meter is out of: the monthly plan allowance, or the add-on credit bought.
* @property {number} percent - `used` as a whole-number share of `capacity`.
* @property {number} barPercent - The share clamped to 0-100, for a bar width.
*/
@@ -83,6 +83,28 @@ export const addonCreditsRemaining = (allowanceInfo) => {
return Math.max(0, purchased - nonNegative(addons.consumedPurchaseCredits));
};
/**
* The add-on credit meter: what has been spent of all the add-on credit the
* account ever bought. Null when it has never had any, as there is no pool
* to meter.
*
* @param {Object | null | undefined} allowanceInfo
* @returns {UsageBudget | null}
*/
export const addonCreditsBudget = (allowanceInfo) => {
const left = addonCreditsRemaining(allowanceInfo);
if ( left === null ) return null;
const capacity = nonNegative(allowanceInfo.addons.purchasedCredits);
const used = capacity - left;
const share = (used / capacity) * 100;
return {
used,
capacity,
percent: meterPercent(share),
barPercent: Math.max(0, Math.min(100, share)),
};
};
/**
* `getMonthlyUsage()` resolves with whatever body the server answered, an
* error included. Throws unless that body is a usage report, so a failed
+36 -1
View File
@@ -18,7 +18,7 @@
*/
import { describe, expect, it } from 'vitest';
import { addonCreditsRemaining, meterPercent, monthlyUsageOrThrow, usageBudget } from './usageBudget.js';
import { addonCreditsBudget, addonCreditsRemaining, meterPercent, monthlyUsageOrThrow, usageBudget } from './usageBudget.js';
const info = (allowance, addons = {}) => ({
monthUsageAllowance: allowance,
@@ -148,6 +148,41 @@ describe('addonCreditsRemaining', () => {
});
});
describe('addonCreditsBudget', () => {
it('has no meter for an account that never had add-on credit', () => {
expect(addonCreditsBudget(info(900_000_000))).toBeNull();
expect(addonCreditsBudget(info(900_000_000, { purchasedCredits: 0 }))).toBeNull();
expect(addonCreditsBudget(undefined)).toBeNull();
});
it('meters what was spent of the credit bought', () => {
expect(addonCreditsBudget(info(900_000_000, {
purchasedCredits: 2_500,
consumedPurchaseCredits: 850,
}))).toEqual({ used: 850, capacity: 2_500, percent: 34, barPercent: 34 });
});
it('starts empty when nothing has been spent, or the spend is unreadable', () => {
expect(addonCreditsBudget(info(0, { purchasedCredits: 1_000 })))
.toEqual({ used: 0, capacity: 1_000, percent: 0, barPercent: 0 });
expect(addonCreditsBudget(info(0, { purchasedCredits: 1_000, consumedPurchaseCredits: NaN })).used).toBe(0);
});
it('reads full, never past it, once the pool is spent', () => {
expect(addonCreditsBudget(info(0, {
purchasedCredits: 1_000,
consumedPurchaseCredits: 1_200,
}))).toEqual({ used: 1_000, capacity: 1_000, percent: 100, barPercent: 100 });
});
it('holds at 99% while any credit is left', () => {
expect(addonCreditsBudget(info(0, {
purchasedCredits: 100_000,
consumedPurchaseCredits: 99_999,
})).percent).toBe(99);
});
});
describe('monthlyUsageOrThrow', () => {
it('passes a usage report through', () => {
const res = { usage: { total: 0 }, allowanceInfo: null };
+6 -2
View File
@@ -5037,17 +5037,21 @@ fieldset[name=number-code] {
align-items: stretch;
}
#storage-bar, #storage-bar-host, .usage-progbar {
#storage-bar, #storage-bar-host, .usage-progbar, .usage-addon-bar {
height: 100%;
background-color: var(--dashboard-bento-usage-bar-fill, #ea580c);
width: 0;
transition: width 0.4s ease, background-color 0.4s ease;
}
#storage-used-percent, .usage-progbar-percent {
#storage-used-percent, .usage-progbar-percent, .usage-addon-percent {
font-weight: 500;
}
.usage-addon {
margin-top: 18px;
}
.usage-addon-credits {
margin: 8px 0 0;
font-size: 13px;
+61 -17
View File
@@ -30,6 +30,7 @@ export const FRAMED_POPUP_ACTIONS = new Set([
'show-save-file-picker',
'show-directory-picker',
'upgrade',
'top-up',
'request-email-confirmation',
]);
@@ -39,17 +40,18 @@ const HOST_USER_ACTIONS = new Set([
'show-save-file-picker',
'show-directory-picker',
'upgrade',
'top-up',
'request-email-confirmation',
]);
// Actions framed over the whole page with no box, which their own dialog's
// scrim covers.
const TRANSPARENT_ACTIONS = new Set(['upgrade', 'request-email-confirmation']);
const TRANSPARENT_ACTIONS = new Set(['upgrade', 'top-up', 'request-email-confirmation']);
const UPGRADE_REASONS = new Set(['funds', 'subscription', 'storage']);
const UPGRADE_METHOD_MAX = 200;
// Modal UI the upgrade dialog and the GUI's windows show. Once some has shown
// Modal UI the upgrade and top-up dialogs and the GUI's windows show. Once some has shown
// and all of it has been gone this long, the dialog closed without reporting
// it, and the frame mustn't be left covering the page.
const UPGRADE_MODAL_UI = '.window, [aria-modal="true"]';
@@ -251,6 +253,23 @@ export const canShowUpgradeFrame = (win, session) =>
!session.requires_card_verification &&
!session.requires_password_change;
/**
* Whether a framed top-up can show for the GUI session `session` (its
* `/whoami`): the deployment has a top-up dialog that reports when it closes,
* and no account check is waiting, which a frame couldn't be closed from.
*
* @param {Window} win
* @param {Record<string, unknown> | null} session
* @returns {boolean}
*/
export const canShowTopUpFrame = (win, session) =>
win.UITopUp?.embeddable === true &&
!!session &&
!session.requires_email_confirmation &&
!session.requires_phone_verification &&
!session.requires_card_verification &&
!session.requires_password_change;
/**
* Whether a framed email confirmation can show for the GUI session `session`
* (its `/whoami`): no check other than the email is waiting, since those show
@@ -300,19 +319,18 @@ export const framedUpgradeDetails = (hash) => {
};
/**
* Shows the deployment's upgrade dialog in a framed popup and tells the
* Shows one of the deployment's dialogs in a framed popup and tells the
* framing page it's ready once it's open, or before it opens when the page
* still hides the frame. Resolves whether the user upgraded,
* once the dialog closes, or false once every dialog and window it showed has
* gone without it reporting a close.
* still hides the frame. Resolves what the dialog reports when it closes, or
* false once every dialog and window it showed has gone without it reporting
* a close.
*
* @param {Window} win
* @param {{ appUid?: string, reason?: string, method?: string }} attribution - what prompted it;
* an empty one is passed as null, so the dialog credits the prompt it remembers
* @param {(closed: (result: boolean) => void) => unknown} open - opens the dialog, which calls `closed` when it closes
* @param {(state: 'ready') => void} tell
* @returns {Promise<boolean>}
*/
export const showFramedUpgrade = (win, attribution, tell) => new Promise((resolve, reject) => {
const showFramedDialog = (win, open, tell) => new Promise((resolve, reject) => {
let settled = false;
let seen = false;
let goneTimer = null;
@@ -345,21 +363,47 @@ export const showFramedUpgrade = (win, attribution, tell) => new Promise((resolv
toldReady = true;
tell('ready');
};
const open = () => {
const dialog = new win.UIUpgradeAccount({
attribution: Object.keys(attribution).length ? attribution : null,
onClose: (result) => settle(() => resolve(result?.upgraded === true)),
});
return dialog.open_as_window();
};
whenFrameSized(win, ready)
.then(open)
.then(() => open((result) => settle(() => resolve(result))))
.then(() => {
check();
ready();
}, (e) => settle(() => reject(e)));
});
/**
* Shows the deployment's upgrade dialog in a framed popup. Resolves whether
* the user upgraded, once the dialog closes.
*
* @param {Window} win
* @param {{ appUid?: string, reason?: string, method?: string }} attribution - what prompted it;
* an empty one is passed as null, so the dialog credits the prompt it remembers
* @param {(state: 'ready') => void} tell
* @returns {Promise<boolean>}
*/
export const showFramedUpgrade = (win, attribution, tell) => showFramedDialog(win, (closed) => {
const dialog = new win.UIUpgradeAccount({
attribution: Object.keys(attribution).length ? attribution : null,
onClose: (result) => closed(result?.upgraded === true),
});
return dialog.open_as_window();
}, tell);
/**
* Shows the deployment's top-up dialog in a framed popup. Resolves whether
* the user bought credit, once the dialog closes.
*
* @param {Window} win
* @param {(state: 'ready') => void} tell
* @returns {Promise<boolean>}
*/
export const showFramedTopUp = (win, tell) => showFramedDialog(win, (closed) => {
const dialog = new win.UITopUp({
onClose: (result) => closed(result?.toppedUp === true),
});
return dialog.open();
}, tell);
/**
* Resolves once the frame has a viewport. The page keeps a loading frame
* hidden, which Chromium and WebKit lay out at 0x0, and a GUI window opened
+95
View File
@@ -3,6 +3,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest';
import {
POPUP_FRAME_MSG,
canShowEmailConfirmationFrame,
canShowTopUpFrame,
canShowUpgradeFrame,
closeAccountChecksOnEscape,
confirmEmailInFrame,
@@ -11,6 +12,7 @@ import {
framedPopupParams,
framedUpgradeDetails,
framesHostUser,
showFramedTopUp,
showFramedUpgrade,
startPopupFrame,
} from './popupFrame.js';
@@ -50,6 +52,7 @@ describe('framedPopupAction', () => {
['/embed/show-save-file-picker', 'show-save-file-picker'],
['/embed/show-directory-picker', 'show-directory-picker'],
['/embed/upgrade', 'upgrade'],
['/embed/top-up', 'top-up'],
['/embed/request-email-confirmation', 'request-email-confirmation'],
['/embed/apps', null],
['/embed/feedback', null],
@@ -85,6 +88,7 @@ describe('framesHostUser', () => {
['show-save-file-picker', true],
['show-directory-picker', true],
['upgrade', true],
['top-up', true],
['request-email-confirmation', true],
['sign-in', false],
[null, false],
@@ -133,6 +137,34 @@ describe('canShowUpgradeFrame', () => {
});
});
describe('canShowTopUpFrame', () => {
const win = { UITopUp: Object.assign(function () {}, { embeddable: true }) };
const session = { uuid: 'u-1' };
it('shows for a session with nothing to verify, when the dialog reports its close', () => {
expect(canShowTopUpFrame(win, session)).toBe(true);
});
it('needs a top-up dialog that reports its close, whatever the upgrade dialog does', () => {
expect(canShowTopUpFrame({}, session)).toBe(false);
expect(canShowTopUpFrame({ UITopUp: function () {} }, session)).toBe(false);
expect(canShowTopUpFrame({ UIUpgradeAccount: Object.assign(function () {}, { embeddable: true }) }, session)).toBe(false);
});
it.each([
'requires_email_confirmation',
'requires_phone_verification',
'requires_card_verification',
'requires_password_change',
])('leaves an account with %s to the tab', (check) => {
expect(canShowTopUpFrame(win, { ...session, [check]: true })).toBe(false);
});
it('needs a session', () => {
expect(canShowTopUpFrame(win, null)).toBe(false);
});
});
describe('canShowEmailConfirmationFrame', () => {
const session = { uuid: 'u-1' };
@@ -361,6 +393,64 @@ describe('showFramedUpgrade', () => {
});
});
describe('showFramedTopUp', () => {
const topUpWindow = (open) => {
const events = new EventTarget();
return {
document,
innerWidth: 1280,
innerHeight: 800,
addEventListener: events.addEventListener.bind(events),
removeEventListener: events.removeEventListener.bind(events),
MutationObserver: window.MutationObserver,
UITopUp: function (options) {
this.open = () => open(options);
},
};
};
afterEach(() => {
vi.useRealTimers();
});
it('tells the page it is ready once open, and resolves whether credit was bought', async () => {
const tell = vi.fn();
let options;
const result = showFramedTopUp(topUpWindow(async (o) => {
options = o;
}), tell);
await vi.waitFor(() => expect(tell).toHaveBeenCalledWith('ready'));
expect(tell).toHaveBeenCalledTimes(1);
options.onClose({ toppedUp: true });
await expect(result).resolves.toBe(true);
});
it('resolves false for a close without a purchase', async () => {
await expect(showFramedTopUp(topUpWindow(async (o) => o.onClose({ toppedUp: false })), vi.fn())).resolves.toBe(false);
await expect(showFramedTopUp(topUpWindow(async (o) => o.onClose()), vi.fn())).resolves.toBe(false);
});
it('resolves false once the dialog is gone without reporting a close', async () => {
vi.useFakeTimers();
const root = document.createElement('div');
root.setAttribute('aria-modal', 'true');
const result = showFramedTopUp(topUpWindow(async () => {
document.body.append(root);
}), vi.fn());
await vi.advanceTimersByTimeAsync(0);
root.remove();
await vi.advanceTimersByTimeAsync(600);
await expect(result).resolves.toBe(false);
});
it('rejects when the dialog fails to open', async () => {
await expect(showFramedTopUp(topUpWindow(async () => {
throw new Error('no dialog');
}), vi.fn())).rejects.toThrow('no dialog');
});
});
describe('startPopupFrame', () => {
it('does nothing without a framing page to answer', () => {
const win = frameWindow();
@@ -396,6 +486,11 @@ describe('startPopupFrame', () => {
);
});
it('lets the framing page show through a top-up frame', () => {
startPopupFrame({ ...frameWindow(), location: { pathname: '/embed/top-up' } }, HOST);
expect(document.documentElement.classList.contains('popup-frame-transparent')).toBe(true);
});
it('lets the framing page show through an upgrade or email confirmation frame only', () => {
startPopupFrame({ ...frameWindow(), location: { pathname: '/embed/sign-in' } }, HOST);
expect(document.documentElement.classList.contains('popup-frame-transparent')).toBe(false);
+2
View File
@@ -28,6 +28,7 @@ const en = {
account_password: 'Verify Account Password',
access_granted_to: 'Access Granted To',
add_another_account: 'Add Another Account',
addon_credits: 'Add-on credits',
addon_credits_available: '+ {{amount}} add-on credits left',
addon_credits_note: 'Used once your monthly credits run out.',
add_app: 'Add an app',
@@ -327,6 +328,7 @@ const en = {
delete_account_description: 'Permanently delete your account and all associated data. This action cannot be undone.',
modified: 'Modified',
monthly_credits: 'monthly credits',
monthly_credits_title: 'Monthly credits',
move: 'Move',
moving_file: 'Moving %%',
my_websites: 'My Websites',
+20
View File
@@ -49,6 +49,7 @@ import { isAttestedOrigin } from './helpers/attestedOrigin.js';
import { embedHostUser } from './helpers/embedHostUser.js';
import {
canShowEmailConfirmationFrame,
canShowTopUpFrame,
canShowUpgradeFrame,
closeAccountChecksOnEscape,
confirmEmailInFrame,
@@ -57,6 +58,7 @@ import {
framedPopupParams,
framedUpgradeDetails,
framesHostUser,
showFramedTopUp,
showFramedUpgrade,
startPopupFrame,
} from './helpers/popupFrame.js';
@@ -1048,6 +1050,23 @@ const postAuthActions = async (action) => {
window.tell_popup_frame('closed', { upgraded });
}
// -------------------------------------------------------------------------------------
// Action: top-up — the deployment's top-up dialog for the first-party page framing it,
// which hears whether the user bought credit once the dialog closes.
// -------------------------------------------------------------------------------------
if ( action === 'top-up' && window.embedded_in_frame ) {
let toppedUp;
try {
toppedUp = await showFramedTopUp(window, window.tell_popup_frame);
} catch (e) {
// A dialog that failed to open leaves the page to open the top-up in a tab.
console.error('top-up action failed', e);
window.tell_popup_frame('unavailable');
return;
}
window.tell_popup_frame('closed', { toppedUp });
}
// -------------------------------------------------------------------------------------
// Action: request-email-confirmation — puter.ui.requestEmailConfirmation() on the
// first-party page framing it, which hears whether the email is confirmed once the
@@ -1841,6 +1860,7 @@ window.initgui = async function (options) {
!host_user ||
session_user !== host_user ||
(action === 'upgrade' && !canShowUpgradeFrame(window, session)) ||
(action === 'top-up' && !canShowTopUpFrame(window, session)) ||
(action === 'request-email-confirmation' && !canShowEmailConfirmationFrame(session))
) {
window.tell_popup_frame('unavailable');
+10 -7
View File
@@ -297,7 +297,9 @@ const mountGuiEmbed = ({ guiOrigin, path, fragment, className, title, stateMsg,
const removal = new MutationObserver(() => {
if ( ! dialog.isConnected ) dismiss();
});
const restoreFocus = document.activeElement;
// Down through shadow roots: a component's host can't take the focus back itself.
let restoreFocus = document.activeElement;
while ( restoreFocus?.shadowRoot?.activeElement ) restoreFocus = restoreFocus.shadowRoot.activeElement;
const wait = (ms) => {
clearTimeout(timer);
timer = setTimeout(() => finish(null), ms);
@@ -454,12 +456,13 @@ const POPUP_FRAME_BOXES = {
'show-directory-picker': { width: 700, height: 400 },
};
// The upgrade has no box: it covers the page and its dialog draws its own
// scrim. A payment in it may use the Payment Request API, which a frame can
// use only when allowed, and Escape is left to the dialog, which doesn't close
// mid-entry.
// The upgrade and the top-up have no box: each covers the page and its dialog
// draws its own scrim. A payment in it may use the Payment Request API, which
// a frame can use only when allowed, and Escape is left to the dialog, which
// doesn't close mid-entry.
const UNBOXED_POPUP_FRAMES = {
upgrade: { allow: 'payment; otp-credentials', escapeDismisses: false },
'top-up': { allow: 'payment; otp-credentials', escapeDismisses: false },
};
/**
@@ -476,8 +479,8 @@ const UNBOXED_POPUP_FRAMES = {
* @template T
* @param {Object} opts
* @param {string} opts.guiOrigin
* @param {'sign-in' | 'show-open-file-picker' | 'show-save-file-picker' | 'show-directory-picker' | 'upgrade' | 'request-email-confirmation'} opts.action
* @param {Record<string, string>} opts.fragment - the popup's parameters, and `user` for a picker, the upgrade or the email confirmation
* @param {'sign-in' | 'show-open-file-picker' | 'show-save-file-picker' | 'show-directory-picker' | 'upgrade' | 'top-up' | 'request-email-confirmation'} opts.action
* @param {Record<string, string>} opts.fragment - the popup's parameters, and `user` for a picker, the upgrade, the top-up or the email confirmation
* @param {string} opts.title - the frame's accessible name
* @param {string} [opts.closeLabel] - puts a close button on the frame
* @param {(data?: Record<string, unknown>) => T} opts.closed
+5 -2
View File
@@ -1149,15 +1149,18 @@ const DRIVER_CONTENT_TYPE = 'text/plain;actually=json';
const callInstance = (call) => call.puter ?? globalThis.puter;
/** The wire body. Fields left `undefined` drop out of the JSON. */
/**
* The wire body. Fields left `undefined` drop out of the JSON. The token goes
* first, where the server can read it before it has the rest of a large body.
*/
const callBody = (call, puter) =>
JSON.stringify({
auth_token: puter.authToken,
interface: call.iface,
driver: call.driver,
test_mode: call.testMode,
method: call.method,
args: call.args,
auth_token: puter.authToken,
});
const logCall = (call, fields) => {
@@ -1233,6 +1233,14 @@ describe('driverCall', () => {
});
});
it('opens the body with the token, ahead of the arguments', async () => {
const xhrs = installFakeXHR(
respond({ body: { success: true, result: 'v' } }),
);
await driverCall({ ...call, args: { key: '"auth_token":"decoy"' } });
expect(xhrs[0].reqBody.startsWith('{"auth_token":"tok",')).toBe(true);
});
it('sends driver and test_mode only when the caller sets them', async () => {
const xhrs = installFakeXHR(
respond({ body: { success: true, result: {} } }),
+3 -2
View File
@@ -26,7 +26,7 @@ import { showUsageLimitDialog } from '../modules/UsageLimitDialog.js';
*
* @typedef {{
* env?: string,
* ui?: { requestUpgrade: Function, canShowUpgradeFrame_?: () => boolean, upgradeFrameShowing_?: () => boolean },
* ui?: { requestUpgrade: Function, canShowUpgradeFrame_?: () => boolean, upgradeFrameShowing_?: () => boolean, topUpShowing_?: () => boolean },
* authToken?: string,
* appID?: string,
* getAppIDFromAuthToken?: (token: unknown) => string | null,
@@ -129,8 +129,9 @@ export const promptIfUpgradeRequired = (error, context = {}, puter = globalThis.
puter.ui.requestUpgrade({ reason, method, message });
} else if ( puter?.env === 'web' || reason === 'storage' ) {
// The upgrade a first-party page already shows answers this refusal
// too, and a dialog over it would block it.
// too, and a dialog over it, or over a top-up, would block it.
if ( puter.ui?.upgradeFrameShowing_?.() === true ) return;
if ( puter.ui?.topUpShowing_?.() === true ) return;
// A first-party page shows the upgrade in a frame over itself
// instead of a dashboard tab.
const framed = puter?.env === 'web' && puter.ui?.canShowUpgradeFrame_?.() === true;
@@ -217,6 +217,14 @@ describe('promptIfUpgradeRequired', () => {
expect(globalThis.puter.ui.requestUpgrade).not.toHaveBeenCalled();
});
it('shows no dialog over a top-up the page already shows', () => {
globalThis.puter.ui.topUpShowing_ = () => true;
promptIfUpgradeRequired({ code: 'insufficient_funds', status: 402 }, { method: 'puter.ai.chat' });
promptIfUpgradeRequired({ code: 'storage_limit_reached' }, { method: 'puter.fs.upload' });
expect(showUsageLimitDialog).not.toHaveBeenCalled();
expect(globalThis.puter.ui.requestUpgrade).not.toHaveBeenCalled();
});
it('keeps the tab when the page cannot frame it', () => {
globalThis.puter.ui.canShowUpgradeFrame_ = () => false;
promptIfUpgradeRequired({ code: 'storage_limit_reached' }, { method: 'puter.fs.upload' });
+71 -7
View File
@@ -5,7 +5,7 @@ import PuterDialog from './PuterDialog.js';
import { ancestorsAllowGuiEmbed, canFrameGuiEmbed, showFeedbackEmbed, showPopupFrame } from '../lib/guiEmbed.js';
import { hasUserActivation } from '../lib/auth-popup.js';
import { appIDOf } from '../lib/upgradePrompt.js';
import { upgradeUrl } from './UsageLimitDialog.js';
import { topUpUrl, upgradeUrl } from './UsageLimitDialog.js';
import { checkPermissions } from './perms/lib/holds.js';
@@ -538,6 +538,9 @@ export class UIModule extends EventListener {
// The framed upgrade showing, which a call made while it's open joins.
#upgradeFrame = null;
// The top-up showing, framed or about to open in a tab, which a call made meanwhile joins.
#topUpFrame = null;
// The framed email confirmation showing, which a call made while it's open joins.
#emailConfirmationFrame = null;
@@ -1803,13 +1806,20 @@ export class UIModule extends EventListener {
/**
* The upgrade in a frame over the page, for the page's signed-in user, or
* in a new tab when the frame can't show it. A call made while one is
* showing settles with it, so a payment in progress isn't replaced.
* showing settles with it, so a payment in progress isn't replaced, and
* one made while the top-up shows resolves `{ upgraded: false }` once
* that closes.
*
* @param {{ reason?: string, method?: string }} details
* @returns {Promise<import('../lib/types.js').UpgradeResult>}
*/
#showUpgradeFrame ({ reason, method }) {
if ( this.#upgradeFrame ) return this.#upgradeFrame;
// A second full-page frame would cover the top-up, and a payment in it.
if ( this.#topUpFrame ) {
const notUpgraded = () => ({ upgraded: false });
return this.#topUpFrame.then(notUpgraded, notUpgraded);
}
this.#upgradeFrame = (async () => {
try {
const user = await this.#frameUser();
@@ -1825,6 +1835,9 @@ export class UIModule extends EventListener {
closed: (data) => ({ upgraded: data?.upgraded === true }),
onMessage: () => {},
});
// The page's cached whoami still reads it as having no plan,
// which is what the user menu offers Add credits by.
if ( framed?.upgraded ) user.subscribed = true;
if ( framed ) return framed;
await this.#openUpgradeTab(upgradeUrl({ reason, method, appID: appIDOf(puter) }));
return { upgraded: false };
@@ -1836,14 +1849,65 @@ export class UIModule extends EventListener {
}
/**
* Opens the upgrade page in a new tab. The click that asked for it may
* have expired by now, so without it the user opens it from a consent
* dialog. Resolves once the tab opens or the user answers the dialog.
* Whether {@link showTopUp_} is showing the top-up in a frame, about to,
* or asking to open it in a tab instead.
*
* @internal
* @returns {boolean}
*/
topUpShowing_ () {
return this.#topUpFrame !== null;
}
/**
* Shows the top-up flow, where a user with a plan buys add-on credit. On
* a signed-in first-party page one label under the Puter origin, it shows
* in a frame over the page, and the promise resolves once it closes. When
* the frame can't show it, and on any other website, it opens on the
* dashboard in a new tab and the promise resolves `{ toppedUp: false }`.
* A call made while one is showing settles with it, so a payment in
* progress isn't replaced.
*
* @internal
* @returns {Promise<{ toppedUp: boolean }>}
*/
showTopUp_ () {
if ( this.#topUpFrame ) return this.#topUpFrame;
const showing = (async () => {
// Not awaited where no frame can show, so the tab opens within the click.
const user = this.#canFrameForUser() ? await this.#frameUser() : null;
const framed = user && await showPopupFrame({
guiOrigin: puter.defaultGUIOrigin,
action: 'top-up',
fragment: { user: user.uuid },
title: 'Add credits',
closed: (data) => ({ toppedUp: data?.toppedUp === true }),
onMessage: () => {},
});
if ( framed ) return framed;
await this.#openUpgradeTab(topUpUrl(puter.defaultGUIOrigin), 'This website wants to open the Puter top-up page.');
return { toppedUp: false };
})();
this.#topUpFrame = showing;
// Cleared from outside: the steps above can fail before their first
// await, which is before the field is set.
const done = () => {
if ( this.#topUpFrame === showing ) this.#topUpFrame = null;
};
showing.then(done, done);
return showing;
}
/**
* Opens the upgrade or top-up page in a new tab. The click that asked for
* it may have expired by now, so without it the user opens it from a
* consent dialog. Resolves once the tab opens or the user answers the dialog.
*
* @param {string} url
* @param {string} [message] - what the consent dialog asks
* @returns {Promise<void>}
*/
#openUpgradeTab (url) {
#openUpgradeTab (url, message = 'This website wants to open the Puter upgrade page.') {
if ( hasUserActivation() && window.open(url, '_blank') ) return Promise.resolve();
return new Promise((resolve) => {
let consent = null;
@@ -1853,7 +1917,7 @@ export class UIModule extends EventListener {
};
try {
consent = PuterDialog.show({
message: 'This website wants to open the Puter upgrade page.',
message,
onContinue: () => {
window.open(url, '_blank');
answered();
@@ -87,6 +87,26 @@ describe('ui.requestUpgrade on a signed-in first-party page', () => {
expect(window.open).not.toHaveBeenCalled();
});
it('marks the page\'s cached account as having a plan once the user upgrades', async () => {
const account = { uuid: USER, subscribed: false };
puter.whoamiCache_ = Promise.resolve(account);
const ui = makeUI();
let result = ui.requestUpgrade(DETAILS);
await vi.waitFor(() => expect(upgradeFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
fromFrame({ msg: 'puterPopupFrame', state: 'closed' });
await result;
expect(account.subscribed).toBe(false);
result = ui.requestUpgrade(DETAILS);
await vi.waitFor(() => expect(upgradeFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
fromFrame({ msg: 'puterPopupFrame', state: 'closed', upgraded: true });
await result;
expect(account.subscribed).toBe(true);
});
it('says the upgrade is showing from the call until it closes', async () => {
const ui = makeUI();
expect(ui.upgradeFrameShowing_()).toBe(false);
@@ -0,0 +1,239 @@
// @vitest-environment jsdom
// @vitest-environment-options {"url": "https://notes.puter.test/doc"}
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const { UIModule } = await import('./UI.js');
// jsdom has no top-layer dialogs.
HTMLDialogElement.prototype.showModal = function () {
this.open = true;
};
HTMLDialogElement.prototype.close = function () {
this.open = false;
};
const GUI_ORIGIN = 'https://puter.test';
const USER = '0b1c9a52-3f0e-4c7a-9a65-2f6d1e8b7c40';
// On the deployment the page belongs to, where the frame would have shown it.
const TOP_UP_TAB = `${GUI_ORIGIN}/dashboard/?topup=#usage`;
const topUpDialog = () => document.querySelector('dialog.puter-popup-frame');
const topUpFrame = () => topUpDialog()?.querySelector('iframe') ?? null;
const fragment = () => Object.fromEntries(new URLSearchParams(new URL(topUpFrame().src).hash.slice(1)));
const fromFrame = (data) => window.dispatchEvent(new MessageEvent('message', {
data, origin: GUI_ORIGIN, source: topUpFrame().contentWindow,
}));
const makeUI = (env = 'web') => new UIModule(
{ env, authToken: 'token-1', APIOrigin: 'https://api.puter.test', util: {} },
{ appInstanceID: 'instance-1' },
);
const fakePuter = ({ signedIn = true, guiOrigin = GUI_ORIGIN } = {}) => {
globalThis.puter = {
defaultGUIOrigin: guiOrigin,
whoamiCache_: Promise.resolve({ uuid: USER }),
auth: {
isSignedIn: () => signedIn,
getUser: vi.fn(async () => ({ uuid: USER })),
},
};
};
const setActivation = (isActive) => Object.defineProperty(navigator, 'userActivation', {
configurable: true,
value: { hasBeenActive: true, isActive },
});
const PENDING = Symbol('pending');
const settled = (promise) => Promise.race([promise, new Promise((r) => setTimeout(() => r(PENDING), 0))]);
beforeEach(() => {
document.body.innerHTML = '';
window.open = vi.fn(() => ({ closed: false }));
setActivation(true);
fakePuter();
});
afterEach(() => {
delete navigator.userActivation;
delete globalThis.puter;
vi.restoreAllMocks();
});
describe('ui.showTopUp_ that fails before anything shows', () => {
it('stops saying a top-up is showing, and tries again on the next call', async () => {
// No scheme, so the dashboard's address can't be built.
fakePuter({ guiOrigin: 'puter.test' });
const ui = makeUI();
await expect(ui.showTopUp_()).rejects.toThrow();
expect(ui.topUpShowing_()).toBe(false);
fakePuter();
const result = ui.showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
fromFrame({ msg: 'puterPopupFrame', state: 'closed' });
await expect(result).resolves.toEqual({ toppedUp: false });
});
});
describe('ui.showTopUp_ on a signed-in first-party page', () => {
it('shows the top-up in a frame over the page and resolves what the dialog reports', async () => {
const result = makeUI().showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
expect(new URL(topUpFrame().src).pathname).toBe('/embed/top-up');
expect(fragment()).toEqual({ user: USER });
// It covers the page, and the dialog inside draws the scrim.
expect(topUpDialog().querySelector('.puter-popup-frame-box')).toBeNull();
expect(topUpDialog().querySelector('.puter-popup-frame-close')).toBeNull();
expect(topUpFrame().getAttribute('allow')).toBe('payment; otp-credentials');
fromFrame({ msg: 'puterPopupFrame', state: 'loading' });
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
expect(topUpDialog().open).toBe(true);
fromFrame({ msg: 'puterPopupFrame', state: 'closed', toppedUp: true });
await expect(result).resolves.toEqual({ toppedUp: true });
expect(topUpFrame()).toBeNull();
expect(window.open).not.toHaveBeenCalled();
});
it('resolves toppedUp false when the dialog closes without a purchase', async () => {
const result = makeUI().showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
fromFrame({ msg: 'puterPopupFrame', state: 'closed' });
await expect(result).resolves.toEqual({ toppedUp: false });
});
it('says a top-up is showing until it closes', async () => {
const ui = makeUI();
expect(ui.topUpShowing_()).toBe(false);
const result = ui.showTopUp_();
expect(ui.topUpShowing_()).toBe(true);
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
expect(ui.topUpShowing_()).toBe(true);
fromFrame({ msg: 'puterPopupFrame', state: 'closed' });
await result;
expect(ui.topUpShowing_()).toBe(false);
});
it('leaves Escape on the page to the dialog once it shows', async () => {
const result = makeUI().showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
topUpDialog().dispatchEvent(new Event('cancel', { cancelable: true }));
expect(topUpFrame()).not.toBeNull();
expect(await settled(result)).toBe(PENDING);
fromFrame({ msg: 'puterPopupFrame', state: 'closed', toppedUp: false });
await expect(result).resolves.toEqual({ toppedUp: false });
});
it('settles a call made while the top-up shows with it, without replacing it', async () => {
const ui = makeUI();
const first = ui.showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
const frame = topUpFrame();
const second = ui.showTopUp_();
expect(second).toBe(first);
expect(document.querySelectorAll('dialog.puter-popup-frame')).toHaveLength(1);
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
fromFrame({ msg: 'puterPopupFrame', state: 'closed', toppedUp: true });
await expect(second).resolves.toEqual({ toppedUp: true });
// A later call shows a new one.
ui.showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
expect(topUpFrame()).not.toBe(frame);
});
it('opens the dashboard top-up in a tab when the frame cannot show it', async () => {
const result = makeUI().showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'loading' });
fromFrame({ msg: 'puterPopupFrame', state: 'unavailable' });
await expect(result).resolves.toEqual({ toppedUp: false });
expect(topUpFrame()).toBeNull();
expect(window.open).toHaveBeenCalledTimes(1);
expect(window.open).toHaveBeenCalledWith(TOP_UP_TAB, '_blank');
});
it('asks for a click before opening the tab once the original click has expired', async () => {
const result = makeUI().showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
setActivation(false);
fromFrame({ msg: 'puterPopupFrame', state: 'unavailable' });
await vi.waitFor(() => expect(document.querySelector('puter-dialog')).not.toBeNull());
const consent = document.querySelector('puter-dialog');
expect(window.open).not.toHaveBeenCalled();
expect(consent.shadowRoot.querySelector('.about').textContent)
.toBe('This website wants to open the Puter top-up page.');
consent.shadowRoot.querySelector('#launch-auth-popup').click();
await expect(result).resolves.toEqual({ toppedUp: false });
expect(window.open).toHaveBeenCalledWith(TOP_UP_TAB, '_blank');
});
});
describe('ui.showTopUp_ elsewhere on the web', () => {
it.each([
['a page outside the Puter origin', { guiOrigin: 'https://puter.com' }],
['a signed-out first-party page', { signedIn: false }],
])('opens the dashboard top-up in a tab within the click on %s', async (_label, options) => {
fakePuter(options);
const result = makeUI().showTopUp_();
// Before anything is awaited, so the browser still counts the click.
expect(window.open).toHaveBeenCalledTimes(1);
expect(window.open).toHaveBeenCalledWith(`${globalThis.puter.defaultGUIOrigin}/dashboard/?topup=#usage`, '_blank');
expect(topUpFrame()).toBeNull();
await expect(result).resolves.toEqual({ toppedUp: false });
});
});
describe('ui.requestUpgrade beside the top-up', () => {
it('waits for the top-up showing instead of framing the upgrade over it', async () => {
const ui = makeUI();
const toppingUp = ui.showTopUp_();
await vi.waitFor(() => expect(topUpFrame()).not.toBeNull());
fromFrame({ msg: 'puterPopupFrame', state: 'ready' });
const upgrading = ui.requestUpgrade({ reason: 'funds' });
await expect(settled(upgrading)).resolves.toBe(PENDING);
expect(document.querySelectorAll('dialog.puter-popup-frame')).toHaveLength(1);
expect(new URL(topUpFrame().src).pathname).toBe('/embed/top-up');
fromFrame({ msg: 'puterPopupFrame', state: 'closed', toppedUp: true });
await expect(toppingUp).resolves.toEqual({ toppedUp: true });
await expect(upgrading).resolves.toEqual({ upgraded: false });
expect(document.querySelector('dialog.puter-popup-frame')).toBeNull();
expect(window.open).not.toHaveBeenCalled();
});
it('still asks before opening the upgrade tab in its own words', async () => {
const result = makeUI().requestUpgrade({ reason: 'funds' });
const frame = () => document.querySelector('dialog.puter-popup-frame iframe');
await vi.waitFor(() => expect(frame()).not.toBeNull());
expect(new URL(frame().src).pathname).toBe('/embed/upgrade');
setActivation(false);
window.dispatchEvent(new MessageEvent('message', {
data: { msg: 'puterPopupFrame', state: 'unavailable' }, origin: GUI_ORIGIN, source: frame().contentWindow,
}));
await vi.waitFor(() => expect(document.querySelector('puter-dialog')).not.toBeNull());
const consent = document.querySelector('puter-dialog');
expect(consent.shadowRoot.querySelector('.about').textContent)
.toBe('This website wants to open the Puter upgrade page.');
consent.shadowRoot.querySelector('#launch-auth-popup-cancel').click();
await expect(result).resolves.toEqual({ upgraded: false });
});
});
@@ -28,6 +28,19 @@ export const upgradeUrl = ({ reason, appID, method } = {}) => {
return url.href;
};
/**
* The dashboard page that opens the top-up flow, for buying add-on credit.
*
* @param {string} [guiOrigin] The deployment whose dashboard it is; Puter's by default.
* @returns {string}
*/
export const topUpUrl = (guiOrigin) => {
const url = new URL('/dashboard/', guiOrigin || UPGRADE_PAGE);
url.searchParams.set('topup', '');
url.hash = 'usage';
return url.href;
};
class UsageLimitDialog extends (globalThis.HTMLElement || Object) {
/**
* @param {string} message Plain-text line saying what was refused.
+2
View File
@@ -343,6 +343,7 @@ The account button and Puter's app browser for a first-party app on a Puter subd
- Signed in, it shows the user's profile picture, or the username's initial when there is none. Clicking it opens a menu with:
- The user's picture, username and email. The email line is hidden when whoami leaves it out, such as for an app that can't read it.
- **Manage account**, **Usage** and **Open Puter**, links that open the GUI origin's `/dashboard/#account`, `/dashboard/#usage` and `/` in a new tab.
- **Add credits**, under Usage, only when whoami reports `subscribed: true` and no `team` that pays for the account. It shows the top-up dialog in a frame over the page (`/embed/top-up`) on a page that can frame the GUI, and otherwise opens it on the dashboard in a new tab.
- **Send feedback**, which calls `puter.ui.showFeedbackDialog()`.
- **Sign out**, which calls `puter.auth.signOut()`.
- The menu opens with focus on its first item. ArrowUp and ArrowDown move between items and wrap around, and Home and End jump to the first and last.
@@ -362,6 +363,7 @@ The account button and Puter's app browser for a first-party app on a Puter subd
| `account-label` | `Account` | Account button label, read as `Account: <username>` |
| `manage-account-label` | `Manage account` | Menu link to the account settings |
| `usage-label` | `Usage` | Menu link to the usage page |
| `add-credits-label` | `Add credits` | Menu item that opens the top-up dialog |
| `open-puter-label` | `Open Puter` | Menu link to Puter |
| `feedback-label` | `Send feedback` | Menu item that opens the feedback dialog |
| `theme` | follows the OS | `light` or `dark`, for the component and the app browser in it |
@@ -5,7 +5,8 @@
* Signed out (a temporary account counts as signed out), it shows a Sign in
* button. Signed in, it shows the user's profile picture (or initial), which
* opens a menu with the account, links to manage it, see its usage and open
* Puter, Send feedback, and Sign out. On a page one subdomain under
* Puter, Add credits for an account with a plan, Send feedback, and Sign out.
* On a page one subdomain under
* the Puter origin, an Apps button opens Puter's app browser in a panel under
* it, for the same user: signed out, or signed in as an account other than
* the Puter session's, the browser shows only recommended apps. The browser
@@ -13,8 +14,8 @@
* Renders nothing inside the Puter desktop.
*
* Attributes: theme, apps-label, sign-in-label, sign-out-label, close-label,
* account-label, manage-account-label, usage-label, open-puter-label,
* feedback-label
* account-label, manage-account-label, usage-label, add-credits-label,
* open-puter-label, feedback-label
* Parts: apps-button, avatar, sign-in, menu, apps-panel
* Events: signin (detail.user), signout
*/
@@ -37,6 +38,7 @@ const LABELS = {
'account-label': 'Account',
'manage-account-label': 'Manage account',
'usage-label': 'Usage',
'add-credits-label': 'Add credits',
'open-puter-label': 'Open Puter',
'feedback-label': 'Send feedback',
};
@@ -48,6 +50,9 @@ const MENU_LINKS = [
{ className: 'open-puter', label: 'open-puter-label', path: '/' },
];
// Sits under Usage, and shows only for an account with a plan.
const ADD_CREDITS_ITEM = '<button type="button" class="menu-item add-credits" role="menuitem" hidden></button>';
// Gap between a panel and the viewport edge, and between its caret and corners.
const EDGE = 8;
const CARET_INSET = 20;
@@ -513,7 +518,7 @@ class PuterUserMenu extends PuterWebComponent {
<span class="menu-email" hidden></span>
</span>
</div>
${MENU_LINKS.map(link => `<a class="menu-item ${link.className}" role="menuitem" target="_blank" rel="noopener"><span class="menu-label"></span>${EXTERNAL_ICON}</a>`).join('')}
${MENU_LINKS.map(link => `<a class="menu-item ${link.className}" role="menuitem" target="_blank" rel="noopener"><span class="menu-label"></span>${EXTERNAL_ICON}</a>${link.className === 'usage' ? ADD_CREDITS_ITEM : ''}`).join('')}
<button type="button" class="menu-item feedback" role="menuitem"></button>
<div class="menu-separator" role="separator"></div>
<button type="button" class="menu-item sign-out" role="menuitem"></button>
@@ -548,6 +553,7 @@ class PuterUserMenu extends PuterWebComponent {
for ( const link of MENU_LINKS ) {
this.$(`.${link.className}`).addEventListener('click', () => this._closePanels());
}
this.$('.add-credits').addEventListener('click', () => this._addCredits());
this.$('.feedback').addEventListener('click', () => this._sendFeedback());
this.$('.sign-out').addEventListener('click', () => this._signOut());
this.$('.apps-close').addEventListener('click', () => this._closePanels({ returnFocus: true }));
@@ -581,6 +587,7 @@ class PuterUserMenu extends PuterWebComponent {
close.title = this._label('close-label');
this.$('.sign-in').textContent = this._label('sign-in-label');
this.$('.sign-out').textContent = this._label('sign-out-label');
this.$('.add-credits').textContent = this._label('add-credits-label');
this.$('.feedback').textContent = this._label('feedback-label');
for ( const link of MENU_LINKS ) {
this.$(`.${link.className} .menu-label`).textContent = this._label(link.label);
@@ -598,6 +605,19 @@ class PuterUserMenu extends PuterWebComponent {
this.$('.avatar').setAttribute('aria-label', username ? `${account}: ${username}` : account);
}
/**
* Add-on credit is sold to accounts with a plan, which whoami reports where there are
* plans. A seat's plan is its team's, and so is paying for its usage.
*/
_renderAddCredits () {
// The SDK reads whoami into a new object when the same token is set again,
// and an upgrade made on the page marks that one.
const cached = globalThis.puter.whoami;
const user = cached?.uuid && cached.uuid === this.#user?.uuid ? cached : this.#user;
const isSeat = typeof user?.team?.uid === 'string' && user.team.uid !== '';
this.$('.add-credits').hidden = user?.subscribed !== true || isSeat;
}
_renderState () {
const signedIn = this._signedIn();
const username = this._username();
@@ -614,6 +634,7 @@ class PuterUserMenu extends PuterWebComponent {
const email = typeof this.#user?.email === 'string' ? this.#user.email : '';
this.$('.menu-email').textContent = email;
this.$('.menu-email').hidden = !email;
this._renderAddCredits();
for ( const link of MENU_LINKS ) {
this.$(`.${link.className}`).href = new URL(link.path, globalThis.puter.defaultGUIOrigin).href;
}
@@ -705,13 +726,15 @@ class PuterUserMenu extends PuterWebComponent {
*/
_openMenu ({ fromKeyboard = false } = {}) {
const menu = this.$('.menu');
// An upgrade made on the page since the account loaded gives it a plan.
this._renderAddCredits();
this._showPanel(menu);
(fromKeyboard ? this.$('.menu-item') : menu).focus({ preventScroll: true });
}
/** Moves focus between menu items for an arrow, Home or End key; false for any other key. */
_moveMenuFocus (key) {
const items = [...this.$$('.menu .menu-item')];
const items = [...this.$$('.menu .menu-item:not([hidden])')];
const current = items.indexOf(this.shadowRoot.activeElement);
const last = items.length - 1;
const next = {
@@ -786,6 +809,13 @@ class PuterUserMenu extends PuterWebComponent {
);
}
_addCredits () {
// Focus goes back to the account button, where the dialog leaves it when it closes.
this._closePanels({ returnFocus: true });
// Called inside the click, so a tab it falls back to isn't blocked.
void globalThis.puter.ui.showTopUp_();
}
_sendFeedback () {
this._closePanels();
void globalThis.puter.ui.showFeedbackDialog();
@@ -26,6 +26,7 @@ const fakePuter = ({ token = null, user = null, picture = null } = {}) => {
},
ui: {
showFeedbackDialog: vi.fn(() => new Promise(() => {})),
showTopUp_: vi.fn(() => new Promise(() => {})),
},
auth: {
getProfilePicture: vi.fn(async () => currentPicture),
@@ -225,6 +226,80 @@ describe('<puter-user-menu>', () => {
expect(shown(el, '.menu')).toBe(false);
});
it('offers Add credits, under Usage, to an account with a plan', async () => {
const sdk = fakePuter({ token: 't', user: { ...USER, subscribed: true } });
vi.stubGlobal('puter', sdk);
const el = await mount();
part(el, '.avatar').click();
expect(shown(el, '.add-credits')).toBe(true);
expect(part(el, '.add-credits').textContent).toBe('Add credits');
expect(part(el, '.usage').nextElementSibling).toBe(part(el, '.add-credits'));
part(el, '.add-credits').click();
expect(sdk.ui.showTopUp_).toHaveBeenCalledTimes(1);
expect(shown(el, '.menu')).toBe(false);
});
it.each([
['an account without a plan', { ...USER, subscribed: false }],
['a server that reports no plans', USER],
['a seat whose team pays for it', { ...USER, subscribed: true, team: { uid: 'team-1', name: 'Acme' } }],
])('leaves Add credits out for %s, and out of the arrow keys', async (_label, user) => {
vi.stubGlobal('puter', fakePuter({ token: 't', user }));
const el = await mount();
part(el, '.avatar').click();
expect(shown(el, '.add-credits')).toBe(false);
part(el, '.usage').focus();
part(el, '.usage').dispatchEvent(new KeyboardEvent('keydown', { key: 'ArrowDown', bubbles: true }));
expect(el.shadowRoot.activeElement).toBe(part(el, '.open-puter'));
});
it('offers Add credits the next time the menu opens after an upgrade made on the page', async () => {
const account = { ...USER, subscribed: false };
vi.stubGlobal('puter', fakePuter({ token: 't', user: account }));
const el = await mount();
part(el, '.avatar').click();
expect(shown(el, '.add-credits')).toBe(false);
part(el, '.avatar').click();
// What the SDK does to its cached whoami when a framed upgrade reports a plan.
account.subscribed = true;
part(el, '.avatar').click();
expect(shown(el, '.add-credits')).toBe(true);
});
it('offers Add credits after an upgrade marks a whoami read again for the same token', async () => {
const sdk = fakePuter({ token: 't', user: { ...USER, subscribed: false } });
vi.stubGlobal('puter', sdk);
const el = await mount();
// Setting the token again reads whoami into a new object, which the upgrade then marks.
const reread = { ...USER, subscribed: false };
sdk.signInAs('t', reread);
sdk.whoami = reread;
await settle();
reread.subscribed = true;
part(el, '.avatar').click();
expect(shown(el, '.add-credits')).toBe(true);
// Never another account's plan.
part(el, '.avatar').click();
sdk.whoami = { ...OTHER, subscribed: true };
part(el, '.avatar').click();
expect(shown(el, '.add-credits')).toBe(false);
});
it('stops offering Add credits once the sign-in changes to an account without a plan', async () => {
const sdk = fakePuter({ token: 't', user: { ...USER, subscribed: true } });
vi.stubGlobal('puter', sdk);
const el = await mount();
expect(part(el, '.add-credits').hidden).toBe(false);
sdk.signInAs('other-token', OTHER);
await settle();
expect(part(el, '.add-credits').hidden).toBe(true);
});
it('moves focus through the menu with the arrow, Home and End keys', async () => {
vi.stubGlobal('puter', fakePuter({ token: 'app-token', user: USER }));
const el = await mount();