Compare commits

...
Author SHA1 Message Date
ProgrammerIn-wonderland 39dcb26f3f secondary custom domain verification 2026-10-26 16:35:44 -04:00
ProgrammerIn-wonderland df9ce2c76f routing fixes 2026-10-26 16:35:44 -04:00
ProgrammerIn-wonderland f61b3ea546 modifications' 2026-10-26 16:35:44 -04:00
ProgrammerIn-wonderland 87b295f564 PUT-2226 2026-10-26 16:35:44 -04:00
Neal Shah 21625de22e Merge pull request #4315 from HeyPuter/codex/kimi-k3-1-pro
Codex/kimi k3 1 pro
2026-10-26 21:01:04 +01:00
Daniel Salazar f850155411 KV share handles: hardening, higher limits, batch share, uninstall data deletion, account-deletion cleanup (#4344)
* fix(kv): sign share handle ids and drop retired handles from every node's cache

New handle ids carry a short HMAC (kvh-<uuid>.<mac>); an id whose mac
does not verify is refused before any lookup. Unsigned ids keep working
but no longer fall back to the primary on a replica miss.

Retiring or changing a handle now broadcasts outer.pubsub.kv.share.forget
so every node drops its cached row. Event subscribes and delivery
rechecks refuse a handle whose owner is suspended, as the driver does.

* fix(kv): an app its user removed stops reaching shares

An app acting for a user exercises a share grant only while the user
still has the app authorized: at least one user-to-app permission row,
which every app token path leaves and revoke-user-app '*' removes. The
share itself stays; authorizing the app again restores access. The
answer rides the permission cache, keyed by the app-under-user
generation the revoke bumps, so live subscriptions go quiet too.

* feat(kv): list keys through a watch-level share handle

list with as: 'keys' through a handle needs only watch, which already
learns every key written in the region. Values and entries still need
read.

* feat(kv): filter share listings by app and by state

GET /kv/shares and GET /kv/shares/with-me take an optional appUid: an
app uid, or null for the app-less store. An account session may name
any app; an app is refused any namespace but its own. GET /kv/shares
also takes state=live|revoked|all, all by default.

* feat(puter.js): appUid and state options on the KV share listings

listShared() and listSharedByMe() pass appUid (null for the app-less
store), and listSharedByMe() passes state. Docs cover the new options
and that a watch share can list keys without their values.

* feat(kv): indexes and store counts for share-handle quotas and retention

Adds covering indexes for the owner's live count, the owner-holder pair,
app-filtered listings and retired rows, plus store methods to count a
pair's live handles and delete rows retired before a cutoff. The live
handle cache holds up to 50,000 entries; the miss cache stays at 10,000.

* feat(kv): higher share-handle caps, a per-holder cap, and sharing with several recipients

Live share handles per account go to 25,000 paid / 5,000 free, and per
app to 10,000 / 2,000. One owner may hold out 256 live handles to one
holder on every plan, which bounds the rows a revoke or unshare walks.
Re-sharing an existing region takes no slot and is never refused.

POST /kv/shares takes recipients (1 to 50) as an alternative to
recipient: one region, level and label for all. Caps and a new
per-recipient grant budget (300/min, 2,000/hour) are checked for the
whole batch up front; failures about one recipient are reported on its
item. The single-recipient request and response are unchanged and run
through the same code path.

* feat(kv): delete share handles 30 days after they stop

A daily job deletes retired share-handle rows older than 30 days, in
batches of 500. One node runs it per day; it stops on shutdown.

* feat(puter.js): share a KV prefix with several users in one call

puter.kv.share() takes an array of recipients, positionally or as
recipients in the options form, and resolves to one item per recipient:
the share, or an error with its message and code. The single-recipient
forms are unchanged.

* feat(kv): delete an app's data on uninstall, and close account-deletion gaps

Uninstall:
- POST /auth/revoke-user-app takes an optional `deleteData` boolean, only
  with `permission: '*'` (400 otherwise). AuthService.uninstallApp revokes
  every grant, deletes the app's worker sessions, then awaits
  `app.uninstalled` { userId, userUuid, appUid, deleteData }.
- With deleteData, KvShareService retires the handles the user shares out
  of that app's namespace (as the platform, in batches) and then empties
  the namespace in the background, 5,000 items per step, stopping on
  shutdown. Handles shared with the user are not touched.
- SystemKVStore.purgeNamespace(namespace, { maxItems }) deletes up to
  maxItems items of one `v1:<uuid>:<appUid>` namespace, reading keys only,
  with no mutation events or usage, and reports `done` once it is empty.
  It refuses the system namespace and anything else. flush shares its
  page-delete path.
- puter.perms.revokeApp(appUid, '*', { deleteData: true }); the desktop's
  uninstall dialog has an unchecked "Also delete this app's data" option.

Account deletion:
- EventsService settles each of a deleted account's apps on `user.delete`
  (they cascade without `app.changed`) and drops the owner's subscription
  keys.
- The flat permission entries the account held are purged in pages.

Suspension:
- `user.suspended` / `user.unsuspended` events; the share-handle store
  drops cached rows owned by that user here and on other nodes.

* fix(permission): await a peer's flat-permission deletes so emitAndWait sees them applied

* fix(kv): strongly consistent purge pages, and an uninstall purge stops once the app is authorized again

A stale empty page could end a purge as done while items remained. The
background purge after an uninstall now checks before each step that the
user hasn't authorized the app again, so data written after that is kept.

* fix(kv): flush keeps eventually consistent page reads; only a purge reads strongly consistent

* fix(user): emit user.suspended and user.unsuspended when the user store changes an account's suspension
2026-10-26 09:43:55 -07:00
Reynaldi Chernando 7c0cf6f6a1 docs(fs): drop batch-write fallback notes from puter.fs.upload() (#4332) 2026-10-26 17:25:23 +07:00
Nariman Jelveh 7f93166560 fix(hosting): disconnect the directory when update gets a null root_dir (#4345)
puter.hosting.update(name, null) sends root_dir: null, which the driver
stringified into a lookup for a folder named "null" and rejected, so
disassociating a website from a folder's properties always failed.
2026-10-25 23:28:45 -07:00
Nariman Jelveh 21846192d1 feat(dashboard): publish websites and workers from Files in dashboard dialogs (#4342)
* feat(dashboard): publish websites and workers from Files in dashboard dialogs

The Files tab's Publish as website and Publish as Worker actions open
dashboard form dialogs instead of desktop windows: the address field shows
the domain inline and shapes the typed name into a valid one, workers get a
Sandboxed switch, and success shows the live address with copy and open.
Published rows get their badges right away.

The form dialog gains suffix and switch fields and a shared copy row, which
the API token dialog now uses too. The save-account / confirm-email gate
before publishing moves to one helper shared by both flows.

* fix(gui): ask a just-saved account to confirm its email before publishing

Saving a temporary account can leave its email unconfirmed when signup
doesn't require confirmation. The publish gate returned right after the
save, skipping the confirm-email step the old inline checks still ran.

* fix(dashboard): link a published site to the address the server reports

The success screen built https://<name>.<hosting domain>, which is wrong on
a self-hosted instance serving sites over http. Use the address the create
response carries, the one the rest of the GUI shows, and fall back to the
built one only when it's missing.

* fix(dashboard): dim a suffixed field as one box while the form submits

Only the input inside it dimmed, leaving the domain after the name at full
contrast while the field was locked.

* fix(dashboard): point a taken or reserved publish name at the address field

The server's reason showed in the status line, but the field stayed
unmarked and focus stayed on Publish, so changing the name took an extra
click. Mark the field invalid and focus it for the rejections that mean
"pick another name".

* fix(dashboard): type into a suffixed field when its suffix is pressed

The domain after the name is drawn inside the field's box but was a plain
span, so pressing it focused nothing. A press there now focuses the field
with the caret at the end, and the suffix shows a text cursor.

* chore(dashboard): drop a phone width rule the icon buttons never used

The form dialog's phone flex rule sizes these buttons, so the width never
applied; touch screens get 44px from the coarse-pointer min-width.

* fix(dashboard): read an underscore in a website address as a word break

A website name can't hold an underscore, so typing My_Cool Site gave
mycool-site. Underscores now become a hyphen, as spaces do; a worker
name keeps them.

* fix(dashboard): keep the letter when an accented one is typed into a publish address

Café became caf: an accented letter isn't a DNS character, so it was
dropped whole. The accent comes off and the letter stays.

* fix(dashboard): cap a website address at a hostname label's 63 characters

The server takes a 64-character subdomain, but DNS caps a label at 63,
so the site it publishes can't be reached. The field now stops at 63,
as the worker one does.

* fix(dashboard): let a publish address finish composing before shaping it

The address was reshaped on every input, composing included: a dead
key's accent turned into a hyphen before its letter arrived, and IME
text was rewritten under the composition. Composed text is now shaped
once it's in.
2026-10-25 21:34:39 -07:00
Nariman JelvehandClaude Opus 5.5 b9f0c8601d fix(dashboard): eighth adversarial review of Files — bugs, UX, a11y, perf (#4330)
* fix(dashboard): don't preview a file named after an image type, or a failed image

Space on a file named "png" (no extension) opened the image preview over a
broken image: the type was read from the last dot-separated part of the
name, which for an extensionless name is the whole name. It's now read from
the extension.

A preview whose image URL couldn't be fetched left the previous image up
under the newly selected row and raised an unhandled rejection. It now
closes instead.

* fix(dashboard): sort Files names by number, and break size/date ties by name

Names were compared as plain text, so "file10" sorted before "file2" and
"dir10" before "dir2". They now compare as a file manager does, with runs
of digits by value.

Sorting by size or date left rows with the same value (every folder, when
sorting by size) in whatever order the server listed them, and a row added
afterwards (a new folder, a rename) landed after its equals instead of where
a fresh listing would put it. Ties now fall back to the name, in the full
sort and in the single-row insert alike.

The sort also parsed each trashed item's metadata on every comparison; it
now reads each entry's sort key once.

* fix(dashboard): ask about "this item" when permanently deleting one Trash item

The Delete key and the phone selection bar's Delete Permanently asked
"Are you sure you want to permanently delete these items?" for a single
item too. One item now gets the single-item question, as from its own
menu. The three copies of the confirm-and-delete step (Delete key,
selection bar, multi-select menu) are now one.

* fix(dashboard): don't let saved column widths squeeze out an auto-width name

Saved widths too wide for the listing fall back to the defaults, but only
when the Name column had a saved width too. Beside an auto-width name, a
Size or Modified column widened on a large screen kept its width on a
smaller one and squeezed the names to nothing, with the row scrolling
sideways. The check now counts an auto-width name at its 100px minimum.

* fix(dashboard): give Files names a phone's full row, not a desktop column width

A narrow listing drops the columns for two-row rows, where each name has
the row's width. Names were still cut to the Name column's saved width
whenever the saved columns happened to fit, so a 120px column set on a
desktop cut names to about 100px on a phone with 280px to spare.

* fix(fs): refuse to replace a folder that contains the item being moved or copied

Moving or copying an item onto a name taken by one of its own parent
folders, with overwrite, removed that folder recursively first, and the
item went with it. Cutting the inner "photos" out of photos/photos, pasting
it next to the outer one and choosing Replace deleted both folders and
everything in them. Such an overwrite is now refused before anything is
removed.

* fix(dashboard): keep the Files selection when an upload lands in another folder

An upload into a folder other than the one on screen (a drop on a folder
row, a sidebar folder or a crumb, or Upload from a folder's menu) still
listed the folder on screen again when it finished. That cleared whatever
was selected there and left an open menu acting on rows no longer on the
page. Only an upload into the folder on screen refreshes it now.

The Upload picker ran its own copy of the upload flow; it now goes through
the same one as drops.

* fix(dashboard): keep Files rows up while a refresh reads web link and app icons

A refresh of the folder on screen keeps its rows until the new listing is
in, so it doesn't flash blank. But it cleared them before reading the
icons, and a web link's or .app's icon is read from the file over the
network, so in such a folder every re-sort, upload or return to the tab
blanked the pane for those reads. The icons are now read before the rows
are swapped.

* fix(dashboard): Ctrl/Cmd-drag a Files selection as a whole

Pressing Ctrl (Cmd on a Mac) on a row that was already selected took it out
of the selection at once, so a drag starting there collapsed the selection
to that one row: a Ctrl-drag of three files to copy them copied one, under
a ghost that still read "3". The row now leaves the selection when the
button is released without a drag, as in a desktop file manager, and a
drag carries the whole selection.

* fix(dashboard): let Enter open the row the arrow keys moved to in Files

Clicking a sidebar folder, crumb or toolbar button leaves it focused, and
the arrow keys and type-to-select still move the row selection from there.
Enter, Space and the menu key then went to that focused control: after
clicking Documents and pressing Down twice, Enter opened Documents again
and dropped the selection instead of opening the highlighted row. The
listing now takes the focus when these keys move the selection, so the
keys after them act on the rows, and Shift+Tab goes back to the toolbar.

* fix(dashboard): let Files' image preview be cancelled while it loads, and close it with the tab

The preview only counted as open once its image URL had come back (two
round-trips), so nothing could cancel it before then: Escape cleared the
selection and the image popped up anyway, a second Space opened it instead
of closing it, and opening a folder left the old image to appear over the
new one. It now counts as open from the start, so each of those cancels
it.

Browser Back/Forward to another dashboard tab left the preview floating
over that tab, where Escape and Space no longer reach it. A route change
away from Files now closes it, as it does Share and Properties.

* fix(dashboard): keep Files keys off a new folder until the server has made it

A new folder's row goes up with the name editor open before mkdir answers.
Clicks and menus already wait on it, but the keyboard didn't: pressing
Escape and then Delete on a slow connection sent a move for the row's
placeholder and showed "Moving New Folder: Subject does not exist", and
Enter opened the predicted path before it existed. Enter, Delete, Ctrl/Cmd+C
and Ctrl/Cmd+X now skip such a row, as do Cut and Delete in the
multi-select menu and the phone's selection bar.

* fix(dashboard): explain a Files action that fails on the network

A request that fails at the network level is rejected with no message.
New Folder then took its row back with no alert at all, and Restore and
the New menu's files showed "[object XMLHttpRequest]". A failed move from
the clipboard named the item and nothing else. Each now falls back to
"An unknown error occurred."

* fix(dashboard): don't offer Trash folders, or view-only shares' crumbs, as Files drop targets

Inside the Trash a folder row lit up as a drop target and then refused the
drop with an untranslated "Cannot move items into a deleted folder.", and a
crumb inside the Trash took the move, though only Restore may take an item
out of the Trash. A crumb of a folder shared with the user to view only
lit up too, and the drop ended in one refusal per item from the server.

Such folders no longer light up, and a drop on one goes back, as a drop on
a view-only folder row already did. A view-only crumb still opens on a
dwell.

* fix(dashboard): count a new folder, not the old selection, in the Files footer

New Folder selects the new row, but the footer was updated before that and
kept describing what was selected before ("3 items selected"), as did the
phone's selection bar, whose buttons by then acted on the new folder. A
Shift+click also still extended from the old anchor. The footer now
follows the new selection, and the new row is the anchor.

* fix(dashboard): update the Files footer when Enter ends a rename

Ending a rename with Enter selects the row, and then called the desktop
Explorer's footer update, which finds no footer in the dashboard. Renaming
a row from its menu while another was selected left the footer at
"1 item selected" with two rows selected.

* fix(dashboard): stop Files' image preview from running out of read URLs

The preview fetched a read URL for every image it showed, and read URLs
share the access-token allowance of 20 an hour. Stepping through a folder
of images with the arrow keys spent it in seconds, after which the preview
(and any app asking for a read URL) failed for the rest of the hour. The
preview now reads the image directly and shows it from memory, freed when
it closes or moves on, and a held arrow key loads only the image it stops
on.

* fix(dashboard): don't offer to publish a folder the user may not share from Files' folder menus

The background, crumb and sidebar menus offered Publish as Website for any
folder, so in a folder shared with the user to view or edit it opened the
publish dialog only for the server to refuse. Publishing takes the right
sharing does; the item now stays disabled without it, as on a folder row's
own menu.

* fix(dashboard): open a Files folder under its new name when it's opened mid-rename

Double-clicking a folder right after typing its new name (the first click
sends the rename) opened it under the old path, which the rename then
took away: the view sat in a folder that no longer existed, with crumbs
and Back/Forward disagreeing about its name, and New Folder or Upload
there failed. Opening a row (double-click, tap, Enter, or Open from its
menu) now waits for a rename still on its way, and goes by one path.

* fix(dashboard): keep Files' Back/Forward in step with folders moved or trashed

Back and Forward kept a folder's old path after it was moved or deleted,
here or from elsewhere: going into a folder, back out, and dragging it
somewhere else (or deleting it) left Forward leading to "This folder
couldn't be opened", with New Folder and Upload gone. History entries now
follow a moved folder and what is inside it, and a trashed or deleted
one's entries are dropped.

* fix(dashboard): keep screen readers from reading Files' icon file names

The crumbs' separators, the folder sidebar's icons and every row's icon
were images with no alt text, so a screen reader announced each one, by
file name in some ("triangle-right.svg"), between every crumb and before
every name. They are decorative beside the text they sit next to and are
now marked so. The crumb of the folder on screen is marked as the current
page.

* fix(dashboard): make double-click on a Files column divider fit the column

Double-clicking a column divider measured the text in system-ui, but the
listing renders in Inter, and took the cells' padding as 8px a side where
Size and Modified have 10. Right after "fit to content", the longest name
was still cut in the middle and every Modified cell ended in an ellipsis
("40 minutes a…"). Each column is now measured in its cells' own font and
padding.

* fix(dashboard): keep a renamed item among its sharer's in Files → Shared

With Shared sorted by "Shared by", renaming an item shared with the user
to edit placed its row by name alone, so it could land among another
person's items until the listing was loaded again. A row put back in
place now goes by its sharer first there, as the listing does.

* fix(dashboard): keep every item, and empty folders' places, in a downloaded zip

Downloading several items as one zip keyed each by its name, so two items
with the same name (in Files → Shared, two people can each share a
"Notes.txt") came out as one file, the other silently gone. Such names
now get " (1)" and so on in the zip, as a folder does for a duplicate.

Empty folders were put at the top of the zip by their own name, not where
they sat: zipping a folder holding x/y gave "y/" next to "x/". They now
keep their place.

dedupedName moves to helpers/, now that the zip uses it as well.

* fix(dashboard): give Files grid tiles both lines of their name

A tile has room for two lines of name, but names were cut to fit one:
"Quarterly financial report final version.pdf" showed as
"Quarterl…l version.pdf" over an empty second line. A name now shows whole
when it wraps into the two lines, and is cut in the middle, keeping its
extension, only as far as it has to be to fit them.

Where the browser wraps a name can't be told from text widths alone, so
the tiles are written and read back together, one layout a round rather
than one a tile.

* fix(dashboard): keep the Files listing when a refresh can't reach the server

A refresh of the folder on screen that failed for any reason replaced the
listing with "This folder couldn't be opened." and took New Folder and
Upload away. Returning to the tab triggers one, so opening a laptop before
its Wi-Fi reconnected, or a moment's lost connection during a re-sort, left
a folder that was fine looking broken until it was opened again. A refresh
that doesn't reach the server now leaves the rows as they are; one the
server answers with an error (the folder is gone) still says so.

* fix(dashboard): don't drop files added to a Files folder while a new folder is made

While New Folder (or New › file) waited for the server, every item added
to the folder on screen was dropped, not only the one being made, so a
file uploaded from another tab or device in that moment didn't show up
until the folder was listed again. Such items are now held until the
create is done and added then; the create's own comes through as an
update of the row already drawn for it.

* perf(dashboard): refresh Files on return to the browser tab only when Files is showing

Coming back to the browser tab listed the Files folder again even with
another dashboard tab in front, and in Files → Shared that meant every
share all over again. The refresh now waits until Files is next shown.

* fix(dashboard): move the keyboard focus off a Files arrow that disables itself

Pressing Enter on Back to reach the first folder disables Back while it has
the focus, and a disabled arrow is drawn at 10% opacity, focus ring
included, so the keyboard user lost track of where they were. The focus
now moves on to an arrow that still works, or to the listing.

* fix(dashboard): show "can't drop here" when dragging files from the computer where Files takes none

Files dragged in from the computer showed the copy pointer everywhere in
Files, then dropping on the Trash, in Shared, on a folder shared to view
only, or while a folder was loading did nothing. The pointer now shows the
drop isn't possible there.

* fix(dashboard): let Files' type-to-select find names with "i" in a Turkish browser

Typed letters were lower-cased for the browser's language and the names
they're matched against without one. In Turkish, "I" lower-cases to "ı",
so typing it never reached "Invoices". Both are now lower-cased alike.

* fix(dashboard): bring Files' Back/Forward along when a folder rename is undone

Renaming a folder points Back and Forward at its new name, but undoing the
rename left them there: Forward then led to "This folder couldn't be
opened" for a name the folder no longer had. Undo now takes them back to
the old name too.

* fix(dashboard): don't shrink Files → Shared's name column when it's dragged wider

In a narrow Shared listing, "Shared by" is laid out narrower beside a name
column with no set width. Dragging the name's divider gave it a width, at
which "Shared by" sprang back to its own and pushed the name narrower than
it started: dragging right made it 30px smaller, and there it stuck.
Sizing the name now keeps "Shared by" at the width it had on screen.

* docs(fs): note that move and copy never overwrite a folder holding the source

* fix(dashboard): keep the Files drop zone after files dragged in cross a folder row

Leaving a folder row while dragging files in from the computer also took
the highlight off the listing around it, as the row's notice bubbled up to
the listing's own. With the "can't drop here" pointer going by that
highlight, a drop on the listing after crossing a folder row was refused.
The row's notices stay with the row, and the listing goes by whether the
folder takes uploads.

* fix(dashboard): don't let a Files folder wait for ever on a rename to open

Opening a row waits for a rename on its way, which settles in the last
step of the rename's handling. A failure after the rename itself (such as
refreshing the folder's websites) never got there, and the folder then
wouldn't open until it was listed again. Opening waits five seconds at
most.

* docs(fs): drop the note on overwriting a folder that holds the source

The refusal is an edge case, and its error message already explains it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-25 18:59:24 -07:00
404oopsandClaude Fable 5.1 68249028ef test(ai): assert replayed Kimi tool arguments reach the wire decoded once
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:59:04 +02:00
404oops cbcbdc2599 Merge branch 'fix/replayed-tool-call-arguments' into codex/kimi-k3-1-pro 2026-10-24 22:57:54 +02:00
404oopsandClaude Fable 5.1 3296c1b0d6 fix(ai): bill Kimi K3.1 Pro from reported usage at the published rates
Cache writes now meter under `cache_write_tokens` at the model's write
rate, split out of `prompt_tokens` by the shared usage splitter, so K3 gets
the same breakdown. The credit hold reserves the prompt and any replayed
reasoning at the input rate, and a stream that ends without a usage chunk
is charged the estimate like every other provider. The per-TTL response
headers, the 502 accounting error and the one-hour write prices are gone:
the cache is scoped to the account and nothing here requests a one-hour
TTL, so those writes never occur and Moonshot publishes no rate for them.

The catalog entries carry the release date so responses normalize by
default, the provider suite no longer imports the SDK, and the docs cover
the reasoning efforts, replay and cache usage keys in their existing
sections.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:55:10 +02:00
404oopsandClaude Fable 5.1 a10a9d9b93 fix(ai): keep replayed tool-call arguments serialized once
An OpenAI-shaped `tool_calls` entry normalizes with its `arguments` still a
JSON string, and hoisting it back into `tool_calls` stringified that string
again, so every OpenAI-compatible provider received a JSON string literal
instead of the call's arguments. Hoisting now sends a string `input` as is
and only serializes object inputs.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:54:39 +02:00
Filip Kujundžić ce5f28e844 Validate Kimi completion accounting and preserve reasoning request semantics 2026-10-24 16:02:17 +02:00
Filip Kujundžić 97beffb572 Make completed Kimi accounting failures terminal without estimated billing 2026-10-23 21:05:39 +02:00
Filip Kujundžić 03eab894a0 fix(ai): reserve Kimi cache writes and preserve tool arguments 2026-10-23 20:56:42 +02:00
Filip Kujundžić 53c77c8333 feat(ai): support Kimi K3.1 Pro models 2026-10-23 20:43:35 +02:00
124 changed files with 8329 additions and 971 deletions

No files matched your search

@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 91;
const CURRENT_SCHEMA_VERSION = 92;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -130,6 +130,7 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[88, ['0093_fsentries-parent-uid-index.sql']],
[89, ['0094_deleted-apps.sql']],
[90, ['0095_kv-share-handle-levels.sql']],
[91, ['0096_kv-share-handle-quota-indexes.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,72 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share-handle quota, listing and purge indexes. See
-- sqlite/0096_kv-share-handle-quota-indexes.sql for what each serves. No
-- per-file applied-state tracking, so every index is guarded and a replay does
-- nothing.
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_quota_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_kv_share_handle_quota_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_owner_live'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_owner_live` (`owner_user_id`, `revoked_at`, `app_uid`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_owner_app'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_owner_app` (`owner_user_id`, `app_uid`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_grantee_app'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_grantee_app` (`grantee_user_id`, `app_uid`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_pair'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_pair` (`owner_user_id`, `grantee_user_id`, `revoked_at`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_revoked'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_revoked` (`revoked_at`);
END IF;
END//
DELIMITER ;
CALL _puter_add_kv_share_handle_quota_indexes();
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_quota_indexes;
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share-handle quota, listing and purge indexes. See
-- sqlite/0096_kv-share-handle-quota-indexes.sql for what each serves.
-- Idempotent via IF NOT EXISTS; there is no per-file applied-state tracking.
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_owner_live
ON kv_share_handles (owner_user_id, revoked_at, app_uid);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_owner_app
ON kv_share_handles (owner_user_id, app_uid, id);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_grantee_app
ON kv_share_handles (grantee_user_id, app_uid, id);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_pair
ON kv_share_handles (owner_user_id, grantee_user_id, revoked_at);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_revoked
ON kv_share_handles (revoked_at);
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Indexes behind the share-handle quotas, the app-filtered listings and the
-- retention purge on `kv_share_handles`.
--
-- - owner_live : the owner's live count, in total and per app, read off
-- the index alone.
-- - owner_app : the owner's listing narrowed to one app, in id order.
-- - grantee_app : the holder's listing narrowed to one app, in id order.
-- - pair : live handles one owner holds out to one holder, counted at
-- mint and walked on revoke and unshare.
-- - revoked : retired rows past retention, for the purge.
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_owner_live`
ON `kv_share_handles` (`owner_user_id`, `revoked_at`, `app_uid`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_owner_app`
ON `kv_share_handles` (`owner_user_id`, `app_uid`, `id`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_grantee_app`
ON `kv_share_handles` (`grantee_user_id`, `app_uid`, `id`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_pair`
ON `kv_share_handles` (`owner_user_id`, `grantee_user_id`, `revoked_at`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_revoked`
ON `kv_share_handles` (`revoked_at`);
@@ -534,6 +534,8 @@ export class DDBClient extends PuterClient {
scanIndexForward?: boolean;
beginsWith?: { key: string; value: string };
select?: 'COUNT';
/** Attributes to return instead of the whole item. */
projection?: string[];
filter?: {
expression: string;
values?: Record<string, unknown>;
@@ -574,6 +576,10 @@ export class DDBClient extends PuterClient {
Object.assign(expressionAttributeValues, options.filter.values);
Object.assign(expressionAttributeNames, options.filter.names);
}
const projection = (options?.projection ?? []).map((name) => {
expressionAttributeNames[`#p_${name}`] = name;
return `#p_${name}`;
});
const command = new QueryCommand({
TableName: table,
@@ -588,6 +594,9 @@ export class DDBClient extends PuterClient {
? { FilterExpression: options.filter.expression }
: {}),
...(options?.select ? { Select: options.select } : {}),
...(projection.length > 0
? { ProjectionExpression: projection.join(', ') }
: {}),
...(options?.scanIndexForward !== undefined
? { ScanIndexForward: options.scanIndexForward }
: {}),
+25 -5
View File
@@ -137,6 +137,17 @@ export type EventMap = {
};
'app.new-icon': { app_uid: string; data_url: string };
'app.opened': { app_uid: string; user_id: number; ts: number };
/**
* A user removed an app: its grants and worker sessions are gone. With
* `deleteData`, the user asked for the app's data in their account to go
* too. Awaited by the emitter.
*/
'app.uninstalled': {
userId: number;
userUuid: string;
appUid: string;
deleteData: boolean;
};
'app.rename': {
app_uid: string;
old_name: string;
@@ -389,6 +400,9 @@ export type EventMap = {
new_username: string;
};
'user.email-changed': { user_id: number; new_email: string };
/** An account was suspended, or its suspension lifted. Local events. */
'user.suspended': { userId: number };
'user.unsuspended': { userId: number };
// Fired after an account is torn down (self-serve, admin, or temp-user
// logout cleanup). Listeners purge external state tied to the account —
// e.g. the marketplace extension cancels the user's Stripe subscriptions.
@@ -665,6 +679,13 @@ export type EventMap = {
* reason as the events generation bump: a per-process cache.
*/
'outer.pubsub.share.ownerShared': { ownerUserId: number };
/**
* Share handles were retired or changed, so every process must drop its
* cached copy. `outer.pubsub.*` because the cache is a per-process map.
*/
'outer.pubsub.kv.share.forget': { handles: string[] };
/** The same, for every cached handle one owner holds out. */
'outer.pubsub.kv.share.forgetOwner': { ownerUserId: number };
'outer.fs.write-hash': { hash: string; uuid: string };
/**
* Cache keys the KV read cache must stop serving, because the entries
@@ -949,11 +970,10 @@ export type EventKey = keyof EventMap & string;
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
? | `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
@@ -7716,6 +7716,12 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
grant,
),
);
// What launching the app records for the holder.
await server.services.permission.grantUserAppPermission(
holderActor,
app.uid,
'flag:app-is-authenticated',
);
const appActor = makeActor({
user: holderActor.user,
app: { id: app.id, uid: app.uid },
@@ -9658,6 +9664,70 @@ describe('AuthController grant/revoke additional branches', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('revoke-user-app: 400 on `deleteData` without "*" or as a non-boolean', async () => {
const { actor } = await makeUserAndActor();
const app = await server.stores.app.create(
{
name: `tr-delete-data-${uuidv4()}`,
title: 'TestDeleteDataApp',
index_url: `https://tr-delete-data-${uuidv4()}.example.test/`,
},
{ ownerUserId: actor.user.id },
);
for (const body of [
{ permission: 'events:background', deleteData: true },
{ permission: '*', deleteData: 'yes' },
]) {
await expect(
controller.handleRevokeUserApp(
makeReq({ app_uid: app.uid, ...body }, { actor }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
}
});
it('revoke-user-app `*`: announces the uninstall, with `deleteData` as sent', async () => {
const { actor } = await makeUserAndActor();
const app = await server.stores.app.create(
{
name: `tr-uninstall-${uuidv4()}`,
title: 'TestUninstallApp',
index_url: `https://tr-uninstall-${uuidv4()}.example.test/`,
},
{ ownerUserId: actor.user.id },
);
const seen: unknown[] = [];
const listener = (_key: string, data: unknown) => {
seen.push(data);
};
server.clients.event.on('app.uninstalled', listener);
try {
for (const deleteData of [undefined, true]) {
await inCtx(actor, () =>
controller.handleRevokeUserApp(
makeReq(
{ app_uid: app.uid, permission: '*', deleteData },
{ actor },
),
makeRes(),
),
);
}
} finally {
server.clients.event.off('app.uninstalled', listener);
}
const expected = {
userId: actor.user.id,
userUuid: actor.user.uuid,
appUid: app.uid,
};
expect(seen).toEqual([
{ ...expected, deleteData: false },
{ ...expected, deleteData: true },
]);
});
it('revoke-user-app: 400 when permission is "*" but app_uid is missing', async () => {
const { actor } = await makeUserAndActor();
await expect(
+18 -12
View File
@@ -3327,7 +3327,8 @@ export class AuthController extends PuterController {
})
async handleRevokeUserApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body ?? {};
const { origin, permission, permissions, meta } = req.body ?? {};
const { origin, permission, permissions, meta, deleteData } =
req.body ?? {};
this.#validateAppPermissionParams({
app_uid,
origin,
@@ -3343,20 +3344,25 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
}
if (deleteData !== undefined && typeof deleteData !== 'boolean') {
throw new HttpError(400, '`deleteData` must be a boolean', {
legacyCode: 'bad_request',
});
}
if (deleteData && permission !== '*') {
throw new HttpError(
400,
'`deleteData` only applies when `permission` is "*"',
{ legacyCode: 'bad_request' },
);
}
// Deliberately not gated by the target's sharing flag: a user must
// always be able to withdraw a grant, whatever the target now says.
if (permission === '*') {
await this.services.permission.revokeUserAppAll(
req.actor!,
app_uid,
meta ?? undefined,
);
// Withdrawing everything also retires the workers the app deployed
// under this user; their tokens would otherwise act as the app.
await this.services.auth.deleteWorkerSessionsForApp(
req.actor!.user!.id as number,
app_uid,
);
await this.services.auth.uninstallApp(req.actor!, app_uid, {
deleteData: deleteData === true,
meta: meta ?? undefined,
});
} else {
for (const entry of list) {
await this.services.permission.revokeUserAppPermission(
@@ -139,14 +139,14 @@ describe('the delivery budgets', () => {
});
it('pins the share-handle quotas by plan', () => {
expect(EVENTS_KV_HANDLES_PER_USER.limit).toBe(512);
expect(EVENTS_KV_HANDLES_PER_APP.limit).toBe(512);
expect(EVENTS_KV_HANDLES_PER_USER.limit).toBe(25_000);
expect(EVENTS_KV_HANDLES_PER_APP.limit).toBe(10_000);
expect(
limitFor(EVENTS_KV_HANDLES_PER_USER, DEFAULT_FREE_SUBSCRIPTION),
).toBe(200);
).toBe(5_000);
expect(
limitFor(EVENTS_KV_HANDLES_PER_APP, DEFAULT_FREE_SUBSCRIPTION),
).toBe(128);
).toBe(2_000);
});
it('keeps `single` well under broadcast — each one costs far more', () => {
+27 -10
View File
@@ -176,20 +176,37 @@ export const KV_SHARE_NOTICE_LIMIT = userWindow(
/**
* Live share handles one account may hold out at a time, across every app.
*
* Each is a standing grant on part of the account's data, and revoking marks
* rather than deletes, so without a ceiling the rate limit alone lets the rows
* grow forever. A temporary account holds none: the grant outlives the session
* that made it, and no account is left to take it back.
* A backstop above the per-app cap: every count reads a covering index, so the
* bound is on standing grants, not query cost. A temporary account holds none:
* the grant outlives the session that made it, and no account is left to take
* it back.
*/
export const EVENTS_KV_HANDLES_PER_USER = tiered(512, 200, 0);
export const EVENTS_KV_HANDLES_PER_USER = tiered(25_000, 5_000, 0);
/**
* Live share handles one app may hold out for one account.
*
* Separately bounded from the account cap so one namespace cannot consume the
* whole budget on free accounts.
* Live share handles one app may hold out for one account, so one namespace
* cannot consume the whole account budget.
*/
export const EVENTS_KV_HANDLES_PER_APP = tiered(512, 128, 0);
export const EVENTS_KV_HANDLES_PER_APP = tiered(10_000, 2_000, 0);
/**
* Live share handles one owner may hold out to one holder, on every plan.
* Bounds the rows a revoke or unshare walks to find the narrower handles inside
* one.
*/
export const EVENTS_KV_HANDLES_PER_HOLDER = 256;
/**
* Recipients one account may share with, per minute and per hour. A batch
* spends one per recipient, on top of the per-call mint budget.
*/
export const KV_SHARE_GRANT_LIMITS = [
userWindow('kvShare:grants:minute', 300),
userWindow('kvShare:grants:hour', 2_000, 60 * 60_000),
] as const;
/** Recipients one `POST /kv/shares` call may name. */
export const KV_SHARE_BATCH_MAX = 50;
// -- Handler surface -------------------------------------------------
@@ -34,7 +34,15 @@ import {
type TestUserCredentials,
} from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import { EVENTS_KV_HANDLE_LIMIT } from '../events/limits.js';
import { claimRateLimit } from '../../core/http/middleware/rateLimit.js';
import { seedKvShareHandles } from '../../services/kvShare/kvShareTestUtil.js';
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
import {
EVENTS_KV_HANDLE_LIMIT,
EVENTS_KV_HANDLES_PER_USER,
KV_SHARE_BATCH_MAX,
KV_SHARE_GRANT_LIMITS,
} from '../events/limits.js';
const BOOT_TIMEOUT_MS = 120_000;
@@ -57,9 +65,11 @@ const call = (
const resetBudgets = async (env: PuterTestEnv) => {
for (const name of [env.users.user.username, env.users.other.username]) {
const user = await env.server.stores.user.getByUsername(name);
await env.server.clients.redis.del(
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user!.id}`,
);
for (const scope of [
EVENTS_KV_HANDLE_LIMIT.scope,
...KV_SHARE_GRANT_LIMITS.map((window) => window.scope),
])
await env.server.clients.redis.del(`rate:${scope}:${user!.id}`);
}
};
@@ -293,6 +303,325 @@ describe('/kv/shares', () => {
const res = await req('GET', '/kv/shares');
expect(res.status).toBeGreaterThanOrEqual(400);
});
describe('filtering a listing', () => {
type Page = { items: Array<Record<string, unknown>>; total?: number };
let appUid: string;
let inApp: Record<string, unknown>;
let global: Record<string, unknown>;
const page = async (path: string, token: string) => {
const res = await req('GET', path, token);
return { status: res.status, body: (await res.json()) as Page };
};
const handlesOf = (body: Page) => body.items.map((row) => row.handle);
const makeApp = async () => {
const owner = await env.server.stores.user.getByUsername(
env.users.user.username,
);
const name = `kv-http-${uuidv4().slice(0, 8)}`;
return env.server.stores.app.create(
{
name,
title: 'Listing App',
index_url: `https://${name}.example.test/index.html`,
},
{ ownerUserId: owner!.id },
);
};
/** The token the desktop launches the app with, for `token`'s user. */
const appToken = async (token: string, uid: string) => {
const res = await req('POST', '/auth/get-user-app-token', token, {
app_uid: uid,
});
expect(res.status).toBe(200);
return ((await res.json()) as { token: string }).token;
};
beforeAll(async () => {
await resetBudgets(env);
appUid = (await makeApp()).uid;
inApp = await shareWithOther({ appUid });
global = await shareWithOther();
});
it('lists every app`s shares for the account, or one app`s, or the app-less store`s with `null`', async () => {
for (const path of ['/kv/shares', '/kv/shares/with-me']) {
const token =
path === '/kv/shares'
? env.users.user.token
: env.users.other.token;
const all = await page(`${path}?limit=200`, token);
expect(handlesOf(all.body)).toEqual(
expect.arrayContaining([inApp.handle, global.handle]),
);
const one = await page(
`${path}?limit=200&appUid=${appUid}`,
token,
);
expect(handlesOf(one.body)).toEqual([inApp.handle]);
const appLess = await page(
`${path}?limit=200&appUid=null`,
token,
);
expect(handlesOf(appLess.body)).toContain(global.handle);
expect(
appLess.body.items.every((row) => row.appUid === null),
).toBe(true);
const bad = await page(`${path}?appUid=not-an-app`, token);
expect(bad.status).toBe(400);
}
});
it('keeps an app to its own namespace, refusing any other it names', async () => {
const token = await appToken(env.users.other.token, appUid);
const own = await page('/kv/shares/with-me?limit=200', token);
expect(handlesOf(own.body)).toEqual([inApp.handle]);
const named = await page(
`/kv/shares/with-me?limit=200&appUid=${appUid}`,
token,
);
expect(handlesOf(named.body)).toEqual([inApp.handle]);
const otherApp = (await makeApp()).uid;
for (const appUidParam of ['null', otherApp]) {
const refused = await page(
`/kv/shares/with-me?appUid=${appUidParam}`,
token,
);
expect(refused.status).toBe(403);
expect(refused.body).toMatchObject({
code: 'events_kv_handle_outside_namespace',
});
}
});
it('narrows what was shared out to live or revoked shares, all by default', async () => {
const gone = await shareWithOther({ appUid });
await req(
'DELETE',
`/kv/shares/${gone.handle}`,
env.users.user.token,
);
const at = async (state?: string) =>
page(
`/kv/shares?limit=200&appUid=${appUid}` +
(state ? `&state=${state}` : ''),
env.users.user.token,
);
expect(handlesOf((await at()).body)).toEqual([
inApp.handle,
gone.handle,
]);
expect(handlesOf((await at('all')).body)).toEqual([
inApp.handle,
gone.handle,
]);
expect(handlesOf((await at('live')).body)).toEqual([inApp.handle]);
expect(handlesOf((await at('revoked')).body)).toEqual([
gone.handle,
]);
const bad = await at('stopped');
expect(bad.status).toBe(400);
});
});
describe('with several recipients', () => {
let third: { username: string; uuid: string; id: number };
// Holds seeded rows, so no recipient's per-holder cap is what trips.
let filler: number;
let ownerId: number;
beforeAll(async () => {
const username = `kv-batch-${uuidv4().slice(0, 8)}`;
await createTestUser(env.server, {
username,
password: 'pw-test-1234',
});
const row = await env.server.stores.user.getByUsername(username);
third = { username, uuid: row!.uuid as string, id: row!.id! };
ownerId = (await env.server.stores.user.getByUsername(
env.users.user.username,
))!.id!;
const fillerName = `kv-filler-${uuidv4().slice(0, 8)}`;
await createTestUser(env.server, {
username: fillerName,
password: 'pw-test-1234',
});
filler = (await env.server.stores.user.getByUsername(fillerName))!
.id!;
});
const shareMany = (body: Record<string, unknown>) =>
req('POST', '/kv/shares', env.users.user.token, {
prefix: `batch-${uuidv4().slice(0, 8)}:`,
...body,
});
it('reports each recipient on its own item, in order', async () => {
const res = await shareMany({
recipients: [
env.users.other.username,
'nobody-by-this-name',
env.users.user.username,
'[email protected]',
{ uid: third.uuid },
],
access: 'write',
label: 'Team list',
});
expect(res.status).toBe(200);
const { items } = (await res.json()) as {
items: Record<string, unknown>[];
};
expect(items).toHaveLength(5);
expect(items[0]).toMatchObject({
recipient: env.users.other.username,
handle: expect.any(String),
access: 'write',
holder: env.users.other.username,
label: 'Team list',
appUid: null,
});
expect(items[1]).toEqual({
recipient: 'nobody-by-this-name',
error: {
message: expect.any(String),
code: 'subject_does_not_exist',
},
});
expect(items[2]).toMatchObject({
error: { code: 'bad_request' },
});
expect(items[3]).toMatchObject({
error: { code: 'invalid_recipient' },
});
expect(items[4]).toMatchObject({
recipient: { uid: third.uuid },
holder: third.username,
});
expect(items[4].handle).not.toBe(items[0].handle);
});
it('answers a recipient named twice with the one share', async () => {
const res = await shareMany({
recipients: [third.username, { uid: third.uuid }],
});
const { items } = (await res.json()) as {
items: Record<string, unknown>[];
};
expect(items[0].handle).toEqual(expect.any(String));
expect(items[1].handle).toBe(items[0].handle);
});
it('refuses a malformed batch outright', async () => {
for (const body of [
{ recipients: [] },
{ recipients: 'alice' },
{
recipients: Array.from(
{ length: KV_SHARE_BATCH_MAX + 1 },
(_, i) => `user-${i}`,
),
},
{
recipient: env.users.other.username,
recipients: [env.users.other.username],
},
]) {
const res = await shareMany(body);
expect(res.status).toBe(400);
await expect(res.json()).resolves.toMatchObject({
code: 'bad_request',
});
}
});
it('refuses the whole batch when it would pass the account cap', async () => {
const prefix = `batch-cap-${uuidv4().slice(0, 8)}:`;
const existing = await shareMany({
prefix,
recipients: [env.users.other.username],
});
const [standing] = (
(await existing.json()) as { items: { handle: string }[] }
).items;
try {
const { total } =
await env.server.stores.kvShareHandle.countLiveForOwner(
ownerId,
);
// One slot left.
await seedKvShareHandles(env.server, {
ownerUserId: ownerId,
ownerUuid,
granteeUserId: filler,
count: EVENTS_KV_HANDLES_PER_USER.limit - total - 1,
appUid: () => KV_GLOBAL_APP_KEY,
});
const refused = await shareMany({
prefix: `batch-over-${uuidv4().slice(0, 8)}:`,
recipients: [env.users.other.username, third.username],
});
expect(refused.status).toBe(409);
await expect(refused.json()).resolves.toMatchObject({
code: 'events_kv_handle_limit_reached',
});
expect(
(
await env.server.stores.kvShareHandle.countLiveForOwner(
ownerId,
)
).total,
).toBe(EVENTS_KV_HANDLES_PER_USER.limit - 1);
// The standing share takes no slot, so one new one still fits.
const fits = await shareMany({
prefix,
recipients: [env.users.other.username, third.username],
});
expect(fits.status).toBe(200);
const { items } = (await fits.json()) as {
items: Record<string, unknown>[];
};
expect(items[0].handle).toBe(standing.handle);
expect(items[1]).toMatchObject({ holder: third.username });
} finally {
await env.server.clients.db.write(
'DELETE FROM `kv_share_handles` WHERE `owner_user_id` = ?',
[ownerId],
);
}
});
it('spends the grant budget per recipient, all or nothing', async () => {
const [minute] = KV_SHARE_GRANT_LIMITS;
await claimRateLimit(
`${minute.scope}:${ownerId}`,
minute.limit,
minute.window,
minute.limit - 1,
);
try {
const refused = await shareMany({
recipients: [env.users.other.username, third.username],
});
expect(refused.status).toBe(429);
await expect(refused.json()).resolves.toMatchObject({
code: 'too_many_requests',
});
} finally {
await resetBudgets(env);
}
});
});
});
describe('/events/kv-handles beside /kv/shares', () => {
@@ -22,27 +22,99 @@ import type { Actor } from '../../core/actor.js';
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { bodyRecord } from '../../core/http/requestBody.js';
import { parseKvShareRecipient } from '../../services/kvShare/kvShares.js';
import type { KvShareHandleListOptions } from '../../stores/events/KvShareHandleStore.js';
import { KV_HANDLE_LIST_LIMIT_CAP } from '../../stores/events/KvShareHandleStore.js';
import {
assertShareableAppUid,
parseKvShareRecipient,
} from '../../services/kvShare/kvShares.js';
import type {
KvShareHandleListOptions,
KvShareOwnerListOptions,
KvShareOwnerListState,
} from '../../stores/events/KvShareHandleStore.js';
import {
KV_HANDLE_LIST_LIMIT_CAP,
KV_SHARE_OWNER_LIST_STATES,
} from '../../stores/events/KvShareHandleStore.js';
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
import { normalizeLimit } from '../../util/pagination.js';
import { EVENTS_LIST_LIMIT } from '../events/limits.js';
import type { KvShareView } from '../../services/kvShare/KvShareService.js';
import { EVENTS_LIST_LIMIT, KV_SHARE_BATCH_MAX } from '../events/limits.js';
import { PuterController } from '../types.js';
/** The app-less namespace is no app, so the wire says `null` for it. */
const publicAppUid = (appUid: string): string | null =>
appUid === KV_GLOBAL_APP_KEY ? null : appUid;
/**
* The namespace a listing's `appUid` names, in the form responses use: an app
* uid, or `null` for the app-less namespace.
*/
const listAppUid = (value: unknown): string | undefined => {
if (value === undefined || value === '') return undefined;
if (typeof value !== 'string')
throw new HttpError(400, '`appUid` must be a string', {
legacyCode: 'bad_request',
});
return value === 'null' ? KV_GLOBAL_APP_KEY : assertShareableAppUid(value);
};
const badRequest = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'bad_request' });
/** A share as `POST /kv/shares` reports it. */
const shareBody = (share: KvShareView) => ({
handle: share.handle,
prefix: share.prefix,
access: share.access,
appUid: publicAppUid(share.appUid),
holder: share.holderUsername,
label: share.label,
createdAt: share.createdAt,
});
/** `recipients` when the body names it: 1 to `KV_SHARE_BATCH_MAX` entries. */
const batchRecipients = (
body: Record<string, unknown>,
): unknown[] | undefined => {
const { recipient, recipients } = body;
if (recipients === undefined) return undefined;
if (recipient !== undefined)
throw badRequest('Send either `recipient` or `recipients`, not both');
if (
!Array.isArray(recipients) ||
recipients.length === 0 ||
recipients.length > KV_SHARE_BATCH_MAX
)
throw badRequest(
`\`recipients\` must be an array of 1 to ${KV_SHARE_BATCH_MAX} recipients`,
);
return recipients;
};
const listState = (value: unknown): KvShareOwnerListState | undefined => {
if (value === undefined || value === '') return undefined;
if (!(KV_SHARE_OWNER_LIST_STATES as readonly unknown[]).includes(value))
throw new HttpError(400, '`state` must be `live`, `revoked` or `all`', {
legacyCode: 'bad_request',
});
return value as KvShareOwnerListState;
};
const pageOptions = (req: Request): KvShareHandleListOptions => {
const query = (req.query ?? {}) as Record<string, unknown>;
return {
limit: normalizeLimit(query.limit, { cap: KV_HANDLE_LIST_LIMIT_CAP }),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
appUid: listAppUid(query.appUid),
};
};
const ownerPageOptions = (req: Request): KvShareOwnerListOptions => ({
...pageOptions(req),
state: listState((req.query ?? {}).state),
});
/**
* Sharing a region of one account's key-value data with another account.
*
@@ -56,7 +128,8 @@ export class KvShareController extends PuterController {
/**
* POST /kv/shares — share a region with another account, `read` unless the
* body asks otherwise. Sharing the same region with the same account again
* changes its level or label in place.
* changes its level or label in place. `recipients` shares it with several
* at once and reports each one on its own item.
*/
@Post('/shares', {
subdomain: 'api',
@@ -67,29 +140,52 @@ export class KvShareController extends PuterController {
async share(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = bodyRecord(req);
const fields = {
prefix: body.prefix,
access: body.access,
appUid: body.appUid,
label: body.label,
};
const recipients = batchRecipients(body);
if (recipients) {
const items = await this.services.kvShare.mintKvHandles(
actor,
{ ...fields, recipients },
{ defaultAccess: 'read' },
);
res.json({
items: items.map((item) =>
item.share
? {
recipient: item.recipient,
...shareBody(item.share),
}
: {
recipient: item.recipient,
error: {
message: item.error?.message ?? 'Not shared',
code:
item.error?.legacyCode ??
item.error?.code ??
'internal_error',
},
},
),
});
return;
}
const share = await this.services.kvShare.mintKvHandle(
actor,
{
prefix: body.prefix,
recipient: parseKvShareRecipient(body.recipient),
access: body.access,
appUid: body.appUid,
label: body.label,
},
{ ...fields, recipient: parseKvShareRecipient(body.recipient) },
{ defaultAccess: 'read' },
);
res.json({
handle: share.handle,
prefix: share.prefix,
access: share.access,
appUid: publicAppUid(share.appUid),
holder: share.holderUsername,
label: share.label,
createdAt: share.createdAt,
});
res.json(shareBody(share));
}
/** GET /kv/shares — what this account has shared out, retired included. */
/**
* GET /kv/shares — what this account has shared out, retired included
* unless `state` narrows it.
*/
@Get('/shares', {
subdomain: 'api',
requireAuth: true,
@@ -100,7 +196,7 @@ export class KvShareController extends PuterController {
const actor = this.#requireActor(req);
const page = await this.services.kvShare.listKvHandles(
actor,
pageOptions(req),
ownerPageOptions(req),
);
res.json({
items: page.items.map((row) => ({
@@ -816,6 +816,9 @@ export class ChatCompletionDriver extends PuterDriver {
// where "done" is the stream draining rather than this method
// returning.
let hold: CreditHold = NO_CREDIT_HOLD;
// The gate's estimate for the current attempt: the message estimate
// plus whatever its provider reads back as prompt beyond the messages.
let attemptPromptTokenEstimate = promptTokenEstimate;
// The current attempt's balance-bound output cap, if it had one.
let fundsCap: number | undefined;
// The cost factor the current attempt's charges are recorded at.
@@ -824,12 +827,16 @@ export class ChatCompletionDriver extends PuterDriver {
// charge, so a rate change mid-request can't split the two.
let pricingTime = new Date();
if (!useFakeProvider) {
({ hold, fundsCap, costFactor } = await this.#applyCreditGate(
actor,
model,
args,
{ promptTokenEstimate, requestedMaxTokens, pricingTime },
));
({
hold,
fundsCap,
costFactor,
promptTokenEstimate: attemptPromptTokenEstimate,
} = await this.#applyCreditGate(actor, model, args, {
promptTokenEstimate,
requestedMaxTokens,
pricingTime,
}));
}
// First attempt
@@ -957,12 +964,16 @@ export class ChatCompletionDriver extends PuterDriver {
// this one starts from nothing held.
pricingTime = new Date();
if (!useFakeProvider) {
({ hold, fundsCap, costFactor } =
await this.#applyCreditGate(actor, fallback, args, {
promptTokenEstimate,
requestedMaxTokens,
pricingTime,
}));
({
hold,
fundsCap,
costFactor,
promptTokenEstimate: attemptPromptTokenEstimate,
} = await this.#applyCreditGate(actor, fallback, args, {
promptTokenEstimate,
requestedMaxTokens,
pricingTime,
}));
}
tried.add(routeId(fallback.provider!, fallback.id));
@@ -1188,7 +1199,7 @@ export class ChatCompletionDriver extends PuterDriver {
actor,
chatStream,
model,
promptTokenEstimate,
promptTokenEstimate: attemptPromptTokenEstimate,
completionId,
username,
intendedProvider,
@@ -1432,10 +1443,16 @@ export class ChatCompletionDriver extends PuterDriver {
/** The instant this attempt is priced at. */
pricingTime: Date;
},
): Promise<{ hold: CreditHold; fundsCap?: number; costFactor: number }> {
): Promise<{
hold: CreditHold;
fundsCap?: number;
costFactor: number;
/** The prompt estimate this attempt was priced on. */
promptTokenEstimate: number;
}> {
const metering = this.services.metering;
const { promptTokenEstimate, requestedMaxTokens, pricingTime } =
estimates;
const { requestedMaxTokens, pricingTime } = estimates;
let { promptTokenEstimate } = estimates;
// A provider can name its own cost keys for this request (Claude's
// fast-mode rate card), cap its output, and size an extra reservation
// the per-token math can't see (a web-search/advisor tool that could
@@ -1446,6 +1463,7 @@ export class ChatCompletionDriver extends PuterDriver {
this.#providers[model.provider!]?.requestPricing?.(args, model, {
promptTokenEstimate,
}) ?? {};
promptTokenEstimate += pricing.additionalPromptTokens ?? 0;
const { inputKey: defaultInputKey, outputKey: defaultOutputKey } =
costKeys(model);
const inputKey = pricing.inputKey ?? defaultInputKey;
@@ -1560,7 +1578,7 @@ export class ChatCompletionDriver extends PuterDriver {
actor,
Math.max(worstCaseCost, minimumCredits),
);
return { hold, fundsCap, costFactor };
return { hold, fundsCap, costFactor, promptTokenEstimate };
}
/** The key `model`'s usage is recorded and cost-factored under. */
@@ -194,7 +194,9 @@ export class OpenAICompatProvider implements IChatProvider {
await OpenAIUtil.process_input_messages(messages, {
preserveToolCallOrder: this.preservesToolCallOrder(model),
keepExtraContent: this.keepsToolCallExtraContent(model),
keepAssistantContent: this.preservesAssistantContent(model),
}),
model,
);
const tools = args.tools
@@ -324,8 +326,19 @@ export class OpenAICompatProvider implements IChatProvider {
return false;
}
/**
* Whether a replayed assistant turn keeps its text beside hoisted tool
* calls.
*/
protected preservesAssistantContent(_model: IChatModel): boolean {
return false;
}
/** Last pass over the messages, after `process_input_messages`. */
protected prepareMessages(messages: PuterMessage[]): PuterMessage[] {
protected prepareMessages(
messages: PuterMessage[],
_model: IChatModel,
): PuterMessage[] {
return messages;
}
@@ -20,8 +20,8 @@
/**
* Integration test for the Moonshot provider.
*
* Uses `kimi-k2.6` (the cheapest current Kimi model). Skipped
* when `PUTER_TEST_AI_MOONSHOT_API_KEY` is unset.
* Uses `kimi-k2.6` (the cheapest current Kimi model) and `kimi-k3.1-pro`.
* Skipped when `PUTER_TEST_AI_MOONSHOT_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
@@ -59,5 +59,36 @@ describe.skipIf(skipUnlessEnv(ENV_VAR))(
?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
});
it('returns a completion and cache usage from kimi-k3.1-pro', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
const provider = new MoonshotProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const result = (await withTestActor(() =>
provider.complete({
model: 'kimi-k3.1-pro',
messages: [
{ role: 'user', content: 'Say hi in one word.' },
],
// Reasoning counts against the cap, so leave room for it.
max_tokens: 2048,
reasoning_effort: 'low',
}),
)) as {
message?: { content?: string };
usage?: Record<string, number>;
};
const text = result.message?.content;
expect(typeof text === 'string' && text.length > 0).toBe(true);
expect(result.usage).toMatchObject({
prompt_tokens: expect.any(Number),
completion_tokens: expect.any(Number),
cached_tokens: expect.any(Number),
cache_write_tokens: expect.any(Number),
});
});
},
);
@@ -20,17 +20,15 @@
/**
* Offline unit tests for MoonshotProvider.
*
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
* redis) and constructs MoonshotProvider directly against the live
* wired `MeteringService` so the recording side is exercised end-to-
* end. Moonshot is OpenAI-compatible so the OpenAI SDK is mocked at
* the module boundary; that's the real network egress point. Image-
* inlining behaviour is covered by `utils/inlineImages.test.ts`; here we
* stub `inlineHttpImageUrls` so http URLs in vision messages don't
* trigger network fetches and only verify the provider invokes it
* for vision-capable models. The companion integration test
* (MoonshotProvider.integration.test.ts) exercises the real Moonshot
* endpoint.
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
* constructs MoonshotProvider directly against the live wired `MeteringService`
* so the recording side is exercised end-to- end. Moonshot is OpenAI-compatible
* so the OpenAI SDK is mocked at the module boundary; that's the real network
* egress point. Image- inlining behaviour is covered by
* `utils/inlineImages.test.ts`; here we stub `inlineHttpImageUrls` so http URLs
* in vision messages don't trigger network fetches and only verify the provider
* invokes it for vision-capable models. The companion integration test
* (MoonshotProvider.integration.test.ts) exercises the real Moonshot endpoint.
*/
import { Writable } from 'node:stream';
@@ -46,14 +44,23 @@ import {
type MockInstance,
} from 'vitest';
import { ChatCompletionDriver } from '../../ChatCompletionDriver.js';
import { clearUnhealthyRoutes } from '../../utils/providerHealth.js';
import { SYSTEM_ACTOR } from '../../../../core/actor.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { PuterServer } from '../../../../server.js';
import { setupTestServer } from '../../../../testUtil.js';
import { withTestActor } from '../../../integrationTestUtil.js';
import {
readStreamResult,
withTestActor,
} from '../../../integrationTestUtil.js';
import { AIChatStream } from '../../utils/Streaming.js';
import { MOONSHOT_MODELS } from './models.js';
import { MoonshotProvider } from './MoonshotProvider.js';
import {
estimatePromptTokens,
estimateTextTokens,
} from '../../utils/usageEstimate.js';
import { modelLookupNames } from '../../utils/modelRouting.js';
// ── OpenAI SDK mock ─────────────────────────────────────────────────
@@ -93,10 +100,23 @@ vi.mock('../../utils/inlineImages.js', () => ({
// ── Test harness ────────────────────────────────────────────────────
let server: PuterServer;
let driver: ChatCompletionDriver;
let recordSpy: MockInstance<MeteringService['utilRecordUsageObject']>;
beforeAll(async () => {
server = await setupTestServer();
driver = new ChatCompletionDriver(
{
providers: {
moonshot: { apiKey: 'test-key' },
ollama: { enabled: false },
},
} as never,
server.clients,
server.stores,
server.services,
);
await driver.onServerStart();
});
afterAll(async () => {
@@ -140,6 +160,7 @@ const makeCapturingChatStream = () => {
};
beforeEach(() => {
clearUnhealthyRoutes();
createMock.mockReset();
openAICtor.mockReset();
inlineHttpImageUrlsMock.mockReset();
@@ -390,7 +411,10 @@ describe('MoonshotProvider image inlining', () => {
await withTestActor(() =>
provider.complete({
model: 'kimi-k3', // kimi-k3 declares image input modality
messages: messages as unknown as { role: string; content: unknown }[],
messages: messages as unknown as {
role: string;
content: unknown;
}[],
}),
);
@@ -782,3 +806,656 @@ describe('MoonshotProvider.checkModeration', () => {
);
});
});
describe('Kimi K3.1 Pro', () => {
const models = ['kimi-k3.1-pro', 'kimi-k3.1-pro-highspeed'];
/** Catalog rates in µ¢ per token: $2/$10 for Pro, $4/$20 for Highspeed. */
const rates = (model: string) =>
model.endsWith('highspeed')
? { input: 400, output: 2000 }
: { input: 200, output: 1000 };
const usage = {
prompt_tokens: 100,
completion_tokens: 10,
prompt_tokens_details: { cached_tokens: 20, cache_write_tokens: 50 },
};
const response = () => ({
choices: [
{
message: {
role: 'assistant',
content: 'answer',
reasoning_content: 'reasoning',
},
finish_reason: 'stop',
},
],
usage: structuredClone(usage),
});
const fund = (balance: number) =>
vi.spyOn(
server.services.metering,
'getUsageHeadroom',
).mockResolvedValue({ balance, held: 0 });
const user = (content: string) => [{ role: 'user', content }];
it.each(models)(
'maps %s caps and reasoning without sampling parameters',
async (model) => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
provider.complete({
model,
messages: user('hi'),
max_tokens: 512,
reasoning_effort: 'high',
temperature: 0.2,
top_p: 0.1,
}),
);
const sent = createMock.mock.calls[0]![0];
expect(sent).toMatchObject({
model,
max_completion_tokens: 512,
reasoning_effort: 'high',
});
expect(sent).not.toHaveProperty('max_tokens');
expect(sent).not.toHaveProperty('temperature');
expect(sent).not.toHaveProperty('top_p');
},
);
it('uses the upstream default output cap and leaves thinking to the model', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
provider.complete({ model: models[0]!, messages: user('hi') }),
);
const sent = createMock.mock.calls[0]![0];
expect(sent.max_completion_tokens).toBe(131072);
expect(sent).not.toHaveProperty('thinking');
expect(sent).not.toHaveProperty('reasoning_effort');
});
it.each(['low', 'max'] as const)(
'forwards %s reasoning effort',
async (effort) => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
provider.complete({
model: models[0]!,
messages: user('hi'),
reasoning_effort: effort,
}),
);
expect(createMock.mock.calls[0]![0].reasoning_effort).toBe(effort);
},
);
it('rejects an unsupported reasoning effort before inference', async () => {
const { provider } = makeProvider();
await expect(
withTestActor(() =>
provider.complete({
model: models[0]!,
messages: user('hi'),
reasoning_effort: 'medium',
}),
),
).rejects.toMatchObject({ statusCode: 400 });
expect(createMock).not.toHaveBeenCalled();
});
it.each([
{ options: { reasoning: { effort: 'low' } }, expected: 'low' },
{ options: { reasoning: { effort: 'max' } }, expected: 'max' },
{
options: { reasoning: { effort: 'low' }, reasoning_effort: 'high' },
expected: 'high',
},
{ options: {}, expected: undefined },
])(
'sends the nested reasoning form through the driver: $options',
async ({ options, expected }) => {
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
driver.complete({
model: models[0]!,
provider: 'moonshotai',
messages: user('hi'),
...options,
} as never),
);
expect(createMock).toHaveBeenCalledTimes(1);
expect(createMock.mock.calls[0]![0].reasoning_effort).toBe(
expected,
);
},
);
it.each(['medium', 'none'])(
'rejects the nested reasoning form %s through the driver',
async (effort) => {
await expect(
withTestActor(() =>
driver.complete({
model: models[0]!,
provider: 'moonshotai',
messages: user('hi'),
reasoning: { effort },
} as never),
),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
});
expect(createMock).not.toHaveBeenCalled();
},
);
it('keeps K3 on max_tokens and bills its cache writes at the K3 write rate', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
provider.complete({
model: 'kimi-k3',
messages: user('hi'),
max_tokens: 512,
}),
);
const sent = createMock.mock.calls[0]![0];
expect(sent).toMatchObject({ model: 'kimi-k3', max_tokens: 512 });
expect(sent).not.toHaveProperty('max_completion_tokens');
expect(recordSpy).toHaveBeenCalledWith(
{
prompt_tokens: 30,
completion_tokens: 10,
cached_tokens: 20,
cache_write_tokens: 50,
},
expect.anything(),
'moonshotai:kimi-k3',
{
prompt_tokens: 30 * 300,
completion_tokens: 10 * 1500,
cached_tokens: 20 * 30,
cache_write_tokens: 50 * 300,
},
);
});
it.each(models)(
'meters cache reads, writes and uncached input separately for %s',
async (model) => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(response());
const result = await withTestActor(() =>
provider.complete({ model, messages: user('hi') }),
);
const { input, output } = rates(model);
expect(recordSpy).toHaveBeenCalledWith(
{
prompt_tokens: 30,
completion_tokens: 10,
cached_tokens: 20,
cache_write_tokens: 50,
},
expect.anything(),
`moonshotai:${model}`,
{
prompt_tokens: 30 * input,
completion_tokens: 10 * output,
cached_tokens: 20 * 20,
cache_write_tokens: 50 * input,
},
);
expect(result).toMatchObject({
usageDetails: {
inputTokens: 30,
outputTokens: 10,
cacheReadTokens: 20,
cacheWrite5mTokens: 50,
},
});
},
);
it('preserves assistant reasoning, text and tool-call order on replay', async () => {
const { provider } = makeProvider();
const toolCalls = ['a', 'b'].map((id) => ({
id,
type: 'function',
function: { name: 'lookup', arguments: '{}' },
}));
const first = response();
Object.assign(first.choices[0]!.message, { tool_calls: toolCalls });
createMock.mockResolvedValueOnce(first);
const result = await withTestActor(() =>
provider.complete({ model: models[0]!, messages: user('hi') }),
);
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
provider.complete({
model: models[0]!,
messages: [(result as { message: unknown }).message],
}),
);
const replayed = createMock.mock.calls[1]![0].messages[0];
expect(replayed).toMatchObject({
role: 'assistant',
content: 'answer',
reasoning_content: 'reasoning',
});
expect(replayed).not.toHaveProperty('reasoning');
expect(replayed.tool_calls).toEqual(toolCalls);
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
provider.complete({
model: models[0]!,
messages: [
{
role: 'assistant',
reasoning: 'reasoning',
content: [
{ type: 'text', text: 'answer' },
...['a', 'b'].map((id) => ({
type: 'tool_use',
id,
name: 'lookup',
input: {},
})),
],
},
],
}),
);
expect(createMock.mock.calls[2]![0].messages[0]).toEqual({
role: 'assistant',
reasoning_content: 'reasoning',
content: [{ type: 'text', text: 'answer' }],
tool_calls: toolCalls,
});
});
it('meters a stream from its final usage chunk', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(
asAsyncIterable([
{ choices: [{ delta: { reasoning_content: 'reasoning' } }] },
{ choices: [{ delta: { content: 'answer' } }] },
{ choices: [], usage: structuredClone(usage) },
]),
);
const result = await withTestActor(() =>
provider.complete({
model: models[0]!,
messages: user('hi'),
stream: true,
}),
);
const harness = makeCapturingChatStream();
await (
result as {
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
}
).init_chat_stream({ chatStream: harness.chatStream });
expect(createMock.mock.calls[0]![0].stream_options).toEqual({
include_usage: true,
});
expect(recordSpy).toHaveBeenCalledTimes(1);
expect(recordSpy.mock.calls[0]![0]).toEqual({
prompt_tokens: 30,
completion_tokens: 10,
cached_tokens: 20,
cache_write_tokens: 50,
});
});
it.each(models)(
'reserves the prompt at the input rate for %s',
async (model) => {
const { input, output } = rates(model);
fund(2_000_000_000);
const reserve = vi.spyOn(
server.services.metering,
'reserveCredits',
);
createMock.mockResolvedValueOnce({
choices: [
{
message: { role: 'assistant', content: 'ok' },
finish_reason: 'stop',
},
],
usage: {
prompt_tokens: 1_000_000,
completion_tokens: 10,
prompt_tokens_details: {
cached_tokens: 100_000,
cache_write_tokens: 900_000,
},
},
});
await withTestActor(() =>
driver.complete({
model,
provider: 'moonshotai',
messages: user('x'.repeat(8_000_000)),
max_tokens: 10,
}),
);
expect(reserve).toHaveBeenCalledTimes(1);
expect(reserve.mock.calls[0]![1]).toBeGreaterThanOrEqual(
1_000_000 * input + 10 * output,
);
expect(reserve.mock.calls[0]![1]).toBeLessThan(
1_000_001 * input + 10 * output,
);
expect(recordSpy.mock.calls[0]![0]).toMatchObject({
prompt_tokens: 0,
completion_tokens: 10,
cached_tokens: 100_000,
cache_write_tokens: 900_000,
});
},
);
it.each(['kimi-k3', 'kimi-k2.6'])(
'keeps the existing reservation for %s',
async (model) => {
const entry = MOONSHOT_MODELS.find(
(candidate) => candidate.id === model,
)!;
fund(2_000_000_000);
const reserve = vi.spyOn(
server.services.metering,
'reserveCredits',
);
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
driver.complete({
model,
provider: 'moonshotai',
messages: [
{
role: 'assistant',
content: 'x'.repeat(8000),
reasoning: 'thought '.repeat(90000),
},
],
max_tokens: 10,
}),
);
expect(reserve.mock.calls[0]![1]).toBe(
1000 * Number(entry.costs.prompt_tokens) +
10 * Number(entry.costs.completion_tokens),
);
},
);
it.each(models)(
'rejects an underfunded request for %s before inference',
async (model) => {
// Short of the prompt alone at the input rate.
fund(model.endsWith('highspeed') ? 300_000_000 : 150_000_000);
await expect(
withTestActor(() =>
driver.complete({
model,
provider: 'moonshotai',
messages: user('x'.repeat(8_000_000)),
max_tokens: 10,
}),
),
).rejects.toMatchObject({ statusCode: 402 });
expect(createMock).not.toHaveBeenCalled();
},
);
it.each([null, ''])(
'round-trips tool-only assistant content %s through the driver',
async (content) => {
const toolCalls = [
{
id: 'call_paris',
type: 'function',
function: {
name: 'weather',
arguments: '{"city":"Paris"}',
},
},
];
createMock.mockResolvedValueOnce({
choices: [
{
message: {
role: 'assistant',
content,
reasoning_content: 'Check weather',
tool_calls: toolCalls,
},
finish_reason: 'tool_calls',
},
],
usage: { prompt_tokens: 1, completion_tokens: 1 },
});
const first = await withTestActor(() =>
driver.complete({
model: models[0]!,
provider: 'moonshotai',
messages: user('weather?'),
}),
);
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
driver.complete({
model: models[0]!,
provider: 'moonshotai',
messages: [
(first as { message: unknown }).message,
{
role: 'tool',
tool_call_id: 'call_paris',
content: 'sunny',
},
],
}),
);
const assistant = createMock.mock.calls[1]![0].messages.find(
(message: { role: string }) => message.role === 'assistant',
);
expect(assistant).toMatchObject({
role: 'assistant',
content: null,
reasoning_content: 'Check weather',
});
expect(assistant).not.toHaveProperty('normalized');
expect(assistant.tool_calls).toEqual(toolCalls);
expect(
JSON.parse(assistant.tool_calls[0].function.arguments),
).toEqual({ city: 'Paris' });
},
);
it.each([{ city: 'Paris' }, '{"city":"Paris"}'])(
'keeps native tool_use input %j through the driver',
async (input) => {
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
driver.complete({
model: models[0]!,
provider: 'moonshotai',
messages: [
{
role: 'assistant',
reasoning_content: 'Check weather',
content: [
{
type: 'tool_use',
id: 'call_paris',
name: 'weather',
input,
},
],
},
{
role: 'tool',
tool_call_id: 'call_paris',
content: 'sunny',
},
],
}),
);
const assistant = createMock.mock.calls[0]![0].messages.find(
(message: { role: string }) => message.role === 'assistant',
);
expect(assistant.reasoning_content).toBe('Check weather');
expect(assistant.tool_calls[0].function.arguments).toBe(
'{"city":"Paris"}',
);
},
);
it.each(
models.flatMap((model) =>
['reasoning', 'reasoning_content', 'both'].map((alias) => ({
model,
alias,
})),
),
)(
'reserves replayed reasoning once with wire precedence: $model $alias',
async ({ model, alias }) => {
const text = 'reasoning '.repeat(90000);
const assistant = {
role: 'assistant',
content: 'answer',
...(alias !== 'reasoning_content'
? { reasoning: alias === 'both' ? 'short' : text }
: {}),
...(alias !== 'reasoning' ? { reasoning_content: text } : {}),
};
const messages = [
...user('hi'),
assistant,
...user('continue'),
];
const tokens =
estimatePromptTokens(messages) + estimateTextTokens(text);
const { input, output } = rates(model);
const headroom = fund(2_000_000_000);
const reserve = vi.spyOn(
server.services.metering,
'reserveCredits',
);
createMock.mockResolvedValueOnce(response());
await withTestActor(() =>
driver.complete({
model,
messages: structuredClone(messages),
max_tokens: 10,
}),
);
expect(reserve.mock.calls[0]![1]).toBe(
tokens * input + 10 * output,
);
expect(
createMock.mock.calls[0]![0].messages[1].reasoning_content,
).toBe(text);
headroom.mockResolvedValue({
balance: tokens * input - 1,
held: 0,
});
await expect(
withTestActor(() =>
driver.complete({
model,
messages: structuredClone(messages),
max_tokens: 10,
}),
),
).rejects.toMatchObject({ statusCode: 402 });
expect(createMock).toHaveBeenCalledTimes(1);
expect(reserve).toHaveBeenCalledTimes(1);
},
);
it.each(models)(
'charges the estimate, replayed reasoning included, when a %s stream is interrupted',
async (model) => {
const reasoning = 'history '.repeat(1000);
const messages = [
{
role: 'assistant',
content: 'answer',
reasoning: 'ignored alias',
reasoning_content: reasoning,
},
...user('hi'),
];
const expectedInput =
estimatePromptTokens(messages) + estimateTextTokens(reasoning);
createMock.mockResolvedValueOnce(
(async function* () {
yield {
choices: [{ delta: { content: 'partial answer' } }],
};
throw new Error('upstream connection lost');
})(),
);
const result = await withTestActor(() =>
driver.complete({
model,
messages: structuredClone(messages),
stream: true,
}),
);
const text = (await readStreamResult(result)).toString();
expect(text).toContain('upstream connection lost');
await vi.waitFor(() => expect(recordSpy).toHaveBeenCalledTimes(1));
expect(
createMock.mock.calls[0]![0].messages[0].reasoning_content,
).toBe(reasoning);
expect(recordSpy.mock.calls[0]![0]).toHaveProperty(
'estimated_prompt_tokens',
expectedInput,
);
expect(recordSpy.mock.calls[0]![0]).toHaveProperty(
'estimated_completion_tokens',
);
},
);
it.each(models)(
'charges the estimate when a %s stream completes without a usage chunk',
async (model) => {
createMock.mockResolvedValueOnce(
asAsyncIterable([
{ choices: [{ delta: { content: 'answer' } }] },
{ choices: [{ delta: {}, finish_reason: 'stop' }] },
]),
);
const result = await withTestActor(() =>
driver.complete({
model,
provider: 'moonshotai',
messages: user('hi'),
stream: true,
}),
);
const events = (await readStreamResult(result))
.toString()
.trim()
.split('\n')
.map((line) => JSON.parse(line));
expect(events.some((event) => event.type === 'error')).toBe(false);
await vi.waitFor(() => expect(recordSpy).toHaveBeenCalledTimes(1));
expect(recordSpy.mock.calls[0]![0]).toMatchObject({
estimated_prompt_tokens: expect.any(Number),
estimated_completion_tokens: expect.any(Number),
});
},
);
});
@@ -18,14 +18,39 @@
*/
import { OpenAI } from 'openai';
import { HttpError } from '../../../../core/http/HttpError.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import type {
IChatModel,
ICompleteArguments,
PuterMessage,
} from '../../types.js';
import { openAICompatParams } from '../../utils/openaiParams.js';
import { sdkClientOptions } from '../../utils/sdkClient.js';
import { estimateTextTokens } from '../../utils/usageEstimate.js';
import {
type ChatProviderConfig,
OpenAICompatProvider,
} from '../OpenAICompatProvider.js';
import { MOONSHOT_MODELS } from './models.js';
/**
* The K3-series request shape: reasoning always on, capped by
* `max_completion_tokens`.
*/
const K31_MODEL_IDS = new Set(['kimi-k3.1-pro', 'kimi-k3.1-pro-highspeed']);
const K31_REASONING_EFFORTS = ['low', 'high', 'max'];
/** Moonshot's own output cap when the request names none. */
const K31_DEFAULT_MAX_COMPLETION_TOKENS = 131_072;
const isK31 = (model: IChatModel) => K31_MODEL_IDS.has(model.id);
/** A replayed assistant turn's reasoning, under the wire key or Puter's. */
const replayedReasoning = (message: PuterMessage): string | undefined => {
const reasoning = message.reasoning_content ?? message.reasoning;
return typeof reasoning === 'string' ? reasoning : undefined;
};
export class MoonshotProvider extends OpenAICompatProvider {
constructor(config: ChatProviderConfig, meteringService: MeteringService) {
super(meteringService, {
@@ -41,4 +66,79 @@ export class MoonshotProvider extends OpenAICompatProvider {
inlineImages: 'vision',
});
}
/**
* K3.1 Pro reads replayed reasoning back as prompt, which the message
* estimate the credit gate works from doesn't count.
*/
requestPricing(args: ICompleteArguments, model: IChatModel) {
if (!isK31(model)) return {};
const additionalPromptTokens = (args.messages ?? []).reduce(
(tokens, message) => {
if (message.role !== 'assistant') return tokens;
const reasoning = replayedReasoning(message);
return tokens + (reasoning ? estimateTextTokens(reasoning) : 0);
},
0,
);
return { additionalPromptTokens };
}
protected override preservesToolCallOrder(model: IChatModel) {
return isK31(model);
}
protected override preservesAssistantContent(model: IChatModel) {
return isK31(model);
}
/** K3.1 Pro wants a turn's reasoning back under `reasoning_content`, once. */
protected override prepareMessages(
messages: PuterMessage[],
model: IChatModel,
) {
if (!isK31(model)) return messages;
return messages.map((message) => {
if (message.role !== 'assistant') return message;
const { reasoning, normalized: _normalized, ...rest } = message;
const replayed = replayedReasoning(message);
return replayed === undefined
? rest
: { ...rest, reasoning_content: replayed };
});
}
protected override vendorParams(
params: Record<string, unknown>,
args: ICompleteArguments,
model: IChatModel,
) {
if (!isK31(model)) return params;
const effort = args.reasoning_effort ?? args.reasoning?.effort;
if (effort !== undefined && !K31_REASONING_EFFORTS.includes(effort)) {
throw new HttpError(
400,
`reasoning_effort must be one of ${K31_REASONING_EFFORTS.join(', ')} for ${model.id}`,
{ legacyCode: 'bad_request' },
);
}
// The K3 series deprecates `max_tokens` in favour of `max_completion_tokens`.
const { max_tokens, ...rest } = params;
return {
...rest,
...openAICompatParams(
{ ...args, reasoning_effort: effort },
'chat',
{
only: [
'tool_choice',
'response_format',
'reasoning_effort',
],
},
),
max_completion_tokens:
max_tokens ?? K31_DEFAULT_MAX_COMPLETION_TOKENS,
};
}
}
@@ -20,6 +20,53 @@
import type { IChatModel } from '../../types.js';
export const MOONSHOT_MODELS: IChatModel[] = [
// -- Kimi K3.1 Pro ------------------------------------------------
{
puterId: 'moonshotai:moonshotai/kimi-k3.1-pro',
id: 'kimi-k3.1-pro',
name: 'Kimi K3.1 Pro',
aliases: ['moonshotai/kimi-k3.1-pro', 'moonshot/kimi-k3.1-pro'],
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 200, // $2.00 per 1M
completion_tokens: 1000, // $10.00 per 1M
cached_tokens: 20, // $0.20 per 1M
cache_write_tokens: 200, // $2.00 per 1M
},
context: 1_048_576,
max_tokens: 1_048_576,
tool_call: true,
release_date: '2026-10-24',
},
{
puterId: 'moonshotai:moonshotai/kimi-k3.1-pro-highspeed',
id: 'kimi-k3.1-pro-highspeed',
name: 'Kimi K3.1 Pro Highspeed',
aliases: [
'moonshotai/kimi-k3.1-pro-highspeed',
'moonshot/kimi-k3.1-pro-highspeed',
],
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 400, // $4.00 per 1M
completion_tokens: 2000, // $20.00 per 1M
cached_tokens: 20, // $0.20 per 1M
cache_write_tokens: 400, // $4.00 per 1M
},
context: 1_048_576,
max_tokens: 1_048_576,
tool_call: true,
release_date: '2026-10-24',
},
// -- Flagship ----------------------------------------------------
{
puterId: 'moonshotai:moonshotai/kimi-k3',
@@ -35,6 +82,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
prompt_tokens: 300, // $3.00 per 1M
completion_tokens: 1500, // $15.00 per 1M
cached_tokens: 30, // $0.30 per 1M
cache_write_tokens: 300, // $3.00 per 1M
},
context: 1_048_576,
max_tokens: 1_048_576,
+3 -1
View File
@@ -238,7 +238,7 @@ export interface ICompleteArguments {
'auto' | 'default' | 'flex' | 'scale' | 'priority' | undefined;
max_tokens?: number;
temperature?: number;
reasoning?: { effort: 'low' | 'medium' | 'high' } | undefined;
reasoning?: { effort: ReasoningEffort } | undefined;
text?: { verbosity?: 'low' | 'medium' | 'high' | undefined } | undefined;
reasoning_effort?: ReasoningEffort | undefined;
verbosity?: 'low' | 'medium' | 'high' | undefined;
@@ -345,6 +345,8 @@ export interface IChatProvider {
inputKey?: string;
outputKey?: string;
extraCost?: number;
/** Provider-specific prompt tokens outside message.content. */
additionalPromptTokens?: number;
maxTokens?: number;
};
/**
@@ -170,7 +170,12 @@ const hoistToolUseBlocks = (
type: 'function',
function: {
name: block.name,
arguments: JSON.stringify(block.input),
// A replayed OpenAI-shaped call arrives with `input` already
// serialized; encoding it again sends a JSON string literal.
arguments:
typeof block.input === 'string'
? block.input
: JSON.stringify(block.input),
},
...(extraContent && block.extra_content
? { extra_content: block.extra_content }
@@ -190,14 +195,22 @@ const hoistToolUseBlocks = (
* make changes in-place.
*
* @param {Message[]} messages - Array of normalized messages
* @param {{ preserveToolCallOrder?: boolean; keepExtraContent?: boolean }} [opts]
* @param {{
* preserveToolCallOrder?: boolean;
* keepExtraContent?: boolean;
* keepAssistantContent?: boolean;
* }} [opts]
* `preserveToolCallOrder` keeps a turn's tool calls in the order the model made
* them. `keepExtraContent` keeps each tool call's `extra_content`.
* @returns {Message[]} - Array of messages in OpenAI format
*/
export const process_input_messages = async (
messages,
{ preserveToolCallOrder = false, keepExtraContent = false } = {},
{
preserveToolCallOrder = false,
keepExtraContent = false,
keepAssistantContent = false,
} = {},
) => {
for (const msg of messages) {
if (!msg.content) continue;
@@ -220,7 +233,12 @@ export const process_input_messages = async (
extraContent: keepExtraContent,
})
)
msg.content = null;
msg.content =
keepAssistantContent &&
msg.role === 'assistant' &&
content.length
? content
: null;
// coerce tool results
// (we assume multiple tool results were already split into separate messages)
@@ -470,11 +488,22 @@ export const extractMeteredUsage = (usage) => {
* @param {import('openai/resources/completions.mjs').CompletionUsage | undefined} usage
*/
export const splitCachedPrompt = (usage) => {
const cachedTokens = usage?.prompt_tokens_details?.cached_tokens ?? 0;
const details = usage?.prompt_tokens_details;
const cachedTokens = details?.cached_tokens ?? 0;
// Moonshot counts the prompt tokens it wrote to its prefix cache here;
// they're billed at the model's write rate, not the plain prompt rate.
const writeTokens =
typeof details?.cache_write_tokens === 'number'
? details.cache_write_tokens
: undefined;
return {
prompt_tokens: (usage?.prompt_tokens ?? 0) - cachedTokens,
prompt_tokens:
(usage?.prompt_tokens ?? 0) - cachedTokens - (writeTokens ?? 0),
completion_tokens: usage?.completion_tokens ?? 0,
cached_tokens: cachedTokens,
...(writeTokens !== undefined
? { cache_write_tokens: writeTokens }
: {}),
};
};
@@ -331,6 +331,29 @@ describe('process_input_messages', () => {
]);
});
it("keeps a replayed call's serialized arguments verbatim", async () => {
// An OpenAI-shaped `tool_calls` entry normalizes with its arguments
// still a JSON string.
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: '{"q":"puter"}',
},
],
},
];
const [out] = (await process_input_messages(messages)) as Array<{
tool_calls: Array<{ function: { arguments: string } }>;
}>;
expect(out!.tool_calls[0]!.function.arguments).toBe('{"q":"puter"}');
});
describe('tool call extra_content', () => {
const turn = (): Array<Record<string, unknown>> => [
{
@@ -772,6 +795,28 @@ describe('process_input_messages_responses_api', () => {
expect(out!.content).toBeUndefined();
});
it("keeps a replayed call's serialized arguments verbatim", async () => {
const messages: Array<Record<string, unknown>> = [
{
role: 'assistant',
content: [
{
type: 'tool_use',
id: 'call_1',
name: 'lookup',
input: '{"q":"puter"}',
},
],
},
];
const [out] = (await process_input_messages_responses_api(
messages,
)) as Array<Record<string, unknown>>;
expect(out!.type).toBe('function_call');
expect(out!.arguments).toBe('{"q":"puter"}');
});
it('rewrites tool_result blocks into function_call_output', async () => {
const messages: Array<Record<string, unknown>> = [
{
@@ -888,6 +933,35 @@ describe('splitCachedPrompt', () => {
cached_tokens: 0,
});
});
it('takes cache writes out of prompt_tokens under their own key', () => {
expect(
splitCachedPrompt({
prompt_tokens: 10,
completion_tokens: 5,
prompt_tokens_details: { cached_tokens: 3, cache_write_tokens: 4 },
}),
).toEqual({
prompt_tokens: 3,
completion_tokens: 5,
cached_tokens: 3,
cache_write_tokens: 4,
});
});
it('leaves cache writes out when the upstream reports no count', () => {
expect(
splitCachedPrompt({
prompt_tokens: 10,
completion_tokens: 5,
prompt_tokens_details: { cached_tokens: 3, cache_write_tokens: null },
}),
).toEqual({
prompt_tokens: 7,
completion_tokens: 5,
cached_tokens: 3,
});
});
});
// ── recorded usage costs ────────────────────────────────────────────
@@ -182,6 +182,14 @@ const appActorFor = (user: TestUser, app: { uid: string; id: number }) =>
app: { uid: app.uid, id: app.id },
});
/** What launching the app records: the user authorizing it. */
const authorizeApp = (user: TestUser, app: { uid: string }) =>
env.server.services.permission.grantUserAppPermission(
user.actor,
app.uid,
'flag:app-is-authenticated',
);
/** One `/drivers/call` to the KV driver, the way the SDK makes it. */
const drive = async (
token: string,
@@ -922,6 +930,8 @@ describe('an app using a handle', () => {
beforeAll(async () => {
app = await makeApp();
otherApp = await makeApp();
await authorizeApp(guest, app);
await authorizeApp(guest, otherApp);
});
it('works in the app whose data it is, and in the holder`s own session', async () => {
@@ -1025,6 +1035,7 @@ describe('a token using a handle', () => {
);
const inApp = await share(appPrefix, 'read', { appUid: app.uid });
await share(appElsewhere, 'read', { appUid: app.uid });
await authorizeApp(guest, app);
const appActor = appActorFor(guest, app);
const carrying = await tokenActor(
@@ -1407,3 +1418,250 @@ describe('what a holder is ever shown', () => {
}
});
});
describe('a handle id', () => {
/** Statements against the handle table while `call` runs. */
const handleReads = async (call: () => Promise<unknown>) => {
const read = vi.spyOn(env.server.clients.db, 'read');
const pread = vi.spyOn(env.server.clients.db, 'pread');
const touches = (spy: typeof read) =>
spy.mock.calls.filter(([sql]) =>
String(sql).includes('kv_share_handles'),
).length;
try {
await call();
} catch {
// The caller asserts on the answer separately.
}
const counts = { read: touches(read), pread: touches(pread) };
read.mockRestore();
pread.mockRestore();
return counts;
};
it('is refused without a store read when its mac does not verify', async () => {
const prefix = region('forged');
await ownerSet(`${prefix}k`, 'v');
const { handle } = await share(prefix, 'read');
const [id, mac] = handle.split('.');
const forged = `${id}.${mac[0] === 'A' ? 'B' : 'A'}${mac.slice(1)}`;
await expect(guestGet(forged, 'k')).rejects.toMatchObject({
legacyCode: 'kv_handle_not_found',
});
expect(await handleReads(() => guestGet(forged, 'k'))).toEqual({
read: 0,
pread: 0,
});
await expect(
events().subscribe(guest.actor, 'kv-forged-socket', {
subject: `kv:${forged}:*`,
}),
).rejects.toMatchObject({ legacyCode: 'subject_does_not_exist' });
expect(await guestGet(handle, 'k')).toBe('v');
});
it('minted before ids were signed still works, and never falls back to the primary', async () => {
const prefix = region('legacy-id');
await ownerSet(`${prefix}k`, 'old');
const permission = kvSharePermission(
owner.uuid,
GLOBAL,
prefix,
'read',
);
await env.server.services.permission.grantUserUserPermission(
owner.actor,
guest.username,
permission,
);
const legacy = `kvh-${uuidv4()}`;
await env.server.clients.db.insert('kv_share_handles', {
handle: legacy,
owner_user_id: owner.id,
grantee_user_id: guest.id,
app_uid: GLOBAL,
key_prefix: prefix,
permission,
label: null,
created_at: Math.floor(Date.now() / 1000),
revoked_at: null,
});
expect(await guestGet(legacy, 'k')).toBe('old');
expect(
await handleReads(() => guestGet(`kvh-${uuidv4()}`, 'k')),
).toEqual({ read: 1, pread: 0 });
});
});
describe('a handle used by an app its holder removed', () => {
const revokeAll = async (appUid: string) => {
const res = await fetch(
new URL('/auth/revoke-user-app', env.apiOrigin),
{
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${guest.creds.token}`,
},
body: JSON.stringify({ app_uid: appUid, permission: '*' }),
},
);
expect(res.status, await res.text()).toBe(200);
};
it('is absent to the app until it is authorized again, and the holder keeps it', async () => {
const app = await makeApp();
const prefix = region('removed');
await as(appActorFor(owner, app), (d) =>
d.set({ key: `${prefix}k`, value: 'in app' }),
);
const { handle } = await share(prefix, 'read', { appUid: app.uid });
await authorizeApp(guest, app);
const appActor = appActorFor(guest, app);
expect(await guestGet(handle, 'k', appActor)).toBe('in app');
await revokeAll(app.uid);
await expect(guestGet(handle, 'k', appActor)).rejects.toMatchObject({
legacyCode: 'kv_handle_not_found',
});
expect(await guestGet(handle, 'k')).toBe('in app');
expect(await handles().getLive(handle)).not.toBeNull();
await authorizeApp(guest, app);
expect(await guestGet(handle, 'k', appActor)).toBe('in app');
});
it('stops delivering to the app`s live subscription', async () => {
const app = await makeApp();
const prefix = region('removed-sub');
const { handle } = await share(prefix, 'read', { appUid: app.uid });
await authorizeApp(guest, app);
const socket = 'kv-removed-socket';
const { sub } = await events().subscribe(
appActorFor(guest, app),
socket,
{ subject: `kv:${handle}:*` },
);
const writes = (key: string) =>
as(appActorFor(owner, app), (d) =>
d.set({ key: `${prefix}${key}`, value: 1 }),
);
delivered.length = 0;
await writes('a');
await vi.waitFor(
() =>
expect(delivered.some((one) => one.subId === sub.subId)).toBe(
true,
),
{ timeout: EVENTS_COALESCE_WINDOW_MS * 12, interval: 25 },
);
await revokeAll(app.uid);
delivered.length = 0;
await writes('b');
await new Promise((resolve) =>
setTimeout(resolve, EVENTS_COALESCE_WINDOW_MS * 3),
);
expect(delivered.some((one) => one.subId === sub.subId)).toBe(false);
await events().reapSocket(guest.id, socket);
});
});
describe('listing keys through a watch handle', () => {
it('returns keys relative to the handle, and refuses values', async () => {
const prefix = region('watch-keys');
await ownerSet(`${prefix}a`, 1);
await ownerSet(`${prefix}b:c`, 2);
const { handle } = await share(prefix, 'watch');
await expect(
as(guest.actor, (d) =>
d.list({ as: 'keys', includeTotal: true, ...via(handle) }),
),
).resolves.toEqual({ items: ['a', 'b:c'], total: 2 });
for (const as_ of ['entries', 'values', undefined] as const)
await expect(
as(guest.actor, (d) => d.list({ as: as_, ...via(handle) })),
).rejects.toMatchObject({ legacyCode: 'kv_handle_access_denied' });
await expect(guestGet(handle, 'a')).rejects.toMatchObject({
legacyCode: 'kv_handle_access_denied',
});
});
it('works for the owner through their own handle', async () => {
const prefix = region('watch-owner');
await ownerSet(`${prefix}a`, 1);
const { handle } = await share(prefix, 'watch');
await expect(
as(owner.actor, (d) => d.list({ as: 'keys', ...via(handle) })),
).resolves.toEqual(['a']);
});
});
describe('a handle whose owner is suspended', () => {
const suspendable = async () =>
userFor(
await createTestUser(env.server, {
username: `kv-susp-${uuidv4().slice(0, 8)}`,
password: 'pw-test-1234',
}),
);
it('cannot be subscribed to, answered as not found', async () => {
const suspendee = await suspendable();
const { handle } = await share(region('susp-sub'), 'read', {
actor: suspendee.actor,
});
await env.server.stores.user.update(suspendee.id, { suspended: 1 });
await expect(
events().subscribe(guest.actor, 'kv-susp-socket', {
subject: `kv:${handle}:*`,
}),
).rejects.toMatchObject({ legacyCode: 'subject_does_not_exist' });
});
it('stops delivering to a subscription made before the suspension', async () => {
const suspendee = await suspendable();
const prefix = region('susp-deliver');
const { handle } = await share(prefix, 'read', {
actor: suspendee.actor,
});
const socket = 'kv-susp-deliver-socket';
const { sub } = await events().subscribe(guest.actor, socket, {
subject: `kv:${handle}:*`,
});
const writes = (key: string) =>
as(suspendee.actor, (d) =>
d.set({ key: `${prefix}${key}`, value: 1 }),
);
const reached = () => delivered.some((one) => one.subId === sub.subId);
delivered.length = 0;
await writes('a');
await vi.waitFor(() => expect(reached()).toBe(true), {
timeout: EVENTS_COALESCE_WINDOW_MS * 12,
interval: 25,
});
// The cached row once it refreshes after the suspension.
const live = await handles().getLive(handle);
const getLive = vi
.spyOn(handles(), 'getLive')
.mockResolvedValue({ ...live!, ownerSuspended: true });
try {
delivered.length = 0;
await writes('b');
await new Promise((resolve) =>
setTimeout(resolve, EVENTS_COALESCE_WINDOW_MS * 3),
);
expect(reached()).toBe(false);
} finally {
getLive.mockRestore();
await events().reapSocket(guest.id, socket);
}
});
});
+5 -5
View File
@@ -83,6 +83,8 @@ interface KvOptConfig {
*/
type KvCallArgs = {
optConfig?: KvOptConfig;
/** `list` only: what the listing returns. */
as?: unknown;
expireAt?: unknown;
ttl?: unknown;
};
@@ -94,8 +96,7 @@ type KvCallArgs = {
type DriverKvOpts = KVOpts & { handle?: string; handlePrefix?: string };
type KvListResult =
| unknown[]
| { items: unknown[]; cursor?: string; total?: number };
unknown[] | { items: unknown[]; cursor?: string; total?: number };
/**
* A listing of another namespace, cross-app or through a handle: what its
@@ -404,7 +405,7 @@ export class KVStoreDriver extends PuterDriver {
if (this.config.kv?.shareHandles !== true) throw kvHandlesDisabled();
const optConfig = args.optConfig ?? {};
const level = kvShareLevelForMethod(method);
const level = kvShareLevelForMethod(method, args);
if (optConfig.appUuid || optConfig.disableSharing)
throw handleOptionInvalid(
'kv: `handle` cannot be combined with `appUuid` or `disableSharing`',
@@ -553,8 +554,7 @@ export class KVStoreDriver extends PuterDriver {
const decoded = decodeCursor(cursor);
const reverse = decoded?.reverse === true;
const position = (reverse ? decoded?.key : decoded) as
| { namespace?: unknown; key?: unknown }
| undefined;
{ namespace?: unknown; key?: unknown } | undefined;
if (position?.namespace !== listing.namespace) throw outsideRegion();
const key = this.#relativeKey(position.key, listing.prefix);
return sealCursor(
@@ -573,6 +573,24 @@ describe('SubdomainDriver.update field handling', () => {
expect(cleared.associated_app).toBeNull();
});
it('disconnects the directory when root_dir is null', async () => {
const { actor } = await makeUser();
const name = uniqueName('unlink');
await createSubdomain(actor, name);
// The shape `puter.hosting.update(name, null)` sends.
const updated = (await withActor(actor, () =>
driver.update({
id: { subdomain: name },
object: { root_dir: null },
}),
)) as Record<string, unknown>;
expect(updated.root_dir).toBeNull();
const row = await server.stores.subdomain.getBySubdomain(name);
expect(row?.root_dir_id ?? null).toBeNull();
});
it('rejects repointing root_dir at a path that does not exist', async () => {
const { actor } = await makeUser();
const created = await createSubdomain(actor, uniqueName('badroot'));
@@ -357,7 +357,10 @@ export class SubdomainDriver extends PuterDriver {
// Subdomain name is immutable — strip if provided
const patch: Record<string, unknown> = {};
if (object.root_dir !== undefined) {
// A null root_dir disconnects the directory; the site stays reserved.
if (object.root_dir === null) {
patch.root_dir_id = null;
} else if (object.root_dir !== undefined) {
const rootDirPath = expandTildePath(
String(object.root_dir),
actor.user.username,
+52
View File
@@ -468,6 +468,58 @@ describe('PuterServer host header validation — permissive modes', () => {
});
});
describe('PuterServer host header validation — custom domains with every host accepted', () => {
let server: PuterServer;
let port: number;
beforeAll(async () => {
port = await allocateEphemeralPort();
server = await setupTestServer(
{
port,
domain: 'puter.localhost',
origin: `http://puter.localhost:${port}`,
allow_all_host_values: true,
custom_domains_enabled: true,
static_hosting_domain: 'site.puter.localhost',
} as unknown as IConfig,
{ listen: true },
);
});
afterAll(async () => {
await server?.shutdown();
});
it('still hands a host outside our domains to site hosting', async () => {
const res = await rawRequest(port, '/', {
host: 'www.my-own-domain.example',
});
expect(res.status).toBe(404);
expect(res.body).toBe('Site not found');
});
it('leaves our own domain to the GUI', async () => {
const res = await rawRequest(port, '/healthcheck', {
host: `puter.localhost:${port}`,
});
expect(res.status).toBe(200);
});
it('never treats an IP or localhost as a custom domain', async () => {
for (const host of [
`127.0.0.1:${port}`,
`[::1]:${port}`,
`localhost:${port}`,
]) {
const res = await rawRequest(port, '/healthcheck', { host });
expect(res.status, host).toBe(200);
const root = await rawRequest(port, '/', { host });
expect(root.body, host).not.toBe('Site not found');
}
});
});
describe('PuterServer request parsing', () => {
let server: PuterServer;
let port: number;
+27 -9
View File
@@ -29,6 +29,7 @@ import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { parseArgs } from 'node:util';
import http from 'node:http';
import { isIP } from 'node:net';
import { puterClients } from './clients';
import { puterControllers } from './controllers';
import { createAuthProbe } from './core/http/middleware/authProbe';
@@ -597,7 +598,32 @@ export class PuterServer {
// Build the allowed-domain set from config.
this.#app.use((req, res, next) => {
const hostName = (req.headers.host ?? '')
.split(':')[0]
.trim()
.toLowerCase();
const isOwnHost =
!hostName ||
this.#getAllowedDomains().some((d) =>
PuterServer.#hostMatchesDomain(hostName, d),
);
// Accepting every host still has to tell custom domains apart, or
// they fall through to the API/GUI instead of site hosting. An IP
// or localhost is never one: load balancer health checks and
// in-cluster callers address the server that way.
if (config.allow_all_host_values) {
if (
config.custom_domains_enabled &&
!isOwnHost &&
req.path !== '/healthcheck' &&
// `[` is what an IPv6 literal's `[::1]:port` splits to.
!hostName.startsWith('[') &&
isIP(hostName) === 0 &&
hostName !== 'localhost'
) {
req.is_custom_domain = true;
}
next();
return;
}
@@ -613,15 +639,7 @@ export class PuterServer {
return;
}
const hostName = (req.headers.host ?? '')
.split(':')[0]
.trim()
.toLowerCase();
const allowed = this.#getAllowedDomains();
if (
allowed.some((d) => PuterServer.#hostMatchesDomain(hostName, d))
) {
if (isOwnHost) {
next();
return;
}
+31 -3
View File
@@ -43,6 +43,7 @@ import { Span } from '../../util/span.js';
import type { puterServices } from '../index';
import { EVENTS_WORKER_SESSION_NAME } from '../events/workerRuntime.js';
import { FULL_API_ACCESS, PERMISSION_MAX_LEN } from '../permission/consts';
import type { GrantMeta } from '../permission/PermissionService.js';
import { PuterService } from '../types';
import type {
AccessTokenPayload,
@@ -941,10 +942,37 @@ export class AuthService extends PuterService {
}
/**
* Delete the sessions of the workers an app deployed for this user, e.g.
* when the user uninstalls it. The workers themselves keep running.
* A user removing an app: every grant they gave it and the sessions of the
* workers it deployed for them go, then `app.uninstalled` tells listeners,
* which delete the app's data in the account when `deleteData` is set.
*/
async deleteWorkerSessionsForApp(
async uninstallApp(
actor: Actor,
appIdentifier: string,
{
deleteData = false,
meta,
}: { deleteData?: boolean; meta?: GrantMeta } = {},
): Promise<void> {
const appUid = await this.services.permission.revokeUserAppAll(
actor,
appIdentifier,
meta,
);
const userId = actor.user!.id as number;
await this.#deleteWorkerSessionsForApp(userId, appUid);
await this.clients.event.emitAndWait(
'app.uninstalled',
{ userId, userUuid: actor.user!.uuid!, appUid, deleteData },
{},
);
}
/**
* Delete the sessions of the workers an app deployed for this user. Their
* tokens would otherwise act as the app; the workers keep running.
*/
async #deleteWorkerSessionsForApp(
userId: number,
appUid: string,
): Promise<void> {
+31 -2
View File
@@ -92,7 +92,10 @@ import {
} from '../../stores/events/types.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import { parseKvNamespace } from '../../stores/systemKv/SystemKVStore.js';
import { awaitLater } from '../../util/concurrency.js';
import {
awaitLater,
runWithConcurrencyLimitSettled,
} from '../../util/concurrency.js';
import {
openCursor,
sealCursor,
@@ -654,6 +657,8 @@ const UNSUBSCRIBE_READS = 3;
* takes. Bounded low: the sweep is a safety net for sessions no API path could
* reach any more, not a bulk cleanup, and it runs every hour.
*/
/** A deleted account's apps settled at once. */
const DELETED_ACCOUNT_SETTLE_CONCURRENCY = 4;
const WORKER_SESSION_SWEEP_BATCH = 500;
const WORKER_SESSION_SWEEP_MAX_PAGES = 2;
@@ -1410,6 +1415,28 @@ export class EventsService extends PuterService {
}
});
// A deleted account's apps cascade away without `app.changed`, so the
// same settle runs for each here, then the owner's subscription keys go.
this.clients.event.on('user.delete', async (_key, data) => {
if (!this.enabled) return;
const settled = await runWithConcurrencyLimitSettled(
[...new Set(data?.appUids ?? [])],
DELETED_ACCOUNT_SETTLE_CONCURRENCY,
(appUid) => this.settleDeletedApp(appUid, data.user_id),
);
for (const outcome of settled)
if (outcome.status === 'rejected')
console.warn(
'[events] deleted-account app settle failed',
outcome.reason,
);
try {
await this.stores.eventSubscription.dropOwner(data.user_id);
} catch (err) {
console.warn('[events] deleted-account key drop failed', err);
}
});
// The KV store is a store, so the bus is the only seam it has to reach
// a service. Same posture as the FS hook: post-commit, and nothing here
// can fail the write that produced it.
@@ -3103,7 +3130,8 @@ export class EventsService extends PuterService {
throw unknownKvShareHandle(handle);
const owner = await this.stores.user.getById(share.ownerUserId);
if (!owner?.uuid) throw unknownKvShareHandle(handle);
if (!owner?.uuid || Boolean(Number(owner.suspended ?? 0)))
throw unknownKvShareHandle(handle);
// Watching is the lowest level, so every level answers it; storing it
// rather than the handle's own grant is what lets a level change leave
@@ -4071,6 +4099,7 @@ export class EventsService extends PuterService {
}
if (
!live ||
live.ownerSuspended ||
(live.row.granteeUserId !== row.holderUserId &&
live.row.ownerUserId !== row.holderUserId)
)
@@ -198,6 +198,39 @@ describe('an app deleted while it has events state', () => {
);
});
it("clears the rows and handlers of a deleted account's apps, and the account's subscription keys", async () => {
const app = await makeApp(userId);
await publish(app.token, {
appUid: app.uid,
name: 'ingestUpload',
source: SOURCE,
});
await subscribeBackgroundHandler(app, 'ingestUpload');
// The account itself stays: other tests here sign in as it. A made-up
// id stands in for it on the keys the delete drops.
const goneUserId = 1_000_000 + Math.floor(Math.random() * 1_000_000);
await env.server.stores.eventSubscription.markRegionWarm(goneUserId);
// What account deletion emits once the row and its apps are gone.
await env.server.clients.event.emitAndWait(
'user.delete',
{ user_id: goneUserId, appUids: [app.uid] },
{},
);
const listed = await env.server.stores.durableSubscription.listForHolder(
userId,
{ appUid: app.uid },
);
expect(listed.items).toEqual([]);
expect(await env.server.stores.eventHandler.listForApp(app.uid)).toEqual(
[],
);
expect(
await env.server.stores.eventSubscription.isRegionWarm(goneUserId),
).toBe(false);
});
it('retires every holder session for a deleted app', async () => {
const app = await makeApp(userId);
await publish(app.token, {
@@ -484,6 +484,12 @@ describe('a handle an app minted', () => {
describe('a grantee subscribing through their own app actor', () => {
beforeAll(async () => {
await delegate();
// What launching the app records for the grantee.
await permissions().grantUserAppPermission(
guest.actor,
appUid,
'flag:app-is-authenticated',
);
});
it('receives the writes when it runs as the region’s own app', async () => {
@@ -33,6 +33,7 @@ import {
EVENTS_COALESCE_WINDOW_MS,
EVENTS_KV_HANDLE_LIMIT,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_HOLDER,
EVENTS_KV_HANDLES_PER_USER,
} from '../../controllers/events/limits.js';
import { makeActor, type Actor } from '../../core/actor.js';
@@ -47,6 +48,7 @@ import type { IConfig } from '../../types.js';
import { DEFAULT_FREE_SUBSCRIPTION } from '../metering/consts.js';
import type { DeliveryEnvelope } from './EventsService.js';
import { kvSharePermission } from '../kvShare/kvShares.js';
import { seedKvShareHandles } from '../kvShare/kvShareTestUtil.js';
import { isKvHandleId, kvAnchorToken } from './subjects.js';
const BOOT_TIMEOUT_MS = 120_000;
@@ -276,42 +278,42 @@ describe('minting a handle', () => {
});
});
// Seeded rows go to `stranger`, so the per-holder cap on `guest` is not
// what trips.
const seedForStranger = (count: number, appUid: (i: number) => string) =>
seedKvShareHandles(env.server, {
ownerUserId: owner.id,
ownerUuid: owner.uuid,
granteeUserId: stranger.id,
count,
appUid,
});
it('stops at the number of handles one account may hold out', async () => {
const seeded: string[] = [];
try {
const { total: live } =
await env.server.stores.kvShareHandle.countLiveForOwner(
owner.id,
);
// Rotate namespaces so the per-account cap is what trips here — no
// namespace sitting anywhere near its own per-app cap.
// Rotate namespaces so the per-account cap is what trips.
const spread = EVENTS_KV_HANDLES_PER_APP.limit - 1;
for (let i = live; i < EVENTS_KV_HANDLES_PER_USER.limit; i++) {
const namespace = `app-ceiling-${Math.floor(i / spread)}`;
const prefix = `ceiling:${i}:`;
const row = await env.server.stores.kvShareHandle.mint({
ownerUserId: owner.id,
granteeUserId: guest.id,
appUid: namespace,
keyPrefix: prefix,
permission: kvSharePermission(owner.uuid, namespace, prefix),
});
seeded.push(row.handle);
}
await seedForStranger(
EVENTS_KV_HANDLES_PER_USER.limit - live,
(i) => `app-ceiling-${Math.floor(i / spread)}`,
);
// `mint()` defaults to `os-global`, a namespace nowhere near any
// per-app cap here.
await expect(
mint({ prefix: 'over:the:line:' }),
).rejects.toMatchObject({
legacyCode: 'events_kv_handle_limit_reached',
message: expect.stringContaining('at a time'),
});
// Retiring one frees a slot: the ceiling counts live handles, not
// the audit trail.
// Retiring one frees a slot: the ceiling counts live handles.
await env.server.clients.db.write(
'UPDATE `kv_share_handles` SET `revoked_at` = ? WHERE `handle` = ?',
[Math.floor(Date.now() / 1000), seeded[0]],
'UPDATE `kv_share_handles` SET `revoked_at` = ? WHERE `id` = ' +
'(SELECT MIN(`id`) FROM `kv_share_handles` WHERE `grantee_user_id` = ?)',
[Math.floor(Date.now() / 1000), stranger.id],
);
await expect(mint({ prefix: 'over:the:line:' })).resolves.toEqual(
expect.objectContaining({ prefix: 'over:the:line:' }),
@@ -324,31 +326,24 @@ describe('minting a handle', () => {
}
});
it('lets a paid namespace fill the account cap and then refuses other namespaces', async () => {
it('refuses a full namespace and leaves room in another', async () => {
const namespace = await registerApp();
const another = await registerApp();
try {
for (let i = 0; i < EVENTS_KV_HANDLES_PER_APP.limit; i++) {
const prefix = `perapp:${i}:`;
await env.server.stores.kvShareHandle.mint({
ownerUserId: owner.id,
granteeUserId: guest.id,
appUid: namespace,
keyPrefix: prefix,
permission: kvSharePermission(owner.uuid, namespace, prefix),
});
}
await seedForStranger(
EVENTS_KV_HANDLES_PER_APP.limit,
() => namespace,
);
await expect(
mint({ appUid: namespace, prefix: 'over:the:app:' }),
).rejects.toMatchObject({
legacyCode: 'events_kv_handle_limit_reached',
message: expect.stringContaining('for one account'),
});
await expect(
mint({ appUid: another, prefix: 'still:room:' }),
).rejects.toMatchObject({
legacyCode: 'events_kv_handle_limit_reached',
});
).resolves.toMatchObject({ appUid: another });
} finally {
await env.server.clients.db.write(
'DELETE FROM `kv_share_handles` WHERE `owner_user_id` = ? AND `app_uid` IN (?, ?)',
@@ -362,17 +357,10 @@ describe('minting a handle', () => {
const prefix = 'idempotent:cap:';
try {
const first = await mint({ appUid: namespace, prefix });
for (let i = 0; i < EVENTS_KV_HANDLES_PER_APP.limit - 1; i++) {
const filler = `filler:${i}:`;
await env.server.stores.kvShareHandle.mint({
ownerUserId: owner.id,
granteeUserId: guest.id,
appUid: namespace,
keyPrefix: filler,
permission: kvSharePermission(owner.uuid, namespace, filler),
});
}
await seedForStranger(
EVENTS_KV_HANDLES_PER_APP.limit - 1,
() => namespace,
);
const before = (
await env.server.stores.kvShareHandle.countLiveForOwner(
@@ -407,6 +395,44 @@ describe('minting a handle', () => {
}
});
it('stops at the number of handles one owner holds out to one holder', async () => {
const prefix = 'pair:existing:';
try {
const first = await mint({ prefix });
await seedKvShareHandles(env.server, {
ownerUserId: owner.id,
ownerUuid: owner.uuid,
granteeUserId: guest.id,
count: EVENTS_KV_HANDLES_PER_HOLDER - 1,
appUid: () => KV_GLOBAL_APP_KEY,
});
expect(
await env.server.stores.kvShareHandle.countLiveForOwnerAndGrantee(
owner.id,
guest.id,
),
).toBe(EVENTS_KV_HANDLES_PER_HOLDER);
await expect(mint({ prefix: 'pair:over:' })).rejects.toMatchObject(
{
legacyCode: 'events_kv_handle_limit_reached',
message: expect.stringContaining('to one user'),
},
);
// Minting the same region again takes no slot, so it isn't refused.
await expect(mint({ prefix })).resolves.toEqual(first);
// Another holder has a cap of their own.
await expect(
mint({ prefix: 'pair:over:', granteeUsername: stranger.username }),
).resolves.toMatchObject({ holderUsername: stranger.username });
} finally {
await env.server.clients.db.write(
'DELETE FROM `kv_share_handles` WHERE `owner_user_id` = ?',
[owner.id],
);
}
});
it('refuses an app its user has not delegated the region to', async () => {
const uid = `app-${uuidv4()}`;
await env.server.clients.db.write(
@@ -688,6 +714,8 @@ describe('what a plan lets an account hold', () => {
let tiered: PuterTestEnv;
let tieredUserId: number;
let tieredUuid: string;
// Seeded rows go to a third user, so the per-holder cap is not what trips.
let thirdUserId: number;
const mintTiered = async (
prefix: string,
@@ -735,6 +763,14 @@ describe('what a plan lets an account hold', () => {
);
tieredUserId = user!.id;
tieredUuid = user!.uuid as string;
const thirdName = `kv-third-${uuidv4().slice(0, 8)}`;
await createTestUser(tiered.server, {
username: thirdName,
password: 'pw-test-1234',
});
thirdUserId = (await tiered.server.stores.user.getByUsername(
thirdName,
))!.id!;
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
@@ -779,9 +815,6 @@ describe('what a plan lets an account hold', () => {
const namespace = await registerApp(tiered, tieredUserId);
const cap =
EVENTS_KV_HANDLES_PER_APP.bySubscription[DEFAULT_FREE_SUBSCRIPTION];
const grantee = await tiered.server.stores.user.getByUsername(
tiered.users.other.username,
);
// Both servers in this file share the process-wide Redis mock, so this
// user id's call budget already carries the other server's mints.
@@ -789,24 +822,19 @@ describe('what a plan lets an account hold', () => {
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${tieredUserId}`,
);
for (let i = 0; i < cap; i++)
await tiered.server.stores.kvShareHandle.mint({
ownerUserId: tieredUserId,
granteeUserId: grantee!.id,
appUid: namespace,
keyPrefix: `seeded:${i}:`,
permission: kvSharePermission(
tieredUuid,
namespace,
`seeded:${i}:`,
),
});
await seedKvShareHandles(tiered.server, {
ownerUserId: tieredUserId,
ownerUuid: tieredUuid,
granteeUserId: thirdUserId,
count: cap,
appUid: () => namespace,
});
const refused = await mintTiered('over:the:app:', namespace);
expect(refused.status).toBe(409);
expect(refused.body.code).toBe('events_kv_handle_limit_reached');
// The account itself is nowhere near its own cap (200), so a mint into
// The account itself is nowhere near its own cap, so a mint into
// another namespace still succeeds.
const accepted = await mintTiered(
'still:room:',
@@ -823,24 +851,14 @@ describe('what a plan lets an account hold', () => {
);
const cap =
EVENTS_KV_HANDLES_PER_APP.bySubscription[DEFAULT_FREE_SUBSCRIPTION];
const grantee = await tiered.server.stores.user.getByUsername(
tiered.users.other.username,
);
try {
for (let i = 0; i < cap; i++) {
const prefix = `global:${i}:`;
await tiered.server.stores.kvShareHandle.mint({
ownerUserId: tieredUserId,
granteeUserId: grantee!.id,
appUid: KV_GLOBAL_APP_KEY,
keyPrefix: prefix,
permission: kvSharePermission(
tieredUuid,
KV_GLOBAL_APP_KEY,
prefix,
),
});
}
await seedKvShareHandles(tiered.server, {
ownerUserId: tieredUserId,
ownerUuid: tieredUuid,
granteeUserId: thirdUserId,
count: cap,
appUid: () => KV_GLOBAL_APP_KEY,
});
const accepted = await mintTiered('global:still:room:');
expect(accepted.status).toBe(200);
} finally {
+38
View File
@@ -4749,6 +4749,25 @@ describe('FSService move', () => {
expect(await readBack(overwritten)).toBe('winner');
});
it('refuses to overwrite a directory the moved entry sits inside', async () => {
const outer = await fs.mkdir(user.userId, {
path: `${user.home}/Documents/nest/photos/photos`,
createMissingParents: true,
});
await writeFile(user, `${user.home}/Documents/nest/photos/photos/a.txt`, 'keep');
const nest = (await entryAt(user, '/Documents/nest'))!;
const refused = await caught(() =>
fs.move(user.userId, {
source: outer,
destinationParent: nest,
overwrite: true,
}),
);
expect(refused.statusCode).toBe(400);
expect(await entryAt(user, '/Documents/nest/photos/photos/a.txt')).toBeTruthy();
});
/**
* Reproduce an occupant arriving after the collision probe: the probe
* misses it (a concurrent move, or a replica that hasn't caught up) and the
@@ -5038,6 +5057,25 @@ describe('FSService copy', () => {
).toBe('cannot_copy_directory_into_itself');
});
it('refuses to overwrite a directory the copied entry sits inside', async () => {
const inner = await fs.mkdir(user.userId, {
path: `${user.home}/Documents/cp-nest/photos/photos`,
createMissingParents: true,
});
await writeFile(user, `${user.home}/Documents/cp-nest/photos/photos/a.txt`, 'keep');
const nest = (await entryAt(user, '/Documents/cp-nest'))!;
const refused = await caught(() =>
fs.copy(user.userId, {
source: inner,
destinationParent: nest,
overwrite: true,
}),
);
expect(refused.statusCode).toBe(400);
expect(await entryAt(user, '/Documents/cp-nest/photos/photos/a.txt')).toBeTruthy();
});
it('handles a destination collision by conflict, overwrite or dedupe', async () => {
const destination = (await entryAt(user, '/Desktop'))!;
const source = await writeFile(
+30 -12
View File
@@ -3673,18 +3673,17 @@ export class FSService extends PuterService {
updates.filter(isValid),
);
let next = 0;
return updates.map(
(update): PromiseSettledResult<FSEntry> =>
isValid(update)
? stored[next++]
: {
status: 'rejected',
reason: new HttpError(
400,
'Invalid file entry identifier for thumbnail update',
{ legacyCode: 'bad_request' },
),
},
return updates.map((update): PromiseSettledResult<FSEntry> =>
isValid(update)
? stored[next++]
: {
status: 'rejected',
reason: new HttpError(
400,
'Invalid file entry identifier for thumbnail update',
{ legacyCode: 'bad_request' },
),
},
);
}
@@ -4990,6 +4989,7 @@ export class FSService extends PuterService {
if (collision && collision.uuid !== source.uuid) {
name = await this.#resolveMoveCollision(
userId,
source,
collision,
destinationParent,
name,
@@ -5042,6 +5042,7 @@ export class FSService extends PuterService {
if (raced && raced.uuid !== source.uuid) {
name = await this.#resolveMoveCollision(
userId,
source,
raced,
destinationParent,
name,
@@ -5104,6 +5105,7 @@ export class FSService extends PuterService {
*/
async #resolveMoveCollision(
userId: number,
source: FSEntry,
occupant: FSEntry,
destinationParent: FSEntry,
name: string,
@@ -5115,6 +5117,7 @@ export class FSService extends PuterService {
return this.#findDedupedName(destinationParent, name);
}
if (input.overwrite) {
this.#assertNotReplacingAncestor(source, occupant);
await this.remove(userId, { entry: occupant, recursive: true });
return name;
}
@@ -5130,6 +5133,20 @@ export class FSService extends PuterService {
});
}
/**
* Refuse to overwrite a directory the source sits inside: removing it would
* take the source down with it.
*/
#assertNotReplacingAncestor(source: FSEntry, occupant: FSEntry): void {
if (occupant.isDir && source.path.startsWith(`${occupant.path}/`)) {
throw new HttpError(
400,
`Cannot replace \`${occupant.name}\`: it contains the item being moved or copied`,
{ legacyCode: 'bad_request' },
);
}
}
/**
* Copy an entry to a new parent. For directories, walks descendants and
* issues S3 CopyObject + DB inserts. Thumbnail URLs on entries ride along
@@ -5199,6 +5216,7 @@ export class FSService extends PuterService {
if (collision) {
if (input.overwrite) {
this.#assertNotReplacingAncestor(source, collision);
await this.remove(userId, {
entry: collision,
recursive: true,
+579 -123
View File
@@ -20,7 +20,9 @@
import {
EVENTS_KV_HANDLE_LIMIT,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_HOLDER,
EVENTS_KV_HANDLES_PER_USER,
KV_SHARE_GRANT_LIMITS,
KV_SHARE_NOTICE_LIMIT,
limitFor,
type TieredQuota,
@@ -32,16 +34,24 @@ import {
userRelatedActor,
type Actor,
} from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { checkRateLimit } from '../../core/http/middleware/rateLimit.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import {
checkRateLimit,
claimRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import {
KV_HANDLE_RETIRE_BATCH,
type FindLiveKvShareHandleInput,
type HeldKvShareHandle,
type KvShareHandle,
type KvShareHandleListOptions,
type KvShareOwnerListOptions,
type KvShareRevokedBy,
} from '../../stores/events/KvShareHandleStore.js';
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
import {
KV_GLOBAL_APP_KEY,
kvNamespace,
} from '../../stores/systemKv/SystemKVStore.js';
import type { UserRow } from '../../stores/user/UserStore.js';
import { runWithConcurrencyLimit } from '../../util/concurrency.js';
import type { PageResult } from '../../util/pagination.js';
@@ -77,6 +87,7 @@ import {
kvSharePrefixCaseConflict,
parseKvShareAccess,
parseKvShareLabel,
parseKvShareRecipient,
type KvShareAccess,
type KvShareRecipient,
} from './kvShares.js';
@@ -99,6 +110,36 @@ export interface KvShareRequest {
label?: unknown;
}
/** A share request's fields other than who it is for. */
export type KvShareMintFields = Omit<KvShareRequest, 'recipient'>;
/** A batch share request: raw recipients, parsed one by one. */
export interface KvShareBatchRequest extends KvShareMintFields {
recipients: readonly unknown[];
}
export interface KvShareMintOptions {
defaultAccess: KvShareAccess;
/** A request naming no level leaves an existing share's level. */
keepLevelUnlessNamed?: boolean;
}
type Grantee = UserRow & { id: number; username: string };
interface KvShareMintResult {
share?: KvShareView;
error?: HttpError;
/** Set while the batch resolves its recipients. */
grantee?: Grantee;
}
/** One recipient's outcome in a batch: its share, or why it was refused. */
export interface KvShareBatchItem {
recipient: unknown;
share?: KvShareView;
error?: HttpError;
}
/** A share as its owner is told about it right after making or changing it. */
export interface KvShareView {
handle: string;
@@ -185,6 +226,29 @@ const handleOutsideNamespace = (): HttpError =>
legacyCode: 'events_kv_handle_outside_namespace',
});
const handleListOutsideNamespace = (): HttpError =>
new HttpError(
403,
'An app may only list shares of its own key-value data',
{
legacyCode: 'events_kv_handle_outside_namespace',
},
);
/**
* The namespace a listing covers. An app sees its own and is refused another;
* the account sees every app's unless it names one.
*/
const listingNamespace = (
app: NonNullable<Actor['effectiveApp']> | null,
named: string | null | undefined,
): string | null => {
if (!app) return named ?? null;
if (named !== undefined && named !== null && named !== app.uid)
throw handleListOutsideNamespace();
return app.uid;
};
/**
* A temporary account holds no share handles: the grant outlives the session
* that made it, and there is no account left behind it to take one back.
@@ -198,17 +262,22 @@ const kvHandleNeedsAccount = (): HttpError =>
* Past a handle ceiling. One code for both of them — the remedy is the same,
* revoke something — with the message saying which one it was.
*/
const QUOTA_MESSAGES = {
account: (limit: number) =>
`An account may hold out ${limit} share handles at a time`,
app: (limit: number) =>
`An app may hold out ${limit} share handles for one account`,
holder: (limit: number) =>
`An account may hold out ${limit} share handles to one user`,
};
const handleQuotaReached = (
limit: number,
scope: 'account' | 'app',
scope: keyof typeof QUOTA_MESSAGES,
): HttpError =>
new HttpError(
409,
scope === 'app'
? `An app may hold out ${limit} share handles for one account`
: `An account may hold out ${limit} share handles at a time`,
{ legacyCode: 'events_kv_handle_limit_reached' },
);
new HttpError(409, QUOTA_MESSAGES[scope](limit), {
legacyCode: 'events_kv_handle_limit_reached',
});
/** An app uid a user session names, for a surface with no app of its own. */
const parseAppUid = (value: unknown): string | null => {
@@ -218,6 +287,9 @@ const parseAppUid = (value: unknown): string | null => {
return value;
};
/** Recipients of one batch looked up or minted at once. */
const MINT_RECIPIENT_CONCURRENCY = 4;
/** Retire announcements settled at once; each is one holder-index read. */
const RETIRE_SETTLE_CONCURRENCY = 8;
@@ -237,9 +309,25 @@ const SHARE_CHANGE_LOCK_MS = 10_000;
const SHARE_CHANGE_LOCK_ATTEMPTS = 20;
const SHARE_CHANGE_LOCK_RETRY_MS = 50;
/** Retired handles are kept this long for their owner's listing, then deleted. */
export const KV_SHARE_REVOKED_RETENTION_SECONDS = 30 * 24 * 60 * 60;
// Every node checks hourly; the gate lets one node through per day.
const PURGE_CHECK_MS = 60 * 60 * 1000;
const PURGE_INITIAL_DELAY_MS = 5 * 60 * 1000;
const PURGE_GATE_KEY = 'kvShare:purge:gate';
const PURGE_GATE_SECONDS = 24 * 60 * 60;
// Rows per delete, with a pause between, so replicas keep up.
const PURGE_BATCH = 500;
const PURGE_PAUSE_MS = 50;
const PURGE_MAX_BATCHES = 2_000;
/** Passes one app deletion takes, so a huge namespace cannot pin a node. */
const APP_RETIRE_MAX_PASSES = 200;
/** Items one app-data purge step deletes, and the steps one purge takes. */
const APP_DATA_PURGE_STEP = 5_000;
const APP_DATA_PURGE_MAX_STEPS = 2_000;
/** Deleted apps whose handles one query looks up. */
const APP_RETIRE_UID_CHUNK = 200;
@@ -259,7 +347,12 @@ const ownerView = (
});
export class KvShareService extends PuterService {
#purgeTimers: ReturnType<typeof setTimeout>[] = [];
#stopping = false;
override onServerStart(): void {
this.#armPurge();
// A check at one level is answered by a grant at any level above it.
this.services.permission.registerExploder(kvShareLevelsExploder());
@@ -268,11 +361,13 @@ export class KvShareService extends PuterService {
this.services.permission.registerImplicator(kvShareOwnerImplicator());
// Lets an app-under-user actor exercise a share grant its user holds,
// scoped to the app the grant's namespace names.
// scoped to the app the grant's namespace names. The answer is cached
// under the app-under-user generation, which removing the app bumps.
this.services.permission.registerImplicator(
kvShareAppDelegateImplicator({
userHolds: (actor, permission) =>
this.services.permission.check(actor, permission),
appAuthorized: (actor) => this.#appAuthorized(actor),
}),
);
@@ -292,6 +387,19 @@ export class KvShareService extends PuterService {
}
});
// Removing an app with its data takes back what the user shared out of
// it first, so no holder reads the namespace while it empties.
this.clients.event.on('app.uninstalled', async (_key, data) => {
if (!data?.deleteData || data.appUid === KV_GLOBAL_APP_KEY) return;
try {
await this.#retireOwnedInApp(data.userId, data.appUid);
} catch (err) {
console.warn('[kvShare] uninstall retire failed', err);
return;
}
void this.#purgeAppData(data.userId, data.userUuid, data.appUid);
});
// A developer's apps go with their account, without `app.changed`.
this.clients.event.on('user.delete', async (_key, data) => {
if (!data?.appUids?.length) return;
@@ -303,6 +411,124 @@ export class KvShareService extends PuterService {
});
}
override onServerPrepareShutdown(): void {
this.#stopping = true;
for (const timer of this.#purgeTimers) clearTimeout(timer);
this.#purgeTimers = [];
}
// -- Retention purge ---------------------------------------------
/**
* Delete handles retired longer ago than the retention, in batches. Returns
* the rows deleted, or null when another node holds today's gate. A missing
* row reads like a retired one everywhere a handle is resolved.
*/
async purgeRevokedHandles(): Promise<number | null> {
const claimed = await this.clients.redis.set(
PURGE_GATE_KEY,
String(Date.now()),
'EX',
PURGE_GATE_SECONDS,
'NX',
);
if (claimed !== 'OK') return null;
const cutoff =
Math.floor(Date.now() / 1000) - KV_SHARE_REVOKED_RETENTION_SECONDS;
let purged = 0;
let finished = false;
try {
for (let i = 0; i < PURGE_MAX_BATCHES; i++) {
if (this.#stopping) return purged;
const batch =
await this.stores.kvShareHandle.deleteRevokedBefore(
cutoff,
PURGE_BATCH,
);
purged += batch;
if (batch < PURGE_BATCH) break;
await new Promise((resolve) =>
setTimeout(resolve, PURGE_PAUSE_MS),
);
}
finished = true;
return purged;
} finally {
// A run cut short frees the gate, so another node picks it up on
// its next check instead of tomorrow.
if (!finished) {
try {
await this.clients.redis.del(PURGE_GATE_KEY);
} catch {
// The gate expires on its own.
}
}
}
}
/**
* Empty one app's namespace for a user in bounded steps, pausing between
* them, until it is empty or the server stops. Stops if the user authorizes
* the app again, so data written after that is kept. A failure is logged,
* not retried.
*/
async #purgeAppData(
userId: number,
userUuid: string,
appUid: string,
): Promise<void> {
const namespace = kvNamespace(userUuid, appUid);
try {
const appId = (await this.stores.app.getByUid(appUid))?.id;
for (let step = 0; step < APP_DATA_PURGE_MAX_STEPS; step++) {
if (this.#stopping) return;
if (
typeof appId === 'number' &&
(
await this.stores.permission.listUserAppPermsFromPrimary(
userId,
appId,
)
).length > 0
)
return;
const { done } = await this.stores.kv.purgeNamespace(
namespace,
{ maxItems: APP_DATA_PURGE_STEP },
);
if (done) return;
await new Promise((resolve) =>
setTimeout(resolve, PURGE_PAUSE_MS),
);
}
console.warn('[kvShare] app data purge hit its step cap');
} catch (err) {
console.warn('[kvShare] app data purge failed', err);
}
}
#armPurge(): void {
const run = async () => {
try {
const purged = await this.purgeRevokedHandles();
if (purged)
console.log(`[kvShare] purged ${purged} retired handles`);
} catch (err) {
console.warn('[kvShare] retired-handle purge failed', err);
}
};
// Jittered so a deploy does not have every node check at once.
const kick = setTimeout(
() => void run(),
PURGE_INITIAL_DELAY_MS * (0.5 + Math.random()),
);
kick.unref?.();
const timer = setInterval(() => void run(), PURGE_CHECK_MS);
timer.unref?.();
this.#purgeTimers = [kick, timer];
}
/** Whether the level a mint asks for is available on this server. */
#assertAccessEnabled(access: KvShareAccess): void {
if (access === 'watch') {
@@ -316,31 +542,68 @@ export class KvShareService extends PuterService {
/**
* Hand another user a region of this account's key-value data, to watch,
* read or write.
*
* Two writes, in this order: the grant, which is the authorization and
* carries its own `manage:` check and its own refusal to grant to yourself,
* and then the handle, which is only a name for it. A handle whose grant
* never landed would be a name for nothing; a grant whose handle never
* landed is unaddressable, since a handle is the only thing that can name
* this family.
*
* An app may mint too, bounded the way sharing bounds an app handing out
* its user's files: authority is the user's, asked of the user behind the
* actor and never of the app, and reach is what the credential itself
* holds. Here reach is structural — an app addresses one namespace — so it
* is asserted rather than worked out, and the app's own consent to delegate
* is a `manage:` grant on the region.
* read or write. One recipient's form of {@link mintKvHandles}.
*/
async mintKvHandle(
actor: Actor,
request: KvShareRequest,
options: {
defaultAccess: KvShareAccess;
/** A request naming no level leaves an existing share's level. */
keepLevelUnlessNamed?: boolean;
},
options: KvShareMintOptions,
): Promise<KvShareView> {
const { recipient, ...fields } = request;
const [item] = await this.#mintShares(
actor,
fields,
[recipient],
options,
);
if (item.error) throw item.error;
return item.share!;
}
/**
* {@link mintKvHandle} for several recipients: one region, level and label
* for all of them. Quotas and budgets are checked for the whole batch
* before anything is granted. A failure about one recipient (unknown,
* yourself, not accepting shares, at the per-holder cap) is reported on its
* item; anything else fails the call.
*/
async mintKvHandles(
actor: Actor,
request: KvShareBatchRequest,
options: KvShareMintOptions,
): Promise<KvShareBatchItem[]> {
const { recipients, ...fields } = request;
const parsed = recipients.map((raw): KvShareRecipient | HttpError => {
try {
return parseKvShareRecipient(raw);
} catch (err) {
if (isHttpError(err)) return err;
throw err;
}
});
const results = await this.#mintShares(actor, fields, parsed, options);
return results.map((result, i) => ({
recipient: recipients[i],
...result,
}));
}
/**
* The one mint path. Two writes per new share, in this order: the grant,
* which is the authorization, and then the handle, which is only a name for
* it. A handle whose grant never landed would be a name for nothing; a
* grant whose handle never landed is unaddressable.
*
* An app may mint too: authority is the user's, asked of the user behind
* the actor, and the app's own consent to delegate is a `manage:` grant on
* the region.
*/
async #mintShares(
actor: Actor,
request: KvShareMintFields,
recipients: readonly (KvShareRecipient | HttpError)[],
options: KvShareMintOptions,
): Promise<KvShareMintResult[]> {
const access = parseKvShareAccess(
request.access,
options.defaultAccess,
@@ -350,18 +613,16 @@ export class KvShareService extends PuterService {
const owner = actor.user;
if (!owner?.uuid || owner.id === undefined) throw handleOwnerOnly();
// `undefined` is an app that could not be resolved, not the absence of
// one — reading it as an account session is what would hand an app a
// surface bounded on the app it is acting as.
// one.
const app = actor.effectiveApp;
if (app === undefined) throw handleOwnerOnly();
// An account's surface is the account's, and a token scoped to less
// than its issuer holds is not the account. A token an app issued is
// refused by the delegation check below, which has its own answer.
// A token scoped to less than its issuer holds is not the account. A
// token an app issued is refused by the delegation check below.
if (app === null && !isAccountContext(actor)) throw handleOwnerOnly();
// The budget is the user's, so an app spends its user's slots rather
// than a machine-rate allowance of its own.
await this.#spendHandleBudget(owner.id);
const ownerUserId = owner.id;
// The budget is the user's, so an app spends its user's slots.
await this.#spendHandleBudget(ownerUserId);
const keyPrefix = assertShareablePrefix(request.prefix);
const label = parseKvShareLabel(request.label);
@@ -372,67 +633,200 @@ export class KvShareService extends PuterService {
keyPrefix,
access,
);
// Before the recipient lookup and the idempotent answer below, so an
// app without the consent learns neither whether a username exists nor
// a handle it may no longer hand out.
// Before any recipient lookup, so an app without the consent learns
// neither whether a username exists nor a handle it may no longer
// hand out.
if (app) await this.#assertKvShareDelegated(actor, permission);
const grantee = await this.#resolveGrantee(request.recipient);
// One unit per named recipient, before any lookup, so a batch is no
// cheaper way to probe usernames than single calls.
const named = recipients.filter((r) => !isHttpError(r)).length;
await this.#spendGrantBudget(ownerUserId, named);
const results: KvShareMintResult[] = recipients.map((r) =>
isHttpError(r) ? { error: r } : {},
);
const grantees = new Map<number, number>();
const plans: Array<{
index: number;
grantee: Grantee;
existing: KvShareHandle | null;
}> = [];
const duplicates: Array<[number, number]> = [];
await runWithConcurrencyLimit(
recipients.map((r, index) => ({ r, index })),
MINT_RECIPIENT_CONCURRENCY,
async ({ r, index }) => {
if (isHttpError(r)) return;
try {
results[index].grantee = await this.#resolveGrantee(r);
} catch (err) {
if (!isHttpError(err)) throw err;
results[index].error = err;
}
},
);
for (let index = 0; index < results.length; index++) {
const grantee = results[index].grantee;
delete results[index].grantee;
if (!grantee) continue;
const first = grantees.get(grantee.id);
if (first !== undefined) {
duplicates.push([index, first]);
continue;
}
grantees.set(grantee.id, index);
plans.push({ index, grantee, existing: null });
}
// Resolved before the idempotent answer below, so a plan that holds
// none is refused whatever it asks for. Kept behind the delegation
// check above: an app with no delegation must not learn anything
// about the account's plan.
// none is refused whatever it asks for. Behind the delegation check: an
// app with no delegation must not learn anything about the plan.
const quota = await this.#handleQuota(actor);
// The same region minted again for the same grantee is the same
// capability, not a second one — handing back what already exists is
// what keeps two handles from ever sharing one permission row. A
// different level or label changes it in place. The ceiling counts
// slots, and this takes none, so it is not asked.
const region = {
ownerUserId: owner.id,
// capability: handing back what exists keeps two handles from ever
// sharing one permission row. It takes no slot, so no cap is asked.
const regionFor = (grantee: Grantee) => ({
ownerUserId,
granteeUserId: grantee.id,
appUid,
keyPrefix,
};
const existing = await this.stores.kvShareHandle.findLive(region);
if (existing) {
const keepLevel =
options.keepLevelUnlessNamed &&
(request.access === undefined || request.access === null);
const changed = await this.#changeShare(actor, existing, grantee, {
permission: keepLevel ? existing.permission : permission,
label,
});
return this.#view(changed, grantee.username);
}
});
await runWithConcurrencyLimit(
plans,
MINT_RECIPIENT_CONCURRENCY,
async (plan) => {
plan.existing = await this.stores.kvShareHandle.findLive(
regionFor(plan.grantee),
);
},
);
// New shares only, so ones made before these rules can still be
// narrowed or taken back.
const added = plans.filter((plan) => !plan.existing);
if (added.length > 0)
assertNewShareSegments(keyPrefixSegments(keyPrefix));
await runWithConcurrencyLimit(
added,
MINT_RECIPIENT_CONCURRENCY,
async (plan) => {
try {
await this.#assertNewShareFor(
ownerUserId,
plan.grantee,
regionFor(plan.grantee),
);
} catch (err) {
if (!isHttpError(err)) throw err;
results[plan.index].error = err;
}
},
);
const fresh = plans.filter(
(plan) => !plan.existing && !results[plan.index].error,
);
await this.#assertHandleCeiling(
ownerUserId,
appUid,
quota,
fresh.length,
);
// New shares only, so ones made before these rules or before a block
// can still be narrowed or taken back. Behind the delegation check,
// so they are no probe for who blocked whom.
assertNewShareSegments(keyPrefixSegments(keyPrefix));
const keepLevel =
options.keepLevelUnlessNamed &&
(request.access === undefined || request.access === null);
// No lock spans the batch: each recipient is its own region, a level
// change takes its handle's own lock, and the caps above were checked
// for the whole batch.
await runWithConcurrencyLimit(
plans.filter((plan) => !results[plan.index].error),
MINT_RECIPIENT_CONCURRENCY,
async ({ index, grantee, existing }) => {
try {
const row = existing
? await this.#changeShare(actor, existing, grantee, {
permission: keepLevel
? existing.permission
: permission,
label,
})
: await this.#grantAndMint(actor, grantee, {
ownerUserId,
appUid,
keyPrefix,
permission,
label: label ?? null,
});
results[index].share = this.#view(row, grantee.username);
} catch (err) {
if (!isHttpError(err)) throw err;
results[index].error = err;
}
},
);
for (const [index, first] of duplicates)
results[index] = { ...results[first] };
return results;
}
/**
* What a new share to this holder must pass beyond the caps: not yourself,
* no case-variant prefix, not blocked, and under the per-holder cap.
*/
async #assertNewShareFor(
ownerUserId: number,
grantee: Grantee,
region: FindLiveKvShareHandleInput,
): Promise<void> {
if (grantee.id === ownerUserId)
throw badRequest(
'cannot grant permissions to yourself',
'bad_request',
);
if (await this.stores.kvShareHandle.hasLiveCaseVariant(region))
throw kvSharePrefixCaseConflict();
if (await refusesSharesFrom(this.stores.userBlock, grantee, owner.id))
if (
await refusesSharesFrom(this.stores.userBlock, grantee, ownerUserId)
)
throw recipientNotAcceptingShares();
const held =
await this.stores.kvShareHandle.countLiveForOwnerAndGrantee(
ownerUserId,
grantee.id,
);
if (held >= EVENTS_KV_HANDLES_PER_HOLDER)
throw handleQuotaReached(EVENTS_KV_HANDLES_PER_HOLDER, 'holder');
}
await this.#assertHandleCeiling(owner.id, appUid, quota);
// The user behind the app, so the authority checked is theirs and the
// issuer recorded is them. Which app acted is carried separately: it
// belongs on the audit row, not in the grant's authority.
/** One new share: the grant, then its handle, then the holder's notice. */
async #grantAndMint(
actor: Actor,
grantee: Grantee,
share: {
ownerUserId: number;
appUid: string;
keyPrefix: string;
permission: string;
label: string | null;
},
): Promise<KvShareHandle> {
// The user behind the app, so the authority checked is theirs. Which
// app acted goes on the audit row, not in the grant's authority.
const authority = userRelatedActor(actor);
const meta = { reason: 'kv share handle', appUid: app?.uid };
const meta = {
reason: 'kv share handle',
appUid: actor.effectiveApp?.uid,
};
// No live handle names this region, so a grant on exactly it at
// another level is left over from a retired one and must not answer
// for the new share. Looked up first, since there rarely is one.
const stale = kvShareGrantAtEveryLevel(permission).filter(
(other) => other !== permission,
// another level is left over from a retired one.
const stale = kvShareGrantAtEveryLevel(share.permission).filter(
(other) => other !== share.permission,
);
if (
await this.services.permission.hasUserUserGrant(
grantee.id,
owner.id,
share.ownerUserId,
stale,
)
)
@@ -445,21 +839,21 @@ export class KvShareService extends PuterService {
await this.services.permission.grantUserUserPermission(
authority,
grantee.username,
permission,
share.permission,
{},
meta,
);
const row = await this.stores.kvShareHandle.mint({
ownerUserId: owner.id,
ownerUserId: share.ownerUserId,
granteeUserId: grantee.id,
appUid,
keyPrefix,
permission,
label: label ?? null,
appUid: share.appUid,
keyPrefix: share.keyPrefix,
permission: share.permission,
label: share.label,
});
await this.#notifyReceived(row);
return this.#view(row, grantee.username);
return row;
}
/**
@@ -624,7 +1018,7 @@ export class KvShareService extends PuterService {
*/
async #mintNamespace(
app: Actor['effectiveApp'],
request: KvShareRequest,
request: KvShareMintFields,
): Promise<string> {
const named = parseAppUid(request.appUid);
if (app) {
@@ -729,6 +1123,16 @@ export class KvShareService extends PuterService {
throw unknownKvShareHandle(named);
}
/** Whether the actor's user still has its app authorized. */
async #appAuthorized(actor: Actor): Promise<boolean> {
const userId = actor.user?.id;
const app = actor.effectiveApp;
if (typeof userId !== 'number' || !app) return false;
const appId = app.id ?? (await this.stores.app.getByUid(app.uid))?.id;
if (typeof appId !== 'number') return false;
return this.stores.permission.hasAnyUserAppPerm(userId, appId);
}
/**
* The account, or an app acting for it. An unresolved app (`undefined`) and
* a token an app issued are neither.
@@ -891,9 +1295,7 @@ export class KvShareService extends PuterService {
/**
* Retire every live handle in deleted apps' namespaces as the platform's,
* in bounded passes. Grants go before handles, so a retry after a failure
* still finds them, and a later share of the same region can't inherit
* them.
* in bounded passes.
*/
async retireForDeletedApps(appUids: readonly string[]): Promise<number> {
const uids = [...new Set(appUids)].filter(
@@ -902,18 +1304,52 @@ export class KvShareService extends PuterService {
let retired = 0;
for (let at = 0; at < uids.length; at += APP_RETIRE_UID_CHUNK) {
const chunk = uids.slice(at, at + APP_RETIRE_UID_CHUNK);
for (let pass = 0; pass < APP_RETIRE_MAX_PASSES; pass++) {
const batch =
await this.stores.kvShareHandle.listLiveForApps(chunk);
await this.#withdrawRetiredGrants(batch);
await this.stores.kvShareHandle.retireMany(
batch.map((row) => row.handle),
'system',
);
await this.#announceRetired(batch);
retired += batch.length;
if (batch.length < KV_HANDLE_RETIRE_BATCH) break;
}
retired += await this.#retireAsSystem(
() => this.stores.kvShareHandle.listLiveForApps(chunk),
'kv share app deleted',
);
}
return retired;
}
/**
* Retire the live handles one user holds out in one app's namespace, as the
* platform's. Handles shared with the user are left alone.
*/
async #retireOwnedInApp(
ownerUserId: number,
appUid: string,
): Promise<number> {
return this.#retireAsSystem(
() =>
this.stores.kvShareHandle.listLiveForOwnerApp(
ownerUserId,
appUid,
),
'kv share app data deleted',
);
}
/**
* Retire what `nextBatch` lists until a batch comes back short. Grants go
* before handles, so a retry after a failure still finds them, and a later
* share of the same region can't inherit them.
*/
async #retireAsSystem(
nextBatch: () => Promise<KvShareHandle[]>,
reason: string,
): Promise<number> {
let retired = 0;
for (let pass = 0; pass < APP_RETIRE_MAX_PASSES; pass++) {
const batch = await nextBatch();
await this.#withdrawRetiredGrants(batch, reason);
await this.stores.kvShareHandle.retireMany(
batch.map((row) => row.handle),
'system',
);
await this.#announceRetired(batch);
retired += batch.length;
if (batch.length < KV_HANDLE_RETIRE_BATCH) break;
}
return retired;
}
@@ -924,6 +1360,7 @@ export class KvShareService extends PuterService {
*/
async #withdrawRetiredGrants(
rows: readonly KvShareHandle[],
reason: string,
): Promise<void> {
const groups = new Map<string, KvShareHandle[]>();
for (const row of rows) {
@@ -950,7 +1387,7 @@ export class KvShareService extends PuterService {
group.flatMap((row) =>
kvShareGrantAtEveryLevel(row.permission),
),
{ reason: 'kv share app deleted', appUid: group[0].appUid },
{ reason, appUid: group[0].appUid },
);
},
);
@@ -988,12 +1425,11 @@ export class KvShareService extends PuterService {
*/
async listKvHandles(
actor: Actor,
options: KvShareHandleListOptions = {},
options: KvShareOwnerListOptions = {},
): Promise<PageResult<KvShareOwnerView>> {
const owner = actor.user;
if (owner?.id === undefined) throw handleOwnerOnly();
this.#assertOwnerSurface(actor);
const app = actor.effectiveApp;
// An app sees its own namespace and nothing else; an account session
// sees across apps, which is what makes the account the surface for a
@@ -1003,7 +1439,11 @@ export class KvShareService extends PuterService {
limit: options.limit,
cursor: options.cursor,
includeTotal: options.includeTotal,
appUid: app?.uid ?? null,
appUid: listingNamespace(
actor.effectiveApp ?? null,
options.appUid,
),
state: options.state,
});
const holders = await this.stores.user.getByIds([
...new Set(page.items.map((row) => row.granteeUserId)),
@@ -1022,8 +1462,8 @@ export class KvShareService extends PuterService {
/**
* Live shares held by this account, by handle, level, app and label only.
* An app sees its own namespace; an unresolved app or a scoped token sees
* nothing.
* An app sees its own namespace, the account every app's; an unresolved app
* or a scoped token sees nothing.
*/
async listKvSharesWithMe(
actor: Actor,
@@ -1039,14 +1479,12 @@ export class KvShareService extends PuterService {
limit: options.limit,
cursor: options.cursor,
includeTotal: options.includeTotal,
appUid: app?.uid ?? null,
appUid: listingNamespace(app, options.appUid),
});
}
/** Who a mint is for. Named by username or uuid; unknown reads as absent. */
async #resolveGrantee(
recipient: KvShareRecipient,
): Promise<UserRow & { id: number; username: string }> {
async #resolveGrantee(recipient: KvShareRecipient): Promise<Grantee> {
const username =
typeof recipient?.username === 'string'
? recipient.username.trim()
@@ -1088,30 +1526,48 @@ export class KvShareService extends PuterService {
}
/**
* Whether this account may hold out another share handle, in total and in
* the namespace this mint names. Retired ones do not count: the row stays
* as the record of what was shared, not as a slot. Check-then-act: two
* mints racing past it can both proceed, which the mint budget bounds well
* enough for an abuse backstop.
* Whether this account may hold out `adding` more share handles, in total
* and in the namespace this mint names. Retired ones do not count.
* Check-then-act: two mints racing past it can both proceed, which the
* budgets bound well enough for an abuse backstop.
*/
async #assertHandleCeiling(
userId: number,
appUid: string,
quota: TieredQuota,
adding: number,
): Promise<void> {
if (adding === 0) return;
const live = await this.stores.kvShareHandle.countLiveForOwner(
userId,
appUid,
);
if (live.total >= quota.perUser)
if (live.total + adding > quota.perUser)
throw handleQuotaReached(quota.perUser, 'account');
// The global namespace is the account's own shelf rather than an
// app's, so the account cap is its only bound — which is also what
// keeps that cap reachable without naming a namespace.
if (appUid !== KV_GLOBAL_APP_KEY && live.forApp >= quota.perApp)
// app's, so the account cap is its only bound.
if (appUid !== KV_GLOBAL_APP_KEY && live.forApp + adding > quota.perApp)
throw handleQuotaReached(quota.perApp, 'app');
}
/**
* One unit per recipient from each grant window. All or nothing: a batch
* that doesn't fit is refused, and what it claimed of the window stays
* spent.
*/
async #spendGrantBudget(userId: number, units: number): Promise<void> {
if (units === 0) return;
for (const window of KV_SHARE_GRANT_LIMITS) {
const admitted = await claimRateLimit(
`${window.scope}:${userId}`,
window.limit,
window.window,
units,
);
if (admitted < units) throw tooManyCalls();
}
}
async #spendHandleBudget(userId: number): Promise<void> {
const ok = await checkRateLimit(
`${EVENTS_KV_HANDLE_LIMIT.scope}:${userId}`,
@@ -186,6 +186,14 @@ const makeApp = async () => {
return app;
};
/** What launching the app records: the user authorizing it. */
const authorizeApp = (user: TestUser, app: { uid: string }) =>
permissions().grantUserAppPermission(
user.actor,
app.uid,
'flag:app-is-authenticated',
);
const appActorFor = (user: TestUser, app: { uid: string; id: number }) =>
makeActor({
user: user.actor.user as never,
@@ -1404,6 +1412,7 @@ describe('an app checking a share through its user', () => {
it('asks its user once, however deep the region', async () => {
const app = await makeApp();
await authorizeApp(guest, app);
const actor = appActorFor(guest, app);
const permission = kvSharePermission(
owner.uuid,
@@ -1423,6 +1432,7 @@ describe('an app checking a share through its user', () => {
it('answers from the user`s grant on any ancestor, at its level or above, until it is revoked', async () => {
const app = await makeApp();
await authorizeApp(guest, app);
const actor = appActorFor(guest, app);
const prefix = region('ancestor');
const { handle } = await share(prefix, 'read', { appUid: app.uid });
@@ -0,0 +1,67 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { v4 as uuidv4 } from 'uuid';
import type { PuterServer } from '../../server.js';
import { kvSharePermission } from './kvShares.js';
const SEED_CHUNK = 500;
/**
* Insert `count` live handle rows straight into the table, many per statement,
* so a test can fill a cap of thousands. Handles are unsigned placeholders:
* they count toward quotas but don't resolve.
*/
export const seedKvShareHandles = async (
server: PuterServer,
seed: {
ownerUserId: number;
ownerUuid: string;
granteeUserId: number;
count: number;
appUid: (i: number) => string;
revokedAt?: number | null;
},
): Promise<void> => {
const now = Math.floor(Date.now() / 1000);
for (let at = 0; at < seed.count; at += SEED_CHUNK) {
const rows: unknown[][] = [];
for (let i = at; i < Math.min(seed.count, at + SEED_CHUNK); i++) {
const appUid = seed.appUid(i);
const prefix = `seeded:${uuidv4()}:`;
rows.push([
`kvh-seed-${uuidv4()}`,
seed.ownerUserId,
seed.granteeUserId,
appUid,
prefix,
kvSharePermission(seed.ownerUuid, appUid, prefix),
now,
seed.revokedAt ?? null,
]);
}
await server.clients.db.write(
'INSERT INTO `kv_share_handles` (`handle`, `owner_user_id`, ' +
'`grantee_user_id`, `app_uid`, `key_prefix`, `permission`, ' +
'`created_at`, `revoked_at`) VALUES ' +
rows.map(() => '(?, ?, ?, ?, ?, ?, ?, ?)').join(', '),
rows.flat(),
);
}
};
@@ -0,0 +1,197 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* A user removing an app, with and without its data: what happens to the
* app's key-value data in their account and to the shares made out of it.
*/
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { makeActor, type Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
const BOOT_TIMEOUT_MS = 120_000;
interface TestUser {
actor: Actor;
username: string;
id: number;
uuid: string;
}
let env: PuterTestEnv;
let owner: TestUser;
let guest: TestUser;
const userFor = async (username: string): Promise<TestUser> => {
const row = await env.server.stores.user.getByUsername(username);
return {
actor: makeActor({ user: row as never }),
username,
id: row!.id,
uuid: row!.uuid as string,
};
};
beforeAll(async () => {
env = await setupPuterTestEnv({
events: { enabled: true, kvHandles: true },
kv: { shareHandles: true },
unlimitedMetering: true,
} as IConfig);
owner = await userFor(env.users.user.username);
guest = await userFor(env.users.other.username);
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await env?.shutdown();
});
const makeApp = async () => {
const name = `kv-uninstall-${uuidv4().slice(0, 8)}`;
return env.server.stores.app.create(
{
name,
title: 'Uninstall App',
index_url: `https://${name}.example.test/index.html`,
},
{ ownerUserId: owner.id },
);
};
const share = (from: TestUser, to: TestUser, appUid: string) =>
env.server.services.kvShare.mintKvHandle(
from.actor,
{
prefix: `region-${uuidv4().slice(0, 8)}:`,
recipient: { username: to.username },
access: 'read',
appUid,
},
{ defaultAccess: 'read' },
);
const kvOpts = (user: TestUser, appUid: string) => ({
actor: user.actor,
appUuid: appUid,
});
const keysIn = async (user: TestUser, appUid: string) =>
(await env.server.stores.kv.list({ as: 'keys' }, kvOpts(user, appUid)))
.res as string[];
const uninstall = (appUid: string, deleteData: boolean) =>
runWithContext({ actor: owner.actor }, () =>
env.server.services.auth.uninstallApp(owner.actor, appUid, {
deleteData,
}),
);
const revokedBy = async (handle: string) =>
(await env.server.stores.kvShareHandle.getByHandle(handle))?.revokedBy ??
null;
describe('uninstalling an app', () => {
it('keeps the data and the shares made out of it without deleteData', async () => {
const app = await makeApp();
await env.server.stores.kv.set(
{ key: 'kept', value: 1 },
kvOpts(owner, app.uid),
);
const owned = await share(owner, guest, app.uid);
await uninstall(app.uid, false);
expect(await keysIn(owner, app.uid)).toEqual(['kept']);
expect(await revokedBy(owned.handle)).toBeNull();
});
it('with deleteData retires the shares the user made out of the app, then deletes its data', async () => {
const app = await makeApp();
const otherApp = await makeApp();
await env.server.stores.kv.batchPut(
{
items: Array.from({ length: 30 }, (_, i) => ({
key: `item-${i}`,
value: i,
})),
},
kvOpts(owner, app.uid),
);
await env.server.stores.kv.set(
{ key: 'elsewhere', value: 1 },
kvOpts(owner, otherApp.uid),
);
await env.server.stores.kv.set(
{ key: 'theirs', value: 1 },
kvOpts(guest, app.uid),
);
const owned = await share(owner, guest, app.uid);
const ownedElsewhere = await share(owner, guest, otherApp.uid);
const held = await share(guest, owner, app.uid);
await uninstall(app.uid, true);
expect(await revokedBy(owned.handle)).toBe('system');
expect(await revokedBy(ownedElsewhere.handle)).toBeNull();
expect(await revokedBy(held.handle)).toBeNull();
await vi.waitFor(async () =>
expect(await keysIn(owner, app.uid)).toEqual([]),
);
expect(await keysIn(owner, otherApp.uid)).toEqual(['elsewhere']);
expect(await keysIn(guest, app.uid)).toEqual(['theirs']);
});
it('stops deleting once the user authorizes the app again', async () => {
const app = await makeApp();
await env.server.stores.kv.set(
{ key: 'after-reinstall', value: 1 },
kvOpts(owner, app.uid),
);
await env.server.stores.permission.upsertUserAppPerm(
owner.id,
app.id as number,
'flag:app-is-authenticated',
{},
);
const purge = vi.spyOn(env.server.stores.kv, 'purgeNamespace');
try {
await env.server.clients.event.emitAndWait(
'app.uninstalled',
{
userId: owner.id,
userUuid: owner.uuid,
appUid: app.uid,
deleteData: true,
},
{},
);
// The purge runs in the background; give it a turn to act.
await new Promise((resolve) => setTimeout(resolve, 200));
expect(purge).not.toHaveBeenCalled();
} finally {
purge.mockRestore();
}
expect(await keysIn(owner, app.uid)).toEqual(['after-reinstall']);
});
});
+85 -13
View File
@@ -42,6 +42,7 @@ import {
kvShareManageNamespaceRoots,
kvShareManagePermission,
kvShareOwnerImplicator,
kvHandleIdKind,
kvSharePermission,
mintKvHandleId,
normalizeKeyPrefix,
@@ -58,6 +59,10 @@ const OWNER = '2a1b0c9d-0000-4000-8000-000000000001';
const OTHER = '2a1b0c9d-0000-4000-8000-000000000002';
const APP = 'app-1234';
const OTHER_APP = 'app-5678';
const SECRET = 'handle-test-secret';
/** The app is still authorized by its user. */
const appAuthorized = async (): Promise<boolean> => true;
const userActor = (uuid: string): Actor =>
({ user: { uuid }, effectiveApp: null }) as unknown as Actor;
@@ -218,7 +223,7 @@ describe('granted namespaces', () => {
});
it('refuses a handle, which shares the slot but is not a namespace', () => {
expect(() => assertShareableAppUid(mintKvHandleId())).toThrow(
expect(() => assertShareableAppUid(mintKvHandleId(SECRET))).toThrow(
HttpError,
);
});
@@ -298,20 +303,41 @@ describe('a delegation request', () => {
describe('handle ids', () => {
it('are told apart from an app uid in the same slot', () => {
const handle = mintKvHandleId();
const handle = mintKvHandleId(SECRET);
expect(isKvHandleId(handle)).toBe(true);
expect(isKvHandleId(APP)).toBe(false);
expect(isKvHandleId('kvh-')).toBe(false);
});
it('carry nothing about the owner', () => {
const handle = mintKvHandleId();
const handle = mintKvHandleId(SECRET);
expect(handle).not.toContain(OWNER);
expect(mintKvHandleId()).not.toBe(handle);
expect(mintKvHandleId(SECRET)).not.toBe(handle);
});
it('carry a mac that only the server secret verifies', () => {
const handle = mintKvHandleId(SECRET);
expect(handle).toMatch(/^kvh-[0-9a-f-]{36}\.[A-Za-z0-9_-]{22}$/);
expect(handle.length).toBeLessThanOrEqual(64);
expect(handle).not.toContain(':');
expect(kvHandleIdKind(handle, SECRET)).toBe('signed');
expect(kvHandleIdKind(handle, 'another-secret')).toBeNull();
const [id, mac] = handle.split('.');
const flipped = `${mac[0] === 'A' ? 'B' : 'A'}${mac.slice(1)}`;
expect(kvHandleIdKind(`${id}.${flipped}`, SECRET)).toBeNull();
expect(kvHandleIdKind(`${id}.`, SECRET)).toBeNull();
expect(kvHandleIdKind(`${id}.${mac}x`, SECRET)).toBeNull();
});
it('minted before ids were signed still read as handles', () => {
expect(kvHandleIdKind(`kvh-${uuidv4()}`, SECRET)).toBe('legacy');
expect(kvHandleIdKind('kvh-1', SECRET)).toBeNull();
expect(kvHandleIdKind(APP, SECRET)).toBeNull();
});
it('are recoverable from the subject a row stored', () => {
const handle = mintKvHandleId();
const handle = mintKvHandleId(SECRET);
expect(kvHandleFromSubject(`kv:${handle}:messages:*`)).toBe(handle);
expect(kvHandleFromSubject(`kv:${APP}:messages:*`)).toBeNull();
expect(kvHandleFromSubject(`fs:${handle}:write`)).toBeNull();
@@ -373,6 +399,7 @@ describe('the app delegate implicator', () => {
it('answers the share family, never its manage arm, nor other families', () => {
const implicator = kvShareAppDelegateImplicator({
userHolds: vi.fn(),
appAuthorized,
});
expect(implicator.matches(permission)).toBe(true);
expect(implicator.matches(`manage:${permission}`)).toBe(false);
@@ -381,7 +408,10 @@ describe('the app delegate implicator', () => {
it("grants when the actor's own app matches the namespace app and the user holds the grant", async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
const actor = appActor(OWNER, APP);
await expect(implicator.check({ actor, permission })).resolves.toEqual(
@@ -396,9 +426,22 @@ describe('the app delegate implicator', () => {
expect(calledPermission).toBe(permission);
});
it('refuses an app its user no longer has authorized, without asking the user', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized: async () => false,
});
await expect(
implicator.check({ actor: appActor(OWNER, APP), permission }),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('refuses when the user does not hold the grant', async () => {
const implicator = kvShareAppDelegateImplicator({
userHolds: vi.fn().mockResolvedValue(false),
appAuthorized,
});
await expect(
implicator.check({ actor: appActor(OWNER, APP), permission }),
@@ -407,7 +450,10 @@ describe('the app delegate implicator', () => {
it("refuses when the actor's app differs from the namespace app", async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
await expect(
implicator.check({
@@ -420,7 +466,10 @@ describe('the app delegate implicator', () => {
it('refuses a plain user actor with no app', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
await expect(
implicator.check({ actor: userActor(OWNER), permission }),
@@ -430,7 +479,10 @@ describe('the app delegate implicator', () => {
it('refuses an access-token actor', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
const actor = {
user: { uuid: OWNER },
app: { uid: APP },
@@ -445,7 +497,10 @@ describe('the app delegate implicator', () => {
it('asks the user once for a deep option and the ancestors its check already asked', async () => {
const userHolds = vi.fn().mockResolvedValue(false);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
const actor = appActor(OWNER, APP);
const memo = new Map<string, unknown>();
const deep = kvSharePermission(OWNER, APP, 'a:b:c:', 'read');
@@ -480,7 +535,10 @@ describe('the app delegate implicator', () => {
it('refuses a permission with no key segments or no app', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const implicator = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
const actor = appActor(OWNER, APP);
await expect(
@@ -618,9 +676,17 @@ describe('the level ladder exploder', () => {
});
describe('the level a driver method needs', () => {
it('is watch for a listing of keys alone', () => {
expect(kvShareLevelForMethod('list', { as: 'keys' })).toBe('watch');
});
it('is read for get and list, and write for every other mapped op', () => {
expect(kvShareLevelForMethod('get')).toBe('read');
expect(kvShareLevelForMethod('list')).toBe('read');
expect(kvShareLevelForMethod('list', { as: 'entries' })).toBe('read');
expect(kvShareLevelForMethod('list', { as: 'values' })).toBe('read');
expect(kvShareLevelForMethod('get', { as: 'keys' })).toBe('read');
expect(kvShareLevelForMethod('set', { as: 'keys' })).toBe('write');
for (const method of [
'set',
'batchPut',
@@ -817,7 +883,10 @@ describe('delegations at every level', () => {
it('are answered for the owner at every level, and never by the app delegate', () => {
const owner = kvShareOwnerImplicator();
const delegate = kvShareAppDelegateImplicator({ userHolds: vi.fn() });
const delegate = kvShareAppDelegateImplicator({
userHolds: vi.fn(),
appAuthorized,
});
for (const access of ['read', 'write'] as const) {
const permission = kvSharePermission(
OWNER,
@@ -840,7 +909,10 @@ describe('delegations at every level', () => {
it('let a holder`s app use a read or write grant only in its own namespace', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const delegate = kvShareAppDelegateImplicator({ userHolds });
const delegate = kvShareAppDelegateImplicator({
userHolds,
appAuthorized,
});
const permission = kvSharePermission(
OWNER,
APP,
+55 -6
View File
@@ -17,7 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import { createHmac, randomUUID } from 'node:crypto';
import {
type Actor,
isAppActor,
@@ -40,6 +40,7 @@ import type {
PermissionImplicator,
} from '../permission/permissionUtil.js';
import { PermissionUtil } from '../permission/permissionUtil.js';
import { secretsEqual } from '../../util/secureCompare.js';
import {
isKvHandleId,
KV_HANDLE_PREFIX,
@@ -113,8 +114,48 @@ export const KV_HANDLE_KEY_MAX_BYTES =
/** Width of the `app_uid` column the handle row stores its namespace in. */
export const KV_SHARE_APP_UID_MAX_LENGTH = KV_SHARE_HANDLE_WIDTHS.appUid.max;
export const mintKvHandleId = (): string =>
`${KV_HANDLE_PREFIX}${randomUUID()}`;
/** Bytes of HMAC a minted handle id carries after its uuid. */
const KV_HANDLE_MAC_BYTES = 16;
/** Handles minted before ids were signed: `kvh-<uuid>`. */
const LEGACY_KV_HANDLE_ID =
/^kvh-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const SIGNED_KV_HANDLE_ID =
/^(kvh-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})\.([A-Za-z0-9_-]{22})$/;
const kvHandleMac = (id: string, secret: string): string =>
createHmac('sha256', secret)
.update(`kv-share-handle:${id}`)
.digest()
.subarray(0, KV_HANDLE_MAC_BYTES)
.toString('base64url');
/**
* `kvh-<uuid>.<mac>`: the mac lets a forged id be refused before any lookup.
* Never contains `:`, since a handle sits in one segment of a `kv:` subject.
*/
export const mintKvHandleId = (secret: string): string => {
const id = `${KV_HANDLE_PREFIX}${randomUUID()}`;
return `${id}.${kvHandleMac(id, secret)}`;
};
/**
* Whether `handle` could name a handle: `signed` for a minted id whose mac
* verifies, `legacy` for an unsigned one minted before ids were signed, and
* `null` for anything else.
*/
export const kvHandleIdKind = (
handle: string,
secret: string,
): 'signed' | 'legacy' | null => {
if (LEGACY_KV_HANDLE_ID.test(handle)) return 'legacy';
const signed = SIGNED_KV_HANDLE_ID.exec(handle);
if (!signed) return null;
return secretsEqual(signed[2], kvHandleMac(signed[1], secret))
? 'signed'
: null;
};
/** The key segments a prefix contributes to a permission string. */
export const keyPrefixSegments = (keyPrefix: string): string[] =>
@@ -195,13 +236,18 @@ export const kvShareGrantCovers = (revoked: string, held: string): boolean => {
/**
* The level a key-value driver method needs through a share, or `null` for a
* method no share reaches. Derived from the cross-app op table, so a method
* missing from it fails closed.
* missing from it fails closed. Listing keys alone needs only `watch`, which
* already learns every key written in the region.
*/
export const kvShareLevelForMethod = (method: string): KvShareAccess | null => {
export const kvShareLevelForMethod = (
method: string,
args: { as?: unknown } = {},
): KvShareAccess | null => {
if (!Object.prototype.hasOwnProperty.call(APP_DATA_KV_METHOD_OPS, method))
return null;
const op = APP_DATA_KV_METHOD_OPS[method];
if (op === null) return null;
if (op === 'list' && args.as === 'keys') return 'watch';
return op === 'get' || op === 'list' ? 'read' : 'write';
};
@@ -590,7 +636,8 @@ const kvShareCheckReaches = (asked: string, option: string): boolean => {
/**
* Letting an app-under-user actor exercise a share grant its user holds, at any
* level, bounded to the app the grant's namespace names.
* level, bounded to the app the grant's namespace names, while the user still
* has the app authorized. Removing the app pauses its reach; the share stays.
*
* Only the grant itself is matched — never `manage:kv-share…` — because
* answering the manage arm here would read as the unbounded namespace-root
@@ -600,6 +647,7 @@ const kvShareCheckReaches = (asked: string, option: string): boolean => {
*/
export const kvShareAppDelegateImplicator = (deps: {
userHolds: (actor: Actor, permission: string) => Promise<boolean>;
appAuthorized: (actor: Actor) => Promise<boolean>;
}): PermissionImplicator => ({
id: 'kv-share-app-delegate',
shortcut: true,
@@ -620,6 +668,7 @@ export const kvShareAppDelegateImplicator = (deps: {
const refused = memo?.get(DELEGATE_REFUSED) as string[] | undefined;
if (refused?.some((asked) => kvShareCheckReaches(asked, permission)))
return undefined;
if (!(await deps.appAuthorized(actor))) return undefined;
if (await deps.userHolds(userRelatedActor(actor), permission))
return {};
memo?.set(DELEGATE_REFUSED, [...(refused ?? []), permission]);
@@ -1728,11 +1728,12 @@ export class PermissionService extends PuterService {
this.#announceRevoked(userId, app.uid, removedPermission);
}
/** Returns the uid of the app `appIdentifier` resolved to. */
async revokeUserAppAll(
actor: Actor,
appIdentifier: string,
meta: GrantMeta = {},
): Promise<void> {
): Promise<string> {
this.#refuseAppActor(actor);
const app = await this.#requireApp(appIdentifier);
const userId = this.#requireUserId(actor);
@@ -1755,6 +1756,7 @@ export class PermissionService extends PuterService {
);
}
this.#announceRevoked(userId, app.uid, null);
return app.uid;
}
async grantDevAppPermission(
@@ -95,6 +95,7 @@ describe('parseAppGrantablePermission', () => {
},
],
['events:background', { family: 'backgroundEvents' }],
['custom-domains:manage', { family: 'customDomains' }],
])('accepts %s', (permission, parsed) => {
expect(parse(permission)).toEqual(parsed);
});
@@ -118,6 +119,8 @@ describe('parseAppGrantablePermission', () => {
'*',
'events',
'events:background:extra',
'custom-domains',
'custom-domains:manage:extra',
// Modeless, or a mode that does not exist.
'fs',
`fs:${FILE}`,
@@ -62,7 +62,8 @@ export type AppGrantablePermission =
appUid: string;
segments: string[];
}
| { family: 'backgroundEvents' };
| { family: 'backgroundEvents' }
| { family: 'customDomains' };
/** One component as it is written canonically: `\` as `\\`, `:` as `\C`. */
const escapeComponent = (component: string): string =>
@@ -117,6 +118,9 @@ export const parseAppGrantablePermission = (
if (permission === 'events:background') {
return { family: 'backgroundEvents' };
}
if (permission === 'custom-domains:manage') {
return { family: 'customDomains' };
}
const parts = PermissionUtil.split(permission);
// Checks that run on the raw string, after this one, match only the
@@ -226,6 +226,47 @@ describe('UserAccountService', () => {
expect(flat?.permission).toBeUndefined();
});
it('clears the flat entries for grants the account held', async () => {
const holder = await seedUser();
const issuer = await seedUser();
const otherHolder = await seedUser();
const permissions = [
`zztest:held-${uuidv4()}:a:read`,
`zztest:held-${uuidv4()}:b:write`,
];
const value = (permission: string) => ({
permission,
deleted: false,
issuer_user_id: issuer.id,
});
for (const permission of permissions)
await server.stores.permission.setFlatUserPerm(
holder.id,
permission,
value(permission),
);
await server.stores.permission.setFlatUserPerm(
otherHolder.id,
permissions[0],
value(permissions[0]),
);
await server.services.userAccount.cascadeDelete(holder.id);
expect(
await server.stores.permission.getFlatUserPerms(
holder.id,
permissions,
),
).toEqual([]);
expect(
await server.stores.permission.getFlatUserPerms(
otherHolder.id,
[permissions[0]],
),
).toHaveLength(1);
});
it('deletes the account’s sites, announcing each one', async () => {
const user = await seedUser();
const slug = Math.random().toString(36).slice(2, 10);
@@ -143,6 +143,17 @@ export class UserAccountService extends PuterService {
}
}
// Grants the account held live in the flat view under its id, which
// nothing else removes.
try {
await this.stores.permission.purgeFlatUserPermsHeldBy(userId);
} catch (e) {
console.warn(
'[cascade-delete-user] held-grant flat purge failed:',
e,
);
}
// Fire-and-forget: let listeners purge external state tied to the
// account (Stripe subscriptions are cancelled immediately, without
// proration). Emitted after the row delete — listeners key off the
@@ -1024,3 +1024,28 @@ describe('the generation counter', () => {
);
});
});
describe('dropping an owner', () => {
it("removes every key kept for the owner and leaves other owners' alone", async () => {
const other = USER + 5000;
await store.add(makeSub({ subId: 'a' }));
await store.add(makeSub({ subId: 'b', token: 'f#two' }));
await store.cacheDurable([
makeSub({
subId: 'd',
token: 'f#durable',
}) as unknown as DurableSubscription,
]);
await store.markRegionWarm(USER);
await store.add(
makeSub({ subId: 'c', holderUserId: other, ownerUserId: other }),
);
await store.dropOwner(USER);
// Socket sets are the holder's, named by socket, and expire on their own.
const left = await redis.keys(`ev:*:{${USER}}*`);
expect(left.filter((key) => !key.startsWith('ev:s:'))).toEqual([]);
await expect(store.userHasAny(other)).resolves.toBe(true);
});
});
@@ -248,6 +248,8 @@ export const REMOTE_WATCH_TTL_SECONDS = 2 * 60 * 60;
/** Bounds how many times a reap re-reads the socket set racing a reanchor. */
const REAP_READS = 3;
/** Keys one owner-drop delete names. */
const OWNER_DROP_CHUNK = 500;
const subscriptionLimitReached = (): HttpError =>
new HttpError(
@@ -1168,6 +1170,30 @@ export class EventSubscriptionStore extends PuterStore {
return null;
}
/**
* Drop every key kept for one owner, for an account that is gone. Socket
* sets are keyed on the holder's sockets and expire on their own.
*/
async dropOwner(ownerUserId: number): Promise<void> {
const tokens = await this.clients.redis.smembers(
watchedKey(ownerUserId),
);
const keys = [
...tokens.map((token) => tokenKey(ownerUserId, token)),
watchedKey(ownerUserId),
generationKey(ownerUserId),
durableMapKey(ownerUserId),
durableWarmKey(ownerUserId),
sessionCountKey(ownerUserId),
remoteWatchKey(ownerUserId),
];
// One hash tag, so every chunk is a single-slot delete.
for (let at = 0; at < keys.length; at += OWNER_DROP_CHUNK)
await this.clients.redis.del(
...keys.slice(at, at + OWNER_DROP_CHUNK),
);
}
// -- Generation --------------------------------------------------
/**
@@ -18,14 +18,11 @@
*/
/**
* The store's own guard against a row wider than the column that holds it.
*
* `KvShareService.mintKvHandle` already bounds `keyPrefix` and `permission`
* well under these widths before the store ever sees them, so reaching the
* guard here means calling the store directly.
* The store against a real database: column-width guards, listings, quota
* counts and retention.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { isHttpError } from '../../core/http/HttpError.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
@@ -259,3 +256,333 @@ describe('the holder listing', () => {
]);
});
});
describe('live handles by owner and app', () => {
it('lists only the owner\'s live handles in that app, oldest first', async () => {
const appUid = 'app-owner-listing';
const first = await store().mint(
input({ appUid, keyPrefix: 'owned-a:' }),
);
const second = await store().mint(
input({ appUid, keyPrefix: 'owned-b:' }),
);
const retired = await store().mint(
input({ appUid, keyPrefix: 'owned-c:' }),
);
await store().retireMany([retired.handle], 'owner');
await store().mint(
input({ appUid: 'app-owner-listing-other', keyPrefix: 'x:' }),
);
const theirs = await store().mint(
input({
appUid,
keyPrefix: 'theirs:',
ownerUserId: granteeUserId,
granteeUserId: ownerUserId,
}),
);
const rows = await store().listLiveForOwnerApp(ownerUserId, appUid);
expect(rows.map((row) => row.handle)).toEqual([
first.handle,
second.handle,
]);
expect(
await store().listLiveForOwnerApp(ownerUserId, appUid, 1),
).toHaveLength(1);
// Other tests count the other account's live handles from zero.
await store().retireMany([theirs.handle], 'owner');
});
});
describe('the live cache across nodes', () => {
const handleReads = (spy: ReturnType<typeof vi.spyOn>) =>
spy.mock.calls.filter(([sql]) =>
String(sql).includes('kv_share_handles'),
).length;
it('tells other nodes to forget a handle it retires or changes', async () => {
const first = await store().mint(
input({ appUid: 'app-forget', keyPrefix: 'forget-a:' }),
);
const second = await store().mint(
input({ appUid: 'app-forget', keyPrefix: 'forget-b:' }),
);
const emit = vi.spyOn(env.server.clients.event, 'emit');
try {
await store().update(first.handle, ownerUserId, { label: 'x' });
await store().retireMany([first.handle, second.handle], 'owner');
const forgets = emit.mock.calls
.filter(([key]) => key === 'outer.pubsub.kv.share.forget')
.map(([, data]) => data);
expect(forgets).toEqual([
{ handles: [first.handle] },
{ handles: [first.handle, second.handle] },
]);
} finally {
emit.mockRestore();
}
});
it('drops its cached row when another node says to', async () => {
const row = await store().mint(
input({ appUid: 'app-forget', keyPrefix: 'forget-c:' }),
);
expect(await store().getLive(row.handle)).not.toBeNull();
// Retired behind this node's back, as another node would.
await env.server.clients.db.write(
'UPDATE `kv_share_handles` SET `revoked_at` = 1 WHERE `handle` = ?',
[row.handle],
);
expect(await store().getLive(row.handle)).not.toBeNull();
// A local emit is ignored: the sender already forgot.
env.server.clients.event.emit(
'outer.pubsub.kv.share.forget',
{ handles: [row.handle] },
{},
);
expect(await store().getLive(row.handle)).not.toBeNull();
env.server.clients.event.emit(
'outer.pubsub.kv.share.forget',
{ handles: [row.handle] },
{ from_outside: true },
);
await vi.waitFor(async () =>
expect(await store().getLive(row.handle)).toBeNull(),
);
});
it('drops an owner\'s cached rows when the owner is suspended, here and on other nodes', async () => {
const row = await store().mint(
input({ appUid: 'app-forget', keyPrefix: 'forget-owner:' }),
);
expect((await store().getLive(row.handle))?.ownerSuspended).toBe(
false,
);
const setSuspended = (value: number) =>
env.server.clients.db.write(
'UPDATE `user` SET `suspended` = ? WHERE `id` = ?',
[value, ownerUserId],
);
const emit = vi.spyOn(env.server.clients.event, 'emit');
try {
await setSuspended(1);
expect((await store().getLive(row.handle))?.ownerSuspended).toBe(
false,
);
env.server.clients.event.emit(
'user.suspended',
{ userId: ownerUserId },
{},
);
expect((await store().getLive(row.handle))?.ownerSuspended).toBe(
true,
);
expect(
emit.mock.calls.filter(
([key]) => key === 'outer.pubsub.kv.share.forgetOwner',
),
).toEqual([
[
'outer.pubsub.kv.share.forgetOwner',
{ ownerUserId },
{},
],
]);
// Another node lifting the suspension reaches this one's cache.
await setSuspended(0);
env.server.clients.event.emit(
'outer.pubsub.kv.share.forgetOwner',
{ ownerUserId },
{ from_outside: true },
);
expect((await store().getLive(row.handle))?.ownerSuspended).toBe(
false,
);
} finally {
emit.mockRestore();
await setSuspended(0);
}
});
it('drops an owner\'s cached rows when the user store suspends or unsuspends them', async () => {
const row = await store().mint(
input({ appUid: 'app-forget', keyPrefix: 'forget-store:' }),
);
expect((await store().getLive(row.handle))?.ownerSuspended).toBe(
false,
);
try {
await env.server.stores.user.update(ownerUserId, {
suspended: true,
});
await vi.waitFor(async () =>
expect(
(await store().getLive(row.handle))?.ownerSuspended,
).toBe(true),
);
await env.server.stores.user.update(ownerUserId, {
suspended: false,
});
await vi.waitFor(async () =>
expect(
(await store().getLive(row.handle))?.ownerSuspended,
).toBe(false),
);
} finally {
await env.server.stores.user.update(ownerUserId, {
suspended: false,
});
}
});
it('reads nothing for an id whose mac does not verify', async () => {
const row = await store().mint(
input({ appUid: 'app-forget', keyPrefix: 'forget-d:' }),
);
const [id, mac] = row.handle.split('.');
const forged = `${id}.${mac[0] === 'A' ? 'B' : 'A'}${mac.slice(1)}`;
const read = vi.spyOn(env.server.clients.db, 'read');
const pread = vi.spyOn(env.server.clients.db, 'pread');
try {
expect(await store().getLive(forged)).toBeNull();
expect(await store().getByHandle(forged)).toBeNull();
expect(handleReads(read) + handleReads(pread)).toBe(0);
} finally {
read.mockRestore();
pread.mockRestore();
}
});
});
describe('the owner listing by state', () => {
it('defaults to every handle, and narrows to live or revoked', async () => {
const appUid = 'app-listing-state';
const live = await store().mint(
input({ appUid, keyPrefix: 'state-live:' }),
);
const gone = await store().mint(
input({ appUid, keyPrefix: 'state-gone:' }),
);
await store().retireMany([gone.handle], 'owner');
const handlesIn = async (state?: 'live' | 'revoked' | 'all') =>
(
await store().listForOwner(ownerUserId, { appUid, state })
).items.map((row) => row.handle);
expect(await handlesIn()).toEqual([live.handle, gone.handle]);
expect(await handlesIn('all')).toEqual([live.handle, gone.handle]);
expect(await handlesIn('live')).toEqual([live.handle]);
expect(await handlesIn('revoked')).toEqual([gone.handle]);
expect(
(
await store().listForOwner(ownerUserId, {
appUid,
state: 'live',
includeTotal: true,
})
).total,
).toBe(1);
});
});
describe('quota counts', () => {
it('counts live handles per owner, per app and per holder, never retired ones', async () => {
const appUid = 'app-quota-counts';
const before = await store().countLiveForOwner(ownerUserId, appUid);
const pairBefore = await store().countLiveForOwnerAndGrantee(
ownerUserId,
granteeUserId,
);
await store().mint(input({ appUid, keyPrefix: 'count-a:' }));
await store().mint(
input({ appUid: 'app-quota-other', keyPrefix: 'count-b:' }),
);
const gone = await store().mint(
input({ appUid, keyPrefix: 'count-c:' }),
);
await store().retireMany([gone.handle], 'owner');
expect(await store().countLiveForOwner(ownerUserId, appUid)).toEqual({
total: before.total + 2,
forApp: before.forApp + 1,
});
expect(
await store().countLiveForOwnerAndGrantee(
ownerUserId,
granteeUserId,
),
).toBe(pairBefore + 2);
expect(
await store().countLiveForOwnerAndGrantee(
granteeUserId,
ownerUserId,
),
).toBe(0);
});
});
describe('retired handle retention', () => {
const DAY = 24 * 60 * 60;
const now = () => Math.floor(Date.now() / 1000);
const retiredAt = async (keyPrefix: string, at: number | null) => {
const row = await store().mint(
input({ appUid: 'app-retention', keyPrefix }),
);
if (at !== null)
await env.server.clients.db.write(
'UPDATE `kv_share_handles` SET `revoked_at` = ? WHERE `handle` = ?',
[at, row.handle],
);
return row.handle;
};
const exists = async (handle: string) =>
(await store().getByHandle(handle)) !== null;
it('deletes rows retired before the cutoff, a batch at a time', async () => {
const cutoff = now() - 30 * DAY;
// Rows other tests retired long ago go first.
await store().deleteRevokedBefore(cutoff, 1_000);
const old = [
await retiredAt('old-1:', cutoff - 10),
await retiredAt('old-2:', cutoff - 5),
];
const recent = await retiredAt('recent:', cutoff + 5);
const live = await retiredAt('live:', null);
expect(await store().deleteRevokedBefore(cutoff, 1)).toBe(1);
// Oldest retirement first.
expect(await exists(old[0])).toBe(false);
expect(await exists(old[1])).toBe(true);
expect(await store().deleteRevokedBefore(cutoff, 10)).toBe(1);
expect(await store().deleteRevokedBefore(cutoff, 10)).toBe(0);
expect(await exists(old[1])).toBe(false);
expect(await exists(recent)).toBe(true);
expect(await exists(live)).toBe(true);
});
it('purges once per day across nodes, keeping live and recent rows', async () => {
const kvShare = env.server.services.kvShare;
await env.server.clients.redis.del('kvShare:purge:gate');
const old = await retiredAt('purge-old:', now() - 31 * DAY);
const recent = await retiredAt('purge-recent:', now() - 29 * DAY);
const live = await retiredAt('purge-live:', null);
expect(await kvShare.purgeRevokedHandles()).toBeGreaterThanOrEqual(1);
expect(await exists(old)).toBe(false);
expect(await exists(recent)).toBe(true);
expect(await exists(live)).toBe(true);
// Another node checking the same day finds the gate held.
const again = await retiredAt('purge-again:', now() - 31 * DAY);
expect(await kvShare.purgeRevokedHandles()).toBeNull();
expect(await exists(again)).toBe(true);
});
});
+156 -10
View File
@@ -19,6 +19,7 @@
import {
KV_SHARE_ACCESS_LEVELS,
kvHandleIdKind,
kvShareRoot,
mintKvHandleId,
type KvShareAccess,
@@ -101,6 +102,15 @@ export interface KvShareHandleListOptions {
appUid?: string | null;
}
/** Which of an owner's handles a listing returns. */
export const KV_SHARE_OWNER_LIST_STATES = ['live', 'revoked', 'all'] as const;
export type KvShareOwnerListState = (typeof KV_SHARE_OWNER_LIST_STATES)[number];
export interface KvShareOwnerListOptions extends KvShareHandleListOptions {
/** Defaults to `all`. */
state?: KvShareOwnerListState;
}
export interface MintKvShareHandleInput {
ownerUserId: number;
granteeUserId: number;
@@ -120,7 +130,11 @@ export interface FindLiveKvShareHandleInput {
/** Handle cache bounds: live rows, and recently missed names. */
const LIVE_CACHE_TTL_MS = 60_000;
const MISS_CACHE_TTL_MS = 5_000;
const LIVE_CACHE_MAX_ENTRIES = 10_000;
const LIVE_CACHE_MAX_ENTRIES = 50_000;
const MISS_CACHE_MAX_ENTRIES = 10_000;
/** Handles one cross-node forget carries. */
const FORGET_BATCH = 500;
/** Rows one app-deletion retire pass takes. */
export const KV_HANDLE_RETIRE_BATCH = 500;
@@ -179,13 +193,13 @@ export class KvShareHandleStore extends PuterStore {
ttlMs: LIVE_CACHE_TTL_MS,
});
readonly #missing = new BoundedTtlMap<string, true>({
maxEntries: LIVE_CACHE_MAX_ENTRIES,
maxEntries: MISS_CACHE_MAX_ENTRIES,
ttlMs: MISS_CACHE_TTL_MS,
});
async mint(input: MintKvShareHandleInput): Promise<KvShareHandle> {
const row: KvShareHandle = {
handle: mintKvHandleId(),
handle: mintKvHandleId(this.config.jwt_secret_v2),
ownerUserId: input.ownerUserId,
granteeUserId: input.granteeUserId,
appUid: input.appUid,
@@ -218,12 +232,67 @@ export class KvShareHandleStore extends PuterStore {
return row;
}
override onServerStart(): void {
// Our own emit already forgot locally before it went out.
this.clients.event.on(
'outer.pubsub.kv.share.forget',
(_key, data, meta) => {
if (!(meta as { from_outside?: boolean })?.from_outside) return;
const handles = (data as { handles?: unknown })?.handles;
if (!Array.isArray(handles)) return;
for (const handle of handles)
if (typeof handle === 'string') this.#forget(handle);
},
);
// Cached rows carry their owner's suspension, so a change to it drops
// them here and on every other node.
const ownerChanged = (_key: string, data: { userId?: unknown }) => {
const ownerUserId = Number(data?.userId);
if (!Number.isInteger(ownerUserId) || ownerUserId <= 0) return;
this.#forgetOwner(ownerUserId);
try {
this.clients.event.emit(
'outer.pubsub.kv.share.forgetOwner',
{ ownerUserId },
{},
);
} catch {
// The cache's TTL covers a node the broadcast misses.
}
};
this.clients.event.on('user.suspended', ownerChanged);
this.clients.event.on('user.unsuspended', ownerChanged);
this.clients.event.on(
'outer.pubsub.kv.share.forgetOwner',
(_key, data, meta) => {
if (!(meta as { from_outside?: boolean })?.from_outside) return;
const ownerUserId = Number(
(data as { ownerUserId?: unknown })?.ownerUserId,
);
if (Number.isInteger(ownerUserId) && ownerUserId > 0)
this.#forgetOwner(ownerUserId);
},
);
}
/** Drop this node's cached live rows owned by one user. */
#forgetOwner(ownerUserId: number): void {
this.#live.deleteWhere(({ row }) => row.ownerUserId === ownerUserId);
}
/**
* A live handle and its owner's state, cached in-process (hits up to 60 s,
* misses 5 s). Authority is the grant, checked on every call, so a stale
* row only keeps an address alive a little longer, never widens it.
*
* A forged id is refused without a read. Only a signed id falls back to the
* primary on a replica miss: that is for read-after-mint, and legacy ids
* are no longer minted.
*/
async getLive(handle: string): Promise<LiveKvShareHandle | null> {
const kind = kvHandleIdKind(handle, this.config.jwt_secret_v2);
if (kind === null) return null;
const cached = this.#live.get(handle);
if (cached) return cached;
if (this.#missing.has(handle)) return null;
@@ -236,7 +305,7 @@ export class KvShareHandleStore extends PuterStore {
`FROM \`${TABLE}\` h JOIN \`user\` u ON u.\`id\` = h.\`owner_user_id\` ` +
'WHERE h.`handle` = ?';
let rows = await this.clients.db.read(query, [handle]);
if (rows.length === 0)
if (rows.length === 0 && kind === 'signed')
rows = await this.clients.db.pread(query, [handle]);
const found = rows[0];
@@ -254,15 +323,31 @@ export class KvShareHandleStore extends PuterStore {
return live;
}
/** Drops this node's cached copy; other nodes converge within the TTL. */
#forget(handle: string): void {
this.#live.delete(handle);
this.#missing.delete(handle);
}
/** Drops the cached copies on this node and every other one. */
#forgetEverywhere(handles: readonly string[]): void {
for (const handle of handles) this.#forget(handle);
for (let at = 0; at < handles.length; at += FORGET_BATCH) {
try {
this.clients.event.emit(
'outer.pubsub.kv.share.forget',
{ handles: handles.slice(at, at + FORGET_BATCH) },
{},
);
} catch {
// The cache's TTL covers a node the broadcast misses.
}
}
}
/**
* Live handles this owner is holding out — in total, and in the namespace a
* mint is naming. Both ceilings read off one count, over the owner index.
* mint is naming. Both ceilings read off one count, covered by the
* `(owner_user_id, revoked_at, app_uid)` index.
*/
async countLiveForOwner(
ownerUserId: number,
@@ -281,6 +366,43 @@ export class KvShareHandleStore extends PuterStore {
};
}
/** Live handles this owner holds out to one holder, over the pair index. */
async countLiveForOwnerAndGrantee(
ownerUserId: number,
granteeUserId: number,
): Promise<number> {
const [row] = await this.clients.db.pread(
`SELECT COUNT(*) AS \`total\` FROM \`${TABLE}\` ` +
'WHERE `owner_user_id` = ? AND `grantee_user_id` = ? ' +
'AND `revoked_at` IS NULL',
[ownerUserId, granteeUserId],
);
return Number(row?.total ?? 0);
}
/**
* Delete up to `limit` rows retired before `cutoff` (unix seconds) and
* report how many went. Ids first, then a delete by id, since not every
* engine takes `LIMIT` on a delete or in an `IN` subquery.
*/
async deleteRevokedBefore(cutoff: number, limit: number): Promise<number> {
const rows = await this.clients.db.pread(
`SELECT \`id\` FROM \`${TABLE}\` ` +
'WHERE `revoked_at` IS NOT NULL AND `revoked_at` < ? ' +
'ORDER BY `revoked_at` LIMIT ?',
[cutoff, limit],
);
if (rows.length === 0) return 0;
const ids = rows.map((row) => Number(row.id));
const result = await this.clients.db.write(
`DELETE FROM \`${TABLE}\` ` +
`WHERE \`id\` IN (${ids.map(() => '?').join(', ')}) ` +
'AND `revoked_at` IS NOT NULL AND `revoked_at` < ?',
[...ids, cutoff],
);
return result.affectedRows;
}
/**
* The live handle already minted over this exact region, if one exists —
* what makes minting the same `(grantee, appUid, keyPrefix)` twice hand
@@ -339,7 +461,7 @@ export class KvShareHandleStore extends PuterStore {
/**
* Every live handle this owner holds out to one grantee, across every
* region — what a revoke or an unshare walks to find the narrower handles
* that go with a wider one. Bounded by the per-account handle cap.
* that go with a wider one. Bounded by the per-holder handle cap.
*/
async listLiveForOwnerAndGrantee(
ownerUserId: number,
@@ -359,6 +481,8 @@ export class KvShareHandleStore extends PuterStore {
* as absent, or a revoked one as live.
*/
async getByHandle(handle: string): Promise<KvShareHandle | null> {
if (kvHandleIdKind(handle, this.config.jwt_secret_v2) === null)
return null;
const rows = await this.clients.db.pread(
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` WHERE \`handle\` = ?`,
[handle],
@@ -425,7 +549,7 @@ export class KvShareHandleStore extends PuterStore {
`WHERE ${where.join(' AND ')}`,
params,
);
for (const handle of unique) this.#forget(handle);
this.#forgetEverywhere(unique);
}
/**
@@ -448,6 +572,25 @@ export class KvShareHandleStore extends PuterStore {
return rows.map(toRow);
}
/**
* One batch of live handles one owner holds out in one app's namespace,
* oldest first, over the `(owner_user_id, app_uid, id)` index. The caller
* repeats until a batch comes back short.
*/
async listLiveForOwnerApp(
ownerUserId: number,
appUid: string,
limit = KV_HANDLE_RETIRE_BATCH,
): Promise<KvShareHandle[]> {
const rows = await this.clients.db.pread(
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` ` +
'WHERE `owner_user_id` = ? AND `app_uid` = ? ' +
'AND `revoked_at` IS NULL ORDER BY `id` LIMIT ?',
[ownerUserId, appUid, limit],
);
return rows.map(toRow);
}
/**
* Change a live handle's level or label in place, scoped to its owner. A
* compare-and-set on `revoked_at IS NULL`, so a handle retired meanwhile
@@ -479,7 +622,7 @@ export class KvShareHandleStore extends PuterStore {
[...params, handle, ownerUserId],
);
}
this.#forget(handle);
this.#forgetEverywhere([handle]);
const row = await this.getByHandle(handle);
return row && row.ownerUserId === ownerUserId ? row : null;
}
@@ -547,7 +690,7 @@ export class KvShareHandleStore extends PuterStore {
*/
async listForOwner(
ownerUserId: number,
options: KvShareHandleListOptions = {},
options: KvShareOwnerListOptions = {},
): Promise<PageResult<KvShareHandle>> {
const limit = clampLimit(options.limit);
const after = openIdCursor(options.cursor, this.config.jwt_secret_v2);
@@ -558,6 +701,9 @@ export class KvShareHandleStore extends PuterStore {
filter.push('`app_uid` = ?');
filterParams.push(options.appUid);
}
if (options.state === 'live') filter.push('`revoked_at` IS NULL');
if (options.state === 'revoked')
filter.push('`revoked_at` IS NOT NULL');
const where = [...filter];
const params = [...filterParams];
@@ -128,6 +128,31 @@ describe('PermissionStore', () => {
expect(values).toEqual([{ permission: present }]);
});
it("purges every one of a holder's entries across pages, and only theirs", async () => {
const holder = await makeUser();
const other = await makeUser();
const perms = Array.from(
{ length: 520 },
(_, i) => `fs:${uuidv4()}:p${i}:read`,
);
await server.stores.kv.batchPut({
items: perms.map((permission) => ({
key: PermissionUtil.join('perm', String(holder.id), permission),
value: { permission },
})),
});
await store.setFlatUserPerm(other.id, perms[0], {
permission: perms[0],
});
expect(await store.purgeFlatUserPermsHeldBy(holder.id)).toBe(520);
expect(await store.getFlatUserPerms(holder.id, perms)).toEqual([]);
expect(await store.getFlatUserPerms(other.id, [perms[0]])).toEqual([
{ permission: perms[0] },
]);
expect(await store.purgeFlatUserPermsHeldBy(holder.id)).toBe(0);
});
it('scopes entries per holder', async () => {
const a = await makeUser();
const b = await makeUser();
@@ -46,6 +46,9 @@ const AUDIT_PAGE_SIZE = 50;
const MAX_AUDIT_PAGE_SIZE = 200;
/** Flat-entry prefix scans one subtree revoke keeps in flight. */
export const FLAT_PERM_SCAN_CONCURRENCY = 4;
/** Flat entries one page of a deleted holder's purge lists, and the pages. */
const HELD_FLAT_PURGE_PAGE = 500;
const HELD_FLAT_PURGE_MAX_PAGES = 200;
/** The user-to-app grant a user's first sign-in to an app records. */
export const APP_AUTHENTICATED_FLAG = 'flag:app-is-authenticated';
@@ -205,7 +208,7 @@ export class PermissionStore extends PuterStore {
#subscribeRemoteFlatInvalidations(): void {
this.clients.event.on(
'outer.permission.flatInvalidated',
(_key, data, meta) => {
async (_key, data, meta) => {
if (!(meta as { from_outside?: boolean })?.from_outside) return;
const raw = (data as { entries?: unknown })?.entries;
if (!Array.isArray(raw)) return;
@@ -221,12 +224,14 @@ export class PermissionStore extends PuterStore {
// Guarded: a transient KV error applying a peer's delete must
// not become an unhandled rejection. The entry stays until the
// next invalidation or its TTL — same as a lost event.
this.#applyFlatUserPermDeletes(entries).catch((err) => {
try {
await this.#applyFlatUserPermDeletes(entries);
} catch (err) {
console.warn(
'[PermissionStore] failed to apply remote flat-perm deletes:',
err,
);
});
}
},
);
}
@@ -333,6 +338,40 @@ export class PermissionStore extends PuterStore {
}
}
/**
* Delete every flat entry one holder has, a page at a time. For an account
* that is gone: its SQL rows cascaded, these are keyed on its id.
*/
async purgeFlatUserPermsHeldBy(holderUserId: number): Promise<number> {
const prefix = `${PermissionUtil.join(PERM_KEY_PREFIX, String(holderUserId))}:`;
let purged = 0;
let cursor: string | undefined;
for (let page = 0; page < HELD_FLAT_PURGE_MAX_PAGES; page++) {
const { res } = await this.stores.kv.list({
as: 'keys',
keyPrefix: prefix,
limit: HELD_FLAT_PURGE_PAGE,
...(cursor ? { cursor } : {}),
});
const { items, cursor: next } = res as {
items: string[];
cursor?: string;
};
await this.delFlatUserPerms(
items.map((key) => ({
holderUserId,
permission: PermissionUtil.unescape_permission_component(
key.slice(prefix.length),
),
})),
);
purged += items.length;
if (!next) break;
cursor = next;
}
return purged;
}
/** Local half of a flat delete. Never emits, so a remote one can't loop. */
async #applyFlatUserPermDeletes(entries: FlatPermRef[]): Promise<void> {
await this.stores.kv.batchDel({
@@ -936,6 +975,16 @@ export class PermissionStore extends PuterStore {
return all.some((row) => row.permission === permission);
}
/**
* Whether this user granted this app anything at all. Every way an app gets
* a token for a user leaves a row, and removing the app takes them all, so
* this is what "still installed" reads off.
*/
async hasAnyUserAppPerm(userId: number, appId: number): Promise<boolean> {
const all = await this.#readAllUserAppPerms(userId, appId);
return all.length > 0;
}
/**
* Every row this user granted this app, read from the primary past the row
* cache — for a write that must see a grant made moments before.
@@ -349,6 +349,14 @@ describe('SystemKVStore read cache', () => {
expect(result.res).toEqual([null, null]);
});
it('serves nothing after a namespace purge', async () => {
await warm('a', 1);
await warm('b', 2);
await target.purgeNamespace(namespace, { maxItems: 10 });
const result = await target.get({ key: ['a', 'b'] }, opts);
expect(result.res).toEqual([null, null]);
});
it('serves nothing after a flush whose delete failed partway', async () => {
await warm('a', 1);
await warm('b', 2);
@@ -789,6 +789,114 @@ describe('SystemKVStore', () => {
batchDel.mockRestore();
}
});
it('reads its pages eventually consistent, as it is billed', async () => {
await target.set({ key: 'f1', value: 1 }, opts);
const query = vi.spyOn(server.clients.dynamo, 'query');
try {
await target.flush(opts);
expect(query.mock.calls.length).toBeGreaterThan(0);
for (const call of query.mock.calls) expect(call[5]).toBe(false);
} finally {
query.mockRestore();
}
});
});
describe('purgeNamespace', () => {
const appOpts = () => ({ actor, appUuid: 'app-purge-test' });
const namespaceOf = () =>
kvNamespace(actor.user!.uuid!, 'app-purge-test');
const seed = (count: number) =>
target.batchPut(
{
items: Array.from({ length: count }, (_, i) => ({
key: `p${i}`,
value: i,
})),
},
appOpts(),
);
it('deletes at most maxItems per call and reports when the namespace is empty', async () => {
await seed(5);
expect(
await target.purgeNamespace(namespaceOf(), { maxItems: 3 }),
).toEqual({ deleted: 3, done: false });
expect(
(await target.list({ as: 'keys' }, appOpts())).res,
).toHaveLength(2);
expect(
await target.purgeNamespace(namespaceOf(), { maxItems: 10 }),
).toEqual({ deleted: 2, done: true });
expect(
await target.purgeNamespace(namespaceOf(), { maxItems: 10 }),
).toEqual({ deleted: 0, done: true });
});
it('leaves the same user\'s other namespaces alone', async () => {
await seed(2);
await target.set({ key: 'global', value: 1 }, opts);
await target.purgeNamespace(namespaceOf(), { maxItems: 10 });
expect((await target.get({ key: 'global' }, opts)).res).toBe(1);
});
it('emits no mutation events', async () => {
await seed(2);
const emit = vi.spyOn(server.clients.event, 'emit');
try {
await target.purgeNamespace(namespaceOf(), { maxItems: 10 });
const keys = emit.mock.calls.map(([key]) => key);
expect(keys).not.toContain('kv.mutated');
expect(keys).not.toContain('kv.flushed');
} finally {
emit.mockRestore();
}
});
it('refuses the system namespace and anything that is not a user namespace', async () => {
for (const namespace of [
`v1:${SYSTEM_ACTOR_UUID}:${KV_GLOBAL_APP_KEY}`,
`v1:${SYSTEM_ACTOR_UUID}:${KV_GLOBAL_APP_KEY}:partition`,
`v1:${SYSTEM_ACTOR_UUID}:some-app`,
'v1:only-two',
'v2:user:app',
'',
]) {
await expect(
target.purgeNamespace(namespace, { maxItems: 10 }),
).rejects.toThrow();
}
});
it('reads its pages with strong consistency', async () => {
await seed(2);
const query = vi.spyOn(server.clients.dynamo, 'query');
try {
await target.purgeNamespace(namespaceOf(), { maxItems: 10 });
const pageReads = query.mock.calls.filter(
([, keys]) =>
(keys as { namespace?: string }).namespace ===
namespaceOf(),
);
expect(pageReads.length).toBeGreaterThan(0);
for (const call of pageReads) expect(call[5]).toBe(true);
} finally {
query.mockRestore();
}
});
it('refuses a maxItems that is not a positive integer', async () => {
for (const maxItems of [0, -1, 1.5, Number.NaN]) {
await expect(
target.purgeNamespace(namespaceOf(), { maxItems }),
).rejects.toThrow();
}
});
});
describe('expireAt / expire', () => {
+102 -46
View File
@@ -186,6 +186,8 @@ const PATH_CLEANER_REGEX = /[^A-Za-z0-9_]/g;
// is bounded; cursors are the recommended way to page.
const MAX_LIST_OFFSET = 5000;
const MAX_FILL_PAGES = 10;
/** Items one purge query reads before deleting them. */
const PURGE_PAGE_ITEMS = 1000;
/** Most items one query reads while a foreign listing skips hidden entries. */
const MAX_HIDDEN_SKIP_READ = 1000;
// Cache keys carried by one invalidation broadcast. A peer applies them in a
@@ -2196,8 +2198,7 @@ export class SystemKVStore extends PuterStore {
settleUsage,
writeUsage(
response.ConsumedCapacity?.CapacityUnits as
| number
| undefined,
number | undefined,
),
),
};
@@ -2331,8 +2332,7 @@ export class SystemKVStore extends PuterStore {
settleUsage,
writeUsage(
(response.ConsumedCapacity?.CapacityUnits as
| number
| undefined) ?? 1,
number | undefined) ?? 1,
),
),
};
@@ -2732,48 +2732,11 @@ export class SystemKVStore extends PuterStore {
let pageKey: Record<string, unknown> | undefined;
try {
do {
const response = await this.clients.dynamo.query(
this.tableName,
{ namespace },
0,
pageKey,
);
usage = addUsage(
usage,
readUsage(
response.ConsumedCapacity?.CapacityUnits as
number | undefined,
),
);
const keys = (response.Items ?? []).map((entry) =>
String(entry.key),
);
if (keys.length > 0) {
try {
const deleted = await this.clients.dynamo.batchDel(
keys.map((key) => ({
table: this.tableName,
key: { namespace, key },
})),
);
usage = addUsage(
usage,
writeUsage(
deleted.ConsumedCapacity?.reduce(
(acc, curr) =>
acc + Number(curr.CapacityUnits ?? 0),
0,
) ?? 0,
),
);
} finally {
// A failed batch may still have deleted some of
// these, so their cached reads go either way.
await this.#invalidate(namespace, keys);
}
}
pageKey = response.LastEvaluatedKey as
Record<string, unknown> | undefined;
const page = await this.#deletePage(namespace, pageKey, {
consistent: false,
});
usage = addUsage(usage, page.usage);
pageKey = page.pageKey;
} while (pageKey);
} finally {
this.#emitMutation(opts, namespace, [], 'flush');
@@ -2782,6 +2745,99 @@ export class SystemKVStore extends PuterStore {
return { res: true, usage };
}
/**
* Delete up to `maxItems` items of one user's app namespace, as the
* platform: no mutation events and no usage. Idempotent; `done` once the
* namespace is empty.
*/
async purgeNamespace(
namespace: string,
{ maxItems }: { maxItems: number },
): Promise<{ deleted: number; done: boolean }> {
const parsed = parseKvNamespace(namespace);
if (
!parsed ||
parsed.userUuid === SYSTEM_ACTOR_UUID ||
kvNamespace(parsed.userUuid, parsed.appUid) !== namespace
)
throw new Error(`kv: not a user namespace: ${namespace}`);
if (!Number.isInteger(maxItems) || maxItems < 1)
throw new Error('kv: maxItems must be a positive integer');
let deleted = 0;
let pageKey: Record<string, unknown> | undefined;
while (deleted < maxItems) {
const page = await this.#deletePage(namespace, pageKey, {
limit: Math.min(maxItems - deleted, PURGE_PAGE_ITEMS),
consistent: true,
});
deleted += page.deleted;
pageKey = page.pageKey;
if (!pageKey) return { deleted, done: true };
}
return { deleted, done: false };
}
/**
* One page of `namespace` deleted, starting after `pageKey`. Reads only the
* keys; `pageKey` comes back while more items may remain. A purge reads
* strongly consistent, so a stale empty page can't end it early; a flush
* keeps the eventually consistent read its usage is billed on.
*/
async #deletePage(
namespace: string,
pageKey: Record<string, unknown> | undefined,
{ limit = 0, consistent }: { limit?: number; consistent: boolean },
): Promise<{
deleted: number;
usage: KVUsage;
pageKey?: Record<string, unknown>;
}> {
const response = await this.clients.dynamo.query(
this.tableName,
{ namespace },
limit,
pageKey,
'',
consistent,
{ projection: ['namespace', 'key'] },
);
let usage = readUsage(
response.ConsumedCapacity?.CapacityUnits as number | undefined,
);
const keys = (response.Items ?? []).map((entry) => String(entry.key));
if (keys.length > 0) {
try {
const deleted = await this.clients.dynamo.batchDel(
keys.map((key) => ({
table: this.tableName,
key: { namespace, key },
})),
);
usage = addUsage(
usage,
writeUsage(
deleted.ConsumedCapacity?.reduce(
(acc, curr) =>
acc + Number(curr.CapacityUnits ?? 0),
0,
) ?? 0,
),
);
} finally {
// A failed batch may still have deleted some of these, so
// their cached reads go either way.
await this.#invalidate(namespace, keys);
}
}
return {
deleted: keys.length,
usage,
pageKey: response.LastEvaluatedKey as
Record<string, unknown> | undefined,
};
}
async expireAt(
{ key, timestamp }: { key: string; timestamp: number },
opts?: KVOpts,
+7
View File
@@ -806,6 +806,13 @@ export class UserStore extends PuterStore {
} else {
await this.invalidateById(userId);
}
if (Object.hasOwn(patch, 'suspended')) {
this.clients.event.emit(
patch.suspended ? 'user.suspended' : 'user.unsuspended',
{ userId },
{},
);
}
return true;
}
+13
View File
@@ -130,4 +130,17 @@ describe('BoundedTtlMap', () => {
expect(map.shouldEmit('k')).toBe(true);
});
});
describe('deleteWhere', () => {
it('deletes only the matching entries and reports how many', () => {
const map = new BoundedTtlMap<string, number>({ maxEntries: 10 });
map.set('a', 1).set('b', 2).set('c', 3);
expect(map.deleteWhere((value) => value % 2 === 1)).toBe(2);
expect(map.has('a')).toBe(false);
expect(map.get('b')).toBe(2);
expect(map.has('c')).toBe(false);
expect(map.size).toBe(1);
});
});
});
+11
View File
@@ -78,6 +78,17 @@ export class BoundedTtlMap<K, V> {
this.#entries.clear();
}
/** Delete every entry whose value matches, in one pass. Returns how many. */
deleteWhere(matches: (value: V, key: K) => boolean): number {
let deleted = 0;
for (const [key, entry] of this.#entries) {
if (!matches(entry.value, key)) continue;
this.#entries.delete(key);
deleted++;
}
return deleted;
}
/**
* True at most once per TTL window for `key`, for "log/post this at most
* once a window" throttles. Calls inside the window return false.
+4 -2
View File
@@ -34,7 +34,7 @@ An object containing the following properties:
- `max_tokens` (Number) - The maximum number of tokens to generate in the completion. By default, the specific model's maximum is used.
- `temperature` (Number) - A number between 0 and 2 indicating the randomness of the completion. Lower values make the output more focused and deterministic, while higher values make it more random. By default, the model's own temperature is used. Some OpenAI reasoning models ignore `temperature` and `top_p`. GPT-5.6 and GPT-6 Sol/Luna accept them only with `reasoning_effort: 'none'`, and GPT-6 Astra, GPT-6.1 Sol, and GPT-5.3 Codex always ignore them. `gemini-4-argon` ignores `temperature` too. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` use it.
- `tools` (Array) (Optional) - Function definitions the AI can call. See [Function Calling](#function-calling) for details.
- `reasoning_effort` / `reasoning.effort` (String) (Optional) - Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Lower values give faster responses with less reasoning. OpenAI models, Meta's Muse Spark models, `gemini-4-argon`, `gemini-3.1-flash-lite` and `gemini-3-flash-preview` only; Muse Spark always reasons, so `none` is ignored for it. `gemini-4-argon` accepts `low`, `medium` and `high`, and uses `medium` when you leave it unset. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` accept `minimal`, `low`, `medium` and `high`, and use Google's default level when you leave it unset. For these three Gemini models, any other value fails with `400 bad_request`, and messages can contain text and images only.
- `reasoning_effort` / `reasoning.effort` (String) (Optional) - Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Lower values give faster responses with less reasoning. OpenAI models, Moonshot's Kimi K3.1 Pro models, Meta's Muse Spark models, `gemini-4-argon`, `gemini-3.1-flash-lite` and `gemini-3-flash-preview` only; Muse Spark always reasons, so `none` is ignored for it. Kimi K3.1 Pro accepts `low`, `high` and `max`, uses `max` when you leave it unset, and fails with `400 bad_request` on any other value. `gemini-4-argon` accepts `low`, `medium` and `high`, and uses `medium` when you leave it unset. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` accept `minimal`, `low`, `medium` and `high`, and use Google's default level when you leave it unset. For these three Gemini models, any other value fails with `400 bad_request`, and messages can contain text and images only.
- `verbosity` / `text.verbosity` (String) (Optional) - Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only.
- `normalize` (Boolean) (Optional) - Controls the format of the non-streaming response. When `true`, the response is normalized to the OpenAI format regardless of the model's vendor: `message.content` is a string, tool calls appear as `message.tool_calls`, and `finish_reason` is one of `stop`, `length`, `tool_calls`, or `content_filter` — or the vendor's own stop reason, passed through unchanged when it has no OpenAI equivalent. When `false`, the response keeps the vendor's native format (for Anthropic models, an array of content blocks). When unset, the SDK-wide `puter.ai.normalize` applies — itself tri-state: set it to `true` to normalize every call regardless of release date, `false` to disable normalization for every call, and when it is left unset too (the default) the release-date policy applies: **models released on or after September 1, 2026 return normalized (OpenAI-format) responses by default**, and for older models the default is unchanged — `message.content` keeps its vendor-native shape. (A handful of reasoning fields were made consistent across all models independently of this option; see [Reasoning fields on existing models](#reasoning-fields-on-existing-models).) Streaming responses are unaffected — chunks already share one format across vendors. See [Response normalization](#response-normalization).
- `compaction` (Boolean | Object) (Optional) - Opt into inline context compaction for long conversations. Pass `true` to enable it with provider defaults, or `{ trigger_tokens: number }` to set the token threshold at which earlier context is summarized. When the model compacts, you receive a `compaction` chunk while streaming (or a `compaction` field on the result when not streaming) containing an opaque `encrypted_content` summary. Resend that item in `messages` on the next turn in place of the summarized history. The compaction chunk shape is identical across providers, so the same code works whether `model` is an OpenAI or Anthropic model. See [Compaction](#compaction).
@@ -219,7 +219,7 @@ A `normalize` option on an individual call always overrides `puter.ai.normalize`
On a normalized response, extended-thinking output (from reasoning models that expose it) is joined into `message.reasoning`, and Anthropic stop reasons are mapped to OpenAI values (`end_turn` → `stop`, `max_tokens` → `length`, `tool_use` → `tool_calls`, `refusal` → `content_filter`). A vendor stop reason with no OpenAI equivalent — Anthropic's `pause_turn`, for instance — passes through unchanged, so treat `finish_reason` as an open set. See [`finish_reason`](/Objects/chatresponse) for the full mapping.
Normalization does not cost you the ability to continue a reasoning turn. The opaque parts a provider needs back — Anthropic thinking-block signatures, OpenAI reasoning item ids and encrypted content — are preserved verbatim on `message.reasoning_details`. Resend that array as-is alongside the message when you continue an extended-thinking tool-use loop. The artifacts are vendor-specific and only meaningful to the model that produced them, so replay them to the same model — don't carry them across vendors.
Normalization does not cost you the ability to continue a reasoning turn. The opaque parts a provider needs back — Anthropic thinking-block signatures, OpenAI reasoning item ids and encrypted content — are preserved verbatim on `message.reasoning_details`. Resend that array as-is alongside the message when you continue an extended-thinking tool-use loop. The artifacts are vendor-specific and only meaningful to the model that produced them, so replay them to the same model — don't carry them across vendors. Kimi K3.1 Pro reads its `reasoning` string back the same way, so keep `reasoning` on the assistant message when you continue a conversation with it.
One caveat. The release-date rule applies to the model that actually serves the request — if a request is rerouted to a fallback provider, the served model's release date decides.
@@ -300,6 +300,8 @@ Pass in the `cache_control` parameter inside the object in the `messages` array.
You can find the implementation in our [prompt caching example](/playground/ai-claude-cache-control/). Find more details about cache control in [Anthropic documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
Moonshot's Kimi K3 and Kimi K3.1 Pro models cache prompt prefixes on their own and take no `cache_control`. Their `usage` reports the tokens read from the cache as `cached_tokens` and the tokens written to it as `cache_write_tokens`. Each is billed at the model's own rate from the [AI models list](https://developer.puter.com/ai/models/).
## Compaction
For long, multi-turn conversations that you keep on the client, enable
+1 -7
View File
@@ -63,15 +63,9 @@ If any part of the upload fails, the promise is rejected — it never resolves t
When every failed item failed the same way, that `code` and `status` are also set on the rejection value itself, because the cause belongs to the request rather than to any one file. An upload that exceeds the account's storage quota is the common case: it rejects with `code: 'storage_limit_reached'` and `status: 413` however many files were in it.
When the signed batch-write endpoint is unavailable and the SDK falls back to the older batch endpoint, the rejection value also carries a stable `code`:
- `batch_upload_failed` — every operation failed, so nothing was written.
- `batch_upload_partially_failed` — some operations succeeded and others didn't. `failedCount` and `totalCount` say how many, and `results` holds every operation's result in the order they were sent.
- `batch_upload_no_results` — the request succeeded but the server didn't report what it wrote.
## Uploading directories
Directory uploads (dropped directory entries, or `createFileParent`) work on every platform: nested paths are recreated under the destination, and files sharing a name stay apart in the directories they came from. They rely on the signed batch-write endpoint, so against a backend that doesn't have one the SDK falls back to the older batch endpoint, which cannot create directories — the upload rejects with `batch_upload_failed`, and the directories have to be created with [`puter.fs.mkdir()`](/FS/mkdir/) and the files uploaded into them.
Directory uploads (dropped directory entries, or `createFileParent`) work on every platform: nested paths are recreated under the destination, and files sharing a name stay apart in the directories they came from.
## Thumbnails
+1
View File
@@ -191,6 +191,7 @@ These hosting features are supported out of the box when using Puter.js:
- **[`puter.hosting.delete()`](/Hosting/delete/)** - Delete a hosting deployment
- **[`puter.hosting.update()`](/Hosting/update/)** - Update hosting settings
- **[`puter.hosting.get()`](/Hosting/get/)** - Get information about a specific deployment
- **[`puter.hosting.attach()`](/Hosting/attach/)** - Let the user connect their own domain to a website
## Examples
+70
View File
@@ -0,0 +1,70 @@
---
title: puter.hosting.attach()
description: Let the user connect one of their own domains to a website hosted on Puter.
platforms: [websites, apps]
---
Opens a Puter dialog where the user connects a domain they own, such as `www.example.com`, to one of their websites. In the dialog, the user can pick a domain already on their account or add a new one. A domain the user picks that currently serves another of their websites moves to this one.
When the user adds a new domain, the dialog shows the DNS records to set at their DNS provider. The domain starts serving the website once those records are in place, usually within a few minutes.
Your app doesn't need any permission to call this method. The user makes the choice in the dialog, and your app only receives the domain they chose. Custom domains are billed to the user's Puter account, and the dialog shows the price before the user adds one.
Inside Puter, the dialog is rendered by the desktop environment. On a website, a puter.com popup hosts the dialog and signs the user in first if needed.
## Syntax
```js
puter.hosting.attach(subdomain)
```
## Parameters
#### `subdomain` (String) (required)
The name of the website to connect a domain to, such as `example` for `example.puter.site`. The website must belong to the signed-in user.
## Return value
A `Promise` that resolves to a [`CustomDomain`](/Objects/customdomain/) object for the domain the user chose, or `null` if the user closed the dialog without choosing one.
A newly added domain resolves with `status` set to `pending`. Its `records` list the DNS records the user still has to set.
On a website that is cross-origin isolated, the popup can't send its result back to your page, so the promise resolves `null` even if the user goes on to connect a domain in the popup.
The promise rejects with an error whose `code` is one of the following:
- `invalid_argument` when `subdomain` is missing or empty.
- `not_available` in an environment that can't show a dialog, such as Node.js or a worker.
- `not_supported` inside a version of the Puter desktop that doesn't have the dialog.
## Examples
<strong class="example-title">Connect a custom domain to a website</strong>
```html;hosting-attach
<html>
<body>
<button id="attach">Use my own domain</button>
<script src="https://js.puter.com/v2/"></script>
<script>
document.getElementById('attach').addEventListener('click', async () => {
// (1) Create a website to connect the domain to
const site = await puter.hosting.create(puter.randName(), '.');
// (2) Let the user pick or add a domain for it
const domain = await puter.hosting.attach(site.subdomain);
if (!domain) {
puter.print('No domain was chosen.');
return;
}
puter.print(`${domain.hostname} is ${domain.status}.<br>`);
for (const record of domain.records) {
puter.print(`${record.type} ${record.name} → ${record.value}<br>`);
}
});
</script>
</body>
</html>
```
+2 -2
View File
@@ -248,8 +248,8 @@ Pick the tool by what you want to do:
A share has one of three levels, and each includes the ones before it:
- **`watch`**: the holder gets [events](/Events/subjects/#sharing-key-value-data-with-another-user) when keys under the prefix change.
- **`read`**: the holder can also [`get()`](/KV/get/) and [`list()`](/KV/list/) keys.
- **`watch`**: the holder gets [events](/Events/subjects/#sharing-key-value-data-with-another-user) when keys under the prefix change, and can [`list()`](/KV/list/) the keys without their values.
- **`read`**: the holder can also read values with [`get()`](/KV/get/) and [`list()`](/KV/list/).
- **`write`**: the holder can also change keys with every other method. Write includes delete, so every holder with write access can change or remove any key under the prefix.
An app shares its user's data only with that user's consent, asked once with [`puter.perms.request('kvShare', …)`](/Perms/request/) at the level it will share or higher. Without it, [`puter.kv.share()`](/KV/share/) fails with `events_kv_handle_not_delegated`.
+2 -2
View File
@@ -39,7 +39,7 @@ If set to `true`, the returned array will contain objects with both `key` and `v
The last argument of the positional forms. It takes these properties:
- `handle` (String): List data another user shared with you, using the `handle` from [`puter.kv.listShared()`](/KV/listShared/). The share needs `read` or `write` access. The `pattern` is relative to the shared prefix and can be at most 768 bytes, and the returned keys are relative to the prefix too. The results and `total` leave out private entries. Every page of a full listing or a stream is fetched through the handle. See [Sharing with other users](/KV/#sharing-with-other-users).
- `handle` (String): List data another user shared with you, using the `handle` from [`puter.kv.listShared()`](/KV/listShared/). Listing only keys works with any access level. Listing values needs `read` or `write` access. The `pattern` is relative to the shared prefix and can be at most 768 bytes, and the returned keys are relative to the prefix too. The results and `total` leave out private entries. Every page of a full listing or a stream is fetched through the handle. See [Sharing with other users](/KV/#sharing-with-other-users).
- `appUuid` (String): List another app's data, which the user let your app reach with [`puter.perms.request('appData', …)`](/Perms/appData/). It can't be combined with `handle`.
#### `options` (Object) (optional)
@@ -73,7 +73,7 @@ Invalid options, such as a non-boolean `reverse` or `offset` combined with `stre
Without pagination params, the full listing is returned as a plain array. It reads the entire store and is metered per page, so on large stores a bare `list()` gets slow and costly. The SDK logs a one-time console warning when a full listing spans multiple pages. Use `stream: true` or `limit` and `cursor` pages instead, and narrow the scan with a `pattern`.
With `handle`, the promise can also reject with `kv_handle_not_found` (the handle doesn't exist, the share stopped, or it isn't shared with you), `kv_handle_access_denied` (the share is `watch` only), `kv_handle_option_invalid` (`handle` combined with `appUuid`), `kv_handle_key_invalid` (`pattern` is over 768 bytes), or `kv_handles_disabled` (shares can't be used this way on this deployment).
With `handle`, the promise can also reject with `kv_handle_not_found` (the handle doesn't exist, the share stopped, or it isn't shared with you), `kv_handle_access_denied` (the listing asks for values and the share is `watch` only), `kv_handle_option_invalid` (`handle` combined with `appUuid`), `kv_handle_key_invalid` (`pattern` is over 768 bytes), or `kv_handles_disabled` (shares can't be used this way on this deployment).
With `stream: true`, the method returns an async iterator of [`KVListPage`](/Objects/kvlistpage) objects instead:
+9 -2
View File
@@ -10,7 +10,7 @@ Lists the live shares other users made with you, a page at a time. Each share co
A share tells you its level, its app, and the owner's label. It never says who shared it or what the prefix is. To tell shares apart, read the label, or read a key the owner keeps for that, such as `meta`, through the handle. Reading needs `read` or `write` access, so a `watch` share only has its label.
In an app, the listing has only the shares of that app's data. Outside an app, it has every share made with you.
In an app, the listing has only the shares of that app's data. Outside an app, it has every share made with you, and `appUid` narrows it to one app.
## Syntax
@@ -26,6 +26,7 @@ puter.kv.listShared(options)
- `limit` (Number): Shares per page. Defaults to `50`, at most `200`.
- `cursor` (String): The `cursor` from the previous page.
- `includeTotal` (Boolean): Add `total` to the result. Defaults to `false`.
- `appUid` (String | null): List only the shares of this app's data. Pass `null` for the store that belongs to no app. In an app, the only uid you can pass is the app's own.
## Return value
@@ -44,7 +45,13 @@ Shares that stopped aren't listed. An access token scoped to less than the accou
## Errors
The promise rejects with `{ message, code }`. A `limit` that isn't a positive number is rejected with `bad_request`, and going over the rate limit with `too_many_requests`. See [Rate Limits and Quotas](/rate-limits-and-quotas/#events).
The promise rejects with `{ message, code }`:
| Code | Meaning |
| -- | -- |
| `events_kv_handle_outside_namespace` | The call came from an app and `appUid` names a different app, or `null`. |
| `bad_request` | `limit` isn't a positive number, or `appUid` isn't an app uid. |
| `too_many_requests` | Over the rate limit. See [Rate Limits and Quotas](/rate-limits-and-quotas/#events). |
## Examples
+9 -6
View File
@@ -6,9 +6,9 @@ platforms: [websites, apps, nodejs, workers]
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
Lists the shares you made with [`puter.kv.share()`](/KV/share/), a page at a time. Shares that stopped stay in the listing with `revokedAt` and `revokedBy` set, as a record of who had access and when it ended.
Lists the shares you made with [`puter.kv.share()`](/KV/share/), a page at a time. Shares that stopped stay in the listing for 30 days with `revokedAt` and `revokedBy` set, as a record of who had access and when it ended. After 30 days they are deleted.
In an app, the listing has only the shares of that app's data. Outside an app, it has every share you made.
In an app, the listing has only the shares of that app's data. Outside an app, it has every share you made, and `appUid` narrows it to one app.
## Syntax
@@ -24,6 +24,8 @@ puter.kv.listSharedByMe(options)
- `limit` (Number): Shares per page. Defaults to `50`, at most `200`.
- `cursor` (String): The `cursor` from the previous page.
- `includeTotal` (Boolean): Add `total` to the result. Defaults to `false`.
- `appUid` (String | null): List only the shares of this app's data. Pass `null` for the store that belongs to no app. In an app, the only uid you can pass is the app's own.
- `state` (String): `'live'` lists only shares that are still live, `'revoked'` only shares that stopped, and `'all'` both. Defaults to `'all'`.
## Return value
@@ -40,7 +42,7 @@ A `Promise` that resolves to:
- `revokedAt` (Number | null): When the share stopped, in Unix seconds, or `null` while it's live.
- `revokedBy` (String | null): Who stopped it. It's `'owner'` when you revoked it, `'holder'` when the holder removed their own access, and `'system'` when the app whose data it was got deleted. It's `null` while the share is live, and on some shares that stopped before Puter recorded who stopped them.
- `cursor` (String): Pass it as `cursor` to get the next page. It's included only if there are more pages to load.
- `total` (Number): The number of shares across every page, stopped ones included. Only included when `includeTotal` is `true`.
- `total` (Number): The number of shares across every page that match `appUid` and `state`. Only included when `includeTotal` is `true`.
## Errors
@@ -49,7 +51,8 @@ The promise rejects with `{ message, code }`:
| Code | Meaning |
| -- | -- |
| `events_kv_handle_owner_only` | The call came from an access token with less access than the account. |
| `bad_request` | `limit` isn't a positive number. |
| `events_kv_handle_outside_namespace` | The call came from an app and `appUid` names a different app, or `null`. |
| `bad_request` | `limit` isn't a positive number, `appUid` isn't an app uid, or `state` isn't `'live'`, `'revoked'`, or `'all'`. |
| `too_many_requests` | Over the rate limit. See [Rate Limits and Quotas](/rate-limits-and-quotas/#events). |
## Examples
@@ -73,9 +76,9 @@ for (const share of items) {
```js
let cursor;
do {
const page = await puter.kv.listSharedByMe({ limit: 100, cursor });
const page = await puter.kv.listSharedByMe({ limit: 100, cursor, state: 'live' });
for (const share of page.items) {
if (share.prefix === 'list:42:' && share.revokedAt === null) {
if (share.prefix === 'list:42:') {
await puter.kv.unshare(share.handle);
}
}
+29 -6
View File
@@ -6,7 +6,7 @@ platforms: [websites, apps, nodejs, workers]
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
Shares the keys under a prefix with another Puter user. The other user becomes the share's holder, and you stay its owner. The keys stay in your store, and the holder reaches them through the share.
Shares the keys under a prefix with another Puter user, or with up to 50 users in one call. The other user becomes the share's holder, and you stay its owner. The keys stay in your store, and the holder reaches them through the share.
Each share has a `handle`. The holder finds it with [`puter.kv.listShared()`](/KV/listShared/) and passes it as `handle` to the other [`puter.kv`](/KV/) methods. [Sharing with other users](/KV/#sharing-with-other-users) shows how a holder uses a share.
@@ -15,6 +15,8 @@ Each share has a `handle`. The holder finds it with [`puter.kv.listShared()`](/K
```js
puter.kv.share(prefix, recipient)
puter.kv.share(prefix, recipient, access)
puter.kv.share(prefix, recipients)
puter.kv.share(prefix, recipients, access)
puter.kv.share(options)
```
@@ -36,12 +38,16 @@ A new share is also rejected with `invalid_kv_share_prefix` when its prefix diff
The user to share with, as a username, `{ username }`, or `{ uid }` with the user's uuid. A string that contains `@` is rejected with `invalid_recipient`.
#### `recipients` (Array)
The users to share with, from 1 to 50, each in any form `recipient` takes. Every recipient gets a share of the same prefix at the same access level, with the same label. A user named twice gets one share. An empty array or more than 50 recipients is rejected with `bad_request`.
#### `access` (String) (optional)
What the holder can do. Defaults to `'read'`. Each level includes the ones before it.
- `'watch'`: receive [events](/Events/subjects/#sharing-key-value-data-with-another-user) when keys under the prefix change.
- `'read'`: also read keys with [`puter.kv.get()`](/KV/get/) and [`puter.kv.list()`](/KV/list/).
- `'watch'`: receive [events](/Events/subjects/#sharing-key-value-data-with-another-user) when keys under the prefix change, and list the keys without their values with [`puter.kv.list()`](/KV/list/).
- `'read'`: also read values with [`puter.kv.get()`](/KV/get/) and [`puter.kv.list()`](/KV/list/).
- `'write'`: also change and delete keys with every other [`puter.kv`](/KV/) method, such as [`puter.kv.set()`](/KV/set/), [`puter.kv.incr()`](/KV/incr/) and [`puter.kv.del()`](/KV/del/). Write access includes delete, so a holder can change or remove any key under the prefix.
#### `options` (Object)
@@ -49,7 +55,8 @@ What the holder can do. Defaults to `'read'`. Each level includes the ones befor
The options form takes these properties:
- `prefix` (String) (required): The key prefix, as above.
- `recipient` (String | Object) (required): Who to share with, as above.
- `recipient` (String | Object): Who to share with, as above.
- `recipients` (Array): Several users to share with, as above. Pass either `recipient` or `recipients`, not both.
- `access` (String): What the holder can do, as above. Defaults to `'read'`.
- `label` (String | null): Plain text the holder is shown with the share, such as `'Groceries from Dan'`. Whitespace at either end is trimmed, and the rest can be at most 128 characters and 512 bytes of UTF-8. Control characters, invisible formatting characters (such as zero-width spaces and text direction marks), line breaks, and more than 3 combining marks on one character are rejected with `kv_share_label_invalid`. An empty string means no label.
- `appUuid` (String): The app whose data to share. Outside an app, such as in a Node.js script using the user's auth token, it can name any app that exists. An app always shares its own data, and passing a different app's id fails with `events_kv_handle_outside_namespace`.
@@ -68,6 +75,12 @@ A `Promise` that resolves to the share:
- `label` (String | null): The label the holder is shown.
- `createdAt` (Number): When the share was made, in Unix seconds.
With `recipients`, the `Promise` resolves to an array with one item per recipient, in the order you passed them. Each item has `recipient`, the recipient as you passed it, and either the share's properties listed above or `error`:
- `error` (Object): Why that recipient didn't get a share, as `{ message, code }`. The code is one of the [errors](#errors) about that recipient: `invalid_recipient`, `subject_does_not_exist`, `bad_request` when it's you, `invalid_kv_share_prefix` when a share with that user differs only in letter case, `recipient_not_accepting_shares`, or `events_kv_handle_limit_reached` when you already hold 256 live shares with that user.
Any other error, such as a bad `prefix`, rejects the whole call. Before it shares with anyone, the call checks that all the new shares fit within your limits and your rate limits, and rejects the whole call if they don't.
## Sharing the same prefix again
Sharing the same prefix of the same app's data with the same user again changes the existing share instead of making a second one. The share keeps its `handle` and `createdAt` and gets the new `access`. Use this to raise or lower a holder's access. The holder's subscriptions through the handle keep working.
@@ -96,14 +109,14 @@ The promise rejects with `{ message, code }`:
| -- | -- |
| `invalid_recipient` | `recipient` contains `@`, or isn't a username, `{ username }`, or `{ uid }`. |
| `subject_does_not_exist` | No user has that username or uuid. |
| `bad_request` | You named yourself as the recipient, or `appUuid` isn't the id of an app that exists. |
| `bad_request` | You named yourself as the recipient, `appUuid` isn't the id of an app that exists, you passed both `recipient` and `recipients`, or `recipients` is empty or has more than 50 entries. |
| `invalid_kv_share_prefix` | `prefix` is empty, too long, has an empty segment, or has a character other than lowercase letters, digits, `_`, `.`, and `-` between its separators, or differs from one of your shares with the same user only in letter case. |
| `kv_share_access_invalid` | `access` isn't `'watch'`, `'read'`, or `'write'`. |
| `kv_share_label_invalid` | `label` isn't a string or `null`, is too long, or contains a character it can't have. |
| `recipient_not_accepting_shares` | The recipient blocked you, or doesn't accept shares. On an existing share, you raised its access or set a new label. |
| `account_is_not_verified` | Your email address isn't confirmed. |
| `events_kv_handle_requires_account` | A temporary account tried to share. |
| `events_kv_handle_limit_reached` | You already hold as many live shares as your plan allows. |
| `events_kv_handle_limit_reached` | You already hold as many live shares as your plan allows, in total, in this app, or with this user. With `recipients`, the new shares would go over your limit in total or in this app. |
| `events_kv_handle_not_delegated` | An app shared without the user's consent for that prefix and level. |
| `events_kv_handle_outside_namespace` | An app named another app's data. |
| `events_kv_handle_owner_only` | The call came from an access token with less access than the account. |
@@ -139,6 +152,16 @@ const share = await puter.kv.share({
await puter.kv.share('trip:japan-2027:', 'alice', 'read');
```
<strong class="example-title">Share with several users at once</strong>
```js
const results = await puter.kv.share('board:planning:', ['alice', 'bob', 'carol'], 'read');
for (const result of results) {
if (result.error) puter.print(`Not shared with ${result.recipient}: ${result.error.code}<br>`);
else puter.print(`Shared with ${result.holder}<br>`);
}
```
## Related
- [`puter.kv.unshare()`](/KV/unshare/) - Stop a share
+1 -1
View File
@@ -11,7 +11,7 @@ Stops a share made with [`puter.kv.share()`](/KV/share/). Either side can call i
- **The owner** revokes the share. The holder loses access, and any narrower share you made with the same holder inside the same prefix stops too.
- **The holder** removes their own access. Any narrower share they hold from the same owner inside the same prefix stops too.
The handle stops working right away for every [`puter.kv`](/KV/) call, and persistent subscriptions on it are suspended with `permission_revoked`. If a wider share the holder still has covers the same keys, calls through the stopped handle can keep working for up to 60 seconds. A stopped share stays in the owner's [`listSharedByMe()`](/KV/listSharedByMe/) listing with `revokedAt` and `revokedBy` set. The keys themselves stay in the owner's store.
The handle stops working right away for every [`puter.kv`](/KV/) call, and persistent subscriptions on it are suspended with `permission_revoked`. If a wider share the holder still has covers the same keys, calls through the stopped handle can keep working for up to 60 seconds. A stopped share stays in the owner's [`listSharedByMe()`](/KV/listSharedByMe/) listing for 30 days with `revokedAt` and `revokedBy` set. The keys themselves stay in the owner's store.
## Syntax
+1
View File
@@ -12,6 +12,7 @@ Various object types and classes that represent different entities in the Puter
- **[ChatResponse](/Objects/chatresponse/)** - Represents an AI chat response
- **[ChatResponseChunk](/Objects/chatresponsechunk/)** - Represents a chunk of streaming chat response data
- **[CreateAppResult](/Objects/createappresult/)** - Represents the result of creating an app
- **[CustomDomain](/Objects/customdomain/)** - Represents a domain connected to a website
- **[DetailedAppUsage](/Objects/detailedappusage/)** - Represents detailed resource usage statistics for a specific application
- **[EmailListPage](/Objects/emaillistpage/)** - Represents one page of mailbox listing results
- **[EmailMessage](/Objects/emailmessage/)** - Represents a fully parsed message from the user's mailbox
+54
View File
@@ -0,0 +1,54 @@
---
title: CustomDomain
description: The CustomDomain object containing the details of a domain connected to a website.
---
The `CustomDomain` object describes a domain the user owns that serves one of their websites hosted on Puter. The [`puter.hosting.attach()`](/Hosting/attach/) method resolves to it.
## Attributes
#### `uuid` (String)
The unique identifier of the custom domain.
#### `hostname` (String)
The domain, such as `www.example.com`.
#### `site` (String)
The name of the website the domain serves, such as `example` for `example.puter.site`.
#### `status` (String)
The state of the domain. It's one of the following:
- `pending` until the DNS records are set and the domain is verified.
- `active` once the domain serves the website.
- `failed` when the domain can't be verified. The `statusDetail` attribute says why.
#### `statusDetail` (String | null)
Why the domain isn't active yet, when that's known.
#### `isApex` (Boolean)
`true` when the hostname is a root domain such as `example.com`. A root domain can't use a regular `CNAME` record, so its DNS provider has to support `CNAME` flattening, `ALIAS`, or `ANAME` records.
#### `records` (Array)
The DNS records to set at the domain's DNS provider. Each record is an object with these attributes:
- `type` (String): `CNAME` or `TXT`.
- `name` (String): the record's name.
- `value` (String): the record's value.
- `purpose` (String): `routing` for the record that sends traffic to the website, `ownership` for the record that proves the user owns the domain, or `certificate` for a record that speeds up issuing the HTTPS certificate.
- `required` (Boolean): `true` for the `routing` and `ownership` records, which the user has to set. A `certificate` record is optional.
#### `createdAt` (Number)
When the domain was added, in milliseconds since the Unix epoch.
#### `activatedAt` (Number | null)
When the domain became active, in milliseconds since the Unix epoch, or `null` if it isn't active.
+6
View File
@@ -775,6 +775,12 @@ const examples = [
slug: 'hosting-get',
source: '/playground/examples/hosting-get.html',
},
{
title: 'Connect a custom domain to a website',
description: 'Let the user connect their own domain to a website with Puter.js hosting API. Run and experiment with this example directly in the playground.',
slug: 'hosting-attach',
source: '/playground/examples/hosting-attach.html',
},
{
title: 'Publish a portfolio page',
description: 'Publish a page built from form input as a public website with Puter.js hosting API. Run and modify this example directly in your browser.',
@@ -0,0 +1,24 @@
<html>
<body>
<button id="attach">Use my own domain</button>
<script src="https://js.puter.com/v2/"></script>
<script>
document.getElementById('attach').addEventListener('click', async () => {
// (1) Create a website to connect the domain to
const site = await puter.hosting.create(puter.randName(), '.');
// (2) Let the user pick or add a domain for it
const domain = await puter.hosting.attach(site.subdomain);
if (!domain) {
puter.print('No domain was chosen.');
return;
}
puter.print(`${domain.hostname} is ${domain.status}.<br>`);
for (const record of domain.records) {
puter.print(`${record.type} ${record.name} → ${record.value}<br>`);
}
});
</script>
</body>
</html>
+9 -4
View File
@@ -430,13 +430,15 @@ See [Events](/Events/) for an overview, [`onLocal()`](/Events/subjects/#gaps) fo
| ------------------------------------ | ------------------ | ---- | ---- | --------- |
| Persistent subscriptions | Per user, all apps | 500 | 100 | 0 |
| Persistent subscriptions | Per user, per app | 100 | 25 | 0 |
| KV share handles | Per user, all apps | 512 | 200 | 0 |
| KV share handles | Per user, per app | 512 | 128 | 0 |
| KV share handles | Per user, all apps | 25,000 | 5,000 | 0 |
| KV share handles | Per user, per app | 10,000 | 2,000 | 0 |
| KV share handles with one other user | Per owner and holder | 256 | 256 | 0 |
| Session subscriptions | Per connection | 50 | 50 | 50 |
- Over a subscription limit: `events_subscription_limit`. Over a share-handle limit: `events_kv_handle_limit_reached`.
- Anonymous accounts get `events_durable_requires_account` for persistent subscriptions and `events_kv_handle_requires_account` for share handles.
- Unsubscribing or revoking frees a slot immediately. Revoked handles stay listed but don't count.
- Unsubscribing or revoking frees a slot immediately. Revoked handles don't count. They stay in [`puter.kv.listSharedByMe()`](/KV/listSharedByMe/) for 30 days after they stop, and are then deleted.
- A [`puter.kv.share()`](/KV/share/) call with several recipients is refused as a whole with `events_kv_handle_limit_reached` when its new shares would go over the all-apps or per-app limit. Sharing a prefix again with a user who already has it takes no slot. The limit per holder applies to each recipient on its own, and a recipient over it gets the error on their own item.
- A share handle minted without naming an app counts only toward the all-apps limit.
- A share's key prefix has a length limit that depends on the app's uid and the access level. For an app uid of the usual form, `app-` and a uuid, the prefix without its trailing `:` can be at most 168 characters for `watch`, 163 for `read`, and 162 for `write`. In the store that belongs to no app, it can be at most 199, 194, and 193. A longer prefix is refused with `invalid_kv_share_prefix`.
- The consent an app asks for with [`puter.perms.request('kvShare', …)`](/Perms/request/) names a prefix too. Its limit is 7 characters lower, so 161, 156, and 155 for an app uid of the usual form.
@@ -466,6 +468,7 @@ Counted per event, per region:
| `unsubscribe` | 600/min | Per user, all apps |
| Acknowledgements | 600/min | Per user, all apps |
| Share-handle mint / revoke | 60/min | Per user, all apps |
| Recipients of `kv.share()` | 300/min, 2,000/hour | Per user, all apps |
| `kv.share.received` notifications | 5/hour | Per owner and holder |
| Handler publish / remove | 60/min | Per user, all apps |
| Subscription and share-handle listings (combined) | 120/min | Per user, per app |
@@ -473,7 +476,9 @@ Counted per event, per region:
| Events worker listings | 120/min | Per user, per app |
| `fetch()` | 120/min | Per user, per app |
Over any of these: `too_many_requests`, except `kv.share.received` notifications: a share past that limit is still made, and the holder isn't notified about it. Listing pages hold up to 200 items. A `fetch()` page defaults to 50 events and holds up to 200.
Over any of these: `too_many_requests`, except `kv.share.received` notifications: a share past that limit is still made, and the holder isn't notified about it. Listing pages hold up to 200 items.
A [`puter.kv.share()`](/KV/share/) call names 1 to 50 recipients. Each one counts toward the recipient budget, and the call counts once toward the mint budget. When the recipients don't all fit in the recipient budget, the whole call fails with `too_many_requests`, and the part that did fit still counts. A `fetch()` page defaults to 50 events and holds up to 200.
Unsubscribing counts toward its own `unsubscribe` limit, not the `subscribe` one.
+15
View File
@@ -648,6 +648,14 @@ let sidebar = [
source: '/Hosting/get.md',
path: '/Hosting/get',
},
{
title: '<code>attach()</code>',
page_title: '<code>puter.hosting.attach()</code>',
title_tag: 'puter.hosting.attach()',
icon: '/assets/img/function.svg',
source: '/Hosting/attach.md',
path: '/Hosting/attach',
},
],
},
{
@@ -1444,6 +1452,13 @@ let sidebar = [
source: '/Objects/speech2txtresult.md',
path: '/Objects/speech2txtresult',
},
{
title: '<code>CustomDomain</code>',
title_tag: 'CustomDomain',
icon: '/assets/img/object.svg',
source: '/Objects/customdomain.md',
path: '/Objects/customdomain',
},
{
title: '<code>Subdomain</code>',
title_tag: 'Subdomain',
+53
View File
@@ -29,6 +29,7 @@ import UIPopover from './UI/UIPopover.js';
import UIPrompt from './UI/UIPrompt.js';
import UIWindow from './UI/UIWindow.js';
import UIWindowAppFeedback from './UI/UIWindowAppFeedback.js';
import UIWindowAttachCustomDomain from './UI/UIWindowAttachCustomDomain.js';
import UIWindowColorPicker from './UI/UIWindowColorPicker.js';
import UIWindowEmailConfirmationRequired from './UI/UIWindowEmailConfirmationRequired.js';
import UIWindowFontPicker from './UI/UIWindowFontPicker.js';
@@ -51,6 +52,8 @@ window.ipc_handlers = {};
// handler below): one dialog at a time, and nothing else — reopening is always
// allowed. See feedbackDialogGuard.js.
const feedback_dialog_guard = createFeedbackDialogGuard();
// Same one-at-a-time rule for the custom domain dialog.
const custom_domain_dialog_guard = createFeedbackDialogGuard();
/**
* In Puter, apps are loaded in iframes and communicate with the graphical user interface (GUI), and each other, using the postMessage API.
@@ -1481,6 +1484,56 @@ const ipc_listener = async (event, handled) => {
}
}
//--------------------------------------------------------
// attachCustomDomain
//--------------------------------------------------------
else if ( event.data.msg === 'attachCustomDomain' ) {
// Always answer, as for showFeedbackDialog: the SDK promise only
// settles on a reply.
const respond = (domain) => {
target_iframe?.contentWindow?.postMessage({
msg: 'customDomainAttached',
domain: domain ?? null,
original_msg_id: msg_id,
}, '*');
};
const subdomain = typeof event.data.subdomain === 'string' ? event.data.subdomain : '';
if ( ! subdomain || ! custom_domain_dialog_guard.mayOpen() ) {
respond(null);
return;
}
custom_domain_dialog_guard.markOpened();
try {
try {
if ( !window.is_auth() && !(await UIWindowSignup({ referrer: app_name })) ) {
respond(null);
return;
}
} catch ( e ) {
console.error('IPC attachCustomDomain: auth gate failed', e);
respond(null);
return;
}
let domain = null;
try {
// The requester shown is the app's registered name from the
// GUI's own records; its window title is app-controlled.
domain = await UIWindowAttachCustomDomain({
subdomain,
requester: app_name,
});
} catch ( e ) {
console.error('IPC attachCustomDomain failed', e);
}
respond(domain);
$(target_iframe).get(0)?.focus({ preventScroll: true });
} finally {
custom_domain_dialog_guard.markClosed();
}
}
//--------------------------------------------------------
// showFontPicker
//--------------------------------------------------------
else if ( event.data.msg === 'showFontPicker' ) {
+6 -1
View File
@@ -426,6 +426,10 @@ function showUninstallModal ({ appName, appTitle, appUid, self, $el_window }) {
<div class="myapps-modal" role="dialog" aria-modal="true" aria-label="${i18n('uninstall_app', { name: displayName })}">
<h3>${i18n('uninstall_app', { name: displayName })}</h3>
<p>${i18n('uninstall_app_body')}</p>
<label class="myapps-modal-option">
<input type="checkbox" class="myapps-modal-delete-data">
<span>${i18n('uninstall_delete_data')}</span>
</label>
<div class="myapps-modal-buttons">
<button class="myapps-modal-btn myapps-modal-cancel">${i18n('cancel')}</button>
<button class="myapps-modal-btn myapps-modal-confirm">${i18n('uninstall')}</button>
@@ -470,6 +474,7 @@ function showUninstallModal ({ appName, appTitle, appUid, self, $el_window }) {
// The saved order intentionally keeps the app's name:
// reconcileAppOrder ignores it while the app is gone and
// restores its position if it comes back.
const deleteData = $overlay.find('.myapps-modal-delete-data').is(':checked');
const removedIndex = self._apps.findIndex(a => a.name === appName);
const removedApp = removedIndex === -1 ? null : self._apps[removedIndex];
// A running instance would be stranded: the tile is a headless
@@ -558,7 +563,7 @@ function showUninstallModal ({ appName, appTitle, appUid, self, $el_window }) {
finishRemoval();
}
puter.perms.revokeApp(appUid, '*').then(async (result) => {
puter.perms.revokeApp(appUid, '*', { deleteData }).then(async (result) => {
// revokeApp resolves (never rejects) with `error` set on failure.
if ( result?.error ) throw result;
// Clearing the grants is only half of it. An app the user already
File diff suppressed because it is too large. Load diff
+526
View File
@@ -27,6 +27,10 @@ vi.mock('../UIAlert.js', () => ({ default: vi.fn() }));
vi.mock('../UIWindowProgress.js', () => ({ default: vi.fn() }));
vi.mock('../UIContextMenu.js', () => ({ default: vi.fn() }));
vi.mock('../../helpers/generateFileContextMenu.js', () => ({ default: vi.fn(async () => []) }));
vi.mock('../../helpers/itemIcon.js', async (importOriginal) => {
const actual = await importOriginal();
return { default: vi.fn(actual.default) };
});
globalThis.$ = jQuery;
globalThis.jQuery = jQuery;
@@ -38,6 +42,7 @@ window.icons = {};
const { default: TabFiles, failedUploadsMessage, rebasePath, typeSelectTarget } = await import('./TabFiles.js');
const { default: generateFileContextMenu } = await import('../../helpers/generateFileContextMenu.js');
const { default: itemIcon } = await import('../../helpers/itemIcon.js');
const TRASH = '/alice/Trash';
@@ -246,6 +251,35 @@ describe('TabFiles opening a folder that fails to list', () => {
expect(TabFiles.acceptsNewItems('/alice/Documents')).toBe(true);
expect(TabFiles.renderingDirectory).toBe(false);
});
it('keeps the rows when a refresh of it can\'t reach the server', async () => {
mountListing(['a.txt'], '/alice/Documents');
TabFiles.renderingDirectory = false;
// puter.js rejects with the XMLHttpRequest itself, which has no code.
window.puter.fs.readdir = vi.fn(async () => {
throw { status: 0 };
});
await TabFiles.renderDirectory('/alice/Documents');
expect($('.files .row')).toHaveLength(1);
expect(TabFiles.acceptsNewItems('/alice/Documents')).toBe(true);
expect(TabFiles.renderingDirectory).toBe(false);
});
it('says so when a refresh finds it gone', async () => {
mountListing(['a.txt'], '/alice/Documents');
TabFiles.renderingDirectory = false;
TabFiles.trashedNames ??= new Map();
window.puter.fs.readdir = vi.fn(async () => {
throw { code: 'subject_does_not_exist', message: 'Subject does not exist' };
});
await TabFiles.renderDirectory('/alice/Documents');
expect($('.files .row')).toHaveLength(0);
expect($('.files').text()).toContain("couldn't be opened");
});
});
describe('TabFiles opening a folder during a refresh', () => {
@@ -294,6 +328,7 @@ describe('TabFiles New Folder', () => {
});
vi.spyOn(TabFiles, 'insertAtSortedPosition').mockImplementation(() => {});
vi.spyOn(TabFiles, 'applyColumnWidths').mockImplementation(() => {});
TabFiles.deferredAdds = [];
});
it('is what Undo takes back, not the upload before it', async () => {
@@ -308,6 +343,62 @@ describe('TabFiles New Folder', () => {
expect($(last.data).attr('data-path')).toBe('/alice/Documents/New Folder');
});
it('becomes the selection the footer counts', async () => {
mountListing(['a.txt', 'b.txt'], '/alice/Documents');
$('.files .row').addClass('selected');
TabFiles.updateFooterStats();
TabFiles.renderingDirectory = false;
TabFiles._creatingItem = 0;
await TabFiles.createFolderInstant('/alice/Documents');
expect($('.files .row.selected')).toHaveLength(1);
expect($('.files-footer-selected-items').text()).toMatch(/^1 item selected/);
});
it('lets items added meanwhile in, once it is made', async () => {
mountListing([], '/alice/Documents');
TabFiles.renderingDirectory = false;
TabFiles._creatingItem = 0;
let finishMkdir;
window.puter.fs.mkdir = vi.fn(() => new Promise((resolve) => {
finishMkdir = resolve;
}));
const added = vi.fn();
const previous = window.UIDashboardFileItem;
window.UIDashboardFileItem = (file) => {
if ( TabFiles._creatingItem ) TabFiles.deferredAdds.push(file);
else added(file);
};
const creating = TabFiles.createFolderInstant('/alice/Documents');
await vi.waitFor(() => expect(finishMkdir).toBeTypeOf('function'));
window.UIDashboardFileItem({ uid: 'u2', path: '/alice/Documents/from-elsewhere.txt' });
expect(added).not.toHaveBeenCalled();
finishMkdir({ uid: 'f1', name: 'New Folder', path: '/alice/Documents/New Folder', is_dir: true });
await creating;
expect(added).toHaveBeenCalledWith({ uid: 'u2', path: '/alice/Documents/from-elsewhere.txt' });
window.UIDashboardFileItem = previous;
});
it('says so when the network fails, rather than vanishing', async () => {
const { default: UIAlert } = await import('../UIAlert.js');
UIAlert.mockClear();
mountListing([], '/alice/Documents');
TabFiles.renderingDirectory = false;
TabFiles._creatingItem = 0;
// puter.js rejects with the XMLHttpRequest itself, which has no message.
window.puter.fs.mkdir = vi.fn(async () => {
throw { status: 0 };
});
await TabFiles.createFolderInstant('/alice/Documents');
expect($('.files .row')).toHaveLength(0);
expect(UIAlert).toHaveBeenCalledWith('An unknown error occurred.');
});
it('is made, not dropped, while the folder is refreshing', async () => {
mountListing([], '/alice/Documents');
TabFiles.trashedNames ??= new Map();
@@ -454,6 +545,25 @@ describe('TabFiles.updateNavButtonStates', () => {
expect($('.path-btn-forward').attr('tabindex')).toBe('-1');
expect($('.path-btn-forward').attr('aria-disabled')).toBe('true');
});
it('moves the focus off an arrow it disables', () => {
document.body.innerHTML = `
<div class="window">
<img class="path-btn path-btn-back" tabindex="0">
<img class="path-btn path-btn-forward" tabindex="0">
<img class="path-btn path-btn-up" tabindex="0">
</div>`;
TabFiles.$el_window = $('.window');
TabFiles.currentPath = '/alice';
window.dashboard_nav_history = ['/alice', '/alice/Documents'];
// Back was pressed from Documents.
window.dashboard_nav_history_current_position = 0;
$('.path-btn-back')[0].focus();
TabFiles.updateNavButtonStates();
expect(document.activeElement).toBe($('.path-btn-forward')[0]);
});
});
describe('TabFiles cancelling a drag that opened a folder', () => {
@@ -502,6 +612,81 @@ describe('TabFiles image preview', () => {
expect(TabFiles.previewOpen).toBe(false);
expect($('.image-preview-popover')).toHaveLength(0);
});
it('is not offered for a file named after an image type', async () => {
mountListing(['png'], '/alice/Pictures');
$('.files .row').attr({ 'data-uid': 'u1', 'data-name': 'png', 'data-is_dir': '0' });
window.puter = { fs: { read: vi.fn(async () => new Blob(['x'])) } };
TabFiles.previewOpen = false;
await TabFiles.showImagePreview($('.files .row'));
expect(window.puter.fs.read).not.toHaveBeenCalled();
expect(TabFiles.previewOpen).toBe(false);
});
it('can be cancelled while its image is still loading', async () => {
window.user = { username: 'alice', directories: {} };
window.home_path = '/alice';
TabFiles.trashedNames ??= new Map();
TabFiles.renderingDirectory = false;
mountListing(['a.png'], '/alice/Pictures');
$('.files .row').attr({ 'data-uid': 'u1', 'data-name': 'a.png', 'data-is_dir': '0' });
let finishUrl;
window.puter = { fs: {
read: vi.fn(() => new Promise(resolve => {
finishUrl = resolve;
})),
readdir: vi.fn(async () => []),
} };
const preview = TabFiles.showImagePreview($('.files .row'));
expect(TabFiles.previewOpen).toBe(true);
// Opening another folder meanwhile, as a double-click on one would.
await TabFiles.renderDirectory('/alice/Documents');
finishUrl(new Blob(['x']));
await preview;
expect(TabFiles.previewOpen).toBe(false);
expect($('.image-preview-popover')).toHaveLength(0);
});
it('reads the image rather than spending an access token on it, typed so an SVG shows', async () => {
mountListing(['logo.svg'], '/alice/Pictures');
$('.files .row').attr({ 'data-uid': 'u1', 'data-name': 'logo.svg', 'data-is_dir': '0' });
window.puter = { fs: { read: vi.fn(async () => new Blob(['<svg/>'])), getReadURL: vi.fn() } };
const created = [];
URL.createObjectURL = vi.fn((blob) => {
created.push(blob);
return 'blob:preview';
});
URL.revokeObjectURL = vi.fn();
await TabFiles.showImagePreview($('.files .row'));
expect(window.puter.fs.getReadURL).not.toHaveBeenCalled();
expect(created[0].type).toBe('image/svg+xml');
expect($('.image-preview-popover img').attr('src')).toBe('blob:preview');
TabFiles.closeImagePreview();
expect(URL.revokeObjectURL).toHaveBeenCalledWith('blob:preview');
});
it('closes, rather than showing the last image, when the next one fails to load', async () => {
mountListing(['b.png'], '/alice/Pictures');
$('.files .row').attr({ 'data-uid': 'u2', 'data-name': 'b.png', 'data-is_dir': '0' });
$('body').append('<div class="image-preview-popover" data-uid="u1"></div>');
TabFiles.previewOpen = true;
TabFiles.previewCurrentUid = 'u1';
window.puter = { fs: { read: vi.fn(async () => { throw new Error('gone'); }) } };
vi.spyOn(console, 'error').mockImplementation(() => {});
await TabFiles.showImagePreview($('.files .row'));
expect(TabFiles.previewOpen).toBe(false);
expect($('.image-preview-popover')).toHaveLength(0);
vi.restoreAllMocks();
});
});
describe('TabFiles.acceptsNewItems', () => {
@@ -729,3 +914,344 @@ describe('TabFiles Remove from Shared', () => {
expect(TabFiles.refreshKeepingSelection).not.toHaveBeenCalled();
});
});
describe('TabFiles sorting', () => {
const entries = [
{ name: 'file10.txt', size: 5, modified: 3 },
{ name: 'dir10', is_dir: true, size: 0, modified: 1 },
{ name: 'file2.txt', size: 5, modified: 3 },
{ name: 'Beta', is_dir: true, size: 0, modified: 1 },
{ name: 'dir2', is_dir: true, size: 0, modified: 1 },
{ name: 'big.bin', size: 50, modified: 2 },
];
const sorted = (column, direction) => {
TabFiles.sortColumn = column;
TabFiles.sortDirection = direction;
return TabFiles.sortFiles(entries).map(e => e.name);
};
beforeEach(() => {
TabFiles.currentPath = '/alice/Documents';
window.shared_path = '/alice/Shared';
});
it('orders names with numbers by value, folders first', () => {
expect(sorted('name', 'asc')).toEqual(['Beta', 'dir2', 'dir10', 'big.bin', 'file2.txt', 'file10.txt']);
});
it('breaks ties in size by name', () => {
expect(sorted('size', 'asc')).toEqual(['Beta', 'dir2', 'dir10', 'file2.txt', 'file10.txt', 'big.bin']);
expect(sorted('size', 'desc')).toEqual(['dir10', 'dir2', 'Beta', 'big.bin', 'file10.txt', 'file2.txt']);
});
it('shows trashed items by the name they had', () => {
TabFiles.sortColumn = 'name';
TabFiles.sortDirection = 'asc';
const trashed = [
{ name: 'uid-b', metadata: JSON.stringify({ original_name: 'b.txt' }) },
{ name: 'uid-a', metadata: JSON.stringify({ original_name: 'a.txt' }) },
];
expect(TabFiles.sortFiles(trashed).map(e => e.name)).toEqual(['uid-a', 'uid-b']);
});
it("keeps a renamed row among its sharer's in the Shared view", () => {
window.shared_path = 'puter://shared';
TabFiles.currentPath = 'puter://shared';
TabFiles.sortColumn = 'shared_by';
TabFiles.sortDirection = 'asc';
const row = (name, sharer) => `<div class="item row" data-name="${name}" data-is_dir="0" data-shared_by="${sharer}"></div>`;
document.body.innerHTML = `<div class="window"><div class="files-tab"><div class="files">
${row('beta.txt', 'alice')}${row('zeta.txt', 'alice')}${row('alpha.txt', 'bob')}
</div></div></div>`;
TabFiles.$el_window = $('.window');
// beta.txt renamed to "zzz.txt": still Alice's, so still before Bob's.
const $renamed = $('.files .row').eq(0).attr('data-name', 'zzz.txt');
TabFiles.insertAtSortedPosition($renamed, { name: 'zzz.txt' });
expect($('.files .row').map((_, r) => r.getAttribute('data-name')).get()).toEqual(['zeta.txt', 'zzz.txt', 'alpha.txt']);
});
it('puts an added row where a listing would', () => {
TabFiles.sortColumn = 'size';
TabFiles.sortDirection = 'asc';
const listed = TabFiles.sortFiles(entries.filter(e => e.name !== 'file2.txt'));
document.body.innerHTML = `<div class="window"><div class="files-tab"><div class="files">${listed.map(e => (
`<div class="item row" data-name="${e.name}" data-is_dir="${e.is_dir ? 1 : 0}" data-size="${e.size}" data-modified="${e.modified}"></div>`
)).join('')}</div></div></div>`;
TabFiles.$el_window = $('.window');
const $row = $('<div class="item row" data-name="file2.txt" data-is_dir="0"></div>').appendTo('.files');
TabFiles.insertAtSortedPosition($row, { name: 'file2.txt', size: 5, modified: 3 });
expect($('.files .row').map((_, r) => r.getAttribute('data-name')).get())
.toEqual(['Beta', 'dir2', 'dir10', 'file2.txt', 'file10.txt', 'big.bin']);
});
});
describe('TabFiles permanent delete', () => {
beforeEach(() => {
window.trash_path = TRASH;
window.delete_item = vi.fn(async () => {});
window.refresh_trash_state = vi.fn(async () => {});
});
it('asks about "this item" for one row and "these items" for several', async () => {
const { default: UIAlert } = await import('../UIAlert.js');
UIAlert.mockClear();
UIAlert.mockResolvedValue('delete');
mountListing(['a', 'b'], TRASH);
const rows = $('.files .row').toArray();
await TabFiles.deleteRowsPermanently([rows[0]]);
expect(UIAlert.mock.calls[0][0].message).toMatch(/this item/);
expect(window.delete_item).toHaveBeenCalledTimes(1);
await TabFiles.deleteRowsPermanently(rows);
expect(UIAlert.mock.calls[1][0].message).toMatch(/these items/);
expect(window.delete_item).toHaveBeenCalledTimes(3);
});
it('deletes nothing when cancelled', async () => {
const { default: UIAlert } = await import('../UIAlert.js');
UIAlert.mockResolvedValue('cancel');
mountListing(['a'], TRASH);
await TabFiles.deleteRowsPermanently($('.files .row').toArray());
expect(window.delete_item).not.toHaveBeenCalled();
});
});
describe('TabFiles column widths that no longer fit', () => {
beforeEach(() => {
vi.restoreAllMocks();
window.shared_path = 'puter://shared';
window.trash_path = TRASH;
mountListing([], '/alice/Documents');
vi.spyOn($.fn, 'width').mockImplementation(function () {
return this.hasClass('files') ? 1200 : 0;
});
vi.spyOn(TabFiles, 'fitItemNames').mockImplementation(() => {});
});
afterEach(() => vi.restoreAllMocks());
it('lay out at the defaults beside an auto-width name', () => {
TabFiles.columnWidths = { name: null, sharedBy: 160, size: 100, modified: 1800 };
TabFiles.applyColumnWidths();
expect(TabFiles.columnGridTemplate).toBe('24px auto 4px 100px 4px 120px 4px 20px');
expect(TabFiles.columnWidths.modified).toBe(1800);
});
it('stay as saved while they fit', () => {
TabFiles.columnWidths = { name: null, sharedBy: 160, size: 100, modified: 900 };
TabFiles.applyColumnWidths();
expect(TabFiles.columnGridTemplate).toBe('24px auto 4px 100px 4px 900px 4px 20px');
});
});
describe('TabFiles names in a narrow listing', () => {
it('are not cut to the name column a wider listing was given', () => {
const name = 'Quarterly financial report final version.pdf';
document.body.innerHTML = `
<div class="window"><div class="files-tab">
<div class="header"><div class="columns" style="display: none"></div></div>
<div class="files files-list-view">
<div class="row item" data-name="${name}"><pre class="item-name"></pre></div>
</div>
</div></div>`;
TabFiles.$el_window = $('.window');
TabFiles.currentView = 'list';
TabFiles.appliedColumnWidths = { name: 120, sharedBy: 160, size: 60, modified: 80 };
TabFiles.fitItemNames($('.files .item-name'));
expect($('.files .item-name').text()).toBe(name);
});
});
describe('TabFiles uploads', () => {
beforeEach(async () => {
vi.restoreAllMocks();
window.trash_path = TRASH;
window.active_uploads = {};
window.actions_history = [];
window.show_save_account_notice_if_needed = vi.fn();
const { default: UIWindowProgress } = await import('../UIWindowProgress.js');
UIWindowProgress.mockResolvedValue({ set_status: vi.fn(), set_progress: vi.fn(), close: vi.fn(), show_error: vi.fn() });
window.puter = { fs: { upload: vi.fn(async (items, dest, callbacks) => {
await callbacks.init('op1', { abort: vi.fn() });
await callbacks.success([{ path: `${dest}/new.txt` }]);
}) } };
vi.spyOn(TabFiles, 'renderDirectory').mockImplementation(async () => {});
});
afterEach(() => vi.restoreAllMocks());
it('into another folder leave the listing and its selection alone', async () => {
mountListing(['a.txt', 'b.txt'], '/alice/Documents');
$('.files .row').addClass('selected');
TabFiles.uploadFiles([new File(['x'], 'new.txt')], '/alice/Documents/sub');
await vi.waitFor(() => expect(window.actions_history).toHaveLength(1));
expect(TabFiles.renderDirectory).not.toHaveBeenCalled();
expect($('.files .row.selected')).toHaveLength(2);
});
it('into the folder on screen list it again', async () => {
mountListing(['a.txt'], '/alice/Documents');
TabFiles.uploadFiles([new File(['x'], 'new.txt')], '/alice/Documents');
await vi.waitFor(() => expect(TabFiles.renderDirectory).toHaveBeenCalledWith('/alice/Documents', { consistency: 'strong' }));
});
});
describe('TabFiles refreshing a folder whose icons are slow to read', () => {
beforeEach(() => {
vi.restoreAllMocks();
window.user = { username: 'alice', directories: {} };
window.home_path = '/alice';
window.trash_path = TRASH;
window.byte_format = bytes => `${bytes} B`;
vi.spyOn(TabFiles, 'renderItem').mockImplementation(async (file) => {
$('.files').append(`<div class="row item" data-uid="${file.uid}" data-path="${file.path}" data-name="${file.name}"></div>`);
});
});
afterEach(() => vi.restoreAllMocks());
it('keeps the rows up until the new ones can be drawn', async () => {
mountListing(['old.txt'], '/alice/Documents');
TabFiles.renderingDirectory = false;
TabFiles.trashedNames ??= new Map();
window.puter = { fs: { readdir: vi.fn(async () => [
{ uid: 'w', name: 'site.weblink', path: '/alice/Documents/site.weblink', is_dir: false, size: 1, modified: 0 },
]) } };
let finishIcon;
itemIcon.mockImplementationOnce(() => new Promise(resolve => {
finishIcon = resolve;
}));
const refresh = TabFiles.renderDirectory('/alice/Documents');
await vi.waitFor(() => expect(finishIcon).toBeTypeOf('function'));
expect($('.files .row').map((_, r) => r.getAttribute('data-path')).get()).toEqual(['/alice/Documents/old.txt']);
finishIcon({ image: 'icon.svg', type: 'icon' });
await refresh;
expect($('.files .row').map((_, r) => r.getAttribute('data-name')).get()).toEqual(['site.weblink']);
});
});
describe('TabFiles rows still waiting on their mkdir', () => {
it('are not moved, cut or trashed', async () => {
window.user = { username: 'alice', directories: {} };
window.trash_path = TRASH;
mountListing(['a.txt', 'New Folder'], '/alice/Documents');
const [file, pending] = $('.files .row').toArray();
pending.setAttribute('data-pending', '1');
expect(await TabFiles.restructurableRows([file, pending])).toEqual([file]);
});
});
describe('TabFiles folder menu', () => {
const publishItem = items => items.find(item => item?.html === i18n('publish_as_website'));
it("doesn't offer to publish a folder shared with the user to view", async () => {
const { remember_shared_root } = await import('../../helpers/sharedAccess.js');
window.user = { username: 'alice', directories: {} };
window.trash_path = TRASH;
window.shared_path = 'puter://shared';
TabFiles.trashedNames ??= new Map();
const viewOnly = '/bob/0e98/Photos';
remember_shared_root({ path: viewOnly, mode: 'read', name: 'Photos' });
TabFiles.currentPath = viewOnly;
expect(publishItem(TabFiles.generateFolderContextMenu(viewOnly)).disabled).toBe(true);
});
it('offers to publish your own folder', () => {
window.user = { username: 'alice', directories: {} };
window.trash_path = TRASH;
window.shared_path = 'puter://shared';
TabFiles.trashedNames ??= new Map();
TabFiles.unreadablePath = '/alice/Documents';
expect(publishItem(TabFiles.generateFolderContextMenu('/alice/Documents')).disabled).toBe(false);
});
});
describe('TabFiles opening a folder being renamed', () => {
afterEach(() => vi.restoreAllMocks());
it('opens it under its new name', async () => {
mountListing(['New Folder'], '/alice/Documents');
const row = $('.files .row')[0];
row.setAttribute('data-is_dir', '1');
let finishRename;
row._pendingRename = new Promise((resolve) => {
finishRename = resolve;
});
vi.spyOn(TabFiles, 'pushNavHistory').mockImplementation(() => {});
vi.spyOn(TabFiles, 'renderDirectory').mockImplementation(async () => {});
const opening = TabFiles.openRow(row);
row.setAttribute('data-path', '/alice/Documents/Photos');
finishRename();
await opening;
expect(TabFiles.renderDirectory).toHaveBeenCalledWith('/alice/Documents/Photos');
});
it('opens it anyway when the rename never reports back', async () => {
vi.useFakeTimers();
mountListing(['Photos'], '/alice/Documents');
const row = $('.files .row')[0];
row.setAttribute('data-is_dir', '1');
row._pendingRename = new Promise(() => {});
vi.spyOn(TabFiles, 'pushNavHistory').mockImplementation(() => {});
vi.spyOn(TabFiles, 'renderDirectory').mockImplementation(async () => {});
const opening = TabFiles.openRow(row);
await vi.advanceTimersByTimeAsync(5000);
await opening;
vi.useRealTimers();
expect(TabFiles.renderDirectory).toHaveBeenCalledWith('/alice/Documents/Photos');
});
});
describe('TabFiles.folderMoved', () => {
beforeEach(() => {
TabFiles.$el_window = $('<div>');
});
it('points Back/Forward at a moved folder', () => {
window.dashboard_nav_history = ['/alice', '/alice/A', '/alice/A/sub'];
window.dashboard_nav_history_current_position = 0;
TabFiles.folderMoved('/alice/A', '/alice/Docs/A');
expect(window.dashboard_nav_history).toEqual(['/alice', '/alice/Docs/A', '/alice/Docs/A/sub']);
});
it('forgets a trashed folder, and what was inside it', () => {
window.dashboard_nav_history = ['/alice', '/alice/A', '/alice/A/sub', '/alice/B'];
window.dashboard_nav_history_current_position = 3;
TabFiles.folderMoved('/alice/A', null);
expect(window.dashboard_nav_history).toEqual(['/alice', '/alice/B']);
expect(window.dashboard_nav_history_current_position).toBe(1);
});
it('keeps the folder on screen, and folds the same folder twice in a row', () => {
window.dashboard_nav_history = ['/alice', '/alice/A', '/alice', '/alice/A'];
window.dashboard_nav_history_current_position = 2;
TabFiles.folderMoved('/alice/A', null);
expect(window.dashboard_nav_history).toEqual(['/alice']);
expect(window.dashboard_nav_history_current_position).toBe(0);
window.dashboard_nav_history = ['/alice', '/alice/A'];
window.dashboard_nav_history_current_position = 1;
TabFiles.folderMoved('/alice/A', null);
expect(window.dashboard_nav_history).toEqual(['/alice', '/alice/A']);
});
});
+2 -33
View File
@@ -26,13 +26,12 @@ import create_access_token from '../../helpers/createAccessToken.js';
import postUserProtected from '../../helpers/postUserProtected.js';
import update_username_in_gui from '../../helpers/updateUsernameInGui.js';
import UIWindowManageSessions from '../UIWindowManageSessions.js';
import UIDashboardFormDialog from './UIDashboardFormDialog.js';
import UIDashboardFormDialog, { copyRowHtml } from './UIDashboardFormDialog.js';
// Keep in sync with ProfileService on the backend.
const DISPLAY_NAME_MAX_LENGTH = 64;
const BIO_MAX_LENGTH = 280;
const TOKEN_LABEL_MAX_LENGTH = 64;
const TOKEN_COPIED_RESET_MS = 2000;
/**
* @typedef {object} AccountDialogOptions
@@ -343,43 +342,13 @@ export function openCreateTokenDialog ({ $sessionsParent, ...opts } = {}) {
const token = await create_access_token({ label: label.trim(), expiresIn: expiresIn || null });
let h = `<div class="dashboard-token-warning">${i18n('token_shown_once_warning')}</div>`;
h += '<div class="dashboard-token-row">';
h += `<input type="text" class="dashboard-dialog-input dashboard-token-value" readonly value="${html_encode(token)}" aria-label="${i18n('api_token')}" spellcheck="false">`;
h += `<button type="button" class="dashboard-dialog-btn dashboard-dialog-btn-quiet dashboard-token-copy">${i18n('copy')}</button>`;
h += '</div>';
h += copyRowHtml({ value: token, label: t('api_token'), copiedLabel: t('token_copied') });
h += `<p class="dashboard-form-hint">${i18n('token_manage_hint')} <button type="button" class="dashboard-token-sessions">${i18n('ui_manage_sessions')}</button>.</p>`;
// The token is shown only once; only Done, ✕ or Escape put it away.
dialog.finish({ html: h, describe: '.dashboard-token-warning', keepOnBackdrop: true });
},
});
let copied_timer = null;
dialog.$dialog.on('click', '.dashboard-token-copy', function () {
const $btn = $(this);
const $value = dialog.$dialog.find('.dashboard-token-value');
const copied = () => {
$btn.text(t('token_copied'));
clearTimeout(copied_timer);
copied_timer = setTimeout(() => $btn.text(t('copy')), TOKEN_COPIED_RESET_MS);
};
// No clipboard access (an insecure origin has no navigator.clipboard
// at all): select it, try the legacy copy, and leave it selected so
// the user can copy it by hand.
const by_hand = () => {
$value.get(0)?.select();
try {
if ( document.execCommand('copy') ) copied();
} catch { /* stays selected */ }
};
if ( navigator.clipboard?.writeText ) {
navigator.clipboard.writeText(String($value.val())).then(copied, by_hand);
} else {
by_hand();
}
});
dialog.$dialog.on('focus', '.dashboard-token-value', function () {
this.select();
});
dialog.$dialog.on('click', '.dashboard-token-sessions', () => {
UIWindowManageSessions({
window_options: {
@@ -275,12 +275,12 @@ describe('create API token', () => {
expect(String(fetch.mock.calls[0][0])).toBe('https://api.puter.test/auth/create-access-token');
expect(bodyOf(0)).toMatchObject({ label: 'CLI', expiresIn: '30d' });
expect($('.dashboard-token-value').val()).toBe('tok-abc');
expect($('.dashboard-copy-value').val()).toBe('tok-abc');
$('.dashboard-token-copy').trigger('click');
$('.dashboard-copy-btn').trigger('click');
await settle();
expect(writeText).toHaveBeenCalledWith('tok-abc');
expect($('.dashboard-token-copy').text()).toBe('token_copied');
expect($('.dashboard-copy-btn').text()).toBe('token_copied');
});
it('shows why minting failed', async () => {
+11 -1
View File
@@ -398,6 +398,13 @@ async function UIDashboard (options) {
});
}
// Files' Back/Forward follows a folder moved elsewhere, and forgets
// one moved to the Trash.
if ( resp.is_dir && old_path && resp.path ) {
const trashed = resp.path.startsWith(`${window.trash_path}/`);
window.dashboard_object?.folderMoved?.(old_path, trashed ? null : resp.path);
}
// Create new item at destination if user is viewing that directory
if ( window.UIDashboardFileItem ) {
window.UIDashboardFileItem(resp);
@@ -407,6 +414,7 @@ async function UIDashboard (options) {
window.socket.on('item.removed', async (item) => {
if ( item.original_client_socket_id === window.socket.id ) return;
if ( item.descendants_only ) return;
if ( item.is_dir && item.path ) window.dashboard_object?.folderMoved?.(item.path, null);
// Match by uid when present (O(1) attribute selector; uids are UUIDs,
// safe to interpolate) and fall back to a path scan for minimal
@@ -533,10 +541,12 @@ async function UIDashboard (options) {
notifications.close({ restoreFocus: false });
// What was open over the tab being left goes with it, as on a
// sidebar click: the phone drawer, a menu, and Files' Share and
// Properties, which act on its rows (their changes are already saved).
// Properties, which act on its rows (their changes are already saved),
// and its image preview.
setMobileSidebar(false);
$('.context-menu').remove();
$el_window.find('.share-modal-close, .item-props-close').trigger('click');
if ( tab !== 'files' && TabFiles.previewOpen ) TabFiles.closeImagePreview();
// Switch to correct tab
const $targetTab = $el_window.find(`.dashboard-sidebar-item[data-section="${tab}"]`);
+108 -10
View File
@@ -28,6 +28,8 @@ import { ICONS, SVG } from './UIDashboardDialog.js';
const FORM_ICONS = {
done: SVG('<circle cx="12" cy="12" r="9"/><path d="M8 12.5l2.5 2.5L16 9.5"/>'),
copy: SVG('<rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/>'),
copied: SVG('<path d="M20 6L9 17l-5-5"/>'),
show: SVG('<path d="M2 12s3.6-7 10-7 10 7 10 7-3.6 7-10 7S2 12 2 12z"/><circle cx="12" cy="12" r="3"/>'),
hide: SVG('<path d="M17.9 17.9A10.4 10.4 0 0 1 12 19c-6.4 0-10-7-10-7a18.5 18.5 0 0 1 5.1-5.9"/><path d="M9.9 5.2A9.1 9.1 0 0 1 12 5c6.4 0 10 7 10 7a18.6 18.6 0 0 1-2.2 3.2"/><path d="M14.1 14.1a3 3 0 1 1-4.2-4.2"/><path d="M2 2l20 20"/>'),
};
@@ -36,8 +38,11 @@ const FORM_ICONS = {
* @typedef {object} FormDialogField
* @property {string} name Key in `values()`.
* @property {string} label HTML the caller has already encoded.
* @property {'text'|'email'|'password'|'textarea'|'select'} [type] Defaults to text.
* @property {'text'|'email'|'password'|'textarea'|'select'|'switch'} [type] Defaults to text.
* @property {string} [value]
* @property {boolean} [checked] A switch's initial state.
* @property {string} [suffix] Plain text shown at the end of a text field, as
* in a domain after the name typed into it.
* @property {string} [placeholder] Already encoded.
* @property {string} [hint] HTML the caller has already encoded, shown under the field.
* @property {boolean} [hidden] Rendered but not shown; see `showField`.
@@ -51,18 +56,21 @@ const FORM_ICONS = {
* @typedef {object} FormDialog
* @property {jQuery} $dialog
* @property {() => void} close
* @property {() => Record<string, string>} values Every field's current value.
* @property {() => Record<string, string|boolean>} values Every field's
* current value; a switch's is a boolean.
* @property {(name: string) => jQuery} field A field's control.
* @property {(name: string, show: boolean) => void} showField
* @property {(text: string, opts?: { kind?: 'error'|'info', field?: string }) => void} setStatus
* Plain text; an empty string clears it. Naming a field marks it invalid and focuses it.
* @property {(opts: { message?: string, html?: string, describe?: string, keepOnBackdrop?: boolean }) => void} finish
* Swaps the form for a done state (a plain-text `message`, or the caller's
* encoded `html`) with a single Done button. `describe` selects the part of
* Swaps the form for a done state (a plain-text `message`, then the
* caller's encoded `html`, either or both) with a single Done button. `describe` selects the part of
* `html` the focused Done button reads out. `keepOnBackdrop` stops a stray
* click outside the card from closing it, for content shown only once.
*/
const COPIED_RESET_MS = 2000;
const attrs_html = (attrs) => Object.entries(attrs ?? {})
.filter(([, value]) => value !== undefined && value !== null && value !== false)
.map(([key, value]) => (value === true ? ` ${key}` : ` ${key}="${html_encode(String(value))}"`))
@@ -71,11 +79,28 @@ const attrs_html = (attrs) => Object.entries(attrs ?? {})
const field_html = (field, id) => {
const control_id = `${id}-${field.name}`;
const hint_id = `${control_id}-hint`;
const suffix_id = `${control_id}-suffix`;
const type = field.type ?? 'text';
const common = ` id="${control_id}" name="${html_encode(field.name)}" class="dashboard-dialog-input dashboard-form-input"`
+ (field.hint ? ` aria-describedby="${hint_id}"` : '')
// A switch is drawn by its slider, not the text-box style.
const control_class = type === 'switch' ? 'dashboard-form-input' : 'dashboard-dialog-input dashboard-form-input';
const described_by = [field.suffix && suffix_id, field.hint && hint_id].filter(Boolean).join(' ');
const common = ` id="${control_id}" name="${html_encode(field.name)}" class="${control_class}"`
+ (described_by ? ` aria-describedby="${described_by}"` : '')
+ attrs_html(field.attrs);
if ( type === 'switch' ) {
return `<div class="dashboard-form-field dashboard-form-switch-field" data-field="${html_encode(field.name)}"${field.hidden ? ' hidden' : ''}>`
+ '<div class="dashboard-form-switch-text">'
+ `<label class="dashboard-form-label" for="${control_id}">${field.label}</label>`
+ (field.hint ? `<p class="dashboard-form-hint" id="${hint_id}">${field.hint}</p>` : '')
+ '</div>'
+ '<label class="dashboard-switch">'
+ `<input type="checkbox" role="switch"${common}${field.checked ? ' checked' : ''}>`
+ '<span class="dashboard-switch-slider"></span>'
+ '</label>'
+ '</div>';
}
let control;
if ( type === 'textarea' ) {
control = `<textarea${common} placeholder="${field.placeholder ?? ''}">${html_encode(field.value ?? '')}</textarea>`;
@@ -85,7 +110,9 @@ const field_html = (field, id) => {
} else {
control = `<input type="${type}"${common} placeholder="${field.placeholder ?? ''}" value="${html_encode(field.value ?? '')}">`;
}
if ( type === 'password' ) {
if ( field.suffix ) {
control = `<div class="dashboard-form-affix">${control}<span class="dashboard-form-suffix" id="${suffix_id}">${html_encode(field.suffix)}</span></div>`;
} else if ( type === 'password' ) {
control = '<div class="dashboard-form-secret">'
+ control
+ `<button type="button" class="dashboard-form-reveal" aria-controls="${control_id}"`
@@ -100,6 +127,29 @@ const field_html = (field, id) => {
+ '</div>';
};
/**
* A read-only value with a Copy button, for a `finish` screen. The dialog
* wires the button.
*
* @param {object} opts
* @param {string} opts.value Plain text.
* @param {string} opts.label Plain text naming the value for screen readers.
* @param {string} [opts.copiedLabel] Plain text the button shows once copied.
* @param {boolean} [opts.compact] Icon buttons, leaving the value the width;
* `after` should hold `dashboard-icon-btn`s too.
* @param {string} [opts.after] HTML the caller has already encoded, after the button.
* @returns {string}
*/
export const copyRowHtml = ({ value, label, copiedLabel, compact = false, after = '' }) =>
`<div class="dashboard-copy-row${compact ? ' dashboard-copy-row-compact' : ''}">`
+ `<input type="text" class="dashboard-dialog-input dashboard-copy-value" readonly value="${html_encode(value)}" aria-label="${html_encode(label)}" spellcheck="false">`
+ `<button type="button" class="dashboard-dialog-btn dashboard-dialog-btn-quiet dashboard-copy-btn${compact ? ' dashboard-icon-btn' : ''}"`
+ ` data-copied="${html_encode(copiedLabel ?? i18n('copied', [], false))}"`
+ (compact ? ` aria-label="${i18n('copy')}" title="${i18n('copy')}">${FORM_ICONS.copy}` : `>${i18n('copy')}`)
+ '</button>'
+ after
+ '</div>';
/**
* @param {object} opts
* @param {string} opts.title Plain text.
@@ -164,7 +214,7 @@ export default function UIDashboardFormDialog ({ title, intro, tone, fields = []
values () {
const out = {};
$controls().each(function () {
out[this.name] = String($(this).val() ?? '');
out[this.name] = this.type === 'checkbox' ? this.checked : String($(this).val() ?? '');
});
return out;
},
@@ -196,10 +246,12 @@ export default function UIDashboardFormDialog ({ title, intro, tone, fields = []
},
finish ({ message, html, describe, keepOnBackdrop = false }) {
keep_on_backdrop = keepOnBackdrop;
const content = html ?? `<div class="dashboard-form-done">${FORM_ICONS.done}<p id="${id}-done">${html_encode(message ?? '')}</p></div>`;
const content = (message === undefined ? '' : `<div class="dashboard-form-done">${FORM_ICONS.done}<p id="${id}-done">${html_encode(message)}</p></div>`)
+ (html ?? '');
$dialog.attr('aria-describedby', null);
$dialog.find('.dashboard-dialog-body').html(`<div class="dashboard-dialog-content" role="status">${content}</div>`);
const described = ! html || (describe && $dialog.find('.dashboard-dialog-body').find(describe).first().attr('id', `${id}-done`).length > 0);
const described = message !== undefined
|| (describe && $dialog.find('.dashboard-dialog-body').find(describe).first().attr('id', `${id}-done`).length > 0);
// A status region that arrives with its text isn't announced;
// the focused Done button reads the message out instead.
$dialog.find('.dashboard-dialog-footer').html(
@@ -219,6 +271,52 @@ export default function UIDashboardFormDialog ({ title, intro, tone, fields = []
$dialog.on('click', '.dashboard-form-cancel, .dashboard-form-done-btn', () => close());
let copied_timer = null;
$dialog.on('click', '.dashboard-copy-btn', function () {
const $btn = $(this);
const $value = $btn.siblings('.dashboard-copy-value');
const show = (done) => {
const text = done ? $btn.attr('data-copied') : i18n('copy', [], false);
if ( $btn.hasClass('dashboard-icon-btn') ) {
$btn.attr({ 'aria-label': text, title: text }).html(done ? FORM_ICONS.copied : FORM_ICONS.copy);
} else {
$btn.text(text);
}
};
const copied = () => {
show(true);
clearTimeout(copied_timer);
copied_timer = setTimeout(() => show(false), COPIED_RESET_MS);
};
// No clipboard access (an insecure origin has no navigator.clipboard
// at all): select it, try the legacy copy, and leave it selected so
// the user can copy it by hand.
const by_hand = () => {
$value.get(0)?.select();
try {
if ( document.execCommand('copy') ) copied();
} catch { /* stays selected */ }
};
if ( navigator.clipboard?.writeText ) {
navigator.clipboard.writeText(String($value.val())).then(copied, by_hand);
} else {
by_hand();
}
});
$dialog.on('focus', '.dashboard-copy-value', function () {
this.select();
});
// The suffix is drawn inside the field's box, so a press on it types into
// the field, at the end of the name.
$dialog.on('mousedown', '.dashboard-form-suffix', function (e) {
const input = $(this).siblings('.dashboard-form-input').get(0);
if ( ! input || input.disabled ) return;
e.preventDefault();
input.focus({ preventScroll: true });
input.setSelectionRange(input.value.length, input.value.length);
});
// Keep the field focused (and a phone's keyboard up) while toggling. The
// label names the action, so the button carries no aria-pressed as well.
$dialog.on('pointerdown mousedown', '.dashboard-form-reveal', (e) => e.preventDefault());
@@ -29,7 +29,7 @@ window.html_encode = globalThis.html_encode;
window.requestAnimationFrame = (fn) => setTimeout(fn, 0);
window.matchMedia = () => ({ matches: false });
const { default: UIDashboardFormDialog } = await import('./UIDashboardFormDialog.js');
const { default: UIDashboardFormDialog, copyRowHtml } = await import('./UIDashboardFormDialog.js');
const tick = () => new Promise(resolve => setTimeout(resolve, 0));
const submit = () => $('.dashboard-form-dialog').trigger('submit');
@@ -207,6 +207,52 @@ describe('UIDashboardFormDialog', () => {
expect($('.dashboard-dialog-overlay').hasClass('dashboard-dialog-show')).toBe(false);
});
it('shows a message and the caller\'s html together, with a working copy row', async () => {
const writeText = vi.fn(async () => {});
Object.defineProperty(navigator, 'clipboard', { value: { writeText }, configurable: true });
open({ onSubmit: (dialog) => dialog.finish({
message: 'Live',
html: copyRowHtml({ value: 'https://a.example', label: 'Address' }),
}) });
submit();
await tick();
expect($('.dashboard-form-done p').text()).toBe('Live');
expect($(`#${$('.dashboard-form-done-btn').attr('aria-describedby')}`).text()).toBe('Live');
expect($('.dashboard-copy-value').val()).toBe('https://a.example');
$('.dashboard-copy-btn').trigger('click');
await tick();
expect(writeText).toHaveBeenCalledWith('https://a.example');
expect($('.dashboard-copy-btn').text()).toBe('copied');
});
it('draws a suffix inside the field and has it read with the field', () => {
open({ fields: [{ name: 'name', label: 'Address', suffix: '.puter.site <b>' }] });
const $input = $('.dashboard-form-input[name="name"]');
expect($input.closest('.dashboard-form-affix')).toHaveLength(1);
expect($(`#${$input.attr('aria-describedby')}`).text()).toBe('.puter.site <b>');
});
it('types into a suffixed field when its suffix is pressed', () => {
open({ fields: [{ name: 'name', label: 'Address', value: 'my-site', suffix: '.puter.site' }] });
const $input = $('.dashboard-form-input[name="name"]');
$input.get(0).blur();
const down = $.Event('mousedown');
$('.dashboard-form-suffix').trigger(down);
expect(down.isDefaultPrevented()).toBe(true);
expect(document.activeElement).toBe($input.get(0));
expect($input.get(0).selectionStart).toBe('my-site'.length);
});
it('reads a switch back as a boolean', () => {
const dialog = open({ fields: [
{ name: 'on', type: 'switch', label: 'On', checked: true },
{ name: 'off', type: 'switch', label: 'Off' },
] });
expect($('.dashboard-form-input[name="on"]').attr('role')).toBe('switch');
expect(dialog.values()).toEqual({ on: true, off: false });
});
it('toggles a password between hidden and shown', () => {
open();
const $input = $('.dashboard-form-input[name="secret"]');
@@ -0,0 +1,195 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
// The Files tab's publish flows, a folder as a website and a .js file as a
// worker: dashboard form dialogs rather than desktop windows.
import ensure_can_publish from '../../helpers/ensureCanPublish.js';
import { SVG } from './UIDashboardDialog.js';
import UIDashboardFormDialog, { copyRowHtml } from './UIDashboardFormDialog.js';
import { finalizePublishName, normalizePublishName, PUBLISH_NAME_RULES } from './publishName.js';
/**
* @typedef {object} PublishTarget
* @property {string} name Display name of the folder or file.
* @property {string} path
* @property {jQuery} [$container] Where the overlay mounts.
* @property {HTMLElement} [returnFocusTo] Where focus goes on close.
*/
const t = (key) => i18n(key, [], false);
const OPEN_ICON = SVG('<path d="M18 13v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/><path d="M15 3h6v6"/><path d="M10 14L21 3"/>');
/** Rows showing `path` or anything under it, on the dashboard or the desktop. */
const each_item_at = (path, fn) => $('.item').each(function () {
const item_path = this.dataset.path ?? '';
if ( item_path === path || item_path.startsWith(`${path}/`) ) fn($(this), item_path);
});
const KINDS = {
website: {
title: 'window_title_publish_website',
intro: 'publish_website_intro',
published: 'website_published',
rule: PUBLISH_NAME_RULES.website,
// Rejections that mean "pick another name" (SubdomainDriver).
nameErrors: new Set(['conflict', 'subdomain_reserved']),
suffix: () => `.${window.hosting_domain}`,
requiresVerification: () => !! window.require_email_verification_to_publish_website,
async publish (name, path) {
const site = await puter.hosting.create(name, path);
// The server's address carries the deployment's real scheme (a
// self-hosted instance may serve sites over http).
return site?.root_dir?.subdomains?.find((entry) => entry.subdomain === name)?.address
?? `https://${name}.${window.hosting_domain}`;
},
markPublished (path, url) {
each_item_at(path, ($item, item_path) => {
if ( item_path === path ) {
$item.attr('data-has_website', '1').find('.item-has-website-badge').show();
const fsentry = $item.data('fsentry');
if ( fsentry ) fsentry.has_website = true;
} else {
$item.attr('data-website_url', url + item_path.slice(path.length))
.find('.item-has-website-url-badge').show();
}
});
window.update_sites_cache?.();
},
},
worker: {
title: 'window_title_publish_worker',
intro: 'publish_worker_intro',
published: 'worker_published',
rule: PUBLISH_NAME_RULES.worker,
nameErrors: new Set(['conflict']),
suffix: () => '.puter.work',
requiresVerification: () => true,
extraFields: () => [{
name: 'sandboxed',
type: 'switch',
label: i18n('worker_sandboxed'),
hint: i18n('worker_sandboxed_hint'),
checked: true,
}],
async publish (name, path, { sandboxed }) {
const result = await puter.workers.create(name, path, { sandbox: sandboxed });
return result?.url ?? `https://${name}.puter.work`;
},
markPublished (path, url) {
each_item_at(path, ($item, item_path) => {
if ( item_path !== path ) return;
$item.attr({ 'data-is_worker': '1', 'data-worker_url': url })
.find('.item-is-worker').show();
});
},
},
};
/** puter.js rejects with an Error, or with the API's `{ error: { message } }` / `{ message }`. */
const error_message = (err) => err?.error?.message ?? err?.message ?? t('error_unknown_cause');
/**
* @param {'website'|'worker'} kind_name
* @param {PublishTarget} target
*/
const open_publish_dialog = async (kind_name, { name, path, ...opts }) => {
const kind = KINDS[kind_name];
if ( ! await ensure_can_publish(kind.requiresVerification()) ) return;
const fields = [{
name: 'name',
label: i18n('publish_address'),
value: normalizePublishName(window.generate_identifier(), kind.rule),
suffix: kind.suffix(),
attrs: {
maxlength: kind.rule.maxLength,
autocomplete: 'off',
autocapitalize: 'off',
autocorrect: 'off',
spellcheck: 'false',
'data-gramm_editor': 'false',
},
}];
fields.push(...(kind.extraFields?.() ?? []));
const dialog = UIDashboardFormDialog({
...opts,
title: t(kind.title),
intro: `<p>${i18n(kind.intro, `<strong>${html_encode(name)}</strong>`, false)}</p>`,
submitLabel: t('publish'),
fields,
async onSubmit () {
const values = dialog.values();
const subdomain = finalizePublishName(values.name, kind.rule);
dialog.field('name').val(subdomain);
if ( ! subdomain ) {
dialog.setStatus(t('publish_name_required'), { field: 'name' });
return;
}
let url;
try {
url = await kind.publish(subdomain, path, values);
} catch ( err ) {
const code = err?.error?.code ?? err?.code;
dialog.setStatus(error_message(err), kind.nameErrors.has(code) ? { field: 'name' } : undefined);
return;
}
kind.markPublished(path, url);
const open_link = `<a class="dashboard-dialog-btn dashboard-dialog-btn-quiet dashboard-icon-btn dashboard-publish-open" href="${html_encode(url)}"`
+ ` target="_blank" rel="noopener noreferrer" aria-label="${i18n('open_in_new_tab')}" title="${i18n('open_in_new_tab')}">${OPEN_ICON}</a>`;
dialog.finish({
message: t(kind.published),
html: copyRowHtml({ value: url, label: t('publish_address'), compact: true, after: open_link }),
});
},
});
// Shape the name as it's typed, keeping the caret where it was.
const el_name = dialog.field('name').get(0);
if ( document.activeElement === el_name ) el_name.select();
const shape = () => {
const next = normalizePublishName(el_name.value, kind.rule);
if ( next === el_name.value ) return;
const caret = normalizePublishName(el_name.value.slice(0, el_name.selectionStart ?? el_name.value.length), kind.rule).length;
el_name.value = next;
el_name.setSelectionRange(caret, caret);
};
// Rewriting the text mid-composition (an IME, or a dead key's accent)
// breaks it; the composed text is shaped once it's in.
$(el_name).on('input', (e) => {
if ( ! e.originalEvent?.isComposing ) shape();
}).on('compositionend', shape);
return dialog;
};
/**
* Publishes a folder as a website, once the account may.
*
* @param {PublishTarget} target
*/
export const openPublishWebsiteDialog = (target) => open_publish_dialog('website', target);
/**
* Deploys a .js file as a worker, once the account may.
*
* @param {PublishTarget} target
*/
export const openPublishWorkerDialog = (target) => open_publish_dialog('worker', target);
@@ -0,0 +1,204 @@
// @vitest-environment jsdom
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { beforeEach, describe, expect, it, vi } from 'vitest';
import jQuery from '../../lib/jquery-3.6.1/jquery-3.6.1.min.js';
vi.mock('../../helpers/ensureCanPublish.js', () => ({ default: vi.fn(async () => true) }));
globalThis.$ = jQuery;
globalThis.jQuery = jQuery;
globalThis.i18n = (key, replacements) => (Array.isArray(replacements) || replacements === undefined ? key : `${key}:${replacements}`);
globalThis.html_encode = (value) => String(value ?? '')
.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;');
window.html_encode = globalThis.html_encode;
window.requestAnimationFrame = (fn) => setTimeout(fn, 0);
window.matchMedia = () => ({ matches: false });
const { openPublishWebsiteDialog, openPublishWorkerDialog } = await import('./UIPublishDialogs.js');
const { default: ensure_can_publish } = await import('../../helpers/ensureCanPublish.js');
const settle = () => new Promise(resolve => setTimeout(resolve, 0));
const nameField = () => $('.dashboard-form-input[name="name"]');
const status = () => $('.dashboard-form-status').text();
const submitWith = async (name) => {
if ( name !== undefined ) nameField().val(name);
$('.dashboard-form-dialog').trigger('submit');
for ( let i = 0; i < 5; i++ ) await settle();
};
const addRow = (path) => {
$('body').append(`<div class="item" data-path="${path}" data-has_website="0" data-is_worker="0">`
+ '<img class="item-has-website-badge" style="display:none"><img class="item-has-website-url-badge" style="display:none">'
+ '<img class="item-is-worker" style="display:none"></div>');
return $('.item').last();
};
beforeEach(() => {
document.body.innerHTML = '';
ensure_can_publish.mockClear();
ensure_can_publish.mockResolvedValue(true);
window.hosting_domain = 'puter.site';
window.generate_identifier = () => 'clever-idea-42';
window.update_sites_cache = vi.fn();
window.require_email_verification_to_publish_website = false;
globalThis.puter = {
hosting: { create: vi.fn(async () => ({})) },
workers: { create: vi.fn(async (name) => ({ success: true, url: `https://${name}.puter.work` })) },
};
});
describe('publish website', () => {
it('names the folder and suggests an address on the hosting domain', async () => {
await openPublishWebsiteDialog({ name: 'Site <1>', path: '/alice/Site' });
expect($('.dashboard-dialog-message').html()).toContain('<strong>Site &lt;1&gt;</strong>');
expect(nameField().val()).toBe('clever-idea-42');
expect($('.dashboard-form-suffix').text()).toBe('.puter.site');
});
it('publishes the folder, shows the live address, and badges its rows', async () => {
const $folder = addRow('/alice/Site');
const $child = addRow('/alice/Site/index.html');
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
await submitWith(' -My Site- ');
expect(puter.hosting.create).toHaveBeenCalledWith('my-site', '/alice/Site');
expect($('.dashboard-form-done p').text()).toBe('website_published');
expect($('.dashboard-copy-value').val()).toBe('https://my-site.puter.site');
expect($('.dashboard-publish-open').attr('href')).toBe('https://my-site.puter.site');
const writeText = vi.fn(async () => {});
Object.defineProperty(navigator, 'clipboard', { value: { writeText }, configurable: true });
$('.dashboard-copy-btn').trigger('click');
await settle();
expect(writeText).toHaveBeenCalledWith('https://my-site.puter.site');
expect($('.dashboard-copy-btn').attr('aria-label')).toBe('copied');
expect($folder.attr('data-has_website')).toBe('1');
expect($folder.find('.item-has-website-badge').css('display')).not.toBe('none');
expect($child.attr('data-website_url')).toBe('https://my-site.puter.site/index.html');
expect(window.update_sites_cache).toHaveBeenCalled();
});
it("links to the server's address for the site when it sends one", async () => {
puter.hosting.create.mockResolvedValueOnce({
subdomain: 'docs',
root_dir: { subdomains: [{ subdomain: 'docs', address: 'http://docs.site.example.test' }] },
});
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
await submitWith('docs');
expect($('.dashboard-copy-value').val()).toBe('http://docs.site.example.test');
expect($('.dashboard-publish-open').attr('href')).toBe('http://docs.site.example.test');
});
it('shapes the address as it is typed', async () => {
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
nameField().val('My_Site 2!').trigger('input');
expect(nameField().val()).toBe('my-site-2');
});
it('leaves text being composed alone and shapes it once composed', async () => {
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
nameField().val('caf\u00B4');
nameField().get(0).dispatchEvent(new InputEvent('input', { bubbles: true, isComposing: true }));
expect(nameField().val()).toBe('caf\u00B4');
nameField().val('café').trigger('compositionend');
expect(nameField().val()).toBe('cafe');
});
it('asks for an address when nothing usable was typed', async () => {
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
await submitWith('---');
expect(status()).toBe('publish_name_required');
expect(nameField().attr('aria-invalid')).toBe('true');
expect(puter.hosting.create).not.toHaveBeenCalled();
});
it('shows why publishing failed and lets the user try again', async () => {
puter.hosting.create.mockRejectedValueOnce({ error: { code: 'internal_error', message: 'Something broke' } });
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
await submitWith('fine');
expect(status()).toBe('Something broke');
expect(nameField().prop('disabled')).toBe(false);
expect(nameField().attr('aria-invalid')).toBeUndefined();
});
it.each([
['conflict', 'A site with this subdomain already exists'],
['subdomain_reserved', "Subdomain 'www' is reserved"],
])('points a %s rejection at the address field', async (code, message) => {
puter.hosting.create.mockRejectedValueOnce({ error: { code, message }, code, message });
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
await submitWith('www');
expect(status()).toBe(message);
expect(nameField().attr('aria-invalid')).toBe('true');
expect(document.activeElement).toBe(nameField().get(0));
});
it('asks for verification only when the deployment needs it', async () => {
window.require_email_verification_to_publish_website = true;
ensure_can_publish.mockResolvedValueOnce(false);
await openPublishWebsiteDialog({ name: 'Site', path: '/alice/Site' });
expect(ensure_can_publish).toHaveBeenCalledWith(true);
expect($('.dashboard-form-dialog')).toHaveLength(0);
});
});
describe('publish worker', () => {
it('always needs a verified account', async () => {
await openPublishWorkerDialog({ name: 'api.js', path: '/alice/api.js' });
expect(ensure_can_publish).toHaveBeenCalledWith(true);
});
it('deploys sandboxed by default and badges the file', async () => {
const $file = addRow('/alice/api.js');
await openPublishWorkerDialog({ name: 'api.js', path: '/alice/api.js' });
expect($('.dashboard-form-suffix').text()).toBe('.puter.work');
await submitWith('my_api');
expect(puter.workers.create).toHaveBeenCalledWith('my_api', '/alice/api.js', { sandbox: true });
expect($('.dashboard-copy-value').val()).toBe('https://my_api.puter.work');
expect($file.attr('data-is_worker')).toBe('1');
expect($file.attr('data-worker_url')).toBe('https://my_api.puter.work');
});
it('deploys unsandboxed when the switch is off', async () => {
await openPublishWorkerDialog({ name: 'api.js', path: '/alice/api.js' });
$('.dashboard-form-input[name="sandboxed"]').prop('checked', false);
await submitWith('api');
expect(puter.workers.create).toHaveBeenCalledWith('api', '/alice/api.js', { sandbox: false });
});
it('shows a failed build in the status line', async () => {
puter.workers.create.mockRejectedValueOnce(new Error('SyntaxError: Unexpected token'));
await openPublishWorkerDialog({ name: 'api.js', path: '/alice/api.js' });
await submitWith('api');
expect(status()).toBe('SyntaxError: Unexpected token');
expect(nameField().attr('aria-invalid')).toBeUndefined();
});
it('points a taken worker name at the address field', async () => {
puter.workers.create.mockRejectedValueOnce({ error: { code: 'conflict', message: 'Worker name is already in use' } });
await openPublishWorkerDialog({ name: 'api.js', path: '/alice/api.js' });
await submitWith('api');
expect(status()).toBe('Worker name is already in use');
expect(nameField().attr('aria-invalid')).toBe('true');
});
});
@@ -14,9 +14,6 @@
* GNU Affero General Public License for more details.
*/
import { describe, it, expect } from 'vitest';
import { dedupedName } from './dedupedName.js';
import { describe, expect, it } from 'vitest';
import { middleTruncate } from './middleTruncate.js';
+68
View File
@@ -0,0 +1,68 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
// The names a website or worker can be published under, as the backend
// checks them: a website's is a DNS label (SubdomainDriver), and a worker's
// also allows underscores (WorkerDriver).
/**
* @typedef {object} PublishNameRule
* @property {RegExp} separators Runs that become one hyphen, as between words.
* @property {RegExp} invalid Characters the name can't hold.
* @property {number} maxLength
*/
// Either name becomes a hostname label, which DNS caps at 63; the backend
// takes a 64-character website name, but nothing can reach it.
const LABEL_MAX_LENGTH = 63;
/** @type {{ website: PublishNameRule, worker: PublishNameRule }} */
export const PUBLISH_NAME_RULES = {
// A website name can't hold an underscore, so it separates words too.
website: { separators: /[\s_]+/g, invalid: /[^a-z0-9-]/g, maxLength: LABEL_MAX_LENGTH },
worker: { separators: /\s+/g, invalid: /[^a-z0-9_-]/g, maxLength: LABEL_MAX_LENGTH },
};
/**
* What a typed name becomes as the user types: lowercase, accents taken off
* their letters, separator runs turned into a hyphen, and anything else the
* rule refuses dropped.
*
* @param {string} value
* @param {PublishNameRule} rule
* @returns {string}
*/
export const normalizePublishName = (value, rule) => String(value ?? '')
.toLowerCase()
.normalize('NFKD')
.replace(/\p{M}/gu, '')
.replace(rule.separators, '-')
.replace(rule.invalid, '');
/**
* The name to submit: normalized, without the leading or trailing hyphens a
* hostname can't start or end with. Empty when nothing usable is left.
*
* @param {string} value
* @param {PublishNameRule} rule
* @returns {string}
*/
export const finalizePublishName = (value, rule) => normalizePublishName(value, rule)
.slice(0, rule.maxLength)
.replace(/^-+|-+$/g, '');
@@ -0,0 +1,64 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { finalizePublishName, normalizePublishName, PUBLISH_NAME_RULES } from './publishName.js';
const { website, worker } = PUBLISH_NAME_RULES;
describe('normalizePublishName', () => {
it('lowercases and turns whitespace runs into one hyphen', () => {
expect(normalizePublishName('My Cool Site', website)).toBe('my-cool-site');
});
it('drops characters a website name cannot hold', () => {
expect(normalizePublishName('my site!.v2', website)).toBe('my-sitev2');
});
it('keeps the letter under an accent', () => {
expect(normalizePublishName('Café Ünïcode', website)).toBe('cafe-unicode');
expect(normalizePublishName('Ñandú_API', worker)).toBe('nandu_api');
});
it('turns underscores into hyphens in a website name', () => {
expect(normalizePublishName('My_Cool _Site', website)).toBe('my-cool-site');
});
it('keeps underscores in a worker name', () => {
expect(normalizePublishName('My_API', worker)).toBe('my_api');
});
it('keeps hyphens at the ends while the user is still typing', () => {
expect(normalizePublishName('my-', website)).toBe('my-');
});
});
describe('finalizePublishName', () => {
it('trims hyphens from both ends', () => {
expect(finalizePublishName(' -my site- ', website)).toBe('my-site');
});
it('is empty when nothing usable is left', () => {
expect(finalizePublishName('!!! ', website)).toBe('');
});
it('caps the length at a hostname label\'s 63', () => {
expect(finalizePublishName('a'.repeat(80), website)).toHaveLength(63);
expect(finalizePublishName('a'.repeat(80), worker)).toHaveLength(63);
});
});
+57
View File
@@ -0,0 +1,57 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* @typedef {Object} SortKey
* @property {string} name - The name the row shows
* @property {number} [size]
* @property {number} [modified]
*/
// One collator for every comparison: localeCompare builds one per call.
const collator = new Intl.Collator(undefined, { numeric: true, sensitivity: 'base' });
/**
* Orders names as a file manager does: case-insensitively, with runs of
* digits compared by value ("file2" before "file10"). Names equal but for
* case or accents still get a fixed order.
*
* @param {string} a
* @param {string} b
* @returns {number}
*/
export function compareNames (a, b) {
return collator.compare(a, b) || (a < b ? -1 : a > b ? 1 : 0);
}
/**
* Orders two listing entries by `column`, ascending. Equal sizes or dates
* fall back to the name, so their order is the same on every listing.
*
* @param {SortKey} a
* @param {SortKey} b
* @param {'name'|'size'|'modified'} column
* @returns {number}
*/
export function compareEntries (a, b, column) {
let comparison = 0;
if ( column === 'size' ) comparison = (a.size || 0) - (b.size || 0);
else if ( column === 'modified' ) comparison = (a.modified || 0) - (b.modified || 0);
return comparison || compareNames(a.name, b.name);
}
@@ -0,0 +1,62 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*/
import { describe, it, expect } from 'vitest';
import { compareEntries, compareNames } from './sortEntries.js';
const sortNames = names => [...names].sort(compareNames);
describe('compareNames', () => {
it('orders runs of digits by value', () => {
expect(sortNames(['file10.txt', 'file2.txt', 'file1.txt'])).toEqual(['file1.txt', 'file2.txt', 'file10.txt']);
expect(sortNames(['IMG_100', 'IMG_9', 'IMG_10'])).toEqual(['IMG_9', 'IMG_10', 'IMG_100']);
});
it('ignores case', () => {
expect(sortNames(['beta', 'Alpha', 'gamma'])).toEqual(['Alpha', 'beta', 'gamma']);
});
it('still orders names that differ only in case', () => {
expect(compareNames('a', 'A')).not.toBe(0);
expect(Math.sign(compareNames('a', 'A'))).toBe(-Math.sign(compareNames('A', 'a')));
});
});
describe('compareEntries', () => {
it('breaks a tie in size or date by name', () => {
const entries = [
{ name: 'zeta', size: 0, modified: 5 },
{ name: 'alpha', size: 0, modified: 5 },
{ name: 'dir10', size: 0, modified: 5 },
{ name: 'dir2', size: 0, modified: 5 },
];
const expected = ['alpha', 'dir2', 'dir10', 'zeta'];
expect([...entries].sort((a, b) => compareEntries(a, b, 'size')).map(e => e.name)).toEqual(expected);
expect([...entries].sort((a, b) => compareEntries(a, b, 'modified')).map(e => e.name)).toEqual(expected);
});
it('orders by the column before the name', () => {
const big = { name: 'a', size: 20, modified: 1 };
const small = { name: 'b', size: 10, modified: 2 };
expect(compareEntries(big, small, 'size')).toBeGreaterThan(0);
expect(compareEntries(big, small, 'modified')).toBeLessThan(0);
expect(compareEntries(big, small, 'name')).toBeLessThan(0);
});
it('reads a missing size or date as 0', () => {
expect(compareEntries({ name: 'a' }, { name: 'b', size: 1 }, 'size')).toBeLessThan(0);
});
});
+4
View File
@@ -712,6 +712,10 @@ export async function get_permission_description (permission, options = {}) {
return { html: i18n('perm_events_background'), icon: 'zap' };
}
if ( permission === 'custom-domains:manage' ) {
return { html: i18n('perm_custom_domains_manage'), icon: 'globe' };
}
const parts = split_permission(permission);
// The grant route refuses any other spelling of the same parts.
if ( parts.map(escape_permission_component).join(':') !== permission ) return null;
@@ -0,0 +1,456 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
let attach_modal_seq = 0;
/** API error code → message key; anything unlisted gets the generic error. */
const ERROR_KEYS = {
invalid_hostname: 'custom_domain_error_invalid',
custom_domain_taken: 'custom_domain_error_taken',
custom_domain_limit_reached: 'custom_domain_error_limit',
custom_domain_rejected: 'custom_domain_error_rejected',
site_not_found: 'custom_domain_error_site',
custom_domain_not_found: 'custom_domain_error_not_found',
custom_domains_not_enabled: 'custom_domain_not_enabled',
custom_domains_permission_required: 'custom_domain_error_generic',
};
const format_price = (pricing) => {
if ( ! pricing || typeof pricing.amount !== 'number' ) return null;
try {
return new Intl.NumberFormat(undefined, {
style: 'currency',
currency: pricing.currency || 'USD',
minimumFractionDigits: 2,
}).format(pricing.amount);
} catch {
return null;
}
};
/**
* "Connect a custom domain to this site" modal, behind
* `puter.hosting.attach()`. The user picks one of their domains (moving it
* from another site if need be) or adds one; the caller learns only the
* domain that ends up chosen.
*
* Built on the app feedback modal's shell (`.app-feedback-*`) so it renders
* the same on the desktop and in the standalone popup. Talks to the
* `/custom-domains` API with the GUI's own session: the requesting app never
* holds a token that can manage domains.
*
* @param {{ subdomain: string, requester?: string }} options `requester` is
* the app title or opener origin shown to the user, as the GUI resolved it.
* @returns {Promise<object|null>} The chosen domain as the API returns it, or
* null when the user closes the dialog first. Never rejects.
*/
async function UIWindowAttachCustomDomain (options) {
options = options ?? {};
const subdomain = String(options.subdomain ?? '');
const site_host = `${subdomain}.${window.hosting_domain}`;
return new Promise((resolve) => {
const modal_id = `custom-domain-${++attach_modal_seq}`;
const authToken = puter.authToken ?? window.auth_token;
let busy = false;
let closed = false;
// A domain added in this dialog is the answer even if the user then
// closes it: it exists, and is billed, either way.
let chosen = null;
const titleId = `${modal_id}-title`;
const h = `
<div class="app-feedback-overlay custom-domain-overlay" role="dialog" aria-modal="true" aria-labelledby="${titleId}">
<div class="app-feedback-modal custom-domain-modal" tabindex="-1">
<div class="app-feedback-head">
<h2 class="app-feedback-title" id="${titleId}">${i18n('custom_domain_title')}</h2>
<button type="button" class="app-feedback-x" aria-label="${html_encode(i18n('close'))}" title="${html_encode(i18n('close'))}">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" aria-hidden="true"><line x1="6" y1="6" x2="18" y2="18"/><line x1="18" y1="6" x2="6" y2="18"/></svg>
</button>
</div>
<div class="app-feedback-body">
<p class="custom-domain-intro">${i18n('custom_domain_intro', [site_host])}</p>
${options.requester ? `<p class="custom-domain-requester">${i18n('custom_domain_requested_by', [options.requester])}</p>` : ''}
<div class="app-feedback-loading custom-domain-pane" data-pane="loading">${i18n('loading')}…</div>
<div class="custom-domain-pane" data-pane="unavailable" style="display:none;">
<p class="app-feedback-unavailable-message"></p>
<div class="app-feedback-actions">
<button type="button" class="app-feedback-btn app-feedback-btn-primary custom-domain-close">${i18n('close')}</button>
</div>
</div>
<div class="custom-domain-pane" data-pane="list" style="display:none;">
<ul class="custom-domain-list"></ul>
<p class="custom-domain-empty" style="display:none;">${i18n('custom_domain_none_yet')}</p>
<p class="app-feedback-error" role="alert" style="display:none;"></p>
<div class="app-feedback-actions">
<button type="button" class="app-feedback-btn custom-domain-close">${i18n('cancel')}</button>
<button type="button" class="app-feedback-btn app-feedback-btn-primary custom-domain-to-add">${i18n('custom_domain_add_new')}</button>
</div>
</div>
<form class="custom-domain-pane" data-pane="add" style="display:none;">
<label class="custom-domain-label" for="${modal_id}-hostname">${i18n('custom_domain_hostname_label')}</label>
<input type="text" class="custom-domain-input" id="${modal_id}-hostname" placeholder="www.example.com" autocomplete="off" autocapitalize="off" spellcheck="false">
<p class="app-feedback-note custom-domain-price" style="display:none;"></p>
<div class="custom-domain-claim" style="display:none;">
<p class="app-feedback-note">${i18n('custom_domain_claim_c2a')}</p>
<table class="custom-domain-records">
<thead><tr><th>${i18n('custom_domain_record_type')}</th><th>${i18n('name')}</th><th>${i18n('custom_domain_record_value')}</th></tr></thead>
<tbody><tr>
<td>TXT</td>
<td><code class="custom-domain-copy custom-domain-claim-name"></code></td>
<td><code class="custom-domain-copy custom-domain-claim-value"></code></td>
</tr></tbody>
</table>
</div>
<p class="app-feedback-error" role="alert" style="display:none;"></p>
<div class="app-feedback-actions">
<button type="button" class="app-feedback-btn custom-domain-back">${i18n('back')}</button>
<button type="submit" class="app-feedback-btn app-feedback-btn-primary custom-domain-add" disabled>${i18n('custom_domain_add')}</button>
</div>
</form>
<div class="custom-domain-pane" data-pane="records" style="display:none;">
<p class="custom-domain-status"></p>
<p class="app-feedback-note custom-domain-records-c2a">${i18n('custom_domain_records_c2a')}</p>
<table class="custom-domain-records">
<thead><tr><th>${i18n('custom_domain_record_type')}</th><th>${i18n('name')}</th><th>${i18n('custom_domain_record_value')}</th></tr></thead>
<tbody></tbody>
</table>
<p class="app-feedback-note custom-domain-apex" style="display:none;">${i18n('custom_domain_apex_hint')}</p>
<p class="app-feedback-error" role="alert" style="display:none;"></p>
<div class="app-feedback-actions">
<button type="button" class="app-feedback-btn custom-domain-check">${i18n('custom_domain_check_status')}</button>
<button type="button" class="app-feedback-btn app-feedback-btn-primary custom-domain-done">${i18n('done')}</button>
</div>
</div>
</div>
</div>
</div>
`;
const $overlay = $(h);
$('body').append($overlay);
requestAnimationFrame(() => $overlay.addClass('app-feedback-open'));
const close = () => {
if ( closed ) return;
closed = true;
$(document).off(`keydown.${modal_id}`);
$overlay.removeClass('app-feedback-open');
setTimeout(() => $overlay.remove(), 160);
resolve(chosen);
};
const finish = (domain) => {
chosen = domain;
close();
};
const showPane = (pane) => {
$overlay.find('.custom-domain-pane').hide();
$overlay.find('.app-feedback-error').hide();
$overlay.find(`.custom-domain-pane[data-pane="${pane}"]`).show();
};
// Plain strings into `.text()`: i18n's HTML encoding would show up
// literally.
const text = (key) => i18n(key, [], false);
const showUnavailable = (key) => {
$overlay.find('.app-feedback-unavailable-message').text(text(key));
showPane('unavailable');
};
const showError = (pane, message) => {
$overlay.find(`.custom-domain-pane[data-pane="${pane}"] .app-feedback-error`)
.text(message).show();
};
const setBusy = (value) => {
busy = value;
$overlay.find('button').prop('disabled', value);
if ( ! value ) updateAddButton();
};
const api = async (method, path = '', body) => {
const res = await fetch(`${window.api_origin}/custom-domains${path}`, {
method,
headers: {
'Authorization': `Bearer ${authToken}`,
...(body ? { 'Content-Type': 'application/json' } : {}),
},
body: body ? JSON.stringify(body) : undefined,
});
let data = null;
try {
data = await res.json();
} catch {
// Non-JSON body; callers go by the status.
}
return { status: res.status, ok: res.ok, data };
};
// A known code gets our wording; anything else shows what the server
// said, which beats a generic message when something is misconfigured.
const errorFor = (res) => {
if ( res.status === 402 ) return text('custom_domain_error_plan');
if ( res.status === 429 ) return text('custom_domain_error_rate_limited');
const key = ERROR_KEYS[res.data?.code];
if ( key ) return text(key);
if ( typeof res.data?.message === 'string' && res.data.message ) {
return res.data.message;
}
return `${text('custom_domain_error_generic')} (HTTP ${res.status})`;
};
// Escape, the backdrop and the close buttons all dismiss, except
// while a request is in flight: closing then would report no domain
// for one that may still land server-side.
$(document).on(`keydown.${modal_id}`, (e) => {
if ( e.key === 'Escape' && ! busy ) close();
});
$overlay.on('mousedown', (e) => {
// Same reason as the feedback modal: keep the press from focusing
// the app window underneath.
window.mouseover_window = undefined;
if ( e.target === $overlay.get(0) && ! busy ) close();
});
$overlay.find('.app-feedback-x, .custom-domain-close').on('click', () => {
if ( ! busy ) close();
});
// -- List ---------------------------------------------------------
const renderList = (items) => {
const $list = $overlay.find('.custom-domain-list').empty();
$overlay.find('.custom-domain-empty').toggle(items.length === 0);
for ( const item of items ) {
const elsewhere = item.site !== subdomain;
const $row = $(`
<li class="custom-domain-row">
<div class="custom-domain-row-main">
<div class="custom-domain-row-host"></div>
<div class="custom-domain-row-meta"></div>
</div>
<button type="button" class="app-feedback-btn custom-domain-use">${i18n('custom_domain_use')}</button>
</li>
`);
$row.find('.custom-domain-row-host').text(item.hostname);
$row.find('.custom-domain-row-meta').text(elsewhere
? i18n('custom_domain_serves', [`${item.site}.${window.hosting_domain}`], false)
: i18n('custom_domain_serves_this', [], false));
$row.find('.custom-domain-use').on('click', () => use(item, $row));
$list.append($row);
}
};
const use = async (item, $row) => {
if ( busy ) return;
if ( item.site === subdomain ) {
finish(item);
return;
}
// Moving takes the domain off another site, so say so first.
if ( ! $row.hasClass('custom-domain-confirming') ) {
$overlay.find('.custom-domain-confirming').removeClass('custom-domain-confirming')
.find('.custom-domain-use').text(text('custom_domain_use'));
$row.addClass('custom-domain-confirming');
$row.find('.custom-domain-row-meta').text(i18n('custom_domain_move_confirm', [
`${item.site}.${window.hosting_domain}`,
site_host,
], false));
$row.find('.custom-domain-use').text(text('custom_domain_move'));
return;
}
setBusy(true);
try {
const res = await api('PATCH', `/${encodeURIComponent(item.uuid)}`, { site: subdomain });
if ( ! res.ok ) {
showError('list', errorFor(res));
return;
}
finish(res.data);
} catch ( e ) {
console.error('custom-domain: move failed', e);
showError('list', text('custom_domain_error_generic'));
} finally {
setBusy(false);
}
};
// -- Add ----------------------------------------------------------
const $input = $overlay.find('.custom-domain-input');
const updateAddButton = () => {
$overlay.find('.custom-domain-add').prop('disabled', busy || String($input.val() || '').trim() === '');
};
// A claim is for one hostname; editing it starts over.
const hideClaim = () => {
$overlay.find('.custom-domain-claim').hide();
$overlay.find('.custom-domain-add').text(text('custom_domain_add'));
};
const showClaim = (record) => {
$overlay.find('.custom-domain-claim-name').text(record.name);
$overlay.find('.custom-domain-claim-value').text(record.value);
$overlay.find('.custom-domain-claim').show();
$overlay.find('.custom-domain-add').text(text('custom_domain_verify_and_add'));
};
$input.on('input', () => {
hideClaim();
updateAddButton();
});
$overlay.find('.custom-domain-claim .custom-domain-copy')
.attr('title', text('copy'))
.on('click', function () {
navigator.clipboard?.writeText($(this).text()).catch(() => {});
});
$overlay.find('.custom-domain-to-add').on('click', () => {
showPane('add');
$input.get(0)?.focus({ preventScroll: true });
});
$overlay.find('.custom-domain-back').on('click', () => {
if ( ! busy ) showPane('list');
});
$overlay.find('form[data-pane="add"]').on('submit', async (e) => {
e.preventDefault();
const hostname = String($input.val() || '').trim();
if ( ! hostname || busy ) return;
setBusy(true);
try {
const res = await api('POST', '', { site: subdomain, hostname });
if ( res.data?.code === 'custom_domain_claim_required' && res.data.verification ) {
// Shown the first time; a retry that still can't see the
// record says so.
const retry = $overlay.find('.custom-domain-claim').is(':visible');
showClaim(res.data.verification);
if ( retry ) showError('add', text('custom_domain_claim_not_found'));
return;
}
if ( ! res.ok ) {
showError('add', errorFor(res));
return;
}
hideClaim();
chosen = res.data;
renderRecords(res.data);
showPane('records');
} catch ( err ) {
console.error('custom-domain: add failed', err);
showError('add', text('custom_domain_error_generic'));
} finally {
setBusy(false);
}
});
// -- Records --------------------------------------------------------
const renderRecords = (domain) => {
const key = domain.status === 'active'
? 'custom_domain_status_active'
: domain.status === 'failed'
? 'custom_domain_status_failed'
: 'custom_domain_status_pending';
const $status = $overlay.find('.custom-domain-status')
.html(i18n(key, [domain.hostname]));
if ( domain.statusDetail && domain.status !== 'active' ) {
$status.append($('<span class="custom-domain-status-detail"></span>').text(domain.statusDetail));
}
$overlay.find('.custom-domain-apex').toggle(domain.isApex === true);
const $tbody = $overlay.find('.custom-domain-records tbody').empty();
for ( const record of domain.records ?? [] ) {
const $tr = $(`
<tr>
<td class="custom-domain-record-type"></td>
<td><code class="custom-domain-copy"></code></td>
<td><code class="custom-domain-copy"></code></td>
</tr>
`);
$tr.find('.custom-domain-record-type').text(record.required
? record.type
: `${record.type} ${i18n('custom_domain_optional', [], false)}`);
const [$name, $value] = $tr.find('code').toArray().map((el) => $(el));
$name.text(record.name);
$value.text(record.value);
$tbody.append($tr);
}
$tbody.find('.custom-domain-copy').attr('title', i18n('copy', [], false)).on('click', function () {
navigator.clipboard?.writeText($(this).text()).catch(() => {});
});
};
$overlay.find('.custom-domain-check').on('click', async () => {
if ( busy || ! chosen ) return;
setBusy(true);
try {
const res = await api('GET', `/${encodeURIComponent(chosen.uuid)}`);
if ( ! res.ok ) {
showError('records', errorFor(res));
return;
}
// A domain that is no longer pending comes back with only its
// routing record; keep the full set shown when it was added.
chosen = { ...res.data, records: res.data.records?.length > 1 ? res.data.records : chosen.records };
renderRecords(chosen);
} catch ( e ) {
console.error('custom-domain: status check failed', e);
showError('records', text('custom_domain_error_generic'));
} finally {
setBusy(false);
}
});
$overlay.find('.custom-domain-done').on('click', () => {
if ( ! busy ) close();
});
// -- Load -------------------------------------------------------------
(async () => {
try {
const res = await api('GET');
if ( closed ) return;
// No custom domain API on this Puter: say so rather than show
// an empty list the user can't add to.
if ( res.status === 404 ) {
showUnavailable('custom_domain_unavailable');
return;
}
if ( res.data?.code === 'custom_domains_not_enabled' ) {
showUnavailable('custom_domain_not_enabled');
return;
}
if ( ! res.ok ) {
showUnavailable('custom_domain_error_generic');
return;
}
renderList(res.data?.items ?? []);
const price = format_price(res.data?.pricing);
if ( price ) {
$overlay.find('.custom-domain-price').text(i18n('custom_domain_price_note', [price], false)).show();
}
showPane('list');
} catch ( e ) {
console.error('custom-domain: list failed', e);
if ( ! closed ) showUnavailable('custom_domain_error_generic');
}
})();
});
}
export default UIWindowAttachCustomDomain;
+266 -5
View File
@@ -1658,6 +1658,19 @@ input.myapps-group-name:focus {
font-size: 14px;
color: var(--dashboard-text-secondary);
}
.myapps-modal-option {
display: flex;
align-items: center;
gap: 8px;
margin: 0 0 16px;
font-size: 14px;
color: var(--dashboard-text-primary);
cursor: pointer;
}
.myapps-modal-option input {
margin: 0;
cursor: pointer;
}
.myapps-modal-buttons {
display: flex;
justify-content: flex-end;
@@ -1993,6 +2006,136 @@ input.myapps-group-name:focus {
}
}
/* Custom domain modal (puter.hosting.attach), on the app feedback shell. */
.custom-domain-modal {
width: min(520px, 100%);
}
.custom-domain-intro,
.custom-domain-requester {
margin: 0 0 12px;
font-size: 14px;
line-height: 1.45;
word-break: break-word;
}
.custom-domain-requester {
margin-top: -6px;
font-size: 12px;
color: var(--afb-muted);
}
.custom-domain-list {
list-style: none;
margin: 0;
padding: 0;
border: 1px solid var(--afb-border);
border-radius: 10px;
overflow: hidden;
}
.custom-domain-list:empty {
display: none;
}
.custom-domain-row {
display: flex;
align-items: center;
gap: 12px;
padding: 10px 12px;
}
.custom-domain-row + .custom-domain-row {
border-top: 1px solid var(--afb-border);
}
.custom-domain-row-main {
flex: 1;
min-width: 0;
}
.custom-domain-row-host {
font-size: 14px;
font-weight: 600;
word-break: break-all;
}
.custom-domain-row-meta {
font-size: 12px;
color: var(--afb-muted);
margin-top: 2px;
}
.custom-domain-confirming .custom-domain-row-meta {
color: var(--afb-fg);
}
.custom-domain-row .app-feedback-btn {
flex: none;
padding: 6px 14px;
}
.custom-domain-empty {
margin: 4px 0 0;
font-size: 14px;
color: var(--afb-muted);
}
.custom-domain-label {
display: block;
font-size: 13px;
font-weight: 500;
margin-bottom: 6px;
}
.custom-domain-input {
display: block;
width: 100%;
box-sizing: border-box;
padding: 9px 12px;
font: inherit;
font-size: 14px;
color: var(--afb-fg);
background: var(--afb-field-bg);
border: 1px solid var(--afb-field-border);
border-radius: 8px;
}
.custom-domain-input:focus {
outline: none;
border-color: #088ef0;
box-shadow: 0 0 0 3px rgba(8, 142, 240, 0.18);
}
.custom-domain-status {
margin: 0 0 8px;
font-size: 14px;
line-height: 1.45;
word-break: break-word;
}
.custom-domain-status-detail {
display: block;
margin-top: 4px;
font-size: 12px;
color: var(--afb-muted);
}
.custom-domain-records {
width: 100%;
margin-top: 10px;
border-collapse: collapse;
font-size: 12px;
table-layout: fixed;
}
.custom-domain-records th {
text-align: left;
font-weight: 500;
color: var(--afb-muted);
padding: 0 6px 6px 0;
}
.custom-domain-records th:first-child {
width: 86px;
}
.custom-domain-records td {
padding: 6px 6px 6px 0;
border-top: 1px solid var(--afb-border);
vertical-align: top;
}
.custom-domain-copy {
display: block;
word-break: break-all;
cursor: copy;
border-radius: 4px;
}
@media (hover: hover) {
.custom-domain-copy:hover {
background: var(--afb-hover);
}
}
/* Add-an-app chooser (see showAddAppModal): the three ways to get an app, as
rows in the same modal shell. min-width gives the two-line rows room without
overflowing a narrow phone; the shell's own max-width caps the other end. */
@@ -4250,6 +4393,11 @@ body.myapps-reordering .myapps-tile {
outline-offset: -2px;
}
/* Holds the focus while the keys move the selection, which marks the place. */
.dashboard-section-files .files-tab .files:focus {
outline: none;
}
.dashboard-section-files .header .path-btn-disabled {
opacity: 0.1;
pointer-events: none;
@@ -9279,6 +9427,95 @@ body.dashboard-mode .notifications-close-all {
}
}
/* A text field with fixed text after it (a domain), drawn as one box. */
.dashboard-form-affix {
display: flex;
align-items: center;
height: 38px;
border: 1px solid var(--dashboard-border);
border-radius: 10px;
background: var(--dashboard-input-background);
box-sizing: border-box;
overflow: hidden;
transition: border-color 0.15s ease, box-shadow 0.15s ease, background-color 0.15s ease;
}
.dashboard-form-affix:focus-within {
border-color: var(--select-color);
background: var(--dashboard-card-background);
box-shadow: 0 0 0 3px var(--select-ring);
}
.dashboard-form-affix:has([aria-invalid='true']) {
border-color: var(--dashboard-danger-text);
}
.dashboard-dialog .dashboard-form-affix .dashboard-dialog-input,
.dashboard-dialog .dashboard-form-affix .dashboard-dialog-input:focus {
flex: 1 1 auto;
min-width: 0;
height: 100%;
padding: 0 2px 0 12px;
border: none;
border-radius: 0;
background: transparent;
box-shadow: none;
text-align: right;
}
/* Locked while the form submits: dim the whole box, suffix included, as
one field rather than the input alone. */
.dashboard-form-affix:has(:disabled) {
opacity: 0.6;
}
.dashboard-dialog .dashboard-form-affix .dashboard-dialog-input:disabled {
opacity: 1;
}
.dashboard-form-suffix {
flex-shrink: 0;
max-width: 60%;
padding-right: 12px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-size: 14px;
color: var(--dashboard-text-muted);
cursor: text;
}
@media (pointer: coarse) {
.dashboard-form-affix {
height: 44px;
}
.dashboard-form-suffix {
font-size: 16px;
}
}
/* A switch sits beside its label and hint, as on the settings cards. */
.dashboard-form-switch-field {
flex-direction: row;
align-items: center;
justify-content: space-between;
gap: 16px;
}
.dashboard-form-switch-text {
display: flex;
flex-direction: column;
gap: 2px;
min-width: 0;
}
.dashboard-form-switch-field .dashboard-form-label {
font-size: 13px;
color: var(--dashboard-text-primary);
cursor: pointer;
}
/* Password fields carry a show/hide toggle inside the box. */
.dashboard-form-secret {
position: relative;
@@ -9338,6 +9575,7 @@ body.dashboard-mode .notifications-close-all {
the content column's gap it would otherwise still claim. */
.dashboard-form-status {
padding: 9px 12px;
white-space: pre-line;
border: 1px solid transparent;
border-radius: 8px;
font-size: 13px;
@@ -9425,12 +9663,13 @@ body.dashboard-mode .notifications-close-all {
line-height: 1.45;
}
.dashboard-token-row {
/* A value to copy: the API token, a published address. */
.dashboard-copy-row {
display: flex;
gap: 8px;
}
.dashboard-dialog .dashboard-dialog-input.dashboard-token-value {
.dashboard-dialog .dashboard-dialog-input.dashboard-copy-value {
flex: 1 1 auto;
min-width: 0;
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
@@ -9439,15 +9678,37 @@ body.dashboard-mode .notifications-close-all {
/* 16px, like the other dialog inputs on touch, so iOS doesn't zoom on focus. */
@media (pointer: coarse) {
.dashboard-dialog .dashboard-dialog-input.dashboard-token-value {
.dashboard-dialog .dashboard-dialog-input.dashboard-copy-value {
font-size: 16px;
}
}
.dashboard-token-copy {
.dashboard-copy-row .dashboard-dialog-btn {
flex-shrink: 0;
}
/* A square button holding only an icon, as in a compact copy row. */
.dashboard-dialog-btn.dashboard-icon-btn {
min-width: 36px;
width: 36px;
padding: 0;
color: var(--dashboard-text-secondary);
text-decoration: none;
}
.dashboard-icon-btn svg {
width: 16px;
height: 16px;
}
@media (pointer: coarse) {
.dashboard-dialog-btn.dashboard-icon-btn {
min-width: 44px;
width: 44px;
padding: 0;
}
}
.dashboard-token-sessions {
display: inline;
padding: 0;
@@ -9471,7 +9732,7 @@ body.dashboard-mode .notifications-close-all {
}
@media (max-width: 600px) {
.dashboard-token-row {
.dashboard-copy-row:not(.dashboard-copy-row-compact) {
flex-direction: column;
}
}
+22 -17
View File
@@ -18,6 +18,7 @@
*/
import { createUploadThumbnailGenerator } from './services/pdfThumbnails/index.js';
import { dedupedName } from './helpers/dedupedName.js';
import { archiveEntryPath } from './helpers/archiveEntryPath.js';
import get_html_element_from_options from './helpers/getHtmlElementFromOptions.js';
import globToRegExp from './helpers/globToRegExp.js';
@@ -1961,6 +1962,12 @@ window.move_items = async function (el_items, dest_path, is_undo = false) {
});
}
// The dashboard's Back/Forward follows a moved folder, and
// forgets a trashed one.
if ( $(el_item).attr('data-is_dir') === '1' ) {
window.dashboard_object?.folderMoved?.($(el_item).attr('data-path'), is_trashing ? null : fsentry.path);
}
if ( is_trashing ) {
// if trashing dir...
if ( $(el_item).attr('data-is_dir') === '1' ) {
@@ -2631,7 +2638,10 @@ window.zipItems = async function (el_items, targetDirPath, download = true) {
progwin?.set_status(i18n('zip', 'Selection(s)'));
}, 500);
let toBeZipped = {};
const toBeZipped = {};
// Shared items from different people can share a name; each keeps its
// own entry rather than overwriting the other.
const topNames = [];
let perItemAdditionProgress = window.zippingProgressConfig.SEQUENCING / el_items.length;
let currentProgress = 0;
@@ -2639,6 +2649,8 @@ window.zipItems = async function (el_items, targetDirPath, download = true) {
const el_item = el_items[idx];
if ( window.operation_cancelled[zip_operation_id] ) return;
let targetPath = $(el_item).attr('data-path');
const topName = dedupedName(path.basename(targetPath), topNames);
topNames.push(topName);
// if directory, zip the directory
if ( $(el_item).attr('data-is_dir') === '1' ) {
@@ -2650,12 +2662,10 @@ window.zipItems = async function (el_items, targetDirPath, download = true) {
for ( let cIdx = 0; cIdx < children.length; cIdx++ ) {
const child = children[cIdx];
if ( ! child.relativePath ) {
// Add empty directiories to the zip
toBeZipped = {
...toBeZipped,
[`${path.basename(child.path)}/`]: [await blobToUint8Array(new Blob()), { level: 9 }],
};
if ( child.emptyDirPath !== undefined ) {
// Add empty directories to the zip, where they sit
const dirPath = el_items.length === 1 ? child.emptyDirPath : `${topName}/${child.emptyDirPath}`;
toBeZipped[dirPath || `${topName}/`] = [await blobToUint8Array(new Blob()), { level: 9 }];
} else {
// Add files from directory to the zip
let relativePath;
@@ -2665,17 +2675,14 @@ window.zipItems = async function (el_items, targetDirPath, download = true) {
}
else
{
relativePath = `${path.basename(targetPath) }/${ child.relativePath}`;
relativePath = `${topName}/${child.relativePath}`;
}
// read file content
progwin?.set_status(i18n('sequencing', child.relativePath));
let content = await puter.fs.read(child.path);
try {
toBeZipped = {
...toBeZipped,
[relativePath]: [await blobToUint8Array(content), { level: 9 }],
};
toBeZipped[relativePath] = [await blobToUint8Array(content), { level: 9 }];
} catch (e) {
console.error(e);
}
@@ -2688,10 +2695,7 @@ window.zipItems = async function (el_items, targetDirPath, download = true) {
else {
progwin?.set_status(i18n('reading', path.basename($(el_items[0]).attr('data-path'))));
let content = await puter.fs.read(targetPath);
toBeZipped = {
...toBeZipped,
[path.basename(targetPath)]: [await blobToUint8Array(content), { level: 9 }],
};
toBeZipped[topName] = [await blobToUint8Array(content), { level: 9 }];
currentProgress += perItemAdditionProgress;
progwin?.set_progress(currentProgress.toPrecision(2));
}
@@ -2762,7 +2766,8 @@ async function readDirectoryRecursive (path, baseDir = '') {
const entries = await puter.fs.readdir(path, { consistency: 'eventual' });
if ( entries.length === 0 ) {
allFiles.push({ path });
// An empty folder, by where it sits in the folder being read.
allFiles.push({ path, emptyDirPath: baseDir });
} else {
// Process each entry
for ( const entry of entries ) {
@@ -17,7 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import path from '../../lib/path.js';
import path from '../lib/path.js';
/** Matches the backend's cap in FSService#findDedupedName. */
const MAX_SUFFIX = 100_000;
+51
View File
@@ -0,0 +1,51 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import UIWindowSaveAccount from '../UI/UIWindowSaveAccount.js';
import UIWindowEmailConfirmationRequired from '../UI/UIWindowEmailConfirmationRequired.js';
/**
* Walks a temporary account through saving itself, and an unconfirmed one
* through confirming its email, before it publishes.
*
* @param {boolean} required - Whether this deployment needs a verified email
* for this kind of publish; when false, resolves true straight away.
* @returns {Promise<boolean>} False if the user backed out of either step.
*/
const ensure_can_publish = async function (required) {
if ( ! required ) return true;
if ( window.user.is_temp && ! await UIWindowSaveAccount({
send_confirmation_code: true,
message: i18n('save_account_to_publish'),
window_options: {
backdrop: true,
close_on_backdrop_click: false,
},
}) ) {
return false;
}
// Saving refreshes window.user; a deployment that doesn't confirm at
// signup leaves the new account unconfirmed, and it still has to confirm.
if ( ! window.user.email_confirmed ) {
return !! await UIWindowEmailConfirmationRequired();
}
return true;
};
export default ensure_can_publish;
@@ -0,0 +1,81 @@
// @vitest-environment jsdom
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { beforeEach, describe, expect, it, vi } from 'vitest';
vi.mock('../UI/UIWindowSaveAccount.js', () => ({ default: vi.fn() }));
vi.mock('../UI/UIWindowEmailConfirmationRequired.js', () => ({ default: vi.fn() }));
globalThis.i18n = (key) => key;
const { default: ensure_can_publish } = await import('./ensureCanPublish.js');
const { default: UIWindowSaveAccount } = await import('../UI/UIWindowSaveAccount.js');
const { default: UIWindowEmailConfirmationRequired } = await import('../UI/UIWindowEmailConfirmationRequired.js');
beforeEach(() => {
UIWindowSaveAccount.mockReset();
UIWindowEmailConfirmationRequired.mockReset();
});
describe('ensure_can_publish', () => {
it('lets anyone through when verification is not required', async () => {
window.user = { is_temp: true };
expect(await ensure_can_publish(false)).toBe(true);
expect(UIWindowSaveAccount).not.toHaveBeenCalled();
});
it('has a temporary account saved first', async () => {
window.user = { is_temp: true };
UIWindowSaveAccount.mockResolvedValueOnce(false);
expect(await ensure_can_publish(true)).toBe(false);
expect(UIWindowEmailConfirmationRequired).not.toHaveBeenCalled();
// Saving replaces window.user with the saved, confirmed account.
UIWindowSaveAccount.mockImplementationOnce(async () => {
window.user = { is_temp: false, email_confirmed: true };
return true;
});
expect(await ensure_can_publish(true)).toBe(true);
expect(UIWindowEmailConfirmationRequired).not.toHaveBeenCalled();
});
it('still has a just-saved account confirm its email when saving left it unconfirmed', async () => {
window.user = { is_temp: true };
UIWindowSaveAccount.mockImplementationOnce(async () => {
window.user = { is_temp: false, email_confirmed: false };
return true;
});
UIWindowEmailConfirmationRequired.mockResolvedValueOnce(false);
expect(await ensure_can_publish(true)).toBe(false);
expect(UIWindowEmailConfirmationRequired).toHaveBeenCalledTimes(1);
});
it('has an unconfirmed email confirmed first', async () => {
window.user = { is_temp: false, email_confirmed: false };
UIWindowEmailConfirmationRequired.mockResolvedValueOnce(false);
expect(await ensure_can_publish(true)).toBe(false);
expect(UIWindowSaveAccount).not.toHaveBeenCalled();
});
it('lets a confirmed account through', async () => {
window.user = { is_temp: false, email_confirmed: true };
expect(await ensure_can_publish(true)).toBe(true);
expect(UIWindowEmailConfirmationRequired).not.toHaveBeenCalled();
});
});
+19 -18
View File
@@ -20,11 +20,10 @@
import UIAlert from '../UI/UIAlert.js';
import UIWindowItemProperties from '../UI/UIWindowItemProperties.js';
import UIWindowSaveAccount from '../UI/UIWindowSaveAccount.js';
import UIWindowEmailConfirmationRequired from '../UI/UIWindowEmailConfirmationRequired.js';
import UIWindowPublishWorker from '../UI/UIWindowPublishWorker.js';
import UIWindowShare from '../UI/UIWindowShare.js';
import publish_as_website from './publishAsWebsite.js';
import ensure_can_publish from './ensureCanPublish.js';
import open_item from './openItem.js';
import openWithAI from './openWithAI.js';
import launch_app from './launchApp.js';
@@ -47,6 +46,8 @@ import { is_default_app } from './defaultApp.js';
* @param {Function} options.onOpen - Optional custom open handler (used by Dashboard)
* @param {Function} options.onShowProperties - Optional custom properties handler (used by Dashboard); receives {name, path, uid, element}
* @param {Function} options.onShare - Optional custom share handler (used by Dashboard); receives {name, path, uid, element}
* @param {Function} options.onPublishWebsite - Optional custom publish-as-website handler (used by Dashboard); receives {uid, name, path}
* @param {Function} options.onPublishWorker - Optional custom publish-as-worker handler (used by Dashboard); receives {uid, name, path}
* @param {Function} options.onPasteInto - Optional custom paste handler (used by Dashboard); receives the folder's path
* @param {Function} options.onRemovedFromShared - Optional, called once Remove from Shared has taken the item's row away (used by Dashboard)
* @returns {Promise<Array>} Array of context menu items
@@ -142,11 +143,16 @@ const generate_file_context_menu = async function (options) {
// enforces that). `write` on someone else's folder is not enough.
disabled: !fsentry.is_dir || fsentry.has_website || !may_share,
onClick: async function () {
await publish_as_website({
const target = {
uid: fsentry.uid,
name: $(el_item).attr('data-name'),
path: $(el_item).attr('data-path'),
});
};
if ( options.onPublishWebsite ) {
options.onPublishWebsite(target);
} else {
await publish_as_website(target);
}
},
});
}
@@ -160,22 +166,17 @@ const generate_file_context_menu = async function (options) {
html: i18n('publish_as_serverless_worker'),
disabled: is_worker,
onClick: async function () {
if ( window.user.is_temp &&
!await UIWindowSaveAccount({
send_confirmation_code: true,
message: i18n('save_account_to_publish'),
window_options: {
backdrop: true,
close_on_backdrop_click: false,
},
}) ) {
const target = {
uid: fsentry.uid,
name: $(el_item).attr('data-name'),
path: $(el_item).attr('data-path'),
};
if ( options.onPublishWorker ) {
options.onPublishWorker(target);
return;
}
else if ( !window.user.email_confirmed && !await UIWindowEmailConfirmationRequired() ) {
return;
}
UIWindowPublishWorker(fsentry.uid, $(el_item).attr('data-name'), $(el_item).attr('data-path'));
if ( ! await ensure_can_publish(true) ) return;
UIWindowPublishWorker(target.uid, target.name, target.path);
},
});
}
+1 -1
View File
@@ -595,7 +595,7 @@ const launch_app = async (options) => {
// for: such a message is never replied to, and a reply timeout can't
// stand in for the check because legitimate replies only arrive when
// the user closes the dialog.
iframe_url.searchParams.append('puter.gui_features', 'feedback-dialog');
iframe_url.searchParams.append('puter.gui_features', 'feedback-dialog,custom-domains');
// Add options.args to URL
iframe_url.searchParams.append('puter.args', JSON.stringify(options.args ?? {}));
Loaded 100 of 124 files, more files were not shown because too many files have changed in this diff. Show more