Compare commits

...
Author SHA1 Message Date
Nariman Jelveh 3dd8e599de fix(dashboard): cancel a transfer once when its Cancel is double-clicked
A cancelled row stays clickable while it fades out, so a double-click
ran the cancel twice: two abort requests for an upload, and for a guest
session two "save your session" prompts.
2026-10-25 19:48:54 -07:00
Nariman Jelveh a62cf80bdf fix(dashboard): word a transfer's time left in Traditional Chinese, Portuguese and Ukrainian
The estimate passed Puter's language code straight to Intl, which
rejects zhtw (leaving "About 9 left" with no unit) and reads br as
Breton and ua as nothing. It now maps the code with intlLocale like the
rest of the dashboard.
2026-10-25 19:48:25 -07:00
Nariman Jelveh 841d283135 fix(dashboard): name the upload a failed transfer row belongs to
Every failed upload in the tray read "Failed to upload files", so with
several running there was no telling which one failed. The row now says
"Failed to upload report.pdf" or "Failed to upload 3 items".
2026-10-25 19:47:44 -07:00
Nariman Jelveh c6350f1a7b fix(dashboard): let a transfer's title wrap so long names keep their extension
A row's title was one line cut at the end, so "Uploading quarterly-
report-final-v2.pdf" lost its extension and most of its name in a
360px tray. It now runs to two lines, like the status under it.
2026-10-25 19:46:53 -07:00
Nariman Jelveh fedc38faa5 fix(dashboard): announce transfer tray status to screen readers
The tray's summary changed silently, so a screen reader user had no
word that a transfer finished. The summary is now a status region.
2026-10-25 19:45:04 -07:00
Nariman Jelveh 2411a2baaf fix(dashboard): title a finished upload "Uploaded" rather than "Uploading"
A finished upload kept its "Uploading report.pdf" title beside its Done
mark. The tray's complete() now takes the finished task's title, and the
Files upload passes "Uploaded report.pdf" or "Uploaded 3 items".
2026-10-25 19:44:52 -07:00
Nariman Jelveh 81fb4ee84c fix: keep one progress window per copy or move across conflict prompts
Copy and move re-arm their progress timer after a conflict prompt, and
the timer opened a second progress window even when one was already up.
The first was then never closed: on the dashboard it stayed in the
transfer tray as a running task that couldn't be cleared. The timer now
leaves an open progress window alone.
2026-10-25 19:44:00 -07:00
Nariman Jelveh 24c2808a00 fix(dashboard): estimate a transfer's time left from its exact progress
The tray rounded progress to whole percentages before estimating, so a
slow upload whose percentage stood still for the 5s rate window lost
its time-left text about a quarter of the time and underestimated the
rest. The estimate now gets the uploader's hundredths; the bar and label
still show whole percentages.
2026-10-25 19:40:17 -07:00
Nariman Jelveh abc8b15d63 fix(dashboard): keep the Files selection bar clear of the transfer tray
On phones the tray and select mode's action bar both docked at the
bottom, so a running transfer hid every selection action. The tray now
rises above the bar while it shows.
2026-10-25 19:39:38 -07:00
Nariman Jelveh d767802c6a fix(dashboard): stack the transfer tray under the dashboard's drawer, sheets and dialogs
The tray sat on body above every window, so on phones it covered the
open sidebar drawer, the Share and Properties sheets, and the drag
cancel zone, and stayed bright and clickable over their scrims. It now
mounts in .dashboard just under the drawer's scrim, and remounts if the
dashboard it was in is re-rendered.
2026-10-25 19:38:09 -07:00
Nariman JelvehandClaude Opus 5.5 3cbc499324 feat(dashboard): show transfer progress in a non-blocking tray
Uploads, copies and moves in the dashboard no longer open a modal.
Each one adds a row to a tray in the bottom-right corner, and the rest
of the dashboard stays usable while it runs. UIWindowProgress hands
off to UIDashboardTransfer, which returns the same handle as before.

The tray shows how many tasks are running and can be collapsed to that
summary. Each row has its own Cancel button, a progress bar, and a
time-left estimate. A finished upload shows Done for a few seconds. A
failure stays until it is dismissed and opens a collapsed tray. The
tray never takes focus, and it sits above app windows and the Files
footer.

With no modal left to sit under, UIAlert no longer needs to stay on
top of dashboard overlays, so that change is reverted.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 19:24:45 -07:00
Nariman JelvehandClaude Opus 5.5 8ca6ecc9ca feat(dashboard): redesign upload, copy, and move progress as dashboard modals
In dashboard mode, progress for uploads, copies, and moves now shows in
the same overlay card as the other dashboard dialogs instead of a
UIWindow. UIWindowProgress hands off to UIDashboardProgressModal there,
so the shared file helpers drive either one through the same handle.

The card shows a titled header ("Uploading 3 items", "Copying",
"Moving", "Deleting"), the current item or destination, an
indeterminate bar until there is a percentage, and then a percentage
with a smoothed time-left estimate. Uploads end on a brief done state.
An error replaces the progress with a message and an OK button. While
a transfer runs, only Cancel stops it: Escape and backdrop clicks do
nothing.

The dashboard's two upload paths now share uploadFiles. A UIAlert
opened while a dashboard overlay is up now stays on top, so a conflict
prompt raised mid-copy can still be answered.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 19:11:58 -07:00
Daniel Salazar 24aac2f6d2 fix: app token (#4338) 2026-10-25 11:50:25 -07:00
Daniel Salazar 1eb4e36cf4 fix: add beta flag to kv share docs to match kvhandle beta status (#4337) 2026-10-25 09:17:12 -07:00
Daniel Salazar 3160b99f11 chore: bump ver (#4335) 2026-10-25 08:24:40 -07:00
Daniel Salazar 589a98b12c feat(kv): share KV prefixes with other users at watch/read/write, and bound what apps can be granted (#4331)
* feat(kv): share key-value prefixes with other users, and bound what apps can be granted

- KV share handles move out of Events into KvShareService, served at /kv/shares
  (the /events/kv-handles routes stay as an alias).
- Shares have watch, read and write levels, encoded in the permission root
  (kv-share, kv-share-read, kv-share-write) with write implying read and watch.
- Every puter.kv method takes optConfig.handle; keys are relative to the shared
  prefix, cursors are sealed, and private entries stay hidden.
- New puter.kv.share, unshare, listShared and listSharedByMe; optional label;
  kv.share.received notice in the app's mailbox only; recipient blocks apply.
- puter.kv.set gains onlyIfAbsent.
- fs and kv events gain viaWorker, and kv events report self from the writer.
- /auth/grant-user-app only accepts canonical permissions from a fixed set of
  families; the consent dialog, puter.perms and docs follow.
- Migrations: kv_share_handles gains revoked_by, label, and grantee and app
  indexes (mysql_mig_49, postgres_mig_38, sqlite 0095).

* fix(kv): never name a private key in a cross-app or handle cursor, and bound flat-permission scans

- Listings of another namespace (cross-app or through a handle) resume after
  the last entry they returned, and their cursors are sealed, bound to the
  namespace or handle, and padded to one length. Old plain cross-app cursors
  are still read when they name the target namespace; other namespaces get 400.
- PermissionStore scans flat permissions a few roots at a time and skips
  roots nested under another root in the same call.
2026-10-25 08:16:19 -07:00
Toshit ChawdaandReynaldi Chernando ec3ad1cb2f docs: easy/medium/hard networking recipes (#4040)
* add net category + http request recipes

* add tcp/tls socket recipes

* add complex recipes

* minor refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-25 19:22:45 +07:00
Reynaldi ChernandoandDaniel Salazar c7cc64e45c docs: medium events recipes (PUT-1971) (#4333)
* docs: easy events recipe (PUT-1961)

* docs: catch-up-on-notifications events recipe (PUT-1961)

* docs: events recipe corrections from a run against a live server (PUT-1961)

* docs: medium events recipes (PUT-1971)

* docs: simpler wording in medium events recipes (PUT-1971)

* refine

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-25 19:18:55 +07:00
Nariman Jelveh 0c67499dc2 fix(dashboard): serve stale profile pictures to every caller, skip detached roots
While a picture refreshed after its TTL, only the first caller got the
stale answer; every later one waited on the network, so in Files → Shared
one row showed the picture at once and the rest fell back to initials.
paintAvatars now also folds a frame's roots into one pass and skips roots
re-rendered away before the frame, so they fire no lookups.
2026-10-24 19:05:54 -07:00
Nariman Jelveh 34d74d36bd feat(dashboard): show profile pictures on people avatars (#4328)
* feat(dashboard): show profile pictures on people avatars

Avatars that name a person now show that person's profile picture once
it loads, and keep the initial-and-color avatar when there is none:

- Files → Shared: the "Shared by" cell and the people filter chips.
- Share dialog: the owner row and each person with access. Teams,
  invited emails and link shares keep their initial.
- Properties: a new Owner row with avatar for items someone else owns,
  and avatars beside each version's author.

Lookups go through one cache: each person is looked up once and shared
by every avatar of theirs, answers (including "no picture") are kept
for five minutes, at most 100 are kept, and at most four run at once.
Calls made while a listing renders row by row fold into one pass per
frame. The signed-in user's own picture is read from their loaded
profile, so it takes no request and follows changes in the Account tab.
The picture is set as an image's src property, never through markup.

* fix(dashboard): profile picture review — stale refresh, picker avatars, limiter

- Serve a stale picture while a fresh one loads, so a listing re-rendered
  after the cache expires no longer drops everyone back to initials.
- Show pictures on people in the share recipient suggestions, matching the
  rows of the dialog they open from.
- Free a lookup slot when a lookup throws before returning a promise.
- Move profilePictures.js to helpers/, now that a shared helper uses it.
2026-10-24 19:01:05 -07:00
Nariman Jelveh 0bdc2197b0 feat(dashboard): show who shared each item in Files → Shared (#4327)
* feat(dashboard): show who shared each item in the Files tab's Shared view

The Shared view now names the person behind every item:

- List view gets a sortable, resizable "Shared by" column with the
  share modal's initial-and-color avatar. The tooltip names the owner
  too when someone else re-shared the item.
- Grid tiles show the sharer under the item name.
- Narrow listings lead each row's second line with the sharer.
- A people filter above the listing (one chip per sharer, with counts)
  narrows the view to one person. It shows once two or more people have
  shared something, and scrolls sideways on small screens.

Sorting by "Shared by" applies only in the Shared view; other folders
keep sorting by name while it is the saved sort.

* fix(dashboard): keep the name column readable beside "Shared by"

In a listing between the phone layout and about 700px wide, the fixed
160px "Shared by" column squeezed an auto-width name down to a few
characters. Beside an auto-width name, "Shared by" now takes at most 40%
of the room the two split, and resizing it stops at the same width.

* fix(dashboard): keep the "Shared by" header on one line

Narrowed, its two-word label wrapped onto a second line that spilled out
of the 32px header row. It now stays on one line and ends in an
ellipsis, as longer translations need too.

* fix(dashboard): fit "Shared by" to its longest name on double-click

Auto-fitting the column measured names in a font the cell doesn't use
and left out part of its padding, so the longest name still ended in an
ellipsis. It now reads the font, padding, avatar and gap off a rendered
cell.

* fix(dashboard): keep the people filter on coming back to Shared

Opening a folder from a filtered Shared view and going Back reset the
filter to Everyone, so going through one person's folders meant picking
them again each time. The filter now lasts while the user browses what
was shared with them, and clears once they go back to their own files.

* fix(dashboard): update Shared after Remove from Shared

Remove from Shared only took the row away, leaving the people filter
counting the item. Removing the last item of the person filtered to left
a blank pane under a chip that still claimed one. Shared is now listed
again afterwards, which updates the filter, the empty notice and the
footer.

* fix(dashboard): show a notified share past the people filter

Opening a "shared with you" notification, or a share link, while Shared
was filtered to someone else picked out nothing: the item was filtered
away. The filter now gives way to everyone's when it hides an item the
notification or link names.

* fix(dashboard): truncate a long sharer name on phone-width rows

In the two-line row layout, a long username grew the details line past
the row, pushing the date off screen and under the ⋯ button. The name
now ends in an ellipsis, and the line keeps clear of the button as the
name line does.

* perf(dashboard): sort the Shared view by sharer with one collator

Each sharer comparison called localeCompare with options, which builds
a collator per call: sorting 2,000 rows took about 25ms in Chrome. One
shared Intl.Collator orders them the same in under 1ms.

* fix(dashboard): make share avatar initials readable for every name

The initial-and-color avatar drew its letter at too little contrast for
nearly half the hues a username can map to: yellows and greens came out
near 3:1 in light mode, and lower still on a selected row. The letter is
now darker in light mode and lighter in dark mode, reaching 4.5:1 for
every hue on the page, selected rows and the share suggestions' hover
row. The share dialog and the Shared listing change together, so a
person keeps one look.

* fix(dashboard): show every folder when a drag springs into Shared

Dragging an item out of a shared folder and holding it over Up, Back or
the Shared crumb opens Shared to drop into a folder there. The people
filter, which now lasts while browsing shared items, could hide those
folders, and its chips can't be pressed mid-drag. A drag arriving in
Shared now sees everyone's folders.

* fix(dashboard): keep filtering Shared when it can't be listed again

A chip click lists Shared again so its rows are current. When that
request failed, the listing was replaced with "This folder couldn't be
opened" and the chips disappeared, though only a filter had changed. A
filter change now applies to the rows already loaded when the listing
can't be fetched.

* fix(dashboard): return a drag let go over Shared to where it began

A drag that springs into Shared and is let go over the listing rather
than a folder tried to move the items into Shared itself, which is a
query and not a folder: nothing moved, nothing said so, and the user was
left in Shared. Let go over a listing that takes no items, the drag now
goes back to the folder it started in, as the cancel zone does.

* fix(dashboard): don't offer read-only shared folders as drop targets

Folders shared read-only lit up under a drag and took the drop, which
the server then refused with "Forbidden". They no longer light up, and
a drop on one sends the drag back where it began; they still open on a
dwell.

* fix(dashboard): slide a drag back when a read-only folder refuses it

A drop a read-only shared folder refused left the drag chip to vanish
where it was let go, as if something had happened. It now slides back
to where the drag began, as any drop that doesn't land does.
2026-10-24 18:10:27 -07:00
Nariman Jelveh 7759f7afe9 fix(dashboard): seventh adversarial review — bugs, UX, a11y, perf (#4326)
* fix(dashboard): show notification times in Brazilian Portuguese, Ukrainian and Traditional Chinese

The notification panel and toasts handed Puter's language code straight to
Intl, and a few of those codes aren't BCP 47 tags. Brazilian Portuguese
(`br`) is Breton to Intl, so times read "5 munut zo"; Ukrainian (`ua`)
fell back to the browser's default language; and `zhtw` threw, so Taiwan
users got English. Those codes are now mapped to pt-BR, uk and zh-TW
before formatting.

* fix(dashboard): count "yesterday" and "N days ago" in calendar days

Notification times rounded the elapsed hours to whole days, so at 23:30
something from 00:45 the same day read "yesterday", and on Wednesday
morning something from Monday night (34 hours earlier) read "yesterday"
too. Days are now counted from midnight to midnight in the reader's time
zone; anything from today stays in hours.

* fix(dashboard): don't let deploy results crowd notifications out of the panel

The panel listed the 30 newest notification rows and then dropped the
ones it never shows, a worker's deploy results among them. A developer
with 30 deploys since their last share saw "No notifications" and no
badge while the share sat behind them. The listing now asks for up to
200 rows and keeps the newest 30 the panel shows.

* fix(dashboard): keep every unread notification in the panel

The socket replays up to 200 unread notifications on connect, but the
panel's listing held only the 30 newest, and the first refresh after that
dropped the rest: 45 unread read "45" until the panel was opened, then
"30", and "Mark all as read" left the other 15 unread on the server for
the next visit. The panel now keeps unread notifications past the newest
30, up to the 200 a listing returns.

* fix(dashboard): let Undo take back a new Files folder or file

The Files tab creates folders and New › files itself rather than through
the desktop's helpers, and never recorded them for Undo. Undo then
reversed whatever came before: upload notes.txt, create a folder, press
Ctrl+Z, and notes.txt was deleted for good (undoing an upload deletes it
outright, not to Trash) while the new folder stayed. Both now record
themselves, so Undo removes what was just created, as on the desktop.

* fix(dashboard): let Undo take back a Files cut and paste

Pasting cut items in the Files tab moves them itself rather than through
the desktop's move_items, and recorded nothing for Undo. Undo after a
paste reversed the action before it instead, which deletes an upload for
good. The paste now records the move as move_items does, so Undo puts
the items back where they were cut from.

* fix(dashboard): don't drop a Files selection onto one of its own folders

With folders A and B selected, dragging A onto B moved A into B and then
stopped on "Cannot move a directory into itself" for B. Hovering B also
sprang it open mid-drag. A folder row that is part of the drag is no
longer a drop target, so the drag reverts as it does over any other
non-target. A drop elsewhere (a sidebar folder, say) also leaves out the
target folder itself and any folder that holds it.

* fix(dashboard): don't drop a Files refresh asked for during another

A refresh of the folder on screen was ignored while another refresh of
it was loading. Drop two files: when the second finished during the
first's refresh, its own refresh was dropped, the earlier listing (read
before the file existed) was drawn, and the file's item.added was
ignored meanwhile too, so it stayed missing until a manual refresh.
Paste, Refresh and returning to the browser tab could lose items the
same way. A refresh now takes over from one in flight, as opening
another folder already did; a navigation in progress still holds.

* fix(dashboard): keep the phone selection bar after cancelling a delete

In select mode on a phone, tapping Delete permanently and then Cancel
left every item selected but hid the floating action bar, the only way
to act on the selection, until the selection next changed. The same
happened when nothing in the selection could be moved to Trash. The bar
now goes away only once something is actually deleted or trashed.

* fix(dashboard): show the phone selection bar when Select starts with a selection

Uploaded or pasted items arrive selected. Tapping Select on a phone then
turned select mode on with them selected but no floating action bar,
until a row was toggled. Entering select mode now brings the bar up for
the selection it starts with.

* fix(dashboard): give Files footer sizes in the rows' format

The footer used the desktop's byte formatter while the rows use their
own, so a 5-byte file read "5 B" in its row and "5.00 Bytes" below it,
1.5 KB read "1.50 KB", and a folder of folders read "0 Byte". The footer
now formats sizes as the rows do.

* fix(dashboard): title a trashed folder's Properties with its name

Properties from the breadcrumb of a folder opened inside Trash was
titled with the folder's uid, the last segment of its trashed path,
while the crumb itself showed the original name. It now uses that name.

* fix(dashboard): show delete and log-out confirmations without entities

The Files tab's "delete these items permanently?" confirmation and the
dashboard's "close your apps and log out?" one passed HTML-encoded i18n
strings to UIAlert, which encodes its message and button labels again.
In Igbo both questions read "Ị ji n&#039;aka …". These alerts, and the
Files tab's name-conflict and folder-depth ones, now pass plain strings.

* fix(gui): read the Trash's state fresh when updating its icon

refresh_trash_state stats the Trash to pick the empty or full icon, and
its comment calls for a strong read, but it asked for an eventual one.
Once a listing of Home had cached the Trash entry (without `is_empty`),
the stat came back without it: permanently deleting the last items from
Trash in Files left the full-Trash icon up, and other tabs were told the
same. The stat is now strong, as intended.

* fix(dashboard): don't add a Files row to a folder opened meanwhile

A file added from elsewhere (an upload, another tab, an app) is checked
against the folder on screen and then drawn once its icon resolves. A web
link's or app's icon is read from the file, and opening another folder
during that read put the row into the new folder's listing, where it
stayed and opened the file from the old one. The checks now run again
once the icon is in.

* fix(dashboard): let a focused folder or crumb take the Menu key

The Files tab answered the Menu key and Shift+F10 with the selection's
menu wherever focus was. With a file selected and a sidebar folder, a
breadcrumb or Home's Shared row focused, the key opened the file's menu;
with nothing selected it cancelled the browser's contextmenu, so the
focused element opened none. A focused control now gets the browser's
contextmenu event, and its own menu, as with a right-click.

* fix(dashboard): don't drop New Folder during a Files refresh

New Folder did nothing while the folder on screen was refreshing (after
an upload or paste, or on returning to the browser tab). A refresh keeps
the rows up with no spinner, so the click just looked ignored. It now
takes over the refresh and makes the folder once the listing is in;
during a navigation to another folder it is still dropped, as before.

* fix(dashboard): step through Files names on a repeated letter

Type-to-select meant to move past the selected row to the next name with
the same start, but returned early whenever the selected row matched, so
the stepping never ran: with readme, report-a and report-b, pressing "r"
again stayed on readme, and "rr" matched nothing. A repeated letter now
steps through the names it starts, wrapping around, as file managers do;
a longer prefix still stays on a selected row it matches.

* fix(dashboard): open one touch menu per long-press

A long-press on Android fires both contextmenu and taphold. The Files
rows already ignore the second, but a breadcrumb, a sidebar folder,
Home's Shared and Trash rows and the listing's empty space each opened a
second sheet over the first; picking an item or tapping the backdrop
then left the other one up. The touch menu now doesn't open while
another is on screen.

* fix(dashboard): treat Ctrl+click on a Mac as a right-click in Files

On a Mac, Ctrl+click opens the context menu, but the Files rows read it
as a Ctrl-toggle first: Ctrl+clicking one of three selected files
deselected it and the menu then acted on that file alone, and on an
unselected file it was added to the selection. On a Mac it now leaves
the selection alone, as a right-click does; Cmd+click still toggles.

* fix(dashboard): keep arrow keys in the open phone drawer on Apps

With the drawer open over the Apps tab, an arrow key moved focus out of
it to the first app tile behind the scrim, and the next Tab then walked
the page behind it. The open drawer now keeps the arrow keys, as the
notifications panel and dialogs already do.

* fix(dashboard): act once on a held Enter or Space in Apps

Holding Enter on an Apps folder opened it, moved focus to its first app,
and the key's repeats then launched that app; held on the "+" tile, the
repeats picked "Browse the App Center" in the dialog it opened, and held
in the search box they launched the first match again and again. Key
repeats now do nothing on a tile, in the search box or on the add-app
dialog's options.

* fix(dashboard): don't blank the Apps grid when it re-flows on resize

When a resize changed how many columns or rows fit (resizing the window,
collapsing the sidebar, zooming, an Android keyboard), the grid was
rebuilt behind the first load's fade, which waits on page one's icons.
With an icon host that was slow or unreachable the grid went blank for
up to 1.5s each time. A grid already on screen now re-flows at once;
one still loading keeps its fade.

* fix(dashboard): open an app from the add-app dialog at once during a search

"Browse the App Center" and "Build with AI" play an intro from the
app's tile, and wait up to 3s for that tile to appear. With a search
that leaves the app out (the case the "+" tile stays for: a search that
found nothing), the tile never appears, so the app opened only after the
full 3s. The intro now gives up as soon as the search is what hides the
tile, and the app opens straight away.

* fix(dashboard): make the Apps search's clear control a button

The "×" that clears the Apps search was a bare SVG with a click handler:
Tab couldn't reach it, screen readers had nothing to announce, and on a
phone it was a 16px target inside the field. It is now a labelled button
after the field in the Tab order, with a focus ring, and on touch it
hit-tests at 44x44 while keeping its look.

* fix(dashboard): tell screen readers when an Apps search finds nothing

Typing in the Apps search re-renders the grid silently, and the "No apps
match your search" line is drawn into it fresh each time, which screen
readers don't announce. A search that matched nothing gave no feedback
at all. A persistent status region now says so.

* fix(dashboard): keep keyboard focus on an Apps tile across a re-render

The grid is rebuilt wholesale whenever its data or size changes: the
refresh that follows opening the tab, a resize or zoom that changes how
many columns fit, an uninstall that fails. A keyboard user who had
arrowed onto a tile lost focus to <body>, so Enter did nothing and the
next arrow started over from the first tile. Focus now moves to the
same app's or folder's new tile.

* fix(dashboard): don't offer Change username to a temporary account

A temporary account's Account tab offered Change username, whose dialog
asks for an account password the account doesn't have: left empty it
said "All fields are required", and anything typed got a 403 from the
server, which refuses temporary accounts. The button is now left out
until the account is saved, as the Password card already is; saving it
re-renders the tab with the button.

* fix(dashboard): don't offer Change email to a team seat

A team seat's Account tab offered Change email, and after the user
filled in the new address and their password the server refused it:
"Your team set this address. Ask a team admin to change it." The Email
card now shows that note in place of the button, as the Username card
already does for seats.

* fix(dashboard): don't let a slow picture removal undo a newer picture

Saving a profile picture is ordered so an older save can't put its
picture back over a newer one, but Remove picture stood outside that
order. Clicking Remove on a slow connection and then saving a new photo
before the removal answered left the default avatar on screen, and
Remove hidden, while the account had the new photo. A removal now
yields to a picture saved after it.

* fix(dashboard): show placeholders in the Usage headers until they load

The Storage and Resources headers started out as " ·  used of " with the
numbers left blank, and stayed that way when reading the storage space
or the month's usage failed. They now start at "--% · -- used of --",
as Home's cards start at "--", and keep that if the read fails.

* fix(dashboard): write Home's trial dates in the dashboard's language

The free-trial note on Home formats its dates with the browser's
language, not the one chosen in Puter, so with Puter in French and the
browser in English it read "…le Mar 5", and the other way round "First
charge on 5 mars". The dates now follow the dashboard's language. The
notification panel's mapping of Puter's language codes to Intl tags
(Brazilian Portuguese, Ukrainian, Traditional Chinese) moves to
i18n/intlLocale.js so both use it.

* fix(dashboard): write credit and dollar amounts in the dashboard's language

Credits were always formatted with English separators and dollar amounts
with "$" and English separators, so in German Home read "5,000
monatliche Credits", which a German reader takes as five, and Usage
showed "2.5" and "$1,234.50". Both now use the dashboard's language
("5.000", "2,5", "1.234,50 $"); English is unchanged.

* fix(dashboard): say why Create token is off where everyone can read it

When the account's email isn't confirmed, Create token is disabled and
the reason ("Verify your email address to create an API token.") lived
only in a hover tooltip: a disabled button isn't reachable by Tab, a
phone has no hover, and screen readers don't read a plain span's title,
so those users met a dead button with no explanation. The reason now
replaces the card's description while the button is off. The tooltip,
which encoded the already-encoded text again, now uses it as is.

* fix(dashboard): announce that a new API token is shown only once

After a token was created, focus moved to Done and a screen reader read
"Done, button" and nothing else: the done state arrives already filled
in a status region, which isn't announced, and Done is described by the
message only for plain-text results. The token dialog's warning ("Copy
this token now — it won't be shown again…") now describes Done, so it
is read out when focus lands there.

* fix(dashboard): show an email change confirmed in another tab

Confirming a change of email from its link sets the new address and
marks it confirmed, but sends no socket event, unlike confirming a code.
Back in the dashboard the user was re-read on returning to the browser
tab, yet nothing redrew: Account kept the old address under the
username, Create token stayed disabled and Security had no 2FA card
until a reload. Account and Security now redraw after that re-read when
the address or its confirmation changed.

* fix(dashboard): close the share dialog after leaving a share

Someone the owner let share could remove their own row in the share
dialog to leave. The removal went through, but the dialog then reloaded
its access list with the access just given up: the server refused it,
the success line gave way to "Subject does not exist" over an empty
list, and the item stayed in the Shared view until a reload. Leaving
now closes the dialog and refreshes the folder on screen, which drops
the item, as "Remove from Shared" does.

* fix(dashboard): say the access list couldn't load, not that sharing failed

When the share dialog couldn't read who has access (a network error,
which carries no message of its own), it said "Could not share this
item." though nothing had been shared or attempted. It now says "Could
not load who has access."

* fix(dashboard): don't pull focus back after a share change saves

When a change in the share dialog finished saving (a level change, Add
to all, a removal, a share), focus was put back on that row's control or
the add field, wherever the user had gone meanwhile. Someone who changed
a person's level and then started typing a name had the next keystrokes
land on that person's level select, where a matching letter picked
another level and granted it. Focus now goes back only when the save
took it away; focus the user moved elsewhere stays there.

* refactor(gui): move the localized time formatting to i18n/formatTime.js

formatRelativeTime and formatAbsoluteTime lived in the notification
center's module; Files and Properties are about to use them too.

* fix(dashboard): write Files and Properties dates in the dashboard's language

The Files tab's Modified column and Properties' Modified, Created and
version times came from timeago, which has no language but English
registered, so a German dashboard read "3 hours ago" beside "Geändert".
They now use the same localized formatter as the notification panel
("vor 3 Stunden"), which also gives a date past a week rather than
"3 weeks ago". Its formatters are now made once per language, since a
listing formats one date per row.

* fix(dashboard): style the Usage table's Show less button in dark mode

The button's dark variant was written for an `html.dark-mode` class that
nothing sets, so in dark mode it kept its light look: grey text at 3.4:1
on the dark card, a light border, and a light chip on hover. Its rules
move under prefers-color-scheme, as the table's fade overlay already
did.

* fix(dashboard): show the Files breadcrumb separators in dark mode

The chevrons between breadcrumbs are a black stroke at 20% opacity,
which vanishes on the dark header: a nested folder read "Home Documents
Work" with nothing between the names. In dark mode they are now
inverted, like the toolbar's icons.

* fix(dashboard): show the website-link badge on dark Files tiles

In grid view a file inside a published folder carries a link badge: a
black glyph on a disc of the page's background color. In dark mode that
is black on near-black and can't be seen. The badge now gets the white
disc the shortcut and worker badges already have.

* fix(dashboard): skip disabled Back, Forward and Up when tabbing in Files

The Files header's arrows stayed in the Tab order while disabled, drawn
at 10% opacity, which takes their focus ring with them. Tabbing into
Files, the first two stops (Back and Forward, disabled on arrival)
looked like nothing had focus. A disabled arrow now leaves the Tab
order, as a disabled button does, and returns once it can be used.

* fix(dashboard): give the Delete account button an AA-contrast red

The Account tab's Delete account button used the desktop's danger red
(#f00808), which puts its white label at 4.4:1, under the 4.5:1 that
WCAG AA asks of text. It now uses the dashboard dialogs' danger red
(#d90707, 5.3:1) and their darker hover, so the two match.

* fix(dashboard): label collapsed sidebar items for keyboard users too

With the sidebar collapsed, each item shows only its icon, and its name
appeared in a tooltip on mouse hover alone. Tabbing through the sidebar
showed a row of icons with no names. The tooltip now also shows while
an item has keyboard focus.

* fix(dashboard): keep Share and Properties inside the visible screen

Both dialogs capped their height at 85vh (88vh as phone sheets), and vh
is the viewport with the browser's bars hidden. With the bars showing,
on a phone in landscape say, the card plus the overlay's padding was
taller than the screen, and the overlay centers without scrolling: the
title and the buttons were cut off with no way to reach them. Each card
is now also held within its overlay, as the dashboard's dialogs are,
and its body scrolls instead.

* fix(dashboard): outline the off state of the 2FA switch

Off, the switch was a light grey track (#e0e0e0) with a white thumb,
1.3:1 against a white card and 1.2:1 against the amber card the 2FA
toggle sits on while 2FA is off, so the control could barely be seen
where it matters most. WCAG asks 3:1 for a control's boundary. The off
track now carries a 1px slate outline (4.3:1 on the amber card); the
on state is unchanged.

* fix(dashboard): make the keyboard focus ring visible on sidebar, tiles and lists

The sidebar items, the Files folders, crumbs and arrows, the app tiles,
Home's cards and recent apps, the notification rows and the Usage sort
buttons show keyboard focus with a 2px outline and nothing else, drawn
in --select-ring: the selection color at 40% (50% dark), about 1.6:1
against the surfaces it sits on. Keyboard users could hardly tell where
focus was; a focus indicator needs 3:1. Those outlines now use the solid
selection color (3.7:1 on white), as the switch and pager dots already
do. Field halos, which come with a colored border, are unchanged.

* fix(dashboard): bring text in the selection blue up to AA contrast

The selection blue is 3.7:1 against white, and a few places set text in
it or white text on it, under the 4.5:1 that text needs: the active
folder in the Files sidebar, the share dialog's "Add to all" link, the
add-app dialog's Send button, the notification count badge, the drag
count badge, and a breadcrumb whose menu is open. They now use the same
hue darkened to 45% (5.3:1; a lighter one for text in dark mode), from
two new tokens, --select-text and --select-fill.

* fix(dashboard): give the Apps page dots a 24px touch target

The Apps pager's dots, and a folder's, are 17px buttons side by side,
under the 24px a target needs when it has neighbors, so on a phone or
tablet a tap meant for one page often opened the next. On a touch
screen each dot is now 24px around the same 7px dot; mouse layouts are
unchanged.

* fix(dashboard): return from a drag-opened folder while it is still loading

Hovering a dragged file over a folder springs it open. Cancelling the
drag (Escape, the Cancel zone, or a drop outside the listing) is meant
to go back where the drag started, but while the sprung folder was still
loading the return was dropped like any navigation during another, and
the user was left in the folder they had backed out of. The return now
takes over from that load.

* fix(dashboard): drop into a drag-opened folder while it is still loading

When a dragged file springs a folder open, the drop anywhere in the
listing moves it into the folder now shown. Until that folder's listing
arrived, though, the Files tab still counted the folder the drag left as
the one on screen, so a drop in that time was taken as a move to where
the file already was and did nothing. It now goes to the folder being
opened.

* fix(dashboard): say "Incorrect password" when the account password is wrong

The account dialogs showed the server's message as is. A wrong current
password in Change password read "Password mismatch", in English in
every language, beside a dialog that says "New Password and Confirm New
Password do not match" when those two differ, so users corrected the
wrong fields. A wrong code read "Incorrect code." and too many attempts
"Too many requests.", also always in English. These now show the
translated messages the GUI already has, and a wrong password puts the
status on the password field, focused for retyping.

* fix(dashboard): don't send an account change to the value it already has

Change username with the account's own name got "This username is
already taken." and used up one of the hour's password attempts. Change
email with the current address was accepted: it mailed a confirmation
and a change notice to the same inbox, and following the link unlinked
any sign-in provider. Change password to the current password signed
out every other session and mailed a "password changed" notice. Each
dialog now says the value is already the account's and sends nothing.

* fix(dashboard): ask for the new password after a provider-only account sets one

An account that signs in only through a linked provider can set a
password with Change password, after which the server treats it as a
password account. This session's copy still said provider-only, so
Change username, Change email and Delete account went on hiding the
password field and sending the user through the provider's popup until
a reload. The copy now changes with the password.

* fix(dashboard): let Escape cancel a session rename without closing Manage sessions

In Manage sessions (Security), Escape in a session's rename field
cancelled the rename and then closed the whole modal, and Escape in the
search box closed it too, search and all. Escape now cancels the rename
or empties the search, and only an Escape with neither closes the
modal.

* fix(dashboard): show the server's message when a session can't be revoked

Manage sessions put a refused request's raw response body in its alert,
so revoking a session already signed out from another tab read
{"message":"Session not found","code":"not_found"}, and the stale row
stayed listed; a failed rename and "Revoke all other sessions" did the
same. The alerts now show the message itself, and a session found gone
reloads the list.

* fix(dashboard): show 2FA as it is when its dialog closes without success

The Security tab's 2FA toggle assumed nothing had changed whenever its
dialog closed without success. With 2FA turned on in another tab, the
setup here refused ("2FA is already enabled."), the toggle went back to
off, and every retry got the same refusal, with Disable never offered;
a code accepted just as the user closed the setup dialog left the same
state. The tab now re-reads the account and shows its real 2FA state.

* fix(gui): leave Ctrl/Cmd+Z to the text field being typed in

Ctrl/Cmd+Z ran the file Undo wherever focus was, and cancelled the
field's own text undo. Typing a new name (F2) or anything in another
field and pressing it undid the last file action instead; when that was
an upload, the uploaded file was deleted for good. Undo now leaves text
fields alone.

* fix(gui): let Escape on a context menu close only the menu

Escape removed any open context menu and then went on to the Files tab's
own Escape handling, which, finding no menu left, cleared the selection
and closed the image preview. Select three files, open their menu,
press Escape: the menu closed and the selection was gone. When Escape
closes a menu, nothing else now acts on it; a second Escape clears the
selection as before.

* fix(dashboard): close the Files image preview when an app opens over it

The image preview sits above every window, and only a double-click
closed it on the way to an app. Opening the image with Enter, or with
Open or Open With from its menu, started the app underneath the
preview, which went on covering it while keyboard focus moved into the
app, so Space and Escape no longer reached the preview. It now closes
whenever an app window comes on screen.

* fix(dashboard): close the Files image preview when its file goes away

Deleting the previewed image (the Delete key, the menu, the phone
selection bar), moving it, or having it removed from another tab took
its row out of the listing but left the preview up, still showing the
image. The preview now closes with its row; a refresh of the folder,
which redraws the row, keeps it.

* fix(dashboard): download one selected file as itself from the phone selection bar

In select mode, the floating bar's Download zipped whatever was
selected, so a single photo arrived as "photo.jpg.zip" where its own
menu's Download gives "photo.jpg". One selected file now downloads as
itself; folders and several items are still zipped.

* fix(dashboard): don't offer New Folder, Upload or Paste in a view-only share

Inside a folder shared with the user to view only, the Files tab still
showed New Folder and Upload, offered New and Paste in its menu, and
took files dropped on the listing or on a folder in it; each then
failed: an upload window ending in an error, or a new folder row with
its name field open, then taken back with a Forbidden alert. Those are
now left out there, as in Trash. A share whose level isn't known yet
stays as before, the server deciding.

* fix(dashboard): move through Files with Home, End, Page Up and Page Down

The Files listing took the arrow keys but not Home, End, Page Up or Page
Down, and as the listing itself never has focus the browser didn't
scroll it on them either: in a folder of hundreds of items the keyboard
could only step through them one row at a time. Home and End now select
the first and last item, Page Up and Page Down move a screenful, and
Shift extends the selection as with the arrows. With a menu open they
are left to it, as the arrows are.

* fix(dashboard): apply the hidden-files setting to a renamed Files row

With Show hidden off, renaming notes.txt to .notes.txt left the row in
the listing until the folder was next loaded, and with it on the row
wasn't dimmed as other dot-files are; renaming .env to env left it
dimmed. A rename made in another tab did the same. A renamed row now
follows the setting as listing the folder would.

* fix(dashboard): open a recent app once on a double-click

Double-clicking an app under Home's "Recently used" opened it in two new
browser tabs, one per click. The second click of a double-click is now
ignored, as on the Apps tab's tiles.

* fix(dashboard): don't say "No recent apps yet" before Home's list has loaded

Home's "Recently used" card was drawn from the list as it stood when the
page opened, which is empty until the request answers: every visit
first read "No recent apps yet / Apps you use will appear here" and
then swapped in the apps, and if the request failed the card kept
saying there were none. It now stays blank until the list arrives, and
a first load that fails says the apps couldn't be loaded; a list
already shown stays when a later refresh fails.

* fix(dashboard): drop "Save session" when the account was saved in another tab

A temporary account saved in one browser tab stayed temporary-looking
in its other dashboard tabs: Home kept its "Save session" button, which
opened Save Account for an account already saved, and Account and
Security kept their temporary-account layout. Coming back to such a tab
re-reads the account; that now removes the button and redraws the two
tabs when the account is no longer temporary.

* fix(dashboard): let go of the phone drawer when the window widens

With the sidebar open as a drawer below 768px, widening the window past
it (an iPad turned to landscape) drew the sidebar as the desktop one but
left it marked open: Tab stayed trapped in it, so the content couldn't
be reached by keyboard, and the Apps grid ignored the arrow keys, until
an Escape or a click in the content. The drawer now closes when the
window widens, leaving focus where it is.

* fix(dashboard): don't let a #files/<path> link pull Files back to its folder

Files doesn't keep the folder on screen in the URL, but a link like
/dashboard#files/alice/Documents stayed in the address bar once opened.
After the user moved on to Pictures, went to Home and pressed Back, the
browser returned to that link and Files jumped back to Documents; a
reload did the same, and Files' own Back/Forward lost track, as the jump
wasn't recorded in it. The link's path is now consumed once opened, as
a share link's is, and a folder opened from such a link is a step in
Files' history.

* fix(dashboard): close the drawer and Files' dialogs on browser Back and Forward

Browser Back or Forward switched tabs but left whatever was open on
screen: on a phone, Android's Back changed the tab under the open menu
drawer and left it open; in Files, the Share or Properties dialog
stayed up over Home, still acting on Files rows; an open context menu
stayed too. These now close with the tab, as with a sidebar click.

* fix(dashboard): fill Home's storage and usage meters without waiting on the plan

Home read the plan first and only then asked for storage and the month's
usage, so every visit paid for one more round-trip before either meter
filled in. The plan request had no timeout either: one that never
answered kept both meters at "--" and, as later loads wait for the one
in flight, held every later refresh of the cards for the rest of the
session. The three now load side by side, and the plan read gives up
after 15 seconds.

* fix(dashboard): link Home's card headers to their tabs

The headers of Home's cards (Recently used, Usage, Your Plan, Your
Storage, Resources) linked to "#", so opening one in a new browser tab
(middle-click, the browser's menu) showed Apps rather than the tab it
names, and Cmd/Ctrl+click switched tabs in place. They now link to the
tab they open, and a modified click is left to the browser, as on the
sidebar.

* fix(dashboard): prefill the username when saving a session from Home

Home's "Save session" button opened Save Account with an empty username,
so a temporary user had to make one up, while the same action in the
user menu fills in the account's current name and starts at the email
field. Both now open it the same way.

* fix(dashboard): measure Home's storage meter by the amount it shows

On a self-hosted install the storage card names the disk's use (Puter's
files and everything else on it) but filled its meter, and gave its
percentage, from Puter's files alone, so it could read "93 GB used ·
0% of 466 GB". The meter and percentage now come from the same amount
the card shows.

* fix(dashboard): refresh Home's recently used apps on returning to the tab

Recent apps open in a new browser tab, and coming back to the dashboard
refreshed the usage cards but not "Recently used", whose comment said
it followed the same triggers: the app just opened only moved up the
list after switching away from Home and back. Returning to the tab now
re-reads the list too.

* fix(dashboard): leave IME keystrokes to the IME in Apps

Typing Japanese or Chinese in the Apps search or a folder's name, the
keys that drive the input method reached the tab's own handlers:
ArrowDown to pick a candidate moved focus to the first tile and cut the
conversion short, Enter to confirm one named the folder half-written or
launched the first match, and Escape put the old name back. The search
box's Enter meant to skip composition, but read isComposing off jQuery's
event, which never carries it. Keys during composition are now left to
the IME.

* fix(dashboard): leave IME keystrokes to the IME in the share recipient field

With a Japanese or Chinese input method on, even a Latin username is
composed, and the share dialog's recipient field took the keys that
drive it: the arrows moved through its suggestions instead of the IME's
candidates, and Enter to confirm the text picked a suggestion instead.
Keys during composition are now left to the IME.

* fix(dashboard): re-read the Apps layout on returning to its browser tab

The Apps tab read its order and folders only when it was opened, and
saves write its whole copy of them. With the dashboard open in two
browser tabs, a folder made in one was wiped as soon as the other,
showing Apps since before, changed anything (a reorder, a new folder, a
rename), as was the order arranged in the first. The tab now reads the
layout again when its browser tab comes back into view.

* fix(dashboard): undo a drag out of an Apps folder when the drag is cancelled

Dragging an app past the edge of an open folder takes it out, closes the
folder and carries the drag on over the grid, and the removal was saved
right then. Pressing Escape, or the window losing focus, cancels the
drag but left the app out; out of a two-app folder that dissolved the
folder for good, its name with it (dragging the app back made a new
"Folder"). A cancelled drag now puts the folder and the order back.

* fix(gui): show the 2FA setup's confirmations without entities

The 2FA setup window, which the dashboard's Security tab opens, encoded
already-encoded translations for its labels, so in French the two
checkboxes before enabling read "J&apos;ai enregistré mes codes…" and
"Je suis prêt à activer l&apos;A2F". The labels, and the window's other
translated text set the same way, are now encoded once.

* fix(gui): don't leave the 2FA dialogs stuck when a request can't be sent

Turning 2FA on or off from the dashboard's Security tab while offline,
or on a dropped connection, left the dialog busy for good: enabling kept
its spinner up with the code boxes disabled, and disabling kept its
button and password field disabled, with nothing said either way. A
request that gets no answer is now reported like a refused one, and the
dialog can be tried again.

* fix(gui): stop disabling 2FA when the provider sign-in doesn't happen

For an account that signs in through a linked provider, turning 2FA off
opens the provider's sign-in popup. When the popup was closed or
blocked, the dialog showed why for a moment and then sent the request
anyway, whose refusal ("OIDC revalidation required") replaced the
reason. It now stops at the popup's error.
2026-10-24 15:05:56 -07:00
Filip KujundžićandClaude Fable 5.1 bbb780eb31 feat(ai): MiniMax video generation (H3 and Hailuo) (#4309)
* feat(ai): MiniMax video generation (H3 and Hailuo)

Adds a minimax-video-generation provider to the video driver. H3 models go
through MiniMax's v2 task API and are billed per second of output; Hailuo
models go through the v1 API, exchange a file id for a download URL, and
are billed per clip. The shared `minimax` chat key is reused, with the
chat block's `/v1` base URL reduced to its host.

WIDTHxHEIGHT sizes now map onto tiers for catalogs that include a 2k or
4k tier.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai): validate MiniMax video inputs and resolution tiers

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-10-24 13:49:21 -07:00
Nariman Jelveh 8a55debdae fix(dashboard): sixth adversarial review — bugs, UX, a11y, perf (#4314)
* fix(dashboard): render in the user's saved language

The dashboard never read the account's preferences: only the desktop did,
on boot. Everyone saw it in English whatever language they had chosen,
until the Files tab's Show hidden toggle switched a running session over,
to the browser's language when none was saved, since that was the
fallback the toggle wrote back.

The desktop's preference loading moves into load_user_preferences, which
the dashboard now runs in the background. The first paint uses the
language this browser last loaded, so nothing waits on the read; a saved
language that differs reloads into it, and a changed Show hidden setting
refreshes the open folder. The tab names are read when the sidebar is
drawn rather than when the modules load. A failed read keeps the cached
preferences instead of resetting them to English.

* fix(dashboard): label credit amounts with a key meant for them

Home's allowance and the Usage tab's total and Credits column used the
`credits` key, which the translations render in its acknowledgments sense:
"5,000 Mitwirkende" (contributors) in German, "Авторы" in Russian,
"特别鸣谢" in Chinese. They now use `usage_credits`, which falls back to
English until it is translated, as `monthly_credits` already does.

* fix(dashboard): enable the token and 2FA cards once the email is confirmed

The dashboard had no handler for the `user.email_confirmed` socket event,
and Account and Security pick the token button's state and whether to
offer 2FA when they render. A user who confirmed their email in another
tab, or in this one through the confirmation prompt, kept a disabled
"Create token" and no 2FA card until a reload.

The dashboard now refreshes the user on that event and renders both tabs
again, as it already does after a temporary account is saved.

* fix(dashboard): save only the profile fields that were edited

Edit profile sent both the display name and the bio on every save, taken
from this session's copy of the profile. That copy comes from the cached
user and can be stale (the profile changed on another device) or empty
(the profile read failed), so changing only the bio wrote the old display
name back, and a save from an empty prefill erased the bio.

The dialog now sends just the fields whose value changed from what it
opened with, and closes without a request when nothing did.

* fix(dashboard): don't show storage as 100% while there is room left

Home and Usage rounded the storage share to a whole percent, so an
account at 99.5% or more read "100%" while uploads still fit. The usage
meter already holds at 99% until the allowance is spent; that rule moves
into meterPercent and the storage meters use it too.

* fix(dashboard): keep keyboard focus inside the open phone drawer

Below 768px the sidebar is a modal drawer over a scrim, but opening it
left focus on the menu button, and Tab past its last entry walked into
the dimmed content behind the scrim. Opening it now focuses the current
page's entry, and Tab and Shift+Tab cycle within it while it is open.

* fix(dashboard): tapping into a Files name field edits the name

On a phone or tablet, tapping the name field of a new folder or a rename
(which iOS needs before it shows the keyboard) reached the row's tap
handler, which opened the folder or launched the file and threw the edit
away. A long-press in the field opened the touch sheet instead of
selecting text, and on desktop a right-click in it showed the file menu
instead of the browser's Cut/Copy/Paste.

The row's tap and menu handlers now leave the name field alone.

* fix(dashboard): don't offer Cut on items the user can't move

A file's menu offered Cut on anything that wasn't immutable, and Shared
rows aren't marked immutable. Cutting a file someone shared with you, or
anything inside a read-only shared folder, then pasting it, ended in a
"Forbidden" alert and an emptied clipboard. Cut now needs the same
permission as Delete in that menu, as Ctrl+X and the selection's Cut
already do.

* fix(dashboard): don't paste or upload into the folder being left

Opening a folder clears the listing at once but only moves currentPath
when the new listing arrives. Ctrl+V, a file dropped from the computer
onto the pane, or Paste from the background menu during that wait went
into the folder the user had just left, and since the refresh then
showed the new one, the items seemed to vanish.

Those three now refuse while another folder is loading, as New Folder
already did.

* fix(dashboard): clear the drop highlight when Escape cancels a drag

Escape ends a Files drag with a synthetic mouseup, which jQuery UI
treats as a release outside every target: it fires no `out`, so the
folder row under the pointer kept its dashed drop outline and tint (and
a sidebar folder or breadcrumb its highlight) after the drag was gone.
The Escape branch now takes those off.

* fix(gui): show the "can't rename in Trash" alert without entities

Pressing F2 on a trashed item (on the desktop or in the dashboard's
Files tab) showed "This item can&apos;t be renamed because it&apos;s in
the trash": the message was encoded by i18n and then again by UIAlert.

* fix(gui): show the failed-upload title without entities

The upload progress window writes its error title with innerText, but
was handed the HTML-encoded i18n string, so French showed "Échec de
l&apos;importation des fichiers". The Files tab's two upload paths and
the desktop's now pass the plain string.

* feat(dashboard): open a Files row's menu with the Menu key or Shift+F10

Share, Download, Open With and Properties were reachable only by
pointer: the ⋯ button isn't focusable, and the keyboard's menu keys
fired a contextmenu on <body>, which the listing ignores. Both keys now
open the selection's menu at its row's ⋯ button, where arrow keys and
Enter already work.

* fix(dashboard): let Shift+Arrow shrink a Files selection

Shift+Arrow only ever added the next row, so after Shift+Down twice,
Shift+Up left all three rows selected instead of going back to two. A
run of Shift+Arrows now selects the range from the row it started on to
the cursor, on top of whatever was selected before the run, so moving
back shrinks it. A click or a plain arrow starts a new run.

* fix(dashboard): name folders in the Back/Forward history menus as the crumbs do

Holding Back or Forward listed each folder by its path's basename: `/`
was a blank entry, the Shared view read "shared", Trash stayed in
English, and a trashed folder showed its uid. The menus now use the same
names the breadcrumbs show.

* fix(dashboard): show a shortened Files name in full on hover

Grid tiles and a resized Name column shorten long names in the middle
("Quarterl…draft v2.xlsx"), and nothing showed the rest short of opening
Rename or Properties. A shortened name now carries the full one as its
tooltip.

* fix(dashboard): keep keyboard focus on the Files breadcrumbs

Every listing rebuilds the breadcrumbs, which took a keyboard-focused
crumb with it: pressing Enter on one, or any refresh of the folder while
a crumb had focus, left focus on <body>, and the next Tab started over
from the top of the page. Focus now moves to the same folder's crumb,
or to the one for the folder just opened.

* fix(dashboard): reach Home's Shared and Trash rows from the keyboard

At 480px wide or less (a phone, or a desktop browser zoomed in) the
folder sidebar is hidden and Home lists Shared and Trash as rows instead.
Those rows only had a click handler, so a keyboard user couldn't get to
either. They are now focusable links that open on Enter or Space, with
the sidebar's focus ring.

* fix(dashboard): keep the Files selection across a sort or view change

Sorting by a column, switching between list and grid, or toggling Show
hidden lists the folder again, and the rebuild dropped the selection and
the shift anchor: select three files, sort by Size, and nothing was
selected. Only the refresh on returning to the browser tab carried them
across. That logic is now refreshKeepingSelection, and those three use
it too.

* fix(dashboard): don't drop a folder open during a Files refresh

A refresh of the folder on screen (after an upload or paste, Refresh, a
sort, or returning to the browser tab) set the same "already rendering"
flag that a navigation sets, so double-clicking a folder, a sidebar
folder, a crumb or Back during that round-trip did nothing, with no
spinner to explain it.

Opening another folder now takes over from a refresh in flight; the
refresh notices it was superseded and drops its result. A navigation
during another navigation is still ignored, as before.

* fix(dashboard): keep the Files marquee when an item arrives mid-drag

Adding a row clears the empty-folder notice by removing every non-row
child of the listing, and the selection marquee is one. A file added
from elsewhere (an upload, another tab, an app) while the user was
drawing a selection made the rectangle vanish under the pointer.

* perf(dashboard): don't rewrite unchanged Files names while resizing

Every width change (a window resize, collapsing the sidebar) re-fits the
item names, at most once a frame, and wrote each name's text back even
when it came out the same. Each write invalidated layout for the whole
list. With 300 rows that took ~6.4ms a frame in list view and ~4.9ms in
grid; skipping the unchanged writes brings it to ~0.7ms and ~3.2ms.

* fix(dashboard): tell assistive tech the Files Select toggle's state

The Select button only toggled a class, so a screen reader announced a
plain button with no state; it now carries aria-pressed. Sort and View
open menus and now say so with aria-haspopup.

* fix(dashboard): show a person's strongest grant in the share list

When someone held two grants on one item (the owner gave them view
access, and someone the owner let share gave them edit), the share
dialog showed whichever grant the listing returned first, so their row
could read "Can view" while they could edit. A second grant on the same
item now counts toward the row with the stronger of the two levels.

* fix(dashboard): only offer share controls on grants the user can change

The backend lets an item's owner change or withdraw any share of it,
and anyone else only the grants they issued (or their own access). The
share dialog offered the level select and Remove on every row. Someone
the owner let share who removed a person the owner had added got
"Removed bob" while nothing changed, and lowering their level added a
second grant that changed nothing either.

The access list now records who issued each row's grants. A user who
doesn't own the selection gets the controls only on rows whose grants
they issued all of, plus Remove on their own row to leave; other rows
show their level as a tag.

* fix(dashboard): update Files' shared markers after the share dialog closes

The share dialog marks the Files rows it covers as shared or not from
the listings it reloads after each change, and skipped that whole reload
once it was closed. Closing it while a share or removal was still saving
left the row without its shared marker (or with a stale one) until the
folder was reloaded. The markers are now updated either way; only the
dialog's own list is skipped once it's gone.

* fix(dashboard): mark rows shared after sharing more than 50 at once

The share dialog updates the Files rows' shared markers from the share
listings it reloads after a change, and doesn't reload them for more
than 50 items. Sharing such a selection, with a person or by link,
reported "Shared with bob on 55 items" while no row showed the marker
until the folder was reloaded. Those grants now mark their rows
directly.

* fix(gui): unbind each context menu's mousemove handler with the menu

The submenu-aim code bound a mousemove handler on the document for every
menu and submenu and never removed it. Each one kept its closed menu and
items alive and ran on every mouse move for the rest of the session; ten
right-clicks in Files left five such handlers behind. The handler now
comes off when its menu is removed. Moving diagonally into a submenu
still keeps it open.

* fix(gui): ignore clicks on a context menu's dividers

A divider is an <li> like the items, so a click on one (they're 11px
tall, easy to hit between two items) ran the item handler with no
action behind it and threw "Cannot read properties of undefined". The
item handler now skips dividers; the menu stays open, as a native
menu's separator does.

* fix(dashboard): warn that removing a team removes everyone in it

Removing a team from a single item's share dialog asked "Remove Design's
access to this item?", the wording used for one person. The desktop's
share window adds "Everyone in the team loses it."; the dashboard now
uses that confirmation for teams too.

* fix(dashboard): count folders, not items, in "via N folders"

A person who reaches a multi-item selection through more than one shared
ancestor gets a "via N folders" note in the share dialog, and N was the
number of items reached that way: five files under two shared folders
read "via 5 folders". It is now the number of distinct folders.

* fix(dashboard): don't treat a long tail after a dot as a file extension

Files middle-truncates long names and keeps everything after the last
dot whole as the extension. For "Report v1.2 final draft for the board
meeting" that kept ".2 final draft for the board meeting", so the tile
read "….2 final draft for the board meeting" and lost the start of the
name. Only a short tail without spaces now counts as an extension; this
one shortens to "Report v1.…d meeting".

* fix(dashboard): keep Tab inside the touch context menu

The touch menu sheet is modal, but Tab and Shift+Tab left it for the
page behind the backdrop while it stayed open; closing it then no
longer returned focus, since focus was elsewhere. Tab and Shift+Tab now
move between its items like the arrow keys, wrapping at the ends.

* fix(dashboard): say "the other item" when one item is missing a grant

The share dialog's "Add to all" link told screen readers "Give bob
access to the other 1 items" when the person was missing from a single
item of the selection. That case now has its own singular string.

* fix(dashboard): announce a failed Properties load, on both tabs

When Properties couldn't read the item, the General list showed the
error but stayed aria-busy, so screen readers kept it unannounced, and
the Versions tab stayed blank. The list is no longer busy once the
error shows, and Versions shows the same error.

* perf(dashboard): launch the add-app modal's apps without reloading the grid

"Browse the App Center" and "Build with AI" go through the same intro
as a deep link, which first awaited loadApps(). With the grid already
loaded that started a fresh fetch of every app list and the saved
layout, and the launch waited on it: with an 800ms connection, ~1.35s
before anything opened instead of ~0.1s. The intro now uses the grid
already loaded, and only joins or starts a load when there is none.

* fix(dashboard): don't hide the Apps grid on every search keystroke

Search re-rendered the grid behind the first load's fade-in, which waits
for page one's icons. Each keystroke made fresh <img> elements, so an
icon that was slow or failing (an unreachable icon host) blanked the
whole grid until it settled, about a second per keystroke with a 1s
icon and up to 1.5s if it never answers. Search, its clear button and
entering reorder mode now render the results at once.

* fix(dashboard): stop the Apps pager's dots jumping back mid-flip

A page arrow, a dot, or the wheel starts a smooth scroll and marks the
target page at once, but the scroll handler then read the pages passed
over on the way as the current one. The active dot went forward, back
to the old page, and forward again, and a second click on the arrow
added to the stale page: two quick clicks showed 0 → 1 → 0 → 2 → 1 → 2.

While such a scroll is on its way, the pager (and the open folder's)
now ignores the positions in between, until it arrives or a second has
passed. The user's own scrolling still moves the dots.

* fix(dashboard): keep paging while a dragged app is held at the edge

Holding a dragged tile still at the Apps grid's edge (or an open
folder's) turned one page and stopped: the next dwell only started on
another pointer move, so a mouse held steady stayed on the second page
of three. Once a flip settles, the dwell starts again if the pointer is
still in the edge zone.

* fix(dashboard): don't turn Apps pages on a trackpad pinch

A pinch on a trackpad reaches the page as wheel events with ctrlKey
set, and the Apps grid (and an open folder) read them as a wheel flick
and turned the page while the user zoomed. Those events are now left
alone.

The handlers also called preventDefault() from a listener the app makes
passive, which does nothing and logged a console warning on every wheel
tick; that call is gone, as the scroller has no vertical scroll to stop.

* feat(dashboard): open the first Apps search result on Enter

Typing in the Apps search and pressing Enter did nothing; reaching a
result took ArrowDown and then Enter. Enter in a non-empty search now
opens the first match (or a folder that matches), as launchers do. It
does nothing while an IME composition is open, or when nothing matches.

* fix(dashboard): include an open folder's page dots in its Tab cycle

The open folder keeps Tab inside it, but its list of stops left out the
page dots, which are buttons. Tab never reached them, and once one had
focus (clicking it focuses it) the next Tab walked out of the modal
folder to the page behind. The dots are now part of the cycle.

* feat(dashboard): open an app tile's menu with the Menu key or Shift+F10

An app or folder tile's menu (Open in New Tab, Copy Link, Uninstall,
Remove from Folder; a folder's Rename and Ungroup) opened only on a
right-click or long-press, and browsers don't all turn the keyboard's
menu keys into a contextmenu event (macOS has none). The Menu key and
Shift+F10 on a focused tile now open its menu at the tile, where arrow
keys and Enter work.

* fix(dashboard): keep keyboard focus on the grid after Ungroup

Ungrouping a folder from its menu re-rendered the grid without the
folder's tile, so a keyboard user who had it focused was left on
<body> and the next Tab started from the top of the page. Focus now
moves to the folder's first app, which takes the folder's place.

* fix(dashboard): give links AA contrast in the light theme

Link-colored text (Home's "View more" and "Upgrade", the notification
panel's "Mark all read" and "Retry", Properties' website links, the
token dialog's sessions link) used #5271ff, 4.07:1 on white and 3.97:1
on the content background, under the 4.5:1 AA minimum for text that
size. The light theme's link is now #3d5bd9 (5.7:1), with a darker
hover. Dark mode already passed.

* fix(dashboard): give success text AA contrast in the light theme

Success text (the 2FA "enabled" line, Home's active-plan badge, the
share dialog's confirmations) used #03933a: 4.01:1 on white and 3.80:1
on its green tint, under the 4.5:1 minimum. It is now #15803d (5.02:1
and 4.75:1); the success check icons that share the token darken with
it. Dark mode already passed.

* fix(dashboard): give danger buttons and error text AA contrast

White text on the red fills fell short of 4.5:1: the Apps uninstall
confirm (#e53935, 4.23:1), the delete-account button (#f00808, 4.43:1)
and the Files drag "Cancel" zone (#ef4444, 3.76:1). So did light-theme
danger text on its pink tint (#dc2626 on #fef2f2, 4.41:1), used for
form, share, crop and notification errors and the past-due badge.

The fills are now #d32f2f, #d90707 and #dc2626 (4.8:1 to 5.3:1), each
with a darker hover, and danger text is #b91c1c (5.9:1 on the tint).

* fix(dashboard): give the app feedback dialog's Send button AA contrast

The feedback dialog's primary button (Send, and Close once sent) put
white text on a light-blue gradient, 2.66:1 at the top and 3.42:1 at the
bottom. It now uses the dashboard's primary button colors, which meet
4.5:1 in both themes, like the dialogs around it.

* fix(dashboard): show the drop outline and dwell fill on grid folders

Dragging files over a folder in grid view only tinted it like a
selection: the grid's own border, hover and selected rules out-ranked
the shared drop-target and spring-load rules, and the hover rule's
background shorthand wiped the dwell fill. The folder then opened after
700ms with no sign it was about to. Grid and compact-grid folders now
get the list's dashed outline and filling dwell, with the padding
trimmed to match so the tile doesn't shift.

* fix(dashboard): keep the Files preview and drag ghost above the window

The Space-bar image preview (z-index 9999) and the drag ghost (10000)
are appended to <body>, beside the dashboard window. On a touch-only
tablet with a keyboard every window is pinned to 9999999, so the preview
opened invisibly behind it from the first press; on desktop the
window's z-index climbs with each click and overtakes both in a long
session. Both now sit in the band the sidebar tooltip and the toasts
already use.

* fix(dashboard): give small dashboard controls 44px touch targets

Several controls a phone or tablet user taps were under the 44px
minimum touch size, with nothing enlarging them on a coarse pointer:
the Files toolbar buttons (32px), the touch menu's rows (36px tall),
the drawer's close button (32px), the notifications close (30px), the
Properties close (32px), the feedback dialog's ✕ (30px), the add-app
Back arrow (26px), the Apps modal buttons (34px tall) and the share
dialog's Copy link (38px tall).

Following the earlier touch fixes, the icon buttons keep their look
and get a 44x44 hit area, and the text buttons and menu rows grow to
44px. Mouse and trackpad layouts are unchanged.

* fix(dashboard): keep a landscape phone's notch clear of the dashboard

A phone held sideways and wider than 768px (an iPhone Pro Max, a large
Android) gets the desktop layout, and the page asks for the full screen
with viewport-fit=cover. Nothing in that layout read the side safe-area
insets, so with the notch on the left the sidebar's icons sat under it
(x=24 against a 47px inset), and with it on the right the content's
edge and the toasts did.

The sidebar (open or collapsed) now adds the left inset, and the content
area, including Apps and Files, and the toasts add the right one. The
insets are 0 everywhere else, so other layouts don't change.

* fix(dashboard): size the touch menu and bottom sheets to the visible height

The touch context menu, the app feedback dialog and the profile-photo
crop sheet capped their height in vh. On phones 100vh is the viewport
with the browser bars hidden, so while they show these could be taller
than the screen. The touch menu, which ContextMenu.js keeps inside the
visible height, then ran its last rows off the bottom on a landscape
phone, and the bottom sheets could push their header off the top. They
now cap in dvh, as style.css already does.

* fix(gui): move keyboard focus into the Manage sessions dialog

The sessions dialog is modal but never took focus: opening it from the
keyboard left focus on the button behind the backdrop, and Tab walked
the page underneath instead of the session list. It now focuses its
close button on open, keeps Tab inside, and hands focus back to the
control that opened it when it closes.

* fix(dashboard): return focus after the Security tab's dialogs close

Closing Blocked people, or the 2FA setup or disable dialog opened from
the switch, left keyboard focus on <body>, so the next Tab started at
the top of the page. The 2FA switch also disables itself while its
dialog is up, which drops focus even before then. Focus now goes back
to the button or switch that opened the dialog once it has gone.
2026-10-24 08:30:57 -07:00
Daniel SalazarandReynaldi Chernando f3e8539eaf docs: easy events recipe (PUT-1961) (#3998)
* docs: easy events recipe (PUT-1961)

* docs: catch-up-on-notifications events recipe (PUT-1961)

* docs: notification examples use notif:app-user, which apps can read

* docs: events recipe corrections from a run against a live server (PUT-1961)

* refine and update agents md

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-24 15:26:07 +07:00
Nariman Jelveh b3b02b1ebb Fix/dashboard fifth review (#4313)
* fix(dashboard): close the Apps modals and folders only on a press that starts on the backdrop

The uninstall and add-app modals and the open folder closed on any click
whose target was their backdrop. Drag-selecting text in the app request
box and releasing outside the card lands that click on the backdrop, so
the modal and the typed request were thrown away.

The backdrop logic the dashboard dialogs already use moves into
bindBackdropDismiss, and the three Apps overlays use it too.

* fix(dashboard): double-clicking "Add an app" no longer opens App Center

The add-app modal opens under the pointer, so the second click of a
double-click on the "+" tile landed on its first option and launched App
Center (or the AI builder, depending on where the tile sat) without the
user choosing anything. An option now ignores the second click of a
double-click.

* fix(dashboard): act on an app tile once per double-click

A tile acted on every click, so a double-click opened an external app's
site in two browser tabs. Double-clicking an option in the add-app modal
also launched whichever tile sat under the pointer once the modal had
closed (App Center and Recorder together, say). A tile now ignores the
second click of a double-click; the first already launched, focused or
opened what was asked for.

* fix(gui): show app launch refusals in a Puter alert, translated

launchApp reported "You don't have access to …" and "Couldn't open …"
through window.UIAlert, which nothing defines, so both fell through to
the browser's native alert(): a blocking, unstyled, English-only dialog
over the dashboard or desktop. They now use UIAlert directly, with the
text in en.js, and pass the app title unencoded since UIAlert encodes
its message itself.

* fix(dashboard): clicking a toast with nothing to open keeps apps on screen

A toast click brings the dashboard forward before acting on the
notification, hiding every app window. Only share notifications lead
anywhere, so clicking an "events subscription ended" toast (or any other
type) to dismiss it minimized the app the user was working in and showed
a dashboard with nothing to look at. Those toasts now just mark the
notification read.

* fix(dashboard): show an emptied Trash as empty

Emptying the Trash from the Files tab took its rows away but left the
footer on "N items · X MB" and the pane blank, without the empty-folder
notice; the selection bar kept offering actions on rows that were gone.
Emptied from another tab, the listing was cleared with .empty(), which
also dropped the loading overlay and likewise never updated the footer.

Both paths now finish with the empty notice and a fresh footer.

* fix(dashboard): say a folder is empty when its last item goes elsewhere

When another tab, device or app deleted or moved away the last item in
the folder Files was showing, its row faded out and left a blank pane:
the item.removed and item.moved handlers updated the footer but never put
the "folder is empty" notice up. They now do, for rows that were in the
Files listing.

* fix(dashboard): list failed drag-and-drop uploads one per line

The upload-button path was fixed to put each failed item on its own line,
but files dropped onto the listing still joined them with "\n", which
UIAlert's HTML collapses into spaces: every failure ran together on one
line. Both paths now build the message in failedUploadsMessage.

* fix(dashboard): dropping items back into their own folder does nothing

Dropping a selection on the sidebar entry for the folder it came from, or
spring-loading into a folder and coming back to release in the listing,
asked move_items to move every item where it already was, and each one
stopped on a "Cannot move item to its current location" alert in turn.
The breadcrumbs already ignore such a drop; moveDraggedRows now leaves
those rows out too, and a drop with nothing left to move does nothing.

* fix(dashboard): a Ctrl- or Alt-drop on the Trash trashes the items

Ctrl-dragging a file onto the sidebar Trash (or a Trash breadcrumb)
copied it into the Trash: the original stayed put, and the copy carried
no original path, so it was offered Restore and Restore failed. Alt-drops
made shortcuts there the same way.

The four drop handlers' copy/shortcut/move choice moves into
dropRowsInto, which only ever trashes what lands on the Trash.

* fix(dashboard): the first row click after a marquee released outside Files counts

A marquee sets a flag so the click that ends it doesn't clear the new
selection; the listing's click handlers consume it. Released over the
sidebar or outside the window, that click lands elsewhere, the flag
stayed set, and it swallowed the user's next click on a row. The flag now
lapses once the release's own click has had its turn.

* fix(dashboard): Back and Forward follow a folder the user renamed

Open a folder, go back, rename it, then press Forward: history still
held the old path, so Files showed "This folder couldn't be opened", and
the Back/Forward long-press menus listed the old name. A folder renamed
in Files now rebases the history entries in and under it, using the
same path rebase the username-change handler does.

* fix(dashboard): a folder that can't be opened becomes the current folder

When a folder failed to list (deleted elsewhere, share revoked, offline),
Files said "This folder couldn't be opened" but stayed on the previous
folder underneath: history had moved on while the breadcrumbs, sidebar
highlight and New Folder/Upload/Paste still pointed at the old folder,
so a new folder or upload landed somewhere other than where the user
seemed to be.

The tab now moves to the folder it was asked for before showing the
message, and creates nothing in it until it lists again.

* fix(dashboard): double-clicking Save in the photo crop uploads it once

Save closes the crop dialog, but the dialog stays in the page for its
200ms fade-out with its handlers live, so the second click of a
double-click ran Save again and sent the profile picture to the server
twice. Save does nothing once the dialog has closed.
2026-10-23 19:04:36 -07:00
Toshit ChawdaandClaude Opus 5 bdc9aa9e93 Revamp networking api with epoxy try 2 (#3476)
* revamp networking api with epoxy

* update to new epoxy build

* force wisp auth

* emit Error objects from socket 'error' events

main tightened the socket error contract to pass an Error (see
types/modules/networking.d.ts, which documents the reason as
`error.message`); the epoxy rewrite still emitted a bare string,
matching the older `(reason: string)` declaration.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* test: cover the epoxy-based networking client

Replaces the coverage lost when the wisp implementation was removed:
PSocket.test.js and PWispHandler.test.js were bound to PWispHandler,
parsers.js, and wispInfo, none of which survive the epoxy rewrite.

  PSocket.test.js  - connect/retry, inbound data, write normalisation,
                     close, and the TLS event remapping. Driven through
                     real ReadableStream/WritableStream pairs so
                     reader locking, cancel, and abort behave as they
                     do in the browser.
  index.test.js    - relay-token exchange (auth header, 401 re-auth and
                     retry, malformed responses) and the epoxy client
                     cache (keying, refresh, in-flight sharing).
  requests.test.js - pFetch delegation, cache invalidation, and the
                     api call logger's request description.
  epoxy.test.js    - the hand-packed wisp password extension payload.

The wasm runtime loader is left to integration coverage; it needs a
network fetch and a browser. createPuterPasswordBuilder is exported so
the byte layout can be tested against an injected fake runtime.

One skipped test records a pre-existing defect: on a failed write,
#readLoop's normal-termination path races #closeStreams and emits
close(false) after an error.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix: report socket close flags and epoxy init failures accurately

PSocket #readLoop raced the teardown it was being torn down by. A failed
write sets #closing and hands the close event to #closeStreams, which has
to await reader.cancel() and writer.close() before emitting close(true).
That cancel resolves the loop's pending read with {done: true}, so the
loop broke, reached its own #emitClose(false), and set #closed first --
callers were told the socket shut down cleanly right after an error.
The loop now defers the close event whenever a teardown is under way, in
both its normal and its throwing path, and #closeStreams always emits.

getEpoxyClient swallowed every init failure and resolved undefined, so a
dead relay reached callers as "cannot read properties of undefined
(reading 'connect')" instead of its cause. It now re-throws, still
without caching the failed attempt. A socket reports "wasm unavailable"
or "Failed to create relay token (HTTP 503 ...)" and closes with the
error flag set.

Two related cache faults fell out of that rewrite: the in-flight
early-return ignored `refresh`, handing a caller the very attempt it
asked to replace, and it ignored the cache key, so an attempt started
before a token change satisfied a request made after it. Reuse is now
conditional on both.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* restore the public API surface dropped in the epoxy revamp

The epoxy rewrite kept the shape of the old networking client but lost
three things callers depend on:

- JSDoc on the exposed methods. Re-added on PSocket (constructor, on,
  addListener, write, close), PTLSSocket and pFetch, along with the
  SocketEvent typedef and a Networking type on netAPI. `on` carries the
  TLS event aliasing docs that used to live on PTLSSocket, since that
  behaviour moved here.

- socket.write's error message, which lost its trailing '!!'. Apps match
  on the message, so it is restored verbatim.

- pFetch's client-side validation. Epoxy reports a bad request as an
  opaque wasm value from deep inside the transport, so the scheme check,
  the Content-Length/body check and the malformed-request TypeError are
  made before a relay is dialled, rejecting with exactly what they
  rejected with before. Validation builds its Request from a clone, or it
  would consume the body epoxy is about to send.

The latter two are what the net API suite asserts on node, browser and
workerd, and it has been failing on all three since the revamp.

addListener takes named parameters so it can be typed, and three unit
tests that pinned pFetch's habit of forwarding unvalidatable input were
updated to the documented behaviour.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* update epoxy client to 04e4930

connectionPrefs changed shape: the handshake and its required extension
list are one object now rather than a tuple, and handing the new build
the old form throws part way through the handshake.

The bump is not optional. The previous build cannot reach the current
relay at all: it enforces the required puter password extension even when
a relay answers with a v1 CONTINUE, and fails with "Protocol extensions
not supported" -- which is what puter.cafe answers with today.

Verified against a local relay: fetch, POST bodies, https, raw sockets and
TLS sockets, the last exercising connectTls, whose third parameter became
an options object in this build.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* bump epoxy

* dont require ext

* chore(puter.js): update Epoxy runtime to 68c6904

* chore(puter.js): update Epoxy runtime to affd9f5

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-10-23 16:57:25 -07:00
Nariman Jelveh 21c1e4142e fix(dashboard): fourth adversarial review — bugs, UX, a11y, perf (#4312)
* fix(dashboard): keep Escape working when two share or properties dialogs are open

Each share and properties dialog bound its document keydown under one fixed
namespace, so closing either instance unbound every instance's Escape.
Double-clicking Share… or Properties in the context menu opens two; after
the top one closed, Escape no longer closed the one left behind. Each dialog
now binds under its own namespace.

* fix(dashboard): don't close Properties when a text selection ends on the backdrop

Drag-selecting the path or UID in the properties dialog and releasing over
the dimmed backdrop closed the dialog: the click lands on the overlay when
press and release are on different elements. The backdrop now closes it
only when the press started there, as the share dialog already does.

* fix(dashboard): keep focus in Properties after disassociating one of several websites

With more than one website on an item, disassociating one removed the
focused button and returned early before refocusing the dialog, so focus
fell to the page and the next Tab landed in the dashboard behind the modal.
The dialog takes focus back as soon as the entry is removed.

* fix(dashboard): handle the Properties stat failure instead of leaving it unhandled

puter.fs.stat both calls the error callback and rejects. The properties
dialog rendered the error from the callback but dropped the promise, so
opening it on a deleted item or offline logged an unhandled rejection.

* fix(dashboard): don't double-encode the "anyone" and withheld-invite share rows

The share dialog encodes row names when it renders them, but these two
labels came from i18n() already HTML-encoded, so a translation containing
an apostrophe or ampersand would show its entity text. They're now taken
unencoded like every other row name.

* fix(dashboard): make the touch context menu reachable by assistive tech and keyboard

The touch action sheet had no roles and never took focus: it was appended
at the end of the page, so a screen reader announced nothing on open and
focus stayed on the page behind it. It's now a menu of menuitems (with
separators and submenu popups marked), takes focus on open, gives it back
on close, moves between enabled items with the arrow keys and Home/End,
and shows an inset focus ring for keyboard use. Taps draw no ring.

* fix(dashboard): keep Apps folders and order when a settings read fails

The Apps tab read the saved order, folders and uninstalled-apps list with
`.catch(() => null)`, so a failed read looked like a user who had never
saved any. One flaky request emptied the folders and reset the grid to
the default order, and the next drag or folder edit wrote that loss back
to kv for good.

A failed read now keeps what the session already has (the last
uninstalled-apps list included). Until a read of the order or folders has
succeeded, saves stay in memory instead of overwriting the stored value.

* fix(dashboard): drop the Apps tile a failed deep-link launch drew

Landing on /app/<name> (or choosing an option in the add-app modal) adds
the app's tile before the launch runs, because the launch is what installs
it. When the launch failed — no access, no app token, a thrown error — the
tile stayed for the rest of the session, survived refreshes, and every
click on it failed again.

The launch outcome is now passed to settleDeepLinkLaunch, which takes a
tile it drew back out (and restores a previous uninstall record) when
nothing was launched.

* fix(dashboard): put a re-installed app back in its saved slot

Uninstall keeps the app's name in the saved order so a re-install returns
it to its place, but a deep-link landing appended the tile at the end of
the grid. The install animation played on the last page and the tile then
jumped back to its saved slot on the next refresh. The landing now places
the tile through reconcileAppOrder like every other load.

* fix(dashboard): don't double-encode the add-app launch failure alert

UIAlert encodes its message, so passing it the already-encoded i18n string
showed entities on screen in languages with an apostrophe (French read
"s&apos;est mal passé").

* fix(dashboard): say so when an app tile fails to launch

A tile click whose launch rejected (for example the app lookup failing)
only logged to the console: the icon flourish played, settled, and nothing
opened. It now shows the same alert the add-app modal shows for that
failure.

* fix(dashboard): keep focus inside the Apps modals and return it on close

The uninstall and add-app modals are aria-modal but had no Tab trap, so
Tab walked into the search box and tiles behind the scrim, where typing
filtered the grid under the modal. Closing the add-app modal, and
confirming an uninstall, dropped focus to the page body.

Tab now cycles inside both cards; closing the add-app modal returns focus
to the add tile, and an uninstall leaves focus on the tile that takes the
removed app's slot.

* fix(dashboard): upload from a folder's menu into that folder

Upload in a sidebar or breadcrumb folder's menu opened the shared file
picker, whose handler always uploaded into the folder on screen: files
picked for Pictures while viewing Documents landed in Documents. The menu
now hands its folder to the picker.

* fix(dashboard): no New, Upload or Paste where nothing can be created

New Folder, Upload, the New/Paste/Upload menu items and file drops were
only refused in the Trash itself and the Shared view. Inside a trashed
folder they created items without the metadata Restore needs, and at /
every create failed (an upload there threw before its progress window
existed and showed nothing). One check now covers the Trash, folders in
it, the Shared view and /.

* fix(dashboard): make Paste Into Folder work after a cut

The row menu's Paste Into Folder handed a cut to the desktop's
move_clipboard_items, which finds items by their on-screen elements and
plain-string paths. The Files tab's cut stores {path, uid} objects, so
nothing moved and the clipboard was emptied. The menu now takes a paste
handler, and Paste, Ctrl+V and Paste Into Folder share one.

* fix(dashboard): don't cut or copy trashed items out of the Trash

The single-item menu still offered Cut on trashed items, and Ctrl+C
copied them; pasted elsewhere, they kept their raw name in the Trash, a
uid. Trashed items leave through Restore, which puts the original name
back.

* fix(dashboard): Delete Permanently works in every language

The confirmation compared the clicked button's label with 'Delete', so in
any other locale confirming did nothing. The buttons carry values now.

* fix(dashboard): refresh Files on return after a cancelled rename

Escape, or keeping a name unchanged, hid the rename editor but left its
-active class, and the refresh on returning to the browser tab takes that
class for an open editor: the listing stopped refreshing until the next
navigation.

* fix(dashboard): clear a drop target's highlight after the drop

jQuery UI fires no `out` on drop, so the folder row a selection was
dropped on stayed selected (and went to the Trash with the next Delete),
a sidebar folder stayed highlighted as the current one, and a breadcrumb
kept its drop outline.

* fix(dashboard): listing drops keep working after a drop on a folder row

Dropping files from the computer onto a folder row stopped the native
event before the listing's dragster saw it, leaving its enter state stuck:
later drags didn't highlight the listing, and drops on its background
uploaded nothing.

* fix(dashboard): offer Restore only for items Restore can handle

Selecting files inside a trashed folder offered Restore in the
multi-select menu and the selection bar, which then failed once per item:
only direct children of Trash carry restore metadata. The trashed
multi-select menu also drew two separators in a row.

* fix(dashboard): show names as typed in conflict and move-failure alerts

These alerts html-encoded the file name before UIAlert encoded the
message again, so "R&D.txt" read "R&amp;D.txt". "already exists." is
translatable now too.

* fix(dashboard): list failed uploads one per line

The failed-upload list was joined with newlines, which the alert collapses
onto a single line.

* fix(dashboard): translate the shortcut badge's tooltip

* fix(dashboard): select mode toggles rows with any pointer

In select mode, a stylus or mouse tap on a phone or tablet toggled the row
in pointerdown and again in click, leaving it as it was; in a narrow
desktop window a click replaced the selection and the selection bar never
showed. Select mode now toggles in click for every pointer and shows its
bar whenever it is on.

* fix(dashboard): show items added elsewhere while New › Web Link prompts

The New menu's create wrapper suppressed item.added for as long as it ran,
which for Web Link includes its URL prompt, so files added to the folder
meanwhile never appeared. It now counts only from the actual create. A
create that fails now says so instead of failing silently.

* fix(dashboard): keep the Files spinner under the mobile drawer

The loading overlay sat at the maximum z-index, so on phones it drew over
the open drawer and its scrim and took their taps while a folder loaded.
It only needs to sit above the rows.

* fix(dashboard): don't show a failed usage request as zero usage

getMonthlyUsage() resolves with the error body when the request fails, and
Usage and Home read it as a month with nothing used: $0.00 of $0.00, 0%,
and on Usage an empty table replacing the one already on screen. A body
without a usage report is treated as the failure it is, so the last good
numbers stay and a first load shows the "unavailable" note.

* fix(dashboard): draw Files select-mode accents in the selection color

The checked checkbox, the active Select button and the Done button were
filled with --primary-color, the desktop's text color rather than an
accent: charcoal by default, and white (on white text and ticks) for
anyone whose desktop theme uses light text. They use --select-color, the
accent the rest of the selection already uses.

* fix(dashboard): stop the mobile drawer animating under reduced motion

The reduced-motion override for the drawer, its scrim and the menu button
came before the drawer rules it was meant to cancel, and the open
drawer's rule outranked it too, so the drawer still slid and the scrim
still faded. The override now follows those rules and covers the open
state.

* fix(dashboard): make the toast close button readable in dark mode

Dark mode gave the close button the card color and then inverted the
whole element, which turned the disc light grey and the cross white:
about 1.4:1. Without the background the invert gives the light-mode
button's mirror, a dark disc with a white cross.

* fix(dashboard): leave the temporary-account state once the account is saved

Saving a temporary account from the user menu promoted the user in place
without a reload, but Home kept its Save session warning (reopening the
save dialog for a saved account) and Account and Security stayed without
their password and email cards until the page was reloaded. The dashboard
now reacts to account-saved, which fires after the user is updated.

* fix(dashboard): give the API token field 16px text on touch screens

The coarse-pointer rule that sets the token field to 16px, as the other
dialog inputs are, came before the field's own 13px rule of the same
specificity, so the 13px won and iOS zoomed the page when the field was
focused. The touch rule now follows it.

* fix(dashboard): lighten the General access dropdown arrows in dark mode

The dark-mode chevron override covered the share modal's recipient
selects but not the two General access ones, which kept the darker
light-theme stroke on the dark card.

* fix(dashboard): don't leave the sidebar collapse button shaded after a tap

Its hover style was the sidebar's only one not limited to devices that
hover, so on a wide touch screen (an iPad in landscape) the button kept
its hover shade after being tapped.

* fix(dashboard): refresh the Usage tab when the subscription changes

An upgrade from the Usage tab's own button, or a return from the billing
portal, refreshed Home behind it while the Usage meter and plan button
kept the old allowance until the user left the tab and came back. Usage
now handles puter:subscription:changed while it is showing.

* perf(dashboard): load usage and storage once when the dashboard opens

Home and Usage both load the monthly usage and the storage space when the
dashboard opens, whichever tab it opens on, so every open sent
/metering/usage and /df twice. Overlapping loads now share one request.
Nothing is cached: a load after the shared one settles fetches again.

* fix(dashboard): keep keyboard focus on the Usage table's Show more/less

The table redraws without the button that was pressed, so focus fell back
to the page and the next Tab started from the top. Focus now moves to the
button that replaces it, as the sort headers already do.

* fix(dashboard): give primary buttons readable contrast

The dialogs' primary buttons, the share modal's Share and the photo
crop's Save were filled with the link color under white text: 4.07:1 in
light mode, 3.08:1 in dark, where hover lightened it further to 2.38:1.
They take their own fill now, #4a67f5 (4.62:1) darkening to #3d5bd9
(5.68:1) on hover in both themes; links keep their color.

* fix(dashboard): keep collapsed-sidebar labels above the dashboard

The hover label is attached to body at z-index 1000, while the dashboard
window's z-index climbs with every click (focusWindow raises it). Over a
long session the window passed 1000 and the labels rendered behind it,
so hovering the collapsed sidebar showed nothing.

* fix(dashboard): bring muted and tertiary text up to AA contrast

The muted and tertiary greys carry real information (form hints, file
dates and sizes, share "via" lines, the sidebar's team name,
notification text and times) but fell short of 4.5:1: 2.56:1 and 3.54:1
on white, 3.18:1 and 3.13:1 on dark cards. They now clear it on the
cards and the sidebar in both themes (light #5f6b7d / #707070, dark
#8b95a5 / #94949c). Disabled buttons keep the fainter grey through their
own token, so they still read as disabled.

* fix(dashboard): name the plan on self-hosted installs instead of "--"

A self-hosted install has no /marketplace/subscriptions/current route, and
Home treated its 404 like any failed read, leaving "Your Plan" at "--" for
good. A 404 now shows the plan every account starts on. The upgrade badge
and link, which have nothing to reach there, go too, unless the hosted
upgrade dialog is loaded.

* fix(dashboard): make hover and press feedback visible in dark mode

Dark mode's hover color, #2a2a2a, was 1.05:1 against the #262626 cards
and #252525 sidebar, so hovering a notification or sidebar item and
pressing a touch-menu or drawer item showed next to nothing. #333 sits
as far off them as light mode's hover sits off white, and stays below
the selected item's #3d3d3d.

* fix(dashboard): follow a username change in the Files tab

Changing the username, or saving a temporary account under a chosen
name, moves the home folder. The GUI rewrote data-path attributes and the
path globals, but the Files tab kept its current folder, back/forward
history, clipboard and drawn rows on the old path, so sorting, New
Folder, Upload, Up, Back and the breadcrumbs all failed with "This folder
couldn't be opened" until a reload. update_username_in_gui now announces
the change, and the Files tab moves its paths over and redraws.

* perf(dashboard): lay out only the new row when an item is added to Files

Every item added to the open folder from elsewhere (another tab, an
upload, a move into it) restyled and re-truncated every row in the
listing, so a burst of adds was quadratic in the folder's size: 100 adds
into a 300-item folder took ~800ms of main-thread work. The added row now
takes the listing's column layout on its own (~130ms for the same burst).

* fix(dashboard): open the Sort and View menus at their buttons from the keyboard

Enter or Space on either button fires a click with no coordinates, so the
menu opened at the top-left corner of the screen, far from the button.
A keyboard press now opens it below the button; a pointer click still
opens it at the pointer.

* fix(dashboard): open a breadcrumb's long-press menu as the touch sheet

A breadcrumb's long-press opened the desktop context menu at the event's
pointer position, which a long-press doesn't have, so on tablets the menu
landed wherever an unset position put it. Breadcrumbs now open the same
touch sheet, anchored to the crumb, as the sidebar folders and rows do.

* fix(dashboard): don't drag-move items the user can't move

Delete, Cut and their buttons already leave out shared roots and items in
a read-only share, but dropping them on a folder, a sidebar entry or a
breadcrumb still sent the moves, and each came back as a "Forbidden"
alert. Drag moves now leave those rows out too; copying them still works.

* fix(dashboard): clear "1 item selected" when a double-click opens a file

Opening a file by double-click deselects its row, but the footer kept
saying "1 item selected" with nothing selected.

* fix(dashboard): only the Trash gets the Trash label in breadcrumbs

Any folder named "Trash" was shown with the translated Trash label in the
breadcrumbs, so in German ~/Documents/Trash read as "Papierkorb". The
label now goes on the user's Trash folder only.

* fix(dashboard): name a trashed folder by its old name in the breadcrumbs

A trashed item's name on disk is its uid, so opening a trashed folder
showed a crumb like "ffaff8cb-dcaf-44f7-…" while its row in the Trash
read "Old project". The crumb now uses the name the row shows.

* fix(dashboard): show why Delete Permanently failed and keep the item

delete_item fades the item out before the request; when the request
failed it left the item hidden, as if deleted, and passed the alert the
undefined `responseText` of a puter.js error, which threw. The item now
comes back and the alert shows the error's message.

* fix(dashboard): translate the New menu's Web Link and templates strings

"Web Link", its URL prompt, validation and failure messages, the "User
templates" submenu, and the account prompt before publishing a worker
were hardcoded English in the Files New and item menus.

* fix(gui): stop leaking a document click handler per context menu

Every context menu bound its item-click handler on document, delegated
to its own id, and never unbound it. Each menu opened therefore left a
handler that ran on every later click in the page and kept the menu's
items alive, rows and fs entries included. The handler is now bound on
the menu itself and goes with it. It still returns false, so item clicks
keep not reaching document-level click handlers, as before.

* fix(gui): run a context menu item once on a double-click

The menu stays clickable during its 200ms fade-out, so the second click
of a double-click ran the item again: double-clicking Share… or
Properties in the dashboard opened two stacked dialogs. A menu that has
started closing now ignores item clicks.

* fix(dashboard): drag a select-mode selection from its own rows

Select mode returned from pointerdown before recording whether the
pressed row was already selected, so a drag started from that stale
value: dragging selected rows didn't move them, and a stale "selected"
could let a drag from an unselected row clear the selection. The value is
recorded first now.

* fix(dashboard): leave Files select mode when the window widens past 480px

Select mode's Select and Done buttons only show up to 480px, so turning
it on in a narrow window and widening it left every click toggling rows,
with no way out short of narrowing the window again.
2026-10-23 15:24:02 -07:00
Neal Shah 03e483e372 PUT-2252 (#4311) 2026-10-23 17:33:59 -04:00
266 changed files with 28759 additions and 4504 deletions

No files matched your search

+3 -1
View File
@@ -184,10 +184,12 @@ Recipes are tested, copy-pasteable patterns for a task, one layer above the per-
- **Description.** Written for someone who has never heard of Puter. Start with "Learn how to", put the action first, and say what the reader's app gains: "Learn how to add file sharing to your app with Puter.js, so users can share files in their Puter account with other people." No API names, no mechanism, no list of example cases, and no "not just …" contrast.
- **Intro.** Explain the situation and why you'd do this, tied to something the reader already knows ("`puter.kv` is your database in Puter.js"). No "Use this when" bullet list.
- **Structure.** Get to the first working call as early as possible, then add validation, rate limits, and sign-in after it. Explain a Puter concept the first time it appears, such as what `me.puter` is or that `puter.workers.exec()` opens the Puter sign-in window. Split recipes by what the reader needs, not by technique, and give a family of recipes a shared tag.
- **Reading order.** A recipe reads top to bottom. Each code block uses only functions and variables from blocks above it, so the reader never has to search further down the page to understand it. Start with a small version that works, and have each section add one case. When a section changes a function from earlier, show the whole new version of that function there. If the code is built up over several sections, end with a `## Full Code` block that has all of it.
- **Link to the reference for full option lists.** Mention only the options the recipe uses, then link to the function page for the rest ("All the sort options are listed on [`puter.fs.readdir()`](/FS/readdir/)."). A copied list of options, rules, or scopes goes stale when the reference changes.
- **Limits.** Mention a limit only when it's the reason for a pattern the recipe uses, such as one wildcard subscription instead of one per key, and link to [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) or the function page for the number. Leave out limits that code following the recipe never reaches.
- **Code.** Platform-neutral: just the code, with top-level `await`. No `<script>` or `<html>` wrapper, no IIFE.
- **Tradeoffs.** Justify a choice with one scenario the reader can picture right away, in about two short paragraphs, or fold each point into the section where it matters. The description states the benefit only.
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit goes in the section it applies to; other facts belong on the linked reference page.
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit the recipe needs goes in the section it applies to, and other facts belong on the linked reference page.
---
+2
View File
@@ -433,6 +433,8 @@
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
// MiniMax. One key powers chat and video generation (H3, Hailuo);
// video calls go to the same host, beside the chat API's `/v1`.
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
+4 -1
View File
@@ -51,7 +51,10 @@ two pages can't be decoded into how fast the table grows; `openCursor` still
reads a plain cursor. What a cursor wraps is an implementation detail per
store:
- DynamoDB-backed lists wrap `LastEvaluatedKey`.
- DynamoDB-backed lists wrap `LastEvaluatedKey`. A kv listing of another
namespace (cross-app or through a share handle) resumes after the last entry
it returned instead, since the scan can stop on a private one, and its cursor
is sealed to that listing and padded to one length.
- SQL-backed lists wrap a keyset position — `(sortValue, id)` of the last
row — and the query seeks past it (`WHERE (col, id) > (?, ?) ORDER BY col,
id`). Any SQL list gaining a cursor must have a deterministic `ORDER BY`
+1 -1
View File
@@ -19099,7 +19099,7 @@
},
"src/puter-js": {
"name": "@heyputer/puter.js",
"version": "2.6.4",
"version": "2.6.5",
"license": "Apache-2.0",
"dependencies": {
"@heyputer/kv.js": "^0.2.1",
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 90;
const CURRENT_SCHEMA_VERSION = 91;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -129,6 +129,7 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[87, ['0092_share-entry-owner.sql']],
[88, ['0093_fsentries-parent-uid-index.sql']],
[89, ['0094_deleted-apps.sql']],
[90, ['0095_kv-share-handle-levels.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,50 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share levels on `kv_share_handles`. See
-- sqlite/0095_kv-share-handle-levels.sql for the rationale. No per-file
-- applied-state tracking, so every step is guarded and a replay does nothing.
CALL _puter_add_col('kv_share_handles', 'revoked_by', '`revoked_by` varchar(8) DEFAULT NULL');
CALL _puter_add_col('kv_share_handles', 'label', '`label` varchar(512) DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_kv_share_handle_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_grantee'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_grantee` (`grantee_user_id`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_app'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_app` (`app_uid`);
END IF;
END//
DELIMITER ;
CALL _puter_add_kv_share_handle_indexes();
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_indexes;
@@ -0,0 +1,29 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share levels on kv_share_handles. See
-- sqlite/0095_kv-share-handle-levels.sql for the rationale. Idempotent via
-- IF NOT EXISTS; there is no per-file applied-state tracking.
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS revoked_by VARCHAR(8) DEFAULT NULL;
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS label VARCHAR(512) DEFAULT NULL;
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_grantee
ON kv_share_handles (grantee_user_id, id);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_app
ON kv_share_handles (app_uid);
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- What share levels need from `kv_share_handles`. The level itself is not a
-- column: it is the root of the stored `permission`.
--
-- - `revoked_by` : `owner`, `holder` or `system` (the namespace app was
-- deleted). Null while live, and on rows retired before
-- this existed.
-- - `label` : optional owner-set text the holder is shown.
-- - the grantee index serves the holder's listing and their unshare, and
-- the app index retiring a deleted app's handles.
ALTER TABLE `kv_share_handles` ADD COLUMN `revoked_by` TEXT DEFAULT NULL;
ALTER TABLE `kv_share_handles` ADD COLUMN `label` TEXT DEFAULT NULL;
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_grantee`
ON `kv_share_handles` (`grantee_user_id`, `id`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_app`
ON `kv_share_handles` (`app_uid`);
@@ -188,6 +188,47 @@ describe('DDBClient — item operations', () => {
).toBeUndefined();
});
it('puts only while its condition holds', async () => {
const absent = {
expression: 'attribute_not_exists(#pk)',
names: { '#pk': 'pk' },
};
await client.put(
TABLE,
{ pk: 'u7', sk: 'once', v: 1 },
{
condition: absent,
},
);
await expect(
client.put(
TABLE,
{ pk: 'u7', sk: 'once', v: 2 },
{
condition: absent,
},
),
).rejects.toMatchObject({ name: 'ConditionalCheckFailedException' });
expect(
(await client.get(TABLE, { pk: 'u7', sk: 'once' })).Item?.v,
).toBe(1);
await client.put(
TABLE,
{ pk: 'u7', sk: 'once', v: 3 },
{
condition: {
expression: '#v = :held',
names: { '#v': 'v' },
values: { ':held': 1 },
},
},
);
expect(
(await client.get(TABLE, { pk: 'u7', sk: 'once' })).Item?.v,
).toBe(3);
});
it('fetches keys from several tables in one batch', async () => {
await client.put(TABLE, { pk: 'b1', sk: 'a', v: 1 });
await client.put(TABLE, { pk: 'b1', sk: 'b', v: 2 });
+30 -1
View File
@@ -252,7 +252,19 @@ export class DDBClient extends PuterClient {
}
@Span('ddb.put', (table: string) => ({ 'db.table': table }))
async put<T extends Record<string, unknown>>(table: string, item: T) {
async put<T extends Record<string, unknown>>(
table: string,
item: T,
opts?: {
/** Refused (`ConditionalCheckFailedException`) unless it holds. */
condition?: {
expression: string;
names?: Record<string, string>;
values?: Record<string, unknown>;
};
},
) {
const condition = opts?.condition;
const client = await this.#getDocumentClient();
return sendRepairingValues(
item,
@@ -262,6 +274,23 @@ export class DDBClient extends PuterClient {
TableName: table,
Item: itemToWrite,
ReturnConsumedCapacity: 'TOTAL',
...(condition
? {
ConditionExpression: condition.expression,
...(condition.names
? {
ExpressionAttributeNames:
condition.names,
}
: {}),
...(condition.values
? {
ExpressionAttributeValues:
condition.values,
}
: {}),
}
: {}),
}),
),
() => `a put to ${table}`,
+27 -1
View File
@@ -397,6 +397,8 @@ export type EventMap = {
user_id: number;
user_uuid?: string;
stripe_customer_id?: string | null;
/** Apps the account owned, which cascaded away with it. */
appUids?: string[];
};
// ---- Team billing ---- the trigger, never the charge ----
@@ -490,13 +492,33 @@ export type EventMap = {
noShareKeys?: string[];
/** Events handler runs behind the write; see `Actor.handlerDepth`. */
handlerDepth?: number;
/**
* The user who made the change. Differs from `userId` when a share
* holder wrote into the owner's namespace.
*/
actingUserId?: number;
/** Made by a worker session or an events handler; see `viaWorkerOf`. */
viaWorker?: true;
};
/**
* A whole namespace was emptied. Namespace-level on purpose: `flush`'s own
* key enumeration is truncated for a large namespace, so the keys it saw
* are not the keys it removed.
*/
'kv.flushed': { namespace: string; userId: number };
'kv.flushed': {
namespace: string;
userId: number;
actingUserId?: number;
viaWorker?: true;
};
/**
* Share handles stopped being addresses, whoever retired them. `userId` is
* one user who could have subscribed through them, their owner or their
* holder. Listeners end what was made through them even where another grant
* still covers the region. Local, like `permission.revoked`: one region
* settling covers every other.
*/
'kv.share.retired': { handles: string[]; userId: number };
// ---- Notifications ----
/**
@@ -679,6 +701,10 @@ export type EventMap = {
requestHash?: string;
mime: string;
};
// Asked for a request whose host is outside every configured domain. A
// listener that recognises the host sets `result.subdomain` to the site
// it serves; an unclaimed host is answered with a 404.
'site.resolveHost': { host: string; result: { subdomain: string | null } };
// Asked before a hosted file is streamed. A listener that owns the site
// sets `result.allowed = false` to withhold the entry; the visitor then
// sees the same 404 as for a missing file.
@@ -49,7 +49,7 @@ import {
} from '../../core/http/types.js';
import { PuterServer } from '../../server.js';
import { createSecret as otpCreateSecret } from '../../services/auth/OTPUtil.js';
import { kvSharePermission } from '../../services/events/kvShares.js';
import { kvSharePermission } from '../../services/kvShare/kvShares.js';
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
import { setupTestServer } from '../../testUtil.js';
import { resetCardVerificationStatusCache } from '../../util/cardFallback.js';
@@ -2151,7 +2151,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = `service:tg-app:ii:read`;
const permission = `events:background`;
// The controller's grant call delegates through PermissionService
// (which uses ALS Context.set), so wrap in runWithContext.
const res = makeRes();
@@ -2192,7 +2192,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = `service:tg-origin-app:ii:read`;
const permission = `events:background`;
const res = makeRes();
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
@@ -2229,7 +2229,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = `service:tr-origin-app:ii:read`;
const permission = `events:background`;
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
@@ -2294,7 +2294,7 @@ describe('AuthController grant flows', () => {
await inCtx(issuerActor, () =>
controller.handleRevokeUserApp(
makeReq(
{ app_uid: app.uid, permission: 'service:x:ii:read' },
{ app_uid: app.uid, permission: 'events:background' },
{ actor: issuerActor },
),
makeRes(),
@@ -2318,12 +2318,12 @@ describe('AuthController grant flows', () => {
it('grant-user-app: 400 on non-string or oversized origin/app_uid/permission', async () => {
const cases = [
{ origin: { host: 'evil' }, permission: 'service:x:ii:read' },
{ origin: 'https://a.test', permission: ['service:x:ii:read'] },
{ app_uid: 12345, permission: 'service:x:ii:read' },
{ origin: { host: 'evil' }, permission: 'events:background' },
{ origin: 'https://a.test', permission: ['events:background'] },
{ app_uid: 12345, permission: 'events:background' },
{
origin: `https://${'a'.repeat(5000)}.test`,
permission: 'service:x:ii:read',
permission: 'events:background',
},
];
for (const body of cases) {
@@ -2354,7 +2354,7 @@ describe('AuthController grant flows', () => {
{ ownerUserId: target.id },
);
const permission = 'service:squat:ii:read';
const permission = 'events:background';
for (const handler of [
'handleGrantUserApp',
'handleRevokeUserApp',
@@ -2398,7 +2398,7 @@ describe('AuthController grant flows', () => {
{ ownerUserId: target.id },
);
const permission = 'service:squat-beside:ii:read';
const permission = 'events:background';
for (const handler of [
'handleGrantUserApp',
'handleRevokeUserApp',
@@ -2451,7 +2451,7 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const permission = 'service:tp-origin:ii:read';
const permission = 'events:background';
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
@@ -2487,7 +2487,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = 'service:tm-app:ii:read';
const permission = 'events:background';
for (const body of [
{ app_uid: app.uid, permission, meta: 'nope' },
{ app_uid: app.uid, permission, meta: [1, 2] },
@@ -2528,7 +2528,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = 'service:tb-app:ii:read';
const permission = 'events:background';
const oversized = { blob: 'z'.repeat(5000) };
for (const body of [
{ app_uid: app.uid, permission, extra: oversized },
@@ -2605,26 +2605,278 @@ describe('AuthController grant flows', () => {
expect(res.body).toEqual({});
});
it('grant-user-app: 400 on a `permission` wider than the column it lands in', async () => {
// 300 chars is under the 4096 input cap but over `varchar(255)`, so it
// used to reach the INSERT and fault on MySQL/Postgres. The check runs
// after the rewrite, so this has to be a permission no rewriter
// shortens — and it has to reject before the app is resolved, since
// this uid names no app.
await expect(
it('grant-user-app: grants read and write share delegations on a region, never on the namespace', async () => {
const appName = `tkvl-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvShareLevelsApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const root of ['kv-share-read', 'kv-share-write']) {
const namespace = `manage:${root}:${issuer.uuid}:${app.uid}`;
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission: namespace },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
const res = makeRes();
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission: `${namespace}:team` },
{ actor: issuerActor },
),
res,
),
);
expect(res.body).toEqual({});
}
const rows = await server.clients.db.read(
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ? ORDER BY `permission`',
[issuer.id, app.id],
);
expect(rows.map((r: { permission: string }) => r.permission)).toEqual([
`manage:kv-share-read:${issuer.uuid}:${app.uid}:team`,
`manage:kv-share-write:${issuer.uuid}:${app.uid}:team`,
]);
});
it('grant-user-app: refuses a share delegation whose region a new share could not name, writing nothing', async () => {
const appName = `tkvc-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvShareCharsApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const region of ['Team', 'team:Abc', 'caf\u00E9', 'a b'])
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: app.uid,
permission: `manage:kv-share-read:${issuer.uuid}:${app.uid}:${region}`,
},
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
const rows = await server.clients.db.read(
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
[issuer.id, app.id],
);
expect(rows).toEqual([]);
});
it('grant-user-app: refuses a permission outside the app-grantable families, writing nothing', async () => {
const appName = `tng-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestNotGrantableApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
const other = await server.stores.app.create(
{
name: `tng-other-${uuidv4()}`,
title: 'TestNotGrantableOther',
index_url: `https://tng-other-${uuidv4()}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
const refused = [
['driver:puter-image-generation:generate'],
[`user:${target.uuid}:email:read`],
// One bad entry refuses the batch.
['events:background', 'random-string'],
// A share delegation names the app's own namespace only.
[`manage:kv-share:${issuer.uuid}:${other.uid}:rooms`],
];
for (const permissions of refused) {
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permissions },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'permission_not_grantable',
});
}
const rows = await server.clients.db.read(
'SELECT 1 FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
[issuer.id, app.id],
);
expect(rows).toEqual([]);
});
it('grant-user-app: refuses a non-canonical spelling of a grantable permission', async () => {
const appName = `tnc-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestNonCanonicalApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const permission of [
`f\\s:/${issuer.username}/Documents:read`,
`app-root-di\\r:${app.uid}:read`,
`manag\\e:kv-share:${issuer.uuid}:${app.uid}:rooms`,
]) {
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'permission_not_grantable' });
}
});
it('grant-user-app: refuses a share delegation over a prefix no share could be made of, as a share would be', async () => {
const appName = `tkvp-${uuidv4().slice(0, 8)}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvSharePrefixApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const root of ['kv-share', 'kv-share-read', 'kv-share-write']) {
// A share grant one character wider than the column.
const head = `${root}:${issuer.uuid}:${app.uid}:`;
const region = 'a'.repeat(256 - head.length);
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: app.uid,
permission: `manage:${head}${region}`,
},
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
}
});
it('grant-user-app: a share delegation must name the app the request names, by uid, name or origin', async () => {
const appName = `tkvn-${uuidv4()}`;
const origin = `https://${appName}.example.test`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvShareNamedApp',
index_url: `${origin}/index.html`,
},
{ ownerUserId: issuer.id },
);
const other = await server.stores.app.create(
{
name: `tkvn-other-${uuidv4()}`,
title: 'TestKvShareNamedOther',
index_url: `https://tkvn-other-${uuidv4()}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
const grant = (requester: Record<string, string>, namespace: string) =>
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: `app-${uuidv4()}`,
permission: `service:${'a'.repeat(300)}:ii:read`,
...requester,
permission: `manage:kv-share:${issuer.uuid}:${namespace}:rooms`,
},
{ actor: issuerActor },
),
makeRes(),
),
);
for (const requester of [{ app_uid: app.name }, { origin }]) {
await grant(requester, app.uid);
await expect(grant(requester, other.uid)).rejects.toMatchObject({
legacyCode: 'permission_not_grantable',
});
}
// The app's name is not its namespace.
await expect(
grant({ app_uid: app.uid }, app.name),
).rejects.toMatchObject({ legacyCode: 'permission_not_grantable' });
const rows = await server.clients.db.read(
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
[issuer.id, app.id],
);
expect(rows.map((r: { permission: string }) => r.permission)).toEqual([
`manage:kv-share:${issuer.uuid}:${app.uid}:rooms`,
]);
});
it('grant-user-app: 400 on a `permission` wider than the column it lands in', async () => {
// Under the 4096 input cap but over `varchar(255)`, so it would reach
// the INSERT and fault on MySQL/Postgres. The check runs after the
// rewrite, so this has to be a grantable permission no rewriter
// shortens.
const appName = `tw-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestWidthApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
// The shares it delegates fit, at 255 characters; the consent's own
// `manage:` arm takes it past the column.
const head = `kv-share:${issuer.uuid}:${app.uid}:`;
const permission = `manage:${head}${'a'.repeat(255 - head.length)}`;
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ statusCode: 400 });
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
message: 'Invalid `permission`',
});
});
it('grant-user-app: an oversized permission that a rewriter shortens is accepted', async () => {
@@ -2636,6 +2888,8 @@ describe('AuthController grant flows', () => {
// on "please try again" for any deeply nested file. Exercised here
// through a rewriter of our own, since the real fs rewriter resolves
// the path through the fsentry store and this suite has no entries.
// It rewrites a share delegation only because that is a grantable
// shape no production rewriter touches.
const appName = `tlp-${uuidv4()}`;
const app = await server.stores.app.create(
{
@@ -2646,13 +2900,15 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const prefix = `tlprw-${uuidv4()}`;
const longPermission = `${prefix}:${'a'.repeat(300)}:read`;
const shortPermission = `${prefix}:${uuidv4()}:read`;
// Over the column only by its `manage:` arm, so the shares it
// delegates still fit.
const head = `kv-share:${issuer.uuid}:${app.uid}:`;
const longPermission = `manage:${head}${'a'.repeat(255 - head.length)}`;
const shortPermission = `manage:${head}${uuidv4()}`;
expect(longPermission.length).toBeGreaterThan(255);
expect(shortPermission.length).toBeLessThanOrEqual(255);
server.services.permission.registerRewriter({
id: `test-shorten-${prefix}`,
id: `test-shorten-${app.uid}`,
matches: (permission: string) => permission === longPermission,
rewrite: async () => shortPermission,
});
@@ -2712,7 +2968,8 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const prefix = `conc-${uuidv4()}`;
// A grantable shape no production rewriter touches.
const prefix = `manage:kv-share:${issuer.uuid}:${app.uid}:conc-${uuidv4()}`;
let release: () => void;
let entered: () => void;
const held = new Promise<void>((r) => {
@@ -2740,7 +2997,7 @@ describe('AuthController grant flows', () => {
makeReq(
{
app_uid: app.uid,
permission: `${prefix}:${app.uid}:write`,
permission: `${prefix}:write`,
},
{ actor: issuerActor },
),
@@ -2781,8 +3038,8 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const prefix = `tardrw-${uuidv4()}`;
const pseudoPermission = `${prefix}:${app.uid}:write`;
const prefix = `manage:kv-share:${issuer.uuid}:${app.uid}:tardrw-${uuidv4()}`;
const pseudoPermission = `${prefix}:write`;
const resolvedPermission = `fs:${uuidv4()}:write`;
const NOTHING = 'nothing-in-particular';
server.services.permission.registerRewriter({
@@ -3023,7 +3280,7 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
// phase A's pre-flight passes, phase C's re-check then fails, so the
// leaf (and its auto-created parent) already exist by the time the
// request unwinds.
const prefix = `rbparent-${uuidv4()}`;
const prefix = `manage:kv-share:${user.uuid}:${appUid}:rbparent-${uuidv4()}`;
let calls = 0;
server.services.permission.registerRewriter({
id: prefix,
@@ -3189,13 +3446,13 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
});
it('does not apply `create` to a `manage:` grant — 404 unchanged, nothing created', async () => {
it('refuses a `manage:` grant outright, creating nothing', async () => {
const p = path(`manage-missing-${rand()}`);
await expect(
grant({ permission: `manage:fs:${p}:write` }),
).rejects.toMatchObject({
statusCode: 404,
legacyCode: 'subject_does_not_exist',
statusCode: 400,
legacyCode: 'permission_not_grantable',
});
expect(await server.stores.fsEntry.getEntryByPath(p)).toBeFalsy();
});
@@ -3285,7 +3542,7 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
// (so nothing is rejected before anything is created) but has grown
// too wide by the time phase C re-runs it — modelling a grant that
// only fails once the fs path from the same batch already exists.
const prefix = `rbtest-${uuidv4()}`;
const prefix = `manage:kv-share:${user.uuid}:${appUid}:rbtest-${uuidv4()}`;
let calls = 0;
server.services.permission.registerRewriter({
id: `test-rollback-${prefix}`,
@@ -3326,6 +3583,37 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
);
});
it('check-permissions answers an app asking about itself through ACL, so a folder grant covers what is inside', async () => {
const dir = path(`acl-self-${rand()}`);
const child = `${dir}/inner`;
await server.services.fs.mkdir(user.id, {
path: child,
createMissingParents: true,
});
expect(await checkAsApp(`fs:${child}:read`)).toBe(false);
await grant({ permission: `fs:${dir}:write`, create: false });
expect(await checkAsApp(`fs:${child}:read`)).toBe(true);
expect(await checkAsApp(`fs:${child}:write`)).toBe(true);
// Another user's home stays out of reach.
expect(await checkAsApp(`fs:/${target.username}/Documents:read`)).toBe(
false,
);
});
it('check-permissions answers an app asking about itself through ACL for a uid too', async () => {
const dir = path(`acl-self-uid-${rand()}`);
const child = await server.services.fs.mkdir(user.id, {
path: `${dir}/inner`,
createMissingParents: true,
});
expect(await checkAsApp(`fs:${child.uuid}:read`)).toBe(false);
await grant({ permission: `fs:${dir}:read`, create: false });
expect(await checkAsApp(`fs:${child.uuid}:read`)).toBe(true);
expect(await checkAsApp(`fs:${child.uuid}:write`)).toBe(false);
});
it('check-permissions on a missing fs path answers false rather than 404ing, without poisoning a mixed batch', async () => {
const missingPath = path(`.checkmissing-${rand()}`);
// A held fs grant: the owning user holds it implicitly (is-owner), so
@@ -4152,9 +4440,9 @@ describe('AuthController.handleCreateAccessToken + handleRevokeAccessToken', ()
['an oversized permission string', ['x'.repeat(4097)]],
[
'an oversized `extra`',
[['service:x:ii:read', { a: 'y'.repeat(5000) }]],
[['events:background', { a: 'y'.repeat(5000) }]],
],
['a non-object `extra`', [['service:x:ii:read', 'nope']]],
['a non-object `extra`', [['events:background', 'nope']]],
['an empty permission string', ['']],
])('rejects %s with 400', async (_label, permissions) => {
await expect(
@@ -7365,6 +7653,88 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
]);
});
it('check-permissions: answers false for share grants, which name what a holder is never told, but not for their manage arm', async () => {
const { user: owner, actor: ownerActor } = await makeUserAndActor();
const { user: holder, actor: holderActor } = await makeUserAndActor();
const region = `notes-${uuidv4().slice(0, 8)}`;
const grant = `kv-share-read:${owner.uuid}:os-global:${region}`;
await inCtx(ownerActor, () =>
server.services.permission.grantUserUserPermission(
ownerActor,
holder.username,
grant,
),
);
expect(await server.services.permission.check(holderActor, grant)).toBe(
true,
);
const asked = [
grant,
`kv-share:${owner.uuid}:os-global:${region}:inner`,
`kv-share-write:${owner.uuid}:os-global:${region}`,
];
const res = makeRes();
await inCtx(holderActor, () =>
controller.handleCheckPermissions(
makeReq({ permissions: asked }, { actor: holderActor }),
res,
),
);
expect(res.body).toEqual({
permissions: Object.fromEntries(asked.map((p) => [p, false])),
});
// What `puter.perms.check('kvShare', …)` asks.
const manage = `manage:kv-share-read:${owner.uuid}:os-global:${region}`;
const ownerRes = makeRes();
await inCtx(ownerActor, () =>
controller.handleCheckPermissions(
makeReq({ permissions: [manage] }, { actor: ownerActor }),
ownerRes,
),
);
expect(ownerRes.body).toEqual({ permissions: { [manage]: true } });
});
it('check-permissions: answers false for share grants asked by an app actor too', async () => {
const { user: owner, actor: ownerActor } = await makeUserAndActor();
const { user: holder, actor: holderActor } = await makeUserAndActor();
const app = await server.stores.app.create(
{
name: `cpk-${uuidv4()}`,
title: 'TestCheckPermsKvApp',
index_url: 'https://check-perms-kv.example.test/index.html',
},
{ ownerUserId: holder.id },
);
const grant = `kv-share-read:${owner.uuid}:${app.uid}:notes-${uuidv4().slice(0, 8)}`;
await inCtx(ownerActor, () =>
server.services.permission.grantUserUserPermission(
ownerActor,
holder.username,
grant,
),
);
const appActor = makeActor({
user: holderActor.user,
app: { id: app.id, uid: app.uid },
});
// The holder's app in the share's namespace holds it.
expect(await server.services.permission.check(appActor, grant)).toBe(
true,
);
const res = makeRes();
await inCtx(appActor, () =>
controller.handleCheckPermissions(
makeReq({ permissions: [grant] }, { actor: appActor }),
res,
),
);
expect(res.body).toEqual({ permissions: { [grant]: false } });
});
// What lets a permission request settle without a prompt. The second string
// is a consent scope nothing implies, so the first's `true` is the grant.
it('check-permissions: an app-under-user actor sees a grant made to that app', async () => {
@@ -7579,7 +7949,7 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
},
{ ownerUserId: user.id },
);
const permission = 'service:tl-app:ii:read';
const permission = 'events:background';
await inCtx(actor, () =>
controller.handleGrantUserApp(
makeReq({ app_uid: app.uid, permission, extra: {} }, { actor }),
@@ -10215,7 +10585,7 @@ describe('AuthController — app-data grants', () => {
await expect(
post('handleGrantUserApp', {
app_uid: grantee.uid,
permission: 'app-data:app-nope/:kv:read',
permission: `app-data:app-${uuidv4()}:kv:read`,
}),
).rejects.toMatchObject({ statusCode: 404 });
});
@@ -10282,7 +10652,7 @@ describe('AuthController — app-data grants', () => {
it('leaves unrelated permissions untouched by the new validation', async () => {
const grantee = await makeAppRow();
const permission = 'service:unrelated:ii:read';
const permission = 'events:background';
await post('handleGrantUserApp', {
app_uid: grantee.uid,
permission,
+65 -9
View File
@@ -33,7 +33,7 @@ import {
hasVerifiedPhone,
} from '../../core/http/middleware/gates.js';
import type { Actor } from '../../core/actor.js';
import { isPlainUserActor, makeActor } from '../../core/actor.js';
import { isAppActor, isPlainUserActor, makeActor } from '../../core/actor.js';
import {
checkRateLimit,
consumeRouteRateLimit,
@@ -88,13 +88,16 @@ import {
import { promoteToVerifiedGroup } from '../../util/userProvisioning.js';
import {
assertBoundedManageGrant,
assertNewShareSegments,
assertPrefixFitsAccess,
isKvSharePermission,
} from '../../services/events/kvShares.js';
} from '../../services/kvShare/kvShares.js';
import {
APP_DATA_PERMISSION_PREFIX,
appDataSharingAllowed,
parseAppDataPermission,
} from '../../services/permission/appDataScopes.js';
import { parseAppGrantablePermission } from '../../services/permission/appGrantable.js';
import {
assertCreatablePath,
isOwnHomePath,
@@ -2997,6 +3000,45 @@ export class AuthController extends PuterController {
return typeof permission === 'string' && permission ? [permission] : [];
}
/**
* Refuses any entry outside the families an app can be granted. Anything
* else is a row the backend never checks for an app, or one wider than the
* consent prompt can describe.
*/
async #assertAppGrantable(
actor: Actor,
appIdentifier: string,
list: string[],
): Promise<void> {
const refuse = (entry: string) =>
new HttpError(
400,
`Permission cannot be granted to an app: ${entry}`,
{ legacyCode: 'permission_not_grantable' },
);
let appUid: string | undefined;
for (const entry of list) {
// A delegation over a whole key-value namespace has no bounded
// description; refused with its own code.
assertBoundedManageGrant(entry);
const parsed = parseAppGrantablePermission(entry, actor.user!.uuid);
if (!parsed) throw refuse(entry);
if (parsed.family !== 'kvShare') continue;
// Held to what a new share's prefix may be, so a consent never
// covers shares that could not be made.
assertNewShareSegments(parsed.segments);
assertPrefixFitsAccess(
parsed.ownerUuid,
parsed.appUid,
parsed.segments.join(':'),
parsed.level,
);
// An app shares regions of its own namespace only.
appUid ??= (await this.stores.app.resolveApp(appIdentifier))?.uid;
if (parsed.appUid !== appUid) throw refuse(entry);
}
}
/**
* Gate a cross-app data grant: the target must exist, must not have opted
* out of sharing, and must be named. Also creates the target's AppData
@@ -3175,6 +3217,7 @@ export class AuthController extends PuterController {
}
// 400 on a bad value, before anything else runs. Absent means create.
const create = parseCreateFlag(req.body?.create);
await this.#assertAppGrantable(req.actor!, app_uid, list);
// Validate every entry before writing or creating anything, so a bad
// one in the list cannot leave a partially-granted set behind: the
@@ -3187,10 +3230,6 @@ export class AuthController extends PuterController {
kind: FsCreateKind;
}> = [];
for (const entry of list) {
// A delegation over a whole key-value namespace has no bounded
// description, so it is refused where it is asked for rather than
// prompted for and then refused at use.
assertBoundedManageGrant(entry);
await this.#prepareAppDataGrant(req.actor!, entry);
const plan = await this.#planFsPathCreate(
req.actor!,
@@ -3397,11 +3436,28 @@ export class AuthController extends PuterController {
const unique = [...new Set(permissions)] as string[];
const result: Record<string, boolean> = {};
// `fs:` reaches down a tree, and only ACL walks that.
// A share grant names its owner and prefix, which its holder is never
// told, so asking about one would confirm a guess. `manage:` arms are
// the app's own consent and stay answerable.
const answerable: string[] = [];
for (const permission of unique) {
if (
typeof permission === 'string' &&
isKvSharePermission(permission)
)
result[permission] = false;
else answerable.push(permission);
}
// `fs:` reaches down a tree, and only ACL walks that. An app asking
// about itself gets the same answer, so a grant on a parent folder
// doesn't read as missing and prompt again.
const viaAcl = new Set(
app_uid ? unique.filter((p) => aclAnswerableFsPermission(p)) : [],
app_uid || isAppActor(actor)
? answerable.filter((p) => aclAnswerableFsPermission(p))
: [],
);
const viaScan = unique.filter((p) => !viaAcl.has(p));
const viaScan = answerable.filter((p) => !viaAcl.has(p));
let granted: Map<string, boolean>;
try {
@@ -154,8 +154,9 @@ export class EventsController extends PuterController {
// -- Cross-user key-value handles --------------------------------
/**
* POST /events/kv-handles — hand another user a watchable region of this
* account's key-value data.
* POST /events/kv-handles — hand another user a region of this account's
* key-value data, `watch` unless the body asks for more. The older name for
* `POST /kv/shares`, keeping its own body and response shapes.
*
* An account session, or an app session holding a `manage:` delegation on
* the region — the service tells those apart and gates each rather than the
@@ -167,12 +168,32 @@ export class EventsController extends PuterController {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
requireVerified: true,
})
async mintKvHandle(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.mintKvHandle(actor, bodyRecord(req)),
const body = bodyRecord(req);
const share = await this.services.kvShare.mintKvHandle(
actor,
{
prefix: body.prefix,
recipient: {
username:
typeof body.granteeUsername === 'string'
? body.granteeUsername
: undefined,
uid:
typeof body.granteeUid === 'string'
? body.granteeUid
: undefined,
},
access: body.access,
appUid: body.appUid,
},
// Callers of this route mint again to get the same handle back.
{ defaultAccess: 'watch', keepLevelUnlessNamed: true },
);
res.json({ handle: share.handle, prefix: share.prefix });
}
/**
@@ -189,7 +210,7 @@ export class EventsController extends PuterController {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.services.events.listKvHandles(actor, {
const page = await this.services.kvShare.listKvHandles(actor, {
limit: normalizeLimit(query.limit, {
cap: KV_HANDLE_LIST_LIMIT_CAP,
}),
@@ -198,7 +219,10 @@ export class EventsController extends PuterController {
});
res.json({
items: page.items,
items: page.items.map(({ holderUsername, ...row }) => ({
...row,
granteeUsername: holderUsername,
})),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
@@ -217,7 +241,10 @@ export class EventsController extends PuterController {
async revokeKvHandle(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.revokeKvHandle(actor, req.params.handle),
await this.services.kvShare.revokeKvHandle(
actor,
req.params.handle,
),
);
}
+10
View File
@@ -163,6 +163,16 @@ export const EVENTS_ACK_LIMIT = userWindow('events:ack', 600);
*/
export const EVENTS_KV_HANDLE_LIMIT = userWindow('events:kvHandles', 60);
/**
* `kv.share.received` notices per hour, per owner and holder. A share past it
* is still made, without a notice.
*/
export const KV_SHARE_NOTICE_LIMIT = userWindow(
'kvShare:notices',
5,
60 * 60_000,
);
/**
* Live share handles one account may hold out at a time, across every app.
*
+2
View File
@@ -27,6 +27,7 @@ import { EventsController } from './events/EventsController.js';
import { FSController } from './fs/FSController.js';
import { HomepageController } from './homepage/HomepageController.js';
import { HostingController } from './hosting/HostingController.js';
import { KvShareController } from './kvShare/KvShareController.js';
import { LegacyFSController } from './fs/LegacyFSController.js';
import { NotificationController } from './notification/NotificationController.js';
import { OIDCController } from './oidc/OIDCController.js';
@@ -58,6 +59,7 @@ export const puterControllers = {
broadcast: BroadcastController,
notification: NotificationController,
events: EventsController,
kvShare: KvShareController,
share: ShareController,
webdav: WebDAVController,
oidc: OIDCController,
@@ -0,0 +1,478 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import {
createTestUser,
setupPuterTestEnv,
type PuterTestEnv,
type TestUserCredentials,
} from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import { EVENTS_KV_HANDLE_LIMIT } from '../events/limits.js';
const BOOT_TIMEOUT_MS = 120_000;
const call = (
env: PuterTestEnv,
method: string,
path: string,
token?: string,
body?: unknown,
) =>
fetch(new URL(path, env.apiOrigin), {
method,
headers: {
'content-type': 'application/json',
...(token ? { authorization: `Bearer ${token}` } : {}),
},
...(body === undefined ? {} : { body: JSON.stringify(body) }),
});
const resetBudgets = async (env: PuterTestEnv) => {
for (const name of [env.users.user.username, env.users.other.username]) {
const user = await env.server.stores.user.getByUsername(name);
await env.server.clients.redis.del(
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user!.id}`,
);
}
};
describe('/kv/shares', () => {
let env: PuterTestEnv;
let ownerUuid: string;
const req = (
method: string,
path: string,
token?: string,
body?: unknown,
) => call(env, method, path, token, body);
beforeAll(async () => {
env = await setupPuterTestEnv({
events: { enabled: true, kvHandles: true },
kv: { shareHandles: true },
unlimitedMetering: true,
} as IConfig);
const owner = await env.server.stores.user.getByUsername(
env.users.user.username,
);
ownerUuid = owner!.uuid as string;
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await env?.shutdown();
});
beforeEach(async () => {
await resetBudgets(env);
});
const shareWithOther = async (body: Record<string, unknown> = {}) => {
const res = await req('POST', '/kv/shares', env.users.user.token, {
prefix: `http-${uuidv4().slice(0, 8)}:`,
recipient: env.users.other.username,
...body,
});
expect(res.status).toBe(200);
return (await res.json()) as Record<string, unknown>;
};
it('shares at read by default and answers in the owner`s shape', async () => {
const shared = await shareWithOther({ label: 'Groceries' });
expect(Object.keys(shared).sort()).toEqual(
[
'access',
'appUid',
'createdAt',
'handle',
'holder',
'label',
'prefix',
].sort(),
);
expect(shared).toMatchObject({
access: 'read',
appUid: null,
holder: env.users.other.username,
label: 'Groceries',
});
});
it('takes the recipient as an object too, and refuses an address', async () => {
await shareWithOther({
recipient: { username: env.users.other.username },
access: 'write',
});
const refused = await req('POST', '/kv/shares', env.users.user.token, {
prefix: 'address:',
recipient: '[email protected]',
});
expect(refused.status).toBe(400);
await expect(refused.json()).resolves.toMatchObject({
code: 'invalid_recipient',
});
});
it('lists what was shared out, and what was shared with the holder without the owner', async () => {
const shared = await shareWithOther({ access: 'write', label: 'Trip' });
const byMe = await req(
'GET',
'/kv/shares?limit=200',
env.users.user.token,
);
const byMePage = (await byMe.json()) as {
items: Array<Record<string, unknown>>;
};
expect(
byMePage.items.find((row) => row.handle === shared.handle),
).toEqual({
handle: shared.handle,
prefix: shared.prefix,
access: 'write',
appUid: null,
holder: env.users.other.username,
label: 'Trip',
createdAt: shared.createdAt,
revokedAt: null,
revokedBy: null,
});
const withMe = await req(
'GET',
'/kv/shares/with-me?limit=200',
env.users.other.token,
);
const withMeText = await withMe.text();
const withMePage = JSON.parse(withMeText) as {
items: Array<Record<string, unknown>>;
};
expect(
withMePage.items.find((row) => row.handle === shared.handle),
).toEqual({
handle: shared.handle,
access: 'write',
appUid: null,
label: 'Trip',
createdAt: shared.createdAt,
});
expect(withMeText).not.toContain(env.users.user.username);
expect(withMeText).not.toContain(ownerUuid);
expect(withMeText).not.toContain(String(shared.prefix));
});
it('lets the holder unshare, and shows the owner they did', async () => {
const shared = await shareWithOther();
const removed = await req(
'DELETE',
`/kv/shares/${shared.handle}`,
env.users.other.token,
);
expect(removed.status).toBe(200);
await expect(removed.json()).resolves.toMatchObject({
handle: shared.handle,
});
const byMe = (await (
await req('GET', '/kv/shares?limit=200', env.users.user.token)
).json()) as { items: Array<Record<string, unknown>> };
expect(
byMe.items.find((row) => row.handle === shared.handle),
).toMatchObject({ revokedBy: 'holder' });
});
it('keeps the owner and prefix out of the holder`s grant audit after they unshare', async () => {
const shared = await shareWithOther({ access: 'write' });
const segment = String(shared.prefix).slice(0, -1);
const other = await env.server.stores.user.getByUsername(
env.users.other.username,
);
// A row an earlier build filed under the holder.
await env.server.stores.permission.auditUserUserPerm({
holder_user_id: other!.id,
issuer_user_id: other!.id,
permission: `kv-share-write:${ownerUuid}:os-global:${segment}`,
action: 'revoke',
reason: 'kv share handle unshared',
});
const removed = await req(
'DELETE',
`/kv/shares/${shared.handle}`,
env.users.other.token,
);
expect(removed.status).toBe(200);
// The revoke is filed under the owner, who granted it.
const auditOf = async (token: string) => {
const res = await req('GET', '/share/audit?limit=200', token);
expect(res.status).toBe(200);
return await res.text();
};
await vi.waitFor(async () => {
const owners = JSON.parse(await auditOf(env.users.user.token)) as {
items: Array<Record<string, unknown>>;
};
expect(owners.items).toContainEqual(
expect.objectContaining({
action: 'revoke',
issuer: env.users.user.username,
holder: env.users.other.username,
permission: expect.stringContaining(segment),
}),
);
});
const holders = await auditOf(env.users.other.token);
expect(holders).not.toContain(ownerUuid);
expect(holders).not.toContain(segment);
});
it('lets the owner revoke', async () => {
const shared = await shareWithOther();
const removed = await req(
'DELETE',
`/kv/shares/${shared.handle}`,
env.users.user.token,
);
expect(removed.status).toBe(200);
});
it('answers anyone else, and an unknown handle, with the same 404', async () => {
const shared = await shareWithOther();
const stranger = await createTestUser(env.server, {
username: `kvs-${uuidv4().slice(0, 8)}`,
password: 'pw-test-1234',
});
const strangers = await req(
'DELETE',
`/kv/shares/${shared.handle}`,
stranger.token,
);
const unknownHandle = `kvh-${uuidv4()}`;
const unknown = await req(
'DELETE',
`/kv/shares/${unknownHandle}`,
env.users.user.token,
);
expect(strangers.status).toBe(404);
expect(unknown.status).toBe(404);
const a = (await strangers.json()) as Record<string, unknown>;
const b = (await unknown.json()) as Record<string, unknown>;
expect(a.code).toBe(b.code);
expect(a.message).toBe(`No such handle: ${shared.handle}`);
expect(b.message).toBe(`No such handle: ${unknownHandle}`);
});
it('turns an anonymous caller away', async () => {
const res = await req('GET', '/kv/shares');
expect(res.status).toBeGreaterThanOrEqual(400);
});
});
describe('/events/kv-handles beside /kv/shares', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv({
events: { enabled: true, kvHandles: true },
kv: { shareHandles: true },
unlimitedMetering: true,
} as IConfig);
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await env?.shutdown();
});
it('keeps its own request and response shapes, at watch by default', async () => {
const minted = await call(
env,
'POST',
'/events/kv-handles',
env.users.user.token,
{
granteeUsername: env.users.other.username,
prefix: 'alias-shape:',
},
);
expect(minted.status).toBe(200);
const body = (await minted.json()) as Record<string, unknown>;
expect(body).toEqual({ handle: body.handle, prefix: 'alias-shape:' });
const listed = (await (
await call(
env,
'GET',
'/events/kv-handles?limit=200',
env.users.user.token,
)
).json()) as { items: Array<Record<string, unknown>> };
const row = listed.items.find((one) => one.handle === body.handle);
expect(row).toMatchObject({
prefix: 'alias-shape:',
appUid: 'os-global',
granteeUsername: env.users.other.username,
access: 'watch',
revokedBy: null,
});
expect(row).not.toHaveProperty('holder');
// The alias revoke stays the owner's: the holder is told it is absent.
const byHolder = await call(
env,
'DELETE',
`/events/kv-handles/${body.handle}`,
env.users.other.token,
);
expect(byHolder.status).toBe(404);
});
it('hands back a share it finds at the level it has, unless the body names one', async () => {
const shared = (await (
await call(env, 'POST', '/kv/shares', env.users.user.token, {
prefix: 'alias-keep:',
recipient: env.users.other.username,
access: 'write',
})
).json()) as Record<string, unknown>;
const accessOf = async () => {
const listed = (await (
await call(
env,
'GET',
'/kv/shares?limit=200',
env.users.user.token,
)
).json()) as { items: Array<Record<string, unknown>> };
return listed.items.find((one) => one.handle === shared.handle)
?.access;
};
const mintAlias = (extra: Record<string, unknown> = {}) =>
call(env, 'POST', '/events/kv-handles', env.users.user.token, {
granteeUsername: env.users.other.username,
prefix: 'alias-keep:',
...extra,
});
const again = await mintAlias();
expect(again.status).toBe(200);
await expect(again.json()).resolves.toEqual({
handle: shared.handle,
prefix: 'alias-keep:',
});
expect(await accessOf()).toBe('write');
expect((await mintAlias({ access: 'read' })).status).toBe(200);
expect(await accessOf()).toBe('read');
});
});
/**
* Sharing asks for a confirmed email and nothing else. Booted with an SMS
* provider configured, so a phone requirement would bite here if one were ever
* added to these routes.
*/
describe('verification on the mint routes', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv({
events: { enabled: true, kvHandles: true },
kv: { shareHandles: true },
unlimitedMetering: true,
strict_email_verification_required: true,
prelude: { apiKey: 'test-key' },
} as IConfig);
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await env?.shutdown();
});
const makeUser = async (
patch: Record<string, unknown> = {},
): Promise<TestUserCredentials> => {
const username = `kvv${uuidv4().slice(0, 8)}`;
const creds = await createTestUser(env.server, {
username,
password: 'pw-test-1234',
});
if (Object.keys(patch).length > 0) {
const row = await env.server.stores.user.getByUsername(username);
await env.server.stores.user.update(row!.id, patch);
}
return creds;
};
const mintBoth = (owner: TestUserCredentials, prefix: string) =>
Promise.all([
call(env, 'POST', '/kv/shares', owner.token, {
prefix: `${prefix}a:`,
recipient: env.users.other.username,
}),
call(env, 'POST', '/events/kv-handles', owner.token, {
prefix: `${prefix}b:`,
granteeUsername: env.users.other.username,
}),
]);
it('refuses an account whose email is not confirmed, on both routes', async () => {
const owner = await makeUser();
for (const res of await mintBoth(owner, 'unconfirmed')) {
expect(res.status).toBe(403);
await expect(res.json()).resolves.toMatchObject({
code: 'account_is_not_verified',
});
}
});
it('lets a confirmed account share with no phone and no card', async () => {
const owner = await makeUser({
email: `kvv-${uuidv4().slice(0, 8)}@example.invalid`,
email_confirmed: 1,
});
for (const res of await mintBoth(owner, 'confirmed'))
expect(res.status).toBe(200);
});
it('never gates listing or unsharing', async () => {
const owner = await makeUser();
for (const path of ['/kv/shares', '/kv/shares/with-me'])
expect((await call(env, 'GET', path, owner.token)).status).toBe(
200,
);
const removed = await call(
env,
'DELETE',
`/kv/shares/kvh-${uuidv4()}`,
owner.token,
);
expect(removed.status).toBe(404);
});
});
@@ -0,0 +1,172 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import type { Actor } from '../../core/actor.js';
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { bodyRecord } from '../../core/http/requestBody.js';
import { parseKvShareRecipient } from '../../services/kvShare/kvShares.js';
import type { KvShareHandleListOptions } from '../../stores/events/KvShareHandleStore.js';
import { KV_HANDLE_LIST_LIMIT_CAP } from '../../stores/events/KvShareHandleStore.js';
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
import { normalizeLimit } from '../../util/pagination.js';
import { EVENTS_LIST_LIMIT } from '../events/limits.js';
import { PuterController } from '../types.js';
/** The app-less namespace is no app, so the wire says `null` for it. */
const publicAppUid = (appUid: string): string | null =>
appUid === KV_GLOBAL_APP_KEY ? null : appUid;
const pageOptions = (req: Request): KvShareHandleListOptions => {
const query = (req.query ?? {}) as Record<string, unknown>;
return {
limit: normalizeLimit(query.limit, { cap: KV_HANDLE_LIST_LIMIT_CAP }),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
};
};
/**
* Sharing a region of one account's key-value data with another account.
*
* Gated like the events surface: `subdomain: 'api'` plus `allowAccessToken`,
* with app-ness decided from `effectiveApp` in the service, which is where the
* owner, holder, delegation and namespace rules live. Only sharing asks for a
* confirmed email; listing, revoking and unsharing only narrow or report.
*/
@Controller('/kv')
export class KvShareController extends PuterController {
/**
* POST /kv/shares — share a region with another account, `read` unless the
* body asks otherwise. Sharing the same region with the same account again
* changes its level or label in place.
*/
@Post('/shares', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
requireVerified: true,
})
async share(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = bodyRecord(req);
const share = await this.services.kvShare.mintKvHandle(
actor,
{
prefix: body.prefix,
recipient: parseKvShareRecipient(body.recipient),
access: body.access,
appUid: body.appUid,
label: body.label,
},
{ defaultAccess: 'read' },
);
res.json({
handle: share.handle,
prefix: share.prefix,
access: share.access,
appUid: publicAppUid(share.appUid),
holder: share.holderUsername,
label: share.label,
createdAt: share.createdAt,
});
}
/** GET /kv/shares — what this account has shared out, retired included. */
@Get('/shares', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async listSharedByMe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const page = await this.services.kvShare.listKvHandles(
actor,
pageOptions(req),
);
res.json({
items: page.items.map((row) => ({
handle: row.handle,
prefix: row.prefix,
access: row.access,
appUid: publicAppUid(row.appUid),
holder: row.holderUsername,
label: row.label,
createdAt: row.createdAt,
revokedAt: row.revokedAt,
revokedBy: row.revokedBy,
})),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
/** GET /kv/shares/with-me — live shares this account holds. */
@Get('/shares/with-me', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async listSharedWithMe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const page = await this.services.kvShare.listKvSharesWithMe(
actor,
pageOptions(req),
);
res.json({
items: page.items.map((row) => ({
handle: row.handle,
access: row.access,
appUid: publicAppUid(row.appUid),
label: row.label,
createdAt: row.createdAt,
})),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
/**
* DELETE /kv/shares/:handle — the owner revokes, the holder gives it up,
* and anyone else is told it is not there.
*/
@Delete('/shares/:handle', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async unshare(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.kvShare.removeKvShare(actor, req.params.handle),
);
}
#requireActor(req: Request): Actor {
const actor = req.actor;
if (!actor?.user)
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
return actor;
}
}
+26
View File
@@ -23,6 +23,7 @@ import {
makeActor,
SYSTEM_ACTOR,
userRelatedActor,
viaWorkerOf,
type Actor,
} from './actor';
@@ -111,3 +112,28 @@ describe('assertResolvedActor', () => {
);
});
});
describe('viaWorkerOf', () => {
const user = { uuid: 'u-1', id: 1 };
it('reads a worker session or an events handler as server-side', () => {
expect(
viaWorkerOf(
makeActor({ user, session: { uid: 's-1', kind: 'worker' } }),
),
).toBe(true);
expect(viaWorkerOf(makeActor({ user, handlerDepth: 1 }))).toBe(true);
});
it('reads a browser session, an app or no actor at all as not', () => {
expect(
viaWorkerOf(
makeActor({ user, session: { uid: 's-1', kind: 'web' } }),
),
).toBe(false);
expect(viaWorkerOf(makeActor({ user, app: { uid: 'app-1' } }))).toBe(
false,
);
expect(viaWorkerOf(undefined)).toBe(false);
});
});
+7
View File
@@ -239,3 +239,10 @@ export const userRelatedActor = (actor: Actor): Actor => {
/** How many events handler runs led to a write by this actor. */
export const handlerDepthOf = (actor: Actor | undefined | null): number =>
actor?.handlerDepth ?? 0;
/**
* Whether a write by this actor was made by server-side code running as the
* account: a worker session, or an events handler.
*/
export const viaWorkerOf = (actor: Actor | undefined | null): boolean =>
actor?.session?.kind === 'worker' || handlerDepthOf(actor) > 0;
@@ -128,6 +128,18 @@ describe('createWwwRedirect', () => {
expect(next).toHaveBeenCalledTimes(1);
});
it('leaves www. alone on a host outside our domains', () => {
const req = makeReq({
subdomains: ['com', 'example', 'www'],
host: 'www.example.com',
});
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
true;
const { out, next } = run(createWwwRedirect(config), req);
expect(out.redirectArgs).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
});
it("passes through when config.domain isn't configured (no target to redirect to)", () => {
const { out, next } = run(
createWwwRedirect({} as IConfig),
@@ -879,6 +891,18 @@ describe('createNativeAppStatic', () => {
const withRoot = () =>
createNativeAppStatic({ native_apps_root: root } as unknown as IConfig);
it('leaves a native app name alone on a host outside our domains', async () => {
const req = staticReq({
subdomains: ['com', 'example', 'editor'],
path: '/index.html',
});
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
true;
const { out, next } = await runStatic(withRoot(), req);
expect(out.sent).toBeUndefined();
expect(next).toHaveBeenCalledTimes(1);
});
it('is a no-op when native_apps_root is unset', async () => {
const { out, next } = await runStatic(
createNativeAppStatic(config),
@@ -207,6 +207,8 @@ const dispatchClaimed = async (
export const createWwwRedirect = (config: IConfig): RequestHandler => {
const domain = (config.domain ?? '').toLowerCase();
return (req, res, next) => {
// `www.` on a host outside our domains belongs to that host's site.
if (req.is_custom_domain) return next();
const active = req.subdomains?.[req.subdomains.length - 1] ?? '';
if (active !== 'www') return next();
if (!domain) return next();
@@ -305,6 +307,7 @@ export const createNativeAppStatic = (config: IConfig): RequestHandler => {
return (_req, _res, next) => next();
}
return async (req, res, next) => {
if (req.is_custom_domain) return next();
const active = (
req.subdomains?.[req.subdomains.length - 1] ?? ''
).toLowerCase();
@@ -950,7 +950,8 @@ describe('resolvePrivateIdentity', () => {
expect(out.userUid).toBeUndefined();
});
it('accepts an app-issued access token minted for the target app', async () => {
it('ignores an app-issued access token even for the target app', async () => {
// A scoped token (e.g. a read URL) is not the account, whoever issued it.
const user = await makeUser();
const targetUid = `app-target-${uuidv4()}`;
const issuer = makeActor({
@@ -970,11 +971,11 @@ describe('resolvePrivateIdentity', () => {
sessionCookieName: 'puter_auth_token',
expectedAppUid: targetUid,
});
expect(out.source).toBe('session-cookie');
expect(out.userUid).toBe(user.uuid);
expect(out.source).toBe('none');
expect(out.userUid).toBeUndefined();
});
it('still accepts a user-issued access token, which is portable', async () => {
it('ignores a user-issued scoped access token', async () => {
const user = await makeUser();
const issuer = makeActor({ user: { uuid: user.uuid } });
const out = await resolvePrivateIdentity({
@@ -983,7 +984,46 @@ describe('resolvePrivateIdentity', () => {
actor: makeActor({
user: { uuid: user.uuid },
accessToken: { uid: `tok-${uuidv4()}`, issuer },
session: { uid: 'sess-1' },
}),
}),
authService,
sessionCookieName: 'puter_auth_token',
expectedAppUid: `app-target-${uuidv4()}`,
});
expect(out.source).toBe('none');
expect(out.userUid).toBeUndefined();
});
it('ignores a scoped access token when the host has no app', async () => {
const user = await makeUser();
const issuer = makeActor({ user: { uuid: user.uuid } });
const out = await resolvePrivateIdentity({
req: reqOf({
cookies: {},
actor: makeActor({
user: { uuid: user.uuid },
accessToken: { uid: `tok-${uuidv4()}`, issuer },
}),
}),
authService,
sessionCookieName: 'puter_auth_token',
});
expect(out.source).toBe('none');
});
it('accepts a full-access token, which holds the account', async () => {
const user = await makeUser();
const issuer = makeActor({ user: { uuid: user.uuid } });
const out = await resolvePrivateIdentity({
req: reqOf({
cookies: {},
actor: makeActor({
user: { uuid: user.uuid },
accessToken: {
uid: `tok-${uuidv4()}`,
issuer,
fullAccess: true,
},
}),
}),
authService,
@@ -25,7 +25,7 @@ import {
} from '../../../services/auth/AuthService';
import type { AppStore } from '../../../stores/app/AppStore.js';
import type { IConfig } from '../../../types';
import type { Actor } from '../../actor';
import { isAccessTokenActor, isAccountContext, type Actor } from '../../actor';
/**
* Support helpers for the private-app access gate — ported from v1's
@@ -196,10 +196,12 @@ export async function resolveOwnedAppForHostedSite(opts: {
appStore: Pick<AppStore, 'listOwnedByIndexUrlCandidates'>;
config: PrivateHostingConfig;
requirePrivate?: boolean;
/** Match against this hosting-domain host instead of the request's. */
host?: string;
}): Promise<AppLike | null> {
if (!opts.site?.user_id) return null;
const host = normalizeHost(opts.req.hostname);
const host = normalizeHost(opts.host ?? opts.req.hostname);
if (!host) return null;
const hostedSubdomain = subdomainFromHost(host, [
@@ -215,7 +217,7 @@ export async function resolveOwnedAppForHostedSite(opts: {
const hostCandidates = new Set<string>();
hostCandidates.add(host);
const headerHost =
typeof opts.req.headers?.host === 'string'
!opts.host && typeof opts.req.headers?.host === 'string'
? opts.req.headers.host.trim().toLowerCase()
: '';
if (headerHost) hostCandidates.add(headerHost);
@@ -458,11 +460,15 @@ export async function resolvePrivateIdentity(opts: {
* "No app anywhere in the chain" is the portable case, so read `effectiveApp` —
* an app-issued access token has no `app` of its own and would otherwise pass
* as a plain session on any private host.
*
* A scoped access token (a read URL, whoever issued it) is never the account
* here: the hosted cookie minted from it would outlive the token's revocation.
*/
function actorMatchesExpectedApp(
actor: Actor,
expectedAppUid: string | undefined,
): boolean {
if (isAccessTokenActor(actor) && !isAccountContext(actor)) return false;
if (!expectedAppUid) return true;
const app = actor.effectiveApp;
if (!app) return true;
@@ -1030,6 +1030,133 @@ describe('createPuterSiteMiddleware — site.access.check', () => {
});
});
describe('createPuterSiteMiddleware — hosts outside the configured domains', () => {
const claims = new Map<string, string>();
const resolveHost = (
_key: string,
data: { host: string; result: { subdomain: string | null } },
) => {
const sub = claims.get(data.host);
if (sub) data.result.subdomain = sub;
};
beforeAll(() => {
server.clients.event.on('site.resolveHost', resolveHost as never);
});
afterAll(() => {
server.clients.event.off('site.resolveHost', resolveHost as never);
});
const customReq = (hostname: string, path = '/index.html') => {
const req = makeReq({ hostname, path });
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
true;
return req;
};
const makeSite = async (opts: { suspended?: boolean } = {}) => {
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
const sub = `cust-${Math.random().toString(36).slice(2, 8)}`;
await server.stores.subdomain.create({
userId: owner.id,
subdomain: sub,
rootDirId: homeEntry!.id,
});
await writeFile(
owner.id,
`${homePath}/index.html`,
Buffer.from('<html>custom</html>'),
'text/html',
);
if (opts.suspended) {
await server.stores.user.update(owner.id, { suspended: 1 });
}
const host = `${sub}.example.test`;
claims.set(host, sub);
return { owner, sub, host };
};
it('serves the claimed site on its host', async () => {
const { host } = await makeSite();
const { out, next } = await runMiddleware(
buildMiddleware(),
customReq(host),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(next).not.toHaveBeenCalled();
expect(out.statusCode).toBe(200);
expect(String(out.body)).toBe('<html>custom</html>');
});
it('404s an unclaimed host instead of falling through to the API/GUI', async () => {
const { out, next } = await runMiddleware(
buildMiddleware(),
customReq('nobody-claims-this.example.test'),
);
expect(next).not.toHaveBeenCalled();
expect(out.statusCode).toBe(404);
expect(out.body).toBe('Site not found');
});
it('404s when a listener throws', async () => {
const host = 'throws.example.test';
const boom = (_key: string, data: { host: string }) => {
if (data.host === host) throw new Error('boom');
};
server.clients.event.on('site.resolveHost', boom as never);
try {
const { out, next } = await runMiddleware(
buildMiddleware(),
customReq(host),
);
expect(next).not.toHaveBeenCalled();
expect(out.statusCode).toBe(404);
} finally {
server.clients.event.off('site.resolveHost', boom as never);
}
});
it('404s a claimed site whose owner is suspended', async () => {
const { host } = await makeSite({ suspended: true });
const { out } = await runMiddleware(buildMiddleware(), customReq(host));
expect(out.statusCode).toBe(404);
expect(out.body).toBe('Subdomain not found');
});
it('404s a private app rather than serving it ungated', async () => {
const { owner, sub, host } = await makeSite();
await server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`, `is_private`) VALUES (?, ?, ?, ?, ?, ?)',
[
`app-${uuidv4()}`,
sub,
sub,
`http://${sub}.site.puter.localhost/`,
owner.id,
1,
],
);
const { out } = await runMiddleware(buildMiddleware(), customReq(host));
expect(out.statusCode).toBe(404);
expect(out.body).toBe('Site not found');
expect(out.redirected).toBeUndefined();
expect(out.cookies).toEqual({});
});
it('passes the host through untouched when custom domains are not flagged', async () => {
const { host } = await makeSite();
const { out, next } = await runMiddleware(
buildMiddleware(),
makeReq({ hostname: host }),
);
expect(next).toHaveBeenCalledTimes(1);
expect(out.statusCode).toBeUndefined();
});
});
describe('createPuterSiteMiddleware — .puter_site_config', () => {
const setupSiteWithConfig = async (config: unknown) => {
const owner = await makeUserWithHome();
@@ -1254,12 +1381,7 @@ describe('createPuterSiteMiddleware — .puter_site_config', () => {
errors: { '404': { file: '/index.html', status: 200 } },
});
const body = Buffer.from('real-page');
await writeFile(
owner.id,
`${homePath}/page.html`,
body,
'text/html',
);
await writeFile(owner.id, `${homePath}/page.html`, body, 'text/html');
const mw = buildMiddleware();
const { res, out } = makeRes();
@@ -1373,9 +1495,9 @@ describe('createPuterSiteMiddleware — .puter_site_config', () => {
// SPA fallback still fires → cache served the deleted config.
expect(second.out.statusCode).toBe(200);
expect(
(second.out.body as Buffer | undefined)?.equals(body),
).toBe(true);
expect((second.out.body as Buffer | undefined)?.equals(body)).toBe(
true,
);
});
it('ignores `errors` entries with status codes outside 4xx/5xx', async () => {
+63 -20
View File
@@ -61,12 +61,12 @@ import {
* - Private-app gate via `app.privateAccess.check` — marketplace extension
* decides; default is denied + redirect to `app-center`
* - Range / ETag / Last-Modified passthrough via `fsEntry.readContent`
* - Hosts outside the configured domains (`custom_domains_enabled`) map to a site
* via `site.resolveHost`; unclaimed ones 404. Public sites only.
*
* Deferred (not yet implemented):
*
* - `.at` username-based sites (UUIDv5-keyed `/user/Public`).
* - Custom domains (subdomains table `domain` column) — requires host validation
* to allow arbitrary hostnames first.
*
* Site config:
*
@@ -228,29 +228,60 @@ export const createPuterSiteMiddleware = (
if (!host) return next();
const matched = matchHostingDomain(host);
if (!matched) return next();
// A host outside every configured domain. Listeners may map it to a
// site; an unclaimed one is answered here rather than falling through
// to the API/GUI routers.
const isCustomHost = !matched && Boolean(req.is_custom_domain);
if (!matched && !isCustomHost) return next();
// Bare hosting domain (e.g. `puter.site`) → redirect to the main site.
if (host === matched) {
if (domain) {
res.redirect(302, `${req.protocol}://${domain}`);
let subdomain: string;
if (isCustomHost) {
const resolveEvent = {
host,
result: { subdomain: null as string | null },
};
try {
await layers.clients.event.emitAndWait(
'site.resolveHost',
resolveEvent,
{},
);
} catch {
resolveEvent.result.subdomain = null;
}
const claimed = resolveEvent.result.subdomain;
if (typeof claimed !== 'string' || !claimed) {
res.status(404).type('text/plain').send('Site not found');
return;
}
res.status(404).type('text/plain').send('Subdomain not found');
return;
}
subdomain = claimed;
} else {
const hostingDomain = matched as string;
// Bare hosting domain (e.g. `puter.site`) → redirect to the main site.
if (host === hostingDomain) {
if (domain) {
res.redirect(302, `${req.protocol}://${domain}`);
return;
}
res.status(404).type('text/plain').send('Subdomain not found');
return;
}
// `host` is `<prefix>.<matched>`; subdomain is the left-most label.
const prefix = host.slice(0, host.length - matched.length - 1);
const subdomain = prefix.split('.')[0] || '';
// `host` is `<prefix>.<matched>`; subdomain is the left-most label.
const prefix = host.slice(
0,
host.length - hostingDomain.length - 1,
);
subdomain = prefix.split('.')[0] || '';
if (!subdomain || subdomain === 'www') {
if (domain) {
res.redirect(302, `${req.protocol}://${domain}`);
if (!subdomain || subdomain === 'www') {
if (domain) {
res.redirect(302, `${req.protocol}://${domain}`);
return;
}
res.status(404).type('text/plain').send('Subdomain not found');
return;
}
res.status(404).type('text/plain').send('Subdomain not found');
return;
}
const site = (await layers.stores.subdomain.getBySubdomain(
@@ -286,6 +317,11 @@ export const createPuterSiteMiddleware = (
site: { user_id: site.user_id },
appStore: layers.stores.app,
config: hostingCfg,
// App index URLs name the hosting-domain address, not the
// custom host the visitor used.
host: isCustomHost
? `${subdomain}.${hostingDomains[0]}`
: undefined,
}),
);
const rootLookup =
@@ -300,7 +336,14 @@ export const createPuterSiteMiddleware = (
const isPrivateApp = Boolean(associatedApp?.is_private);
const privateApp = isPrivateApp ? associatedApp : null;
if (isPrivateApp) {
if (isCustomHost) {
// The private-app gate and its cookies are bound to the private
// hosting domain, so these hosts serve public sites only.
if (isPrivateApp) {
res.status(404).type('text/plain').send('Site not found');
return;
}
} else if (isPrivateApp) {
// Private apps must run on the private hosting domain. If a
// visitor arrives via the public domain (puter.site), redirect
// them to the equivalent private-host URL so the cookie scope
@@ -448,7 +491,7 @@ export const createPuterSiteMiddleware = (
res.redirect(302, cleanPath);
return;
}
} else if (privateHostingDomains.has(matched)) {
} else if (privateHostingDomains.has(matched as string)) {
// Non-private content landed on the private hosting domain —
// mirror of the private redirect above. Covers two cases:
// - a paid app that just flipped to free (is_private 1→0)
@@ -103,6 +103,11 @@ beforeAll(async () => {
// block, so this one alone must not register a provider.
'gemini-video-generation': { apiKey: 'gem-key' },
kling: { apiKey: 'kling-key' },
// The chat block's base URL ends in `/v1`; video must drop it.
minimax: {
apiKey: 'mm-key',
apiBaseUrl: 'https://api.minimax.io/v1',
},
},
} as never);
driver = server.drivers.aiVideo as unknown as VideoGenerationDriver;
@@ -507,6 +512,54 @@ describe('VideoGenerationDriver.generate provider routing', () => {
]);
});
it.each([
'minimax-h3',
'minimax/minimax-h3',
'minimax:minimax/minimax-h3',
])('routes %s to the MiniMax provider from the shared minimax key', async (model) => {
const fetchSpy = vi.spyOn(globalThis, 'fetch');
fetchSpy.mockResolvedValueOnce(
new Response(JSON.stringify({ task_id: 'mm-task' })),
);
fetchSpy.mockResolvedValueOnce(
new Response(
JSON.stringify({
task: {
id: 'mm-task',
status: 'succeeded',
content: { url: 'https://minimax/out.mp4' },
usage: { output_seconds: 5 },
},
}),
),
);
const result = await withActor(() =>
driver.generate({ prompt: 'hi', model } as never),
);
expect(result).toBe('https://minimax/out.mp4');
expect(String(fetchSpy.mock.calls[0]![0])).toBe(
'https://api.minimax.io/v2/video_generation',
);
expect(togetherVideosCreateMock).not.toHaveBeenCalled();
});
it('lists the MiniMax models under the minimax-video-generation provider', async () => {
const minimax = (await driver.models()).filter(
(m) => m.provider === 'minimax-video-generation',
);
expect(minimax.map((m) => m.id).sort()).toEqual([
'minimax-h3',
'minimax-h3-highspeed',
'minimax-h3-lite',
'minimax-h3-max',
'minimax-hailuo-02',
'minimax-hailuo-2.3',
'minimax-hailuo-2.3-fast',
]);
});
it('routes a known togetherai:minimax/video-01-director id to the Together provider', async () => {
togetherVideosCreateMock.mockResolvedValueOnce({ id: 'tg-job' });
togetherVideosRetrieveMock.mockResolvedValueOnce({
@@ -948,6 +1001,49 @@ describe('VideoGenerationDriver.generate puter_output_path', () => {
// ── Size unification ───────────────────────────────────────────────
describe('VideoGenerationDriver.generate size unification', () => {
it.each(['912x512', '512x912'])(
'selects and bills Hailuo 02 at 512p for %s',
async (size) => {
const fetchSpy = vi.spyOn(globalThis, 'fetch');
const incrementUsageSpy = vi.spyOn(
server.services.metering,
'incrementUsage',
);
fetchSpy.mockResolvedValueOnce(
new Response(JSON.stringify({ task_id: 'mm-task' })),
);
fetchSpy.mockResolvedValueOnce(
new Response(JSON.stringify({
status: 'Success',
file_id: 'file-1',
})),
);
fetchSpy.mockResolvedValueOnce(
new Response(JSON.stringify({
file: { download_url: 'https://minimax/out.mp4' },
})),
);
await withActor(() => driver.generate({
prompt: 'a ball rolls',
model: 'minimax-hailuo-02',
size,
input_reference: 'https://example.com/first.png',
} as never));
const body = JSON.parse(
fetchSpy.mock.calls[0]![1]!.body as string,
);
expect(body.resolution).toBe('512P');
expect(incrementUsageSpy).toHaveBeenCalledWith(
SYSTEM_ACTOR,
'minimax-video-generation:minimax-hailuo-02:per-video:512p:6',
1,
10 * 1_000_000,
);
},
);
it('maps a WIDTHxHEIGHT size onto a tier plus aspect ratio for tier-based models', async () => {
completeTogetherJob();
@@ -981,6 +1077,77 @@ describe('VideoGenerationDriver.generate size unification', () => {
expect('ratio' in sent).toBe(false);
});
it('treats a catalog with a 2k or 4k tier as tier-based', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch');
fetchSpy.mockResolvedValueOnce(
new Response(
JSON.stringify({
code: 0,
data: { id: 'kl-task', status: 'submitted' },
}),
),
);
fetchSpy.mockResolvedValueOnce(
new Response(
JSON.stringify({
code: 0,
data: [
{
id: 'kl-task',
status: 'succeeded',
outputs: [
{
type: 'video',
url: 'https://kling/out.mp4',
duration: '5',
},
],
},
],
}),
),
);
await withActor(() =>
driver.generate({
prompt: 'hi',
model: 'kling-3.0',
size: '1080x1920',
} as never),
);
const sent = JSON.parse(
(fetchSpy.mock.calls[0]![1] as RequestInit).body as string,
);
expect(sent.settings.resolution).toBe('1080p');
expect(sent.settings.aspect_ratio).toBe('9:16');
fetchSpy.mockResolvedValueOnce(
new Response(JSON.stringify({ task_id: 'mm-task' })),
);
fetchSpy.mockResolvedValueOnce(
new Response(
JSON.stringify({
task: {
id: 'mm-task',
status: 'succeeded',
content: { url: 'https://minimax/out.mp4' },
},
}),
),
);
await withActor(() =>
driver.generate({
prompt: 'hi',
model: 'minimax-h3-lite',
size: '854x480',
} as never),
);
const sentToMiniMax = JSON.parse(
(fetchSpy.mock.calls[2]![1] as RequestInit).body as string,
);
expect(sentToMiniMax.resolution).toBe('480P');
expect(sentToMiniMax.ratio).toBe('16:9');
});
it('fills width/height from a WIDTHxHEIGHT size for pixel-sized models', async () => {
completeTogetherJob();
@@ -41,6 +41,7 @@ import { readProviderKey } from '../util/providerRegistry.js';
import { BytePlusVideoProvider } from './providers/byteplus/BytePlusVideoProvider.js';
import { GeminiVideoProvider } from './providers/gemini/GeminiVideoProvider.js';
import { KlingVideoProvider } from './providers/kling/KlingVideoProvider.js';
import { MiniMaxVideoProvider } from './providers/minimax/MiniMaxVideoProvider.js';
import { TogetherVideoProvider } from './providers/together/TogetherVideoProvider.js';
import type {
IGenerateVideoParams,
@@ -50,7 +51,9 @@ import type {
const DEFAULT_PROVIDER = 'together-video-generation';
const isResolutionTier = (value: string): boolean => /^\d{3,4}p$/i.test(value);
/** A resolution tier such as `720p`, `2k` or `4k`, as opposed to pixels. */
const isResolutionTier = (value: string): boolean =>
/^(\d{3,4}p|\dk)$/i.test(value);
const parsePixelSize = (
value: string | undefined,
@@ -63,14 +66,15 @@ const parsePixelSize = (
};
/** Resolution tier of the shorter side, e.g. 1920x1080 and 1080x1920 → 1080p. */
const tierForPixels = ({
width,
height,
}: {
width: number;
height: number;
}) => {
const tierForPixels = (
{ width, height }: { width: number; height: number },
dimensions: readonly string[],
) => {
const shortSide = Math.min(width, height);
const exactTier = dimensions.find(
(tier) => tier.toLowerCase() === `${shortSide}p`,
);
if (exactTier) return exactTier;
if (shortSide <= 480) return '480p';
if (shortSide <= 720) return '720p';
if (shortSide <= 1080) return '1080p';
@@ -81,7 +85,7 @@ const tierForPixels = ({
* Driver implementing the `puter-video-generation` interface.
*
* Manages multiple upstream providers (Together, BytePlus/Seedance, Gemini,
* Kling) and handles model resolution, provider routing, and parameter
* Kling, MiniMax) and handles model resolution, provider routing, and parameter
* normalisation. Each provider is a plain `IVideoProvider` -- the driver
* instantiates them from config on boot.
*
@@ -98,6 +102,7 @@ export class VideoGenerationDriver extends PuterDriver {
'byteplus-video-generation',
'gemini-video-generation',
'kling-video-generation',
'minimax-video-generation',
];
readonly isDefault = true;
@@ -293,7 +298,7 @@ export class VideoGenerationDriver extends PuterDriver {
const catalogIsTiers = model.dimensions.every(isResolutionTier);
const wanted =
catalogIsTiers && requestedPixels
? tierForPixels(requestedPixels)
? tierForPixels(requestedPixels, model.dimensions)
: requestedResolution;
// Case-insensitive so '4K' matches a catalog entry spelled '4k';
@@ -468,6 +473,26 @@ export class VideoGenerationDriver extends PuterDriver {
{ stores: this.stores, fs: this.services.fs },
);
}
// Falls back to the shared `minimax` (ai-chat) key. The chat block's
// `apiBaseUrl` names the chat API under `/v1`; the provider keeps its
// host, since the video API lives beside it.
const minimaxVideoCfg = providers['minimax-video-generation'] as
Record<string, unknown> | undefined;
const minimaxSharedCfg = providers['minimax'] as
Record<string, unknown> | undefined;
const minimaxKey = readProviderKey(minimaxVideoCfg, minimaxSharedCfg);
if (minimaxKey) {
this.#providers['minimax-video-generation'] =
new MiniMaxVideoProvider(
{
apiKey: minimaxKey,
apiBaseUrl: (minimaxVideoCfg?.apiBaseUrl ??
minimaxSharedCfg?.apiBaseUrl) as string | undefined,
},
m,
);
}
}
// -- Model map -----------------------------------------------------------
@@ -0,0 +1,66 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Integration test for the MiniMax video provider.
*
* Generates a 4s 480P clip on minimax-h3-lite ($0.10). Skipped when
* `PUTER_TEST_AI_MINIMAX_API_KEY` is unset.
*/
import { describe, expect, it } from 'vitest';
import {
makeMeteringStub,
optionalEnv,
skipUnlessEnv,
withTestActor,
} from '../../../integrationTestUtil.js';
import { MiniMaxVideoProvider } from './MiniMaxVideoProvider.js';
const ENV_VAR = 'PUTER_TEST_AI_MINIMAX_API_KEY';
// Task-based generation can take a couple of minutes.
const VIDEO_TIMEOUT_MS = 5 * 60 * 1000;
describe.skipIf(skipUnlessEnv(ENV_VAR))(
'MiniMaxVideoProvider (integration)',
() => {
it(
'generates a video URL from minimax-h3-lite',
{ timeout: VIDEO_TIMEOUT_MS },
async () => {
const provider = new MiniMaxVideoProvider(
{ apiKey: optionalEnv(ENV_VAR)! },
makeMeteringStub(),
);
const url = await withTestActor(() =>
provider.generate({
model: 'minimax-h3-lite',
prompt: 'a red ball rolls to the right',
resolution: '480p',
seconds: 4,
}),
);
expect(typeof url).toBe('string');
expect((url as string).startsWith('https://')).toBe(true);
},
);
},
);
@@ -0,0 +1,876 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Offline unit tests for MiniMaxVideoProvider.
*
* Boots a real PuterServer and constructs the provider against the live wired
* `MeteringService`. MiniMax's task APIs are called through global `fetch`,
* which is stubbed as the network boundary.
*/
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
type MockInstance,
} from 'vitest';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { PuterServer } from '../../../../server.js';
import { setupTestServer } from '../../../../testUtil.js';
import { withTestActor } from '../../../integrationTestUtil.js';
import { withAiCostFactor } from '../../../util/aiCostFactor.js';
import type { IGenerateVideoParams } from '../../types.js';
import { MiniMaxVideoProvider } from './MiniMaxVideoProvider.js';
import { MINIMAX_VIDEO_GENERATION_MODELS } from './models.js';
// -- Test harness ----------------------------------------------------
let server: PuterServer;
let fetchSpy: MockInstance<typeof fetch>;
let remainingUsageSpy: MockInstance<MeteringService['getRemainingUsage']>;
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const aiMetering = () =>
withAiCostFactor(
server.services.metering,
server.clients.event,
'ai-video',
);
const makeProvider = (config: { apiKey?: string; apiBaseUrl?: string } = {}) =>
new MiniMaxVideoProvider(
{
apiKey: config.apiKey ?? 'test-key',
...(config.apiBaseUrl ? { apiBaseUrl: config.apiBaseUrl } : {}),
pollIntervalMs: 1,
},
aiMetering(),
);
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
remainingUsageSpy = vi.spyOn(server.services.metering, 'getRemainingUsage');
remainingUsageSpy.mockResolvedValue(Number.MAX_SAFE_INTEGER);
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
incrementUsageSpy.mockResolvedValue({} as never);
});
afterEach(() => {
vi.restoreAllMocks();
});
const jsonResponse = (body: unknown, status = 200) =>
new Response(JSON.stringify(body), {
status,
headers: { 'content-type': 'application/json' },
});
const ok = { status_code: 0, status_msg: 'success' };
const h3Task = (overrides: Record<string, unknown> = {}) => ({
id: 'task-1',
model: 'MiniMax-H3',
status: 'succeeded',
content: { url: 'https://minimax.example/out.mp4' },
resolution: '768P',
duration: 5,
usage: { output_seconds: 5 },
...overrides,
});
/** Queue the v2 create response followed by task query responses. */
const mockH3Flow = (...polled: unknown[]) => {
fetchSpy.mockResolvedValueOnce(jsonResponse({ task_id: 'task-1' }));
for (const task of polled) {
fetchSpy.mockResolvedValueOnce(jsonResponse({ task }));
}
};
const hailuoTask = (status: string, fileId = 'file-1') => ({
task_id: 'task-1',
status,
file_id: status === 'Success' ? fileId : '',
base_resp: ok,
});
/** Queue the v1 create, query and file retrieve responses. */
const mockHailuoFlow = (...polled: unknown[]) => {
fetchSpy.mockResolvedValueOnce(
jsonResponse({ task_id: 'task-1', base_resp: ok }),
);
for (const task of polled) {
fetchSpy.mockResolvedValueOnce(jsonResponse(task));
}
fetchSpy.mockResolvedValueOnce(
jsonResponse({
file: {
file_id: 1,
download_url: 'https://minimax.example/hailuo.mp4',
},
base_resp: ok,
}),
);
};
const sentUrl = (callIndex = 0) => String(fetchSpy.mock.calls[callIndex]![0]);
const sentBody = (callIndex = 0): Record<string, any> =>
JSON.parse(
(fetchSpy.mock.calls[callIndex]![1] as RequestInit).body as string,
);
const generate = (params: Partial<IGenerateVideoParams>) =>
withTestActor(() =>
makeProvider().generate({ prompt: 'a ball rolls', ...params }),
);
const rate = (id: string, key: string) =>
MINIMAX_VIDEO_GENERATION_MODELS.find((m) => m.id === id)!.costs![key]!;
// -- Construction / catalog ------------------------------------------
describe('MiniMaxVideoProvider construction and catalog', () => {
it('throws when no apiKey is supplied', () => {
expect(
() => new MiniMaxVideoProvider({ apiKey: '' }, aiMetering()),
).toThrow(/API key/i);
});
it('defaults to MiniMax H3 and lists the seven models', () => {
const provider = makeProvider();
expect(provider.getDefaultModel()).toBe('minimax-h3');
expect(provider.models().map((m) => m.id)).toEqual([
'minimax-h3',
'minimax-h3-highspeed',
'minimax-h3-lite',
'minimax-h3-max',
'minimax-hailuo-2.3',
'minimax-hailuo-2.3-fast',
'minimax-hailuo-02',
]);
expect(fetchSpy).not.toHaveBeenCalled();
});
});
// -- H3 request shape ------------------------------------------------
describe('MiniMaxVideoProvider.generate H3 request shape', () => {
it('creates a v2 task with a ratio and polls it', async () => {
mockH3Flow(h3Task());
const result = await generate({
model: 'minimax-h3-lite',
seconds: 6,
size: '480p',
width: 1080,
height: 1920,
});
expect(result).toBe('https://minimax.example/out.mp4');
expect(sentUrl()).toBe('https://api.minimax.io/v2/video_generation');
const init = fetchSpy.mock.calls[0]![1] as RequestInit & {
headers: Record<string, string>;
};
expect(init.method).toBe('POST');
expect(init.headers.Authorization).toBe('Bearer test-key');
expect(sentBody()).toEqual({
model: 'MiniMax-H3-Lite',
content: [{ type: 'text', text: 'a ball rolls' }],
duration: 6,
resolution: '480P',
ratio: '9:16',
});
expect(sentUrl(1)).toBe(
'https://api.minimax.io/v2/query/video_generation/task-1',
);
});
it('sends the model name as MiniMax spells it and 2K upper-cased', async () => {
mockH3Flow(h3Task());
await generate({ model: 'MiniMax:minimax/minimax-h3', size: '2K' });
expect(sentBody()).toMatchObject({
model: 'MiniMax-H3',
resolution: '2K',
duration: 5,
});
});
it('defaults the ratio to 16:9 and ignores ratios MiniMax lacks', async () => {
mockH3Flow(h3Task());
await generate({ model: 'minimax-h3', width: 1000, height: 300 });
expect(sentBody().ratio).toBe('16:9');
});
it('sends frames with roles and no ratio', async () => {
mockH3Flow(h3Task());
await generate({
model: 'minimax-h3',
input_reference: 'data:image/png;base64,AAAA',
last_frame: 'https://example.com/last.png',
width: 1920,
height: 1080,
});
const body = sentBody();
expect(body.content).toEqual([
{ type: 'text', text: 'a ball rolls' },
{
type: 'image_url',
image_url: { url: 'data:image/png;base64,AAAA' },
role: 'first_frame',
},
{
type: 'image_url',
image_url: { url: 'https://example.com/last.png' },
role: 'last_frame',
},
]);
expect('ratio' in body).toBe(false);
});
it('takes a last frame on its own', async () => {
mockH3Flow(h3Task());
await generate({
model: 'minimax-h3',
last_frame: 'https://example.com/last.png',
});
expect(
sentBody().content.map((c: { role?: string }) => c.role),
).toEqual([undefined, 'last_frame']);
});
it('wraps raw base64 as a PNG data URI', async () => {
mockH3Flow(h3Task());
await generate({ model: 'minimax-h3', input_reference: 'BBBB' });
expect(sentBody().content[1].image_url.url).toBe(
'data:image/png;base64,BBBB',
);
});
it('sends references with a ratio', async () => {
mockH3Flow(h3Task());
await generate({
model: 'minimax-h3-max',
reference_images: ['https://example.com/a.png', 'CCCC'],
width: 4,
height: 3,
});
const body = sentBody();
expect(body.content).toEqual([
{ type: 'text', text: 'a ball rolls' },
{
type: 'image_url',
image_url: { url: 'https://example.com/a.png' },
role: 'reference_image',
},
{
type: 'image_url',
image_url: { url: 'data:image/png;base64,CCCC' },
role: 'reference_image',
},
]);
expect(body.ratio).toBe('4:3');
});
it("drops the chat API's /v1 from a base URL override", async () => {
mockH3Flow(h3Task());
await withTestActor(() =>
makeProvider({
apiBaseUrl: 'https://api.minimaxi.com/v1/',
}).generate({ prompt: 'hi' }),
);
expect(sentUrl()).toBe('https://api.minimaxi.com/v2/video_generation');
});
it('keeps polling while the task is queued or running', async () => {
mockH3Flow(
h3Task({ status: 'queued', content: undefined, usage: {} }),
h3Task({ status: 'running', content: undefined, usage: {} }),
h3Task(),
);
const result = await generate({});
expect(result).toBe('https://minimax.example/out.mp4');
expect(fetchSpy).toHaveBeenCalledTimes(4);
});
it('treats a rate-limited poll as a missed poll', async () => {
fetchSpy.mockResolvedValueOnce(jsonResponse({ task_id: 'task-1' }));
fetchSpy.mockResolvedValueOnce(
jsonResponse(
{
type: 'error',
error: {
type: 'rate_limit_error',
message: 'rate limit (1002)',
},
},
429,
),
);
fetchSpy.mockResolvedValueOnce(jsonResponse({ task: h3Task() }));
const result = await generate({});
expect(result).toBe('https://minimax.example/out.mp4');
expect(fetchSpy).toHaveBeenCalledTimes(3);
});
it('returns the sample clip in test mode without calling MiniMax', async () => {
const result = await generate({ test_mode: true });
expect(result).toBe('https://assets.puter.site/txt2vid.mp4');
expect(fetchSpy).not.toHaveBeenCalled();
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
});
// -- Hailuo request shape --------------------------------------------
describe('MiniMaxVideoProvider.generate Hailuo request shape', () => {
it('creates a v1 task, polls it and exchanges the file id for a URL', async () => {
mockHailuoFlow(hailuoTask('Preparing'), hailuoTask('Success'));
const result = await generate({
model: 'minimax-hailuo-2.3',
seconds: 10,
width: 1920,
height: 1080,
});
expect(result).toBe('https://minimax.example/hailuo.mp4');
expect(sentUrl()).toBe('https://api.minimax.io/v1/video_generation');
expect(sentBody()).toEqual({
model: 'MiniMax-Hailuo-2.3',
prompt: 'a ball rolls',
duration: 10,
resolution: '768P',
});
expect(sentUrl(1)).toBe(
'https://api.minimax.io/v1/query/video_generation?task_id=task-1',
);
expect(sentUrl(3)).toBe(
'https://api.minimax.io/v1/files/retrieve?file_id=file-1',
);
});
it('falls back to 6 seconds at 1080p, which takes no 10-second clip', async () => {
mockHailuoFlow(hailuoTask('Success'));
await generate({
model: 'minimax-hailuo-2.3-fast',
seconds: 10,
size: '1080p',
input_reference: 'https://example.com/first.png',
});
expect(sentBody()).toMatchObject({
model: 'MiniMax-Hailuo-2.3-Fast',
duration: 6,
resolution: '1080P',
first_frame_image: 'https://example.com/first.png',
});
});
it.each(['768p', '1080p'])('sends first and last frames at %s', async (size) => {
mockHailuoFlow(hailuoTask('Success'));
await generate({
model: 'minimax-hailuo-02',
size,
input_reference: 'DDDD',
last_frame: 'https://example.com/last.png',
});
expect(sentBody()).toEqual({
model: 'MiniMax-Hailuo-02',
prompt: 'a ball rolls',
duration: 6,
resolution: size.toUpperCase(),
first_frame_image: 'data:image/png;base64,DDDD',
last_frame_image: 'https://example.com/last.png',
});
});
});
// -- Validation ------------------------------------------------------
describe('MiniMaxVideoProvider.generate rejects invalid input before calling MiniMax', () => {
it.each<[string, Partial<IGenerateVideoParams>]>([
['a blank prompt', { prompt: ' ' }],
[
'a prompt over 7000 characters on H3',
{ model: 'minimax-h3', prompt: 'x'.repeat(7001) },
],
[
'a prompt over 2000 characters on Hailuo',
{ model: 'minimax-hailuo-2.3', prompt: 'x'.repeat(2001) },
],
[
'a source video',
{ model: 'minimax-h3', input_video: 'https://example.com/v.mp4' },
],
[
'reference images on Hailuo',
{
model: 'minimax-hailuo-02',
reference_images: ['https://example.com/r.png'],
},
],
[
'reference images with a first frame on H3',
{
model: 'minimax-h3',
input_reference: 'https://example.com/f.png',
reference_images: ['https://example.com/r.png'],
},
],
[
'more than 5 images on H3',
{
model: 'minimax-h3',
reference_images: Array(6).fill('https://example.com/r.png'),
},
],
[
'more than 2 images on H3 Max',
{
model: 'minimax-h3-max',
reference_images: Array(3).fill('https://example.com/r.png'),
},
],
[
'a last frame on Hailuo 2.3',
{
model: 'minimax-hailuo-2.3',
input_reference: 'https://example.com/f.png',
last_frame: 'https://example.com/l.png',
},
],
[
'a last frame without a first frame on Hailuo 02',
{
model: 'minimax-hailuo-02',
last_frame: 'https://example.com/l.png',
},
],
[
'512p without a first frame on Hailuo 02',
{ model: 'minimax-hailuo-02', size: '512p' },
],
[
'a missing first frame on Hailuo 2.3 Fast',
{ model: 'minimax-hailuo-2.3-fast' },
],
[
'512p with first and last frames on Hailuo 02',
{
model: 'minimax-hailuo-02',
size: '512p',
input_reference: 'https://example.com/first.png',
last_frame: 'https://example.com/last.png',
},
],
[
'a non-string image',
{ model: 'minimax-h3', input_reference: 42 as never },
],
[
'a non-array reference_images',
{ model: 'minimax-h3', reference_images: 'x' as never },
],
])('rejects %s', async (_label, params) => {
fetchSpy.mockRejectedValue(new Error('Unexpected MiniMax request'));
const reserveCreditsSpy = vi.spyOn(
server.services.metering,
'reserveCredits',
);
await expect(generate(params)).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
});
expect(fetchSpy).not.toHaveBeenCalled();
expect(reserveCreditsSpy).not.toHaveBeenCalled();
});
it('accepts a 7000-character prompt on H3', async () => {
mockH3Flow(h3Task());
await generate({ model: 'minimax-h3', prompt: 'x'.repeat(7000) });
expect(fetchSpy).toHaveBeenCalled();
});
it('accepts five images on H3', async () => {
mockH3Flow(h3Task());
await generate({
model: 'minimax-h3',
reference_images: Array(5).fill('https://example.com/r.png'),
});
expect(sentBody().content).toHaveLength(6);
});
});
// -- Billing ---------------------------------------------------------
describe('MiniMaxVideoProvider billing', () => {
it.each<[Partial<IGenerateVideoParams>, string, string, number]>([
[
{ model: 'minimax-h3', seconds: 5 },
'minimax-h3',
'per-second:768p',
5,
],
[
{ model: 'minimax-h3', seconds: 8, size: '2k' },
'minimax-h3',
'per-second:2k',
8,
],
[
{ model: 'minimax-h3-lite', seconds: 4, size: '480p' },
'minimax-h3-lite',
'per-second:480p',
4,
],
])(
'meters %o per second under its cost key',
async (params, id, key, secs) => {
mockH3Flow(
h3Task({ duration: secs, usage: { output_seconds: secs } }),
);
await generate(params);
expect(incrementUsageSpy).toHaveBeenCalledTimes(1);
const [, usageType, amount, cost] =
incrementUsageSpy.mock.calls[0]!;
expect(usageType).toBe(`minimax-video-generation:${id}:${key}`);
expect(amount).toBe(secs);
expect(cost).toBeCloseTo(secs * rate(id, key) * 1_000_000);
},
);
it('bills the seconds of output the task reports', async () => {
mockH3Flow(h3Task({ duration: 5, usage: { output_seconds: 4 } }));
await generate({ model: 'minimax-h3', seconds: 5 });
const [, , amount, cost] = incrementUsageSpy.mock.calls[0]!;
expect(amount).toBe(4);
expect(cost).toBeCloseTo(4 * 8 * 1_000_000);
});
it('bills the requested seconds when the task reports no usage', async () => {
mockH3Flow(h3Task({ duration: undefined, usage: undefined }));
await generate({ model: 'minimax-h3', seconds: 7 });
expect(incrementUsageSpy.mock.calls[0]![2]).toBe(7);
});
it('shortens an H3 clip to what the balance buys', async () => {
remainingUsageSpy.mockResolvedValue(6 * 8 * 1_000_000);
mockH3Flow(h3Task({ duration: 6, usage: { output_seconds: 6 } }));
await generate({ model: 'minimax-h3', seconds: 10 });
expect(sentBody().duration).toBe(6);
});
it('rejects with insufficient_funds before creating an H3 task it cannot afford', async () => {
remainingUsageSpy.mockResolvedValue(1000);
await expect(generate({ model: 'minimax-h3' })).rejects.toMatchObject({
statusCode: 402,
});
expect(fetchSpy).not.toHaveBeenCalled();
});
it.each<[Partial<IGenerateVideoParams>, string, string]>([
[
{ model: 'minimax-hailuo-2.3', seconds: 10 },
'minimax-hailuo-2.3',
'per-video:768p:10',
],
[
{ model: 'minimax-hailuo-02', size: '1080p' },
'minimax-hailuo-02',
'per-video:1080p:6',
],
[
{
model: 'minimax-hailuo-02',
size: '512p',
seconds: 10,
input_reference: 'https://example.com/f.png',
},
'minimax-hailuo-02',
'per-video:512p:10',
],
])('meters a Hailuo clip %o as one video', async (params, id, key) => {
mockHailuoFlow(hailuoTask('Success'));
await generate(params);
const [, usageType, amount, cost] = incrementUsageSpy.mock.calls[0]!;
expect(usageType).toBe(`minimax-video-generation:${id}:${key}`);
expect(amount).toBe(1);
expect(cost).toBe(rate(id, key) * 1_000_000);
});
it('rejects with insufficient_funds before creating a Hailuo task it cannot afford', async () => {
remainingUsageSpy.mockResolvedValue(1000);
await expect(
generate({ model: 'minimax-hailuo-2.3' }),
).rejects.toMatchObject({
statusCode: 402,
legacyCode: 'insufficient_funds',
});
expect(fetchSpy).not.toHaveBeenCalled();
});
});
// -- Failures --------------------------------------------------------
describe('MiniMaxVideoProvider failures', () => {
it.each(['failed', 'cancelled'])(
'maps a %s H3 task to upstream_failed and meters nothing',
async (status) => {
mockH3Flow(
h3Task({
status,
content: undefined,
error: { code: '1000', message: 'unknown error' },
}),
);
await expect(generate({})).rejects.toMatchObject({
statusCode: 502,
legacyCode: 'upstream_failed',
});
expect(incrementUsageSpy).not.toHaveBeenCalled();
},
);
it('maps an H3 task failed by the content filter to moderation_flagged', async () => {
mockH3Flow(
h3Task({
status: 'failed',
content: undefined,
error: {
code: '1026',
message: 'video description contains sensitive content',
},
}),
);
await expect(generate({})).rejects.toMatchObject({
statusCode: 400,
code: 'moderation_flagged',
});
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('carries a v2 parameter rejection with its status and code', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse(
{
type: 'error',
error: {
type: 'bad_request_error',
message:
'invalid params, model MiniMax-H3-Lite does not support resolution 2K (2013)',
http_code: '400',
},
},
400,
),
);
await expect(generate({})).rejects.toMatchObject({
status: 400,
code: '2013',
provider: 'minimax',
});
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('reports a model MiniMax no longer serves as upstream_failed', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse(
{
type: 'error',
error: {
type: 'bad_request_error',
message:
'invalid params, this model is not supported by /v2/video_generation (2013)',
},
},
400,
),
);
await expect(generate({})).rejects.toMatchObject({
statusCode: 502,
legacyCode: 'upstream_failed',
});
});
it('hides the auth message behind a 401', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse(
{
type: 'error',
error: {
type: 'authorized_error',
message:
'login fail: carry the key in Authorization (1004)',
},
},
401,
),
);
await expect(generate({})).rejects.toMatchObject({
status: 401,
message: 'MiniMax rejected the API key',
});
});
it('surfaces an empty balance as status 402 with the MiniMax code', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse(
{
type: 'error',
error: {
type: 'insufficient_balance_error',
message: 'insufficient balance (1008)',
},
},
402,
),
);
await expect(generate({})).rejects.toMatchObject({
status: 402,
code: '1008',
provider: 'minimax',
});
});
it('maps a content block at submit to moderation_flagged', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse(
{
type: 'error',
error: {
type: 'unprocessable_entity_error',
message: 'input new_sensitive (1026)',
},
},
422,
),
);
await expect(generate({})).rejects.toMatchObject({
statusCode: 400,
code: 'moderation_flagged',
});
});
it('reads a v1 verdict out of base_resp under HTTP 200', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse({
task_id: '',
base_resp: {
status_code: 1008,
status_msg: 'insufficient balance',
},
}),
);
await expect(
generate({ model: 'minimax-hailuo-2.3' }),
).rejects.toMatchObject({
status: 402,
code: '1008',
provider: 'minimax',
});
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('reports a Hailuo model MiniMax no longer serves as upstream_failed', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse({
task_id: '',
base_resp: {
status_code: 2013,
status_msg: 'invalid params, incorrect model param input',
},
}),
);
await expect(
generate({ model: 'minimax-hailuo-2.3' }),
).rejects.toMatchObject({
statusCode: 502,
legacyCode: 'upstream_failed',
});
});
it('maps a failed Hailuo task to upstream_failed and meters nothing', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse({ task_id: 'task-1', base_resp: ok }),
);
fetchSpy.mockResolvedValueOnce(jsonResponse(hailuoTask('Fail')));
await expect(
generate({ model: 'minimax-hailuo-2.3' }),
).rejects.toMatchObject({
statusCode: 502,
legacyCode: 'upstream_failed',
});
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('maps a Hailuo clip removed by the output filter to moderation_flagged', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse({ task_id: 'task-1', base_resp: ok }),
);
fetchSpy.mockResolvedValueOnce(
jsonResponse({
task_id: 'task-1',
status: 'Fail',
base_resp: {
status_code: 1027,
status_msg: 'output new_sensitive',
},
}),
);
await expect(
generate({ model: 'minimax-hailuo-2.3' }),
).rejects.toMatchObject({
statusCode: 400,
code: 'moderation_flagged',
});
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('fails when the finished H3 task has no video', async () => {
mockH3Flow(h3Task({ content: {} }));
await expect(generate({})).rejects.toThrow(/video URL/);
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
it('fails when the Hailuo file has no download URL', async () => {
fetchSpy.mockResolvedValueOnce(
jsonResponse({ task_id: 'task-1', base_resp: ok }),
);
fetchSpy.mockResolvedValueOnce(jsonResponse(hailuoTask('Success')));
fetchSpy.mockResolvedValueOnce(
jsonResponse({ file: { file_id: 1 }, base_resp: ok }),
);
await expect(generate({ model: 'minimax-hailuo-2.3' })).rejects.toThrow(
/video URL/,
);
expect(incrementUsageSpy).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,690 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Actor } from '../../../../core/actor.js';
import { Context } from '../../../../core/context.js';
import { HttpError } from '../../../../core/http/HttpError.js';
import { insufficientCreditsError } from '../../../../services/metering/enforcement.js';
import type { CreditHold } from '../../../../services/metering/types.js';
import type { AiMeteringService } from '../../../util/aiCostFactor.js';
import { readBoundedBody } from '../../../util/fileInput.js';
import { toUrlOrDataUri } from '../../../util/imageInput.js';
import { sanitizeUpstreamMessage } from '../../../util/upstreamErrors.js';
import { capSecondsToRemainingCredits } from '../../creditCap.js';
import type { IGenerateVideoParams, IVideoModel } from '../../types.js';
import { VideoProvider } from '../VideoProvider.js';
import {
pollUntilSettled,
VIDEO_HOLD_TTL_MS,
videoJobFailure,
} from '../polling.js';
import {
MINIMAX_VIDEO_GENERATION_MODELS,
MINIMAX_VIDEO_SPECS,
type MiniMaxVideoSpec,
} from './models.js';
const DEFAULT_TEST_VIDEO_URL = 'https://assets.puter.site/txt2vid.mp4';
const DEFAULT_BASE_URL = 'https://api.minimax.io';
const DEFAULT_MODEL = 'minimax-h3';
const DEFAULT_POLL_INTERVAL_MS = 5_000;
const REQUEST_TIMEOUT_MS = 60_000;
const MAX_RESPONSE_BYTES = 1024 * 1024;
const PROVIDER = 'minimax';
const ASPECT_RATIOS = ['21:9', '16:9', '4:3', '1:1', '3:4', '9:16'];
const DEFAULT_ASPECT_RATIO = '16:9';
// MiniMax's business codes, from the trailing "(NNNN)" of a v2 error message
// or `base_resp.status_code` of a v1 reply.
// https://platform.minimax.io/docs/api-reference/errorcode
const AUTH_CODES = new Set([1004, 2049]);
const BALANCE_CODES = new Set([1008, 2053, 2056, 2067]);
const RATE_LIMIT_CODES = new Set([1002, 1039, 1041, 1045]);
const MODERATION_CODES = new Set([1026, 1027]);
const MODEL_UNAVAILABLE_CODES = new Set([2022, 2061, 2068, 2069]);
const INVALID_PARAM_CODES = new Set([2013, 2014]);
/** A 2013 about the model itself, which means the catalog has drifted. */
const UNSUPPORTED_MODEL_PATTERN =
/this model is not supported|incorrect model param/i;
type MiniMaxVideoConfig = {
apiKey: string;
apiBaseUrl?: string;
/** Test hook: polling cadence for task status checks. */
pollIntervalMs?: number;
};
interface MiniMaxEnvelope {
base_resp?: { status_code?: number; status_msg?: string };
error?: { type?: string; message?: string; http_code?: string };
}
interface H3Task {
id: string;
status: 'queued' | 'running' | 'succeeded' | 'failed' | 'cancelled';
content?: { url?: string };
error?: { code?: string; message?: string };
duration?: number;
usage?: { output_seconds?: number };
}
interface HailuoTask {
task_id?: string;
status?: 'Preparing' | 'Queueing' | 'Processing' | 'Success' | 'Fail';
file_id?: string;
}
const badRequest = (message: string) =>
new HttpError(400, message, { legacyCode: 'bad_request' });
const present = (value: unknown): boolean =>
value !== undefined && value !== null;
const statusForCode = (code: number, httpStatus: number): number => {
if (AUTH_CODES.has(code)) return 401;
if (BALANCE_CODES.has(code)) return 402;
if (RATE_LIMIT_CODES.has(code)) return 429;
if (INVALID_PARAM_CODES.has(code)) return 400;
return httpStatus >= 400 ? httpStatus : 502;
};
/** The code a v2 error message ends with, e.g. "invalid params (2013)". */
const parseCode = (message: string): number | undefined => {
const match = message.match(/\((\d{4,5})\)\s*$/);
return match ? Number.parseInt(match[1], 10) : undefined;
};
/** A request MiniMax refused, shaped for `translateProviderError`. */
const minimaxError = (
code: number | undefined,
message: string,
status: number,
): Error => {
// A model MiniMax doesn't serve is Puter's pick, so the catalog has
// drifted from MiniMax's: our fault, not the caller's, and worth
// someone's attention.
if (
code !== undefined &&
(MODEL_UNAVAILABLE_CODES.has(code) ||
(code === 2013 && UNSUPPORTED_MODEL_PATTERN.test(message)))
) {
return new HttpError(
502,
'MiniMax does not support the model Puter requested',
{
legacyCode: 'upstream_failed',
alarmSeverity: 'warning',
fields: {
provider: PROVIDER,
upstreamCode: String(code),
upstreamMessage: sanitizeUpstreamMessage(message),
},
},
);
}
if (code !== undefined && MODERATION_CODES.has(code)) {
return new HttpError(
400,
sanitizeUpstreamMessage(message) ||
'MiniMax rejected the request under its content policy',
{
legacyCode: 'bad_request',
code: 'moderation_flagged',
fields: { provider: PROVIDER, upstreamCode: String(code) },
},
);
}
// MiniMax's auth messages point at its own console; the caller can't act
// on them.
const isAuthFailure = code !== undefined && AUTH_CODES.has(code);
return Object.assign(
new Error(isAuthFailure ? 'MiniMax rejected the API key' : message),
{
status:
code === undefined
? status >= 400
? status
: 502
: statusForCode(code, status),
code: code === undefined ? undefined : String(code),
provider: PROVIDER,
},
);
};
/**
* MiniMax video generation (H3 and Hailuo).
*
* H3 models go through the v2 task API: a `content` list of text, frames and
* references creates a task that is polled until it leaves queued/running, and
* the clip's URL comes back on the task. Billing is per second of output at a
* rate set by resolution. Hailuo models go through the v1 API, which takes a
* prompt and frame images, is polled on its own status endpoint and hands back
* a file id to exchange for a download URL. They are billed per clip, by
* resolution and length.
* https://platform.minimax.io/docs/api-reference/video-generation-v2-create
*/
export class MiniMaxVideoProvider extends VideoProvider {
#apiKey: string;
#baseUrl: string;
#pollIntervalMs: number;
#meteringService: AiMeteringService;
constructor(
config: MiniMaxVideoConfig,
meteringService: AiMeteringService,
) {
super();
if (!config.apiKey) {
throw new Error('MiniMax video generation requires an API key');
}
this.#apiKey = config.apiKey;
// The shared chat block's base URL ends in the chat API's `/v1`; the
// video API lives beside it on the same host.
this.#baseUrl = (config.apiBaseUrl ?? DEFAULT_BASE_URL)
.replace(/\/+$/, '')
.replace(/\/v1$/, '');
this.#pollIntervalMs =
config.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS;
this.#meteringService = meteringService;
}
getDefaultModel(): string {
return DEFAULT_MODEL;
}
models(): IVideoModel[] {
return MINIMAX_VIDEO_GENERATION_MODELS;
}
async generate(params: IGenerateVideoParams): Promise<unknown> {
const {
prompt,
model: requestedModel,
seconds,
duration,
size,
resolution,
width,
height,
input_reference: inputReference,
last_frame: lastFrame,
reference_images: referenceImages,
input_video: inputVideo,
test_mode: testMode,
} = params ?? {};
if (typeof prompt !== 'string' || !prompt.trim()) {
throw badRequest('prompt must be a non-empty string');
}
const model = this.#getModel(requestedModel);
const spec = MINIMAX_VIDEO_SPECS[model.id];
if (prompt.length > spec.maxPromptChars) {
throw badRequest(
`${model.id} takes a prompt of at most ${spec.maxPromptChars} characters`,
);
}
if (present(inputVideo)) {
throw badRequest(`${model.id} does not accept a source video`);
}
const resolutionKey = this.#resolution(size ?? resolution, model);
const firstFrame = this.#image(inputReference, 'input_reference');
const endFrame = this.#image(lastFrame, 'last_frame');
const references = this.#references(referenceImages);
this.#checkInputs(spec, model, resolutionKey, {
firstFrame,
endFrame,
references,
});
if (testMode) {
return DEFAULT_TEST_VIDEO_URL;
}
const actor = Context.get('actor') as Actor | undefined;
if (!actor) {
throw new HttpError(401, 'Authentication required', {
legacyCode: 'unauthorized',
});
}
const requestedSeconds =
this.#coerceSeconds(seconds ?? duration) ??
model.durationSeconds![0];
const aspectRatio = this.#aspectRatio(width, height);
const images = { firstFrame, endFrame, references };
if (spec.api === 'v2') {
return this.#generateH3(actor, model, spec, {
prompt,
resolutionKey,
requestedSeconds,
aspectRatio,
...images,
});
}
return this.#generateHailuo(actor, model, spec, {
prompt,
resolutionKey,
requestedSeconds,
...images,
});
}
// -- H3 (v2 task API, per second) ---------------------------------------
async #generateH3(
actor: Actor,
model: IVideoModel,
spec: MiniMaxVideoSpec,
input: {
prompt: string;
resolutionKey: string;
requestedSeconds: number;
aspectRatio: string | undefined;
firstFrame?: string;
endFrame?: string;
references: string[];
},
): Promise<string> {
const costKey = `per-second:${input.resolutionKey}`;
const centsPerSecond = model.costs?.[costKey];
if (!centsPerSecond) {
throw new Error(
`No pricing configured for video model ${model.id} at ${costKey}`,
);
}
const usageType = `minimax-video-generation:${model.id}:${costKey}`;
const { seconds: cappedSeconds, hold } =
await capSecondsToRemainingCredits({
metering: this.#meteringService,
actor,
usageType,
perSecondMicroCents: centsPerSecond * 1_000_000,
requestedSeconds: input.requestedSeconds,
allowedSeconds: model.durationSeconds,
});
try {
const content: Array<Record<string, unknown>> = [
{ type: 'text', text: input.prompt },
];
if (input.firstFrame) {
content.push({
type: 'image_url',
image_url: { url: input.firstFrame },
role: 'first_frame',
});
}
if (input.endFrame) {
content.push({
type: 'image_url',
image_url: { url: input.endFrame },
role: 'last_frame',
});
}
for (const url of input.references) {
content.push({
type: 'image_url',
image_url: { url },
role: 'reference_image',
});
}
const body: Record<string, unknown> = {
model: spec.apiModel,
content,
duration: cappedSeconds,
resolution: input.resolutionKey.toUpperCase(),
};
// A clip from frames takes their shape; MiniMax requires a ratio
// for everything else.
if (!input.firstFrame && !input.endFrame) {
body.ratio = input.aspectRatio ?? DEFAULT_ASPECT_RATIO;
}
const created = await this.#request<{ task_id?: string }>(
'POST',
'/v2/video_generation',
body,
);
if (!created.task_id) {
throw new Error('MiniMax response did not include a task id');
}
const task = await pollUntilSettled<H3Task>({
provider: PROVIDER,
providerLabel: 'MiniMax',
intervalMs: this.#pollIntervalMs,
fetch: async () => {
const { task } = await this.#request<{ task?: H3Task }>(
'GET',
`/v2/query/video_generation/${encodeURIComponent(created.task_id!)}`,
);
if (!task) {
throw new Error('MiniMax task query returned no task');
}
return task;
},
isPending: (task) =>
task.status === 'queued' || task.status === 'running',
});
if (task.status !== 'succeeded') {
throw videoJobFailure(
PROVIDER,
task.error?.message || `Video generation ${task.status}`,
task.error?.code,
);
}
const url = task.content?.url;
if (typeof url !== 'string' || !url.trim()) {
throw new Error('MiniMax response did not include a video URL');
}
// MiniMax bills the seconds of output the task reports.
const outputSeconds = task.usage?.output_seconds ?? task.duration;
const billedSeconds =
typeof outputSeconds === 'number' &&
Number.isFinite(outputSeconds) &&
outputSeconds > 0
? Math.max(1, Math.round(outputSeconds))
: cappedSeconds;
await this.#meteringService.incrementUsage(
actor,
usageType,
billedSeconds,
Math.round(billedSeconds * centsPerSecond * 1_000_000),
);
return url;
} finally {
await hold.release();
}
}
// -- Hailuo (v1 API, per clip) ------------------------------------------
async #generateHailuo(
actor: Actor,
model: IVideoModel,
spec: MiniMaxVideoSpec,
input: {
prompt: string;
resolutionKey: string;
requestedSeconds: number;
firstFrame?: string;
endFrame?: string;
},
): Promise<string> {
// A clip length the resolution doesn't offer falls back to the first
// one it does, the way the driver treats any unsupported duration.
const offered = (secs: number) =>
model.costs?.[`per-video:${input.resolutionKey}:${secs}`];
const clipSeconds = offered(input.requestedSeconds)
? input.requestedSeconds
: model.durationSeconds!.find(offered)!;
const costKey = `per-video:${input.resolutionKey}:${clipSeconds}`;
const centsPerVideo = model.costs![costKey]!;
const usageType = `minimax-video-generation:${model.id}:${costKey}`;
const microCents = centsPerVideo * 1_000_000;
const hold: CreditHold | null =
await this.#meteringService.reserveAiCredits(
actor,
usageType,
microCents,
{ ttlMs: VIDEO_HOLD_TTL_MS },
);
if (!hold) {
throw insufficientCreditsError();
}
try {
const body: Record<string, unknown> = {
model: spec.apiModel,
prompt: input.prompt,
duration: clipSeconds,
resolution: input.resolutionKey.toUpperCase(),
};
if (input.firstFrame) body.first_frame_image = input.firstFrame;
if (input.endFrame) body.last_frame_image = input.endFrame;
const created = await this.#request<{ task_id?: string }>(
'POST',
'/v1/video_generation',
body,
);
if (!created.task_id) {
throw new Error('MiniMax response did not include a task id');
}
const task = await pollUntilSettled<HailuoTask>({
provider: PROVIDER,
providerLabel: 'MiniMax',
intervalMs: this.#pollIntervalMs,
fetch: () =>
this.#request<HailuoTask>(
'GET',
`/v1/query/video_generation?task_id=${encodeURIComponent(created.task_id!)}`,
),
isPending: (task) =>
task.status === 'Preparing' ||
task.status === 'Queueing' ||
task.status === 'Processing',
});
if (task.status !== 'Success' || !task.file_id) {
throw videoJobFailure(
PROVIDER,
'MiniMax video generation failed',
);
}
// The clip is a file in MiniMax's store; its download link is
// minted on request and works for an hour.
const { file } = await this.#request<{
file?: { download_url?: string };
}>(
'GET',
`/v1/files/retrieve?file_id=${encodeURIComponent(task.file_id)}`,
);
const url = file?.download_url;
if (typeof url !== 'string' || !url.trim()) {
throw new Error('MiniMax response did not include a video URL');
}
await this.#meteringService.incrementUsage(
actor,
usageType,
1,
microCents,
);
return url;
} finally {
await hold.release();
}
}
// -- Validation ---------------------------------------------------------
#checkInputs(
spec: MiniMaxVideoSpec,
model: IVideoModel,
resolutionKey: string,
inputs: {
firstFrame?: string;
endFrame?: string;
references: string[];
},
) {
const { firstFrame, endFrame, references } = inputs;
if (spec.firstFrameRequired && !firstFrame) {
throw badRequest(`${model.id} requires an input_reference image`);
}
if (endFrame && !spec.lastFrame) {
throw badRequest(`${model.id} does not support last_frame`);
}
if (endFrame && !firstFrame && spec.api === 'v1') {
throw badRequest(
'last_frame requires a first-frame image via input_reference',
);
}
if (references.length > 0) {
if (!spec.references) {
throw badRequest(
`${model.id} does not support reference_images`,
);
}
if (firstFrame || endFrame) {
throw badRequest(
'reference_images cannot be combined with input_reference or last_frame',
);
}
}
const imageCount =
(firstFrame ? 1 : 0) + (endFrame ? 1 : 0) + references.length;
if (imageCount > spec.maxImages) {
throw badRequest(
`${model.id} takes at most ${spec.maxImages} image(s), frames and references together`,
);
}
if (
spec.imageOnlyResolutions?.includes(resolutionKey) &&
(!firstFrame || endFrame)
) {
throw badRequest(
`${model.id} renders ${resolutionKey} only from an input_reference image without last_frame`,
);
}
}
async #request<T>(
method: 'GET' | 'POST',
path: string,
body?: Record<string, unknown>,
): Promise<T> {
const response = await fetch(`${this.#baseUrl}${path}`, {
method,
headers: {
Authorization: `Bearer ${this.#apiKey}`,
...(body ? { 'Content-Type': 'application/json' } : {}),
},
...(body ? { body: JSON.stringify(body) } : {}),
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
});
const raw = await readBoundedBody(response, MAX_RESPONSE_BYTES, () =>
minimaxError(undefined, 'MiniMax response too large', 502),
);
let payload: MiniMaxEnvelope = {};
try {
const parsed: unknown = JSON.parse(raw.toString('utf8'));
if (parsed && typeof parsed === 'object') {
payload = parsed as MiniMaxEnvelope;
}
} catch {
// Not JSON; the status alone decides.
}
// A v1 reply carries its verdict in `base_resp` under HTTP 200; a v2
// reply uses the HTTP status and an OpenAI-style `error`.
const v1Code = payload.base_resp?.status_code;
if (typeof v1Code === 'number' && v1Code !== 0) {
throw minimaxError(
v1Code,
payload.base_resp?.status_msg ||
`MiniMax request failed (code ${v1Code})`,
response.status,
);
}
if (!response.ok) {
const message =
payload.error?.message ||
`MiniMax request failed (status ${response.status})`;
throw minimaxError(parseCode(message), message, response.status);
}
return payload as unknown as T;
}
#getModel(requestedModel?: string): IVideoModel {
const wanted = (requestedModel ?? '').trim().toLowerCase();
return (
MINIMAX_VIDEO_GENERATION_MODELS.find(
(m) =>
m.id === wanted ||
m.puterId === wanted ||
m.aliases?.includes(wanted),
) ??
MINIMAX_VIDEO_GENERATION_MODELS.find((m) => m.id === DEFAULT_MODEL)!
);
}
#resolution(candidate: unknown, model: IVideoModel): string {
const tiers = model.dimensions!;
if (typeof candidate === 'string') {
const wanted = candidate.trim().toLowerCase();
if (tiers.includes(wanted)) return wanted;
}
return tiers[0];
}
/** An image as MiniMax takes it: a URL or a data URI. */
#image(value: unknown, label: string): string | undefined {
if (!present(value)) return undefined;
if (typeof value !== 'string' || !value.trim()) {
throw badRequest(
`${label} must be an image URL, data URI or base64`,
);
}
return toUrlOrDataUri(value);
}
#references(value: unknown): string[] {
if (!present(value)) return [];
if (!Array.isArray(value)) {
throw badRequest('reference_images must be an array');
}
return value.map((image) => this.#image(image, 'reference_images')!);
}
#coerceSeconds(value: unknown): number | undefined {
const numeric =
typeof value === 'string' ? Number.parseInt(value, 10) : value;
return typeof numeric === 'number' &&
Number.isFinite(numeric) &&
numeric > 0
? Math.round(numeric)
: undefined;
}
/** Snap width/height to one of MiniMax's aspect ratios. */
#aspectRatio(width?: number, height?: number): string | undefined {
if (
typeof width !== 'number' ||
typeof height !== 'number' ||
!Number.isFinite(width) ||
!Number.isFinite(height) ||
width <= 0 ||
height <= 0
) {
return undefined;
}
const gcd = (a: number, b: number): number =>
b === 0 ? a : gcd(b, a % b);
const w = Math.round(width);
const h = Math.round(height);
const d = gcd(w, h) || 1;
const candidate = `${w / d}:${h / d}`;
// Other ratios are left to the 16:9 default.
return ASPECT_RATIOS.includes(candidate) ? candidate : undefined;
}
}
@@ -0,0 +1,234 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { IVideoModel } from '../../types.js';
/** MiniMax-specific model behavior the shared IVideoModel shape can't carry. */
export interface MiniMaxVideoSpec {
/** The model name as MiniMax spells it on the wire. */
apiModel: string;
/**
* `v2`: the H3 task API, billed per second of output. `v1`: the Hailuo API,
* billed per clip, whose result is fetched through a file id.
*/
api: 'v1' | 'v2';
/**
* Frames and reference images together. MiniMax charges per image above
* this quota, so more are refused rather than billed.
*/
maxImages: number;
/** Whether `reference_images` are taken (H3 reference-to-video). */
references: boolean;
firstFrameRequired?: boolean;
lastFrame: boolean;
/** Resolutions MiniMax renders only from a first frame. */
imageOnlyResolutions?: string[];
maxPromptChars: number;
}
// Duration ladder for the driver's normalization: element 0 is the fallback
// default, the rest enumerate the model's contiguous valid range.
const seconds = (def: number, min: number, max: number): number[] => [
def,
...Array.from({ length: max - min + 1 }, (_, i) => min + i).filter(
(s) => s !== def,
),
];
// Prices from https://platform.minimax.io/docs/guides/pricing-video. H3 models
// are per second of output, keyed `per-second:<resolution>`; Hailuo models are
// per clip, keyed `per-video:<resolution>:<seconds>`, and a key's absence means
// the model doesn't offer that combination. `default-duration-per-video` is
// the default clip at the default resolution, for cross-provider sorting and
// display only.
export const MINIMAX_VIDEO_GENERATION_MODELS: IVideoModel[] = [
{
id: 'minimax-h3',
puterId: 'minimax:minimax/minimax-h3',
name: 'MiniMax H3',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-second:768p': 8,
'per-second:2k': 13,
'default-duration-per-video': 40,
},
durationSeconds: seconds(5, 4, 15),
dimensions: ['768p', '2k'],
fps: [24],
},
{
id: 'minimax-h3-highspeed',
puterId: 'minimax:minimax/minimax-h3-highspeed',
name: 'MiniMax H3 Highspeed',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-second:768p': 6,
'default-duration-per-video': 30,
},
durationSeconds: seconds(5, 4, 15),
dimensions: ['768p'],
fps: [24],
},
{
id: 'minimax-h3-lite',
puterId: 'minimax:minimax/minimax-h3-lite',
name: 'MiniMax H3 Lite',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-second:768p': 4,
'per-second:480p': 2.5,
'default-duration-per-video': 20,
},
durationSeconds: seconds(5, 4, 15),
dimensions: ['768p', '480p'],
fps: [24],
},
{
id: 'minimax-h3-max',
puterId: 'minimax:minimax/minimax-h3-max',
name: 'MiniMax H3 Max',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-second:768p': 8,
'per-second:480p': 5,
'default-duration-per-video': 40,
},
durationSeconds: seconds(5, 5, 15),
dimensions: ['768p', '480p'],
fps: [24],
},
{
id: 'minimax-hailuo-2.3',
puterId: 'minimax:minimax/minimax-hailuo-2.3',
name: 'MiniMax Hailuo 2.3',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-video:768p:6': 28,
'per-video:768p:10': 56,
'per-video:1080p:6': 49,
'default-duration-per-video': 28,
},
durationSeconds: [6, 10],
dimensions: ['768p', '1080p'],
fps: [24],
},
{
id: 'minimax-hailuo-2.3-fast',
puterId: 'minimax:minimax/minimax-hailuo-2.3-fast',
name: 'MiniMax Hailuo 2.3 Fast',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-video:768p:6': 19,
'per-video:768p:10': 32,
'per-video:1080p:6': 33,
'default-duration-per-video': 19,
},
durationSeconds: [6, 10],
dimensions: ['768p', '1080p'],
fps: [24],
},
{
id: 'minimax-hailuo-02',
puterId: 'minimax:minimax/minimax-hailuo-02',
name: 'MiniMax Hailuo 02',
costs_currency: 'usd-cents',
output_cost_key: 'default-duration-per-video',
costs: {
'per-video:768p:6': 28,
'per-video:768p:10': 56,
'per-video:1080p:6': 49,
'per-video:512p:6': 10,
'per-video:512p:10': 15,
'default-duration-per-video': 28,
},
durationSeconds: [6, 10],
dimensions: ['768p', '1080p', '512p'],
fps: [24],
},
];
// Capabilities per https://platform.minimax.io/docs/api-reference/video-generation-v2-create
// (H3) and the text-, image- and frames-to-video pages under
// https://platform.minimax.io/docs/api-reference/video-generation-t2v (Hailuo).
export const MINIMAX_VIDEO_SPECS: Record<string, MiniMaxVideoSpec> = {
'minimax-h3': {
apiModel: 'MiniMax-H3',
api: 'v2',
maxImages: 5,
references: true,
lastFrame: true,
maxPromptChars: 7000,
},
'minimax-h3-highspeed': {
apiModel: 'MiniMax-H3-Highspeed',
api: 'v2',
maxImages: 5,
references: true,
lastFrame: true,
maxPromptChars: 7000,
},
'minimax-h3-lite': {
apiModel: 'MiniMax-H3-Lite',
api: 'v2',
maxImages: 5,
references: true,
lastFrame: true,
maxPromptChars: 7000,
},
'minimax-h3-max': {
apiModel: 'MiniMax-H3-Max',
api: 'v2',
maxImages: 2,
references: true,
lastFrame: true,
maxPromptChars: 7000,
},
'minimax-hailuo-2.3': {
apiModel: 'MiniMax-Hailuo-2.3',
api: 'v1',
maxImages: 1,
references: false,
lastFrame: false,
maxPromptChars: 2000,
},
'minimax-hailuo-2.3-fast': {
apiModel: 'MiniMax-Hailuo-2.3-Fast',
api: 'v1',
maxImages: 1,
references: false,
firstFrameRequired: true,
lastFrame: false,
maxPromptChars: 2000,
},
'minimax-hailuo-02': {
apiModel: 'MiniMax-Hailuo-02',
api: 'v1',
maxImages: 2,
references: false,
lastFrame: true,
imageOnlyResolutions: ['512p'],
maxPromptChars: 2000,
},
};
File diff suppressed because it is too large. Load diff
@@ -14,7 +14,10 @@ import {
APP_DATA_KV_METHOD_OPS,
appDataPermission,
} from '../../services/permission/appDataScopes.ts';
import { kvNamespace } from '../../stores/systemKv/SystemKVStore.ts';
import { createTestUser, setupTestServer } from '../../testUtil.ts';
import type { IConfig } from '../../types.ts';
import { sealCursor, unsealCursor } from '../../util/pagination.ts';
import { KV_COSTS } from './costs.ts';
import type { KVStoreDriver } from './KVStoreDriver.ts';
import {
@@ -1552,6 +1555,301 @@ describe('KVStoreDriver', () => {
expect(own).toContain('token');
});
// -- Cross-app cursors ----------------------------------------------
const secret = () =>
(target as unknown as { config: IConfig }).config.jwt_secret_v2;
/** A cursor as the server reads it, its key decoded, or null. */
const opened = (cursor: string) => {
const payload = unsealCursor(cursor, secret());
return payload
? {
...payload,
k: Buffer.from(String(payload.k), 'base64url').toString(
'utf8',
),
}
: null;
};
/** Everything a cursor shows to the server and to a base64 decoder. */
const revealed = (cursor: string) =>
[
JSON.stringify(opened(cursor)),
Buffer.from(cursor, 'base64').toString('utf8'),
Buffer.from(cursor, 'base64url').toString('utf8'),
].join('\n');
const encode = (payload: unknown) =>
Buffer.from(JSON.stringify(payload)).toString('base64');
type Page = { items: string[]; cursor?: string; total?: number };
const writeAs = (
actorForCall: Actor,
key: string,
disableSharing = false,
) =>
inCtx(
() =>
target.set({
key,
value: key,
...(disableSharing
? { optConfig: { disableSharing: true } }
: {}),
}),
actorForCall,
);
it('resumes a cross-app page after a private entry it skipped, padding every cursor to one length', async () => {
const { owner, calendar, contacts, calendarActor, contactsActor } =
await setup();
await writeAs(contactsActor, 'a-hidden-oauth-token-name', true);
await writeAs(contactsActor, 'b');
await writeAs(contactsActor, 'c-a-much-longer-public-key-name');
await grant(owner, calendar.uid, appDataPermission(contacts.uid));
const page = (cursor?: string) =>
crossApp(calendarActor, contacts.uid, (optConfig) =>
target.list({
as: 'keys',
limit: 1,
...(cursor ? { cursor } : {}),
optConfig,
}),
) as Promise<Page>;
// The scan's first row is the private one.
const first = await page();
expect(first.items).toEqual(['b']);
expect(revealed(first.cursor!)).not.toContain('hidden-oauth');
expect(opened(first.cursor!)?.k).toBe('b');
const second = await page(first.cursor);
expect(second.items).toEqual(['c-a-much-longer-public-key-name']);
expect(second.cursor!.length).toBe(first.cursor!.length);
});
// Over a thousand writes and reads: slow when the whole suite runs.
it('pads a cursor left on a long run of private entries to the same length', async () => {
const { owner, calendar, contacts, calendarActor, contactsActor } =
await setup();
// More than one call reads past, so the page ends on a hidden row.
const hidden = Array.from(
{ length: 1100 },
(_, i) => `h-hidden-${String(i).padStart(4, '0')}`,
);
for (let at = 0; at < hidden.length; at += 1000)
await inCtx(
() =>
target.batchPut({
items: hidden
.slice(at, at + 1000)
.map((key) => ({ key, value: 1 })),
optConfig: { disableSharing: true },
}),
contactsActor,
);
await writeAs(contactsActor, 'z');
await grant(owner, calendar.uid, appDataPermission(contacts.uid));
const keys: string[] = [];
const cursors: string[] = [];
let emptyPages = 0;
let cursor: string | undefined;
for (let pages = 0; pages < 10; pages++) {
const page = (await crossApp(
calendarActor,
contacts.uid,
(optConfig) =>
target.list({
as: 'keys',
limit: 1,
...(cursor ? { cursor } : {}),
optConfig,
}),
)) as Page;
keys.push(...page.items);
if (page.items.length === 0) emptyPages++;
cursor = page.cursor;
if (!cursor) break;
cursors.push(cursor);
expect(
Buffer.from(cursor, 'base64').toString('utf8'),
).not.toContain('h-hidden');
}
expect(keys).toEqual(['entry', 'z']);
expect(emptyPages).toBeGreaterThan(0);
expect(new Set(cursors.map((c) => c.length)).size).toBe(1);
}, 30_000);
describe('paging a namespace with private entries in it', () => {
// Lengths differ, so an unpadded cursor would show which it is.
const PUBLIC = [
'entry',
'k1',
'k3-longer',
'k4',
'k6-longest-of-all',
];
const PRIVATE = ['k2', 'k5', 'k7', 'k8'];
const seeded = async () => {
const fixture = await setup();
for (const key of PUBLIC.slice(1))
await writeAs(fixture.contactsActor, key);
for (const key of PRIVATE)
await writeAs(fixture.contactsActor, key, true);
await grant(
fixture.owner,
fixture.calendar.uid,
appDataPermission(fixture.contacts.uid),
);
return fixture;
};
it.each(
[false, true].flatMap((reverse) =>
[false, true].map((includeTotal) => ({
reverse,
includeTotal,
})),
),
)(
'returns every public entry once, reverse $reverse, includeTotal $includeTotal',
async ({ reverse, includeTotal }) => {
const { contacts, calendarActor } = await seeded();
const keys: string[] = [];
const lengths = new Set<number>();
let cursor: string | undefined;
let pages = 0;
do {
const page = (await crossApp(
calendarActor,
contacts.uid,
(optConfig) =>
target.list({
as: 'keys',
limit: 2,
...(reverse ? { reverse } : {}),
...(includeTotal ? { includeTotal } : {}),
...(cursor ? { cursor } : {}),
optConfig,
}),
)) as Page;
keys.push(...page.items);
if (includeTotal)
expect(page.total).toBe(PUBLIC.length);
cursor = page.cursor;
if (cursor) {
lengths.add(cursor.length);
const position = opened(cursor);
expect(position).not.toBeNull();
expect(PRIVATE).not.toContain(position!.k);
}
pages++;
} while (cursor && pages < 10);
expect(keys).toEqual(
reverse ? [...PUBLIC].reverse() : PUBLIC,
);
expect(lengths.size).toBe(1);
},
);
it('still reads a plain cursor from before they were sealed, and seals the next one', async () => {
const { owner, contacts, calendarActor } = await seeded();
const namespace = kvNamespace(owner.user.uuid!, contacts.uid);
const list = (cursor: unknown, limit = 10) =>
crossApp(calendarActor, contacts.uid, (optConfig) =>
target.list({ as: 'keys', limit, cursor, optConfig }),
) as Promise<Page>;
expect(
(await list(encode({ namespace, key: 'k1' }))).items,
).toEqual(PUBLIC.slice(2));
expect(
(
await list(
encode({
reverse: true,
key: { namespace, key: 'k4' },
}),
)
).items,
).toEqual(PUBLIC.slice(0, 3).reverse());
const next = await list(encode({ namespace, key: 'k1' }), 1);
expect(next.items).toEqual(['k3-longer']);
expect(opened(next.cursor!)).toMatchObject({ k: 'k3-longer' });
});
it('refuses a plain or sealed cursor for another namespace', async () => {
const { owner, calendar, contacts, calendarActor } =
await seeded();
const other = await makeRealApp(owner.user.id!);
await writeAs(asApp(owner, other), 'k1');
await writeAs(asApp(owner, other), 'k2');
await grant(owner, calendar.uid, appDataPermission(other.uid));
const fromOther = (
(await crossApp(calendarActor, other.uid, (optConfig) =>
target.list({ as: 'keys', limit: 1, optConfig }),
)) as Page
).cursor!;
await writeAs(calendarActor, 'own-1');
await writeAs(calendarActor, 'own-2');
const ownCursor = (
(await inCtx(
() => target.list({ as: 'keys', limit: 1 }),
calendarActor,
)) as Page
).cursor!;
const elsewhere = kvNamespace(owner.user.uuid!, other.uid);
const stranger = kvNamespace('someone-else', contacts.uid);
const forged: unknown[] = [
fromOther,
encode({ namespace: elsewhere, key: 'k1' }),
encode({ namespace: stranger, key: 'k1' }),
{ namespace: elsewhere, key: 'k1' },
encode({
reverse: true,
key: { namespace: elsewhere, key: 'k1' },
}),
encode({
namespace: kvNamespace(owner.user.uuid!, contacts.uid),
key: 'k1',
extra: true,
}),
sealCursor({ id: 1 }, secret()),
sealCursor(
{
s: elsewhere,
k: Buffer.from('k1').toString('base64url'),
},
secret(),
),
ownCursor,
];
for (const cursor of forged)
await expect(
crossApp(calendarActor, contacts.uid, (optConfig) =>
target.list({
as: 'keys',
limit: 1,
cursor,
optConfig,
}),
),
JSON.stringify(cursor),
).rejects.toMatchObject({
statusCode: 400,
message: 'invalid kv: cursor',
});
});
});
it('refuses cross-app writes and deletes against a private entry', async () => {
const { owner, calendar, contacts, calendarActor, contactsActor } =
await setup();
+550 -78
View File
@@ -32,9 +32,29 @@ import {
appDataPermission,
appDataSharingAllowed,
} from '../../services/permission/appDataScopes.js';
import type { KVOpts, KVUsage } from '../../stores/systemKv/SystemKVStore.js';
import {
KV_GLOBAL_APP_KEY,
KV_MAX_KEY_BYTES,
kvNamespace,
type KVOpts,
type KVUsage,
} from '../../stores/systemKv/SystemKVStore.js';
import { KV_SHARE_HANDLE_WIDTHS } from '../../stores/events/columnWidths.js';
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
import { normalizeLimit } from '../../util/pagination.js';
import { isKvHandleId } from '../../services/events/subjects.js';
import {
KV_HANDLE_KEY_MAX_BYTES,
KV_SHARE_PREFIX_MAX_BYTES,
kvHandlesDisabled,
kvShareLevelForMethod,
kvSharePermission,
} from '../../services/kvShare/kvShares.js';
import {
decodeCursor,
normalizeLimit,
sealCursor,
unsealCursor,
} from '../../util/pagination.js';
import { KV_COSTS } from './costs.js';
import {
KV_MAX_BATCH_PUT_ITEMS,
@@ -42,16 +62,143 @@ import {
KV_MAX_LIST_LIMIT,
} from './limits.js';
/** `optConfig` as every KV method takes it. */
interface KvOptConfig {
/** Another app's namespace, which the user let this app reach. */
appUuid?: string;
/** Writes the entry private to the app that wrote it. */
disableSharing?: boolean;
/**
* A share handle: the call reaches that region of its owner's data, with
* keys relative to it.
*/
handle?: string;
/** `set` only: write only when no live entry holds the key. */
onlyIfAbsent?: boolean;
}
/**
* Every KV method's argument object, as far as option resolution cares: the
* namespace override, plus the expiry parameters that can delete an entry.
*/
type KvCallArgs = {
optConfig?: { appUuid?: string; disableSharing?: boolean };
optConfig?: KvOptConfig;
expireAt?: unknown;
ttl?: unknown;
};
/**
* Store options plus, for a call through a share handle, the handle and its
* region's prefix: keys go in under it and come back out relative to it.
*/
type DriverKvOpts = KVOpts & { handle?: string; handlePrefix?: string };
type KvListResult =
| unknown[]
| { items: unknown[]; cursor?: string; total?: number };
/**
* A listing of another namespace, cross-app or through a handle: what its
* cursors are sealed to, the namespace a position rebuilds in, and the prefix
* its keys are relative to.
*/
interface ForeignListing {
bind: string;
namespace: string;
prefix: string;
/** Cross-app cursors were plain before they were sealed. */
acceptsPlain: boolean;
}
/**
* Plaintext bytes a foreign listing's cursor is padded to: a key at the store's
* limit in base64url, plus room for the binding and the envelope.
*/
const FOREIGN_CURSOR_BYTES = Math.ceil((KV_MAX_KEY_BYTES * 4) / 3) + 256;
/** Whether `value` is a store position, `{ namespace, key }`, in `namespace`. */
const isPositionIn = (value: unknown, namespace: string): boolean => {
if (!value || typeof value !== 'object' || Array.isArray(value))
return false;
const { namespace: named, key, ...rest } = value as Record<string, unknown>;
return (
named === namespace &&
typeof key === 'string' &&
Object.keys(rest).length === 0
);
};
// -- Share-handle errors ----------------------------------------------
/**
* Unknown, retired, not the caller's, owner gone or suspended, or namespace app
* deleted: one body for all of them, so no call can tell them apart.
*/
const handleNotFound = (): HttpError =>
new HttpError(404, 'kv: no such share handle', {
legacyCode: 'kv_handle_not_found',
});
/** Only ever told to the handle's holder. */
const handleAccessDenied = (): HttpError =>
new HttpError(403, 'kv: this share handle does not allow this call', {
legacyCode: 'kv_handle_access_denied',
});
const handleOptionInvalid = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'kv_handle_option_invalid' });
const handleKeyInvalid = (): HttpError =>
new HttpError(
400,
`kv: a key used with a share handle must be a non-empty string of at most ${KV_HANDLE_KEY_MAX_BYTES} bytes`,
{ legacyCode: 'kv_handle_key_invalid' },
);
/** The key limit, so no pattern length can tell how long the prefix is. */
const handlePatternInvalid = (): HttpError =>
new HttpError(
400,
`kv: a pattern used with a share handle may be at most ${KV_HANDLE_KEY_MAX_BYTES} bytes`,
{ legacyCode: 'kv_handle_key_invalid' },
);
const invalidCursor = (): HttpError =>
new HttpError(400, 'invalid kv: cursor', { legacyCode: 'bad_request' });
/** A stored key outside the handle's region, which a scan can't return. */
const outsideRegion = (): HttpError =>
new HttpError(500, 'kv: internal error', { legacyCode: 'internal_error' });
const viaHandle = (args?: KvCallArgs): boolean =>
args?.optConfig?.handle !== undefined && args.optConfig.handle !== null;
/** `onlyIfAbsent` as a boolean, refusing anything else a raw body sends. */
const parseOnlyIfAbsent = (value: unknown): boolean => {
if (value === undefined || value === null || value === false) return false;
if (value === true) return true;
throw new HttpError(400, 'kv: onlyIfAbsent must be a boolean', {
legacyCode: 'bad_request',
});
};
/**
* A store error with every stored key it names put back the way the caller
* named it, so the handle's prefix never leaves in a message.
*/
const relativizeError = (
err: unknown,
prefix: string,
keys: readonly string[],
): unknown => {
if (!(err instanceof Error) || typeof err.message !== 'string') return err;
let message = err.message;
for (const key of keys)
message = message.split(`${prefix}${key}`).join(key);
if (message !== err.message) err.message = message;
return err;
};
/**
* Methods that stay available to an account with nothing left of its budget.
* Each one only ever reduces what the account is storing, and turning those
@@ -141,13 +288,25 @@ export class KVStoreDriver extends PuterDriver {
}));
}
#coerceKey(key: unknown): string {
/**
* A key as the caller sent it. Through a handle it is relative to the
* region, and held to the handle's own fixed limit.
*/
#coerceKey(key: unknown, args?: KvCallArgs): string {
if (key === null || key === undefined) {
throw new HttpError(400, 'Missing `key`', {
legacyCode: 'bad_request',
}); // legacyCode for backward compatibility with old error handling in controllers
}
const str = typeof key === 'string' ? key : String(key);
if (viaHandle(args)) {
if (
str === '' ||
Buffer.byteLength(str, 'utf8') > KV_HANDLE_KEY_MAX_BYTES
)
throw handleKeyInvalid();
return str;
}
if (str === '')
throw new HttpError(400, 'Missing `key`', {
legacyCode: 'bad_request',
@@ -155,7 +314,38 @@ export class KVStoreDriver extends PuterDriver {
return str;
}
async #opts(method: string, args: KvCallArgs): Promise<KVOpts> {
/** The key as stored: through a handle, placed under its prefix. */
#storeKey(key: string, opts: DriverKvOpts): string {
return opts.handlePrefix === undefined
? key
: `${opts.handlePrefix}${key}`;
}
/** A stored key as a handle's caller names it; fails closed outside it. */
#relativeKey(key: unknown, prefix: string): string {
if (typeof key !== 'string' || !key.startsWith(prefix))
throw outsideRegion();
return key.slice(prefix.length);
}
/**
* Run one store call. Through a handle, an error naming a stored key names
* the caller's relative key instead.
*/
async #inRegion<T>(
opts: DriverKvOpts,
keys: readonly string[],
call: () => Promise<T>,
): Promise<T> {
if (opts.handlePrefix === undefined) return call();
try {
return await call();
} catch (err) {
throw relativizeError(err, opts.handlePrefix, keys);
}
}
async #opts(method: string, args: KvCallArgs): Promise<DriverKvOpts> {
const actor = Context.get('actor') as Actor | undefined;
// Every method resolves its options here first, so this is the one
@@ -170,6 +360,8 @@ export class KVStoreDriver extends PuterDriver {
);
}
if (viaHandle(args)) return this.#handleOpts(actor, method, args);
const appUuid = args.optConfig?.appUuid;
// Through the issuer chain, not `actor.app`: an access-token actor
// carries no app of its own, so keying off `app` would read a token an
@@ -198,6 +390,214 @@ export class KVStoreDriver extends PuterDriver {
return { actor, namespaceAppUuid: appUuid };
}
/**
* A call through a share handle, resolved to the owner's namespace. The
* handle row only names the region and whose it is; the grant, checked here
* on every call, is the authority. One check covers every key of the call,
* since they all sit under the handle's prefix.
*/
async #handleOpts(
actor: Actor | undefined,
method: string,
args: KvCallArgs,
): Promise<DriverKvOpts> {
if (this.config.kv?.shareHandles !== true) throw kvHandlesDisabled();
const optConfig = args.optConfig ?? {};
const level = kvShareLevelForMethod(method);
if (optConfig.appUuid || optConfig.disableSharing)
throw handleOptionInvalid(
'kv: `handle` cannot be combined with `appUuid` or `disableSharing`',
);
if (level === null)
throw handleOptionInvalid(
`kv: \`${method}\` is not available through a share handle`,
);
const handle: unknown = optConfig.handle;
if (typeof handle !== 'string')
throw handleOptionInvalid('kv: `handle` must be a string');
const userId = actor?.user?.id;
if (
!actor ||
typeof userId !== 'number' ||
handle.length > KV_SHARE_HANDLE_WIDTHS.handle.max ||
!isKvHandleId(handle)
)
throw handleNotFound();
const live = await this.stores.kvShareHandle.getLive(handle);
if (!live || live.ownerSuspended) throw handleNotFound();
const { row, ownerUuid } = live;
// Bound to its grantee: a handle id is an address, not a bearer
// credential, so another holder's overlapping grant doesn't reach it.
if (userId !== row.granteeUserId && userId !== row.ownerUserId)
throw handleNotFound();
// Side by side, so the app lookup costs no round trip of its own.
const [appGone, allowed] = await Promise.all([
row.appUid === KV_GLOBAL_APP_KEY
? false
: this.stores.app.getByUid(row.appUid).then((app) => !app),
this.services.permission.check(
actor,
kvSharePermission(ownerUuid, row.appUid, row.keyPrefix, level),
),
]);
if (appGone) throw handleNotFound();
if (!allowed) {
// Asked only on the way out: whoever can still watch the region
// is its holder, and only they may learn the level is too low.
const holds = await this.services.permission.check(
actor,
kvSharePermission(
ownerUuid,
row.appUid,
row.keyPrefix,
'watch',
),
);
throw holds ? handleAccessDenied() : handleNotFound();
}
// Past the cap only a watch handle can be, reached through a wider
// grant; the fixed key limit holders are shown assumes the cap.
if (
Buffer.byteLength(row.keyPrefix, 'utf8') > KV_SHARE_PREFIX_MAX_BYTES
)
throw handleAccessDenied();
return {
actor,
namespaceUser: { uuid: ownerUuid, id: row.ownerUserId },
namespaceAppUuid: row.appUid,
handle,
handlePrefix: row.keyPrefix,
};
}
/** Set when a listing covers another namespace, cross-app or by handle. */
#foreignListing(opts: DriverKvOpts): ForeignListing | undefined {
if (opts.handle !== undefined)
return {
bind: opts.handle,
namespace: kvNamespace(
opts.namespaceUser!.uuid,
opts.namespaceAppUuid!,
),
prefix: opts.handlePrefix!,
acceptsPlain: false,
};
if (!opts.namespaceAppUuid) return undefined;
const namespace = kvNamespace(
opts.actor!.user.uuid!,
opts.namespaceAppUuid,
);
return { bind: namespace, namespace, prefix: '', acceptsPlain: true };
}
/**
* A foreign listing's cursor as a store position, rebuilt in the listing's
* own namespace and region. Only a cursor sealed to this listing opens, or
* a plain cross-app one naming the same namespace.
*/
#cursorIn(
cursor: unknown,
listing: ForeignListing,
): Record<string, unknown> | undefined {
if (cursor === undefined || cursor === null) return undefined;
if (typeof cursor === 'string') {
if (cursor.trim() === '') return undefined;
const sealed = unsealCursor(
cursor.trim(),
this.config.jwt_secret_v2,
);
if (sealed) {
const { s, k, r, ...rest } = sealed;
if (
s !== listing.bind ||
typeof k !== 'string' ||
(r !== undefined && r !== true) ||
Object.keys(rest).length > 0
)
throw invalidCursor();
const position = {
namespace: listing.namespace,
key: `${listing.prefix}${Buffer.from(k, 'base64url').toString('utf8')}`,
};
return r ? { reverse: true, key: position } : position;
}
} else if (typeof cursor !== 'object') throw invalidCursor();
if (!listing.acceptsPlain) throw invalidCursor();
const plain = decodeCursor(
cursor as string | Record<string, unknown>,
'kv: cursor',
);
if (plain === undefined || isPositionIn(plain, listing.namespace))
return plain;
const { reverse, key, ...rest } = plain;
if (
reverse === true &&
Object.keys(rest).length === 0 &&
isPositionIn(key, listing.namespace)
)
return plain;
throw invalidCursor();
}
/**
* The store's next-page position, sealed to the listing and padded to one
* length, with its key relative to the region.
*/
#cursorOut(cursor: string, listing: ForeignListing): string | undefined {
const decoded = decodeCursor(cursor);
const reverse = decoded?.reverse === true;
const position = (reverse ? decoded?.key : decoded) as
| { namespace?: unknown; key?: unknown }
| undefined;
if (position?.namespace !== listing.namespace) throw outsideRegion();
const key = this.#relativeKey(position.key, listing.prefix);
return sealCursor(
{
s: listing.bind,
k: Buffer.from(key, 'utf8').toString('base64url'),
...(reverse ? { r: true } : {}),
},
this.config.jwt_secret_v2,
{ padTo: FOREIGN_CURSOR_BYTES },
);
}
/** A foreign listing as its caller sees it: keys relative, cursor sealed. */
#foreignResult(
res: KvListResult,
as: string | undefined,
listing: ForeignListing,
): KvListResult {
const { prefix } = listing;
const relative = (items: unknown[]): unknown[] => {
if (as === 'values' || prefix === '') return items;
if (as === 'keys')
return items.map((key) => this.#relativeKey(key, prefix));
return items.map((entry) => {
const { key, value } = entry as {
key: unknown;
value: unknown;
};
return { key: this.#relativeKey(key, prefix), value };
});
};
if (Array.isArray(res)) return relative(res);
const cursor = res.cursor
? this.#cursorOut(res.cursor, listing)
: undefined;
return {
items: relative(res.items),
...(cursor ? { cursor } : {}),
...(res.total !== undefined ? { total: res.total } : {}),
};
}
async #assertCrossAppKvAccess(
actor: Actor,
targetAppUid: string,
@@ -322,7 +722,7 @@ export class KVStoreDriver extends PuterDriver {
async get(args: {
key: unknown;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const { key } = args;
if (key === undefined || key === null) {
@@ -343,45 +743,55 @@ export class KVStoreDriver extends PuterDriver {
if (Array.isArray(key)) {
if (key.length === 0) return [];
const coerced = key.map((k) => this.#coerceKey(k));
const { res, usage } = await this.stores.kv.get(
{ key: coerced },
opts,
const coerced = key.map((k) => this.#coerceKey(k, args));
const { res, usage } = await this.#inRegion(opts, coerced, () =>
this.stores.kv.get(
{ key: coerced.map((k) => this.#storeKey(k, opts)) },
opts,
),
);
this.#meter(opts.actor, usage);
return res;
}
const { res, usage } = await this.stores.kv.get(
{ key: this.#coerceKey(key) },
opts,
const coerced = this.#coerceKey(key, args);
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.get({ key: this.#storeKey(coerced, opts) }, opts),
);
this.#meter(opts.actor, usage);
return res;
}
/**
* `false` only for an `onlyIfAbsent` write that found a live entry, which
* is left as it was.
*/
async set(args: {
key: unknown;
value: unknown;
expireAt?: number | null;
optConfig?: { appUuid?: string; disableSharing?: boolean };
optConfig?: KvOptConfig;
}): Promise<boolean> {
const { key, value, expireAt } = args;
const coerced = this.#coerceKey(key);
const coerced = this.#coerceKey(key, args);
if (value === undefined)
throw new HttpError(400, 'Missing `value`', {
legacyCode: 'bad_request',
}); // legacyCode for backward compatibility with old error handling in controllers
const onlyIfAbsent = parseOnlyIfAbsent(args.optConfig?.onlyIfAbsent);
const opts = await this.#opts('set', args);
const { res, usage } = await this.stores.kv.set(
{
key: coerced,
value,
expireAt,
disableSharing: args.optConfig?.disableSharing,
},
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.set(
{
key: this.#storeKey(coerced, opts),
value,
expireAt,
disableSharing: args.optConfig?.disableSharing,
onlyIfAbsent,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -389,7 +799,7 @@ export class KVStoreDriver extends PuterDriver {
async batchPut(args: {
items: Array<{ key: string; value: unknown; expireAt?: number | null }>;
optConfig?: { appUuid?: string; disableSharing?: boolean };
optConfig?: KvOptConfig;
}): Promise<boolean> {
const { items } = args;
if (!Array.isArray(items) || items.length === 0) {
@@ -404,17 +814,27 @@ export class KVStoreDriver extends PuterDriver {
{ legacyCode: 'bad_request' },
);
}
// A batch write takes no condition, so it can't promise this per item.
if (parseOnlyIfAbsent(args.optConfig?.onlyIfAbsent)) {
throw new HttpError(
400,
'kv: onlyIfAbsent is not available on a batch write',
{ legacyCode: 'invalid_request' },
);
}
const coerced = items.map((item) => ({
key: this.#coerceKey(item.key),
key: this.#coerceKey(item.key, args),
value: item.value,
expireAt: item.expireAt,
}));
const opts = await this.#opts('batchPut', args);
// Per-item expiry, which the top-level scan in `#opts` cannot see.
// Per-item expiry, which the top-level scan in `#opts` cannot see. A
// handle's `write` level already carries expiries.
if (
opts.namespaceAppUuid &&
opts.handlePrefix === undefined &&
coerced.some(
(item) => item.expireAt !== undefined && item.expireAt !== null,
)
@@ -424,12 +844,20 @@ export class KVStoreDriver extends PuterDriver {
opts.namespaceAppUuid,
);
}
const { res, usage } = await this.stores.kv.batchPut(
{
items: coerced,
disableSharing: args.optConfig?.disableSharing,
},
const { res, usage } = await this.#inRegion(
opts,
coerced.map((item) => item.key),
() =>
this.stores.kv.batchPut(
{
items: coerced.map((item) => ({
...item,
key: this.#storeKey(item.key, opts),
})),
disableSharing: args.optConfig?.disableSharing,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -437,11 +865,13 @@ export class KVStoreDriver extends PuterDriver {
async del(args: {
key: unknown;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<boolean> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
const opts = await this.#opts('del', args);
const { res, usage } = await this.stores.kv.del({ key: coerced }, opts);
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.del({ key: this.#storeKey(coerced, opts) }, opts),
);
this.#meter(opts.actor, usage);
return res;
}
@@ -455,7 +885,7 @@ export class KVStoreDriver extends PuterDriver {
includeTotal?: boolean;
fetchUntilFull?: boolean;
reverse?: boolean;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const opts = await this.#opts('list', args);
// Naming what it holds is how an account with nothing left decides what
@@ -469,6 +899,14 @@ export class KVStoreDriver extends PuterDriver {
this.config,
);
}
const handlePrefix = opts.handlePrefix;
const listing = this.#foreignListing(opts);
if (
handlePrefix !== undefined &&
typeof args.pattern === 'string' &&
Buffer.byteLength(args.pattern, 'utf8') > KV_HANDLE_KEY_MAX_BYTES
)
throw handlePatternInvalid();
const { res, usage } = await this.stores.kv.list(
{
as: args.as,
@@ -476,20 +914,26 @@ export class KVStoreDriver extends PuterDriver {
cap: KV_MAX_LIST_LIMIT,
label: 'kv: limit',
}),
cursor: args.cursor,
cursor: listing
? this.#cursorIn(args.cursor, listing)
: args.cursor,
pattern: args.pattern,
offset: args.offset,
includeTotal: args.includeTotal,
fetchUntilFull: args.fetchUntilFull,
reverse: args.reverse,
...(handlePrefix === undefined
? {}
: { keyPrefix: handlePrefix }),
},
opts,
);
this.#meter(opts.actor, usage);
return res;
if (!listing) return res;
return this.#foreignResult(res as KvListResult, args.as, listing);
}
async flush(args: { optConfig?: { appUuid?: string } }): Promise<boolean> {
async flush(args: { optConfig?: KvOptConfig }): Promise<boolean> {
const opts = await this.#opts('flush', args);
const { res, usage } = await this.stores.kv.flush(opts);
this.#meter(opts.actor, usage);
@@ -499,9 +943,9 @@ export class KVStoreDriver extends PuterDriver {
async incr(args: {
key: unknown;
pathAndAmountMap: Record<string, number>;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (
!args.pathAndAmountMap ||
typeof args.pathAndAmountMap !== 'object'
@@ -511,9 +955,14 @@ export class KVStoreDriver extends PuterDriver {
});
}
const opts = await this.#opts('incr', args);
const { res, usage } = await this.stores.kv.incr(
{ key: coerced, pathAndAmountMap: args.pathAndAmountMap },
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.incr(
{
key: this.#storeKey(coerced, opts),
pathAndAmountMap: args.pathAndAmountMap,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -522,9 +971,9 @@ export class KVStoreDriver extends PuterDriver {
async decr(args: {
key: unknown;
pathAndAmountMap: Record<string, number>;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (
!args.pathAndAmountMap ||
typeof args.pathAndAmountMap !== 'object'
@@ -534,9 +983,14 @@ export class KVStoreDriver extends PuterDriver {
});
}
const opts = await this.#opts('decr', args);
const { res, usage } = await this.stores.kv.decr(
{ key: coerced, pathAndAmountMap: args.pathAndAmountMap },
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.decr(
{
key: this.#storeKey(coerced, opts),
pathAndAmountMap: args.pathAndAmountMap,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -545,18 +999,23 @@ export class KVStoreDriver extends PuterDriver {
async expireAt(args: {
key: unknown;
timestamp: number;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<boolean> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (typeof args.timestamp !== 'number') {
throw new HttpError(400, '`timestamp` must be a number', {
legacyCode: 'bad_request',
});
}
const opts = await this.#opts('expireAt', args);
const { res, usage } = await this.stores.kv.expireAt(
{ key: coerced, timestamp: args.timestamp },
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.expireAt(
{
key: this.#storeKey(coerced, opts),
timestamp: args.timestamp,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -565,18 +1024,20 @@ export class KVStoreDriver extends PuterDriver {
async expire(args: {
key: unknown;
ttl: number;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<boolean> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (typeof args.ttl !== 'number') {
throw new HttpError(400, '`ttl` must be a number (seconds)', {
legacyCode: 'bad_request',
});
}
const opts = await this.#opts('expire', args);
const { res, usage } = await this.stores.kv.expire(
{ key: coerced, ttl: args.ttl },
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.expire(
{ key: this.#storeKey(coerced, opts), ttl: args.ttl },
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -586,22 +1047,24 @@ export class KVStoreDriver extends PuterDriver {
key: unknown;
pathAndValueMap: Record<string, unknown>;
ttl?: number | null;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (!args.pathAndValueMap || typeof args.pathAndValueMap !== 'object') {
throw new HttpError(400, 'Missing or invalid `pathAndValueMap`', {
legacyCode: 'bad_request',
});
}
const opts = await this.#opts('update', args);
const { res, usage } = await this.stores.kv.update(
{
key: coerced,
pathAndValueMap: args.pathAndValueMap,
ttl: args.ttl,
},
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.update(
{
key: this.#storeKey(coerced, opts),
pathAndValueMap: args.pathAndValueMap,
ttl: args.ttl,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -610,18 +1073,23 @@ export class KVStoreDriver extends PuterDriver {
async add(args: {
key: unknown;
pathAndValueMap: Record<string, unknown>;
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (!args.pathAndValueMap || typeof args.pathAndValueMap !== 'object') {
throw new HttpError(400, 'Missing or invalid `pathAndValueMap`', {
legacyCode: 'bad_request',
});
}
const opts = await this.#opts('add', args);
const { res, usage } = await this.stores.kv.add(
{ key: coerced, pathAndValueMap: args.pathAndValueMap },
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.add(
{
key: this.#storeKey(coerced, opts),
pathAndValueMap: args.pathAndValueMap,
},
opts,
),
);
this.#meter(opts.actor, usage);
return res;
@@ -630,23 +1098,27 @@ export class KVStoreDriver extends PuterDriver {
async remove(args: {
key: unknown;
paths: string[];
optConfig?: { appUuid?: string };
optConfig?: KvOptConfig;
}): Promise<unknown> {
const coerced = this.#coerceKey(args.key);
const coerced = this.#coerceKey(args.key, args);
if (!Array.isArray(args.paths) || args.paths.length === 0) {
throw new HttpError(400, 'Missing or invalid `paths`', {
legacyCode: 'bad_request',
});
}
const opts = await this.#opts('remove', args);
const { res, usage } = await this.stores.kv.remove(
{ key: coerced, paths: args.paths },
opts,
const { res, usage } = await this.#inRegion(opts, [coerced], () =>
this.stores.kv.remove(
{ key: this.#storeKey(coerced, opts), paths: args.paths },
opts,
),
);
this.#meter(opts.actor, usage);
// The result is the entry's value, so a delete-only grant gets none.
// The result is the entry's value, so a delete-only grant gets none. A
// handle's `write` level reads too.
if (
opts.namespaceAppUuid &&
opts.handlePrefix === undefined &&
!(await this.services.permission.check(
opts.actor!,
appDataPermission(opts.namespaceAppUuid, 'kv', 'get'),
@@ -36,6 +36,7 @@ import {
} from '../../services/notification/notificationAudience.js';
import {
NOTIFICATION_VALUE_MAX_BYTES,
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
resolveNotificationWrite,
type NotificationTypeName,
} from '../../services/notification/notificationTypes.js';
@@ -221,7 +222,13 @@ export class NotificationDriver extends PuterDriver {
scope: MailboxScope | null;
filter?: string;
onlyUnacknowledged?: boolean;
excludeTypes?: readonly string[];
} = { limit, scope };
// The whole mailbox, or a slice naming no app, is the desktop's view of
// it, which leaves out the types only an app has a use for. A slice
// naming the app still returns them.
if (scope === null || scope.appUid === undefined)
query.excludeTypes = OUTSIDE_ACCOUNT_MAILBOX_TYPES;
switch (predicateName) {
case 'unseen':
+5 -2
View File
@@ -441,6 +441,7 @@ describe('PuterServer host header validation — permissive modes', () => {
allow_no_host_header: false,
custom_domains_enabled: true,
allow_nipio_domains: true,
static_hosting_domain: 'site.puter.localhost',
} as unknown as IConfig,
{ listen: true },
);
@@ -450,11 +451,13 @@ describe('PuterServer host header validation — permissive modes', () => {
await server?.shutdown();
});
it('lets an unknown host through when custom domains are enabled', async () => {
it('hands an unknown host to site hosting when custom domains are enabled', async () => {
const res = await rawRequest(port, '/', {
host: 'my-own-domain.example',
});
expect(res.status).not.toBe(400);
// No site claims it, so it 404s instead of reaching the GUI.
expect(res.status).toBe(404);
expect(res.body).toBe('Site not found');
});
it('accepts nip.io hosts when they are opted in', async () => {
@@ -51,8 +51,9 @@ import {
EVENTS_UNSUBSCRIBE_VERB,
type DeliveryEnvelope,
type EventSocket,
type KvDispatchOptions,
} from './EventsService.js';
import { kvSharePermission } from './kvShares.js';
import { kvSharePermission } from '../kvShare/kvShares.js';
import { FILTER_EVALUATIONS_PER_EVENT } from './matcher.js';
import { SUBSCRIPTION_CACHE_TTL_MS } from './subscriptionCache.js';
import { kvAnchorToken } from './subjects.js';
@@ -203,6 +204,15 @@ let handles: Map<string, KvShareHandle>;
const kvShareHandleStore = {
getByHandle: async (handle: string) => handles.get(handle) ?? null,
getLive: async (handle: string) => {
const row = handles.get(handle);
if (!row || row.revokedAt !== null) return null;
return {
row,
ownerUuid: `user-${row.ownerUserId}`,
ownerSuspended: false,
};
},
};
/** Apps the cross-app gate can be asked about, and what they share. */
@@ -2108,6 +2118,7 @@ it('sends only the projected shape, never an internal row', async () => {
'subject',
'ts',
'uid',
'viaWorker',
]);
expect(sent[0].envelope.event).not.toHaveProperty('userId');
});
@@ -2554,6 +2565,47 @@ describe('delivering a kv change', () => {
expect(sent).toEqual([]);
});
it('tells a subscriber the change was theirs only when they made it', async () => {
vi.useFakeTimers();
await subscribeKv(`kv:${OWN_APP}:cart`);
const write = (options: KvDispatchOptions) =>
service.dispatchKv(
{
userId,
namespace: `v1:user-${userId}:${OWN_APP}`,
keys: ['cart'],
op: 'set',
},
options,
);
// One window each: writes to one key inside a window coalesce.
for (const options of [
{ actingUserId: userId },
{ actingUserId: userId + 1 },
{},
{ actingUserId: userId, viaWorker: true },
]) {
await write(options);
await vi.advanceTimersByTimeAsync(EVENTS_COALESCE_WINDOW_MS + 1);
}
expect(
sent.map(({ envelope }) => {
const { self, viaWorker } = envelope.event as {
self: boolean;
viaWorker: boolean;
};
return { self, viaWorker };
}),
).toEqual([
{ self: true, viaWorker: false },
{ self: false, viaWorker: false },
{ self: false, viaWorker: false },
{ self: true, viaWorker: true },
]);
});
it('sends only the projected shape', async () => {
vi.useFakeTimers();
await subscribeKv(`kv:${OWN_APP}:cart`);
@@ -2569,6 +2621,7 @@ describe('delivering a kv change', () => {
'seq',
'subject',
'ts',
'viaWorker',
]);
});
});
@@ -2843,6 +2896,7 @@ describe('the cross-app kv gate', () => {
'seq',
'subject',
'ts',
'viaWorker',
]);
});
@@ -2951,8 +3005,10 @@ describe('cross-user kv handles', () => {
appUid: OWN_APP,
keyPrefix: PREFIX,
permission,
label: null,
createdAt: 0,
revokedAt: null,
revokedBy: null,
...overrides,
};
handles.set(row.handle, row);
+136 -531
View File
@@ -31,9 +31,6 @@ import {
EVENTS_HANDLER_DEPTH,
EVENTS_HANDLER_PUBLISH_BATCH,
EVENTS_HANDLER_PUBLISH_LIMIT,
EVENTS_KV_HANDLE_LIMIT,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_USER,
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT,
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT,
EVENTS_KV_VALUE_MAX_BYTES,
@@ -53,7 +50,6 @@ import {
isAccessTokenActor,
isAccountContext,
makeActor,
userRelatedActor,
type Actor,
} from '../../core/actor.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
@@ -62,15 +58,12 @@ import type {
ReanchorInput,
SuspendedReason,
} from '../../stores/events/DurableSubscriptionStore.js';
import type { LiveKvShareHandle } from '../../stores/events/KvShareHandleStore.js';
import {
DurableQuotaRaceLost,
HANDLER_SETTLE_BATCH,
isSuspendedReason,
} from '../../stores/events/DurableSubscriptionStore.js';
import type {
KvShareHandle,
KvShareHandleListOptions,
} from '../../stores/events/KvShareHandleStore.js';
import {
HANDLER_NAME_MAX_LENGTH,
hashContent,
@@ -98,10 +91,7 @@ import {
type SubscriptionTarget,
} from '../../stores/events/types.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import {
KV_GLOBAL_APP_KEY,
parseKvNamespace,
} from '../../stores/systemKv/SystemKVStore.js';
import { parseKvNamespace } from '../../stores/systemKv/SystemKVStore.js';
import { awaitLater } from '../../util/concurrency.js';
import {
openCursor,
@@ -111,13 +101,17 @@ import {
import type { AclMode, ResourceDescriptor } from '../acl/ACLService.js';
import { resolveNode } from '../fs/resolveNode.js';
import { maskUnderAnchor } from '../fs/sharePathMask.js';
import { assertActorHasCredits } from '../metering/enforcement.js';
import { actorPlanId, assertActorHasCredits } from '../metering/enforcement.js';
import {
canViewNotification,
notificationRowScope,
ownedAppUids,
} from '../notification/notificationAudience.js';
import { notificationsFoldInEnabled } from '../notification/notificationSocket.js';
import {
inAccountMailbox,
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
} from '../notification/notificationTypes.js';
import {
appSocketRoom,
type SocketSpecifier,
@@ -196,18 +190,10 @@ import {
} from './registry.js';
import { SubscriptionCache } from './subscriptionCache.js';
import {
assertBoundedManageGrant,
assertShareableAppUid,
assertShareablePermission,
assertShareablePrefix,
kvShareAppDelegateImplicator,
kvShareGrantCovers,
kvShareManageNamespaceRoot,
kvShareManagePermission,
kvShareOwnerImplicator,
kvSharePermission,
relativeToKvShareRoot,
} from './kvShares.js';
} from '../kvShare/kvShares.js';
import {
KV_MATCH_SEPARATOR,
NOTIF_MATCH_SEPARATOR,
@@ -216,6 +202,7 @@ import {
isKvToken,
isNotifToken,
kvHandleFromSubject,
notifMatchSpansEveryApp,
parseSubject,
type FsOp,
type ParsedSubject,
@@ -432,35 +419,6 @@ export interface DurableSubscriptionView extends SubscriptionView {
suspendedReason: string | null;
}
/** Body of the handle-minting surface. The grantee is named either way. */
export interface MintKvHandleRequest {
granteeUsername?: unknown;
granteeUid?: unknown;
appUid?: unknown;
prefix?: unknown;
}
/**
* What minting returns. Only the handle and the region it covers: the owner
* already knows the rest, and the grantee is handed this verbatim, so anything
* else here would be something the handle exists to not say.
*/
export interface MintedKvHandle {
handle: string;
prefix: string;
}
/**
* One handle as its owner sees it, revoked ones included — they are the record
* of what was shared and when it stopped.
*/
export interface KvShareHandleView extends MintedKvHandle {
appUid: string;
granteeUsername: string | null;
createdAt: number;
revokedAt: number | null;
}
export type VerbAck<T extends object> =
| ({ ok: true } & T)
| { ok: false; error: { code: string; message: string } };
@@ -576,6 +534,8 @@ export interface FsDispatchOptions {
actingUserId?: number;
/** Handler runs behind the write; see `Actor.handlerDepth`. */
handlerDepth?: number;
/** The write was made by server-side code; see `viaWorkerOf`. */
viaWorker?: boolean;
/** Present only for a move: where the entry lived before it, lazily. */
movedFrom?: {
path: string;
@@ -596,6 +556,8 @@ export interface FsDispatchOptions {
export interface KvDispatchOptions {
/** Who performed the write, for the `self` flag. */
actingUserId?: number;
/** The write was made by server-side code; see `viaWorkerOf`. */
viaWorker?: boolean;
/** See {@link FsDispatchOptions.forwarded}. */
forwarded?: true;
/** Carried over from the emitting region so both copies share one id/ts. */
@@ -633,6 +595,9 @@ export interface KvDispatchInput {
noShareKeys?: readonly string[];
/** Handler runs behind the write; see `Actor.handlerDepth`. */
handlerDepth?: number;
/** Who made the change; the listener hands it on as an option. */
actingUserId?: number;
viaWorker?: true;
}
interface RouteLimits {
@@ -831,19 +796,6 @@ const handlerAppRequired = (): HttpError =>
legacyCode: 'events_handler_app_required',
});
/**
* Listing and revoking are the owner's, and an app's only where the handle is
* the app's own: its namespace, its user, and a delegation it still holds.
* Anything else — another app, a token an app issued, a handle outside the
* namespace — reads as somebody else's surface.
*/
const handleOwnerOnly = (): HttpError =>
new HttpError(
403,
'Only the account, or the app whose namespace this is, may manage share handles',
{ legacyCode: 'events_kv_handle_owner_only' },
);
/** A scoped token, whoever issued it: neither the account nor the app. */
const eventsWorkerOwnerOnly = (): HttpError =>
new HttpError(
@@ -852,62 +804,6 @@ const eventsWorkerOwnerOnly = (): HttpError =>
{ legacyCode: 'events_worker_owner_only' },
);
/**
* An app minting is delegation, and the `manage:` grant on the region is the
* consent for it. Without one there is nothing authorizing the app to dispose
* of its user's data, whatever the user themselves could do here.
*/
const handleNotDelegated = (): HttpError =>
new HttpError(
403,
'This app has not been given permission to share this data',
{ legacyCode: 'events_kv_handle_not_delegated' },
);
/**
* A delegation is the app's to hold, not to pass on: a token it minted may
* carry the `manage:` permission and still not mint through it. The user's own
* token is not this case — it acts for the user, who needs no delegation.
*/
const handleAccessTokenForbidden = (): HttpError =>
new HttpError(403, 'An access token may not mint a share handle', {
legacyCode: 'forbidden',
});
/**
* An app addresses exactly one key-value namespace — its own, under its user —
* so that is the only one it can hand a region of out.
*/
const handleOutsideNamespace = (): HttpError =>
new HttpError(403, 'An app may only share its own key-value data', {
legacyCode: 'events_kv_handle_outside_namespace',
});
/**
* A temporary account holds no share handles: the grant outlives the session
* that made it, and there is no account left behind it to take one back.
*/
const kvHandleNeedsAccount = (): HttpError =>
new HttpError(403, 'A temporary account may not share its key-value data', {
legacyCode: 'events_kv_handle_requires_account',
});
/**
* Past a handle ceiling. One code for both of them — the remedy is the same,
* revoke something — with the message saying which one it was.
*/
const handleQuotaReached = (
limit: number,
scope: 'account' | 'app',
): HttpError =>
new HttpError(
409,
scope === 'app'
? `An app may hold out ${limit} share handles for one account`
: `An account may hold out ${limit} share handles at a time`,
{ legacyCode: 'events_kv_handle_limit_reached' },
);
/**
* Deploying an app's code is the developer's, and an app token cannot borrow
* its user's ownership of some other app. Same answer for an app that is not
@@ -1474,6 +1370,17 @@ export class EventsService extends PuterService {
});
});
// A retired handle is a dead address: what was made through it ends,
// even where another grant still covers the region. Awaited by the
// emitter, so a revoke returns once its subscriptions are settled.
this.clients.event.on('kv.share.retired', async (_key, data) => {
try {
await this.settleRetiredKvHandles(data);
} catch (err) {
console.warn('[events] retired-handle settle failed', err);
}
});
// A deleted app's rows can never deliver again — the grant identity
// they are re-checked under no longer resolves — so they are torn
// down rather than left to cost their holder an anchor slot forever.
@@ -1507,9 +1414,17 @@ export class EventsService extends PuterService {
// a service. Same posture as the FS hook: post-commit, and nothing here
// can fail the write that produced it.
this.clients.event.on('kv.mutated', (_key, data) => {
void this.dispatchKv(data as KvDispatchInput).catch((err) => {
console.warn('[events] kv dispatch failed', err);
});
const input = data as KvDispatchInput;
void (async () => {
try {
await this.dispatchKv(input, {
actingUserId: input.actingUserId,
viaWorker: input.viaWorker === true,
});
} catch (err) {
console.warn('[events] kv dispatch failed', err);
}
})();
});
// Emitted once the row is written, so a delivery never names a
@@ -1520,19 +1435,6 @@ export class EventsService extends PuterService {
});
});
// Owning a key-value namespace is holding every share grant over it,
// which is what lets its owner mint a handle on their own data.
this.services.permission.registerImplicator(kvShareOwnerImplicator());
// Lets an app-under-user actor exercise a share grant its user holds,
// scoped to the app the grant's namespace names.
this.services.permission.registerImplicator(
kvShareAppDelegateImplicator({
userHolds: (actor, permission) =>
this.services.permission.check(actor, permission),
}),
);
this.#armExpirySweep();
this.#armPendingSweep();
this.#armCreditSweep();
@@ -1934,15 +1836,8 @@ export class EventsService extends PuterService {
}
/** The plan this actor is on, or `null` where there are no plans. */
async #planId(actor: Actor): Promise<string | null> {
const metering = this.services.metering;
if (!metering) return null;
try {
return (await metering.getActorSubscription(actor)).id;
} catch (err) {
console.warn('[events] could not resolve a plan', err);
return null;
}
#planId(actor: Actor): Promise<string | null> {
return actorPlanId(this.services.metering, actor);
}
/**
@@ -2047,11 +1942,16 @@ export class EventsService extends PuterService {
);
// One extra row answers "is there another page" without a count.
// The generic slice is the desktop's view of the mailbox, so it leaves
// out the types kept off the desktop, as the live delivery does.
const rows = await this.stores.notification.listScoped(user.id, {
audience: scope.audience,
appUid: scope.appUid,
after: Number.isFinite(cursored) ? cursored : null,
limit: limit + 1,
...(scope.appUid === undefined
? { excludeTypes: OUTSIDE_ACCOUNT_MAILBOX_TYPES }
: {}),
});
const page = rows.slice(0, limit);
@@ -2399,373 +2299,6 @@ export class EventsService extends PuterService {
return { appUid, removed, suspended };
}
// -- Cross-user key-value handles --------------------------------
/**
* Hand another user a watchable region of this account's key-value data.
*
* Two writes, in this order: the grant, which is the authorization and
* carries its own `manage:` check and its own refusal to grant to yourself,
* and then the handle, which is only a name for it. A handle whose grant
* never landed would be a name for nothing; a grant whose handle never
* landed is unaddressable, since a handle is the only thing that can name
* this family.
*
* An app may mint too, bounded the way sharing bounds an app handing out
* its user's files: authority is the user's, asked of the user behind the
* actor and never of the app, and reach is what the credential itself
* holds. Here reach is structural — an app addresses one namespace — so it
* is asserted rather than worked out, and the app's own consent to delegate
* is a `manage:` grant on the region.
*/
async mintKvHandle(
actor: Actor,
request: MintKvHandleRequest,
): Promise<MintedKvHandle> {
if (!this.enabled) throw disabled();
if (!this.kvHandlesEnabled) throw kvShareHandleDisabled();
const owner = actor.user;
if (!owner?.uuid || owner.id === undefined) throw disabled();
// `undefined` is an app that could not be resolved, not the absence of
// one — reading it as an account session is what would hand an app a
// surface bounded on the app it is acting as.
const app = actor.effectiveApp;
if (app === undefined) throw handleOwnerOnly();
// An account's surface is the account's, and a token scoped to less
// than its issuer holds is not the account. A token an app issued is
// refused by the delegation check below, which has its own answer.
if (app === null && !isAccountContext(actor)) throw handleOwnerOnly();
// The budget is the user's, so an app spends its user's slots rather
// than a machine-rate allowance of its own.
await this.#spendHandleBudget(owner.id);
const keyPrefix = assertShareablePrefix(request?.prefix);
const appUid = this.#mintNamespace(app, request);
const grantee = await this.#resolveGrantee(request);
const permission = assertShareablePermission(
kvSharePermission(owner.uuid, appUid, keyPrefix),
);
// Before the idempotent answer below, or an app that lost its
// delegation could read back a handle it may no longer hand out.
if (app) await this.#assertKvShareDelegated(actor, permission);
// Resolved before the idempotent answer below, so a plan that holds
// none is refused whatever it asks for. Kept behind the delegation
// check above: an app with no delegation must not learn anything
// about the account's plan.
const quota = await this.#handleQuota(actor);
// The same region minted again for the same grantee is the same
// capability, not a second one — handing back what already exists is
// what keeps two handles from ever sharing one permission row. The
// ceiling counts slots, and this takes none, so it is not asked.
const existing = await this.stores.kvShareHandle.findLive({
ownerUserId: owner.id,
granteeUserId: grantee.id,
appUid,
keyPrefix,
});
if (existing)
return { handle: existing.handle, prefix: existing.keyPrefix };
await this.#assertHandleCeiling(owner.id, appUid, quota);
// The user behind the app, so the authority checked is theirs and the
// issuer recorded is them. Which app acted is carried separately: it
// belongs on the audit row, not in the grant's authority.
await this.services.permission.grantUserUserPermission(
userRelatedActor(actor),
grantee.username,
permission,
{},
{ reason: 'kv share handle', appUid: app?.uid },
);
const row = await this.stores.kvShareHandle.mint({
ownerUserId: owner.id,
granteeUserId: grantee.id,
appUid,
keyPrefix,
permission,
});
return { handle: row.handle, prefix: row.keyPrefix };
}
/**
* Which namespace a mint may name. An app reaches `v1:<user>:<its own app>`
* and nothing else, so a request naming another one is refused rather than
* quietly minted somewhere the app cannot even write.
*/
#mintNamespace(
app: Actor['effectiveApp'],
request: MintKvHandleRequest,
): string {
const named = parseAppUid(request?.appUid);
if (!app) return assertShareableAppUid(named ?? KV_GLOBAL_APP_KEY);
if (named !== null && named !== app.uid) throw handleOutsideNamespace();
return app.uid;
}
/**
* Whether this app was given the region to hand out. The `manage:` grant is
* the user's consent, and prefix implication makes one taken on a region
* cover the keys beneath it — so a handle deeper than the consent is still
* inside it, and one above it is not.
*/
async #assertKvShareDelegated(
actor: Actor,
permission: string,
): Promise<void> {
// Consequential enough to require the app's own session, whatever a
// token it minted happens to carry.
if (isAccessTokenActor(actor)) throw handleAccessTokenForbidden();
// The consent surface refuses a namespace-root delegation because no
// prompt can describe it; refused here too, so a row written any other
// way cannot authorize a mint.
assertBoundedManageGrant(kvShareManagePermission(permission));
const delegated = await this.services.permission.canManagePermission(
actor,
permission,
);
if (!delegated) throw handleNotDelegated();
// `canManagePermission` walks ancestors, so a namespace-root grant
// written by any path other than the consent surface would otherwise
// authorize a mint anywhere in the namespace, bounded region or not.
const unbounded = await this.services.permission.check(
actor,
kvShareManageNamespaceRoot(permission),
);
if (unbounded) throw handleNotDelegated();
}
/**
* Take a shared region back.
*
* The grant goes first — the subtree with it, since a deeper grant would
* leave access to part of a region that has just been withdrawn — and the
* handle is retired after. Removing the grant is what actually settles the
* subscriptions, and it is the only half that stops delivery, so it is the
* half that must not be able to fail after the other has succeeded. In that
* order both steps are idempotent and a failure anywhere is a retry: the
* reverse leaves a handle nothing can address standing on a grant
* everything still passes, and no way to try again.
*
* Not gated on the feature flag. Withdrawing access only ever narrows, and
* an install that turned handles off must still be able to retire the ones
* it has.
*/
async revokeKvHandle(
actor: Actor,
handle: unknown,
): Promise<{ handle: string; revokedAt: number }> {
if (!this.enabled) throw disabled();
const owner = actor.user;
if (owner?.id === undefined) throw disabled();
// `undefined` is an app that could not be resolved, not the absence of
// one — reading it as an account session is what would hand an app the
// account's whole surface.
const app = actor.effectiveApp;
if (app === undefined) throw handleOwnerOnly();
// A delegation is the app's to hold, not to pass on, so a token an app
// issued is refused here as it is on the mint path — and a user's own
// token acts for the user only when it carries the user's whole reach.
if (isAccessTokenActor(actor) && !isAccountContext(actor))
throw handleOwnerOnly();
await this.#spendHandleBudget(owner.id);
const named = typeof handle === 'string' ? handle : '';
const share = await this.stores.kvShareHandle.getByHandle(named);
// Unknown and somebody else's read the same way, so revoking cannot be
// used to find out that a handle exists.
if (!share || share.ownerUserId !== owner.id)
throw unknownKvShareHandle(named);
// An app reaches one namespace, so a handle outside it is not its to
// take back — checked after the owner test, which must answer first.
if (app && share.appUid !== app.uid) throw handleOwnerOnly();
// The same consent the mint took: a withdrawn delegation withdraws the
// app's surface on the region with it.
if (app) await this.#assertKvShareDelegated(actor, share.permission);
const grantee = await this.stores.user.getById(share.granteeUserId);
if (!grantee?.username)
throw new HttpError(500, 'The grantee of this handle is gone', {
legacyCode: 'internal_error',
});
await this.services.permission.revokeUserUserPermissionSubtree(
userRelatedActor(actor),
grantee.username,
share.permission,
{ reason: 'kv share handle revoked', appUid: app?.uid },
);
const revoked = await this.stores.kvShareHandle.retire(named, owner.id);
if (!revoked?.revokedAt) throw unknownKvShareHandle(named);
// A narrower handle to the same grantee shares this one's permission
// row whenever its region nests under it, and the subtree revoke
// above just withdrew that row — so its grant is already gone even
// though its own row still reads live.
await this.#retireCoveredHandles(revoked);
return { handle: revoked.handle, revokedAt: revoked.revokedAt };
}
/**
* Every other live handle to this grantee that the just-withdrawn grant
* covered. Coverage runs on the permission string, whose third component is
* the namespace app uid, so a covered handle is always inside the same
* one.
*/
async #retireCoveredHandles(revoked: KvShareHandle): Promise<void> {
const siblings =
await this.stores.kvShareHandle.listLiveForOwnerAndGrantee(
revoked.ownerUserId,
revoked.granteeUserId,
);
await Promise.all(
siblings
.filter((row) =>
kvShareGrantCovers(revoked.permission, row.permission),
)
.map((row) =>
this.stores.kvShareHandle.retire(
row.handle,
revoked.ownerUserId,
),
),
);
}
/**
* What this account has shared out of its key-value data, retired handles
* included: they are the only record of what was shared and when it
* stopped. The grantee has no listing of their own — what they hold is the
* handle they were given.
*/
async listKvHandles(
actor: Actor,
options: KvShareHandleListOptions = {},
): Promise<PageResult<KvShareHandleView>> {
if (!this.enabled) throw disabled();
const owner = actor.user;
if (owner?.id === undefined) throw disabled();
const app = actor.effectiveApp;
if (app === undefined) throw handleOwnerOnly();
if (isAccessTokenActor(actor) && !isAccountContext(actor))
throw handleOwnerOnly();
// An app sees its own namespace and nothing else; an account session
// sees across apps, which is what makes the account the surface for a
// row whose app is long gone. Listing disposes of nothing, so the gate
// is app-ness and the namespace rather than the region's delegation.
const page = await this.stores.kvShareHandle.listForOwner(owner.id, {
limit: options.limit,
cursor: options.cursor,
includeTotal: options.includeTotal,
appUid: app?.uid ?? null,
});
const grantees = await this.stores.user.getByIds([
...new Set(page.items.map((row) => row.granteeUserId)),
]);
return {
...page,
items: page.items.map((row) => ({
handle: row.handle,
prefix: row.keyPrefix,
appUid: row.appUid,
granteeUsername:
grantees.get(row.granteeUserId)?.username ?? null,
createdAt: row.createdAt,
revokedAt: row.revokedAt,
})),
};
}
/** Who a mint is for. Named by username or uuid; unknown reads as absent. */
async #resolveGrantee(
request: MintKvHandleRequest,
): Promise<{ id: number; username: string }> {
const username =
typeof request?.granteeUsername === 'string'
? request.granteeUsername.trim()
: '';
const uid =
typeof request?.granteeUid === 'string'
? request.granteeUid.trim()
: '';
if (!username && !uid)
throw badRequest(
'Name the grantee with `granteeUsername` or `granteeUid`',
'bad_request',
);
const user = username
? await this.stores.user.getByUsername(username)
: await this.stores.user.getByUuid(uid);
if (!user?.username || user.id === undefined)
throw new HttpError(404, 'user_does_not_exist', {
legacyCode: 'subject_does_not_exist',
});
return { id: user.id, username: user.username };
}
/**
* How many handles this caller's plan lets the account hold out, in total
* and in one namespace. Resolved before the idempotent answer on the mint
* path, so a plan that may hold none is refused whatever it asks for.
*
* A deployment with no metering has no plans to read, and is held to the
* paid caps.
*/
async #handleQuota(actor: Actor): Promise<TieredQuota> {
const plan = await this.#planId(actor);
if (plan !== null && limitFor(EVENTS_KV_HANDLES_PER_USER, plan) === 0)
throw kvHandleNeedsAccount();
return {
perUser: limitFor(EVENTS_KV_HANDLES_PER_USER, plan),
perApp: limitFor(EVENTS_KV_HANDLES_PER_APP, plan),
};
}
/**
* Whether this account may hold out another share handle, in total and in
* the namespace this mint names. Retired ones do not count: the row stays
* as the record of what was shared, not as a slot. Check-then-act: two
* mints racing past it can both proceed, which the mint budget bounds well
* enough for an abuse backstop.
*/
async #assertHandleCeiling(
userId: number,
appUid: string,
quota: TieredQuota,
): Promise<void> {
const live = await this.stores.kvShareHandle.countLiveForOwner(
userId,
appUid,
);
if (live.total >= quota.perUser)
throw handleQuotaReached(quota.perUser, 'account');
// The global namespace is the account's own shelf rather than an
// app's, so the account cap is its only bound — which is also what
// keeps that cap reachable without naming a namespace.
if (appUid !== KV_GLOBAL_APP_KEY && live.forApp >= quota.perApp)
throw handleQuotaReached(quota.perApp, 'app');
}
async #spendHandleBudget(userId: number): Promise<void> {
const ok = await checkRateLimit(
`${EVENTS_KV_HANDLE_LIMIT.scope}:${userId}`,
EVENTS_KV_HANDLE_LIMIT.limit,
EVENTS_KV_HANDLE_LIMIT.window,
);
if (!ok) throw tooManyCalls();
}
// -- Suspension --------------------------------------------------
/**
@@ -3547,9 +3080,11 @@ export class EventsService extends PuterService {
* keyspace that is matters: the row is indexed under the **owner**, because
* a write only ever knows whose namespace it touched.
*
* Authority is the grant, never the handle: an actor holding the mirrored
* permission may subscribe, and one who does not is told the handle is not
* there rather than that it is theirs to want.
* Authority is the grant, never the handle: an actor holding the grant at
* any level may subscribe, and one who does not is told the handle is not
* there rather than that it is theirs to want. The handle is also bound to
* its holder, so another user whose own grant overlaps the region cannot
* use it.
*/
async #resolveKvHandleSubscribe(
actor: Actor,
@@ -3561,17 +3096,31 @@ export class EventsService extends PuterService {
const share = await this.stores.kvShareHandle.getByHandle(handle);
if (!share || share.revokedAt !== null)
throw unknownKvShareHandle(handle);
const held = await kvSharedRegionAuthorized(
actor,
share.permission,
this.#kvShareDeps(),
);
if (!held) throw unknownKvShareHandle(handle);
if (
actor.user?.id !== share.granteeUserId &&
actor.user?.id !== share.ownerUserId
)
throw unknownKvShareHandle(handle);
const owner = await this.stores.user.getById(share.ownerUserId);
if (!owner?.uuid) throw unknownKvShareHandle(handle);
// Watching is the lowest level, so every level answers it; storing it
// rather than the handle's own grant is what lets a level change leave
// the subscription alone.
const watch = kvSharePermission(
owner.uuid,
share.appUid,
share.keyPrefix,
'watch',
);
const held = await kvSharedRegionAuthorized(
actor,
watch,
this.#kvShareDeps(),
);
if (!held) throw unknownKvShareHandle(handle);
const anchor = resolveKvHandleAnchor(parsed, {
ownerUserUuid: owner.uuid,
appUid: share.appUid,
@@ -3590,7 +3139,7 @@ export class EventsService extends PuterService {
ownerUserId: share.ownerUserId,
// The grant string rather than a mode: it is what a revoke names,
// and what the delivery re-check asks again.
permission: share.permission,
permission: watch,
subject: anchor.subject,
};
}
@@ -3716,6 +3265,7 @@ export class EventsService extends PuterService {
id: options.id ?? randomUUID(),
ts: options.ts ?? Date.now(),
handlerDepth: options.handlerDepth,
viaWorker: options.viaWorker === true,
};
const { local, remote } =
@@ -3733,6 +3283,7 @@ export class EventsService extends PuterService {
family: 'fs',
ownerUserId,
actingUserId: options.actingUserId,
...(context.viaWorker ? { viaWorker: true as const } : {}),
id: context.id,
ts: context.ts,
fs: {
@@ -3823,6 +3374,7 @@ export class EventsService extends PuterService {
id: options.forwarded && options.id ? options.id : randomUUID(),
ts,
handlerDepth: input.handlerDepth,
viaWorker: options.viaWorker === true,
...(noShareKeys.has(kvKey) ? { noShare: true as const } : {}),
}));
const carried: Array<{ value: unknown } | undefined> = [];
@@ -3865,6 +3417,9 @@ export class EventsService extends PuterService {
family: 'kv',
ownerUserId,
actingUserId: options.actingUserId,
...(context.viaWorker
? { viaWorker: true as const }
: {}),
id: context.id,
ts: context.ts,
kv: {
@@ -3931,7 +3486,7 @@ export class EventsService extends PuterService {
subject,
context,
forKey,
ownerUserId,
options.actingUserId,
(matched) =>
this.#kvStillAuthorized(
matched,
@@ -4032,6 +3587,7 @@ export class EventsService extends PuterService {
ancestors: async () => item.fs!.movedFrom!.ancestors,
},
actingUserId: item.actingUserId,
viaWorker: item.viaWorker === true,
forwarded: true,
id: item.id,
ts: item.ts,
@@ -4068,6 +3624,7 @@ export class EventsService extends PuterService {
},
{
actingUserId: item.actingUserId,
viaWorker: item.viaWorker === true,
forwarded: true,
id: item.id,
ts: item.ts,
@@ -4123,10 +3680,14 @@ export class EventsService extends PuterService {
);
if (watched.length === 0) return;
const rows = await this.stores.eventSubscription.getForTokens(
const fetched = await this.stores.eventSubscription.getForTokens(
userId,
watched,
);
// A subscription naming no app is the desktop's view of the mailbox.
const rows = inAccountMailbox(context.type)
? fetched
: fetched.filter((row) => !notifMatchSpansEveryApp(row.match));
if (rows.length === 0) return;
await this.#route(subject, context, rows, userId, (matched) =>
@@ -4494,14 +4055,27 @@ export class EventsService extends PuterService {
}
/**
* Whether one row's holder may still watch the region it was made on.
*
* Held in the cross-event cache under the row's anchor, which for a shared
* region is the whole of what it can address: nothing above the handle is
* nameable, so the answer does not vary by key and one evaluation covers
* every event under it until a grant or a revoke moves the generation.
* Whether one row's holder may still watch the region it was made on. The
* handle must be live and theirs, so a retired one goes silent within the
* handle cache's lifetime even where a wider grant covers the region. The
* grant check doesn't vary by key, so it's cached under the row's anchor.
*/
async #kvShareHolds(row: DispatchSubscription): Promise<boolean> {
const handle = kvHandleFromSubject(row.subject);
if (handle === null) return false;
let live: LiveKvShareHandle | null;
try {
live = await this.stores.kvShareHandle.getLive(handle);
} catch {
return false;
}
if (
!live ||
(live.row.granteeUserId !== row.holderUserId &&
live.row.ownerUserId !== row.holderUserId)
)
return false;
const identity = await this.#grantIdentity(row);
if (!identity) return false;
@@ -4673,6 +4247,37 @@ export class EventsService extends PuterService {
return suspended.length;
}
/**
* Take out of service every durable subscription one user, the share's
* owner or its holder, made through these share handles, with their
* backlogs. One holder-index read, bounded by the user's subscription cap.
* Session rows are left to the delivery re-check, which reads the handle's
* liveness.
*/
async settleRetiredKvHandles(retired: {
handles: readonly string[];
userId: number;
}): Promise<number> {
if (!this.enabled || retired.handles.length === 0) return 0;
const held = await this.stores.durableSubscription.listActiveForHolder(
retired.userId,
null,
{ includeSuspended: true },
);
const handles = new Set(retired.handles);
const settling = held.filter((row) => {
const handle = kvHandleFromSubject(row.subject);
return handle !== null && handles.has(handle);
});
if (settling.length === 0) return 0;
const suspended = await this.#suspend(settling, 'permission_revoked', {
override: RESUMABLE_REASONS,
});
await this.#notifyEnded(suspended, 'permission_revoked');
return suspended.length;
}
/**
* Revoke the reused `events:handlers` worker session for (holder, app), if
* one was ever minted. Best-effort: a lookup or revoke failure must not
@@ -224,6 +224,7 @@ describe('an app connection', () => {
'subject',
'ts',
'uid',
'viaWorker',
]);
});
@@ -81,6 +81,8 @@ export interface ForwardEvent {
ownerUserId: number;
/** Who caused it, so `self` is right on the far side. */
actingUserId?: number;
/** Made by server-side code, so `viaWorker` is right on the far side. */
viaWorker?: true;
/** The emitter's event id and timestamp, kept so the two copies match. */
id: string;
ts: number;
+52 -1
View File
@@ -2044,6 +2044,34 @@ describe('a kv value crossing to another region', () => {
expect(eventsIn(west)[0].kv).toMatchObject({ value: { items: 3 } });
});
it('carries who wrote it, so `self` and `viaWorker` hold on the far side', async () => {
const west = makeRegion('west', ['east'], forwardCfg);
const east = makeRegion('east', ['west'], forwardCfg);
await subscribeKv(east, 'socket-writer');
await announced(east);
await west.events.dispatchKv(
{
userId,
namespace: `v1:user-${userId}:${APP}`,
keys: ['cart'],
op: 'set',
},
{ actingUserId: userId + 1, viaWorker: true },
);
await posted(west);
await arrived(east);
expect(eventsIn(west)[0]).toMatchObject({
actingUserId: userId + 1,
viaWorker: true,
});
expect(east.sent[0].event).toMatchObject({
self: false,
viaWorker: true,
});
});
});
describe('a session subscription in another region', () => {
@@ -2058,10 +2086,33 @@ describe('a session subscription in another region', () => {
await dispatch(west);
await arrived(east);
expect(east.sent[0].event).toMatchObject({ op: 'write', self: true });
expect(east.sent[0].event).toMatchObject({
op: 'write',
self: true,
viaWorker: false,
});
expect(west.sent).toEqual([]);
});
it('carries a server-side write as one on the far side', async () => {
const west = makeRegion('west', ['east'], forwardCfg);
const east = makeRegion('east', ['west'], forwardCfg);
await subscribeSession(east);
await posted(east);
await west.events.dispatchFs('fs.write.file', entry(), {
actingUserId: userId,
viaWorker: true,
ancestors: async () => ancestors(),
});
await arrived(east);
expect(east.sent[0].event).toMatchObject({
self: true,
viaWorker: true,
});
});
it('delivers to a subscriber who is not the writer', async () => {
const west = makeRegion('west', ['east'], forwardCfg);
const east = makeRegion('east', ['west'], forwardCfg);
@@ -42,7 +42,10 @@ import { runWithContext } from '../../core/context.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import type { DeliveryEnvelope } from './EventsService.js';
import { kvShareManagePermission, kvSharePermission } from './kvShares.js';
import {
kvShareManagePermission,
kvSharePermission,
} from '../kvShare/kvShares.js';
const BOOT_TIMEOUT_MS = 120_000;
const SOCKET_ID = 'kv-delegate-socket';
@@ -64,6 +67,7 @@ let appId: number;
let appActor: Actor;
const events = () => env.server.services.events;
const shares = () => env.server.services.kvShare;
const permissions = () => env.server.services.permission;
const settled = (count = 1) =>
@@ -111,11 +115,15 @@ const undelegate = (prefix = PREFIX) =>
);
const mint = (request: Record<string, unknown> = {}, actor = appActor) =>
events().mintKvHandle(actor, {
granteeUsername: guest.username,
prefix: PREFIX,
...request,
});
shares().mintKvHandle(
actor,
{
prefix: PREFIX,
...request,
recipient: { username: guest.username },
},
{ defaultAccess: 'watch' },
);
const clearRows = async () => {
await env.server.clients.db.write(`DELETE FROM \`${TABLE}\``, []);
@@ -191,7 +199,7 @@ describe('an app minting without consent', () => {
legacyCode: 'events_kv_handle_not_delegated',
});
// The share itself is untouched: what was refused is the app asking.
const page = await events().listKvHandles(owner.actor, { limit: 100 });
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
expect(page.items).toContainEqual(
expect.objectContaining({ handle, revokedAt: null }),
);
@@ -405,14 +413,14 @@ describe('an app minting with consent', () => {
it('shows up in the owner`s listing of what they have shared out', async () => {
const { handle } = await mint();
const page = await events().listKvHandles(owner.actor, { limit: 100 });
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
expect(page.items).toContainEqual(
expect.objectContaining({
handle,
prefix: PREFIX,
appUid,
granteeUsername: guest.username,
holderUsername: guest.username,
revokedAt: null,
}),
);
@@ -454,7 +462,7 @@ describe('a handle an app minted', () => {
handlerName: 'onChange',
});
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
await vi.waitFor(
async () => {
const [row] = (await env.server.clients.db.pread(
@@ -540,7 +548,7 @@ describe('a grantee subscribing through their own app actor', () => {
await settled();
expect(delivered[0].subId).toBe(sub.subId);
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
delivered.length = 0;
await appWrites(`${PREFIX}live:2`, 2);
@@ -573,7 +581,7 @@ describe('an app managing what it minted', () => {
handlerName: 'onChange',
});
await events().revokeKvHandle(appActor, handle);
await shares().revokeKvHandle(appActor, handle);
await vi.waitFor(
async () => {
const [row] = (await env.server.clients.db.pread(
@@ -592,7 +600,7 @@ describe('an app managing what it minted', () => {
),
).resolves.toBe(false);
const page = await events().listKvHandles(owner.actor, { limit: 100 });
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
expect(
page.items.find((one) => one.handle === handle)?.revokedAt,
).toBeTypeOf('number');
@@ -607,10 +615,10 @@ describe('an app managing what it minted', () => {
const prefix = `${PREFIX}session-in-app-${uuidv4().slice(0, 8)}:`;
const { handle } = await mint({ appUid, prefix }, owner.actor);
const revoked = await events().revokeKvHandle(appActor, handle);
const revoked = await shares().revokeKvHandle(appActor, handle);
expect(revoked.handle).toBe(handle);
const page = await events().listKvHandles(owner.actor, { limit: 100 });
const page = await shares().listKvHandles(owner.actor, { limit: 100 });
expect(
page.items.find((one) => one.handle === handle)?.revokedAt,
).toBeTypeOf('number');
@@ -624,7 +632,7 @@ describe('an app managing what it minted', () => {
);
await expect(
events().revokeKvHandle(appActor, handle),
shares().revokeKvHandle(appActor, handle),
).rejects.toMatchObject({ legacyCode: 'events_kv_handle_owner_only' });
});
@@ -647,7 +655,7 @@ describe('an app managing what it minted', () => {
});
await expect(
events().revokeKvHandle(siblingActor, handle),
shares().revokeKvHandle(siblingActor, handle),
).rejects.toMatchObject({ legacyCode: 'events_kv_handle_owner_only' });
});
@@ -665,7 +673,7 @@ describe('an app managing what it minted', () => {
});
await expect(
events().revokeKvHandle(tokenActor, handle),
shares().revokeKvHandle(tokenActor, handle),
).rejects.toMatchObject({ legacyCode: 'events_kv_handle_owner_only' });
});
@@ -679,12 +687,12 @@ describe('an app managing what it minted', () => {
await undelegate(prefix);
await expect(
events().revokeKvHandle(appActor, handle),
shares().revokeKvHandle(appActor, handle),
).rejects.toMatchObject({
legacyCode: 'events_kv_handle_not_delegated',
});
const page = await events().listKvHandles(owner.actor, {
const page = await shares().listKvHandles(owner.actor, {
limit: 100,
});
expect(
@@ -698,7 +706,7 @@ describe('an app managing what it minted', () => {
await mint({ appUid: 'os-global', prefix: sessionPrefix }, owner.actor);
await mint({ prefix: appPrefix });
const appPage = await events().listKvHandles(appActor, {
const appPage = await shares().listKvHandles(appActor, {
limit: 100,
includeTotal: true,
});
@@ -708,7 +716,7 @@ describe('an app managing what it minted', () => {
false,
);
const ownerPage = await events().listKvHandles(owner.actor, {
const ownerPage = await shares().listKvHandles(owner.actor, {
limit: 100,
includeTotal: true,
});
@@ -721,7 +729,7 @@ describe('an app managing what it minted', () => {
const key = `rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${owner.id}`;
const before = await env.server.clients.redis.zcard(key);
await events().revokeKvHandle(appActor, handle);
await shares().revokeKvHandle(appActor, handle);
const after = await env.server.clients.redis.zcard(key);
expect(after).toBe(before + 1);
@@ -729,14 +737,14 @@ describe('an app managing what it minted', () => {
it('answers another user`s handle in the same namespace as absent, not as forbidden', async () => {
const prefix = `cross-user-${uuidv4().slice(0, 8)}:`;
const { handle } = await events().mintKvHandle(guest.actor, {
granteeUsername: owner.username,
appUid,
prefix,
});
const { handle } = await shares().mintKvHandle(
guest.actor,
{ recipient: { username: owner.username }, appUid, prefix },
{ defaultAccess: 'watch' },
);
await expect(
events().revokeKvHandle(appActor, handle),
shares().revokeKvHandle(appActor, handle),
).rejects.toMatchObject({ legacyCode: 'subject_does_not_exist' });
});
@@ -744,7 +752,7 @@ describe('an app managing what it minted', () => {
const prefix = `${PREFIX}revoke-audit-probe-${uuidv4().slice(0, 8)}:`;
const { handle } = await mint({ prefix });
await events().revokeKvHandle(appActor, handle);
await shares().revokeKvHandle(appActor, handle);
const row = await vi.waitFor(async () => {
const [found] = (await env.server.clients.db.pread(
@@ -60,6 +60,7 @@ let stranger: TestUser;
let delivered: DeliveryEnvelope[];
const events = () => env.server.services.events;
const shares = () => env.server.services.kvShare;
const settled = (count = 1) =>
vi.waitFor(() => expect(delivered.length).toBeGreaterThanOrEqual(count), {
@@ -88,10 +89,11 @@ const ownerWrites = (key: string, value: unknown): Promise<unknown> =>
);
const mint = (prefix = PREFIX) =>
events().mintKvHandle(owner.actor, {
granteeUsername: guest.username,
prefix,
});
shares().mintKvHandle(
owner.actor,
{ prefix, recipient: { username: guest.username } },
{ defaultAccess: 'watch' },
);
/** A durable `single` row, which is the only kind that holds a backlog. */
const subscribeDurable = async (handle: string) =>
@@ -182,7 +184,7 @@ describe('revoking a handle', () => {
);
expect(await ownerWatches()).toBe(true);
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
await suspendedRow(sub.subId);
// The anchor leaves the *owner's* watched set, which is the keyspace a
@@ -211,7 +213,7 @@ describe('revoking a handle', () => {
await settled();
expect(delivered[0].subId).toBe(sub.subId);
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
delivered.length = 0;
await ownerWrites(`${PREFIX}live:2`, 2);
@@ -229,16 +231,21 @@ describe('revoking a handle', () => {
const settle = vi.spyOn(events(), 'settleRevokedGrant');
try {
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
await suspendedRow(sub.subId);
// One announcement, one settle pass, one indexed read of the
// holder's rows — no lookup keyed on the handle anywhere.
// One grant settle for the holder, and a retired-handle settle
// for each side, each a single indexed read of that user's rows —
// no lookup keyed on the handle anywhere.
expect(settle).toHaveBeenCalledTimes(1);
expect(byHolder).toHaveBeenCalledTimes(1);
expect(byHolder).toHaveBeenCalledWith(guest.id, null, {
includeSuspended: true,
});
expect(
byHolder.mock.calls.map(([userId]) => userId).sort(),
).toEqual([guest.id, guest.id, owner.id].sort());
for (const call of byHolder.mock.calls)
expect(call.slice(1)).toEqual([
null,
{ includeSuspended: true },
]);
} finally {
byHolder.mockRestore();
settle.mockRestore();
@@ -250,7 +257,7 @@ describe('revoking a handle', () => {
const { handle } = await mint();
const deeper = await mint(`${PREFIX}messages:`);
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
// The subtree goes with the region: a grant left underneath would keep
// access to part of what was just taken back.
@@ -271,12 +278,12 @@ describe('revoking a handle', () => {
const deeper = await mint(`${PREFIX}messages:`);
expect((await countLive()).total).toBe(before.total + 2);
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
// The deeper handle's grant died with the subtree revoke above (proven
// by the sibling case), so its row reading live would disagree with
// reality — both in the listing and in the slot it appears to hold.
const page = await events().listKvHandles(owner.actor, {
const page = await shares().listKvHandles(owner.actor, {
limit: 100,
});
const deeperRow = page.items.find((one) => one.handle === deeper.handle);
@@ -289,9 +296,9 @@ describe('revoking a handle', () => {
const { handle } = await mint();
const sibling = await mint('workspace:other:');
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
const page = await events().listKvHandles(owner.actor, {
const page = await shares().listKvHandles(owner.actor, {
limit: 100,
});
const siblingRow = page.items.find(
@@ -303,7 +310,7 @@ describe('revoking a handle', () => {
it('lets a fresh handle over the same region work, and leaves the old one dead', async () => {
await clearRows();
const first = await mint();
await events().revokeKvHandle(owner.actor, first.handle);
await shares().revokeKvHandle(owner.actor, first.handle);
const second = await mint();
expect(second.handle).not.toBe(first.handle);
@@ -334,7 +341,7 @@ describe('revoking a handle', () => {
: owner.actor;
await expect(
events().revokeKvHandle(actor, handle),
shares().revokeKvHandle(actor, handle),
).rejects.toMatchObject({
legacyCode: 'subject_does_not_exist',
});
@@ -345,8 +352,8 @@ describe('revoking a handle', () => {
// between them leaves a retry to finish the job — which only works if
// reaching a retired handle again succeeds.
const { handle } = await mint();
const first = await events().revokeKvHandle(owner.actor, handle);
const again = await events().revokeKvHandle(owner.actor, handle);
const first = await shares().revokeKvHandle(owner.actor, handle);
const again = await shares().revokeKvHandle(owner.actor, handle);
expect(again).toEqual(first);
});
@@ -365,7 +372,7 @@ describe('revoking a handle', () => {
.spyOn(store, 'retire')
.mockRejectedValueOnce(new Error('write failed'));
await expect(
events().revokeKvHandle(owner.actor, handle),
shares().revokeKvHandle(owner.actor, handle),
).rejects.toThrow('write failed');
failing.mockRestore();
@@ -377,7 +384,7 @@ describe('revoking a handle', () => {
expect(delivered).toEqual([]);
// And the retry converges rather than answering "no such handle".
const revoked = await events().revokeKvHandle(owner.actor, handle);
const revoked = await shares().revokeKvHandle(owner.actor, handle);
expect(revoked.handle).toBe(handle);
expect(await retire(handle, owner.id)).toMatchObject({
revokedAt: revoked.revokedAt,
@@ -398,7 +405,7 @@ describe('revoking a handle', () => {
});
await expect(
events().revokeKvHandle(appActor, handle),
shares().revokeKvHandle(appActor, handle),
).rejects.toMatchObject({
legacyCode: 'events_kv_handle_owner_only',
});
@@ -408,9 +415,9 @@ describe('revoking a handle', () => {
describe('the owner`s handle listing', () => {
it('shows what was shared, retired handles included', async () => {
const { handle } = await mint();
await events().revokeKvHandle(owner.actor, handle);
await shares().revokeKvHandle(owner.actor, handle);
const page = await events().listKvHandles(owner.actor, {
const page = await shares().listKvHandles(owner.actor, {
includeTotal: true,
});
const row = page.items.find((one) => one.handle === handle);
@@ -418,7 +425,9 @@ describe('the owner`s handle listing', () => {
expect(row).toMatchObject({
prefix: PREFIX,
appUid: 'os-global',
granteeUsername: guest.username,
holderUsername: guest.username,
access: 'watch',
revokedBy: 'owner',
});
expect(row?.revokedAt).toBeTypeOf('number');
expect(page.total).toBe(page.items.length + (page.cursor ? 1 : 0));
@@ -426,7 +435,7 @@ describe('the owner`s handle listing', () => {
it('shows the grantee nothing of the owner`s', async () => {
await mint();
const page = await events().listKvHandles(guest.actor);
const page = await shares().listKvHandles(guest.actor);
expect(page.items).toEqual([]);
});
@@ -434,11 +443,11 @@ describe('the owner`s handle listing', () => {
await mint();
await mint();
const first = await events().listKvHandles(owner.actor, { limit: 1 });
const first = await shares().listKvHandles(owner.actor, { limit: 1 });
expect(first.items).toHaveLength(1);
expect(first.cursor).toBeTypeOf('string');
const second = await events().listKvHandles(owner.actor, {
const second = await shares().listKvHandles(owner.actor, {
limit: 1,
cursor: first.cursor,
});
@@ -46,7 +46,7 @@ import {
import type { IConfig } from '../../types.js';
import { DEFAULT_FREE_SUBSCRIPTION } from '../metering/consts.js';
import type { DeliveryEnvelope } from './EventsService.js';
import { kvSharePermission } from './kvShares.js';
import { kvSharePermission } from '../kvShare/kvShares.js';
import { isKvHandleId, kvAnchorToken } from './subjects.js';
const BOOT_TIMEOUT_MS = 120_000;
@@ -68,6 +68,7 @@ let stranger: TestUser;
let delivered: DeliveryEnvelope[];
const events = () => env.server.services.events;
const shares = () => env.server.services.kvShare;
const settled = (count = 1) =>
vi.waitFor(() => expect(delivered.length).toBeGreaterThanOrEqual(count), {
@@ -97,14 +98,34 @@ const ownerWrites = (key: string, value: unknown): Promise<unknown> =>
);
const mint = (
request: Record<string, unknown> = {},
{
granteeUsername = guest.username,
...request
}: Record<string, unknown> = {},
actor: Actor = owner.actor,
) =>
events().mintKvHandle(actor, {
granteeUsername: guest.username,
prefix: PREFIX,
...request,
});
shares().mintKvHandle(
actor,
{
prefix: PREFIX,
...request,
recipient: { username: granteeUsername as string },
},
{ defaultAccess: 'watch' },
);
/** An app row under a fresh uid, since a share may only name an app that exists. */
const registerApp = async (
on: PuterTestEnv = env,
ownerUserId: number = owner.id,
): Promise<string> => {
const uid = `app-${uuidv4()}`;
await on.server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[uid, uid, uid, `https://${uid}.example/`, ownerUserId],
);
return uid;
};
const clearRows = async () => {
await env.server.clients.db.write(`DELETE FROM \`${TABLE}\``, []);
@@ -148,7 +169,12 @@ describe('minting a handle', () => {
const minted = await mint();
expect(isKvHandleId(minted.handle)).toBe(true);
expect(minted).toEqual({ handle: minted.handle, prefix: PREFIX });
expect(minted).toMatchObject({
handle: minted.handle,
prefix: PREFIX,
access: 'watch',
holderUsername: guest.username,
});
const wire = JSON.stringify(minted);
expect(wire).not.toContain(owner.uuid);
expect(wire).not.toContain(owner.username);
@@ -299,7 +325,8 @@ describe('minting a handle', () => {
});
it('lets a paid namespace fill the account cap and then refuses other namespaces', async () => {
const namespace = 'app-perapp-probe';
const namespace = await registerApp();
const another = await registerApp();
try {
for (let i = 0; i < EVENTS_KV_HANDLES_PER_APP.limit; i++) {
const prefix = `perapp:${i}:`;
@@ -318,20 +345,20 @@ describe('minting a handle', () => {
legacyCode: 'events_kv_handle_limit_reached',
});
await expect(
mint({ appUid: 'app-another-probe', prefix: 'still:room:' }),
mint({ appUid: another, prefix: 'still:room:' }),
).rejects.toMatchObject({
legacyCode: 'events_kv_handle_limit_reached',
});
} finally {
await env.server.clients.db.write(
'DELETE FROM `kv_share_handles` WHERE `owner_user_id` = ? AND `app_uid` IN (?, ?)',
[owner.id, namespace, 'app-another-probe'],
[owner.id, namespace, another],
);
}
});
it('hands back an existing handle at the cap, taking no slot', async () => {
const namespace = 'app-idempotent-probe';
const namespace = await registerApp();
const prefix = 'idempotent:cap:';
try {
const first = await mint({ appUid: namespace, prefix });
@@ -749,7 +776,7 @@ describe('what a plan lets an account hold', () => {
it('holds a free account to the free per-app cap, with room left in the account cap', async () => {
await setEmail(`${tiered.users.user.username}@example.invalid`);
const namespace = 'app-tiered-free-probe';
const namespace = await registerApp(tiered, tieredUserId);
const cap =
EVENTS_KV_HANDLES_PER_APP.bySubscription[DEFAULT_FREE_SUBSCRIPTION];
const grantee = await tiered.server.stores.user.getByUsername(
@@ -783,7 +810,7 @@ describe('what a plan lets an account hold', () => {
// another namespace still succeeds.
const accepted = await mintTiered(
'still:room:',
'app-tiered-free-other',
await registerApp(tiered, tieredUserId),
);
expect(accepted.status).toBe(200);
});
@@ -1,451 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { v4 as uuidv4 } from 'uuid';
import { describe, expect, it, vi } from 'vitest';
import type { Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { PermissionUtil } from '../permission/permissionUtil.js';
import {
assertBoundedManageGrant,
assertShareableAppUid,
assertShareablePermission,
assertShareablePrefix,
keyPrefixSegments,
kvShareAppDelegateImplicator,
kvShareGrantCovers,
kvShareManagePermission,
kvShareOwnerImplicator,
kvSharePermission,
mintKvHandleId,
normalizeKeyPrefix,
relativeToKvShareRoot,
} from './kvShares.js';
import { isKvHandleId, kvHandleFromSubject } from './subjects.js';
const OWNER = '2a1b0c9d-0000-4000-8000-000000000001';
const OTHER = '2a1b0c9d-0000-4000-8000-000000000002';
const APP = 'app-1234';
const OTHER_APP = 'app-5678';
const userActor = (uuid: string): Actor =>
({ user: { uuid }, effectiveApp: null }) as unknown as Actor;
const appActor = (uuid: string, appUid: string): Actor =>
({
user: { uuid },
app: { uid: appUid },
effectiveApp: { uid: appUid },
}) as unknown as Actor;
describe('the share permission family', () => {
it('names the owner, the app and the granted prefix', () => {
expect(kvSharePermission(OWNER, APP, 'workspace:abc:')).toBe(
`kv-share:${OWNER}:${APP}:workspace:abc`,
);
});
it('makes key segments components, so a deeper key is a descendant', () => {
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
const deeper = kvSharePermission(
OWNER,
APP,
'workspace:abc:messages:1',
);
// The exploder walks parents of the string being checked, so implication
// over key segments needs nothing of its own.
expect(deeper.startsWith(`${granted}:`)).toBe(true);
});
it('escapes a `:` inside a component rather than splitting on it', () => {
const permission = kvSharePermission(OWNER, 'app:odd', 'cart:');
expect(PermissionUtil.split(permission)).toEqual([
'kv-share',
OWNER,
'app:odd',
'cart',
]);
});
it('reads a withdrawn grant as covering itself and everything under it', () => {
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
expect(kvShareGrantCovers(granted, granted)).toBe(true);
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OWNER, APP, 'workspace:abc:messages:'),
),
).toBe(true);
// A sibling that merely shares a text prefix is a different region.
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OWNER, APP, 'workspace:abcdef:'),
),
).toBe(false);
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OTHER, APP, 'workspace:abc:'),
),
).toBe(false);
// Coverage never crosses namespaces: the app uid is what keeps a
// revoke inside the one the handle was minted in.
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OWNER, OTHER_APP, 'workspace:abc:messages:'),
),
).toBe(false);
});
it('never reads a withdrawn child as covering its own parent', () => {
// Revoking the deeper of two grants must not read as covering the
// shallower one — coverage only ever runs downward.
const child = kvSharePermission(OWNER, APP, 'workspace:abc:messages:');
const parent = kvSharePermission(OWNER, APP, 'workspace:abc:');
expect(kvShareGrantCovers(child, parent)).toBe(false);
});
});
describe('granted prefixes', () => {
it('always ends on the key delimiter', () => {
expect(normalizeKeyPrefix('workspace:abc')).toBe('workspace:abc:');
expect(normalizeKeyPrefix('workspace:abc:')).toBe('workspace:abc:');
expect(keyPrefixSegments('workspace:abc:')).toEqual([
'workspace',
'abc',
]);
});
it.each([
['', 'the whole namespace'],
[':', 'delimiters only'],
['workspace:*', 'a pattern'],
['workspace:a?c', 'a single-character pattern'],
// Normalizing would drop the empty segment and grant `a:b:` instead —
// a region other than the one asked for.
['workspace::abc:', 'an empty key segment'],
[':workspace:abc:', 'a leading empty segment'],
// `escape_permission_component` leaves `\\` alone while escaping `:`,
// so the stored prefix and the permission string would disagree and
// the share would silently never fire.
['work\\space:abc:', 'a backslash'],
])('refuses %s (%s)', (prefix) => {
let thrown: unknown;
try {
assertShareablePrefix(prefix);
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(HttpError);
expect((thrown as HttpError).legacyCode).toBe(
'invalid_kv_share_prefix',
);
});
it('refuses a region too deep to fit the grant column', () => {
const deep = kvSharePermission(OWNER, APP, 'a'.repeat(300));
expect(() => assertShareablePermission(deep)).toThrow(HttpError);
expect(
assertShareablePermission(
kvSharePermission(OWNER, APP, 'workspace:abc:'),
),
).toBeTypeOf('string');
});
it('keeps the trailing delimiter optional', () => {
expect(assertShareablePrefix('workspace:abc')).toBe('workspace:abc:');
expect(assertShareablePrefix('workspace:abc:')).toBe('workspace:abc:');
});
});
describe('granted namespaces', () => {
it('takes an app uid, and the app-less namespace', () => {
expect(assertShareableAppUid(APP)).toBe(APP);
expect(assertShareableAppUid('os-global')).toBe('os-global');
});
it('refuses one past the column it is stored in', () => {
// A longer one reaches the insert as a write the column cannot hold.
let thrown: unknown;
try {
assertShareableAppUid('a'.repeat(41));
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(HttpError);
expect((thrown as HttpError).statusCode).toBe(400);
});
it('takes both shapes a real app uid comes in', () => {
// Minted (`app-<uuidv4>`) and derived from an origin (`app-<uuidv5>`)
// — 40 characters either way, which is the column exactly.
const minted = `app-${uuidv4()}`;
expect(assertShareableAppUid(minted)).toBe(minted);
expect(minted).toHaveLength(40);
});
it('refuses a handle, which shares the slot but is not a namespace', () => {
expect(() => assertShareableAppUid(mintKvHandleId())).toThrow(
HttpError,
);
});
it('refuses anything else short enough to fit the column', () => {
// The slot ends up in a subject and in a permission string, so a
// namespace carrying a delimiter of either grammar is refused before
// it can be composed into one.
for (const uid of [
'os-globals',
'workspace:abc',
'app',
'app-',
'app-a:b',
'app-a#b',
'v1:someone:app-x',
'',
])
expect(() => assertShareableAppUid(uid)).toThrow(HttpError);
});
});
describe('keys inside a granted region', () => {
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
it('are named relative to the granted root', () => {
expect(
relativeToKvShareRoot(permission, 'workspace:abc:messages:1'),
).toBe('messages:1');
});
it('read as outside when they are', () => {
expect(
relativeToKvShareRoot(permission, 'workspace:abcdef:x'),
).toBeNull();
expect(relativeToKvShareRoot(permission, 'workspace:abc')).toBeNull();
});
});
describe('a delegation request', () => {
const region = kvShareManagePermission(
kvSharePermission(OWNER, APP, 'workspace:abc:'),
);
it('is the manage arm of the grant it would let an app issue', () => {
expect(region).toBe(`manage:kv-share:${OWNER}:${APP}:workspace:abc`);
expect(assertBoundedManageGrant(region)).toBe(region);
});
it.each([
['manage:kv-share', 'the family itself'],
[`manage:kv-share:${OWNER}`, 'an owner and no namespace'],
[`manage:kv-share:${OWNER}:${APP}`, 'a whole namespace'],
])('refuses %s (%s)', (permission) => {
let thrown: unknown;
try {
assertBoundedManageGrant(permission);
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(HttpError);
expect((thrown as HttpError).legacyCode).toBe(
'invalid_kv_share_prefix',
);
});
it('leaves permissions from other families alone', () => {
const unrelated = `manage:fs:${OWNER}:write`;
expect(assertBoundedManageGrant(unrelated)).toBe(unrelated);
// The read arm of this family is not a delegation, so it is not this
// check's to bound.
expect(assertBoundedManageGrant(`kv-share:${OWNER}:${APP}`)).toBeTypeOf(
'string',
);
});
});
describe('handle ids', () => {
it('are told apart from an app uid in the same slot', () => {
const handle = mintKvHandleId();
expect(isKvHandleId(handle)).toBe(true);
expect(isKvHandleId(APP)).toBe(false);
expect(isKvHandleId('kvh-')).toBe(false);
});
it('carry nothing about the owner', () => {
const handle = mintKvHandleId();
expect(handle).not.toContain(OWNER);
expect(mintKvHandleId()).not.toBe(handle);
});
it('are recoverable from the subject a row stored', () => {
const handle = mintKvHandleId();
expect(kvHandleFromSubject(`kv:${handle}:messages:*`)).toBe(handle);
expect(kvHandleFromSubject(`kv:${APP}:messages:*`)).toBeNull();
expect(kvHandleFromSubject(`fs:${handle}:write`)).toBeNull();
});
});
describe('the owner implicator', () => {
const implicator = kvShareOwnerImplicator();
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
it('answers the share family and its manage arm', () => {
expect(implicator.matches(permission)).toBe(true);
expect(implicator.matches(`manage:${permission}`)).toBe(true);
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
});
it('holds for the owner named in the permission, and nobody else', () => {
expect(
implicator.check({ actor: userActor(OWNER), permission }),
).toEqual({});
expect(
implicator.check({
actor: userActor(OWNER),
permission: `manage:${permission}`,
}),
).toEqual({});
expect(
implicator.check({ actor: userActor(OTHER), permission }),
).toBeUndefined();
});
it('reads the owner past a `manage` key segment', () => {
// Stripping every `manage:` rather than the leading one would rewrite
// the string the owner is read out of.
const nested = kvSharePermission(OWNER, APP, 'manage:secrets:');
expect(implicator.matches(nested)).toBe(true);
expect(
implicator.check({ actor: userActor(OWNER), permission: nested }),
).toEqual({});
expect(
implicator.check({ actor: userActor(OTHER), permission: nested }),
).toBeUndefined();
});
it('never answers for an app or a token acting through the owner', () => {
const appActor = {
user: { uuid: OWNER },
app: { uid: APP },
} as unknown as Actor;
expect(
implicator.check({ actor: appActor, permission }),
).toBeUndefined();
});
});
describe('the app delegate implicator', () => {
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
it('answers the share family, never its manage arm, nor other families', () => {
const implicator = kvShareAppDelegateImplicator({
userHolds: vi.fn(),
});
expect(implicator.matches(permission)).toBe(true);
expect(implicator.matches(`manage:${permission}`)).toBe(false);
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
});
it("grants when the actor's own app matches the namespace app and the user holds the grant", async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = appActor(OWNER, APP);
await expect(implicator.check({ actor, permission })).resolves.toEqual(
{},
);
expect(userHolds).toHaveBeenCalledTimes(1);
const [calledActor, calledPermission] = userHolds.mock.calls[0];
expect(calledActor.app).toBeUndefined();
expect(calledActor.effectiveApp).toBeNull();
expect(calledActor.user).toEqual({ uuid: OWNER });
expect(calledPermission).toBe(permission);
});
it('refuses when the user does not hold the grant', async () => {
const implicator = kvShareAppDelegateImplicator({
userHolds: vi.fn().mockResolvedValue(false),
});
await expect(
implicator.check({ actor: appActor(OWNER, APP), permission }),
).resolves.toBeUndefined();
});
it("refuses when the actor's app differs from the namespace app", async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
await expect(
implicator.check({
actor: appActor(OWNER, OTHER_APP),
permission,
}),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('refuses a plain user actor with no app', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
await expect(
implicator.check({ actor: userActor(OWNER), permission }),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('refuses an access-token actor', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = {
user: { uuid: OWNER },
app: { uid: APP },
accessToken: { uid: 't' },
} as unknown as Actor;
await expect(
implicator.check({ actor, permission }),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('refuses a permission with no key segments or no app', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = appActor(OWNER, APP);
await expect(
implicator.check({
actor,
permission: PermissionUtil.join('kv-share', OWNER, APP),
}),
).resolves.toBeUndefined();
await expect(
implicator.check({
actor,
permission: PermissionUtil.join('kv-share', OWNER),
}),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
});
-327
View File
@@ -1,327 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import {
type Actor,
isAppActor,
isPlainUserActor,
userRelatedActor,
} from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { KV_SHARE_HANDLE_WIDTHS } from '../../stores/events/columnWidths.js';
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
import {
MANAGE_PERM_PREFIX,
PERMISSION_MAX_LEN,
} from '../permission/consts.js';
import type { PermissionImplicator } from '../permission/permissionUtil.js';
import { PermissionUtil } from '../permission/permissionUtil.js';
import {
isKvHandleId,
KV_HANDLE_PREFIX,
KV_KEY_SEPARATOR,
} from './subjects.js';
/**
* Letting one user watch a region of another's key-value namespace.
*
* The grant is an ordinary user-to-user permission, which is the whole point:
* `manage:` answers "may I delegate this", the exploder's prefix implication
* makes a grant on `workspace:abc:` answer every key beneath it, and revocation
* settling and the delivery re-check key off the stored string exactly as they
* do for a shared folder. Putting the ACL on the key-value item instead would
* take part in none of that, and would tax every write to the item besides.
*
* The permission names the owner, so its components are the three things a
* shared region is:
*
* kv-share:<ownerUserUuid>:<appUid>:<keySegment>...
*
* Key segments are components rather than one escaped blob because that is what
* makes prefix implication work without a rule of its own: the check a deep key
* runs is a descendant of the grant string, and the existing parent walk finds
* it. The grantee never sees this string — the handle is the name on the wire.
*/
/** Root of the cross-user key-value share namespace. */
export const KV_SHARE_PERMISSION_PREFIX = 'kv-share';
/** Width of the `app_uid` column the handle row stores its namespace in. */
export const KV_SHARE_APP_UID_MAX_LENGTH = KV_SHARE_HANDLE_WIDTHS.appUid.max;
export const mintKvHandleId = (): string =>
`${KV_HANDLE_PREFIX}${randomUUID()}`;
/** The key segments a prefix contributes to a permission string. */
export const keyPrefixSegments = (keyPrefix: string): string[] =>
keyPrefix.split(KV_KEY_SEPARATOR).filter((segment) => segment.length > 0);
/**
* A granted prefix as it is stored: always ending on the key delimiter, so
* `keyPrefix + relative` is a whole key and the region never accidentally
* includes the key that names it.
*/
export const normalizeKeyPrefix = (keyPrefix: string): string => {
const segments = keyPrefixSegments(keyPrefix);
return segments.length === 0
? ''
: `${segments.join(KV_KEY_SEPARATOR)}${KV_KEY_SEPARATOR}`;
};
export const kvSharePermission = (
ownerUserUuid: string,
appUid: string,
keyPrefix: string,
): string =>
PermissionUtil.join(
KV_SHARE_PERMISSION_PREFIX,
ownerUserUuid,
appUid,
...keyPrefixSegments(keyPrefix),
);
export const isKvSharePermission = (permission: string): boolean =>
permission === KV_SHARE_PERMISSION_PREFIX ||
permission.startsWith(`${KV_SHARE_PERMISSION_PREFIX}:`);
/**
* Whether withdrawing `revoked` takes `held` with it. Prefix implication read
* backwards: a grant answers every check at or beneath it, so withdrawing it
* puts every one of those in question.
*/
export const kvShareGrantCovers = (revoked: string, held: string): boolean =>
held === revoked || held.startsWith(`${revoked}:`);
/** The granted root a share grant names, read back as a key prefix. */
export const kvShareGrantPrefix = (permission: string): string =>
normalizeKeyPrefix(
PermissionUtil.split(permission).slice(3).join(KV_KEY_SEPARATOR),
);
/**
* A key inside a granted region, named the way its holder addresses it, or
* `null` for one outside. The handle is the granted root, so everything the
* holder is shown is relative to it — an absolute key would name a namespace
* they cannot address and were never told about.
*/
export const relativeToKvShareRoot = (
permission: string,
key: string,
): string | null => {
const prefix = kvShareGrantPrefix(permission);
if (!prefix || !key.startsWith(prefix)) return null;
return key.slice(prefix.length);
};
// -- Minting input ----------------------------------------------------
export const invalidPrefix = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'invalid_kv_share_prefix' });
export const invalidAppUid = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'bad_request' });
/**
* The region a handle may be minted on. A pattern is refused because a handle
* names a region rather than selecting within one, and the namespace root is
* refused because a handle over everything the app holds is not a bounded
* capability — it is the app's data, and sharing that is a different decision
* with a different consent.
*/
export const assertShareablePrefix = (keyPrefix: unknown): string => {
if (typeof keyPrefix !== 'string')
throw invalidPrefix('`prefix` must be a string');
if (keyPrefix.includes('*') || keyPrefix.includes('?'))
throw invalidPrefix('A share prefix is a key prefix, not a pattern');
// `escape_permission_component` escapes `:` as `\C` but leaves `\` alone,
// so a prefix carrying one desynchronizes the stored `key_prefix` from the
// permission string and the share silently never fires.
if (keyPrefix.includes('\\'))
throw invalidPrefix('A share prefix may not contain a backslash');
// Normalizing drops empty segments, so `a::b:` would silently become a
// grant on `a:b:` — a region other than the one asked for. Refused rather
// than rewritten; only the trailing delimiter is optional.
const written = keyPrefix.split(KV_KEY_SEPARATOR);
if (written[written.length - 1] === '') written.pop();
if (written.some((segment) => segment === ''))
throw invalidPrefix('A share prefix may not have an empty key segment');
const normalized = normalizeKeyPrefix(keyPrefix);
if (normalized === '')
throw invalidPrefix('A share prefix may not be the whole namespace');
return normalized;
};
/**
* The shape every app uid takes, minted or derived from an origin. Narrow on
* purpose: `:` and `#` delimit the anchor token and permission grammars, so
* excluding them matters as much as excluding the handle prefix does.
*/
const APP_UID_SHAPE = /^app-[A-Za-z0-9_-]{1,64}$/;
/** Fixed namespaces with no app of their own. */
const FIXED_KV_NAMESPACES: readonly string[] = [KV_GLOBAL_APP_KEY];
/**
* The namespace a handle may be minted over: the caller's own app slot, or the
* app-less one. Bounded by the column that stores it, shaped like a real app
* uid or one of the platform's fixed namespaces, and never handle-shaped, since
* the two share the app slot of a `kv:` subject.
*/
export const assertShareableAppUid = (appUid: string): string => {
if (appUid.length > KV_SHARE_APP_UID_MAX_LENGTH)
throw invalidAppUid(
`\`appUid\` may not exceed ${KV_SHARE_APP_UID_MAX_LENGTH} characters`,
);
if (isKvHandleId(appUid))
throw invalidAppUid('A share handle does not name a namespace');
if (!APP_UID_SHAPE.test(appUid) && !FIXED_KV_NAMESPACES.includes(appUid))
throw invalidAppUid('`appUid` does not name a namespace');
return appUid;
};
/**
* The grant a mint would issue, refused when it would not fit the column every
* permission table declares. Key prefixes run to a kilobyte and permissions do
* not, so the real bound on how deep a region may be is this one.
*/
export const assertShareablePermission = (permission: string): string => {
if (permission.length > PERMISSION_MAX_LEN)
throw invalidPrefix(
`A share prefix must leave the grant under ${PERMISSION_MAX_LEN} characters`,
);
return permission;
};
// -- Delegation -------------------------------------------------------
/** The delegation arm of a share grant: authority to hand the region out. */
export const kvShareManagePermission = (permission: string): string =>
PermissionUtil.join(
MANAGE_PERM_PREFIX,
...PermissionUtil.split(permission),
);
export const KV_SHARE_MANAGE_PREFIX = `${MANAGE_PERM_PREFIX}:${KV_SHARE_PERMISSION_PREFIX}`;
export const isKvShareManagePermission = (permission: string): boolean =>
permission === KV_SHARE_MANAGE_PREFIX ||
permission.startsWith(`${KV_SHARE_MANAGE_PREFIX}:`);
/**
* A delegation must name a subtree. Consent on the namespace root reads as "let
* this app hand out anything it has stored for you", which is not a bounded
* capability and so is not something a prompt can put to a user.
*/
export const assertBoundedManageGrant = (permission: string): string => {
if (!isKvShareManagePermission(permission)) return permission;
const [, , owner, appUid, ...segments] = PermissionUtil.split(permission);
if (!owner || !appUid || segments.length === 0)
throw invalidPrefix(
'A share delegation must name a region, not the whole namespace',
);
return permission;
};
/**
* The unbounded `manage:` grant a bounded delegation's own permission descends
* from. The consent surface refuses to ever write this row, but
* `canManagePermission` walks ancestors to decide "may I delegate this", so if
* it exists by any other path it would silently authorize a mint anywhere in
* the namespace — this is what lets the mint path notice it regardless.
*/
export const kvShareManageNamespaceRoot = (permission: string): string => {
const [, owner, appUid] = PermissionUtil.split(permission);
return PermissionUtil.join(
MANAGE_PERM_PREFIX,
KV_SHARE_PERMISSION_PREFIX,
owner,
appUid,
);
};
// -- Permission rules -------------------------------------------------
/**
* The permission a `manage:` arm delegates over. Only the leading component is
* stripped: a key segment may itself be `manage`, and taking those out would
* change which string the owner is read from.
*/
const withoutManageArm = (permission: string): string =>
permission.startsWith(`${MANAGE_PERM_PREFIX}:`)
? permission.slice(MANAGE_PERM_PREFIX.length + 1)
: permission;
/**
* Owning the namespace is holding every share grant over it, and being able to
* issue them. Without this the owner cannot mint a handle on their own data:
* `grantUserUserPermission` asks `canManagePermission` first, and there is no
* row anywhere saying a user may manage what is already theirs.
*
* Restricted to plain user actors, as the filesystem's `is-owner` is. An app
* disposing of a region of its user's namespace is delegation, which is the
* `manage:` grant's job rather than this one's.
*/
export const kvShareOwnerImplicator = (): PermissionImplicator => ({
id: 'kv-share-is-owner',
shortcut: true,
matches: (permission: string): boolean =>
isKvSharePermission(withoutManageArm(permission)),
check: ({ actor, permission }): unknown => {
if (!isPlainUserActor(actor)) return undefined;
const uuid = actor.user?.uuid;
if (!uuid) return undefined;
const owner = PermissionUtil.split(withoutManageArm(permission))[1];
return owner === uuid ? {} : undefined;
},
});
/**
* Letting an app-under-user actor exercise a `kv-share:` grant its user holds,
* bounded to the app the grant's namespace names.
*
* Only the read arm is matched — never `manage:kv-share:…` — because answering
* the manage arm here would read as the unbounded namespace-root delegation and
* break minting for everyone (see `kvShareManageNamespaceRoot`). The
* namespace-app check keeps one app from reading a region shared with another
* app of the same user.
*/
export const kvShareAppDelegateImplicator = (deps: {
userHolds: (actor: Actor, permission: string) => Promise<boolean>;
}): PermissionImplicator => ({
id: 'kv-share-app-delegate',
shortcut: true,
matches: (permission: string): boolean => isKvSharePermission(permission),
check: async ({ actor, permission }): Promise<unknown> => {
if (!isAppActor(actor)) return undefined;
const app = actor.effectiveApp;
if (!app?.uid) return undefined;
const [, owner, namespaceApp, ...segments] =
PermissionUtil.split(permission);
if (!owner || !namespaceApp || segments.length === 0) return undefined;
if (namespaceApp !== app.uid) return undefined;
return (await deps.userHolds(userRelatedActor(actor), permission))
? {}
: undefined;
},
});
+24 -1
View File
@@ -43,10 +43,20 @@ const PROJECTED_KEYS = [
'subject',
'ts',
'uid',
'viaWorker',
];
/** A KV event names a key where an FS event names a node. */
const PROJECTED_KV_KEYS = ['id', 'key', 'op', 'self', 'seq', 'subject', 'ts'];
const PROJECTED_KV_KEYS = [
'id',
'key',
'op',
'self',
'seq',
'subject',
'ts',
'viaWorker',
];
const entry = {
id: 42,
@@ -195,11 +205,24 @@ describe('the kv subject', () => {
op: 'set',
key: 'cart:items',
self: true,
viaWorker: false,
ts: 1_700_000_002,
seq: 0,
});
});
it('says whether server-side code made the change, on kv and fs alike', () => {
expect(
subject().project({ ...kvDelivery, viaWorker: true }),
).toMatchObject({ viaWorker: true });
expect(
lookupFsSubject('fs.write.file')!.project({
...delivery,
viaWorker: true,
}),
).toMatchObject({ viaWorker: true });
});
it('names the namespace app, never the namespace itself', () => {
const projected = subject().project(kvDelivery) as Record<
string,
+15
View File
@@ -64,6 +64,11 @@ export interface ProjectedFsEvent extends ProjectedEventBase {
path: string;
/** Present only on a `move` a subscribed folder lost the node from. */
from?: string;
/**
* The change was made by server-side code running as the account (a worker,
* or an events handler) rather than by the account directly.
*/
viaWorker: boolean;
}
/**
@@ -80,6 +85,8 @@ export interface ProjectedKvEvent extends ProjectedEventBase {
* value is small enough to inline; an `expire` never carries one.
*/
value?: unknown;
/** See {@link ProjectedFsEvent.viaWorker}. */
viaWorker: boolean;
}
/**
@@ -142,6 +149,8 @@ export interface EventContextBase {
ts: number;
/** Handler runs behind the change. Never projected. */
handlerDepth?: number;
/** Made by server-side code running as the account; see `viaWorkerOf`. */
viaWorker?: boolean;
}
/** What dispatch knows about one committed filesystem change. */
@@ -284,6 +293,7 @@ const fsProject =
path: delivery.entry.path,
...(delivery.movedFrom ? { from: delivery.movedFrom.path } : {}),
self: delivery.self,
viaWorker: delivery.viaWorker === true,
ts: delivery.ts,
seq: delivery.seq,
});
@@ -309,6 +319,7 @@ const kvProject = (delivery: KvDeliveryContext): ProjectedKvEvent => ({
key: delivery.kvKey,
...(delivery.value !== undefined ? { value: delivery.value } : {}),
self: delivery.self,
viaWorker: delivery.viaWorker === true,
ts: delivery.ts,
seq: delivery.seq,
});
@@ -429,6 +440,10 @@ export const UNPUBLISHED_INTERNAL_EVENTS = [
event: 'kv.flushed',
reason: 'No subject addresses "the whole namespace went": the grammar anchors on a key or a prefix, and fanning a flush out per key would report keys a truncated enumeration never saw.',
},
{
event: 'kv.share.retired',
reason: 'A share handle stopping is not a change to any key; it ends the subscriptions made through the handle rather than being delivered to them.',
},
] as const satisfies readonly UnpublishedInternalEvent[];
// Every `fs.*`/`kv.*`/`notif.*` bus key needs a decision. A new key that is in
+4
View File
@@ -194,6 +194,10 @@ export const notifMatchOn = (
audience: NotificationAudience | string,
): string => `${ref}${NOTIF_MATCH_SEPARATOR}${audience}`;
/** Whether a notif filter spans every app: a session's generic slice. */
export const notifMatchSpansEveryApp = (match: string | null): boolean =>
match?.startsWith(`*${NOTIF_MATCH_SEPARATOR}`) === true;
/** Which family a stored row belongs to, without re-parsing its subject. */
export const isKvToken = (token: string): boolean =>
token.startsWith(KV_TOKEN_PREFIX);
@@ -425,6 +425,7 @@ describe('the call an owed delivery makes', () => {
'subject',
'ts',
'uid',
'viaWorker',
]);
expect(call.body.event).toMatchObject({
path: `${anchor}/note.txt`,
+3
View File
@@ -41,6 +41,7 @@ import {
isAccountContext,
isAppActor,
isPlainUserActor,
viaWorkerOf,
} from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
@@ -4794,6 +4795,7 @@ export class FSService extends PuterService {
const actor = Context.get('actor') as Actor | undefined;
const actingUserId = actor?.user?.id;
const handlerDepth = handlerDepthOf(actor);
const viaWorker = viaWorkerOf(actor);
const movedFromChain = movedFrom && {
path: movedFrom.path,
ancestors: () => fsEntry.getAncestorChain(movedFrom.path),
@@ -4807,6 +4809,7 @@ export class FSService extends PuterService {
events.dispatchFs(key, entry, {
actingUserId,
handlerDepth,
viaWorker,
ancestors: () => ancestors(entry),
...(movedFromChain
? { movedFrom: movedFromChain }
+5
View File
@@ -33,6 +33,7 @@ import { EventForwardService } from './events/EventForwardService';
import { EventsService } from './events/EventsService';
import { AppFeedbackService } from './feedback/AppFeedbackService';
import { FSService } from './fs/FSService';
import { KvShareService } from './kvShare/KvShareService';
import { ServerHealthService } from './health/ServerHealthService';
import { PuterHomepageService } from './homepage/PuterHomepageService';
import { LocalWorkerService } from './localworker/LocalWorkerService';
@@ -74,6 +75,7 @@ declare module './types' {
suggestedApps: SuggestedAppsService;
app: AppService;
socket: SocketService;
kvShare: KvShareService;
events: EventsService;
eventForward: EventForwardService;
notification: NotificationService;
@@ -134,6 +136,9 @@ export const puterServices = {
// Reaches `auth` and `permission` at call time only.
app: AppService,
socket: SocketService,
// Registers the share-grant implicators with `permission`, and reaches
// `permission`, `metering` and `notification` at call time only.
kvShare: KvShareService,
// Delivers through `socket` and resolves paths through `fs`, so it follows
// both; `fs` reaches back for dispatch at call time only.
events: EventsService,
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,365 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Withdrawing a share against a replica that lags the primary, and the MySQL
* matching rules a share's grants and rows meet. Runs on SQLite, and also on
* MySQL when `PUTER_TEST_MYSQL_PORT` names one.
*/
import { fileURLToPath } from 'node:url';
import mysql from 'mysql2/promise';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import { EVENTS_KV_HANDLE_LIMIT } from '../../controllers/events/limits.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import {
kvShareGrantAtEveryLevel,
kvSharePermission,
type KvShareAccess,
} from './kvShares.js';
const BOOT_TIMEOUT_MS = 300_000;
const GLOBAL = 'os-global';
const MYSQL = {
host: '127.0.0.1',
port: Number(process.env.PUTER_TEST_MYSQL_PORT ?? 3307),
user: 'root',
password: 'puter',
};
const MIGRATIONS = fileURLToPath(
new URL('../../clients/database/migrations/mysql', import.meta.url),
);
const ENGINES = [
'sqlite',
...(process.env.PUTER_TEST_MYSQL_PORT ? ['mysql'] : []),
] as const;
interface TestUser {
actor: Actor;
username: string;
id: number;
uuid: string;
}
describe.each(ENGINES)('key-value shares on %s', (engine) => {
const database = `kv_share_${Date.now().toString(36)}`;
let env: PuterTestEnv;
let owner: TestUser;
let guest: TestUser;
const db = () => env.server.clients.db;
const shares = () => env.server.services.kvShare;
const permissions = () => env.server.services.permission;
const handles = () => env.server.stores.kvShareHandle;
const flat = () => env.server.stores.permission;
const userFor = async (username: string): Promise<TestUser> => {
const row = await env.server.stores.user.getByUsername(username);
return {
actor: makeActor({ user: row as never }),
username,
id: row!.id,
uuid: row!.uuid as string,
};
};
const region = (label: string) => `${label}-${uuidv4().slice(0, 8)}:`;
const share = (prefix: string, access: KvShareAccess) =>
shares().mintKvHandle(
owner.actor,
{ prefix, recipient: { username: guest.username }, access },
{ defaultAccess: 'read' },
);
const grantRows = async (prefix: string) =>
(
await db().pread(
'SELECT `permission` FROM `user_to_user_permissions` ' +
'WHERE `holder_user_id` = ? AND `permission` IN (?, ?, ?)',
[
guest.id,
...kvShareGrantAtEveryLevel(
kvSharePermission(owner.uuid, GLOBAL, prefix),
),
],
)
).map((row) => String(row.permission));
/**
* From here until the returned function runs, replica reads see the grant
* table as it is now and primary reads see it as it really is.
*/
const lagReplica = async () => {
await db().write('DROP TABLE IF EXISTS `u2u_lagging`');
await db().write(
'CREATE TABLE `u2u_lagging` AS SELECT * FROM `user_to_user_permissions`',
);
const read = db().read.bind(db());
const replica = vi
.spyOn(db(), 'read')
.mockImplementation((query, params) =>
read(
query.replaceAll(
'`user_to_user_permissions`',
'`u2u_lagging`',
),
params,
),
);
// SQLite's own primary read goes through `read`, so route it past the
// lagging view explicitly.
const primary = vi
.spyOn(db(), 'pread')
.mockImplementation((query, params) => read(query, params));
return async () => {
replica.mockRestore();
primary.mockRestore();
await db().write('DROP TABLE `u2u_lagging`');
};
};
/** No row, no flat entry, and no write reaching a later nested share. */
const expectWithdrawn = async (prefix: string) => {
expect(await grantRows(prefix)).toEqual([]);
expect(
await flat().getFlatUserPerms(
guest.id,
kvShareGrantAtEveryLevel(
kvSharePermission(owner.uuid, GLOBAL, prefix),
),
),
).toEqual([]);
const nested = await share(`${prefix}nested:`, 'watch');
await expect(
runWithContext({ actor: guest.actor }, () =>
env.server.drivers.kvStore.set({
key: 'k',
value: 1,
optConfig: { handle: nested.handle },
}),
),
).rejects.toMatchObject({ legacyCode: 'kv_handle_access_denied' });
};
beforeAll(async () => {
const flags = {
events: { enabled: true, kvHandles: true },
kv: { shareHandles: true },
unlimitedMetering: true,
};
if (engine === 'mysql') {
const admin = await mysql.createConnection(MYSQL);
await admin.query(`CREATE DATABASE \`${database}\``);
await admin.end();
env = await setupPuterTestEnv({
...flags,
database: {
engine: 'mysql',
...MYSQL,
database,
migrationPaths: [MIGRATIONS],
},
} as never);
} else {
env = await setupPuterTestEnv(flags as IConfig);
}
owner = await userFor(env.users.user.username);
guest = await userFor(env.users.other.username);
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await env?.shutdown();
if (engine !== 'mysql') return;
const admin = await mysql.createConnection(MYSQL);
await admin.query(`DROP DATABASE IF EXISTS \`${database}\``);
await admin.end();
});
beforeEach(async () => {
for (const user of [owner, guest])
await env.server.clients.redis.del(
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user.id}`,
);
});
describe('an owner revoke while the replica lags', () => {
it('withdraws a share minted inside the lag', async () => {
const prefix = region('lag-mint');
const thaw = await lagReplica();
try {
const { handle } = await share(prefix, 'write');
await shares().removeKvShare(owner.actor, handle);
} finally {
await thaw();
}
await expectWithdrawn(prefix);
});
it('withdraws the write grant of a read share raised inside the lag', async () => {
const prefix = region('lag-raise');
const { handle } = await share(prefix, 'read');
const thaw = await lagReplica();
try {
await share(prefix, 'write');
await shares().removeKvShare(owner.actor, handle);
} finally {
await thaw();
}
await expectWithdrawn(prefix);
});
it('clears a flat entry under the region that no row backs', async () => {
const prefix = region('orphan');
const { handle } = await share(prefix, 'read');
const orphan = kvSharePermission(
owner.uuid,
GLOBAL,
`${prefix}deep:`,
'write',
);
await flat().setFlatUserPerm(guest.id, orphan, {
permission: orphan,
issuer_user_id: owner.id,
deleted: false,
});
await shares().removeKvShare(owner.actor, handle);
expect(await flat().getFlatUserPerms(guest.id, [orphan])).toEqual(
[],
);
});
});
describe('matching a share`s prefix', () => {
it('finds a live share by its exact bytes only', async () => {
const prefix = region('exact');
await share(prefix, 'read');
const input = {
ownerUserId: owner.id,
granteeUserId: guest.id,
appUid: GLOBAL,
keyPrefix: prefix,
};
expect(await handles().findLive(input)).not.toBeNull();
expect(
await handles().findLive({
...input,
keyPrefix: prefix.toUpperCase(),
}),
).toBeNull();
});
it.skipIf(engine !== 'mysql')(
'uses no operator MySQL warns is deprecated, whatever the connection charset',
async () => {
const pread = vi.spyOn(db(), 'pread');
let call: unknown[] | undefined;
try {
await handles().findLive({
ownerUserId: owner.id,
granteeUserId: guest.id,
appUid: GLOBAL,
keyPrefix: region('warn'),
});
call = pread.mock.calls.find(([query]) =>
query.includes('`key_prefix`'),
);
} finally {
pread.mockRestore();
}
expect(call).toBeDefined();
// The text protocol, so the statement is parsed (and warned
// about) as it runs.
const connection = await mysql.createConnection({
...MYSQL,
database,
charset: 'LATIN1_SWEDISH_CI',
});
try {
await connection.query(
String(call![0]),
call![1] as never[],
);
const [warnings] = await connection.query('SHOW WARNINGS');
expect(warnings).toEqual([]);
} finally {
await connection.end();
}
},
);
});
describe('revoking a grant the database spells differently', () => {
it.skipIf(engine !== 'mysql')(
'clears the flat entry under the spelling it removed',
async () => {
const tag = uuidv4().slice(0, 8);
const stored = kvSharePermission(
owner.uuid,
GLOBAL,
`Notes-${tag}:`,
'read',
);
const asked = kvSharePermission(
owner.uuid,
GLOBAL,
`notes-${tag}:`,
'read',
);
await permissions().grantUserUserPermission(
owner.actor,
guest.username,
stored,
);
expect(await permissions().check(guest.actor, stored)).toBe(
true,
);
const removed = await permissions().revokeUserUserPermissions(
owner.actor,
guest.username,
[asked],
);
expect(removed).toEqual([stored]);
expect(
await flat().getFlatUserPerms(guest.id, [stored]),
).toEqual([]);
expect(await permissions().check(guest.actor, stored)).toBe(
false,
);
},
);
});
});
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,549 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* What the recipient of a key-value share sees and controls: the notification
* that tells their app a share arrived, kept off their desktop, and the share
* blocks that refuse one.
*/
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import {
EVENTS_COALESCE_WINDOW_MS,
EVENTS_KV_HANDLE_LIMIT,
KV_SHARE_NOTICE_LIMIT,
} from '../../controllers/events/limits.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
import type { IConfig } from '../../types.js';
import type { DeliveryEnvelope } from '../events/EventsService.js';
import type { KvShareAccess } from './kvShares.js';
const BOOT_TIMEOUT_MS = 120_000;
interface TestUser {
actor: Actor;
username: string;
id: number;
uuid: string;
}
/** One server, its two users, and a fresh app the owner can share data of. */
const bootEnv = (config: Record<string, unknown>) => {
const state = {
env: null as unknown as PuterTestEnv,
owner: null as unknown as TestUser,
guest: null as unknown as TestUser,
};
const userFor = async (username: string): Promise<TestUser> => {
const row = await state.env.server.stores.user.getByUsername(username);
return {
actor: makeActor({ user: row as never }),
username,
id: row!.id,
uuid: row!.uuid as string,
};
};
beforeAll(async () => {
state.env = await setupPuterTestEnv({
events: { enabled: true, kvHandles: true },
kv: { shareHandles: true },
unlimitedMetering: true,
...config,
} as IConfig);
state.owner = await userFor(state.env.users.user.username);
state.guest = await userFor(state.env.users.other.username);
}, BOOT_TIMEOUT_MS);
afterAll(async () => {
await state.env?.shutdown();
});
beforeEach(async () => {
for (const user of [state.owner, state.guest])
await state.env.server.clients.redis.del(
`rate:${EVENTS_KV_HANDLE_LIMIT.scope}:${user.id}`,
);
await state.env.server.clients.redis.del(
`rate:${KV_SHARE_NOTICE_LIMIT.scope}:${state.owner.id}:${state.guest.id}`,
);
});
const makeApp = async () => {
const name = `kv-recipient-${uuidv4().slice(0, 8)}`;
return state.env.server.stores.app.create(
{
name,
title: 'Recipient App',
index_url: `https://${name}.example.test/index.html`,
},
{ ownerUserId: state.owner.id },
);
};
const share = (
prefix: string,
access: KvShareAccess,
opts: { appUid?: string; label?: unknown } = {},
) =>
state.env.server.services.kvShare.mintKvHandle(
state.owner.actor,
{
prefix,
recipient: { username: state.guest.username },
access,
...(opts.appUid ? { appUid: opts.appUid } : {}),
...(opts.label !== undefined ? { label: opts.label } : {}),
},
{ defaultAccess: 'read' },
);
/** The holder's share notifications naming one handle. */
const notificationsFor = async (handle: string) =>
(
await state.env.server.clients.db.pread(
'SELECT `uid`, `app_uid`, `audience`, `value` FROM `notification` ' +
"WHERE `user_id` = ? AND `type` = 'kv.share.received'",
[state.guest.id],
)
)
.map((row) => ({
uid: String(row.uid),
appUid: row.app_uid,
audience: row.audience,
value: (typeof row.value === 'string'
? JSON.parse(row.value)
: row.value) as Record<string, unknown>,
}))
.filter((row) => row.value.handle === handle);
return { state, makeApp, share, notificationsFor };
};
const region = (label: string) => `${label}-${uuidv4().slice(0, 8)}:`;
describe('the share notification', () => {
const { state, makeApp, share, notificationsFor } = bootEnv({});
it('reaches the holder once, in the app the data belongs to, without the owner', async () => {
const app = await makeApp();
const prefix = region('notify');
const shared = await share(prefix, 'write', {
appUid: app.uid,
label: 'Groceries',
});
const rows = await notificationsFor(shared.handle);
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
appUid: app.uid,
audience: 'app-user',
});
expect(rows[0].value).toEqual({
type: 'kv.share.received',
handle: shared.handle,
access: 'write',
label: 'Groceries',
createdAt: shared.createdAt,
});
const wire = JSON.stringify(rows);
expect(wire).not.toContain(state.owner.uuid);
expect(wire).not.toContain(state.owner.username);
expect(wire).not.toContain(prefix);
});
it('is not sent again for a re-share, a level change or a label change', async () => {
const app = await makeApp();
const prefix = region('again');
const { handle } = await share(prefix, 'read', { appUid: app.uid });
await share(prefix, 'read', { appUid: app.uid });
await share(prefix, 'write', { appUid: app.uid });
await share(prefix, 'write', { appUid: app.uid, label: 'Renamed' });
expect(await notificationsFor(handle)).toHaveLength(1);
});
it('is sent at most five times an hour per owner and holder, and the share is made anyway', async () => {
const app = await makeApp();
const prefix = region('churn');
const handles: string[] = [];
for (let i = 0; i < KV_SHARE_NOTICE_LIMIT.limit + 2; i++) {
const { handle } = await share(prefix, 'read', { appUid: app.uid });
handles.push(handle);
await state.env.server.services.kvShare.removeKvShare(
state.owner.actor,
handle,
);
}
expect(new Set(handles).size).toBe(handles.length);
const counts = await Promise.all(
handles.map(
async (handle) => (await notificationsFor(handle)).length,
),
);
expect(counts.reduce((sum, n) => sum + n, 0)).toBe(
KV_SHARE_NOTICE_LIMIT.limit,
);
expect(counts.slice(KV_SHARE_NOTICE_LIMIT.limit)).toEqual([0, 0]);
});
it('is not sent for a share with no app', async () => {
const { handle } = await share(region('global'), 'read');
expect(await notificationsFor(handle)).toEqual([]);
});
it('never reaches the desktop wire, and the mint stands when it fails', async () => {
const app = await makeApp();
const emit = vi.spyOn(state.env.server.clients.event, 'emit');
try {
const { handle } = await share(region('wire'), 'read', {
appUid: app.uid,
});
const [row] = await notificationsFor(handle);
const pushed = emit.mock.calls.filter(
([key, data]) =>
key === 'outer.gui.notif.message' &&
JSON.stringify(data).includes(row.uid),
);
expect(pushed).toEqual([]);
} finally {
emit.mockRestore();
}
const notify = vi
.spyOn(state.env.server.services.notification, 'notify')
.mockRejectedValueOnce(new Error('mailbox down'));
try {
await expect(
share(region('failing'), 'read', { appUid: app.uid }),
).resolves.toMatchObject({ access: 'read' });
} finally {
notify.mockRestore();
}
});
it('stays out of the whole-mailbox listing, but not out of the app`s slice', async () => {
const app = await makeApp();
const { handle } = await share(region('select'), 'read', {
appUid: app.uid,
});
const [row] = await notificationsFor(handle);
const select = (actor: Actor, args: Record<string, unknown> = {}) =>
runWithContext({ actor }, () =>
state.env.server.drivers.notifications.select(args),
) as Promise<Array<{ uid: unknown }>>;
const uids = (rows: Array<{ uid: unknown }>) =>
rows.map((one) => one.uid);
expect(uids(await select(state.guest.actor))).not.toContain(row.uid);
expect(
uids(
await select(state.guest.actor, { subject: 'notif:app-user' }),
),
).not.toContain(row.uid);
expect(
uids(
await select(state.guest.actor, {
subject: `notif:${app.uid}:app-user`,
}),
),
).toContain(row.uid);
const appActor = makeActor({
user: state.guest.actor.user as never,
app: { uid: app.uid, id: app.id },
});
expect(uids(await select(appActor))).toContain(row.uid);
});
it('is left out of the unread replay a desktop gets on connecting', async () => {
const app = await makeApp();
const { handle } = await share(region('replay'), 'read', {
appUid: app.uid,
});
const [row] = await notificationsFor(handle);
const emit = vi.spyOn(state.env.server.clients.event, 'emit');
try {
state.env.server.clients.event.emit(
'web.socket.user-connected',
{ user: { id: state.guest.id } } as never,
{},
);
// The replay is debounced by two seconds.
await new Promise((resolve) => setTimeout(resolve, 2_300));
const replayed = emit.mock.calls.filter(
([key, data]) =>
key === 'outer.gui.notif.unreads' &&
JSON.stringify(data).includes(row.uid),
);
expect(replayed).toEqual([]);
} finally {
emit.mockRestore();
}
});
});
describe('the share notification with notification dispatch on', () => {
const { state, makeApp, share, notificationsFor } = bootEnv({
events: {
enabled: true,
kvHandles: true,
notificationsFoldIn: true,
},
});
it('reaches the app`s notif:app-user subscription and never the desktop', async () => {
const app = await makeApp();
const delivered: DeliveryEnvelope[] = [];
state.env.server.services.events.onDelivered = (envelope) =>
delivered.push(envelope);
const appActor = makeActor({
user: state.guest.actor.user as never,
app: { uid: app.uid, id: app.id },
});
await state.env.server.services.events.subscribe(
appActor,
'kv-recipient-socket',
{ subject: 'notif:app-user' },
);
const send = vi.spyOn(state.env.server.services.socket, 'send');
try {
const { handle } = await share(region('folded'), 'read', {
appUid: app.uid,
});
const [row] = await notificationsFor(handle);
await vi.waitFor(
() =>
expect(
delivered.some(
(one) =>
(one.event as { uid?: string }).uid === row.uid,
),
).toBe(true),
{ timeout: EVENTS_COALESCE_WINDOW_MS * 12, interval: 25 },
);
const desktop = send.mock.calls.filter(
([, key, data]) =>
key === 'notif.message' &&
JSON.stringify(data).includes(row.uid),
);
expect(desktop).toEqual([]);
} finally {
send.mockRestore();
await state.env.server.services.events.reapSocket(
state.guest.id,
'kv-recipient-socket',
);
}
});
it('stays out of the desktop`s own notif:app-user feed, live and fetched', async () => {
const events = state.env.server.services.events;
const app = await makeApp();
const delivered: DeliveryEnvelope[] = [];
events.onDelivered = (envelope) => delivered.push(envelope);
const appActor = makeActor({
user: state.guest.actor.user as never,
app: { uid: app.uid, id: app.id },
});
const { sub: desktopSub } = await events.subscribe(
state.guest.actor,
'kv-recipient-desktop',
{ subject: 'notif:app-user' },
);
const { sub: appSub } = await events.subscribe(
appActor,
'kv-recipient-app',
{ subject: 'notif:app-user' },
);
const deliveredTo = (subId: string, uid: string) =>
delivered.some(
(one) =>
one.subId === subId &&
(one.event as { uid?: string }).uid === uid,
);
// Deliveries on one subject within the coalesce window collapse to the
// last, so each notification is waited out before the next is sent.
const waitFor = (check: () => boolean) =>
vi.waitFor(() => expect(check()).toBe(true), {
timeout: EVENTS_COALESCE_WINDOW_MS * 12,
interval: 25,
});
try {
const { handle } = await share(region('desktop'), 'read', {
appUid: app.uid,
});
const [row] = await notificationsFor(handle);
await waitFor(() => deliveredTo(appSub.subId, row.uid));
// Another app-user row about the same app still reaches the desktop.
const ended = (await state.env.server.services.notification.notify(
[state.guest.id],
{ title: 'Subscription ended' },
{ type: 'app.events.ended', appUid: app.uid },
))!;
await waitFor(() => deliveredTo(desktopSub.subId, ended));
expect(deliveredTo(desktopSub.subId, row.uid)).toBe(false);
const fetched = async (subject: string) =>
(
await events.fetchMissed(state.guest.actor, { subject })
).items.map((item) => item.uid);
const generic = await fetched('notif:app-user');
expect(generic).toContain(ended);
expect(generic).not.toContain(row.uid);
expect(await fetched(`notif:${app.uid}:app-user`)).toContain(
row.uid,
);
} finally {
for (const socketId of ['kv-recipient-desktop', 'kv-recipient-app'])
await events.reapSocket(state.guest.id, socketId);
}
});
});
describe('recipient share blocks', () => {
const { state, makeApp, share, notificationsFor } = bootEnv({});
it('refuse a share from a sender the recipient blocked', async () => {
await state.env.server.stores.userBlock.create(
state.guest.id,
state.owner.id,
);
try {
const app = await makeApp();
await expect(
share(region('blocked'), 'read', { appUid: app.uid }),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'recipient_not_accepting_shares',
});
const listed =
await state.env.server.services.kvShare.listKvHandles(
state.owner.actor,
{ limit: 200 },
);
expect(
listed.items.filter((row) => row.appUid === app.uid),
).toEqual([]);
} finally {
await state.env.server.stores.userBlock.deleteByPair(
state.guest.id,
state.owner.id,
);
}
});
it('refuse a share when the recipient refuses everyone', async () => {
await state.env.server.stores.user.updateMetadata(state.guest.id, {
blockAllShares: true,
});
try {
await expect(share(region('all'), 'watch')).rejects.toMatchObject({
legacyCode: 'recipient_not_accepting_shares',
});
} finally {
await state.env.server.stores.user.updateMetadata(state.guest.id, {
blockAllShares: false,
});
}
});
it('leave a share made before the block to be narrowed, its label cleared, and revoked', async () => {
const app = await makeApp();
const prefix = region('before');
const { handle } = await share(prefix, 'write', {
appUid: app.uid,
label: 'Before',
});
await state.env.server.stores.userBlock.create(
state.guest.id,
state.owner.id,
);
try {
await expect(
share(prefix, 'read', { appUid: app.uid, label: null }),
).resolves.toMatchObject({
handle,
access: 'read',
label: null,
});
await expect(
state.env.server.services.kvShare.removeKvShare(
state.owner.actor,
handle,
),
).resolves.toMatchObject({ handle });
// Still only the notification from before the block.
expect(await notificationsFor(handle)).toHaveLength(1);
} finally {
await state.env.server.stores.userBlock.deleteByPair(
state.guest.id,
state.owner.id,
);
}
});
it.each([
['a raised level', { access: 'write' as const }],
['a new label', { label: 'Later' }],
[
'a lower level with a new label',
{ access: 'watch' as const, label: 'Later' },
],
])('refuse %s on a share made before the block', async (_name, change) => {
const app = await makeApp();
const prefix = region('raise');
const { handle } = await share(prefix, 'read', { appUid: app.uid });
await state.env.server.stores.userBlock.create(
state.guest.id,
state.owner.id,
);
try {
await expect(
share(prefix, 'access' in change ? change.access : 'read', {
appUid: app.uid,
...('label' in change ? { label: change.label } : {}),
}),
).rejects.toMatchObject({
legacyCode: 'recipient_not_accepting_shares',
});
const row =
await state.env.server.stores.kvShareHandle.getByHandle(handle);
expect(row).toMatchObject({ revokedAt: null, label: null });
expect(row?.permission.startsWith('kv-share-read:')).toBe(true);
} finally {
await state.env.server.stores.userBlock.deleteByPair(
state.guest.id,
state.owner.id,
);
}
});
});
@@ -0,0 +1,868 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { v4 as uuidv4 } from 'uuid';
import { describe, expect, it, vi } from 'vitest';
import type { Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { PermissionUtil } from '../permission/permissionUtil.js';
import {
assertBoundedManageGrant,
assertNewShareSegments,
assertPrefixFitsAccess,
assertShareableAppUid,
assertShareablePrefix,
isKvShareManagePermission,
isKvSharePermission,
keyPrefixSegments,
KV_SHARE_PREFIX_MAX_BYTES,
kvShareAccessOf,
kvShareAppDelegateImplicator,
kvShareGrantAtEveryLevel,
kvShareGrantCovers,
kvShareGrantPrefix,
kvShareLevelForMethod,
kvShareLevelsExploder,
kvShareManageNamespaceRoots,
kvShareManagePermission,
kvShareOwnerImplicator,
kvSharePermission,
mintKvHandleId,
normalizeKeyPrefix,
parseKvShareAccess,
parseKvShareLabel,
parseKvShareRecipient,
relativeToKvShareRoot,
withKvShareAccess,
} from './kvShares.js';
import { APP_DATA_KV_METHOD_OPS } from '../permission/appDataScopes.js';
import { isKvHandleId, kvHandleFromSubject } from '../events/subjects.js';
const OWNER = '2a1b0c9d-0000-4000-8000-000000000001';
const OTHER = '2a1b0c9d-0000-4000-8000-000000000002';
const APP = 'app-1234';
const OTHER_APP = 'app-5678';
const userActor = (uuid: string): Actor =>
({ user: { uuid }, effectiveApp: null }) as unknown as Actor;
const appActor = (uuid: string, appUid: string): Actor =>
({
user: { uuid },
app: { uid: appUid },
effectiveApp: { uid: appUid },
}) as unknown as Actor;
describe('the share permission family', () => {
it('names the owner, the app and the granted prefix', () => {
expect(kvSharePermission(OWNER, APP, 'workspace:abc:')).toBe(
`kv-share:${OWNER}:${APP}:workspace:abc`,
);
});
it('makes key segments components, so a deeper key is a descendant', () => {
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
const deeper = kvSharePermission(
OWNER,
APP,
'workspace:abc:messages:1',
);
// The exploder walks parents of the string being checked, so implication
// over key segments needs nothing of its own.
expect(deeper.startsWith(`${granted}:`)).toBe(true);
});
it('escapes a `:` inside a component rather than splitting on it', () => {
const permission = kvSharePermission(OWNER, 'app:odd', 'cart:');
expect(PermissionUtil.split(permission)).toEqual([
'kv-share',
OWNER,
'app:odd',
'cart',
]);
});
it('reads a withdrawn grant as covering itself and everything under it', () => {
const granted = kvSharePermission(OWNER, APP, 'workspace:abc:');
expect(kvShareGrantCovers(granted, granted)).toBe(true);
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OWNER, APP, 'workspace:abc:messages:'),
),
).toBe(true);
// A sibling that merely shares a text prefix is a different region.
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OWNER, APP, 'workspace:abcdef:'),
),
).toBe(false);
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OTHER, APP, 'workspace:abc:'),
),
).toBe(false);
// Coverage never crosses namespaces: the app uid is what keeps a
// revoke inside the one the handle was minted in.
expect(
kvShareGrantCovers(
granted,
kvSharePermission(OWNER, OTHER_APP, 'workspace:abc:messages:'),
),
).toBe(false);
});
it('never reads a withdrawn child as covering its own parent', () => {
// Revoking the deeper of two grants must not read as covering the
// shallower one — coverage only ever runs downward.
const child = kvSharePermission(OWNER, APP, 'workspace:abc:messages:');
const parent = kvSharePermission(OWNER, APP, 'workspace:abc:');
expect(kvShareGrantCovers(child, parent)).toBe(false);
});
});
describe('granted prefixes', () => {
it('always ends on the key delimiter', () => {
expect(normalizeKeyPrefix('workspace:abc')).toBe('workspace:abc:');
expect(normalizeKeyPrefix('workspace:abc:')).toBe('workspace:abc:');
expect(keyPrefixSegments('workspace:abc:')).toEqual([
'workspace',
'abc',
]);
});
it.each([
['', 'the whole namespace'],
[':', 'delimiters only'],
['workspace:*', 'a pattern'],
['workspace:a?c', 'a single-character pattern'],
// Normalizing would drop the empty segment and grant `a:b:` instead —
// a region other than the one asked for.
['workspace::abc:', 'an empty key segment'],
[':workspace:abc:', 'a leading empty segment'],
// `escape_permission_component` leaves `\\` alone while escaping `:`,
// so the stored prefix and the permission string would disagree and
// the share would silently never fire.
['work\\space:abc:', 'a backslash'],
])('refuses %s (%s)', (prefix) => {
let thrown: unknown;
try {
assertShareablePrefix(prefix);
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(HttpError);
expect((thrown as HttpError).legacyCode).toBe(
'invalid_kv_share_prefix',
);
});
it('refuses a region too deep to fit the grant column', () => {
expect(() =>
assertPrefixFitsAccess(OWNER, APP, 'a'.repeat(300), 'watch'),
).toThrow(HttpError);
expect(
assertPrefixFitsAccess(OWNER, APP, 'workspace:abc:', 'watch'),
).toBe(kvSharePermission(OWNER, APP, 'workspace:abc:'));
});
it('keeps the trailing delimiter optional', () => {
expect(assertShareablePrefix('workspace:abc')).toBe('workspace:abc:');
expect(assertShareablePrefix('workspace:abc:')).toBe('workspace:abc:');
});
});
describe('granted namespaces', () => {
it('takes an app uid, and the app-less namespace', () => {
expect(assertShareableAppUid(APP)).toBe(APP);
expect(assertShareableAppUid('os-global')).toBe('os-global');
});
it('refuses one past the column it is stored in', () => {
// A longer one reaches the insert as a write the column cannot hold.
let thrown: unknown;
try {
assertShareableAppUid('a'.repeat(41));
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(HttpError);
expect((thrown as HttpError).statusCode).toBe(400);
});
it('takes both shapes a real app uid comes in', () => {
// Minted (`app-<uuidv4>`) and derived from an origin (`app-<uuidv5>`)
// — 40 characters either way, which is the column exactly.
const minted = `app-${uuidv4()}`;
expect(assertShareableAppUid(minted)).toBe(minted);
expect(minted).toHaveLength(40);
});
it('refuses a handle, which shares the slot but is not a namespace', () => {
expect(() => assertShareableAppUid(mintKvHandleId())).toThrow(
HttpError,
);
});
it('refuses anything else short enough to fit the column', () => {
// The slot ends up in a subject and in a permission string, so a
// namespace carrying a delimiter of either grammar is refused before
// it can be composed into one.
for (const uid of [
'os-globals',
'workspace:abc',
'app',
'app-',
'app-a:b',
'app-a#b',
'v1:someone:app-x',
'',
])
expect(() => assertShareableAppUid(uid)).toThrow(HttpError);
});
});
describe('keys inside a granted region', () => {
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
it('are named relative to the granted root', () => {
expect(
relativeToKvShareRoot(permission, 'workspace:abc:messages:1'),
).toBe('messages:1');
});
it('read as outside when they are', () => {
expect(
relativeToKvShareRoot(permission, 'workspace:abcdef:x'),
).toBeNull();
expect(relativeToKvShareRoot(permission, 'workspace:abc')).toBeNull();
});
});
describe('a delegation request', () => {
const region = kvShareManagePermission(
kvSharePermission(OWNER, APP, 'workspace:abc:'),
);
it('is the manage arm of the grant it would let an app issue', () => {
expect(region).toBe(`manage:kv-share:${OWNER}:${APP}:workspace:abc`);
expect(assertBoundedManageGrant(region)).toBe(region);
});
it.each([
['manage:kv-share', 'the family itself'],
[`manage:kv-share:${OWNER}`, 'an owner and no namespace'],
[`manage:kv-share:${OWNER}:${APP}`, 'a whole namespace'],
])('refuses %s (%s)', (permission) => {
let thrown: unknown;
try {
assertBoundedManageGrant(permission);
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(HttpError);
expect((thrown as HttpError).legacyCode).toBe(
'invalid_kv_share_prefix',
);
});
it('leaves permissions from other families alone', () => {
const unrelated = `manage:fs:${OWNER}:write`;
expect(assertBoundedManageGrant(unrelated)).toBe(unrelated);
// The read arm of this family is not a delegation, so it is not this
// check's to bound.
expect(assertBoundedManageGrant(`kv-share:${OWNER}:${APP}`)).toBeTypeOf(
'string',
);
});
});
describe('handle ids', () => {
it('are told apart from an app uid in the same slot', () => {
const handle = mintKvHandleId();
expect(isKvHandleId(handle)).toBe(true);
expect(isKvHandleId(APP)).toBe(false);
expect(isKvHandleId('kvh-')).toBe(false);
});
it('carry nothing about the owner', () => {
const handle = mintKvHandleId();
expect(handle).not.toContain(OWNER);
expect(mintKvHandleId()).not.toBe(handle);
});
it('are recoverable from the subject a row stored', () => {
const handle = mintKvHandleId();
expect(kvHandleFromSubject(`kv:${handle}:messages:*`)).toBe(handle);
expect(kvHandleFromSubject(`kv:${APP}:messages:*`)).toBeNull();
expect(kvHandleFromSubject(`fs:${handle}:write`)).toBeNull();
});
});
describe('the owner implicator', () => {
const implicator = kvShareOwnerImplicator();
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
it('answers the share family and its manage arm', () => {
expect(implicator.matches(permission)).toBe(true);
expect(implicator.matches(`manage:${permission}`)).toBe(true);
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
});
it('holds for the owner named in the permission, and nobody else', () => {
expect(
implicator.check({ actor: userActor(OWNER), permission }),
).toEqual({});
expect(
implicator.check({
actor: userActor(OWNER),
permission: `manage:${permission}`,
}),
).toEqual({});
expect(
implicator.check({ actor: userActor(OTHER), permission }),
).toBeUndefined();
});
it('reads the owner past a `manage` key segment', () => {
// Stripping every `manage:` rather than the leading one would rewrite
// the string the owner is read out of.
const nested = kvSharePermission(OWNER, APP, 'manage:secrets:');
expect(implicator.matches(nested)).toBe(true);
expect(
implicator.check({ actor: userActor(OWNER), permission: nested }),
).toEqual({});
expect(
implicator.check({ actor: userActor(OTHER), permission: nested }),
).toBeUndefined();
});
it('never answers for an app or a token acting through the owner', () => {
const appActor = {
user: { uuid: OWNER },
app: { uid: APP },
} as unknown as Actor;
expect(
implicator.check({ actor: appActor, permission }),
).toBeUndefined();
});
});
describe('the app delegate implicator', () => {
const permission = kvSharePermission(OWNER, APP, 'workspace:abc:');
it('answers the share family, never its manage arm, nor other families', () => {
const implicator = kvShareAppDelegateImplicator({
userHolds: vi.fn(),
});
expect(implicator.matches(permission)).toBe(true);
expect(implicator.matches(`manage:${permission}`)).toBe(false);
expect(implicator.matches(`fs:${OWNER}:read`)).toBe(false);
});
it("grants when the actor's own app matches the namespace app and the user holds the grant", async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = appActor(OWNER, APP);
await expect(implicator.check({ actor, permission })).resolves.toEqual(
{},
);
expect(userHolds).toHaveBeenCalledTimes(1);
const [calledActor, calledPermission] = userHolds.mock.calls[0];
expect(calledActor.app).toBeUndefined();
expect(calledActor.effectiveApp).toBeNull();
expect(calledActor.user).toEqual({ uuid: OWNER });
expect(calledPermission).toBe(permission);
});
it('refuses when the user does not hold the grant', async () => {
const implicator = kvShareAppDelegateImplicator({
userHolds: vi.fn().mockResolvedValue(false),
});
await expect(
implicator.check({ actor: appActor(OWNER, APP), permission }),
).resolves.toBeUndefined();
});
it("refuses when the actor's app differs from the namespace app", async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
await expect(
implicator.check({
actor: appActor(OWNER, OTHER_APP),
permission,
}),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('refuses a plain user actor with no app', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
await expect(
implicator.check({ actor: userActor(OWNER), permission }),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('refuses an access-token actor', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = {
user: { uuid: OWNER },
app: { uid: APP },
accessToken: { uid: 't' },
} as unknown as Actor;
await expect(
implicator.check({ actor, permission }),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
it('asks the user once for a deep option and the ancestors its check already asked', async () => {
const userHolds = vi.fn().mockResolvedValue(false);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = appActor(OWNER, APP);
const memo = new Map<string, unknown>();
const deep = kvSharePermission(OWNER, APP, 'a:b:c:', 'read');
for (const permission of [
deep,
kvSharePermission(OWNER, APP, 'a:b:c:', 'write'),
kvSharePermission(OWNER, APP, 'a:b:', 'read'),
kvSharePermission(OWNER, APP, 'a:', 'write'),
])
await expect(
implicator.check({ actor, permission, memo }),
).resolves.toBeUndefined();
expect(userHolds).toHaveBeenCalledTimes(1);
// A lower level, or a region beside it, is a question of its own.
await implicator.check({
actor,
permission: kvSharePermission(OWNER, APP, 'a:b:', 'watch'),
memo,
});
await implicator.check({
actor,
permission: kvSharePermission(OWNER, APP, 'a:x:', 'read'),
memo,
});
expect(userHolds).toHaveBeenCalledTimes(3);
// Nothing carries over to another scan.
await implicator.check({ actor, permission: deep });
expect(userHolds).toHaveBeenCalledTimes(4);
});
it('refuses a permission with no key segments or no app', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const implicator = kvShareAppDelegateImplicator({ userHolds });
const actor = appActor(OWNER, APP);
await expect(
implicator.check({
actor,
permission: PermissionUtil.join('kv-share', OWNER, APP),
}),
).resolves.toBeUndefined();
await expect(
implicator.check({
actor,
permission: PermissionUtil.join('kv-share', OWNER),
}),
).resolves.toBeUndefined();
expect(userHolds).not.toHaveBeenCalled();
});
});
/** The refusal a call threw, for asserting on its code. */
const refusal = (fn: () => unknown): HttpError => {
try {
fn();
} catch (err) {
if (err instanceof HttpError) return err;
throw err;
}
throw new Error('expected a refusal');
};
describe('share levels', () => {
const REGION = 'workspace:abc:';
it('put the level in the root, leaving watch grants as they were', () => {
expect(kvSharePermission(OWNER, APP, REGION)).toBe(
`kv-share:${OWNER}:${APP}:workspace:abc`,
);
expect(kvSharePermission(OWNER, APP, REGION, 'read')).toBe(
`kv-share-read:${OWNER}:${APP}:workspace:abc`,
);
expect(kvSharePermission(OWNER, APP, REGION, 'write')).toBe(
`kv-share-write:${OWNER}:${APP}:workspace:abc`,
);
});
it('read the level back off any root, and nothing else as the family', () => {
for (const access of ['watch', 'read', 'write'] as const) {
const permission = kvSharePermission(OWNER, APP, REGION, access);
expect(isKvSharePermission(permission)).toBe(true);
expect(kvShareAccessOf(permission)).toBe(access);
expect(kvShareGrantPrefix(permission)).toBe(REGION);
expect(
relativeToKvShareRoot(permission, `${REGION}messages:1`),
).toBe('messages:1');
}
expect(isKvSharePermission(`kv-shared:${OWNER}`)).toBe(false);
expect(isKvSharePermission(`kv-share-admin:${OWNER}`)).toBe(false);
expect(kvShareAccessOf(`fs:${OWNER}:read`)).toBeNull();
});
it('swap only the root when moving a grant between levels', () => {
const read = kvSharePermission(OWNER, APP, REGION, 'read');
expect(withKvShareAccess(read, 'write')).toBe(
kvSharePermission(OWNER, APP, REGION, 'write'),
);
expect(kvShareGrantAtEveryLevel(read)).toEqual([
kvSharePermission(OWNER, APP, REGION, 'watch'),
read,
kvSharePermission(OWNER, APP, REGION, 'write'),
]);
});
it('cover a region withdrawn at one level at every level', () => {
const writeRegion = kvSharePermission(OWNER, APP, REGION, 'write');
const watchInside = kvSharePermission(OWNER, APP, `${REGION}messages:`);
expect(kvShareGrantCovers(writeRegion, watchInside)).toBe(true);
expect(
kvShareGrantCovers(
kvSharePermission(OWNER, APP, REGION, 'read'),
kvSharePermission(OWNER, APP, REGION, 'write'),
),
).toBe(true);
// Still never upward, and never across namespaces.
expect(kvShareGrantCovers(watchInside, writeRegion)).toBe(false);
expect(
kvShareGrantCovers(
writeRegion,
kvSharePermission(OWNER, OTHER_APP, REGION, 'write'),
),
).toBe(false);
// Nothing outside the family is ever covered.
expect(kvShareGrantCovers(writeRegion, `fs:${OWNER}:read`)).toBe(false);
});
});
describe('the level ladder exploder', () => {
const exploder = kvShareLevelsExploder();
const at = (access: 'watch' | 'read' | 'write') =>
kvSharePermission(OWNER, APP, 'workspace:abc:', access);
it('lets a check at a level be answered by every level above it', async () => {
expect(await exploder.explode({ permission: at('watch') })).toEqual([
at('watch'),
at('read'),
at('write'),
]);
expect(await exploder.explode({ permission: at('read') })).toEqual([
at('read'),
at('write'),
]);
expect(exploder.matches(at('write'))).toBe(false);
});
it('ladders a manage arm the same way', async () => {
expect(
await exploder.explode({ permission: `manage:${at('read')}` }),
).toEqual([`manage:${at('read')}`, `manage:${at('write')}`]);
});
it('never ladders a string with no key segment', async () => {
for (const permission of [
PermissionUtil.join('kv-share', OWNER, APP),
PermissionUtil.join('manage', 'kv-share-read', OWNER, APP),
PermissionUtil.join('kv-share', OWNER),
'kv-share',
])
expect(await exploder.explode({ permission })).toEqual([
permission,
]);
});
it('leaves other families alone', () => {
expect(exploder.matches(`fs:${OWNER}:read`)).toBe(false);
expect(exploder.matches(`app-data:${APP}:kv:read`)).toBe(false);
});
});
describe('the level a driver method needs', () => {
it('is read for get and list, and write for every other mapped op', () => {
expect(kvShareLevelForMethod('get')).toBe('read');
expect(kvShareLevelForMethod('list')).toBe('read');
for (const method of [
'set',
'batchPut',
'add',
'incr',
'decr',
'update',
'del',
'remove',
'expire',
'expireAt',
])
expect(kvShareLevelForMethod(method)).toBe('write');
});
it('refuses flush and anything the op table does not name', () => {
expect(kvShareLevelForMethod('flush')).toBeNull();
expect(kvShareLevelForMethod('toString')).toBeNull();
expect(kvShareLevelForMethod('somethingNew')).toBeNull();
});
it('covers every method the op table knows', () => {
for (const [method, op] of Object.entries(APP_DATA_KV_METHOD_OPS))
expect(kvShareLevelForMethod(method) === null).toBe(op === null);
});
});
describe('share request parsing', () => {
it('takes the three levels and a fallback, and refuses anything else', () => {
expect(parseKvShareAccess(undefined, 'read')).toBe('read');
expect(parseKvShareAccess(null, 'watch')).toBe('watch');
expect(parseKvShareAccess('write', 'read')).toBe('write');
for (const value of ['delete', 'WRITE', '', 2, {}])
expect(
refusal(() => parseKvShareAccess(value, 'read')).legacyCode,
).toBe('kv_share_access_invalid');
});
it('caps a prefix at what the grant column leaves for its owner, app and level', () => {
const owner = uuidv4();
const app = `app-${uuidv4()}`;
// 255 less the root, the owner uuid, the app uid and two separators.
const longest = { watch: 169, read: 164, write: 163 } as const;
for (const access of ['watch', 'read', 'write'] as const) {
const fits = `${'a'.repeat(longest[access] - 1)}:`;
const over = `${'a'.repeat(longest[access])}:`;
expect(
assertPrefixFitsAccess(owner, app, fits, access),
).toHaveLength(255);
const refused = refusal(() =>
assertPrefixFitsAccess(owner, app, over, access),
);
expect(refused.legacyCode).toBe('invalid_kv_share_prefix');
expect(refused.message).toContain(`${longest[access]} characters`);
}
// The app-less namespace has a shorter name, so more room.
expect(() =>
assertPrefixFitsAccess(
owner,
'os-global',
`${'a'.repeat(193)}:`,
'write',
),
).not.toThrow();
});
it('holds a multibyte read or write prefix to the byte cap', () => {
const multibyte = `${'é'.repeat(KV_SHARE_PREFIX_MAX_BYTES / 2)}:`;
expect(
refusal(() => assertPrefixFitsAccess(OWNER, APP, multibyte, 'read'))
.legacyCode,
).toBe('invalid_kv_share_prefix');
expect(() =>
assertPrefixFitsAccess(OWNER, APP, multibyte, 'watch'),
).not.toThrow();
});
it('takes a recipient as a username or an object, never an address', () => {
expect(parseKvShareRecipient(' alice ')).toEqual({
username: 'alice',
});
expect(parseKvShareRecipient({ username: 'alice' })).toEqual({
username: 'alice',
});
expect(parseKvShareRecipient({ uid: OWNER })).toEqual({ uid: OWNER });
for (const value of [
'[email protected]',
{ username: 'a@b' },
'',
{},
[],
42,
null,
undefined,
])
expect(refusal(() => parseKvShareRecipient(value)).legacyCode).toBe(
'invalid_recipient',
);
});
it('trims a label, reads empty as none, and keeps absent apart from null', () => {
expect(parseKvShareLabel(undefined)).toBeUndefined();
expect(parseKvShareLabel(null)).toBeNull();
expect(parseKvShareLabel(' ')).toBeNull();
expect(parseKvShareLabel(' Japan 2027 · from dan ')).toBe(
'Japan 2027 · from dan',
);
});
it.each([
['a control character', 'trip\u0007'],
['a newline', 'trip\nplans'],
['a line separator', 'trip\u2028plans'],
['a paragraph separator', 'trip\u2029plans'],
['a bidi override', 'trip\u202Eplans'],
['a bidi isolate', 'trip\u2067plans'],
['a left-to-right mark', 'trip\u200Eplans'],
['a right-to-left mark', 'trip\u200Fplans'],
['an Arabic letter mark', 'trip\u061Cplans'],
['a zero-width space', 'trip\u200Bplans'],
['a zero-width joiner', 'trip\u200Dplans'],
['a zero-width non-joiner', 'trip\u200Cplans'],
['a word joiner', 'trip\u2060plans'],
['a soft hyphen', 'trip\u00ADplans'],
['a byte order mark', 'trip\uFEFFplans'],
['a tag character', 'trip\u{E0041}plans'],
['an interlinear annotation anchor', 'trip\uFFF9plans'],
['four combining marks on one character', 'e\u0301\u0302\u0303\u0304'],
['a non-string', 42],
['too many characters', 'x'.repeat(129)],
['too many bytes', '😀'.repeat(129)],
])('refuses a label with %s', (_case, value) => {
expect(refusal(() => parseKvShareLabel(value)).legacyCode).toBe(
'kv_share_label_invalid',
);
});
it('holds a new share`s key segments to lowercase letters, digits, `_`, `.` and `-`', () => {
expect(() =>
assertNewShareSegments(['room_1.a-b', 'x9']),
).not.toThrow();
for (const segment of ['Room', 'café', 'a b', 'a*', 'ß', 'İ'])
expect(
refusal(() => assertNewShareSegments(['ok', segment]))
.legacyCode,
).toBe('invalid_kv_share_prefix');
});
it('takes a label at the character cap', () => {
expect(parseKvShareLabel('x'.repeat(128))).toBe('x'.repeat(128));
});
it('takes up to three combining marks on a character, and emoji with a variation selector', () => {
expect(parseKvShareLabel('e\u0301\u0302\u0303 caf\u00E9')).toBe(
'e\u0301\u0302\u0303 caf\u00E9',
);
expect(parseKvShareLabel('Trip \u2764\uFE0F')).toBe(
'Trip \u2764\uFE0F',
);
});
});
describe('delegations at every level', () => {
it('are bounded the same way at every level', () => {
for (const access of ['watch', 'read', 'write'] as const) {
const region = kvShareManagePermission(
kvSharePermission(OWNER, APP, 'workspace:abc:', access),
);
expect(isKvShareManagePermission(region)).toBe(true);
expect(assertBoundedManageGrant(region)).toBe(region);
const root = PermissionUtil.join(
'manage',
access === 'watch' ? 'kv-share' : `kv-share-${access}`,
OWNER,
APP,
);
expect(
refusal(() => assertBoundedManageGrant(root)).legacyCode,
).toBe('invalid_kv_share_prefix');
}
});
it('name the namespace root at every level for the unbounded check', () => {
expect(
kvShareManageNamespaceRoots(
kvSharePermission(OWNER, APP, 'workspace:abc:', 'read'),
),
).toEqual([
`manage:kv-share:${OWNER}:${APP}`,
`manage:kv-share-read:${OWNER}:${APP}`,
`manage:kv-share-write:${OWNER}:${APP}`,
]);
});
it('are answered for the owner at every level, and never by the app delegate', () => {
const owner = kvShareOwnerImplicator();
const delegate = kvShareAppDelegateImplicator({ userHolds: vi.fn() });
for (const access of ['read', 'write'] as const) {
const permission = kvSharePermission(
OWNER,
APP,
'workspace:abc:',
access,
);
expect(owner.matches(permission)).toBe(true);
expect(owner.matches(`manage:${permission}`)).toBe(true);
expect(
owner.check({ actor: userActor(OWNER), permission }),
).toEqual({});
expect(
owner.check({ actor: userActor(OTHER), permission }),
).toBeUndefined();
expect(delegate.matches(permission)).toBe(true);
expect(delegate.matches(`manage:${permission}`)).toBe(false);
}
});
it('let a holder`s app use a read or write grant only in its own namespace', async () => {
const userHolds = vi.fn().mockResolvedValue(true);
const delegate = kvShareAppDelegateImplicator({ userHolds });
const permission = kvSharePermission(
OWNER,
APP,
'workspace:abc:',
'write',
);
await expect(
delegate.check({ actor: appActor(OTHER, APP), permission }),
).resolves.toEqual({});
await expect(
delegate.check({ actor: appActor(OTHER, OTHER_APP), permission }),
).resolves.toBeUndefined();
await expect(
delegate.check({
actor: appActor(OTHER, OTHER_APP),
permission: kvSharePermission(
OWNER,
'os-global',
'workspace:abc:',
'read',
),
}),
).resolves.toBeUndefined();
});
});
+628
View File
@@ -0,0 +1,628 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import {
type Actor,
isAppActor,
isPlainUserActor,
userRelatedActor,
} from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { KV_SHARE_HANDLE_WIDTHS } from '../../stores/events/columnWidths.js';
import {
KV_GLOBAL_APP_KEY,
KV_MAX_KEY_BYTES,
} from '../../stores/systemKv/SystemKVStore.js';
import {
MANAGE_PERM_PREFIX,
PERMISSION_MAX_LEN,
} from '../permission/consts.js';
import { APP_DATA_KV_METHOD_OPS } from '../permission/appDataScopes.js';
import type {
PermissionExploder,
PermissionImplicator,
} from '../permission/permissionUtil.js';
import { PermissionUtil } from '../permission/permissionUtil.js';
import {
isKvHandleId,
KV_HANDLE_PREFIX,
KV_KEY_SEPARATOR,
} from '../events/subjects.js';
/**
* Letting one user watch, read or write a region of another's key-value
* namespace.
*
* The grant is an ordinary user-to-user permission, which is the whole point:
* `manage:` answers "may I delegate this", the exploder's prefix implication
* makes a grant on `workspace:abc:` answer every key beneath it, and revocation
* settling and the delivery re-check key off the stored string exactly as they
* do for a shared folder. Putting the ACL on the key-value item instead would
* take part in none of that, and would tax every write to the item besides.
*
* The permission names the owner, so its components are the three things a
* shared region is, under a root that names the access level:
*
* kv-share:<ownerUserUuid>:<appUid>:<keySegment>... watch
* kv-share-read:<ownerUserUuid>:<appUid>:<keySegment>... read
* kv-share-write:<ownerUserUuid>:<appUid>:<keySegment>... write
*
* The level is in the root rather than a component so the permission system
* enforces it, and so the strings `kv-share:` grants already use keep their
* meaning.
*
* Key segments are components rather than one escaped blob because that is what
* makes prefix implication work without a rule of its own: the check a deep key
* runs is a descendant of the grant string, and the existing parent walk finds
* it. The grantee never sees this string — the handle is the name on the wire.
*/
/** Root of the cross-user key-value share namespace, at the watch level. */
export const KV_SHARE_PERMISSION_PREFIX = 'kv-share';
/** Access levels, lowest first. Each implies the ones before it. */
export const KV_SHARE_ACCESS_LEVELS = ['watch', 'read', 'write'] as const;
export type KvShareAccess = (typeof KV_SHARE_ACCESS_LEVELS)[number];
const KV_SHARE_ROOTS: Record<KvShareAccess, string> = {
watch: KV_SHARE_PERMISSION_PREFIX,
read: `${KV_SHARE_PERMISSION_PREFIX}-read`,
write: `${KV_SHARE_PERMISSION_PREFIX}-write`,
};
/** The permission root a level is granted under. */
export const kvShareRoot = (access: KvShareAccess): string =>
KV_SHARE_ROOTS[access];
const ACCESS_BY_ROOT = new Map<string, KvShareAccess>(
KV_SHARE_ACCESS_LEVELS.map((access) => [KV_SHARE_ROOTS[access], access]),
);
/**
* Byte ceiling on a `read` or `write` prefix, which the fixed relative-key
* limit assumes. The grant column binds first for every prefix a new share may
* name; this one only bites on a multibyte prefix shared before that rule.
*/
export const KV_SHARE_PREFIX_MAX_BYTES = 256;
/**
* Longest key a call through a handle may name, relative to the handle, in
* UTF-8 bytes: what the store's key limit leaves after the longest prefix, so
* the limit a holder hits says nothing about how long the prefix is.
*/
export const KV_HANDLE_KEY_MAX_BYTES =
KV_MAX_KEY_BYTES - KV_SHARE_PREFIX_MAX_BYTES;
/** Width of the `app_uid` column the handle row stores its namespace in. */
export const KV_SHARE_APP_UID_MAX_LENGTH = KV_SHARE_HANDLE_WIDTHS.appUid.max;
export const mintKvHandleId = (): string =>
`${KV_HANDLE_PREFIX}${randomUUID()}`;
/** The key segments a prefix contributes to a permission string. */
export const keyPrefixSegments = (keyPrefix: string): string[] =>
keyPrefix.split(KV_KEY_SEPARATOR).filter((segment) => segment.length > 0);
/**
* A granted prefix as it is stored: always ending on the key delimiter, so
* `keyPrefix + relative` is a whole key and the region never accidentally
* includes the key that names it.
*/
export const normalizeKeyPrefix = (keyPrefix: string): string => {
const segments = keyPrefixSegments(keyPrefix);
return segments.length === 0
? ''
: `${segments.join(KV_KEY_SEPARATOR)}${KV_KEY_SEPARATOR}`;
};
export const kvSharePermission = (
ownerUserUuid: string,
appUid: string,
keyPrefix: string,
access: KvShareAccess = 'watch',
): string =>
PermissionUtil.join(
KV_SHARE_ROOTS[access],
ownerUserUuid,
appUid,
...keyPrefixSegments(keyPrefix),
);
/** The family root a permission starts with, or `null` outside the family. */
const kvShareRootOf = (permission: string): string | null => {
const end = permission.indexOf(':');
const head = end === -1 ? permission : permission.slice(0, end);
return ACCESS_BY_ROOT.has(head) ? head : null;
};
export const isKvSharePermission = (permission: string): boolean =>
kvShareRootOf(permission) !== null;
/** The access level a share grant carries, or `null` outside the family. */
export const kvShareAccessOf = (permission: string): KvShareAccess | null => {
const root = kvShareRootOf(permission);
return root === null ? null : (ACCESS_BY_ROOT.get(root) ?? null);
};
/** The same grant at another level: only the root changes. */
export const withKvShareAccess = (
permission: string,
access: KvShareAccess,
): string => {
const root = kvShareRootOf(permission);
if (root === null) return permission;
return `${KV_SHARE_ROOTS[access]}${permission.slice(root.length)}`;
};
/** The grant at every level, for withdrawing a region whatever it was. */
export const kvShareGrantAtEveryLevel = (permission: string): string[] =>
KV_SHARE_ACCESS_LEVELS.map((access) =>
withKvShareAccess(permission, access),
);
/**
* Whether withdrawing `revoked` takes `held` with it. Prefix implication read
* backwards: a grant answers every check at or beneath it, so withdrawing it
* puts every one of those in question. Compared below the root, because a
* region withdrawn at one level is in question at every level.
*/
export const kvShareGrantCovers = (revoked: string, held: string): boolean => {
const revokedRoot = kvShareRootOf(revoked);
const heldRoot = kvShareRootOf(held);
if (revokedRoot === null || heldRoot === null) return false;
const region = revoked.slice(revokedRoot.length);
const inside = held.slice(heldRoot.length);
return inside === region || inside.startsWith(`${region}:`);
};
/**
* The level a key-value driver method needs through a share, or `null` for a
* method no share reaches. Derived from the cross-app op table, so a method
* missing from it fails closed.
*/
export const kvShareLevelForMethod = (method: string): KvShareAccess | null => {
if (!Object.prototype.hasOwnProperty.call(APP_DATA_KV_METHOD_OPS, method))
return null;
const op = APP_DATA_KV_METHOD_OPS[method];
if (op === null) return null;
return op === 'get' || op === 'list' ? 'read' : 'write';
};
/** The granted root a share grant names, read back as a key prefix. */
export const kvShareGrantPrefix = (permission: string): string =>
normalizeKeyPrefix(
PermissionUtil.split(permission).slice(3).join(KV_KEY_SEPARATOR),
);
/**
* A key inside a granted region, named the way its holder addresses it, or
* `null` for one outside. The handle is the granted root, so everything the
* holder is shown is relative to it — an absolute key would name a namespace
* they cannot address and were never told about.
*/
export const relativeToKvShareRoot = (
permission: string,
key: string,
): string | null => {
const prefix = kvShareGrantPrefix(permission);
if (!prefix || !key.startsWith(prefix)) return null;
return key.slice(prefix.length);
};
/**
* `kv.shareHandles` is off: no `read` or `write` shares, and no calls through
* one.
*/
export const kvHandlesDisabled = (): HttpError =>
new HttpError(
403,
'Sharing key-value data for reading or writing is not available',
{ legacyCode: 'kv_handles_disabled' },
);
// -- Minting input ----------------------------------------------------
export const invalidPrefix = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'invalid_kv_share_prefix' });
const invalidAccess = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'kv_share_access_invalid' });
/** The level a mint asks for, `fallback` when it names none. */
export const parseKvShareAccess = (
value: unknown,
fallback: KvShareAccess,
): KvShareAccess => {
if (value === undefined || value === null) return fallback;
if (
typeof value !== 'string' ||
!(KV_SHARE_ACCESS_LEVELS as readonly string[]).includes(value)
)
throw invalidAccess('`access` must be `watch`, `read` or `write`');
return value as KvShareAccess;
};
/**
* The grant a share of `keyPrefix` at `access` would issue, refused with
* `invalid_kv_share_prefix` when it would not fit the permission column. The
* room left depends on the owner uuid, the app uid and the level's root, so the
* longest prefix differs per namespace and level.
*/
export const assertPrefixFitsAccess = (
ownerUuid: string,
appUid: string,
keyPrefix: string,
access: KvShareAccess,
): string => {
const permission = kvSharePermission(ownerUuid, appUid, keyPrefix, access);
const prefix = normalizeKeyPrefix(keyPrefix);
const room = PERMISSION_MAX_LEN - (permission.length - prefix.length);
const overBytes =
access !== 'watch' &&
Buffer.byteLength(prefix, 'utf8') > KV_SHARE_PREFIX_MAX_BYTES;
if (permission.length > PERMISSION_MAX_LEN || overBytes)
throw invalidPrefix(
`With \`${access}\` access, a share prefix in this namespace may be at most ${room} characters` +
(access === 'watch'
? ''
: ` and ${KV_SHARE_PREFIX_MAX_BYTES} bytes`),
);
return permission;
};
const invalidRecipient = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'invalid_recipient' });
/** Who a share is for, as a username or a user uuid. */
export interface KvShareRecipient {
username?: string;
uid?: string;
}
/**
* A recipient as `POST /kv/shares` takes it: a username, `{ username }` or `{
* uid }`. An address is refused rather than looked up, so email recipients can
* later mean what they mean for file shares.
*/
export const parseKvShareRecipient = (value: unknown): KvShareRecipient => {
const named = (username: unknown): KvShareRecipient => {
if (typeof username !== 'string' || username.trim() === '')
throw invalidRecipient('`recipient` must name a user');
if (username.includes('@'))
throw invalidRecipient(
'`recipient` must be a username, not an email address',
);
return { username: username.trim() };
};
if (typeof value === 'string') return named(value);
if (value && typeof value === 'object' && !Array.isArray(value)) {
const { username, uid } = value as Record<string, unknown>;
if (username !== undefined) return named(username);
if (typeof uid === 'string' && uid.trim() !== '')
return { uid: uid.trim() };
}
throw invalidRecipient(
'`recipient` must be a username, `{ username }` or `{ uid }`',
);
};
const invalidLabel = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'kv_share_label_invalid' });
export const KV_SHARE_LABEL_MAX_CHARS = 128;
export const KV_SHARE_LABEL_MAX_BYTES = 512;
/**
* Control and format characters (bidi marks, overrides and isolates, zero-width
* characters, BOM, tags) and line or paragraph separators.
*/
const LABEL_FORBIDDEN = /[\p{Cc}\p{Cf}\p{Zl}\p{Zp}]/u;
/** Combining marks a label may stack on one character. */
export const KV_SHARE_LABEL_MAX_COMBINING_MARKS = 3;
const LABEL_MARK_RUN = new RegExp(
`\\p{M}{${KV_SHARE_LABEL_MAX_COMBINING_MARKS + 1},}`,
'u',
);
/**
* An owner-set label. `undefined` leaves the stored one alone, `null` clears
* it, and a string that trims to nothing reads as `null`.
*/
export const parseKvShareLabel = (
value: unknown,
): string | null | undefined => {
if (value === undefined) return undefined;
if (value === null) return null;
if (typeof value !== 'string')
throw invalidLabel('`label` must be a string');
const label = value.trim();
if (label === '') return null;
if (LABEL_FORBIDDEN.test(label))
throw invalidLabel(
'`label` may not contain control, formatting, line-break or text-direction characters',
);
if (LABEL_MARK_RUN.test(label))
throw invalidLabel(
`\`label\` may not stack more than ${KV_SHARE_LABEL_MAX_COMBINING_MARKS} combining marks on one character`,
);
if (
[...label].length > KV_SHARE_LABEL_MAX_CHARS ||
Buffer.byteLength(label, 'utf8') > KV_SHARE_LABEL_MAX_BYTES
)
throw invalidLabel(
`\`label\` may not exceed ${KV_SHARE_LABEL_MAX_CHARS} characters or ${KV_SHARE_LABEL_MAX_BYTES} bytes`,
);
return label;
};
export const invalidAppUid = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'bad_request' });
/**
* The region a handle may be minted on. A pattern is refused because a handle
* names a region rather than selecting within one, and the namespace root is
* refused because a handle over everything the app holds is not a bounded
* capability — it is the app's data, and sharing that is a different decision
* with a different consent.
*/
export const assertShareablePrefix = (keyPrefix: unknown): string => {
if (typeof keyPrefix !== 'string')
throw invalidPrefix('`prefix` must be a string');
if (keyPrefix.includes('*') || keyPrefix.includes('?'))
throw invalidPrefix('A share prefix is a key prefix, not a pattern');
// `escape_permission_component` escapes `:` as `\C` but leaves `\` alone,
// so a prefix carrying one desynchronizes the stored `key_prefix` from the
// permission string and the share silently never fires.
if (keyPrefix.includes('\\'))
throw invalidPrefix('A share prefix may not contain a backslash');
// Normalizing drops empty segments, so `a::b:` would silently become a
// grant on `a:b:` — a region other than the one asked for. Refused rather
// than rewritten; only the trailing delimiter is optional.
const written = keyPrefix.split(KV_KEY_SEPARATOR);
if (written[written.length - 1] === '') written.pop();
if (written.some((segment) => segment === ''))
throw invalidPrefix('A share prefix may not have an empty key segment');
const normalized = normalizeKeyPrefix(keyPrefix);
if (normalized === '')
throw invalidPrefix('A share prefix may not be the whole namespace');
return normalized;
};
const NEW_SHARE_SEGMENT = /^[a-z0-9_.-]+$/;
/**
* The key segments a new share or share consent may name. One spelling per
* region, so no two grants differ only in case or accents, which some databases
* compare as equal. Existing shares are not held to it.
*/
export const assertNewShareSegments = (segments: readonly string[]): void => {
if (segments.some((segment) => !NEW_SHARE_SEGMENT.test(segment)))
throw invalidPrefix(
'A share prefix may only contain lowercase letters (a-z), digits, `_`, `.` and `-` between its `:` separators',
);
};
/** A new share over a prefix that an existing share spells differently. */
export const kvSharePrefixCaseConflict = (): HttpError =>
invalidPrefix(
'This user already has a share of a prefix that differs from this one only in letter case',
);
/**
* The shape every app uid takes, minted or derived from an origin. Narrow on
* purpose: `:` and `#` delimit the anchor token and permission grammars, so
* excluding them matters as much as excluding the handle prefix does.
*/
export const APP_UID_SHAPE = /^app-[A-Za-z0-9_-]{1,64}$/;
/** Fixed namespaces with no app of their own. */
const FIXED_KV_NAMESPACES: readonly string[] = [KV_GLOBAL_APP_KEY];
/**
* The namespace a handle may be minted over: the caller's own app slot, or the
* app-less one. Bounded by the column that stores it, shaped like a real app
* uid or one of the platform's fixed namespaces, and never handle-shaped, since
* the two share the app slot of a `kv:` subject.
*/
export const assertShareableAppUid = (appUid: string): string => {
if (appUid.length > KV_SHARE_APP_UID_MAX_LENGTH)
throw invalidAppUid(
`\`appUid\` may not exceed ${KV_SHARE_APP_UID_MAX_LENGTH} characters`,
);
if (isKvHandleId(appUid))
throw invalidAppUid('A share handle does not name a namespace');
if (!APP_UID_SHAPE.test(appUid) && !FIXED_KV_NAMESPACES.includes(appUid))
throw invalidAppUid('`appUid` does not name a namespace');
return appUid;
};
// -- Delegation -------------------------------------------------------
/** The delegation arm of a share grant: authority to hand the region out. */
export const kvShareManagePermission = (permission: string): string =>
PermissionUtil.join(
MANAGE_PERM_PREFIX,
...PermissionUtil.split(permission),
);
export const isKvShareManagePermission = (permission: string): boolean =>
PermissionUtil.isManage(permission) &&
isKvSharePermission(permission.slice(MANAGE_PERM_PREFIX.length + 1));
/**
* A delegation must name a subtree. Consent on the namespace root reads as "let
* this app hand out anything it has stored for you", which is not a bounded
* capability and so is not something a prompt can put to a user.
*/
export const assertBoundedManageGrant = (permission: string): string => {
if (!isKvShareManagePermission(permission)) return permission;
const [, , owner, appUid, ...segments] = PermissionUtil.split(permission);
if (!owner || !appUid || segments.length === 0)
throw invalidPrefix(
'A share delegation must name a region, not the whole namespace',
);
return permission;
};
/**
* The unbounded `manage:` grants, one per level, that a bounded delegation's
* own permission sits in. The consent surface refuses to ever write these rows,
* but `canManagePermission` walks ancestors to decide "may I delegate this", so
* one that exists by any other path would silently authorize a mint anywhere in
* the namespace. This is what lets the mint path notice it regardless.
*/
export const kvShareManageNamespaceRoots = (permission: string): string[] => {
const [, owner, appUid] = PermissionUtil.split(permission);
return KV_SHARE_ACCESS_LEVELS.map((access) =>
PermissionUtil.join(
MANAGE_PERM_PREFIX,
KV_SHARE_ROOTS[access],
owner,
appUid,
),
);
};
// -- Permission rules -------------------------------------------------
/**
* The level ladder: a check at one level is also answered by a grant at any
* level above it, `manage:` arms included. Only a string with at least one key
* segment is laddered, so no level of a namespace-root grant answers another.
*/
export const kvShareLevelsExploder = (): PermissionExploder => ({
id: 'kv-share-levels',
matches: (permission: string): boolean => {
const access = kvShareAccessOf(
PermissionUtil.stripManageArms(permission),
);
return access !== null && access !== 'write';
},
explode: ({ permission }): string[] => {
const grant = PermissionUtil.stripManageArms(permission);
const arms = permission.slice(0, permission.length - grant.length);
const access = kvShareAccessOf(grant);
// Owner, app, and at least one key segment after the root.
if (access === null || grant.split(':').length < 4) return [permission];
const above = KV_SHARE_ACCESS_LEVELS.slice(
KV_SHARE_ACCESS_LEVELS.indexOf(access) + 1,
);
return [
permission,
...above.map(
(level) => `${arms}${withKvShareAccess(grant, level)}`,
),
];
},
});
/**
* The permission a `manage:` arm delegates over. Only the leading component is
* stripped: a key segment may itself be `manage`, and taking those out would
* change which string the owner is read from.
*/
const withoutManageArm = (permission: string): string =>
permission.startsWith(`${MANAGE_PERM_PREFIX}:`)
? permission.slice(MANAGE_PERM_PREFIX.length + 1)
: permission;
/**
* Owning the namespace is holding every share grant over it, and being able to
* issue them. Without this the owner cannot mint a handle on their own data:
* `grantUserUserPermission` asks `canManagePermission` first, and there is no
* row anywhere saying a user may manage what is already theirs.
*
* Restricted to plain user actors, as the filesystem's `is-owner` is. An app
* disposing of a region of its user's namespace is delegation, which is the
* `manage:` grant's job rather than this one's.
*/
export const kvShareOwnerImplicator = (): PermissionImplicator => ({
id: 'kv-share-is-owner',
shortcut: true,
matches: (permission: string): boolean =>
isKvSharePermission(withoutManageArm(permission)),
check: ({ actor, permission }): unknown => {
if (!isPlainUserActor(actor)) return undefined;
const uuid = actor.user?.uuid;
if (!uuid) return undefined;
const owner = PermissionUtil.split(withoutManageArm(permission))[1];
return owner === uuid ? {} : undefined;
},
});
/** Scan memo key for the grants a user check already refused. */
const DELEGATE_REFUSED = 'kv-share-app-delegate:refused';
/**
* Whether a check of `asked` also asks `option`: `option` is `asked` or an
* ancestor of it, at the same level or above.
*/
const kvShareCheckReaches = (asked: string, option: string): boolean => {
const askedAccess = kvShareAccessOf(asked);
const optionAccess = kvShareAccessOf(option);
if (askedAccess === null || optionAccess === null) return false;
return (
KV_SHARE_ACCESS_LEVELS.indexOf(optionAccess) >=
KV_SHARE_ACCESS_LEVELS.indexOf(askedAccess) &&
kvShareGrantCovers(option, asked)
);
};
/**
* Letting an app-under-user actor exercise a share grant its user holds, at any
* level, bounded to the app the grant's namespace names.
*
* Only the grant itself is matched — never `manage:kv-share…` — because
* answering the manage arm here would read as the unbounded namespace-root
* delegation and break minting for everyone (see
* `kvShareManageNamespaceRoots`). The namespace-app check keeps one app from
* reading a region shared with another app of the same user.
*/
export const kvShareAppDelegateImplicator = (deps: {
userHolds: (actor: Actor, permission: string) => Promise<boolean>;
}): PermissionImplicator => ({
id: 'kv-share-app-delegate',
shortcut: true,
matches: (permission: string): boolean => isKvSharePermission(permission),
check: async ({ actor, permission, memo }): Promise<unknown> => {
if (!isAppActor(actor)) return undefined;
const app = actor.effectiveApp;
if (!app?.uid) return undefined;
const [, owner, namespaceApp, ...segments] =
PermissionUtil.split(permission);
if (!owner || !namespaceApp || segments.length === 0) return undefined;
if (namespaceApp !== app.uid) return undefined;
// A scan offers the deepest option first, and the user check of one
// also asks its ancestors at its level or above, so its refusal
// answers those without asking again.
const refused = memo?.get(DELEGATE_REFUSED) as string[] | undefined;
if (refused?.some((asked) => kvShareCheckReaches(asked, permission)))
return undefined;
if (await deps.userHolds(userRelatedActor(actor), permission))
return {};
memo?.set(DELEGATE_REFUSED, [...(refused ?? []), permission]);
return undefined;
},
});
@@ -223,6 +223,23 @@ export const actorOnPaidPlan = async (
return subscriptionSatisfies(subscription.id, true);
};
/**
* The actor's plan id, for tiered limits. `null` with no metering to ask, or
* when the lookup fails, which the tier tables read as their default.
*/
export const actorPlanId = async (
metering: SubscriptionMetering | undefined,
actor: Actor,
): Promise<string | null> => {
if (!metering) return null;
try {
return (await metering.getActorSubscription(actor)).id;
} catch (err) {
console.warn('[metering] could not resolve a plan', err);
return null;
}
};
/**
* Where a plan gate sits, so a `subscription.gate.<surface>` listener can waive
* it for a call it recognizes. `surface` is the key segment after
@@ -28,6 +28,8 @@ import {
} from './notificationSocket.js';
import {
findNotificationType,
inAccountMailbox,
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
resolveNotificationWrite,
} from './notificationTypes.js';
import type { NotificationTypeName } from './notificationTypes.js';
@@ -243,6 +245,7 @@ export class NotificationService extends PuterService {
* message the GUI has always listened for.
*/
deliverOverSocket(context: NotifEventContext): void {
if (!inAccountMailbox(context.type)) return;
this.#socket().message(
context.userId,
context.uid,
@@ -299,7 +302,9 @@ export class NotificationService extends PuterService {
payload: Record<string, unknown>,
scope: { type: string; audience: string; appUid: string | null },
): Promise<void> {
const desktop = inAccountMailbox(scope.type);
if (!notificationsFoldInEnabled(this.config)) {
if (!desktop) return;
this.#socket().message(userId, uid, payload);
this.#markDelivered(uid, userId);
return;
@@ -309,6 +314,7 @@ export class NotificationService extends PuterService {
// looked up only on this path, and user rows are cached.
const user = await this.stores.user.getById(userId);
if (!user?.uuid) {
if (!desktop) return;
this.#socket().message(userId, uid, payload);
this.#markDelivered(uid, userId);
return;
@@ -332,7 +338,7 @@ export class NotificationService extends PuterService {
},
{},
);
this.#markDelivered(uid, userId);
if (desktop) this.#markDelivered(uid, userId);
}
/**
@@ -372,6 +378,7 @@ export class NotificationService extends PuterService {
const rows = await this.stores.notification.listByUserId(userId, {
filter: 'unseen',
limit: UNREAD_REPLAY_LIMIT,
excludeTypes: OUTSIDE_ACCOUNT_MAILBOX_TYPES,
});
if (rows.length === 0) return;
@@ -20,7 +20,9 @@
import { describe, expect, it } from 'vitest';
import {
NOTIFICATION_TYPES,
OUTSIDE_ACCOUNT_MAILBOX_TYPES,
findNotificationType,
inAccountMailbox,
resolveNotificationWrite,
} from './notificationTypes.js';
@@ -128,3 +130,28 @@ describe('resolveNotificationWrite', () => {
);
});
});
describe('types kept off the account mailbox', () => {
it('are only the ones an app alone has a use for', () => {
expect(OUTSIDE_ACCOUNT_MAILBOX_TYPES).toEqual(['kv.share.received']);
for (const entry of NOTIFICATION_TYPES)
expect(inAccountMailbox(entry.type)).toBe(
entry.type !== 'kv.share.received',
);
// An unclassified legacy row still belongs on the desktop.
expect(inAccountMailbox('')).toBe(true);
expect(inAccountMailbox(undefined)).toBe(true);
});
it('include the key-value share notice, which always names its app', () => {
expect(findNotificationType('kv.share.received')).toMatchObject({
audience: 'app-user',
appScoped: true,
groupable: false,
accountMailbox: false,
});
expect(() =>
resolveNotificationWrite('kv.share.received', null),
).toThrow(/requires an app uid/);
});
});
@@ -38,6 +38,12 @@ export interface NotificationType {
appScoped: boolean;
/** Whether `notifyUpdate` may fold a later one into an open row. */
groupable: boolean;
/**
* `false` keeps the type off the account's own desktop: out of the
* whole-mailbox listing, the unread replay and the desktop wire. Apps still
* read it through its subject. Absent means it shows there.
*/
accountMailbox?: false;
/** Where a delivery of this type publishes. */
subject: (ref: NotificationSubjectRef) => string;
}
@@ -109,6 +115,16 @@ export const NOTIFICATION_TYPES = [
groupable: false,
subject: subjectFor('developer'),
},
// A key-value share is news inside the app whose data it is, and nowhere
// else: the desktop has nothing to open it with.
{
type: 'kv.share.received',
audience: 'app-user',
appScoped: true,
groupable: false,
accountMailbox: false,
subject: subjectFor('app-user'),
},
] as const satisfies readonly NotificationType[];
/** Every registered type name; what `notify` accepts. */
@@ -125,6 +141,16 @@ export const findNotificationType = (
type: string,
): NotificationType | undefined => BY_NAME.get(type);
/** Types kept off the account's own desktop; see `accountMailbox`. */
export const OUTSIDE_ACCOUNT_MAILBOX_TYPES: readonly string[] =
NOTIFICATION_TYPES.filter(
(entry: NotificationType) => entry.accountMailbox === false,
).map((entry) => entry.type);
/** Whether rows of this type belong on the account's own desktop. */
export const inAccountMailbox = (type: unknown): boolean =>
typeof type !== 'string' || !OUTSIDE_ACCOUNT_MAILBOX_TYPES.includes(type);
/**
* The registry entry for a write, or a throw. `account` rows never name an app;
* app-scoped rows always do.
@@ -1267,6 +1267,93 @@ describe('PermissionService (integration)', () => {
expect(flat.filter((v) => !v.deleted)).toHaveLength(0);
});
it('revokeUserUserPermissions removes several grants with one delete and one generation bump, announcing each', async () => {
const { user: issuer, actor: issuerActor } = await makeUserActor();
const { user: target } = await makeUserActor();
const tag = uuidv4();
const granted = [
`zztest:rvk-many-${tag}:a:read`,
`zztest:rvk-many-${tag}:b:read`,
];
const absent = `zztest:rvk-many-${tag}:c:read`;
for (const permission of [...granted, absent])
await grantManage(issuer, permission);
for (const permission of granted)
await runWithContext({ actor: issuerActor }, () =>
permService.grantUserUserPermission(
issuerActor,
target.username,
permission,
),
);
const store = server.stores.permission;
const single = vi.spyOn(store, 'deleteUserUserPermByHolder');
const bump = vi.spyOn(store, 'bumpCacheGeneration');
const emit = vi.spyOn(server.clients.event, 'emit');
let removed: string[];
try {
removed = await runWithContext({ actor: issuerActor }, () =>
permService.revokeUserUserPermissions(
issuerActor,
target.username,
[...granted, absent],
),
);
expect(single).not.toHaveBeenCalled();
expect(bump).toHaveBeenCalledTimes(1);
expect(
emit.mock.calls
.filter(([key]) => key === 'permission.revoked')
.map(
([, data]) =>
(data as { permission: string }).permission,
)
.sort(),
).toEqual([...granted].sort());
} finally {
single.mockRestore();
bump.mockRestore();
emit.mockRestore();
}
expect(removed.sort()).toEqual([...granted].sort());
expect(
await store.readLinkedUserUserPerms(target.id, granted),
).toHaveLength(0);
const flat = await store.getFlatUserPerms(target.id, granted);
expect(flat.filter((v) => !v.deleted)).toHaveLength(0);
});
it('revokeUserUserPermissions refuses an issuer who may not manage one of them, removing nothing', async () => {
const { user: issuer, actor: issuerActor } = await makeUserActor();
const { user: target } = await makeUserActor();
const managed = `zztest:rvk-auth-${uuidv4()}:ii:read`;
const unmanaged = `zztest:rvk-auth-${uuidv4()}:ii:read`;
await grantManage(issuer, managed);
await runWithContext({ actor: issuerActor }, () =>
permService.grantUserUserPermission(
issuerActor,
target.username,
managed,
),
);
await expect(
runWithContext({ actor: issuerActor }, () =>
permService.revokeUserUserPermissions(
issuerActor,
target.username,
[managed, unmanaged],
),
),
).rejects.toMatchObject({ statusCode: 403 });
expect(
await server.stores.permission.readLinkedUserUserPerms(
target.id,
[managed],
),
).toHaveLength(1);
});
it('grantUserUserPermission persists the linked SQL row before resolving', async () => {
const { user: issuer, actor: issuerActor } = await makeUserActor();
const { user: target } = await makeUserActor();
@@ -510,12 +510,17 @@ export class PermissionService extends PuterService {
}
// -- shortcut implicators --
const memo = new Map<string, unknown>();
let shortCircuit = false;
for (const permission of options) {
for (const implicator of this.implicators) {
if (!implicator.shortcut) continue;
if (!implicator.matches(permission)) continue;
const implied = await implicator.check({ actor, permission });
const implied = await implicator.check({
actor,
permission,
memo,
});
if (!implied) continue;
reading.push({
$: 'option',
@@ -542,7 +547,7 @@ export class PermissionService extends PuterService {
// immediately. Scanners only push into `reading`; they don't
// read each other's writes, so there are no ordering hazards.
await Promise.all([
this.#scanNonShortcutImplicators(actor, options, reading),
this.#scanNonShortcutImplicators(actor, options, reading, memo),
this.#scanAccessToken(actor, options, reading),
this.#scanUserUser(actor, options, reading, workingState),
this.#scanUserGroup(actor, options, reading, workingState),
@@ -578,12 +583,17 @@ export class PermissionService extends PuterService {
actor: Actor,
options: string[],
reading: ReadingNode[],
memo: Map<string, unknown>,
): Promise<void> {
for (const permission of options) {
for (const implicator of this.implicators) {
if (implicator.shortcut) continue;
if (!implicator.matches(permission)) continue;
const implied = await implicator.check({ actor, permission });
const implied = await implicator.check({
actor,
permission,
memo,
});
if (!implied) continue;
reading.push({
$: 'option',
@@ -1267,6 +1277,7 @@ export class PermissionService extends PuterService {
permission = await this.rewritePermissionForActor(actor, permission);
const user = await this.#requireUser(username);
const issuerId = this.#requireUserId(actor);
const grantIssuerId = opts.issuerUserId ?? issuerId;
// Giving up access you hold needs no authority over the permission —
// it can only ever narrow what you can reach.
@@ -1284,7 +1295,7 @@ export class PermissionService extends PuterService {
const revoked = await this.stores.permission.deleteUserUserPermByHolder(
user.id,
permission,
opts.issuerUserId ?? issuerId,
grantIssuerId,
);
// The flat key isn't issuer-scoped, so it may only go once no issuer
@@ -1311,11 +1322,11 @@ export class PermissionService extends PuterService {
this.#audit(
this.stores.permission.auditUserUserPerm({
holder_user_id: user.id,
issuer_user_id: issuerId,
issuer_user_id: grantIssuerId,
permission,
action: 'revoke',
reason: meta.reason ?? 'revoked via PermissionService',
extra: this.#auditActorContext(actor),
extra: this.#revokeAuditExtra(actor, grantIssuerId),
}),
'user-user revoke',
);
@@ -1330,6 +1341,86 @@ export class PermissionService extends PuterService {
return revoked;
}
/**
* {@link revokeUserUserPermission} for several permissions of one holder:
* one delete, one flat-view cleanup and one generation bump. Returns the
* permissions that were removed, each announced on its own.
*/
async revokeUserUserPermissions(
actor: Actor,
username: string,
permissions: readonly string[],
meta: GrantMeta = {},
opts: { issuerUserId?: number } = {},
): Promise<string[]> {
const rewritten = [
...new Set(
await Promise.all(
permissions.map((permission) =>
this.rewritePermissionForActor(actor, permission),
),
),
),
];
if (rewritten.length === 0) return [];
const user = await this.#requireUser(username);
const issuerId = this.#requireUserId(actor);
const grantIssuerId = opts.issuerUserId ?? issuerId;
if (user.id !== issuerId) {
await Promise.all(
rewritten.map((permission) =>
this.#assertCanManage(actor, permission),
),
);
}
const removed =
await this.stores.permission.deleteUserUserPermsByHolder(
user.id,
rewritten,
grantIssuerId,
);
// As the single revoke: the flat key goes only where no issuer's grant
// remains, read from the primary the delete just landed on. A database
// that matches case-insensitively may have removed a row spelled other
// than asked, and its flat key is under that spelling.
const spellings = [...new Set([...rewritten, ...removed])];
const remaining = new Set(
(
await this.stores.permission.readLinkedUserUserPermsFromPrimary(
user.id,
spellings,
)
).map((row) => row.permission),
);
await this.stores.permission.delFlatUserPerms(
spellings
.filter((permission) => !remaining.has(permission))
.map((permission) => ({ holderUserId: user.id, permission })),
);
for (const permission of removed) {
this.#audit(
this.stores.permission.auditUserUserPerm({
holder_user_id: user.id,
issuer_user_id: grantIssuerId,
permission,
action: 'revoke',
reason: meta.reason ?? 'revoked via PermissionService',
extra: this.#revokeAuditExtra(actor, grantIssuerId),
}),
'user-user revoke',
);
}
if (user.uuid) await this.#bumpUserCacheGeneration(user.uuid);
for (const permission of removed)
this.#announceRevoked(user.id, null, permission);
return removed;
}
/**
* What an audit row records about who was acting: which app asked, which is
* the whole question for anything minted under a standing consent.
@@ -1350,6 +1441,45 @@ export class PermissionService extends PuterService {
return acting?.uid ? { appUid: acting.uid } : null;
}
/**
* A revoke row is filed under the grant's issuer, so a holder giving up
* access never has it in their own trail; who acted goes in `extra`.
*/
#revokeAuditExtra(
actor: Actor,
grantIssuerId: number,
): Record<string, unknown> | null {
const context = this.#auditActorContext(actor);
const actingUserId = actor.user?.id;
if (actingUserId === undefined || actingUserId === grantIssuerId)
return context;
return { ...context, actingUserId };
}
/**
* Whether this holder has any of `permissions` from this issuer, as a SQL
* row or a flat entry: a point lookup, for skipping a revoke with nothing
* to remove.
*/
async hasUserUserGrant(
holderUserId: number,
issuerUserId: number,
permissions: readonly string[],
): Promise<boolean> {
if (permissions.length === 0) return false;
const [rows, flat] = await Promise.all([
this.stores.permission.findUserUserPermsByIssuer(
holderUserId,
issuerUserId,
permissions,
),
this.stores.permission.getFlatUserPerms(holderUserId, [
...permissions,
]),
]);
return rows.length > 0 || flat.some((value) => !value.deleted);
}
/**
* Withdraw a grant and everything the holder was given beneath it.
*
@@ -1361,27 +1491,34 @@ export class PermissionService extends PuterService {
* business.
*
* Returns the permissions removed. Each is announced separately, because
* each is a distinct thing something may have been standing on.
* each is a distinct thing something may have been standing on. Several
* roots go in one delete and one generation bump.
*/
async revokeUserUserPermissionSubtree(
actor: Actor,
username: string,
permission: string,
permission: string | readonly string[],
meta: GrantMeta = {},
): Promise<string[]> {
permission = await this.rewritePermissionForActor(actor, permission);
const roots = await Promise.all(
(typeof permission === 'string' ? [permission] : permission).map(
(root) => this.rewritePermissionForActor(actor, root),
),
);
const user = await this.#requireUser(username);
const issuerId = this.#requireUserId(actor);
if (user.id !== issuerId) {
await this.#assertCanManage(actor, permission);
await Promise.all(
roots.map((root) => this.#assertCanManage(actor, root)),
);
}
const removed =
await this.stores.permission.deleteUserUserPermSubtreeForHolder(
user.id,
issuerId,
permission,
roots,
);
for (const removedPermission of removed) {
@@ -0,0 +1,179 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { parseAppGrantablePermission } from './appGrantable.js';
const ME = '2f5d8a8e-6b8f-4a43-9a0b-3d6f3f0f9b11';
const OTHER = '9c1e4b7a-1f2d-4c3b-8a5e-6d7f8091a2b3';
const APP = 'app-5d4f2c1e-8b3a-4f6d-9e7c-1a2b3c4d5e6f';
const FILE = 'b3c1d2e4-5f6a-4b7c-8d9e-0f1a2b3c4d5e';
const parse = (permission: string) =>
parseAppGrantablePermission(permission, ME);
describe('parseAppGrantablePermission', () => {
it.each([
[
`fs:/alice/Documents:read`,
{ family: 'fs', target: '/alice/Documents', mode: 'read' },
],
[`fs:${FILE}:write`, { family: 'fs', target: FILE, mode: 'write' }],
[
`fs:/alice/a\\Cb.txt:see`,
{ family: 'fs', target: '/alice/a:b.txt', mode: 'see' },
],
[
`fs:/alice/a\\\\b.txt:read`,
{ family: 'fs', target: '/alice/a\\b.txt', mode: 'read' },
],
[`fs:/alice:list`, { family: 'fs', target: '/alice', mode: 'list' }],
[`user:${ME}:email:read`, { family: 'email' }],
[`apps-of-user:${ME}:read`, { family: 'apps', access: 'read' }],
[
`subdomains-of-user:${ME}:write`,
{ family: 'subdomains', access: 'write' },
],
[`app-data:${APP}`, { family: 'appData', appUid: APP }],
[`app-data:${APP}:kv`, { family: 'appData', appUid: APP, store: 'kv' }],
[
`app-data:${APP}:kv:expireAt`,
{ family: 'appData', appUid: APP, store: 'kv', scope: 'expireAt' },
],
[
`app-data:${APP}:fs:delete`,
{ family: 'appData', appUid: APP, store: 'fs', scope: 'delete' },
],
[
`app-root-dir:${APP}:write`,
{ family: 'appRootDir', appUid: APP, access: 'write' },
],
[
`manage:kv-share:${ME}:${APP}:rooms:abc`,
{
family: 'kvShare',
level: 'watch',
ownerUuid: ME,
appUid: APP,
segments: ['rooms', 'abc'],
},
],
[
`manage:kv-share-read:${ME}:${APP}:rooms:abc`,
{
family: 'kvShare',
level: 'read',
ownerUuid: ME,
appUid: APP,
segments: ['rooms', 'abc'],
},
],
[
`manage:kv-share-write:${ME}:${APP}:rooms`,
{
family: 'kvShare',
level: 'write',
ownerUuid: ME,
appUid: APP,
segments: ['rooms'],
},
],
['events:background', { family: 'backgroundEvents' }],
])('accepts %s', (permission, parsed) => {
expect(parse(permission)).toEqual(parsed);
});
it.each([
// Never checked for an app, so a grant would only confuse the prompt.
'service:ai-chat:ii:puter-chat-completion',
'driver:puter-image-generation:generate',
'thread:abc:post',
'site:example',
'feature',
// Internal or wider than any prompt can describe.
`manage:fs:${FILE}:write`,
`manage:fs:/alice/Documents`,
`kv-share:${ME}:${APP}:rooms`,
`kv-share-read:${ME}:${APP}:rooms`,
`kv-share-write:${ME}:${APP}:rooms`,
`app:uid#${APP}:access`,
'flag:app-is-authenticated',
'full-api-access',
'*',
'events',
'events:background:extra',
// Modeless, or a mode that does not exist.
'fs',
`fs:${FILE}`,
`fs:${FILE}:delete`,
`fs:${FILE}:read:extra`,
'fs:relative/path:read',
'fs:not-a-uuid:read',
// Another user's data.
`user:${OTHER}:email:read`,
`apps-of-user:${OTHER}:read`,
`subdomains-of-user:${OTHER}:write`,
`manage:kv-share:${OTHER}:${APP}:rooms`,
`manage:kv-share-read:${OTHER}:${APP}:rooms`,
`manage:kv-share-write:${OTHER}:${APP}:rooms`,
// Shape errors inside an allowed family.
`user:${ME}:email`,
`user:${ME}:email:write`,
`apps-of-user:${ME}`,
`apps-of-user:${ME}:manage`,
'app-data',
'app-data:',
`app-data:${APP}:sql:read`,
`app-data:${APP}:kv:flush`,
`app-data:${APP}:fs:get`,
`app-data:${APP}:kv:get:extra`,
'app-data:app-nope/:kv:read',
'app-data:not-an-app:kv:read',
`app-data:${APP}:`,
`app-data:${APP}:kv:`,
`app-data:${APP}::read`,
`app-root-dir:${APP}`,
`app-root-dir:${APP}:delete`,
'app-root-dir:not-an-app:read',
`app-root-dir:${APP}:read:extra`,
`apps-of-user:${ME}:read:extra`,
// Spelled other than canonically, which later checks would not match.
'f\\s:/x:read',
'fs:/alice/a\\b.txt:read',
`app-dat\\a:${APP}:kv:read`,
`app-root-di\\r:${APP}:read`,
`manag\\e:kv-share:${ME}:${APP}:rooms`,
`manage:kv-shar\\e:${ME}:${APP}:rooms`,
`manage:kv-share\\Crooms:${ME}:${APP}:abc`,
'event\\s:background',
`manage:kv-share:${ME}:${APP}`,
`manage:kv-share:${ME}:${APP}:rooms::abc`,
`manage:kv-share:${ME}:not-an-app:rooms`,
`manage:kv-share-read:${ME}:${APP}`,
`manage:kv-share-write:${ME}:${APP}:rooms::abc`,
`manage:kv-share-delete:${ME}:${APP}:rooms`,
// Prototype members are not families.
`constructor:${ME}:read`,
`manage:toString:${ME}:${APP}:rooms`,
'random-string',
'',
])('refuses %s', (permission) => {
expect(parse(permission)).toBeNull();
});
});
@@ -0,0 +1,198 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { validate as validateUuid } from 'uuid';
import {
APP_UID_SHAPE,
KV_SHARE_ACCESS_LEVELS,
type KvShareAccess,
kvShareRoot,
} from '../kvShare/kvShares.js';
import {
APP_DATA_CLASSES,
APP_DATA_KV_OPS,
APP_DATA_PERMISSION_PREFIX,
type AppDataStore,
} from './appDataScopes.js';
import { MANAGE_PERM_PREFIX } from './consts.js';
import { PermissionUtil } from './permissionUtil.js';
const ACCESS_LEVELS = ['read', 'write'] as const;
type AccessLevel = (typeof ACCESS_LEVELS)[number];
const FS_MODES = ['see', 'list', 'read', 'write'] as const;
type FsMode = (typeof FS_MODES)[number];
/**
* A permission a user may grant an app through the consent prompt, parsed. No
* app-actor check reads `apps-of-user:<uuid>:read` or `subdomains-of-user`;
* they stay grantable because apps already request them.
*/
export type AppGrantablePermission =
| { family: 'fs'; target: string; mode: FsMode }
| { family: 'email' }
| { family: 'apps' | 'subdomains'; access: AccessLevel }
| {
family: 'appData';
appUid: string;
store?: AppDataStore;
scope?: string;
}
| { family: 'appRootDir'; appUid: string; access: AccessLevel }
| {
family: 'kvShare';
level: KvShareAccess;
ownerUuid: string;
appUid: string;
segments: string[];
}
| { family: 'backgroundEvents' };
/** One component as it is written canonically: `\` as `\\`, `:` as `\C`. */
const escapeComponent = (component: string): string =>
component.replace(/\\/g, '\\\\').replace(/:/g, '\\C');
const isOneOf = <T extends string>(
values: readonly T[],
value: string | undefined,
): value is T =>
value !== undefined && (values as readonly string[]).includes(value);
const ACCESS_FAMILIES: Record<string, 'apps' | 'subdomains'> = {
'apps-of-user': 'apps',
'subdomains-of-user': 'subdomains',
};
/** Own keys only, so `constructor` or `toString` never names a family. */
const lookup = <T>(map: Record<string, T>, key: string | undefined) =>
key !== undefined && Object.hasOwn(map, key) ? map[key] : undefined;
const parseAppData = (parts: string[]): AppGrantablePermission | null => {
const [, appUid, store, scope, ...rest] = parts;
if (!appUid || !APP_UID_SHAPE.test(appUid) || rest.length > 0) return null;
if (store === undefined) return { family: 'appData', appUid };
if (store === 'kv') {
if (
scope !== undefined &&
!isOneOf(APP_DATA_CLASSES, scope) &&
!isOneOf(APP_DATA_KV_OPS, scope)
) {
return null;
}
} else if (store === 'fs') {
if (scope !== undefined && !isOneOf(APP_DATA_CLASSES, scope)) {
return null;
}
} else {
return null;
}
return { family: 'appData', appUid, store, scope };
};
/**
* `permission` as an app-grantable permission, or null when a user may not
* grant it to an app. Families naming a user (email, apps, subdomains, share
* delegation) must name `userUuid`, the user granting it.
*/
export const parseAppGrantablePermission = (
permission: string,
userUuid: string,
): AppGrantablePermission | null => {
if (permission === 'events:background') {
return { family: 'backgroundEvents' };
}
const parts = PermissionUtil.split(permission);
// Checks that run on the raw string, after this one, match only the
// canonical spelling, so `f\s:` must not reach them as `fs:`.
if (parts.map(escapeComponent).join(':') !== permission) return null;
const [head] = parts;
if (head === 'fs') {
const [, target, mode] = parts;
if (parts.length !== 3 || !target || !isOneOf(FS_MODES, mode)) {
return null;
}
if (!target.startsWith('/') && !validateUuid(target)) return null;
return { family: 'fs', target, mode };
}
if (head === 'user') {
const [, uuid, field, mode] = parts;
if (
parts.length !== 4 ||
uuid !== userUuid ||
field !== 'email' ||
mode !== 'read'
) {
return null;
}
return { family: 'email' };
}
const accessFamily = lookup(ACCESS_FAMILIES, head);
if (accessFamily) {
const [, uuid, access] = parts;
if (
parts.length !== 3 ||
uuid !== userUuid ||
!isOneOf(ACCESS_LEVELS, access)
) {
return null;
}
return { family: accessFamily, access };
}
if (head === APP_DATA_PERMISSION_PREFIX) return parseAppData(parts);
if (head === 'app-root-dir') {
const [, appUid, access] = parts;
if (
parts.length !== 3 ||
!appUid ||
!APP_UID_SHAPE.test(appUid) ||
!isOneOf(ACCESS_LEVELS, access)
) {
return null;
}
return { family: 'appRootDir', appUid, access };
}
const level =
head === MANAGE_PERM_PREFIX
? KV_SHARE_ACCESS_LEVELS.find(
(access) => kvShareRoot(access) === parts[1],
)
: undefined;
if (level) {
const [, , ownerUuid, appUid, ...segments] = parts;
if (
ownerUuid !== userUuid ||
!appUid ||
!APP_UID_SHAPE.test(appUid) ||
segments.length === 0 ||
segments.some((segment) => segment === '')
) {
return null;
}
return { family: 'kvShare', level, ownerUuid, appUid, segments };
}
return null;
};
@@ -211,6 +211,11 @@ export interface ImplicatorCheckInput {
actor: Actor;
permission: string;
recurse?: (actor: Actor, permission: string) => Promise<ReadingNode[]>;
/**
* Shared by every implicator call of one scan, so a rule can answer the
* options of one check from a single lookup. Never outlives the scan.
*/
memo?: Map<string, unknown>;
}
export interface PermissionImplicator {
@@ -43,7 +43,8 @@ import {
} from './shareDeepLink';
import { NOTIFY_FANOUT_CAP } from '../../core/teams.js';
import { abuseKey, cleanEmail, isBlockedEmail } from '../../util/email.js';
import { blocksAllShares, type ResolvedShare } from './ShareService';
import { blocksAllShares } from './shareBlocks.js';
import type { ResolvedShare } from './ShareService';
import { withRedisLock } from '../../util/redisLock.js';
/**
@@ -2469,6 +2469,52 @@ describe('ShareService', () => {
);
});
it('files a recipient leaving under the issuer, naming who acted', async () => {
const owner = await makeUser();
const recipient = await makeUser();
const file = await makeFile(owner.user);
await share(owner.actor, {
uid: file.uuid,
recipient: { email: recipient.email },
mode: 'read',
});
await unshare(recipient.actor, {
uid: file.uuid,
recipient: { email: recipient.email },
});
const rows = await vi.waitFor(async () => {
const found = await server.clients.db.read(
'SELECT `issuer_user_id`, `extra` FROM `audit_user_to_user_permissions` ' +
'WHERE `holder_user_id` = ? AND `action` = ?',
[recipient.user.id, 'revoke'],
);
expect(found.length).toBeGreaterThan(0);
return found;
});
for (const row of rows) {
expect(Number(row.issuer_user_id)).toBe(owner.user.id);
const extra =
typeof row.extra === 'string'
? JSON.parse(row.extra)
: row.extra;
expect(extra?.actingUserId).toBe(recipient.user.id);
}
const ownerTrail = await audit(owner.actor, { uid: file.uuid });
expect(
ownerTrail.items.some(
(i) =>
i.action === 'revoke' &&
i.issuer.username === owner.user.username,
),
).toBe(true);
const recipientTrail = await audit(recipient.actor);
expect(
recipientTrail.items.some((i) => i.entryUid === file.uuid),
).toBe(false);
});
it('lists what the caller granted when no item is named', async () => {
const owner = await makeUser();
const other = await makeUser();
+21 -28
View File
@@ -59,7 +59,14 @@ import { carriesClosedAppData } from '../permission/appDataScopes';
import { MANAGE_PERM_PREFIX } from '../permission/consts';
import { PermissionUtil } from '../permission/permissionUtil.js';
import { SubscriptionCache } from '../events/subscriptionCache.js';
import { KV_SHARE_ACCESS_LEVELS, kvShareRoot } from '../kvShare/kvShares.js';
import { PuterService } from '../types';
import {
BLOCK_ALL_SHARES_KEY,
blocksAllShares,
recipientNotAcceptingShares,
refusesSharesFrom,
} from './shareBlocks.js';
// -- Types ------------------------------------------------------------
@@ -288,21 +295,6 @@ const RETIRE_CHUNK_SIZE = 100;
/** Shares one user may create per UTC day, absent a config override. */
export const DEFAULT_DAILY_SHARE_LIMIT = 200;
/**
* Where "refuse shares from everyone" lives on the user row.
*
* A key in the existing `metadata` blob rather than a column of its own: the
* share path already holds the recipient's row by the time it asks, so reading
* it costs nothing either way, and a one-bit preference doesn't earn a
* migration per dialect.
*/
const BLOCK_ALL_SHARES_KEY = 'blockAllShares';
/** Whether this account refuses shares from everyone. */
export const blocksAllShares = (
user: Pick<UserRow, 'metadata'> | null,
): boolean => Boolean(user?.metadata?.[BLOCK_ALL_SHARES_KEY]);
/**
* What a share recipient's browser is told about someone else's entry.
*
@@ -2317,13 +2309,19 @@ export class ShareService extends PuterService {
}> {
const userId = this.#requireUserId(actor);
// A key-value share's grant names its owner and prefix, which its
// holder is never told, so only the owner sees those rows.
const ownedRoots = {
roots: KV_SHARE_ACCESS_LEVELS.map(kvShareRoot),
ownerUuid: actor.user.uuid ?? '',
};
let filter: UserUserAuditFilter;
if (target.uid || target.path) {
const entry = await this.#resolveEntry(target, actor);
await this.#assertCanManage(actor, entry);
filter = { permissions: entryPermissions(entry.uuid) };
filter = { permissions: entryPermissions(entry.uuid), ownedRoots };
} else {
filter = { issuerUserId: userId };
filter = { issuerUserId: userId, ownedRoots };
}
const page = await this.stores.permission.listUserUserAudit(filter, {
@@ -2727,13 +2725,8 @@ export class ShareService extends PuterService {
* switch is free; only a caller who cleared it pays for the pair lookup.
*/
async #assertNotBlocked(issuer: number, holder: UserRow): Promise<void> {
const blocked =
blocksAllShares(holder) ||
(await this.stores.userBlock.isBlocked(holder.id, issuer));
if (!blocked) return;
throw new HttpError(403, 'recipient is not accepting shares', {
legacyCode: 'recipient_not_accepting_shares',
});
if (await refusesSharesFrom(this.stores.userBlock, holder, issuer))
throw recipientNotAcceptingShares();
}
// -- Internals ----------------------------------------------------
@@ -2801,11 +2794,11 @@ export class ShareService extends PuterService {
// stopped accepting shares — from this sender, or from anyone
// — since it was sent.
if (
blocksAllShares(holder) ||
(await this.stores.userBlock.isBlocked(
holderUserId,
await refusesSharesFrom(
this.stores.userBlock,
holder,
Number(row.issuer_user_id),
))
)
) {
await this.stores.share.deleteByUid(row.uid);
continue;
+58
View File
@@ -0,0 +1,58 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { HttpError } from '../../core/http/HttpError.js';
import type { UserRow } from '../../stores/user/UserStore.js';
import type { UserBlockStore } from '../../stores/userBlock/UserBlockStore.js';
/**
* Where "refuse shares from everyone" lives on the user row.
*
* A key in the existing `metadata` blob rather than a column of its own: the
* share path already holds the recipient's row by the time it asks, so reading
* it costs nothing either way, and a one-bit preference doesn't earn a
* migration per dialect.
*/
export const BLOCK_ALL_SHARES_KEY = 'blockAllShares';
/** Whether this account refuses shares from everyone. */
export const blocksAllShares = (
user: Pick<UserRow, 'metadata'> | null,
): boolean => Boolean(user?.metadata?.[BLOCK_ALL_SHARES_KEY]);
/**
* Whether `holder` refuses shares from `issuerId`: from everyone, or from that
* sender. The blanket switch is on the row already in hand, so only a holder
* who cleared it costs the pair lookup.
*/
export const refusesSharesFrom = async (
userBlock: Pick<UserBlockStore, 'isBlocked'>,
holder: Pick<UserRow, 'id' | 'metadata'>,
issuerId: number,
): Promise<boolean> =>
blocksAllShares(holder) || (await userBlock.isBlocked(holder.id, issuerId));
/**
* Refusing everyone and refusing this sender read the same: which of the two it
* is is the recipient's business, not the sender's.
*/
export const recipientNotAcceptingShares = (): HttpError =>
new HttpError(403, 'recipient is not accepting shares', {
legacyCode: 'recipient_not_accepting_shares',
});
@@ -27,6 +27,9 @@ import { PuterService } from '../types.js';
*/
const PROVISIONED_FSENTRY_COUNT = 8;
/** Apps read per page when listing the ones an account owns. */
const OWNED_APPS_PAGE = 500;
/**
* Account-lifecycle operations that more than one caller needs: deleting an
* account and everything hanging off it, and measuring whether an account has
@@ -101,8 +104,16 @@ export class UserAccountService extends PuterService {
console.warn('[cascade-delete-user] subdomain cleanup failed:', e);
}
// The user's apps cascade away with the row; record them first so a
// hosted origin doesn't hand their uids to the subdomain's next owner.
// The user's apps cascade away with the row, with no `app.changed`, so
// `user.delete` names them for what was tied to them.
let appUids: string[] = [];
try {
appUids = await this.#ownedAppUids(userId);
} catch (e) {
console.warn('[cascade-delete-user] owned-app lookup failed:', e);
}
// Record them first so a hosted origin doesn't hand their uids to the
// subdomain's next owner.
await this.stores.app.recordDeletedForOwner(userId);
// Other users' sites and workers tied to those apps cascade too.
const appSites = await this.stores.subdomain.listAppCascade({
@@ -143,6 +154,7 @@ export class UserAccountService extends PuterService {
user_id: userId,
user_uuid: userUuid,
stripe_customer_id: stripeCustomerId,
appUids,
},
{},
);
@@ -164,6 +176,22 @@ export class UserAccountService extends PuterService {
}
}
/** The uids of every app this user owns, a page at a time. */
async #ownedAppUids(userId: number): Promise<string[]> {
const uids: string[] = [];
let afterId: number | undefined;
for (;;) {
const apps = (await this.stores.app.list({
ownerUserId: userId,
afterId,
limit: OWNED_APPS_PAGE,
})) as Array<{ id: number; uid: string }>;
uids.push(...apps.map((app) => app.uid));
if (apps.length < OWNED_APPS_PAGE) return uids;
afterId = apps[apps.length - 1]!.id;
}
}
/**
* Delete the user's sites one by one, emitting `subdomain.delete` for each
* as the driver does. The FK cascade would drop the rows silently, leaving
+1
View File
@@ -520,6 +520,7 @@ export class AppStore extends PuterStore {
* @param {number} [filters.ownerUserId] — only apps owned by this user
* @param {number} [filters.appOwner] — only apps created by another app
* @param {string} [filters.name] — exact name match
* @param {number} [filters.afterId] — only apps with a greater `id`
* @param {number} [filters.limit=500] Default is `500`
*/
async list(filters = {}) {
@@ -20,9 +20,9 @@
/**
* The store's own guard against a row wider than the column that holds it.
*
* `EventsService.mintKvHandle` already bounds `keyPrefix` and `permission` well
* under these widths before the store ever sees them, so reaching the guard
* here means calling the store directly.
* `KvShareService.mintKvHandle` already bounds `keyPrefix` and `permission`
* well under these widths before the store ever sees them, so reaching the
* guard here means calling the store directly.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
@@ -128,3 +128,134 @@ describe('the owner listing', () => {
expect(second.cursor).toBeUndefined();
});
});
describe('retiring an app`s handles', () => {
it('works through them a batch at a time, as the platform', async () => {
const appUid = 'app-retire-batches';
const minted = [];
for (let i = 0; i < 3; i++)
minted.push(
await store().mint(
input({
appUid,
keyPrefix: `batch-${i}:`,
permission: `kv-share:owner-uuid:${appUid}:batch-${i}`,
}),
),
);
const batches: number[] = [];
for (;;) {
const batch = await store().listLiveForApps([appUid], 2);
batches.push(batch.length);
if (batch.length === 0) break;
await store().retireMany(
batch.map((row) => row.handle),
'system',
);
}
expect(batches).toEqual([2, 1, 0]);
for (const { handle } of minted)
expect(await store().getByHandle(handle)).toMatchObject({
revokedBy: 'system',
revokedAt: expect.any(Number),
});
});
});
describe('changing a live handle', () => {
it('never brings a retired one back', async () => {
const row = await store().mint(input({ keyPrefix: 'cas:' }));
await store().retire(row.handle, ownerUserId);
const after = await store().update(row.handle, ownerUserId, {
permission: 'kv-share-write:owner-uuid:app-column-widths:cas',
label: 'late',
});
expect(after).toMatchObject({
revokedAt: expect.any(Number),
permission: row.permission,
label: null,
});
});
it('is scoped to the owner', async () => {
const row = await store().mint(input({ keyPrefix: 'scoped:' }));
expect(
await store().update(row.handle, granteeUserId, { label: 'x' }),
).toBeNull();
expect((await store().getByHandle(row.handle))?.label).toBeNull();
});
});
describe('finding the live handle on a region', () => {
it('matches the prefix byte for byte, and reports one that differs only in case', async () => {
const appUid = `app-case-${Date.now().toString(36)}`;
const region = { ownerUserId, granteeUserId, appUid };
const minted = await store().mint(
input({
appUid,
keyPrefix: 'Notes:',
permission: `kv-share:o:${appUid}:Notes`,
}),
);
expect(
await store().findLive({ ...region, keyPrefix: 'Notes:' }),
).toMatchObject({ handle: minted.handle });
expect(
await store().findLive({ ...region, keyPrefix: 'notes:' }),
).toBeNull();
expect(
await store().hasLiveCaseVariant({
...region,
keyPrefix: 'notes:',
}),
).toBe(true);
expect(
await store().hasLiveCaseVariant({
...region,
keyPrefix: 'Notes:',
}),
).toBe(false);
expect(
await store().hasLiveCaseVariant({
...region,
keyPrefix: 'other:',
}),
).toBe(false);
await store().retire(minted.handle, ownerUserId);
expect(
await store().hasLiveCaseVariant({
...region,
keyPrefix: 'notes:',
}),
).toBe(false);
});
});
describe('the holder listing', () => {
it('carries no owner, prefix or grant string', async () => {
const row = await store().mint(
input({
appUid: 'app-holder-listing',
keyPrefix: 'secret-prefix:',
permission:
'kv-share-read:owner-uuid:app-holder-listing:secret-prefix',
label: 'Shown',
}),
);
const page = await store().listLiveForGrantee(granteeUserId, {
appUid: 'app-holder-listing',
});
expect(page.items).toEqual([
{
handle: row.handle,
appUid: 'app-holder-listing',
access: 'read',
label: 'Shown',
createdAt: row.createdAt,
},
]);
});
});
+333 -32
View File
@@ -17,7 +17,13 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { mintKvHandleId } from '../../services/events/kvShares.js';
import {
KV_SHARE_ACCESS_LEVELS,
kvShareRoot,
mintKvHandleId,
type KvShareAccess,
} from '../../services/kvShare/kvShares.js';
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
import {
keysetPage,
openIdCursor,
@@ -40,6 +46,11 @@ const TABLE = 'kv_share_handles';
export const KV_HANDLE_LIST_DEFAULT_LIMIT = 50;
export const KV_HANDLE_LIST_LIMIT_CAP = 200;
/** Who retired a handle: its owner, its holder, or the platform. */
export type KvShareRevokedBy = 'owner' | 'holder' | 'system';
const REVOKED_BY: readonly KvShareRevokedBy[] = ['owner', 'holder', 'system'];
export interface KvShareHandle {
handle: string;
ownerUserId: number;
@@ -47,10 +58,39 @@ export interface KvShareHandle {
appUid: string;
/** The granted root, ending on the key delimiter. */
keyPrefix: string;
/** The user-to-user grant this handle mirrors. */
/** The user-to-user grant this handle mirrors; its root is the level. */
permission: string;
/** Owner-set text the holder is shown, or `null`. */
label: string | null;
createdAt: number;
revokedAt: number | null;
/** `null` while live, and on rows retired before this was recorded. */
revokedBy: KvShareRevokedBy | null;
}
/** A live handle as the hot path needs it: the row and its owner's state. */
export interface LiveKvShareHandle {
row: Omit<KvShareHandle, 'label'>;
ownerUuid: string;
ownerSuspended: boolean;
}
/**
* A handle as its holder may know it. Selected without the owner, the prefix or
* the grant string, so none of them can reach a holder-facing response.
*/
export interface HeldKvShareHandle {
handle: string;
appUid: string;
access: KvShareAccess;
label: string | null;
createdAt: number;
}
/** What a level or label change writes; an absent field is left alone. */
export interface KvShareHandleUpdate {
permission?: string;
label?: string | null;
}
export interface KvShareHandleListOptions {
@@ -67,6 +107,7 @@ export interface MintKvShareHandleInput {
appUid: string;
keyPrefix: string;
permission: string;
label?: string | null;
}
export interface FindLiveKvShareHandleInput {
@@ -76,8 +117,21 @@ export interface FindLiveKvShareHandleInput {
keyPrefix: string;
}
/** Handle cache bounds: live rows, and recently missed names. */
const LIVE_CACHE_TTL_MS = 60_000;
const MISS_CACHE_TTL_MS = 5_000;
const LIVE_CACHE_MAX_ENTRIES = 10_000;
/** Rows one app-deletion retire pass takes. */
export const KV_HANDLE_RETIRE_BATCH = 500;
const nowSeconds = (): number => Math.floor(Date.now() / 1000);
const toRevokedBy = (value: unknown): KvShareRevokedBy | null =>
REVOKED_BY.includes(value as KvShareRevokedBy)
? (value as KvShareRevokedBy)
: null;
const toRow = (row: Record<string, unknown>): KvShareHandle => ({
handle: String(row.handle),
ownerUserId: Number(row.owner_user_id),
@@ -85,15 +139,50 @@ const toRow = (row: Record<string, unknown>): KvShareHandle => ({
appUid: String(row.app_uid),
keyPrefix: String(row.key_prefix),
permission: String(row.permission),
label:
row.label === null || row.label === undefined
? null
: String(row.label),
createdAt: Number(row.created_at) || 0,
revokedAt: row.revoked_at === null ? null : Number(row.revoked_at),
revokedBy: toRevokedBy(row.revoked_by),
});
const SELECT_COLUMNS =
'`handle`, `owner_user_id`, `grantee_user_id`, `app_uid`, ' +
'`key_prefix`, `permission`, `created_at`, `revoked_at`';
'`key_prefix`, `permission`, `label`, `created_at`, `revoked_at`, ' +
'`revoked_by`';
/**
* The level as SQL reads it off the grant's root, so the holder listing can
* report it without selecting the grant string itself.
*/
const ACCESS_COLUMN =
`CASE WHEN \`permission\` LIKE '${kvShareRoot('write')}:%' THEN 'write' ` +
`WHEN \`permission\` LIKE '${kvShareRoot('read')}:%' THEN 'read' ` +
"ELSE 'watch' END AS `access`";
const toAccess = (value: unknown): KvShareAccess =>
KV_SHARE_ACCESS_LEVELS.includes(value as KvShareAccess)
? (value as KvShareAccess)
: 'watch';
const clampLimit = (limit: number | undefined): number =>
Math.min(
Math.max(1, Math.floor(limit ?? KV_HANDLE_LIST_DEFAULT_LIMIT)),
KV_HANDLE_LIST_LIMIT_CAP,
);
export class KvShareHandleStore extends PuterStore {
readonly #live = new BoundedTtlMap<string, LiveKvShareHandle>({
maxEntries: LIVE_CACHE_MAX_ENTRIES,
ttlMs: LIVE_CACHE_TTL_MS,
});
readonly #missing = new BoundedTtlMap<string, true>({
maxEntries: LIVE_CACHE_MAX_ENTRIES,
ttlMs: MISS_CACHE_TTL_MS,
});
async mint(input: MintKvShareHandleInput): Promise<KvShareHandle> {
const row: KvShareHandle = {
handle: mintKvHandleId(),
@@ -102,8 +191,10 @@ export class KvShareHandleStore extends PuterStore {
appUid: input.appUid,
keyPrefix: input.keyPrefix,
permission: input.permission,
label: input.label ?? null,
createdAt: nowSeconds(),
revokedAt: null,
revokedBy: null,
};
assertColumnWidths(KV_SHARE_HANDLE_WIDTHS, {
handle: row.handle,
@@ -119,12 +210,56 @@ export class KvShareHandleStore extends PuterStore {
app_uid: row.appUid,
key_prefix: row.keyPrefix,
permission: row.permission,
label: row.label,
created_at: row.createdAt,
revoked_at: null,
});
this.#forget(row.handle);
return row;
}
/**
* A live handle and its owner's state, cached in-process (hits up to 60 s,
* misses 5 s). Authority is the grant, checked on every call, so a stale
* row only keeps an address alive a little longer, never widens it.
*/
async getLive(handle: string): Promise<LiveKvShareHandle | null> {
const cached = this.#live.get(handle);
if (cached) return cached;
if (this.#missing.has(handle)) return null;
const query =
'SELECT h.`handle`, h.`owner_user_id`, h.`grantee_user_id`, ' +
'h.`app_uid`, h.`key_prefix`, h.`permission`, h.`created_at`, ' +
'h.`revoked_at`, h.`revoked_by`, ' +
'u.`uuid` AS `owner_uuid`, u.`suspended` AS `owner_suspended` ' +
`FROM \`${TABLE}\` h JOIN \`user\` u ON u.\`id\` = h.\`owner_user_id\` ` +
'WHERE h.`handle` = ?';
let rows = await this.clients.db.read(query, [handle]);
if (rows.length === 0)
rows = await this.clients.db.pread(query, [handle]);
const found = rows[0];
if (!found || found.revoked_at !== null || !found.owner_uuid) {
this.#missing.set(handle, true);
return null;
}
const { label: _label, ...row } = toRow({ ...found, label: null });
const live: LiveKvShareHandle = {
row,
ownerUuid: String(found.owner_uuid),
ownerSuspended: Boolean(Number(found.owner_suspended ?? 0)),
};
this.#live.set(handle, live);
return live;
}
/** Drops this node's cached copy; other nodes converge within the TTL. */
#forget(handle: string): void {
this.#live.delete(handle);
this.#missing.delete(handle);
}
/**
* Live handles this owner is holding out — in total, and in the namespace a
* mint is naming. Both ceilings read off one count, over the owner index.
@@ -155,10 +290,17 @@ export class KvShareHandleStore extends PuterStore {
async findLive(
input: FindLiveKvShareHandleInput,
): Promise<KvShareHandle | null> {
// Byte-exact, so `Notes:` and `notes:` are two regions everywhere. The
// collation goes on the utf8mb4 column, so the connection's charset
// doesn't matter.
const exactPrefix = this.clients.db.case({
mysql: '`key_prefix` COLLATE utf8mb4_bin = ?',
otherwise: '`key_prefix` = ?',
});
const rows = await this.clients.db.pread(
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` WHERE ` +
'`owner_user_id` = ? AND `grantee_user_id` = ? AND ' +
'`app_uid` = ? AND `key_prefix` = ? AND `revoked_at` IS NULL ' +
`\`app_uid\` = ? AND ${exactPrefix} AND \`revoked_at\` IS NULL ` +
'LIMIT 1',
[
input.ownerUserId,
@@ -170,10 +312,34 @@ export class KvShareHandleStore extends PuterStore {
return rows.length > 0 ? toRow(rows[0]) : null;
}
/**
* Whether a live handle for the same owner, grantee and app has a prefix
* equal to this one ignoring case but not byte-equal: a grant on it would
* collide with this one's where the database compares case-insensitively.
*/
async hasLiveCaseVariant(
input: FindLiveKvShareHandleInput,
): Promise<boolean> {
const rows = await this.clients.db.pread(
`SELECT \`key_prefix\` FROM \`${TABLE}\` WHERE ` +
'`owner_user_id` = ? AND `grantee_user_id` = ? AND ' +
'`app_uid` = ? AND LOWER(`key_prefix`) = LOWER(?) AND ' +
// At most one live row is byte-exact, so two find a variant.
'`revoked_at` IS NULL LIMIT 2',
[
input.ownerUserId,
input.granteeUserId,
input.appUid,
input.keyPrefix,
],
);
return rows.some((row) => String(row.key_prefix) !== input.keyPrefix);
}
/**
* Every live handle this owner holds out to one grantee, across every
* region — what a revoke walks to find a narrower handle whose grant just
* died as a side effect of a wider one being withdrawn.
* region — what a revoke or an unshare walks to find the narrower handles
* that go with a wider one. Bounded by the per-account handle cap.
*/
async listLiveForOwnerAndGrantee(
ownerUserId: number,
@@ -206,32 +372,173 @@ export class KvShareHandleStore extends PuterStore {
* unknown handle and somebody else's both get, so retiring cannot be used
* to find out that a handle exists.
*
* Idempotent: an already-retired handle keeps the timestamp it has. The
* caller withdraws the grant first and that step can fail, so this one has
* to be safe to reach twice.
*
* The `revoked_at IS NULL` predicate makes the read-then-write a
* compare-and-set: two callers racing means one stamps the row and the
* other reports the stamp it found, rather than both writing.
* Idempotent: an already-retired handle keeps the timestamp it has, so a
* retry after a failed step is safe.
*/
async retire(
handle: string,
ownerUserId: number,
revokedBy: 'owner' | 'system' = 'owner',
): Promise<KvShareHandle | null> {
const existing = await this.getByHandle(handle);
if (!existing || existing.ownerUserId !== ownerUserId) return null;
if (existing.revokedAt !== null) return existing;
await this.retireMany([handle], revokedBy, { ownerUserId });
const row = await this.getByHandle(handle);
return row?.ownerUserId === ownerUserId ? row : null;
}
const at = nowSeconds();
const result = await this.clients.db.write(
`UPDATE \`${TABLE}\` SET \`revoked_at\` = ? ` +
'WHERE `handle` = ? AND `owner_user_id` = ? ' +
'AND `revoked_at` IS NULL',
[at, handle, ownerUserId],
/** {@link retire}, scoped to the holder instead, recorded as theirs. */
async retireAsHolder(
handle: string,
granteeUserId: number,
): Promise<KvShareHandle | null> {
await this.retireMany([handle], 'holder', { granteeUserId });
const row = await this.getByHandle(handle);
return row?.granteeUserId === granteeUserId ? row : null;
}
/**
* Retire several handles in one statement, optionally scoped to their owner
* or holder. A compare-and-set on `revoked_at IS NULL`, so a handle already
* retired keeps its timestamp and who retired it.
*/
async retireMany(
handles: readonly string[],
revokedBy: KvShareRevokedBy,
scope: { ownerUserId?: number; granteeUserId?: number } = {},
): Promise<void> {
const unique = [...new Set(handles)];
if (unique.length === 0) return;
const where = [
`\`handle\` IN (${unique.map(() => '?').join(', ')})`,
'`revoked_at` IS NULL',
];
const params: unknown[] = [nowSeconds(), revokedBy, ...unique];
if (scope.ownerUserId !== undefined) {
where.push('`owner_user_id` = ?');
params.push(scope.ownerUserId);
}
if (scope.granteeUserId !== undefined) {
where.push('`grantee_user_id` = ?');
params.push(scope.granteeUserId);
}
await this.clients.db.write(
`UPDATE \`${TABLE}\` SET \`revoked_at\` = ?, \`revoked_by\` = ? ` +
`WHERE ${where.join(' AND ')}`,
params,
);
if (result?.anyRowsAffected === false)
return await this.getByHandle(handle);
return { ...existing, revokedAt: at };
for (const handle of unique) this.#forget(handle);
}
/**
* One batch of live handles in these apps' namespaces, oldest first. A
* deleted app's handles are retired off this; the caller repeats until a
* batch comes back short.
*/
async listLiveForApps(
appUids: readonly string[],
limit = KV_HANDLE_RETIRE_BATCH,
): Promise<KvShareHandle[]> {
const unique = [...new Set(appUids)];
if (unique.length === 0) return [];
const rows = await this.clients.db.pread(
`SELECT ${SELECT_COLUMNS} FROM \`${TABLE}\` ` +
`WHERE \`app_uid\` IN (${unique.map(() => '?').join(', ')}) ` +
'AND `revoked_at` IS NULL ORDER BY `id` LIMIT ?',
[...unique, limit],
);
return rows.map(toRow);
}
/**
* Change a live handle's level or label in place, scoped to its owner. A
* compare-and-set on `revoked_at IS NULL`, so a handle retired meanwhile
* stays retired; returns the row as it now stands, or `null` for none.
*/
async update(
handle: string,
ownerUserId: number,
changes: KvShareHandleUpdate,
): Promise<KvShareHandle | null> {
const sets: string[] = [];
const params: unknown[] = [];
if (changes.permission !== undefined) {
assertColumnWidths(KV_SHARE_HANDLE_WIDTHS, {
permission: changes.permission,
});
sets.push('`permission` = ?');
params.push(changes.permission);
}
if (changes.label !== undefined) {
sets.push('`label` = ?');
params.push(changes.label);
}
if (sets.length > 0) {
await this.clients.db.write(
`UPDATE \`${TABLE}\` SET ${sets.join(', ')} ` +
'WHERE `handle` = ? AND `owner_user_id` = ? ' +
'AND `revoked_at` IS NULL',
[...params, handle, ownerUserId],
);
}
this.#forget(handle);
const row = await this.getByHandle(handle);
return row && row.ownerUserId === ownerUserId ? row : null;
}
/**
* Live handles held by one user, newest grants last, keyset-paginated on
* the grantee index. Selected without the owner, prefix or grant string:
* this is the holder's listing, and none of those are theirs to see.
*/
async listLiveForGrantee(
granteeUserId: number,
options: KvShareHandleListOptions = {},
): Promise<PageResult<HeldKvShareHandle>> {
const limit = clampLimit(options.limit);
const after = openIdCursor(options.cursor, this.config.jwt_secret_v2);
const filter = ['`grantee_user_id` = ?', '`revoked_at` IS NULL'];
const filterParams: unknown[] = [granteeUserId];
if (options.appUid !== undefined && options.appUid !== null) {
filter.push('`app_uid` = ?');
filterParams.push(options.appUid);
}
const where = [...filter];
const params = [...filterParams];
if (after !== null) {
where.push('`id` > ?');
params.push(after);
}
const rows = await this.clients.db.read(
'SELECT `id`, `handle`, `app_uid`, `label`, `created_at`, ' +
`${ACCESS_COLUMN} FROM \`${TABLE}\` ` +
`WHERE ${where.join(' AND ')} ORDER BY \`id\` LIMIT ?`,
[...params, limit + 1],
);
const page = keysetPage(rows, limit, this.config.jwt_secret_v2);
const result: PageResult<HeldKvShareHandle> = {
items: page.rows.map((row) => ({
handle: String(row.handle),
appUid: String(row.app_uid),
access: toAccess(row.access),
label:
row.label === null || row.label === undefined
? null
: String(row.label),
createdAt: Number(row.created_at) || 0,
})),
};
if (page.cursor) result.cursor = page.cursor;
if (options.includeTotal) {
const [count] = await this.clients.db.read(
`SELECT COUNT(*) AS \`total\` FROM \`${TABLE}\` ` +
`WHERE ${filter.join(' AND ')}`,
filterParams,
);
result.total = Number(count?.total ?? 0);
}
return result;
}
/**
@@ -242,13 +549,7 @@ export class KvShareHandleStore extends PuterStore {
ownerUserId: number,
options: KvShareHandleListOptions = {},
): Promise<PageResult<KvShareHandle>> {
const limit = Math.min(
Math.max(
1,
Math.floor(options.limit ?? KV_HANDLE_LIST_DEFAULT_LIMIT),
),
KV_HANDLE_LIST_LIMIT_CAP,
);
const limit = clampLimit(options.limit);
const after = openIdCursor(options.cursor, this.config.jwt_secret_v2);
const filter = ['`owner_user_id` = ?'];
@@ -56,7 +56,8 @@ export class NotificationStore extends PuterStore {
/**
* A mailbox, newest first. `scope` narrows to one audience/app slice on the
* way out of the database — `appUid: null` means the rows naming no app,
* which is not the same question as "any app".
* which is not the same question as "any app". `excludeTypes` leaves rows
* of those types out.
*
* @param {number} userId
* @param {{
@@ -67,6 +68,7 @@ export class NotificationStore extends PuterStore {
* audiences: readonly string[];
* appUid: string | null;
* } | null;
* excludeTypes?: readonly string[];
* }} [opts]
*/
async listByUserId(
@@ -76,11 +78,19 @@ export class NotificationStore extends PuterStore {
onlyUnacknowledged = false,
filter = undefined,
scope = null,
excludeTypes = [],
} = {},
) {
const where = ['`user_id` = ?'];
const params = [userId];
if (excludeTypes.length > 0) {
where.push(
`\`type\` NOT IN (${excludeTypes.map(() => '?').join(', ')})`,
);
params.push(...excludeTypes);
}
if (onlyUnacknowledged || filter === 'unacknowledged') {
where.push('`acknowledged` IS NULL');
} else if (filter === 'unseen') {
@@ -127,6 +137,7 @@ export class NotificationStore extends PuterStore {
* about no app, which is not the same question as "any app". `undefined` is
* "any app": a session's own generic slice, which spans every app the
* audience predicate already grants it rather than one named ref.
* `excludeTypes` leaves rows of those types out.
*
* @param {number} userId
* @param {{
@@ -134,15 +145,25 @@ export class NotificationStore extends PuterStore {
* appUid?: string | null;
* after?: number | null;
* limit?: number;
* excludeTypes?: readonly string[];
* }} opts
*/
async listScoped(userId, { audience, appUid, after = null, limit = 50 }) {
async listScoped(
userId,
{ audience, appUid, after = null, limit = 50, excludeTypes = [] },
) {
const n = Number(limit);
const safeLimit = Number.isFinite(n) ? Math.max(0, Math.floor(n)) : 50;
if (safeLimit === 0) return [];
const where = ['`user_id` = ?', '`audience` = ?'];
const params = [userId, audience];
if (excludeTypes.length > 0) {
where.push(
`\`type\` NOT IN (${excludeTypes.map(() => '?').join(', ')})`,
);
params.push(...excludeTypes);
}
if (appUid === null) {
where.push('`app_uid` IS NULL');
} else if (appUid !== undefined) {
@@ -22,8 +22,10 @@ import { readFileSync } from 'fs';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { PuterServer } from '../../server.ts';
import { PermissionUtil } from '../../services/permission/permissionUtil.ts';
import { setupTestServer } from '../../testUtil.ts';
import { decodeCursor } from '../../util/pagination.ts';
import { FLAT_PERM_SCAN_CONCURRENCY } from './PermissionStore.ts';
describe('PermissionStore', () => {
let server: PuterServer;
@@ -1232,6 +1234,172 @@ describe('PermissionStore', () => {
).toHaveLength(2);
});
it('takes several roots in one delete', async () => {
const issuer = await makeUser();
const holder = await makeUser();
const roots = ['kv-share:owner:app:a', 'kv-share-read:owner:app:b'];
for (const permission of [...roots, `${roots[0]}:deep`])
await store.upsertUserUserPerm(
holder.id,
issuer.id,
permission,
{},
);
await store.upsertUserUserPerm(
holder.id,
issuer.id,
'kv-share:owner:app:c',
{},
);
const removed = await store.deleteUserUserPermSubtreeForHolder(
holder.id,
issuer.id,
roots,
);
expect(removed.sort()).toEqual(
[...roots, `${roots[0]}:deep`].sort(),
);
expect(
await store.readLinkedUserUserPerms(holder.id, [
'kv-share:owner:app:c',
]),
).toHaveLength(1);
});
it('scans flat entries a few roots at a time, once per outermost root', async () => {
const issuer = await makeUser();
const holder = await makeUser();
const outer = Array.from(
{ length: 12 },
(_, i) => `kv-share:owner:app:r${i}`,
);
const nested = outer.map((root) => `${root}:deep`);
for (const root of outer) {
await store.upsertUserUserPerm(holder.id, issuer.id, root, {});
await store.setFlatUserPerm(holder.id, root, {
permission: root,
issuer_user_id: issuer.id,
});
}
// Flat only, beneath a nested root: the outer scan has to find it.
const orphan = `${nested[0]}:orphan`;
await store.setFlatUserPerm(holder.id, orphan, {
permission: orphan,
issuer_user_id: issuer.id,
});
const holderPrefix = PermissionUtil.join('perm', String(holder.id));
const dynamo = server.clients.dynamo;
const query = dynamo.query.bind(dynamo);
const scanned: string[] = [];
let inFlight = 0;
let peak = 0;
const spy = vi
.spyOn(dynamo, 'query')
.mockImplementation(async (...args) => {
const prefix = args[6]?.beginsWith?.value;
if (!prefix?.startsWith(holderPrefix))
return query(...args);
scanned.push(prefix);
peak = Math.max(peak, ++inFlight);
try {
await new Promise((resolve) => setTimeout(resolve, 5));
return await query(...args);
} finally {
inFlight--;
}
});
let removed: string[];
try {
removed = await store.deleteUserUserPermSubtreeForHolder(
holder.id,
issuer.id,
[...outer, ...nested, outer[0]],
);
} finally {
spy.mockRestore();
}
expect(peak).toBeGreaterThan(1);
expect(peak).toBeLessThanOrEqual(FLAT_PERM_SCAN_CONCURRENCY);
expect(scanned.sort()).toEqual(
outer
.map((root) =>
PermissionUtil.join('perm', String(holder.id), root),
)
.sort(),
);
expect(removed.sort()).toEqual([...outer].sort());
expect(
await store.getFlatUserPerms(holder.id, [...outer, orphan]),
).toEqual([]);
});
it('deletes several exact grants of one issuer in one statement, reporting the ones it found', async () => {
const issuer = await makeUser();
const other = await makeUser();
const holder = await makeUser();
await store.upsertUserUserPerm(
holder.id,
issuer.id,
'kv-share:o:a:x',
{},
);
await store.upsertUserUserPerm(
holder.id,
issuer.id,
'kv-share:o:a:x:deep',
{},
);
await store.upsertUserUserPerm(
holder.id,
other.id,
'kv-share-read:o:a:x',
{},
);
const write = vi.spyOn(server.clients.db, 'write');
let removed: string[];
try {
removed = await store.deleteUserUserPermsByHolder(
holder.id,
[
'kv-share:o:a:x',
'kv-share-read:o:a:x',
'kv-share-write:o:a:x',
],
issuer.id,
);
expect(
write.mock.calls.filter(([sql]) =>
String(sql).startsWith('DELETE'),
),
).toHaveLength(1);
} finally {
write.mockRestore();
}
expect(removed).toEqual(['kv-share:o:a:x']);
// Exact, not a subtree, and never another issuer's grant.
expect(
(
await store.readLinkedUserUserPerms(holder.id, [
'kv-share:o:a:x:deep',
'kv-share-read:o:a:x',
])
)
.map((row) => row.permission)
.sort(),
).toEqual(['kv-share-read:o:a:x', 'kv-share:o:a:x:deep']);
expect(
await store.deleteUserUserPermsByHolder(
holder.id,
[],
issuer.id,
),
).toEqual([]);
});
it('leaves another issuer`s subtree grant standing', async () => {
const issuerA = await makeUser();
const issuerB = await makeUser();
+188 -18
View File
@@ -27,6 +27,7 @@ import {
PERMISSION_CACHE_GENERATION_TTL_SECONDS,
PERMISSION_SCAN_CACHE_TTL_SECONDS,
} from '../../services/permission/consts';
import { runWithConcurrencyLimit } from '../../util/concurrency';
import { kv } from '../../util/kvSingleton';
import { keysetPage, openIdCursor } from '../../util/pagination';
import { bumpGeneration } from '../../util/redisGeneration';
@@ -43,6 +44,8 @@ const ACCESS_TOKEN_PERM_INSERT_CHUNK = 100;
const AUDIT_PAGE_SIZE = 50;
const MAX_AUDIT_PAGE_SIZE = 200;
/** Flat-entry prefix scans one subtree revoke keeps in flight. */
export const FLAT_PERM_SCAN_CONCURRENCY = 4;
/** The user-to-app grant a user's first sign-in to an app records. */
export const APP_AUTHENTICATED_FLAG = 'flag:app-is-authenticated';
@@ -127,6 +130,12 @@ export interface UserUserAuditFilter {
issuerUserId?: number;
holderUserId?: number;
permissions?: string[];
/**
* Rows under these roots are kept only where the component after the root
* is `ownerUuid`, for families whose grants name the owner of what they
* grant. Narrows the other fields; it is not a filter on its own.
*/
ownedRoots?: { roots: readonly string[]; ownerUuid: string };
}
/** One entry in the flat KV view, as addressed by a delete. */
@@ -156,6 +165,20 @@ const subtreeClause = (
]),
});
/** `roots` less any at or beneath another of them, which that one's scan covers. */
const outermostRoots = (roots: readonly string[]): string[] => {
const all = new Set(roots);
return [...all].filter((root) => {
for (
let at = root.indexOf(':');
at !== -1;
at = root.indexOf(':', at + 1)
)
if (all.has(root.slice(0, at))) return false;
return true;
});
};
/**
* PermissionStore owns the _persistence_ side of permissions:
*
@@ -481,6 +504,74 @@ export class PermissionStore extends PuterStore {
return true;
}
/**
* {@link deleteUserUserPermByHolder} for several permissions in one
* statement. Returns the ones that were there to delete.
*/
async deleteUserUserPermsByHolder(
holderUserId: number,
permissions: readonly string[],
issuerUserId: number,
): Promise<string[]> {
const found = await this.findUserUserPermsByIssuer(
holderUserId,
issuerUserId,
permissions,
);
if (found.length === 0) return [];
const { scope, params } = this.#issuerHolderScope(
holderUserId,
issuerUserId,
permissions,
);
await this.clients.db.write(
`DELETE FROM \`user_to_user_permissions\` WHERE ${scope}`,
params,
);
await this.publishCacheKeys({
keys: [this.#u2uCacheKey(holderUserId)],
broadcast: true,
});
return found;
}
/**
* Which of `permissions` this issuer granted this holder, as the primary
* spells them. A point lookup on the table's key.
*/
async findUserUserPermsByIssuer(
holderUserId: number,
issuerUserId: number,
permissions: readonly string[],
): Promise<string[]> {
if (permissions.length === 0) return [];
const { scope, params } = this.#issuerHolderScope(
holderUserId,
issuerUserId,
permissions,
);
const rows = await this.clients.db.pread(
`SELECT \`permission\` FROM \`user_to_user_permissions\` WHERE ${scope}`,
params,
);
return rows.map((row) => String(row.permission));
}
#issuerHolderScope(
holderUserId: number,
issuerUserId: number,
permissions: readonly string[],
): { scope: string; params: unknown[] } {
const unique = [...new Set(permissions)];
return {
scope:
'`holder_user_id` = ? AND `issuer_user_id` = ? AND ' +
`\`permission\` IN (${unique.map(() => '?').join(', ')})`,
params: [holderUserId, issuerUserId, ...unique],
};
}
/**
* The grants an account issued, as flat-view refs. Read before the account
* row goes: the SQL rows cascade with it, the holders' flat entries do
@@ -588,35 +679,104 @@ export class PermissionStore extends PuterStore {
async deleteUserUserPermSubtreeForHolder(
holderUserId: number,
issuerUserId: number,
permission: string,
permission: string | readonly string[],
): Promise<string[]> {
const { where, params } = subtreeClause([permission]);
const roots =
typeof permission === 'string' ? [permission] : [...permission];
if (roots.length === 0) return [];
const { where, params } = subtreeClause(roots);
const scope = '`holder_user_id` = ? AND `issuer_user_id` = ?';
const scoped = [holderUserId, issuerUserId, ...params];
const rows = (await this.clients.db.read(
'SELECT `permission` FROM `user_to_user_permissions` ' +
`WHERE ${scope} AND (${where})`,
scoped,
)) as Array<{ permission: string }>;
if (rows.length === 0) return [];
// The primary: a replica behind it misses a grant just made, whose
// flat entry would then outlive the revoke.
const removed = (
await this.clients.db.pread(
'SELECT `permission` FROM `user_to_user_permissions` ' +
`WHERE ${scope} AND (${where})`,
scoped,
)
).map((row) => String(row.permission));
if (removed.length > 0) {
await this.clients.db.write(
`DELETE FROM \`user_to_user_permissions\` WHERE ${scope} AND (${where})`,
scoped,
);
await this.publishCacheKeys({
keys: [this.#u2uCacheKey(holderUserId)],
broadcast: true,
});
}
await this.clients.db.write(
`DELETE FROM \`user_to_user_permissions\` WHERE ${scope} AND (${where})`,
scoped,
// The whole flat subtree, not only what SQL returned: an entry can
// outlive its row. An entry another issuer's row still backs is kept.
const candidates = [
...new Set([
...removed,
...(await this.#flatPermsUnder(holderUserId, roots)),
]),
];
if (candidates.length === 0) return removed;
const remaining = new Set(
(
await this.readLinkedUserUserPermsFromPrimary(
holderUserId,
candidates,
)
).map((row) => row.permission),
);
const removed = rows.map((row) => String(row.permission));
await this.delFlatUserPerms(
removed.map((perm) => ({ holderUserId, permission: perm })),
candidates
.filter((perm) => !remaining.has(perm))
.map((perm) => ({ holderUserId, permission: perm })),
);
await this.publishCacheKeys({
keys: [this.#u2uCacheKey(holderUserId)],
broadcast: true,
});
return removed;
}
/**
* The flat entries a holder has at or beneath any of `roots`, found by key
* prefix, so an entry with no SQL row behind it is found too.
*/
async #flatPermsUnder(
holderUserId: number,
roots: readonly string[],
): Promise<string[]> {
const holderKey = PermissionUtil.join(
PERM_KEY_PREFIX,
String(holderUserId),
);
const escapedSeparator =
PermissionUtil.escape_permission_component(':');
const found = await runWithConcurrencyLimit(
outermostRoots(roots),
FLAT_PERM_SCAN_CONCURRENCY,
async (root) => {
const base = PermissionUtil.join(
PERM_KEY_PREFIX,
String(holderUserId),
root,
);
const { res } = await this.stores.kv.list({
as: 'keys',
keyPrefix: base,
});
// The prefix also matches siblings that merely start the same.
return (res as string[])
.filter(
(key) =>
key === base ||
key.startsWith(`${base}${escapedSeparator}`),
)
.map((key) =>
PermissionUtil.unescape_permission_component(
key.slice(holderKey.length + 1),
),
);
},
);
return found.flat();
}
async auditUserUserPerm(
entry: AuditEntry & {
holder_user_id: number;
@@ -719,6 +879,16 @@ export class PermissionStore extends PuterStore {
if (clauses.length === 0) {
throw new Error('listUserUserAudit requires a filter');
}
const owned = filter.ownedRoots;
for (const root of owned?.roots ?? []) {
clauses.push(
"NOT (`permission` LIKE ? ESCAPE '!' AND `permission` NOT LIKE ? ESCAPE '!')",
);
params.push(
`${escapeLike(root)}:%`,
`${escapeLike(`${root}:${owned?.ownerUuid}`)}:%`,
);
}
return { sql: clauses.join(' AND '), params };
}
@@ -3312,14 +3312,70 @@ describe('SystemKVStore', () => {
it('bills the probe as a read', async () => {
// The probe is a real round trip; metering runs off the usage the
// store reports, so swallowing it under-bills the caller.
const { usage } = await target.set(
{ key: 'metered', value: 'v' },
const { usage } = await target.update(
{ key: 'metered', pathAndValueMap: { a: 1 } },
crossOpts,
);
expect(usage.read).toBeGreaterThan(0);
expect(usage.write).toBeGreaterThan(0);
});
it('checks set and del in the write itself, with no read before it', async () => {
const get = vi.spyOn(server.clients.dynamo, 'get');
try {
const written = await target.set(
{ key: 'one-trip', value: 'v' },
crossOpts,
);
expect(written.usage.read).toBe(0);
expect(written.usage.write).toBeGreaterThan(0);
const deleted = await target.del(
{ key: 'one-trip' },
crossOpts,
);
expect(deleted.usage.read).toBe(0);
expect(get).not.toHaveBeenCalled();
} finally {
get.mockRestore();
}
});
it('refuses set and del on a private entry, leaving it as it was', async () => {
const own = { actor, appUuid: 'app-other' };
await target.set(
{ key: 'kept-secret', value: 's', disableSharing: true },
own,
);
await expect(
target.set({ key: 'kept-secret', value: 'x' }, crossOpts),
).rejects.toMatchObject({ statusCode: 403 });
await expect(
target.del({ key: 'kept-secret' }, crossOpts),
).rejects.toMatchObject({ statusCode: 403 });
expect((await target.get({ key: 'kept-secret' }, own)).res).toBe(
's',
);
});
it('writes over a private entry whose expiry has passed', async () => {
const own = { actor, appUuid: 'app-other' };
await target.set(
{
key: 'lapsed-secret',
value: 's',
disableSharing: true,
expireAt: Math.floor(Date.now() / 1000) - 10,
},
own,
);
await expect(
target.set({ key: 'lapsed-secret', value: 'x' }, crossOpts),
).resolves.toMatchObject({ res: true });
expect((await target.get({ key: 'lapsed-secret' }, own)).res).toBe(
'x',
);
});
it('still refuses a private entry through the batch probe', async () => {
// Into the *target* namespace: a plain user actor may address any of
// its own app namespaces via `appUuid`, which is how the owning app's
@@ -3463,6 +3519,188 @@ describe('SystemKVStore', () => {
});
});
// -- Another user's namespace -------------------------------------
//
// Reached only through `namespaceUser`, which the KV driver sets after a
// share handle's check has passed.
describe('another user`s namespace', () => {
let owner: Actor;
let holderOpts: {
actor: Actor;
namespaceUser: { uuid: string; id: number };
namespaceAppUuid: string;
};
beforeEach(() => {
owner = {
user: {
id: 7001,
uuid: `owner-${Math.random().toString(36).slice(2)}`,
},
effectiveApp: null,
};
holderOpts = {
actor,
namespaceUser: {
uuid: owner.user.uuid!,
id: owner.user.id!,
},
namespaceAppUuid: KV_GLOBAL_APP_KEY,
};
});
it('reads and writes the owner`s data, not the caller`s', async () => {
await target.set(
{ key: 'shared', value: 'mine' },
{ actor: owner },
);
expect((await target.get({ key: 'shared' }, holderOpts)).res).toBe(
'mine',
);
await target.set({ key: 'shared', value: 'theirs' }, holderOpts);
expect(
(await target.get({ key: 'shared' }, { actor: owner })).res,
).toBe('theirs');
expect((await target.get({ key: 'shared' }, opts)).res).toBeNull();
});
it('hides private entries from get, list and total, and refuses writes to them', async () => {
await target.set({ key: 'p:open', value: 1 }, { actor: owner });
await target.set(
{ key: 'p:hidden', value: 2, disableSharing: true },
{ actor: owner },
);
expect(
(await target.get({ key: 'p:hidden' }, holderOpts)).res,
).toBeNull();
const listed = await target.list(
{ as: 'keys', limit: 10, includeTotal: true, pattern: 'p:*' },
holderOpts,
);
expect(listed.res).toEqual({ items: ['p:open'], total: 1 });
await expect(
target.incr(
{ key: 'p:hidden', pathAndAmountMap: { '': 1 } },
holderOpts,
),
).rejects.toMatchObject({ statusCode: 403 });
await expect(
target.set({ key: 'p:hidden', value: 3 }, holderOpts),
).rejects.toMatchObject({ statusCode: 403 });
});
});
describe('onlyIfAbsent', () => {
it('writes once, then leaves a live entry alone', async () => {
const first = await target.set(
{ key: 'claim', value: 'a', onlyIfAbsent: true },
opts,
);
const second = await target.set(
{ key: 'claim', value: 'b', onlyIfAbsent: true },
opts,
);
expect(first.res).toBe(true);
expect(second.res).toBe(false);
expect(second.usage.write).toBeGreaterThan(0);
expect((await target.get({ key: 'claim' }, opts)).res).toBe('a');
});
it('treats an expired row that has not been swept as absent', async () => {
await target.set(
{
key: 'lapsed',
value: 'old',
expireAt: Math.floor(Date.now() / 1000) - 10,
},
opts,
);
const res = await target.set(
{ key: 'lapsed', value: 'new', onlyIfAbsent: true },
opts,
);
expect(res.res).toBe(true);
expect((await target.get({ key: 'lapsed' }, opts)).res).toBe('new');
});
it('announces only the write that landed', async () => {
// A user row id is what a mutation event keys on.
const withId = {
actor: { ...actor, user: { ...actor.user, id: 9 } },
};
const emit = vi.spyOn(server.clients.event, 'emit');
try {
await target.set(
{ key: 'once', value: 1, onlyIfAbsent: true },
withId,
);
await target.set(
{ key: 'once', value: 2, onlyIfAbsent: true },
withId,
);
const mutations = emit.mock.calls.filter(
([key]) => key === 'kv.mutated',
);
expect(mutations).toHaveLength(1);
} finally {
emit.mockRestore();
}
});
it('answers false for a live private entry in a foreign namespace', async () => {
const own = { actor, appUuid: 'app-other' };
await target.set(
{ key: 'private-claim', value: 's', disableSharing: true },
own,
);
const res = await target.set(
{ key: 'private-claim', value: 'x', onlyIfAbsent: true },
{ actor, namespaceAppUuid: 'app-other' },
);
expect(res.res).toBe(false);
expect((await target.get({ key: 'private-claim' }, own)).res).toBe(
's',
);
});
});
describe('list within a key prefix', () => {
beforeEach(async () => {
for (const key of [' room:a', ' room:b', 'room:c', ' roomy'])
await target.set({ key, value: key }, opts);
});
it('lists only under the prefix, reading the pattern below it', async () => {
const all = await target.list(
{ as: 'keys', keyPrefix: ' room:' },
opts,
);
expect(all.res).toEqual([' room:a', ' room:b']);
const narrowed = await target.list(
{ as: 'keys', keyPrefix: ' room:', pattern: 'b*' },
opts,
);
expect(narrowed.res).toEqual([' room:b']);
});
it('keeps a prefix that starts with whitespace, where a pattern is trimmed', async () => {
// Trimmed as a pattern, ` room:` would widen to `room:`.
const res = await target.list(
{
as: 'keys',
keyPrefix: ' room:',
pattern: ' ',
limit: 10,
includeTotal: true,
},
opts,
);
expect(res.res).toEqual({
items: [' room:a', ' room:b'],
total: 2,
});
});
});
describe('take', () => {
it('returns the value to exactly one caller, null after', async () => {
await target.set({ key: 'claim-me', value: { by: 'me' } }, opts);
@@ -3956,6 +4194,85 @@ describe('SystemKVStore', () => {
expect(listed.res).not.toContain('legacyStringPast');
});
it('onlyIfAbsent writes over a legacy expired string-ttl row', async () => {
const namespace = namespaceOf(actor);
const past = String(Math.floor(Date.now() / 1000) - 3600);
await server.clients.dynamo.put(PUTER_KV_STORE_TABLE_NAME, {
namespace,
key: 'legacyStringClaim',
value: 'old',
ttl: past,
});
const res = await target.set(
{ key: 'legacyStringClaim', value: 'new', onlyIfAbsent: true },
opts,
);
expect(res.res).toBe(true);
expect(
(await target.get({ key: 'legacyStringClaim' }, opts)).res,
).toBe('new');
});
it('onlyIfAbsent still answers false over a legacy future string-ttl row', async () => {
const namespace = namespaceOf(actor);
const future = String(Math.floor(Date.now() / 1000) + 3600);
await server.clients.dynamo.put(PUTER_KV_STORE_TABLE_NAME, {
namespace,
key: 'legacyStringHeld',
value: 'old',
ttl: future,
});
const res = await target.set(
{ key: 'legacyStringHeld', value: 'new', onlyIfAbsent: true },
opts,
);
expect(res.res).toBe(false);
expect(
(await target.get({ key: 'legacyStringHeld' }, opts)).res,
).toBe('old');
});
it('a foreign set and del go through a private entry whose legacy string ttl has passed', async () => {
const namespace = kvNamespace(actor.user.uuid!, 'app-other');
const foreign = { actor, namespaceAppUuid: 'app-other' };
const past = String(Math.floor(Date.now() / 1000) - 3600);
for (const key of ['legacyPrivateSet', 'legacyPrivateDel'])
await server.clients.dynamo.put(PUTER_KV_STORE_TABLE_NAME, {
namespace,
key,
value: 'hidden',
ttl: past,
noShare: true,
});
await expect(
target.set({ key: 'legacyPrivateSet', value: 'mine' }, foreign),
).resolves.toMatchObject({ res: true });
expect(
(await target.get({ key: 'legacyPrivateSet' }, foreign)).res,
).toBe('mine');
await expect(
target.del({ key: 'legacyPrivateDel' }, foreign),
).resolves.toMatchObject({ res: true });
});
it('a foreign set still refuses a private entry whose legacy string ttl is in the future', async () => {
const namespace = kvNamespace(actor.user.uuid!, 'app-other');
await server.clients.dynamo.put(PUTER_KV_STORE_TABLE_NAME, {
namespace,
key: 'legacyPrivateLive',
value: 'hidden',
ttl: String(Math.floor(Date.now() / 1000) + 3600),
noShare: true,
});
await expect(
target.set(
{ key: 'legacyPrivateLive', value: 'mine' },
{ actor, namespaceAppUuid: 'app-other' },
),
).rejects.toMatchObject({ statusCode: 403 });
});
it('a legacy row with a non-numeric string ttl is readable by get and list', async () => {
const namespace = namespaceOf(actor);
await server.clients.dynamo.put(PUTER_KV_STORE_TABLE_NAME, {
+223 -77
View File
@@ -24,6 +24,7 @@ import {
isSystemActor,
SYSTEM_ACTOR,
SYSTEM_ACTOR_UUID,
viaWorkerOf,
} from '../../core/actor';
import { HttpError } from '../../core/http';
import {
@@ -129,6 +130,12 @@ export interface KVOpts {
* only after its cross-app permission check has passed.
*/
namespaceAppUuid?: string;
/**
* Namespace override that wins over the actor's own user — how a share
* holder addresses the owner's data. Set only by the KV driver, after its
* handle check has passed.
*/
namespaceUser?: { uuid: string; id: number };
/**
* Files system data under its own partition of the system namespace, so a
* high-volume key family doesn't share one partition key with the rest of
@@ -149,7 +156,8 @@ export const KV_GLOBAL_APP_KEY = 'os-global';
const SYSTEM_NAMESPACE = `v1:${SYSTEM_ACTOR_UUID}:${KV_GLOBAL_APP_KEY}`;
// The extra segment keeps a partition from parsing as a user's namespace.
const SYSTEM_PARTITION_PREFIX = `${SYSTEM_NAMESPACE}:`;
const MAX_KEY_BYTES = 1024;
/** Longest key the store accepts, in UTF-8 bytes. */
export const KV_MAX_KEY_BYTES = 1024;
/**
* Whether a write was refused because the condition it carried no longer held.
@@ -178,6 +186,8 @@ const PATH_CLEANER_REGEX = /[^A-Za-z0-9_]/g;
// is bounded; cursors are the recommended way to page.
const MAX_LIST_OFFSET = 5000;
const MAX_FILL_PAGES = 10;
/** Most items one query reads while a foreign listing skips hidden entries. */
const MAX_HIDDEN_SKIP_READ = 1000;
// Cache keys carried by one invalidation broadcast. A peer applies them in a
// single pass either way; the cap only keeps an individual message a sane size.
const KV_CACHE_BROADCAST_CHUNK = 500;
@@ -239,6 +249,31 @@ const expiredRowFilter = (now: number) => {
return { ...live, expression: `NOT (${live.expression})` };
};
/** No item, or an expired one: where an `onlyIfAbsent` write may land. */
const absentRowCondition = (now: number) => {
const expired = expiredRowFilter(now);
return {
expression: `attribute_not_exists(#key) OR (${expired.expression})`,
names: { ...expired.names, '#key': 'key' },
values: expired.values,
};
};
/** No live private entry: where a write into a foreign namespace may land. */
const notPrivateCondition = (now: number) => {
const expired = expiredRowFilter(now);
return {
expression: `attribute_not_exists(#privAttr) OR (${expired.expression})`,
names: { ...expired.names, '#privAttr': KV_PRIVATE_ATTR },
values: expired.values,
};
};
const privateEntryError = (): HttpError =>
new HttpError(403, 'kv: this entry is private to the app that wrote it', {
legacyCode: 'forbidden',
});
/**
* A timestamp at or before now is stored as `floor(now)` rather than as-is, so
* it still sweeps (never `0`, which reads as no expiry).
@@ -319,13 +354,14 @@ const isNestingTooDeep = (err: unknown): boolean =>
const MAX_LIVE_WRITE_ATTEMPTS = 3;
/**
* Expired entries, plus private ones for a cross-app caller. Filtered in the
* query rather than afterwards so `includeTotal`'s COUNT excludes them too — a
* total counting rows the caller can't see would leak what the flag hides.
* Expired entries, plus private ones for a caller in a foreign namespace.
* Filtered in the query rather than afterwards so `includeTotal`'s COUNT
* excludes them too — a total counting rows the caller can't see would leak
* what the flag hides.
*/
const listFilter = (now: number, crossApp: boolean) => {
const listFilter = (now: number, foreign: boolean) => {
const live = liveRowFilter(now);
if (!crossApp) return live;
if (!foreign) return live;
return {
expression: `(${live.expression}) AND attribute_not_exists(#privAttr)`,
names: { ...live.names, '#privAttr': KV_PRIVATE_ATTR },
@@ -361,7 +397,12 @@ const addUsage = (a: KVUsage, b: KVUsage): KVUsage => ({
const ensureActor = (opts?: KVOpts): Actor => opts?.actor ?? SYSTEM_ACTOR;
const isCrossApp = (opts?: KVOpts): boolean => Boolean(opts?.namespaceAppUuid);
/**
* Another app's namespace, or another user's: where private entries read as
* absent and refuse writes.
*/
const isForeignNamespace = (opts?: KVOpts): boolean =>
Boolean(opts?.namespaceAppUuid || opts?.namespaceUser);
// `effectiveApp`, not `app`, so the namespace agrees with the gate the driver
// applied: both read an access-token actor as the app that issued it, rather
@@ -378,7 +419,7 @@ const getNamespace = (actor: Actor, opts?: KVOpts): string => {
actor.effectiveApp?.uid ??
opts?.appUuid ??
KV_GLOBAL_APP_KEY;
return kvNamespace(actor.user.uuid!, appUuid);
return kvNamespace(opts?.namespaceUser?.uuid ?? actor.user.uuid!, appUuid);
};
/** The namespace one user's data for one app lives in. */
@@ -405,10 +446,10 @@ const assertKey = (key: string): void => {
throw new HttpError(400, 'kv: key is empty', {
legacyCode: 'bad_request',
});
if (Buffer.byteLength(key, 'utf8') > MAX_KEY_BYTES) {
if (Buffer.byteLength(key, 'utf8') > KV_MAX_KEY_BYTES) {
throw new HttpError(
400,
`kv: key exceeds ${MAX_KEY_BYTES} byte limit`,
`kv: key exceeds ${KV_MAX_KEY_BYTES} byte limit`,
{ legacyCode: 'bad_request' },
);
}
@@ -1413,7 +1454,7 @@ export class SystemKVStore extends PuterStore {
* the change.
*/
async #committed(
actor: Actor,
opts: KVOpts | undefined,
namespace: string,
keys: string[],
op: KvMutation,
@@ -1421,7 +1462,7 @@ export class SystemKVStore extends PuterStore {
noShareKeys?: string[],
): Promise<void> {
await this.#invalidate(namespace, keys);
this.#emitMutation(actor, namespace, keys, op, values, noShareKeys);
this.#emitMutation(opts, namespace, keys, op, values, noShareKeys);
}
/**
@@ -1429,24 +1470,32 @@ export class SystemKVStore extends PuterStore {
* large namespace doesn't turn into an unbounded event.
*/
#emitMutation(
actor: Actor,
opts: KVOpts | undefined,
namespace: string,
keys: string[],
op: KvMutation,
values?: unknown[],
noShareKeys?: string[],
): void {
const actor = ensureActor(opts);
// Internal system data has no subscribable subject, and it is written
// often enough that the emit itself would be the cost.
if (isSystemActor(actor)) return;
const userId = actor.user?.id;
// Dispatch keys on the namespace's owner, who is not the writer when a
// share holder wrote it.
const userId = opts?.namespaceUser?.id ?? actor.user?.id;
if (typeof userId !== 'number') return;
const actingUserId = actor.user?.id;
const writer = {
...(typeof actingUserId === 'number' ? { actingUserId } : {}),
...(viaWorkerOf(actor) ? { viaWorker: true as const } : {}),
};
try {
if (op === 'flush') {
this.clients.event.emit(
'kv.flushed',
{ namespace, userId },
{ namespace, userId, ...writer },
{},
);
return;
@@ -1471,6 +1520,7 @@ export class SystemKVStore extends PuterStore {
...(actor.handlerDepth
? { handlerDepth: actor.handlerDepth }
: {}),
...writer,
},
{},
);
@@ -1713,16 +1763,17 @@ export class SystemKVStore extends PuterStore {
// -- Public API ---------------------------------------------------
/**
* Refuse a cross-app mutation against a private entry. Not atomic with the
* write that follows, so one in-flight write can still land on an entry the
* owner flags in between.
* Refuse a path write into a foreign namespace against a private entry. Not
* atomic with the write that follows, so one in-flight write can still land
* on an entry the owner flags in between; `set` and `del` carry the check
* in their own write instead.
*/
async #assertNotPrivate(
namespace: string,
key: string,
opts?: KVOpts,
): Promise<KVUsage> {
if (!isCrossApp(opts)) return emptyUsage();
if (!isForeignNamespace(opts)) return emptyUsage();
let response = await this.clients.dynamo.get(this.tableName, {
namespace,
key,
@@ -1753,11 +1804,7 @@ export class SystemKVStore extends PuterStore {
response.Item?.[KV_PRIVATE_ATTR] &&
!isExpiredTtl(response.Item.ttl, Date.now() / 1000)
) {
throw new HttpError(
403,
'kv: this entry is private to the app that wrote it',
{ legacyCode: 'forbidden' },
);
throw privateEntryError();
}
// Returned rather than swallowed: the probe is a real read, and a caller
// that did not pay for it is under-billed for the operation.
@@ -1773,7 +1820,7 @@ export class SystemKVStore extends PuterStore {
keys: string[],
opts?: KVOpts,
): Promise<KVUsage> {
if (!isCrossApp(opts) || keys.length === 0) return emptyUsage();
if (!isForeignNamespace(opts) || keys.length === 0) return emptyUsage();
const { entries, usage } = await this.getBatches(namespace, keys);
const now = Date.now() / 1000;
@@ -1811,13 +1858,7 @@ export class SystemKVStore extends PuterStore {
const isPrivate = confirmedEntries.some(
(entry) => entry?.noShare && !isExpiredTtl(entry.ttl, now),
);
if (isPrivate) {
throw new HttpError(
403,
'kv: this entry is private to the app that wrote it',
{ legacyCode: 'forbidden' },
);
}
if (isPrivate) throw privateEntryError();
return addUsage(usage, recheckUsage);
}
@@ -1948,6 +1989,34 @@ export class SystemKVStore extends PuterStore {
}
}
/**
* A write whose condition may refuse it, retried once after settling a row
* the condition reads as live but reads treat as expired (a legacy text or
* boolean `ttl`). `response` is `null` when the condition still refuses.
*/
async #writeUnlessRefused<R>(
namespace: string,
key: string,
write: () => Promise<R>,
): Promise<{ response: R | null; usage: KVUsage }> {
try {
return { response: await write(), usage: emptyUsage() };
} catch (e) {
if (!isConditionRefused(e)) throw e;
}
// A refused conditional write still consumes capacity.
const usage = addUsage(
writeUsage(1),
await this.#dropIfExpired(namespace, key),
);
try {
return { response: await write(), usage };
} catch (e) {
if (!isConditionRefused(e)) throw e;
return { response: null, usage: addUsage(usage, writeUsage(1)) };
}
}
async get(
{
key,
@@ -1957,7 +2026,7 @@ export class SystemKVStore extends PuterStore {
): Promise<KVResult<unknown | null | (unknown | null)[]>> {
const actor = ensureActor(opts);
const namespace = getNamespace(actor, opts);
const crossApp = isCrossApp(opts);
const foreign = isForeignNamespace(opts);
const multi = Array.isArray(key);
const keys = multi ? key : [key];
@@ -2028,19 +2097,24 @@ export class SystemKVStore extends PuterStore {
if (!entry) return null;
if (isExpiredTtl(entry.ttl, now)) return null;
// Absent rather than refused: the flag must not confirm the key.
if (crossApp && entry.noShare) return null;
if (foreign && entry.noShare) return null;
return entry.value ?? null;
});
return { res: multi ? values : values[0], usage };
}
/**
* `res` is `false` only for an `onlyIfAbsent` write that found a live
* entry, which is left as it was.
*/
async set(
{
key,
value,
expireAt,
disableSharing,
onlyIfAbsent,
}: {
key: string;
value: unknown;
@@ -2051,6 +2125,11 @@ export class SystemKVStore extends PuterStore {
* on a later write is how the owner re-shares the entry.
*/
disableSharing?: boolean;
/**
* Write only when no live entry holds the key; an expired one that
* hasn't been swept counts as absent.
*/
onlyIfAbsent?: boolean;
},
opts?: KVOpts,
): Promise<KVResult<boolean>> {
@@ -2059,23 +2138,51 @@ export class SystemKVStore extends PuterStore {
const ttl = coerceExpiry(expireAt, 'expireAt');
const actor = ensureActor(opts);
const namespace = getNamespace(actor, opts);
const probeUsage = await this.#assertNotPrivate(namespace, key, opts);
// The put carries the check rather than a read before it: any live
// entry refuses `onlyIfAbsent`, which covers a private one too.
const now = Date.now() / 1000;
const condition = onlyIfAbsent
? absentRowCondition(now)
: isForeignNamespace(opts)
? notPrivateCondition(now)
: undefined;
let response: Awaited<ReturnType<typeof this.clients.dynamo.put>>;
const put = () =>
this.clients.dynamo.put(
this.tableName,
{
namespace,
key,
value,
...(ttl ? { ttl: storedExpiry(ttl) } : {}),
...(disableSharing ? { [KV_PRIVATE_ATTR]: true } : {}),
},
condition ? { condition } : undefined,
);
let response: Awaited<ReturnType<typeof put>> | null;
let settleUsage = emptyUsage();
try {
response = await this.clients.dynamo.put(this.tableName, {
namespace,
key,
value,
...(ttl ? { ttl: storedExpiry(ttl) } : {}),
...(disableSharing ? { [KV_PRIVATE_ATTR]: true } : {}),
});
if (condition) {
const outcome = await this.#writeUnlessRefused(
namespace,
key,
put,
);
response = outcome.response;
settleUsage = outcome.usage;
} else {
response = await put();
}
} catch (e) {
if (isNestingTooDeep(e)) throw tooDeeplyNestedError(undefined, e);
throw e;
}
if (response === null) {
if (onlyIfAbsent) return { res: false, usage: settleUsage };
throw privateEntryError();
}
await this.#committed(
actor,
opts,
namespace,
[key],
'set',
@@ -2086,10 +2193,11 @@ export class SystemKVStore extends PuterStore {
return {
res: true,
usage: addUsage(
probeUsage,
settleUsage,
writeUsage(
response.ConsumedCapacity?.CapacityUnits as
number | undefined,
| number
| undefined,
),
),
};
@@ -2163,7 +2271,7 @@ export class SystemKVStore extends PuterStore {
throw e;
}
await this.#committed(
actor,
opts,
namespace,
[...byKey.keys()],
'set',
@@ -2189,17 +2297,28 @@ export class SystemKVStore extends PuterStore {
assertKey(key);
const actor = ensureActor(opts);
const namespace = getNamespace(actor, opts);
const probeUsage = await this.#assertNotPrivate(namespace, key, opts);
// The delete carries the private-entry check rather than a read first.
const condition = isForeignNamespace(opts)
? notPrivateCondition(Date.now() / 1000)
: undefined;
// The old item is only worth reading back for the private-key check,
// which a system write never needs an event for anyway.
const response = await this.clients.dynamo.del(
this.tableName,
{ namespace, key },
{ returnOld: !isSystemActor(actor) },
);
const del = () =>
this.clients.dynamo.del(
this.tableName,
{ namespace, key },
{
returnOld: !isSystemActor(actor),
...(condition ? { condition } : {}),
},
);
const { response, usage: settleUsage } = condition
? await this.#writeUnlessRefused(namespace, key, del)
: { response: await del(), usage: emptyUsage() };
if (response === null) throw privateEntryError();
await this.#committed(
actor,
opts,
namespace,
[key],
'del',
@@ -2209,10 +2328,11 @@ export class SystemKVStore extends PuterStore {
return {
res: true,
usage: addUsage(
probeUsage,
settleUsage,
writeUsage(
(response.ConsumedCapacity?.CapacityUnits as
number | undefined) ?? 1,
| number
| undefined) ?? 1,
),
),
};
@@ -2238,7 +2358,7 @@ export class SystemKVStore extends PuterStore {
{ returnOld: true },
);
await this.#committed(
actor,
opts,
namespace,
[key],
'del',
@@ -2300,15 +2420,15 @@ export class SystemKVStore extends PuterStore {
})),
);
await this.#committed(
actor,
opts,
namespace,
uniqueKeys,
'del',
uniqueKeys.map((): unknown => null),
// A batch delete reports no prior state, so a same-app batch is
// marked private wholesale; a cross-app caller was already
// refused any private key.
isCrossApp(opts) ? undefined : uniqueKeys,
// marked private wholesale; a foreign caller was already refused
// any private key.
isForeignNamespace(opts) ? undefined : uniqueKeys,
);
const units =
response.ConsumedCapacity?.reduce(
@@ -2332,6 +2452,7 @@ export class SystemKVStore extends PuterStore {
includeTotal,
fetchUntilFull,
reverse,
keyPrefix,
}: {
as?: 'keys' | 'values' | 'entries';
limit?: number;
@@ -2341,6 +2462,12 @@ export class SystemKVStore extends PuterStore {
includeTotal?: boolean;
fetchUntilFull?: boolean;
reverse?: boolean;
/**
* Confines the listing to keys under this prefix, with `pattern`
* read below it. Taken as-is, where `pattern` is trimmed, so no
* pattern can widen it.
*/
keyPrefix?: string;
},
opts?: KVOpts,
): Promise<
@@ -2408,7 +2535,7 @@ export class SystemKVStore extends PuterStore {
}
}
const effectiveReverse = reverse ?? cursorReverse;
const normalizedPattern = normalizePattern(pattern);
const scanPrefix = `${keyPrefix ?? ''}${normalizePattern(pattern) ?? ''}`;
if (pageKey !== undefined && normalizedOffset !== undefined) {
throw new HttpError(
@@ -2457,15 +2584,15 @@ export class SystemKVStore extends PuterStore {
false,
{
scanIndexForward: !effectiveReverse,
...(normalizedPattern
...(scanPrefix
? {
beginsWith: {
key: 'key',
value: normalizedPattern,
value: scanPrefix,
},
}
: {}),
filter: listFilter(now, isCrossApp(opts)),
filter: listFilter(now, isForeignNamespace(opts)),
...(select ? { select } : {}),
},
);
@@ -2501,6 +2628,9 @@ export class SystemKVStore extends PuterStore {
}
}
// Another namespace's page goes on past entries it hides, reading
// further each time, so its cursor can sit on an entry it shows.
const foreign = isForeignNamespace(opts);
const collected: Array<Record<string, unknown>> = [];
let nextKey = startKey;
if (!exhausted) {
@@ -2509,7 +2639,13 @@ export class SystemKVStore extends PuterStore {
const want = normalizedLimit
? normalizedLimit - collected.length
: 0;
const response = await runQuery(want, nextKey);
const skipping = foreign && pages > 0 && collected.length === 0;
const response = await runQuery(
skipping
? Math.min(want * 2 ** pages, MAX_HIDDEN_SKIP_READ)
: want,
nextKey,
);
collected.push(
...((response.Items ?? []) as Array<
Record<string, unknown>
@@ -2523,7 +2659,7 @@ export class SystemKVStore extends PuterStore {
// result is complete rather than capped at one response.
if (!nextKey) break;
} else if (
!fetchUntilFull ||
(!fetchUntilFull && !(foreign && collected.length === 0)) ||
collected.length >= normalizedLimit ||
!nextKey ||
pages >= MAX_FILL_PAGES
@@ -2533,6 +2669,16 @@ export class SystemKVStore extends PuterStore {
} while (true);
}
// The scan may have stopped on a private entry, so a foreign cursor
// resumes after the last entry kept, which a read-ahead can overshoot.
let resumeKey = nextKey;
if (foreign && normalizedLimit !== undefined) {
const overshot = collected.splice(normalizedLimit).length > 0;
const last = collected[collected.length - 1];
if (last && (nextKey || overshot))
resumeKey = { namespace, key: last.key };
}
const entries = collected
.filter((e) => e && !isExpiredTtl(e.ttl, now))
.map((e) => ({ key: e.key as string, value: e.value }));
@@ -2560,9 +2706,9 @@ export class SystemKVStore extends PuterStore {
}
const nextCursor = encodeCursor(
nextKey && effectiveReverse
? { reverse: true, key: nextKey }
: nextKey,
resumeKey && effectiveReverse
? { reverse: true, key: resumeKey }
: resumeKey,
);
return {
res: {
@@ -2630,7 +2776,7 @@ export class SystemKVStore extends PuterStore {
Record<string, unknown> | undefined;
} while (pageKey);
} finally {
this.#emitMutation(actor, namespace, [], 'flush');
this.#emitMutation(opts, namespace, [], 'flush');
}
return { res: true, usage };
@@ -2651,7 +2797,7 @@ export class SystemKVStore extends PuterStore {
const probeUsage = await this.#assertNotPrivate(namespace, key, opts);
const { usage, isPrivate } = await this.rawExpireAt(namespace, key, ts);
await this.#committed(
actor,
opts,
namespace,
[key],
'expire',
@@ -2681,7 +2827,7 @@ export class SystemKVStore extends PuterStore {
timestamp,
);
await this.#committed(
actor,
opts,
namespace,
[key],
'expire',
@@ -2813,7 +2959,7 @@ export class SystemKVStore extends PuterStore {
throw pathWriteError(e, 'incr', key, paths);
}
await this.#committed(
actor,
opts,
namespace,
[key],
'set',
@@ -2949,7 +3095,7 @@ export class SystemKVStore extends PuterStore {
throw pathWriteError(e, 'add', key, paths);
}
await this.#committed(
actor,
opts,
namespace,
[key],
'set',
@@ -3028,7 +3174,7 @@ export class SystemKVStore extends PuterStore {
};
}
await this.#committed(
actor,
opts,
namespace,
[key],
'set',
@@ -3174,7 +3320,7 @@ export class SystemKVStore extends PuterStore {
}
await this.#committed(
actor,
opts,
namespace,
[key],
'set',
@@ -101,6 +101,7 @@ describe('what a mutation announces', () => {
keys: [KEY],
op: 'set',
values: [1],
actingUserId: 42,
});
expect(mutations()[0].key).toBe('kv.mutated');
});
@@ -279,10 +280,54 @@ describe('flush', () => {
expect(mutations()[0].data).toEqual({
namespace: namespaceOf(),
userId: 42,
actingUserId: 42,
});
});
});
describe('who made the change', () => {
it('keys a write into another user`s namespace on its owner, naming the writer', async () => {
const ownerUuid = `owner-${Math.random().toString(36).slice(2)}`;
await store.set(
{ key: KEY, value: 1 },
{
actor,
namespaceUser: { uuid: ownerUuid, id: 7 },
namespaceAppUuid: KV_GLOBAL_APP_KEY,
},
);
expect(onlyMutation()).toMatchObject({
namespace: `v1:${ownerUuid}:${KV_GLOBAL_APP_KEY}`,
userId: 7,
actingUserId: 42,
});
expect(onlyMutation()).not.toHaveProperty('viaWorker');
});
it.each([
['a worker session', { session: { uid: 's-1', kind: 'worker' } }],
['an events handler', { handlerDepth: 1 }],
])(
'marks a write made by %s, on writes and flushes alike',
async (_name, extra) => {
const server = { ...actor, ...extra } as Actor;
await store.set({ key: KEY, value: 1 }, { actor: server });
expect(onlyMutation()).toMatchObject({
actingUserId: 42,
viaWorker: true,
});
emitted = [];
await store.flush({ actor: server });
expect(mutations()[0].data).toMatchObject({
actingUserId: 42,
viaWorker: true,
});
},
);
});
describe('who is left out', () => {
it('says nothing for internal system data', async () => {
await store.set({ key: KEY, value: 1 }, { actor: SYSTEM_ACTOR });
+13
View File
@@ -1268,6 +1268,19 @@ interface IConfigOptional {
internalSecret?: string;
};
/**
* Key-value sharing beyond watching.
*
* - `shareHandles` — whether a share handle may carry `read` or `write`
* access, and be used for key-value calls. Absent means off: those mints
* and upgrades reject with `kv_handles_disabled`. Watch-only handles stay
* under `events.kvHandles`, and revoking, unsharing and listing are never
* gated.
*/
kv?: {
shareHandles?: boolean;
};
/**
* Display multiplier converting metered amounts into the "credits" clients
* show. Applied server-side by the usage-reporting endpoints, so raw
+25
View File
@@ -8,6 +8,7 @@ import {
openCursor,
openIdCursor,
sealCursor,
unsealCursor,
} from './pagination';
import { HttpError } from '../core/http';
@@ -107,6 +108,30 @@ describe('pagination util', () => {
expect(plain).toBe(encodeCursor({ id: 3 }));
expect(openCursor(plain, undefined)).toEqual({ id: 3 });
});
it('pads to one length whatever the payload holds', () => {
const short = sealCursor({ k: 'a' }, SECRET, { padTo: 256 })!;
const long = sealCursor({ k: 'a'.repeat(200) }, SECRET, {
padTo: 256,
})!;
expect(short.length).toBe(long.length);
expect(unsealCursor(short, SECRET)).toEqual({ k: 'a' });
expect(openCursor(long, SECRET)).toEqual({ k: 'a'.repeat(200) });
// Past the size, the next multiple rather than a cut.
const over = sealCursor({ k: 'a'.repeat(300) }, SECRET, {
padTo: 256,
})!;
expect(Buffer.from(over, 'base64url').length).toBe(
Buffer.from(short, 'base64url').length + 256,
);
});
it('opens only what it sealed with unsealCursor', () => {
expect(unsealCursor(encodeCursor({ id: 7 })!, SECRET)).toBeNull();
expect(
unsealCursor(sealCursor({ id: 7 }, 'other-secret')!, SECRET),
).toBeNull();
});
});
describe('normalizeLimit', () => {
+16 -6
View File
@@ -93,20 +93,29 @@ const sealKey = (secret: string): Buffer => {
* A cursor its holder cannot read, for keysets over a global sequence: a plain
* one would let two pages show how fast the whole table grows. Encrypted and
* authenticated, so an altered cursor is refused like any malformed one.
* Without a secret it is a plain `encodeCursor` cursor.
* Without a secret it is a plain `encodeCursor` cursor. `padTo` rounds the
* plaintext up to a multiple of that many bytes, so the length hides the size.
*/
export const sealCursor = (
payload: Record<string, unknown> | undefined,
secret: string | undefined,
{ padTo }: { padTo?: number } = {},
): string | undefined => {
if (!payload || Object.keys(payload).length === 0) return undefined;
if (!secret) return encodeCursor(payload);
let plain = Buffer.from(JSON.stringify(payload), 'utf8');
if (padTo) {
// Trailing spaces, which JSON.parse skips.
const padded = Buffer.alloc(
Math.ceil(plain.length / padTo) * padTo,
' ',
);
plain.copy(padded);
plain = padded;
}
const iv = randomBytes(SEAL_IV_BYTES);
const cipher = createCipheriv('aes-256-gcm', sealKey(secret), iv);
const body = Buffer.concat([
cipher.update(JSON.stringify(payload), 'utf8'),
cipher.final(),
]);
const body = Buffer.concat([cipher.update(plain), cipher.final()]);
return Buffer.concat([
Buffer.from([SEALED_CURSOR_VERSION]),
iv,
@@ -115,7 +124,8 @@ export const sealCursor = (
]).toString('base64url');
};
const unsealCursor = (
/** What `sealCursor` sealed under `secret`, or null if `cursor` is not that. */
export const unsealCursor = (
cursor: string,
secret: string,
): Record<string, unknown> | null => {
+1 -1
View File
@@ -202,7 +202,7 @@ These AI features are supported out of the box when using Puter.js:
- **[`puter.ai.txt2speech.listEngines()`](/AI/txt2speech.listEngines/)** - List available TTS engines/models
- **[`puter.ai.txt2speech.listVoices()`](/AI/txt2speech.listVoices/)** - List available TTS voices
- **[`puter.ai.speech2speech()`](/AI/speech2speech/)** - Convert speech in one voice to another voice
- **[`puter.ai.txt2vid()`](/AI/txt2vid/)** - Generate short video clips from text or a reference image with Wan, Seedance, Veo, Kling and other models, or edit a video you supply with Gemini Omni, FLUX 3 or Kling Omni
- **[`puter.ai.txt2vid()`](/AI/txt2vid/)** - Generate short video clips from text or a reference image with Wan, Seedance, Veo, Kling, MiniMax H3 and other models, or edit a video you supply with Gemini Omni, FLUX 3 or Kling Omni
- **[`puter.ai.speech2txt()`](/AI/speech2txt/)** - Transcribe audio recordings into text
## Examples
+50 -7
View File
@@ -1,10 +1,10 @@
---
title: puter.ai.txt2vid()
description: Generate short video clips from text or reference images, or edit a video you supply, with Wan, Seedance, Veo, Kling, Gemini Omni and other models through Puter.js.
description: Generate short video clips from text or reference images, or edit a video you supply, with Wan, Seedance, Veo, Kling, MiniMax H3, Gemini Omni and other models through Puter.js.
platforms: [websites, apps, nodejs, workers]
---
Given a prompt, generate a short video clip using AI. Puter routes the request to one of four upstream providers (Together AI, BytePlus Seedance, Google Gemini, Kling) based on the model you pick, waits for the clip to render, and resolves with a ready-to-play video. Every model can also start from an image you supply (image-to-video), and the same option names work across providers. Gemini Omni, FLUX 3 and the Kling Omni models can also edit a video you supply.
Given a prompt, generate a short video clip using AI. Puter routes the request to one of five upstream providers (Together AI, BytePlus Seedance, Google Gemini, Kling, MiniMax) based on the model you pick, waits for the clip to render, and resolves with a ready-to-play video. Every model can also start from an image you supply (image-to-video), and the same option names work across providers. Gemini Omni, FLUX 3 and the Kling Omni models can also edit a video you supply.
## Syntax
@@ -34,12 +34,12 @@ Additional settings for the generation request.
|--------|------|-------------|
| `prompt` | `String` | Text description for the video generation |
| `model` | `String` | Video model to use. Defaults to `'togetherai:wan-ai/wan2.7-t2v'` (Wan 2.7 text-to-video). See [Choosing a model](#choosing-a-model) |
| `provider` | `String` | Pin the request to one provider: `'together-video-generation'`, `'byteplus-video-generation'`, `'gemini-video-generation'` or `'kling-video-generation'`. Only needed when a model id exists on more than one provider |
| `provider` | `String` | Pin the request to one provider: `'together-video-generation'`, `'byteplus-video-generation'`, `'gemini-video-generation'`, `'kling-video-generation'` or `'minimax-video-generation'`. Only needed when a model id exists on more than one provider |
| `seconds` | `Number` | Target clip length in seconds. Each model supports a fixed set of durations; an unsupported value falls back to the model default instead of failing. Gemini Omni has no length control and rejects this option. `duration` is an alias |
| `size` | `String` | Output size as `'WIDTHxHEIGHT'` (e.g. `'1280x720'`) on every provider. Models that work in resolution tiers take the tier of the shorter side plus the aspect ratio, and also accept the tier directly (`'720p'`). An unsupported value falls back to the model default, except on Gemini Omni, which rejects it. `resolution` is an alias |
| `width`, `height` | `Number` | Output size in pixels on Together AI models sized in pixels; the aspect ratio on Seedance, Kling and Wan 2.7. Filled in from `size` when you leave them out |
| `width`, `height` | `Number` | Output size in pixels on Together AI models sized in pixels; the aspect ratio on Seedance, Kling, MiniMax H3 and Wan 2.7. Filled in from `size` when you leave them out |
| `input_reference` | `String \| Blob` | Image the clip starts from (image-to-video), on every provider. See [Image inputs](#image-inputs) for the formats it takes |
| `last_frame` | `String \| Blob` | Image the clip ends on, same formats. Requires `input_reference` on Seedance |
| `last_frame` | `String \| Blob` | Image the clip ends on, same formats. Requires `input_reference` on Seedance, Kling and Hailuo 02; the MiniMax H3 models also take it alone |
| `reference_images` | `Array<String \| Blob>` | Images the model keeps consistent as subjects or style, same formats. Limits per provider are listed below; not combined with `input_reference` or `last_frame`, except on Kling |
| `inputVideo` | `String \| Blob` | Video to edit, with the prompt describing the change. Only Gemini Omni, FLUX 3, Kling 3.0 Omni and Kling O1 accept it. See [Video input](#video-input) |
| `negative_prompt` | `String` | Text describing what to avoid in the video (Together AI) |
@@ -79,6 +79,7 @@ What each provider does with image inputs:
| BytePlus Seedance | Yes | Yes, with `input_reference` | Seedance 2.0: up to 9; Seedance 2.5: up to 30; cannot be combined with frames |
| Google Gemini | Yes, one PNG, JPEG or WebP image up to 5 MiB | No | No |
| Kling | Yes, JPEG or PNG | Yes, with `input_reference`, on the models listed in [Kling options](#kling-options) | Kling 3.0 Omni and O1: up to 7, or 4 with `inputVideo` |
| MiniMax | Yes, JPEG, PNG or WebP | Yes, on the H3 models, and with `input_reference` on Hailuo 02 | H3 models: up to 5 images in total with the frames, 2 on H3 Max; cannot be combined with frames |
#### Video input
@@ -274,6 +275,47 @@ An option the model lacks, or a combination it doesn't support, is rejected with
For more details, see the [Kling video API reference](https://kling.ai/document-api/api/video/3-0-omni/text-to-video).
#### MiniMax options
Available when using a MiniMax model (provider `'minimax-video-generation'`). The ids are MiniMax's own model names in lower case, and every model also answers to `minimax/<model>`. The H3 models take any whole number of seconds in their range and are priced per second. The Hailuo models render a 6 or 10 second clip for a flat price. All of them work in resolution tiers: pass `size` as a tier, or as `'WIDTHxHEIGHT'`, which selects the tier of the shorter side and the aspect ratio. Pass `'2k'` by name. They render at 24 frames per second.
| Model | Duration (s) | Resolutions | Audio | Last frame | Reference images |
|-------|--------------|-------------|-------|------------|------------------|
| `minimax-h3` (default) | 4 to 15, default 5 | `768p`, `2k` | Always | Yes | Up to 5 images in total |
| `minimax-h3-highspeed` | 4 to 15, default 5 | `768p` | Always | Yes | Up to 5 images in total |
| `minimax-h3-lite` | 4 to 15, default 5 | `768p`, `480p` | Always | Yes | Up to 5 images in total |
| `minimax-h3-max` | 5 to 15, default 5 | `768p`, `480p` | Always | Yes | Up to 2 images in total |
| `minimax-hailuo-2.3` | 6 or 10 | `768p`, `1080p` (6 seconds only) | No | No | No |
| `minimax-hailuo-2.3-fast` | 6 or 10 | `768p`, `1080p` (6 seconds only) | No | No | No |
| `minimax-hailuo-02` | 6 or 10 | `768p`, `1080p` (6 seconds only), `512p` (with `input_reference` only) | No | Yes | No |
The first resolution listed is the default. "Images in total" counts `input_reference`, `last_frame` and `reference_images` together. Hailuo 2.3 Fast requires an `input_reference` image. Hailuo 02 supports `512p` with a first frame alone. First and last frames together require `768p` or `1080p`.
| Model | Price |
|-------|-------|
| `minimax-h3` | $0.08 per second at `768p`, $0.13 at `2k` |
| `minimax-h3-highspeed` | $0.06 per second |
| `minimax-h3-lite` | $0.04 per second at `768p`, $0.025 at `480p` |
| `minimax-h3-max` | $0.08 per second at `768p`, $0.05 at `480p` |
| `minimax-hailuo-2.3` | $0.28 for 6 seconds at `768p`, $0.56 for 10 seconds at `768p`, $0.49 for 6 seconds at `1080p` |
| `minimax-hailuo-2.3-fast` | $0.19 for 6 seconds at `768p`, $0.32 for 10 seconds at `768p`, $0.33 for 6 seconds at `1080p` |
| `minimax-hailuo-02` | As Hailuo 2.3, plus $0.10 for 6 seconds and $0.15 for 10 seconds at `512p` |
| Option | Type | Description |
|--------|------|-------------|
| `model` | `String` | Video model to use, from the table above |
| `prompt` | `String` | Up to 7,000 characters on the H3 models, 2,000 on the Hailuo models |
| `seconds` | `Number` | Clip length, from the table above. Any other value falls back to the default. On the Hailuo models, 10 seconds at `1080p` falls back to 6 |
| `size` | `String` | Resolution tier from the table above (case-insensitive), or `'WIDTHxHEIGHT'`. `resolution` is an alias |
| `width`, `height` | `Number` | Optional. Only their ratio is used, to pick the aspect ratio of an H3 clip made from text or `reference_images`: `21:9`, `16:9`, `4:3`, `1:1`, `3:4` or `9:16`. Any other ratio gives `16:9`. A clip from `input_reference` or `last_frame` takes the image's shape. The Hailuo models ignore them |
| `input_reference` | `String \| Blob` | Image used as the first frame, in any [image input](#image-inputs) format. Required on Hailuo 2.3 Fast. JPEG, PNG or WebP, at least 256 pixels per side on the H3 models and more than 300 pixels on the short side on the Hailuo models |
| `last_frame` | `String \| Blob` | Image used as the last frame, same formats. Requires `input_reference` and `768p` or `1080p` on Hailuo 02. The H3 models also take it alone |
| `reference_images` | `Array<String \| Blob>` | Images the model uses as subject, style or scene references, same formats, on the H3 models. Cannot be combined with `input_reference` or `last_frame` |
An option the model lacks, or a combination it doesn't support, is rejected with `bad_request` before any credits are spent. MiniMax returns a hosted URL that is temporary, so pass `puter_output_path` to keep a clip. A Hailuo link works for one hour.
For more details, see the [MiniMax video API reference](https://platform.minimax.io/docs/api-reference/video-generation-v2-create).
Any properties not set fall back to provider defaults.
#### Saving to Puter filesystem
@@ -297,6 +339,7 @@ Every successful generation is charged to the user's AI credits according to the
- **Seedance** is priced per second. If the balance cannot cover the requested length, Puter shortens the clip to the longest duration the model supports that the balance does cover, and rejects with `insufficient_funds` only when even the shortest clip is unaffordable. The response does not flag a shortened clip, so read `video.duration` once metadata has loaded if the exact length matters.
- **Together AI** prices its older models per clip, so those requests are either accepted at the model's duration or rejected with `insufficient_funds`. Its per-second models (marked in the Together AI table), including the default Wan 2.7, are shortened like Seedance, and charged at the model's highest published rate per second.
- **Kling** is priced per second, and clips are shortened like Seedance. An edit of an `inputVideo` is as long as its source, so Puter holds the price of the longest source the model accepts (16 seconds on Kling 3.0 Omni, 10 on Kling O1) and charges for the length of the clip Kling returns.
- **MiniMax** H3 models are priced per second, and clips are shortened like Seedance. The charge is for the seconds of output MiniMax reports. The Hailuo models are priced per clip, so those requests are either accepted at the clip's price or rejected with `insufficient_funds`.
- **Gemini Omni** is priced per token and has no length control, so it's never shortened. Before generating, Puter counts the input tokens and holds an estimate of about $3.63 plus the input. The estimate covers 173,760 video tokens and 65,536 text and thinking tokens. Puter doesn't count the tokens of an `inputVideo`, so a request with one holds another 16,000 input tokens, about $0.02, which covers a 10-second video. Puter reads the video only after the estimate is held. A balance that can't cover it fails with `insufficient_funds` before generation starts. The charge is for the tokens Google reports for the clip. It can be higher than the estimate, because Google can go past the requested output limit. When Google's reply reports token usage but fails or has no usable clip, those tokens are charged and the request still fails.
Apart from that Gemini Omni case, a request that fails or times out is not charged. The request and concurrency limits that apply to every AI call are listed in [Rate limits and quotas](/rate-limits-and-quotas/).
@@ -309,7 +352,7 @@ Video generation is slow: expect anywhere from tens of seconds to several minute
A `Promise` that resolves to an `HTMLVideoElement` (in browsers) that you can append to the DOM straight away:
- `src` is the clip's URL: an `https:` URL on the provider's storage (Together AI or Seedance), or a `data:video/mp4;base64,…` URI holding the whole clip (Gemini Omni). Provider URLs are temporary; keep a copy with `puter_output_path` if you need the clip later.
- `src` is the clip's URL: an `https:` URL on the provider's storage (Together AI, Seedance, Kling or MiniMax), or a `data:video/mp4;base64,…` URI holding the whole clip (Gemini Omni). Provider URLs are temporary; keep a copy with `puter_output_path` if you need the clip later.
- `controls` is enabled and `preload` is `"metadata"`.
- The `data-source` attribute carries the same URL, and `data-mime-type` the MIME type when it is known (for example `video/mp4`).
- `String(video)` returns the URL, so the element can be dropped into a template or passed to `fetch()`.
@@ -329,7 +372,7 @@ A rejection carries the error body exactly as the backend sent it, or `{ message
| `message` | Human-readable reason. `error` carries the same text for older clients. |
| `code` | Stable error code; see the table below. |
| `errorCode` | A more specific code alongside a general `code`. Today the only value is `moderation_flagged`. |
| `provider` | Which upstream handled the request: `together`, `byteplus` or `gemini`. Present on errors raised while a job was running. |
| `provider` | Which upstream handled the request: `together`, `byteplus`, `gemini`, `kling` or `minimax`. Present on errors raised while a job was running. |
| `upstreamCode` | The provider's own error code, when it gave one. |
| `upstreamStatus` | The HTTP status the provider returned, when it rejected the request before a job started. |
+1 -1
View File
@@ -6,7 +6,7 @@ platforms: [websites, apps, nodejs, workers]
Reads one message by the `id` a listing returned. The whole raw message is downloaded and parsed, so the result carries the full subject, sender and recipients, the text and HTML bodies, every header, and each attachment with its bytes. Pass `raw: true` to get the unparsed `message/rfc822` bytes as a `Blob` instead.
Reading the mailbox requires the `fs:/{username}/.mail:read` permission, the same grant [`puter.email.list()`](/Email/list/) needs.
Reading the mailbox requires read access to the `~/.mail` folder, the same grant [`puter.email.list()`](/Email/list/) needs.
## Syntax
+5 -1
View File
@@ -6,7 +6,11 @@ platforms: [websites, apps, nodejs, workers]
Lists the messages in one folder of the user's mailbox, newest first. Every call returns one page of `items` plus, while more pages exist, a `cursor` to pass back for the next one. Listing reads the mailbox's folder structure only, so it never downloads a message: each item carries the subject, date, and size, and `puter.email.get()` fetches the rest.
The mailbox belongs to the user, so it is protected like the rest of their files. An app reads it only after the user grants `fs:/{username}/.mail:read`, for example with `puter.perms.request('permission', { permission: `fs:/${username}/.mail:read` })`.
The mailbox belongs to the user, so it is protected like the rest of their files. An app reads it only after the user grants read access to the `~/.mail` folder. To ask for it, call [`puter.perms.request()`](/Perms/request/):
```js
await puter.perms.request('fs', { path: '~/.mail', access: 'read' });
```
## Syntax
+3 -3
View File
@@ -20,9 +20,9 @@ What you watch is written as a subject string:
| `fs:~/Documents` | Everything under a folder, or one file by path or uid |
| `fs:~/inbox/*.json:add` | Matching paths, one kind of change |
| `kv:cart` | One key in your app's key-value store (`kv:cart*` for a prefix) |
| `notif:account` | The user's notifications |
| `notif:app-user` | Notifications about the user's use of your app |
See [Subjects and events](/Events/subjects/) for the full subject syntax, the event shape, and how to share key-value events with another user.
See [Subjects and events](/Events/subjects/) for the full subject syntax, the event shape, and how to watch key-value data another user shared with you.
With the [User-Pays Model](/user-pays-model/), deliveries are billed to the user holding the subscription, not to you. See [Rate Limits and Quotas](/rate-limits-and-quotas/#events) for limits and costs.
@@ -239,7 +239,7 @@ With the [User-Pays Model](/user-pays-model/), deliveries are billed to the user
let seen = 0;
do {
const page = await puter.events.fetch({
subject: 'notif:account',
subject: 'notif:app-user',
cursor,
});
for (const event of page.items) {
+6 -5
View File
@@ -25,7 +25,7 @@ puter.events.fetch(options)
#### `options` (Object) (required)
- `subject` (String) (required): What to read: `notif:account` (the account's notifications), `notif:app-user` (the ones belonging to the app you're running as), or `notif:<appId>:<audience>`. Audiences are `account`, `developer` (about an app, sent to its owner), and `app-user` (about your data inside an app).
- `subject` (String) (required): What to read: `notif:app-user` (the ones belonging to the app you're running as), `notif:developer` (about your app, for its owner), or `notif:<appId>:<audience>`. Audiences are `account`, `developer` (about an app, sent to its owner), and `app-user` (about your data inside an app). Apps and websites never see `account` notifications: `notif:account` returns an empty page for them.
- `cursor` (String): The `cursor` from the previous page. Leave it off to start from the oldest notification still kept.
- `after` (String): The older name for `cursor`, still accepted. If both are given, `cursor` is used.
- `limit` (Number): Events per page. Defaults to 50, max 200.
@@ -48,7 +48,7 @@ Each item is a notification event:
| `type` | String | The kind of notification, from the published catalog: `share.received`, `app.worker.deployed`, and so on. |
| `audience` | String | `account`, `developer`, or `app-user`. |
| `appUid` | String \| null | The app it's about, or `null` for platform notifications. |
| `notification` | Object | The stored payload. It always has `type`, and everything Puter sends has a `title`; other fields depend on `type` (below). |
| `notification` | Object | The stored payload. It always has `type`, and every type except `kv.share.received` has a `title`. Other fields depend on `type` (below). |
| `self` | Boolean | Always `true`: a mailbox is your own. |
| `ts` | Number | When it was created, in milliseconds since the epoch. |
| `seq` | Number | Position within the page. |
@@ -60,6 +60,7 @@ Besides `type` and `title`, `notification` carries:
- `app.worker.deployed`, `app.worker.deployFailed`: nothing else.
- `app.events.ended`: `reason` (`anchor_deleted`, `permission_revoked`, or `no_credit`); `anchor_deleted` and `permission_revoked` also add `subject`, `subjects` (up to 20) and `count`.
- `app.events.suspended`: `handler` and `subscriptions`.
- `kv.share.received`: `handle`, `access`, `label`, and `createdAt` (in Unix seconds) of a key-value share another user made with you (see [`puter.kv.listShared()`](/KV/listShared/)). It arrives on `notif:app-user` in the app whose data was shared, and the Puter desktop doesn't show it. Outside that app, `notif:app-user` leaves it out. `notif:<appId>:app-user` returns it to the account itself and to that app, and not to other apps.
Other notifications can carry other fields, so read them defensively.
@@ -83,7 +84,7 @@ The promise rejects with `{ message, code }`: `auth_canceled` if nobody was sign
let seen = 0;
do {
const page = await puter.events.fetch({
subject: 'notif:account',
subject: 'notif:app-user',
cursor,
});
for (const event of page.items) {
@@ -118,10 +119,10 @@ The promise rejects with `{ message, code }`: `auth_canceled` if nobody was sign
// Live first, so nothing arriving during the catch-up is lost —
// `id` is what makes the overlap harmless.
const sub = await puter.events.onLocal('notif:account',
const sub = await puter.events.onLocal('notif:app-user',
({ event }) => show(event));
const page = await puter.events.fetch({ subject: 'notif:account' });
const page = await puter.events.fetch({ subject: 'notif:app-user' });
page.items.forEach(show);
await sub.off();
+1 -1
View File
@@ -57,7 +57,7 @@ The promise rejects with `{ message, code }`:
| `invalid_kv_handle_key` | A `kv:<handle>:…` subject names no key, or a key that tries to leave the shared region. |
| `invalid_include_value` | `includeValue` isn't a boolean, or the subject isn't `kv:`. |
| `events_cross_app_disabled` | The subject names another app's key-value data, and that's turned off on this server. |
| `forbidden` | The other app doesn't share its data, or your app hasn't been granted `app-data:<appId>:kv:read`. |
| `forbidden` | The other app doesn't share its data, or your app hasn't been granted read access to its key-value data (see [`puter.perms.request('appData')`](/Perms/appData/)). |
| `subject_does_not_exist` | The subject doesn't exist, or this account can't read it. |
| `events_subscription_limit` | This connection already has 50 subscriptions. |
| `too_many_requests` | Over the subscribe rate limit. |
+2 -2
View File
@@ -35,10 +35,10 @@ puter.events.onPersistent(options)
## Background delivery needs consent
Running your handler while the user isn't there needs its own per-app permission, **`events:background`**. Subscribing with `worker` among the `targets` (the default for an app) without it fails with `events_background_consent_required`. Request it like any other permission:
Running your handler while the user isn't there needs its own per-app permission, **`events:background`**. Subscribing with `worker` among the `targets` (the default for an app) without it fails with `events_background_consent_required`. To ask for it, call [`puter.perms.request()`](/Perms/request/) with the `'backgroundEvents'` resource:
```js
await puter.perms.request(['events:background']);
await puter.perms.request('backgroundEvents');
```
The user can revoke it wherever they manage the app's access. That suspends every worker-target subscription the app holds for them with `permission_revoked`, and granting it again doesn't resume them: subscribe again. A subscription that only wants deliveries while your app is open needs no consent: pass `targets: ['socket']`.
+31 -28
View File
@@ -98,49 +98,50 @@ Watching another app's key-value data takes the same consent as reading it: the
Entries the other app wrote with `disableSharing` are never delivered. Marking a key private produces no event, so a value you already have for it may be stale.
## Sharing key-value events with another user
<a id="sharing-key-value-events-with-another-user"></a>
`kv:` always means your own data. To let another account watch part of yours, mint a **share handle** over a key prefix and give it to them. They subscribe with the handle where an app id would go:
## Sharing key-value data with another user
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
`kv:` always means your own data. When another user shares part of theirs with you using [`puter.kv.share()`](/KV/share/), you watch it through the share's **handle**, which goes where an app id would go. For a walkthrough, see [Send Live Updates to Other Users](/recipes/send-live-updates-to-other-users/).
```js
// The owner shares one workspace with another account.
const res = await fetch(`${puter.APIOrigin}/events/kv-handles`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${puter.authToken}`,
},
body: JSON.stringify({
granteeUsername: 'bob',
// Use a segment you'll never rename: the handle is fixed to this prefix.
prefix: `workspace:${workspaceId}:`,
}),
});
const { handle } = await res.json();
// The owner shares one workspace with another user.
await puter.kv.share(`workspace:${workspaceId}:`, 'bob', 'watch');
```
```js
// Bob watches every key written in that workspace.
// Bob finds the share and watches every key written in that workspace.
// `event.key` is relative to the handle: `messages:1`, not `workspace:<id>:messages:1`.
const { items } = await puter.kv.listShared();
const { handle } = items[0];
await puter.events.onLocal(`kv:${handle}:*`, ({ event }) => render(event.key));
```
- **The holder learns only the handle**: not whose data it is, where it sits, or anything above the prefix. `subject` and `key` in their deliveries are relative to the handle. `kv:<handle>:messages:*` narrows to part of the shared region.
- **Values:** `includeValue` works through a handle, with the same limits. A handle grants events, not reads, so when a value is left out the holder needs some other authorized way to read it.
- **Every level can watch.** A share has `watch`, `read`, or `write` access, and all three can subscribe. With `watch`, the holder only gets events. With `read`, they can also read the keys with [`puter.kv.get()`](/KV/get/) and [`puter.kv.list()`](/KV/list/), passing the handle, to catch up after a [gap](#gaps) or to read a value an event left out. With `write`, they can also change and delete the keys. [Sharing with other users](/KV/#sharing-with-other-users) covers the levels.
- **The holder learns only the handle.** It doesn't say whose data it is or where it sits. The `subject` and `key` in their deliveries are relative to the handle. `kv:<handle>:messages:*` narrows to part of the shared region.
- **Values.** `includeValue` works through a handle, with the same limits.
- **Private entries stay private.** An entry written with `disableSharing` is never delivered through a handle: no event, key, or value.
- **Use a stable prefix.** A handle stays on the prefix it was minted on. Rename `workspace:<uuid>:` to `project:<uuid>:` and existing handles point at keys nothing writes. Prefer synthetic ids (`workspace:<uuid>:`, `thread:<uuid>:`) over names that might change (`acme-corp:`).
- **Prefixes are literal.** `*`, `?` and empty segments (`workspace::abc:`) are refused with `invalid_kv_share_prefix`. The trailing `:` is optional.
- **Keys can't leave the handle.** A handle with no key after it, or a key using `..`, is refused with `invalid_kv_handle_key`.
- **Writes from every side arrive.** A change the owner makes, and a change any holder makes through a `write` share, reach the owner's subscriptions and every holder's. On a holder's own write through the handle, `self` is `true`, so `self && !viaWorker` skips your own changes. See [the event](#the-event).
**Managing handles.** `GET /events/kv-handles` lists the handles the account has minted, revoked ones included. `DELETE /events/kv-handles/<handle>` revokes one, and every persistent subscription on it is suspended with `permission_revoked` and its backlog dropped. Revoking an already-revoked handle isn't an error. See [Rate Limits and Quotas](/rate-limits-and-quotas/#events) for how many handles an account can hold. Temporary accounts can't mint handles (`events_kv_handle_requires_account`). Where the feature is off, minting and handle subjects fail with `events_kv_handles_disabled`.
When a share stops, deliveries through its handle stop, and every persistent subscription on it is suspended with `permission_revoked` and its backlog dropped. If another share to the same holder still covers its keys, live deliveries through the stopped handle can continue for up to 60 seconds. This happens when the owner revokes it with [`puter.kv.unshare()`](/KV/unshare/), when the holder removes their own access the same way, and when the app whose data it is gets deleted. Where `watch` shares are off on the deployment, handle subjects fail with `events_kv_handles_disabled`.
**Apps.** An app can mint, list and revoke handles for its user, but only inside its own namespace and with the user's consent: `manage:kv-share:<userUuid>:<appId>:<prefix>`, requested with [`puter.perms.request()`](/Perms/request/). The prefix supplies its segments, so `workspace:abc:` ends the permission as `…:workspace:abc`. The consent must name a prefix; asking for the whole namespace is refused with `invalid_kv_share_prefix`. An app that lists handles sees only its own namespace.
**Apps.** An app can share, list, and unshare its user's data with [`puter.kv.share()`](/KV/share/) and its sibling methods, but only inside its own namespace and with the user's consent. To ask for it, call [`puter.perms.request()`](/Perms/request/) with the `'kvShare'` resource, the prefix the shares will cover, and the highest level they'll have:
```js
await puter.perms.request('kvShare', { prefix: 'workspace:abc:', access: 'watch' });
```
The consent covers that prefix and every key under it, at that level and the levels below it. It must name a prefix. A prefix that covers the whole namespace is rejected with `invalid_argument`. An app that lists shares sees only its own namespace.
| Refused with | When |
| --- | --- |
| `events_kv_handle_not_delegated` | Minting or revoking outside the consented prefix, or after the consent was withdrawn (the handle stays in place). |
| `events_kv_handle_outside_namespace` | Minting outside the app's own namespace. |
| `events_kv_handle_owner_only` | The caller is an access token the app issued, a different app of the same user, or the handle is outside the app's namespace. |
| `events_kv_handle_not_delegated` | Sharing or revoking outside the consented prefix or level, or after the consent was withdrawn (the share stays in place). |
| `events_kv_handle_outside_namespace` | Sharing outside the app's own namespace. |
| `events_kv_handle_owner_only` | Listing or revoking from an access token the app issued, or revoking a share outside the app's namespace. |
| `forbidden` | Sharing from an access token the app issued. |
An app can also subscribe through a handle, but only when the shared region belongs to that same app. A different app, even for the same user, is refused as if the handle didn't exist.
@@ -160,7 +161,8 @@ The handler is called with `{ event }`. A filesystem change carries:
| `uid` | String | The uid of the node that changed. |
| `path` | String | The path of the node that changed. |
| `from` | String | On a `move`, the path the node left. Only present when the subscription was watching that location. |
| `self` | Boolean | `true` when the account holding the subscription made the change. Use it to ignore your own writes. |
| `self` | Boolean | `true` when the account holding the subscription made the change. |
| `viaWorker` | Boolean | `true` when the change came from server-side code running as an account, such as a worker using `me.puter` or an [events handler](/Events/handlers/) running in the events worker. A worker using `user.puter` acts as the user who called it, so its changes are that user's own, with `viaWorker` set to `false`. A handler running in a client writes as that client, also with `viaWorker` set to `false`. Skip your own changes with `self && !viaWorker`, which still lets through the ones a worker made as your account. |
| `ts` | Number | When it happened, in milliseconds since the epoch. |
| `seq` | Number | Position within one dispatch, when a change goes to several subscriptions. |
@@ -169,11 +171,12 @@ A key-value change carries `key` instead of `uid` and `path`, and different ops:
| Field | Type | Description |
| --- | --- | --- |
| `id` | String | Unique id for the event. |
| `subject` | String | `kv:<appId>:<key>`, naming the key that changed. |
| `subject` | String | `kv:<appId>:<key>`, naming the key that changed. Through a share, `kv:<handle>:<key>`. |
| `op` | String | `set` for a write, `del` for a removal, `expire` when only the key's lifetime changed. |
| `key` | String | The key that changed. |
| `value` | Any | The new value (`null` on a `del`). Only with `includeValue`, and only within the [limits above](#getting-the-new-value). Never on an `expire`. |
| `self` | Boolean | As above. |
| `viaWorker` | Boolean | As above. |
| `ts` | Number | As above. |
| `seq` | Number | As above. |
+66 -1
View File
@@ -9,7 +9,7 @@ It supports various operations such as set, get, delete, list keys, increment an
Puter.js handles all the infrastructure for you, so you don't need to set up servers, handle scaling, or manage backups. And thanks to the [User-Pays Model](/user-pays-model/), you don't have to worry about storage, read, or write costs, as users of your application cover their own usage.
<div class="info"><strong>Need to share data across users?</strong> Each user's key-value store lives in their own account, so one user can't read another's data. To keep a single, centralized store that every user reads from and writes to, use a <a href="/Workers/">Serverless Worker</a> — its code can act on the worker owner's resources, giving all users one shared backend.</div>
<div class="info"><strong>Need to share data across users?</strong> Each user's key-value store lives in their own account. A user can share part of theirs with another user, as described in <a href="#sharing-with-other-users">Sharing with other users</a>. To keep a single, centralized store that every user reads from and writes to, use a <a href="/Workers/">Serverless Worker</a>. Its code can act on the worker owner's resources, giving all users one shared backend.</div>
## Features
@@ -223,9 +223,74 @@ These Key-Value Store features are supported out of the box when using Puter.js:
- **[`puter.kv.expireAt()`](/KV/expireAt/)** - Set key expiration timestamp
- **[`puter.kv.list()`](/KV/list/)** - List keys in ascending or descending order
- **[`puter.kv.flush()`](/KV/flush/)** - Clear all data
- **[`puter.kv.share()`](/KV/share/)** - Share the keys under a prefix with another user
- **[`puter.kv.unshare()`](/KV/unshare/)** - Stop a share, as its owner or its holder
- **[`puter.kv.listShared()`](/KV/listShared/)** - List the shares other users made with you
- **[`puter.kv.listSharedByMe()`](/KV/listSharedByMe/)** - List the shares you made
- **[`puter.kv.MAX_KEY_SIZE`](/KV/MAX_KEY_SIZE/)** - Maximum key size in bytes
- **[`puter.kv.MAX_VALUE_SIZE`](/KV/MAX_VALUE_SIZE/)** - Maximum value size in bytes
## Sharing with other users
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
A user can share the keys under a prefix with another Puter user. The user who shares is the share's owner, and the user it's shared with is its holder. The keys stay in the owner's store.
Pick the tool by what you want to do:
| You want to… | Use |
| --- | --- |
| Let another user see or change part of your app's data | [`puter.kv.share()`](/KV/share/) |
| Let another app of the same user use your data | [`puter.perms.request('appData', …)`](/Perms/appData/) |
| Share files | [`puter.fs.share()`](/FS/share/) |
| Show data to anyone, signed in or not | [Hosting](/Hosting/) or a [worker](/Workers/). A share is for one user, who needs a Puter account. |
| Enforce rules per writer, such as letting each user edit only their own posts | A prefix per writer, each shared separately, or a [worker](/Workers/) that checks each write |
A share has one of three levels, and each includes the ones before it:
- **`watch`**: the holder gets [events](/Events/subjects/#sharing-key-value-data-with-another-user) when keys under the prefix change.
- **`read`**: the holder can also [`get()`](/KV/get/) and [`list()`](/KV/list/) keys.
- **`write`**: the holder can also change keys with every other method. Write includes delete, so every holder with write access can change or remove any key under the prefix.
An app shares its user's data only with that user's consent, asked once with [`puter.perms.request('kvShare', …)`](/Perms/request/) at the level it will share or higher. Without it, [`puter.kv.share()`](/KV/share/) fails with `events_kv_handle_not_delegated`.
The holder finds the share with [`puter.kv.listShared()`](/KV/listShared/), and passes its `handle` to the other methods. Keys are relative to the shared prefix. Here the owner writes `list:<id>:meta`, and the holder reads the same key as `meta`:
```js
// The owner, once per app: let this app share `list:` keys with write access
await puter.perms.request('kvShare', { prefix: 'list:', access: 'write' });
const prefix = `list:${crypto.randomUUID()}:`;
await puter.kv.set(`${prefix}meta`, { title: 'Groceries' });
await puter.kv.share(prefix, 'alice', 'write');
// Alice, in the same app
const { items } = await puter.kv.listShared();
const { handle } = items[0];
const meta = await puter.kv.get('meta', { handle });
await puter.kv.set(`item:${crypto.randomUUID()}`, { text: 'milk' }, { handle });
const entries = await puter.kv.list('item:*', true, { handle });
```
- **What the holder knows.** A share tells the holder its level, its app, and an optional label. It never says who shared it or what the prefix is. The label is plain text the owner sets with [`puter.kv.share()`](/KV/share/). Render it with `textContent`, never as HTML.
- **Notification.** When a user shares an app's data, the holder gets a `kv.share.received` notification in that app, on [`notif:app-user`](/Events/subjects/#notifications). It carries the `handle`, `access`, `label`, and `createdAt`. Listen for it with [`puter.events.onLocal()`](/Events/onLocal/), or read missed ones with [`puter.events.fetch()`](/Events/fetch/). The Puter desktop doesn't show it, so your app is where the holder finds out about the share. The live notification needs notifications to be turned on for events on the deployment. Calling [`puter.kv.listShared()`](/KV/listShared/) when your app opens finds new shares in every case.
- **Blocks.** A user who blocked the owner, or who doesn't accept shares from anyone, can't be shared with, and [`puter.kv.share()`](/KV/share/) fails with `recipient_not_accepting_shares`.
- **Account.** Sharing needs a confirmed email address where the deployment requires one, and no phone number, card, or paid plan.
- **Private entries.** Entries written with `disableSharing` aren't readable or listed through a share. Writing to one through a share is refused, so a holder with write access can learn that a private key exists, but never its value. Keep private entries outside the prefixes you share.
- **Writes from several users.** When two users write the same key at about the same time, the last write wins. For a log of changes, write each change to its own key, such as `chg:<timestamp>-<random>`, instead of updating one shared counter.
- **Cost.** Each user pays for their own reads and writes, including the ones they make through a share.
Either side ends a share with [`puter.kv.unshare()`](/KV/unshare/). See [Rate Limits and Quotas](/rate-limits-and-quotas/#events) for how many shares an account can make and hold.
These recipes show sharing in a full app:
- [Share a List With Someone](/recipes/share-a-list-with-someone/)
- [Let Someone View Your Data](/recipes/let-someone-view-your-data/)
- [Send Live Updates to Other Users](/recipes/send-live-updates-to-other-users/)
- [Manage Who Has Access to Shared Data](/recipes/manage-who-has-access-to-shared-data/)
- [Build a Group Chat](/recipes/build-a-group-chat/)
- [Share Data From Your Backend](/recipes/share-data-from-your-backend/)
## Examples
You can see various Puter.js Key-Value Store features in action from the following examples:
+17
View File
@@ -11,6 +11,7 @@ Add values to an existing key. When you pass an array, its elements are appended
```js
puter.kv.add(key, value)
puter.kv.add(key, pathAndValue)
puter.kv.add(key, value, optConfig)
```
## Parameters
@@ -35,6 +36,17 @@ Paths support dot notation, array indexes at any level (`[0]`, `items[0]`, or `s
A plain object is read as a path map, never appended as an element: to append an object, wrap it in an array, as in `puter.kv.add('log', [{ at, event }])`.
#### `optConfig` (Object) (optional)
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
The last argument. It takes these properties:
- `handle` (String): Work on data another user shared with you, using the `handle` from [`puter.kv.listShared()`](/KV/listShared/). The key is then relative to the shared prefix. The share needs `write` access. See [Sharing with other users](/KV/#sharing-with-other-users).
- `appUuid` (String): Work on another app's data, which the user let your app reach with [`puter.perms.request('appData', …)`](/Perms/appData/). It can't be combined with `handle`.
Pass `optConfig` after `value`, as in `puter.kv.add('tags', ['new'], { handle })`. An object in the place of `value` is read as a path map.
## Return value
Returns a `Promise` that resolves to the updated value stored at `key`.
@@ -57,6 +69,11 @@ A rejection carries an `Error` with a stable `code`:
| `subject_does_not_exist` | `optConfig.appUuid` names an app that doesn't exist. |
| `response_timeout` | Too many writers on the key right now — retry. |
| `too_many_requests` | The rate limit was exceeded. See [Rate Limits and Quotas](/rate-limits-and-quotas/). |
| `kv_handle_not_found` | `handle` doesn't exist, the share stopped, or it isn't shared with you. |
| `kv_handle_access_denied` | The share's access doesn't allow this method. |
| `kv_handle_option_invalid` | `handle` was combined with `appUuid` or `disableSharing`. |
| `kv_handle_key_invalid` | With `handle`, the key is empty or longer than 768 bytes. |
| `kv_handles_disabled` | Shares can't be used this way on this deployment. |
## Examples
+17
View File
@@ -12,6 +12,7 @@ Decrements the value of a key. A missing or expired key starts from 0 and loses
puter.kv.decr(key)
puter.kv.decr(key, amount)
puter.kv.decr(key, pathAndAmount)
puter.kv.decr(key, amount, optConfig)
```
## Parameters
@@ -33,6 +34,17 @@ When `amount` is an object: Decrements a property within an object value stored
Paths support dot notation, array indexes at any level (`[0]`, `items[0]`, or `some.path[1].to.value`), and quoted property names (`["key.with.dots"]`). An empty path (`""`) targets the whole stored value. Use non-negative integer indexes in brackets to address arrays. When a path continues through an array element (for example, `[0].score`), that element must already exist. Missing object parents are created automatically; sparse array elements are not created. A path may chain at most 31 levels; a deeper one rejects with `bad_request`. Quoted names can't be empty. All of a call's paths go into one write, so a call fits at most 1,500 path segments and, with short field names, about 60 paths; split larger changes across calls. See [Rate Limits and Quotas](/rate-limits-and-quotas/).
#### `optConfig` (Object) (optional)
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
The last argument. It takes these properties:
- `handle` (String): Work on data another user shared with you, using the `handle` from [`puter.kv.listShared()`](/KV/listShared/). The key is then relative to the shared prefix. The share needs `write` access. See [Sharing with other users](/KV/#sharing-with-other-users).
- `appUuid` (String): Work on another app's data, which the user let your app reach with [`puter.perms.request('appData', …)`](/Perms/appData/). It can't be combined with `handle`.
Pass `optConfig` after `amount`, as in `puter.kv.decr('stats', 1, { handle })`. An object in the place of `amount` is read as a path map.
## Return Value
Returns the new value of the key after the decrement operation — a number for the plain form, or the whole stored value when `amount` is a path map.
@@ -55,6 +67,11 @@ A rejection carries an `Error` with a stable `code`:
| `subject_does_not_exist` | `optConfig.appUuid` names an app that doesn't exist. |
| `response_timeout` | Too many writers on the key right now — retry. |
| `too_many_requests` | The rate limit was exceeded. See [Rate Limits and Quotas](/rate-limits-and-quotas/). |
| `kv_handle_not_found` | `handle` doesn't exist, the share stopped, or it isn't shared with you. |
| `kv_handle_access_denied` | The share's access doesn't allow this method. |
| `kv_handle_option_invalid` | `handle` was combined with `appUuid` or `disableSharing`. |
| `kv_handle_key_invalid` | With `handle`, the key is empty or longer than 768 bytes. |
| `kv_handles_disabled` | Shares can't be used this way on this deployment. |
## Examples
+22
View File
@@ -9,15 +9,37 @@ When passed a key, will remove that key from the key-value storage. If there is
## Syntax
```js
puter.kv.del(key)
puter.kv.del(key, optConfig)
```
## Parameters
#### `key` (String) (required)
A string containing the name of the key you want to remove.
#### `optConfig` (Object) (optional)
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
The last argument. It takes these properties:
- `handle` (String): Work on data another user shared with you, using the `handle` from [`puter.kv.listShared()`](/KV/listShared/). The key is then relative to the shared prefix. The share needs `write` access. See [Sharing with other users](/KV/#sharing-with-other-users).
- `appUuid` (String): Work on another app's data, which the user let your app reach with [`puter.perms.request('appData', …)`](/Perms/appData/). It can't be combined with `handle`.
## Return value
A `Promise` that will resolve to `true` when the key has been removed.
## Errors
With `handle`, the promise can reject with these codes:
| Code | Meaning |
| -- | -- |
| `kv_handle_not_found` | `handle` doesn't exist, the share stopped, or it isn't shared with you. |
| `kv_handle_access_denied` | The share's access doesn't allow this method. |
| `kv_handle_option_invalid` | `handle` was combined with `appUuid` or `disableSharing`. |
| `kv_handle_key_invalid` | With `handle`, the key is empty or longer than 768 bytes. |
| `kv_handles_disabled` | Shares can't be used this way on this deployment. |
## Examples
<strong class="example-title">Delete the key 'name'</strong>
+22
View File
@@ -10,6 +10,7 @@ Set the time-to-live (TTL) in seconds for a key in the key-value store.
```js
puter.kv.expire(key, ttlSeconds)
puter.kv.expire(key, ttlSeconds, optConfig)
```
## Parameters
@@ -22,10 +23,31 @@ A string containing the name of the key.
The number of seconds until the key is removed from the key-value store. `0` or less expires the key immediately.
#### `optConfig` (Object) (optional)
<div class="info">KV sharing is in beta. Method shapes, limits, and behavior may change between releases.</div>
The last argument. It takes these properties:
- `handle` (String): Work on data another user shared with you, using the `handle` from [`puter.kv.listShared()`](/KV/listShared/). The key is then relative to the shared prefix. The share needs `write` access. See [Sharing with other users](/KV/#sharing-with-other-users).
- `appUuid` (String): Work on another app's data, which the user let your app reach with [`puter.perms.request('appData', …)`](/Perms/appData/). It can't be combined with `handle`.
## Return value
A `Promise` that will resolve to `true` when the expiration has been set.
## Errors
With `handle`, the promise can reject with these codes:
| Code | Meaning |
| -- | -- |
| `kv_handle_not_found` | `handle` doesn't exist, the share stopped, or it isn't shared with you. |
| `kv_handle_access_denied` | The share's access doesn't allow this method. |
| `kv_handle_option_invalid` | `handle` was combined with `appUuid` or `disableSharing`. |
| `kv_handle_key_invalid` | With `handle`, the key is empty or longer than 768 bytes. |
| `kv_handles_disabled` | Shares can't be used this way on this deployment. |
## Keys That Don't Exist
A missing or already-expired key becomes an empty entry with the TTL you gave it, rather than staying absent. [`puter.kv.list()`](/KV/list/) shows it and [`puter.kv.get()`](/KV/get/) reads it as `null`, the same as a missing key — the old value, if there was one, never comes back. [`puter.kv.incr()`](/KV/incr/) rejects such an entry, the same as any non-number value.
Loaded 100 of 266 files, more files were not shown because too many files have changed in this diff. Show more