Compare commits

...
Author SHA1 Message Date
Nariman Jelveh 4ce24e63b2 Load the app browser embed's script from beside the GUI bundle
The embed page always loaded /dist/apps-embed.min.js from the server,
while the GUI bundle and the embed build are published to gui_bundle's
location (a CDN in production). The server's /dist copy can be stale, so
the embed ran an old build. It now loads apps-embed.min.js from
gui_bundle's directory, keeping /dist when gui_bundle isn't set.
2026-11-05 20:39:16 -07:00
Nariman Jelveh fb9e7d70c6 Open apps from the embedded app browser at their own URL (#4510)
* Open apps from the embedded app browser at their own URL

Selecting an app in the /embed/apps browser now opens it in a new tab at
its index_url, with the same launch params the dashboard's tab launch
uses, instead of the GUI's /app/<name> landing. The tab is opened inside
the click and without noopener; the app's SDK asks the embed for its
token through the opener, so the token stays out of the URL.

Apps that need the desktop (built-in and godmode apps, private apps the
user can't open, or a failed token request) still open /app/<name>, with
the tab's opener cleared. Website shortcuts are unchanged.

The app URL, the app token request and the /rao report move out of
launch_app into helpers without desktop globals (appLaunchUrl.js,
appLaunchApi.js), so launch_app and the embed build the same URL.
uuidv4 moves to a helper the embed can import.

* Tag app launches with utm_medium and a utm_campaign naming the launch surface

Apps now get utm_medium=referral and utm_campaign=desktop, dashboard, or
app_browser_embed alongside utm_source=puter.com, so their analytics can
tell where in Puter they were opened from.
2026-11-05 20:31:30 -07:00
Nariman Jelveh 819e77bb4c Open dashboard apps and files in a new tab at the app's index_url (#4508)
* Open dashboard app tiles in a new tab at the app's index_url

Clicking an app tile in the dashboard now opens the app in a new browser
tab at its index_url, with the same query params (puter.app_instance_id,
puter.app.*, puter.auth.token, puter.domain, puter.api_origin,
puter.locale, puter.args, ...) its in-page iframe gets.

launch_app takes a newTab option: it runs the normal launch (private
access, file signing, token) and sends the caller's tab to the built URL
instead of opening a window. The tile click opens the blank tab inside
the click so popup blockers allow it, and closes it if the launch fails.

* Hand an app tab its token through the opener, not the URL

An app opened in its own tab no longer gets puter.auth.token in its URL,
which kept the token in browser history, and which puter.js ignored anyway
outside an iframe.

The GUI now holds the token for the tab it opened. Once the app loads,
puter.js (standalone, with puter.app_instance_id in its URL and an opener)
asks its opener for it at the configured GUI origin, takes only a reply
from that window at that origin, and then clears window.opener. The GUI
answers once, only to that tab at the origin it sent the tab to, and
drops the offer after two minutes. Sign-in and implicit auth wait for the
handoff, then fall back to the normal sign-in flow when no token came
(opener closed, slow load, redirect to another origin).

Built-in apps join explorer in opening in-page, since they belong to the
desktop.

* Open files from the dashboard Files tab in a new tab

Opening a file in the Files tab (double-click, Enter, tap, or the Open
menu item) now opens its app in a new browser tab, the same way app
tiles do. The tab is opened inside the gesture and handed to open_item,
which passes it to launch_app or closes it when the file opens no app.
Web links reuse the tab, cut off from the dashboard. A second file of a
multi-select Enter, whose tab the popup blocker refuses, opens in-page.

The opened file's signed read/write URLs don't expire, so a tab gets
them through the opener handoff with its token instead of in its URL;
the URL keeps only puter.item.uid as a marker. puter.js delivers the
handed-over item to onLaunchedWithItems / onItemsOpened, and
wasLaunchedWithItems answers true while it's on its way.

launch_app now closes a tab it was given whenever the launch fails, so
callers don't each have to.

* Open "Open With" apps from the Files tab in a new tab

Picking an app under Open With in the dashboard Files tab now opens it
in a new browser tab, like opening the row. The tab is opened in the
menu click, or, when the "always open with" prompt comes first, right
after it's answered, since that click counts as a fresh user gesture.
If the popup blocker refuses the tab, the app opens in-page. The desktop
keeps opening in-page.

* Wait for a GUI-opened tab's token in getUser, whoami and getProfile

A tab the GUI opens for an app gets its token over postMessage moments
after puter.js loads. These calls short-circuited on a missing token, so
an app reading the user at startup failed with a 401 (or a null profile)
even though the token was on its way. They now wait for the handoff
first, the same way sign-in already does.

* Close an app's blank tab before alerting that its launch failed

A launch denied for a private app, or one whose token couldn't be
fetched, awaited its alert while the blank tab it was given stayed in
front. The alert renders on the dashboard behind it, so the user was
left on an empty tab until they went looking. The tab is now closed
before the alert shows.

* Launch an app tile in-page when the browser refuses its tab

When window.open returned null (a popup-blocking setting, or a context
that can't open tabs), clicking an app tile did nothing at all. It now
falls back to the in-page launch it had before, as opening a file from
the Files tab and Open With already do: an open window is surfaced,
otherwise the app opens maximized with the tile's open morph.

* Open no tab for a file opened from the Trash

Opening a file in the Trash only explains that it can't be opened, but
the Files tab still opened a blank tab for it first, which then closed
again: a flash of a new tab before the alert. Trashed files (by the
path a shortcut resolves to) now get no tab.

* Don't count a tab launch as one that can report interactivity

launch.ipc_tracked on app-is-ready is the denominator for the
app-interactive series: launches that could produce one. An app opened
in a new tab never signals over IPC, yet was recorded as tracked, so
every tab launch counted as an interactive span that never came. Tab
launches are now recorded as untracked.

* Leave openerLaunch_ unset when there is no GUI opener to ask

requestLaunchFromOpener resolved null asynchronously even when there was
no opener at all, so openerLaunch_ was briefly set on every boot with an
app instance id in the URL. A reloaded app tab (opener already cut off,
URL still marking the opened item with puter.item.uid) reported
wasLaunchedWithItems() as true to code running in the same task as the
boot, though no item would ever arrive. With no opener to ask it now
returns null synchronously and nothing is left pending.

* Call an app tab launch off when the user closed its tab

Closing the blank tab while the launch's fetches were in flight didn't
stop the launch: it still navigated the closed tab, offered it a token,
and recorded the app as opened (recent apps and /rao). The launch now
checks the tab before sending the app there and ends as not launched,
without an alert, when the user has closed it.

* Keep a same-origin opener instead of asking it for a GUI launch

Any top-level page with an app instance id and an opener asked that
opener for its launch, waited out the timeout when no answer came, and
then nulled window.opener. A page an app opened from its own origin (a
pop-out of itself, say) can't have the GUI as its opener, yet it waited
five seconds before its launch item and sign-in, and lost its link to
the app window that opened it. An opener readable from this page, on an
origin other than the GUI's, is now left alone and not asked.
2026-11-05 19:27:46 -07:00
Daniel Salazar 7355eaa77b feat(realtime): run the host prompt check per response and log its cost (#4507)
Each response that starts runs `ai.prompt.validate` on the text the client
sent since the previous check (instructions, message text and transcripts,
function call output), and its usage follows as `ai.prompt.cost-calculated`
under the same completion id. A refusal closes the session with
`permission_denied`. Nothing runs when no host listens for the check.
2026-11-05 18:05:43 -07:00
Nariman Jelveh d55aac1f98 Add embeddable Puter toolbar and shared app browser (#4019)
* Add embeddable Puter toolbar and shared app browser

Provide deployment-scoped iframe routes, a responsive host helper, read-only app browsing, saved-account switching, and new-tab account and upgrade links. Validate account sessions before switching and keep credentials inside the GUI origin.

Verified with 182 targeted tests, production GUI build, backend type checking, lint, and a real-backend browser smoke test. Translation checker could not run because tools/check-translations.js is missing.

* Prevent embedded toolbar from scrolling when menus open

* Stop double-encoding translated toolbar labels

i18n() already HTML-encodes its result, so wrapping it in escape() again
turned accented labels into literal entities (e.g. "Schlie&szlig;en") for
screen readers, and .text() did the same for the error message.

* Load only the preferred language in the toolbar and app browser embeds

Importing i18n.js pulled every translation into each embed, making the
toolbar bundle 1.1 MB. The embeds now bundle English and fetch the user's
language on demand (toolbar 1.12 MB -> 179 KB, app browser 1.18 MB -> 237 KB).
The desktop bundle keeps all translations.

* Keep toolbar buttons in place when a menu opens near the viewport edge

On phones the expanded frame was clamped to an 8px viewport margin, which
moved the whole toolbar row (by the header padding minus 8px) under the
user's finger. The frame now shrinks to fit beside and below the collapsed
toolbar, and only falls back to shifting when there isn't enough room.

* Don't pull focus back into the toolbar when the host page closes its menu

Clicking a host text field while a toolbar menu was open closed the menu and
then refocused the toolbar button, so the host field lost focus and typing
went nowhere. Focus is now restored only while the toolbar document has it.

* Close toolbar menus when the empty part of the expanded frame is pressed

While a menu is open the toolbar frame covers host content beside the
buttons; presses there reached neither the host nor the toolbar's outside
click handling, so the menu stayed open. They now close it.

* Omit the empty email line in the toolbar account menu

Temporary accounts and saved accounts without an email rendered an empty
element, adding a stray gap under the username.

* Stop user-hosted sites from framing the toolbar and app browser embeds

frame-ancestors allowed every *.<domain> subdomain, which on the default
self-hosted layout includes user sites (*.site.<domain>) and private apps
(*.app.<domain>), letting any hosted page clickjack sign-out and account
switching. The policy now names the embedding page exactly, from the
request's referrer, and refuses the user-content hosting domains.

* Use the dashboard danger color for the toolbar sign-out confirmation

The hardcoded red had 3:1 contrast on the dark panel, below WCAG AA for
14px text. The dashboard's danger token is 4.8:1 in light and 6:1 in dark.

* Drop the dashboard's hamburger clearance from the embedded app browser

At narrow widths the dashboard pads the search 60px down to clear its fixed
menu button. The embeds have no such button, so the toolbar's app panel
lost 90px of a ~500px frame to empty space above the search.

* Keep the toolbar's Upgrade button legible in dark mode

Upgrade was filled with the link color, which dark mode lightens to
#6b8cff: 3.1:1 under white text. Use the dashboard's primary-button fill
(4.6:1 in both themes) and its hover color, and drop the button classes
nothing styles on this page.

* Keep the toolbar's apps icon legible on hosts in the other theme

The toolbar follows the OS theme but draws on the host page, so in dark
mode on a light host its icon was #d4d4d4 on white (and the reverse on a
dark host). Give the apps toggle a surface of its own, like the avatar
beside it, and the session-unavailable notice one too.

* Let the toolbar sign out of a session that already ended

When the session was ended elsewhere (revoked, or signed out in another
tab), the anti-CSRF and logout requests answer 401 and sign-out failed
on every retry, stranding the stale token. Treat 401 as already signed
out and finish clearing it locally, as the desktop does; other failures
still keep the session and show the error.

* Keep the toolbar's slot following the host layout while a menu is open

The helper reserved the toolbar's space with a span frozen at the
frame's resolved size and margins, so a host that right-aligns it with
margin-left: auto (or sizes it in percentages or media queries) left the
open menu anchored to a stale spot, overflowing the header, after a
resize or rotation. Reserve it with an empty clone of the frame instead,
which the page's own rules keep laying out; if they target
data-puter-toolbar, which the clone drops so hosts still find one frame,
hold the frame's measured size.

* Reuse isOrgSeat for the toolbar's Upgrade check

The toolbar re-implemented the dashboard's organization-seat test
inline. Use the shared helper so both surfaces agree on who sees an
upgrade offer.

* Point comments at AppBrowser.js, where the Apps tab code now lives

The constants and methods these comments name moved out of TabApps.js
when the app browser became shared.

* Restore the license header on the new embed and toolbar files

TabApps.js lost its AGPL header when its code moved to AppBrowser.js,
and the new embed, toolbar, and host-helper modules were added without
one; every other GUI source file carries it.

* Remove the embeddable toolbar, keeping the standalone app browser

Drop /embed/toolbar, its page and host helper, their styles, strings and
tests, and the dashboard's ?upgrade=1 deep link that only the toolbar's
Upgrade button used. /embed/apps no longer allows the GUI origin to frame
it with a wildcard over deployment subdomains, which existed only for the
toolbar's nested app browser, and its bootstrap config no longer carries
the domain the toolbar read.
2026-11-05 17:19:05 -07:00
Daniel Salazar c6987d9606 feat(notifications): push unless a page is visible and focused (#4506)
A push is skipped only when a page that got the notification live was
visible and focused. Hidden, unfocused or unanswering pages no longer hold
it back, and pages no longer show browser notifications themselves, so the
push reaches every subscribed browser.

- Receipts carry the page's visibility and focus as of arrival; the settle
  gains `handled`. `showsOwn` is gone.
- Pages send `notif.handled { uid }` on the socket a notification arrived on
  when they got it while visible and focused. Core relays it as the
  node-local `notif.handled` event (60 a minute per user) for extensions.
- puter.js drops the page-shown notification path; push-sw.js drops its
  page-show message. Handlers still run live in the page.
- The desktop registers /sw.js only when push is turned on, updates it when
  subscribed, and removes one left without a subscription. Its hidden-tab
  banners are gone.
- Docs describe the focus rule.
2026-11-05 14:43:33 -07:00
Filip Kujundžić 796b42c74d Merge pull request #4475 from HeyPuter/fix/openai-cache-write-tiers 2026-11-05 21:27:46 +01:00
404oopsandClaude Opus 5.5 e2043a5e58 Merge remote-tracking branch 'origin/main' into fix/openai-cache-write-tiers
Main's docs cleanup (#4504) moved the prompt caching docs under
per-provider sections; the OpenAI paragraphs now live under
OpenAI models > Prompt Caching.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 19:55:55 +01:00
Filip Kujundžić 8b98c7ada8 Merge pull request #4503 from HeyPuter/codex/openai-image-default
feat(ai): add GPT Image 2.6 and default to Flare
2026-11-05 19:53:05 +01:00
Filip Kujundžić b27e84d181 Merge pull request #4457 from HeyPuter/fix/convo-bounded-recovery
Bound conversation recovery and report omitted replay context
2026-11-05 19:46:10 +01:00
404oopsandClaude Opus 5.5 add0e7be7d fix(convo): report routine context trimming as context_trimmed
A reconnect that left older items out of the 40-item, 48 KiB restore window
fired an error event with context_restore_failed, the code a provider
refusal uses, with the count only in the message. Every reconnect of a long
conversation raised an error that apps treating error as fatal would surface.

Trimming now fires a nonfatal context_trimmed event, before ready, with the
count in omittedItems. context_restore_failed is kept for a provider refusal
of a restored item or a failed send of the replay.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 19:28:02 +01:00
404oops b5d221ede6 fix(convo): bound recovery attempts and report omitted context 2026-11-05 19:28:02 +01:00
404oops 3137a631a7 fix(realtime): reuse settled credit for post-record continuation 2026-11-05 19:28:02 +01:00
404oopsandClaude Opus 5.5 57706d8093 fix(realtime): bound unacknowledged audio by duration, not a byte cap
The per-commit cap of 1,280,000 bytes assumed G.711, so a PCM16 session
(48,000 bytes/s, the SDK's format) with transcription on was closed with
1013 after about 27 seconds without a commit: silence under server VAD,
one long utterance, or a long assistant reply.

Unacknowledged audio is now a duration at the provider-confirmed input
format (the lower rate while a requested format is unconfirmed). Without
turn detection it counts all audio since the last commit or clear; with
server VAD only speech from speech_started, plus the buffer after an
explicit commit, since the provider drops silence. Each started 160 s
claims one spare item as its hold instead of closing the session; a
commit, clear or input_audio_buffer_commit_empty returns the holds to the
pool. If the upstream closes first, every claimed item is billed. The
settle no longer commits silence.

The relay test file now expires the in-memory per-IP pre-auth window
after each test; every test connects from loopback and the added tests
pushed the file past 120 upgrades a minute.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 19:27:54 +01:00
Reynaldi ChernandoandClaude Opus 5.5 fec1d1916f Merge remote-tracking branch 'upstream/main' into codex/openai-image-default
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-06 01:03:08 +07:00
Juan Fernando CastroandReynaldi Chernando da369039e0 docs: complex teams recipe (PUT-1986) (#4095)
* docs: complex teams recipe (PUT-1986)

Showing which teammates have a file open, built from a folder shared with the
team plus `puter.events` watching it. There is no presence API, so the recipe
is about composing the two.

One use case per the recipe guidance. Uses the masked path from
`listShared()` so members address the owner's folder, keys presence files on
`uuid` rather than username, and re-reads the folder on each event so a missed
delivery cannot leave the list wrong.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-11-06 00:53:25 +07:00
404oopsandClaude Opus 5.5 4f36152850 fix(realtime): hold transcription credit per item at the selected model
Input transcription reserved 64 items up front at the costliest model's
price and largest cost factor, about $7 before factors per session. Ordinary
subscribers couldn't enable it, and concurrent sessions were refused while
balance was left.

Each item now holds its own credit at the selected transcription model's
budget and cost factor. Two spare items are reserved before configuration or
audio is forwarded and refilled as the provider commits items, within the
64-item pending cap. A commit with no spare reserves its own item. A model
change releases the spares and reserves new ones at the new price; committed
items keep their hold. When the account can't cover another item, client
reads pause while the session's own holds may still free credit, and the
session is refused with insufficient_funds once it holds none.

Audio awaiting a commit acknowledgement is now tracked separately from pool
capacity and reset by each acknowledged commit or clear. If the upstream
closes before that acknowledgement, the audio bills as one item. Previously
the fallback required an untouched window, so a lost acknowledgement after an
earlier completed item went unbilled; the cumulative byte count could not
simply be reused without charging earlier work twice. The uncommitted-audio
cap is now 160 seconds of audio instead of a per-window total.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:49 -07:00
404oops 86db96d587 fix(realtime): reserve and settle pending transcription work 2026-11-05 10:23:49 -07:00
404oops 214ffb6605 fix(realtime): evict revoked credentials and recheck paid eligibility 2026-11-05 10:23:48 -07:00
404oops 871cc2cf87 fix(realtime): bound upgrades before credential admission 2026-11-05 10:23:48 -07:00
404oops ba002be0f7 fix(convo): reject interrupt readiness failures asynchronously 2026-11-05 10:23:47 -07:00
404oops 2832926f8d test(realtime): isolate credential attempt budgets between cases 2026-11-05 10:23:47 -07:00
404oops 1ab1c04760 fix(realtime): redact structured provider diagnostics 2026-11-05 10:23:47 -07:00
404oopsandClaude Opus 5.5 63a98f89ff test(metering): find the actor key in any meterIncr argument
The buffered write now passes the claim key too, which moves the counter
key's position. The image provider's write-ordering test matched it by
index and stopped seeing the write.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oops d2cef27085 perf(metering): avoid retries for growing buffers 2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 74b60c1227 fix(metering): type the array read's retry callback
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 2ed3e78dfa fix(metering): keep buffered and store reads from missing or doubling usage
Three reads still got a flush wrong:

- A write re-sent after its reply was lost, found already applied, left
  the claim without touching a warm cached base, so balance reads stayed
  short of what the store held. The claim and the base are now dropped
  together, and the next read seeds a fresh base from the store.
- A read with no cached base took the buffer, then the store; a flush
  that wrote a batch and left its claim in between had it counted twice.
- An exact read added another deployment's claim but not the usage that
  arrived behind it, which that read can't flush, so a charge crossing
  the allowance went entirely to the allowance.

Reads that go to the store now take everything buffered (delta and
claim), read the store consistently, and read the buffer again, retrying
when it shrank in between. Single and array reads and exact reads all
use it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 22ebd353b8 fix(metering): read the claim before the store in an exact read
`readExact` read the store and the outstanding claim side by side. When
a flush wrote a batch and took it off its claim between the two, the
store read had the counter from before the write and the claim read had
nothing, so the batch was missed: 10 units recorded, 0 read.

The claim is now read first. An amount gone from it by then was written
before the store read, which is consistent, so the read has it. When the
claim changes during the store read, the batch may be in both, so the
read is retried, up to three times.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 7551e12370 fix(metering): count claims on the write path and read missing bases exactly
The running total `incr` returns and `readExact` also missed usage a
flush had claimed but not yet written. Both feed how a charge is split
between the allowance and purchased credit, so an account could be
charged to its allowance for usage that had already used it up. Both
now add the counter's outstanding claim.

A read or seed with no cached base asked the store with an ordinary
read, which could miss a write a settle had just made; it now reads
consistently. A write-through after a cache failure drops the cached
base, which doesn't have the write, so the next read reseeds it.

The dirty-set reconcile test no longer depends on how many flush cycles
earlier tests ran.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 6e29e32fe9 fix(metering): count usage mid-flush in balance reads
A flush takes a counter's buffered amounts aside before writing them
onward, and reads only looked at the live delta and the cached base.
From the claim until the settle replaced the base, those amounts were
missing from every read: one store write normally, up to the orphan
sweep if the write failed. Balance checks, credit reservations among
them, could see an account as having more than it does.

Reads now add the counter's outstanding claim. A batch written onward
leaves the claim and joins the cached base in one step, so it stays
counted until the settle replaces the base with the store's view.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 54bfc0ffe5 test(puter-js): prove microphone chunks hit the closing-socket gap
The browser test now counts the chunks the transport refused while the
socket read as closing, and requires at least one, so it can't pass
without exercising the gap. The socket stays closing until the
connection drops instead of going back to open.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:45 -07:00
404oopsandClaude Opus 5.5 c27883134b fix(puter-js): keep microphone capture when the socket closes first
A conversation still reads as connected for a moment after its socket
starts closing. A microphone chunk sent in that moment got
`session_closed`, and the capture handler treated that like a real
media failure: it stopped the tracks and the AudioContext, so the
replacement session after the reconnect had no microphone.

The built-in microphone now drops a chunk the transport refuses with
`session_closed` or `session_not_ready`, as it already did for
`queue_full`, and keeps capturing. A reconnect resumes sending; a final
close still stops capture through the conversation, and processor or
other errors still release it. Dropped chunks aren't replayed, and
public sends keep reporting the transport error.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:45 -07:00
404oopsandClaude Opus 5.5 cf4c1f6b64 fix(puter-js): end the conversation on an oversized event mid-reconnect
A server event over the 4 MiB limit closes the socket with 4009, which
ends a connected conversation. During a reconnect attempt the same close
was reported as `session_closed`, which is retried, so the conversation
tried ten more times and finally closed with the original drop code.
A replacement session opened after a restart notice was reopened again
and again for the same reason.

A 4009 close now ends the conversation during a reconnect too, reporting
4009, and a replacement session that hits it isn't reopened. Tests cover
both, a multibyte event of exactly 4 MiB, and a whole retrieved item
through the relay.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 305d20ab1c fix(puter-js): accept provider events up to the relay's 4 MiB cap
The conversation transport refused any server event over 64 KiB with a
terminal 4009 close, the limit meant for what the SDK sends. The relay
forwards provider events up to 4 MiB, and a session.updated echo or a
whole audio item easily passes 64 KiB, so valid conversations ended.

Incoming events now have their own 4 MiB limit, matching the relay,
counted in UTF-8 bytes (a string short enough that it can't exceed the
limit isn't encoded to check). An event over it still closes with 4009
and doesn't reconnect. Uploads keep their 64 KiB cap, and malformed
events are still ignored.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 0ce6b0646a perf(metering): stop releases from churning the reservation version
Every release bumped the balance version, so an account's own short-lived
holds (each realtime session probes one and releases it at once) could
keep its other reservations reading a stale version until they were
refused. The usage a hold stands for already bumps the version when it's
recorded, before the hold is released, and a release without usage
leaves the balance as it was, so releases no longer bump it.

The admission read also uses the cached add-ons record when it shows no
purchased credit left, which can only understate the balance; it reads
consistently only while credit remains.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 6b5c4f3d9e fix(metering): refuse a reservation whose balance never holds still
A reservation that found the balance changed three times in a row used
to skip the version check on its fourth attempt, which brought back the
race the check closes; an account could provoke that itself. It now
waits a short randomized time between attempts and refuses after five.

A missing version key now starts from a random value, so a version read
before the key expired can't match its successor. Credit grants,
allowance changes, manual usage corrections and a usage write that
fails partway also bump the version, and the store refuses an amount
it can't hold instead of storing a negative hold.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 d6fdc5f268 fix(metering): reserve AI credit only when it fits, in one step
`reserveAiCredits` checked the balance and then took an unconditional
hold, so concurrent requests could all pass the check before any hold
was visible; two 16.2M µ¢ holds were admitted against a 20M µ¢ balance.

The hold store now takes a hold only if it fits beside the live holds,
in one Lua script on the actor's hold key. A per-user version, bumped
when usage is recorded and when a hold is released, lets the script
refuse a balance read that a settlement has since overtaken; the
service reads the balance again and retries, and skips the version
check on its last attempt rather than refuse an account whose balance
keeps moving. Both keys share the actor's hash tag, so the script runs
on Redis Cluster. The balance read for this check reads purchased
credit consistently.

`reserveCreditsIfAffordable` carries this through the service, and the
AI facade calls it once with the cost factor applied. `reserveCredits`
keeps its unconditional semantics. When the cache is unreachable, the
balance alone decides and nothing is held, as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 4f3f946941 fix(realtime): judge a refusal by the plan it was decided on
A plan or funds refusal was classified by a plan lookup made after the
decision. A fallback plan is cached for only a few seconds, so when it
ran out in between, the refusal was judged against the recovered plan
and still ended the session with a terminal 1008. The plan is now also
checked just before each decision, and a refusal is retryable if either
check found a fallback.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 4e36672b6b test(puter-js): fail credential lookups until the relay hits one
The reconnect test made the next credential lookup fail once, but the
SDK's own socket.io connection could take that failure before the
relay's reconnect attempt did, so the relay never saw a failed lookup
and the test failed in CI. Lookups now fail until the relay has refused
an attempt on one, then recover.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 6b0b530ab1 fix(realtime): retry instead of refusing when the plan lookup fails
When the billing lookup failed, metering answered with a provisional
fallback plan, so the relay refused the session with
`subscription_required` or, against the fallback's allowance,
`insufficient_funds`, both terminal 1008 closes, and the SDK showed the
upgrade prompt to a paying account. A plan or funds refusal decided on a
provisional plan now sends `internal_error` and closes with 1011, at
connection and mid-session, so the client retries once the lookup
recovers.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 68b8f341e9 test(realtime): cover a revoked session credential
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 c3756d3ac3 fix(realtime): tell transient server failures apart from bad credentials
A credential lookup that threw (a database or cache outage) was treated
as no actor, so the session got `invalid_api_key` and a 1008 close. That
is terminal for the SDK, which then ended an established conversation
that only needed to reconnect. A failed lookup now gets `internal_error`
and closes with 1011, which the SDK retries; missing, invalid and
revoked credentials keep `invalid_api_key` and 1008. Nothing is cached
for the failed lookup, and the cause is logged server-side only.

A session whose usage records pile up behind metering now closes with
1013 instead of 1008, still with `rate_limit_exceeded`. Queued usage is
still recorded and its holds released. An unexpected setup failure now
sends `internal_error` before its 1011 close.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 0c927ac5a4 fix(realtime): listen for client socket errors before refusing a session
A session refused while the server drains returned before its client
error and close listeners were registered, so a frame that errored the
socket during the close handshake (an oversized one, for example)
emitted an unhandled `error` and ended the process. The listeners now
go on as soon as the session exists, ahead of every refusal.

The 404 answer to an upgrade for another host also ends a raw socket
that has no error listener once Node hands it over; it now has one.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:42 -07:00
Neal Shah 64c3b3c0ce subdomain and worker recipes complex, writing full stack applications with both (#4206) 2026-11-05 21:33:21 +07:00
Reynaldi Chernando e9a4c7a461 ai docs cleanup (#4504)
* ai docs cleanup

* fixes
2026-11-05 19:15:32 +07:00
Filip Kujundžić be9cd9eb89 feat(ai): add GPT Image 2.6 and default to Flare 2026-11-05 11:13:43 +01:00
Daniel Salazar 8b64cc5076 docs(notifications): recipes and format pass for dismissal, priority, buttons and schedule updates (#4502)
- dismiss.md is the one place that explains closing a notification on other
  devices; on/onNotification/push link to it instead of repeating it.
- Pages follow the house format: section order, error tables, playground
  examples for dismiss() and markShown(), sidebar and index listings.
- updateSchedule.md describes what a developer sees instead of internals.
- New recipes: mark notifications as read, incoming calls (high priority,
  short ttl, buttons), unread counts on the app icon.
- Existing recipes move reminders, snoozes and recurring summaries with
  updateSchedule() instead of cancel-and-recreate; the buttons recipe follows
  the recipe format.
2026-11-05 00:32:53 -07:00
Daniel Salazar d3869b7846 feat(notifications): dismissal across devices, high priority, buttons, schedule updates, docs fixes (#4501)
- Docs: status(), browser limits, events worker fetch/ack, update() across
  pages, stored-notification retries, device removal, push delivery, queue
  coalescing, existing service workers and missing error codes match the code.
- puter.notifications.markShown(uid) / dismiss(uid).
- Dismissals reach the app's live feed (on('dismiss')); pages close the
  notification on screen, and a later page load closes ones already dismissed.
  The desktop closes its own copies on notif.ack.
- Live notification postings are forwarded to sessions in other regions.
- notify({ priority: 'high' }).
- onPush() results accept requireInteraction and actions; clicks carry action.
- replaced reaches service-worker onPush() handlers.
- puter.events.updateSchedule(scheduleId, changes), with the invoker
  re-checking a schedule run's guard right before it is sent.
2026-11-05 00:17:27 -07:00
Daniel Salazar 91230697cd fix(backend): shutdown hooks don't skip each other, and final flushes wait for in-flight work (#4499) (#4500)
A throwing prepare/shutdown hook stopped the loop, so shutdown() skipped
closeAllConnections and every later hook, including the final metering
flush; each hook now runs and failures are logged. Broadcast, events
forwarding and share digests joined nothing already in flight and could
drop what was queued behind it; their final flushes now wait for it, ship
the tail, and drain once more at shutdown. Local worker disposal and the
OpenAI image metering call are awaited. settleWithin and
FINAL_FLUSH_TIMEOUT_MS bound each final flush.
2026-11-04 21:47:35 -07:00
Daniel Salazar 56e4856518 feat(backend): drain-start hooks run concurrently and can hold the listener open (#4498)
The listener closes once every onServerDrainStart hook settles, and the
drain window now starts at the signal, so time a hook spends holding the
listener open counts against drainMs instead of extending shutdown. A
failing hook no longer skips the others.
2026-11-04 19:40:54 -07:00
Reynaldi Chernando 8acd73675c use gpt 6 luna as example in docs (#4497) 2026-11-05 08:20:12 +07:00
Nariman Jelveh fc4af97885 fix(gui): the custom domain empty state holds its add button
With no domains, the only way on was the add button in the footer, which
the full-page popup pins to the bottom edge, far from the empty state the
eye lands on. The button now sits in the empty state under its hint, and
the footer keeps only Cancel until there's a domain to list.
2026-11-04 17:08:06 -07:00
Nariman Jelveh 7cec710014 feat(gui): redesign the custom domain dialog, full page in its popup (#4496)
* feat(gui): redesign the custom domain dialog, full page in its popup

- Domains list as cards with a status badge, trash icon, and inline
  Cancel/Confirm for remove and move
- DNS records as cards with copy buttons that confirm with a check
- Status card on the records step; a green check once the domain serves
  the site, after which the dialog closes
- Spinners on the button that's working, skeleton while loading, empty
  and unavailable states
- Pasted URLs are trimmed to their hostname before adding
- In the attach-custom-domain popup the dialog fills the page, with the
  actions pinned to the bottom

* fix(gui): Escape backs out of removing the chosen custom domain

Escape already put back a list row's pending confirmation. On the records
step, it closed the dialog while Remove was asking to confirm; it now puts
the button back first, like the rows.

* fix(gui): each custom domain step opens scrolled to the top

Switching steps reset the scroll of the dialog's body, which only scrolls
when the dialog fills a popup. The floating modal scrolls itself, so a
step opened from far down the list started scrolled past its status card.

* fix(gui): keyboard focus stays in the custom domain dialog

Focus on a control that got hidden or disabled fell to the page behind the
dialog, so Tab started over from there. Switching steps now focuses the
dialog, backing out of a row's confirmation returns to the button that
asked, a request gives focus back to the button that started it, and
removing a row moves focus to the next one.

* fix(gui): a removed custom domain row takes no input as it fades

Its buttons came back on once the delete finished, so a second click in
the fade sent the delete again, and Cancel brought back Use for a domain
that no longer exists. The row is inert until it's gone.

* fix(gui): DNS record copy buttons work without the clipboard API

On an origin with no navigator.clipboard (a Puter served over plain http)
or when the browser refuses the write, the copy buttons did nothing. They
now select the value and try the legacy copy, like the dashboard's form
dialog, and leave it selected to copy by hand if that fails too.

* fix(gui): custom domain rows keep the icon beside the name on phones

The name was sized to the row less 46px for the icon and gap, but the icon
is 36px wide with its border, so the row overflowed by 2px and every name
dropped below its icon. Only the actions wrap now; the name takes the rest
of the first line.

* fix(gui): hostnames in the custom domain dialog wrap whole

Every bold hostname broke at any letter, so one that would fit on the
next line was split mid-word at the end of this one ("puter.lo / calhost").
It now moves down whole and breaks only when it's wider than the line.

* fix(gui): readable Remove and Confirm buttons in dark mode

The red fill under their white labels used the danger text color, which
dark mode lightens to #f87171: 2.8:1 against white. Fills now use their
own token that stays #dc2626 (4.8:1); light mode is unchanged.

* fix(gui): the switch-to-www link shows when it's working

The link that switches a root domain to www had no disabled or loading
style, so while its request ran it still looked clickable and showed
nothing, while every other control dimmed. It now dims like them when
waiting and shows a spinner when it's the one working.
2026-11-04 16:45:56 -07:00
Daniel Salazar 48cd4ea355 fix: oidc toggle (#4494) 2026-11-04 13:13:15 -07:00
Neal ShahandClaude Opus 5.5 a110be0340 feat(gui): remove a custom domain from the attach dialog (#4493)
Each domain in the list gets a Remove button, confirmed in place like Move,
and the records pane can remove the domain just added. A removed domain is
never handed back as the dialog's answer.

Also fixes two strings that went stale: the claim prompt (an active domain
can be claimed too now) and the redirect note (root domains get two A
records).

PUT-2338

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 13:23:04 -05:00
Juan Fernando Castro 93798cba93 fix(puter-js): saved push handlers stay while any tab uses them, and the worker can skip them (#4481)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment

* test(puter-js): keep the push boot test comment to one line

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.

* fix(static): push-sw.js refreshes at the CDN within five minutes

* feat(puter-js): the push worker can skip saved handlers

* fix(puter-js): off() keeps a saved push handler another tab still uses

* docs(notifications): self-hosting the push worker

* docs(notifications): name the pinned push worker version as a placeholder

* fix(puter-js): pages with push handlers can enter the back/forward cache

* test(puter-js): a restored page saves its push handler again

* docs(notifications): pin the push worker only with a pinned puter.js
2026-11-04 10:15:36 -05:00
Juan Fernando Castro 848e336455 fix(puter-js): page loads check push status instead of registering again (#4480)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment

* test(puter-js): keep the push boot test comment to one line

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.

* fix(puter-js): a page load checks push status instead of registering again

* fix(puter-js): subscriptions made with an earlier Puter key are replaced

* fix(puter-js): a replaced browser subscription is picked up on the next visit

* fix(puter-js): a page load that only checks push status still looks for a newer worker

* fix(puter-js): the worker subscribes again only where a Puter page used it

* docs(puter-js): soften the note on subscriptions Puter made earlier

* fix(puter-js): a status answer on load leaves a subscription another tab just turned on
2026-11-04 10:15:25 -05:00
Juan Fernando Castro a9a8df6409 feat(events): fetch can start at the newest notification, and the feed uses it (#4479)
* feat(events): fetch can start from the newest event

GET /events/fetch takes from=end: with no cursor it answers an empty page
whose cursor sits at the newest row of the slice, so later reads return
only rows created or replaced after it. puter.events.fetch() passes
`from` through. Adds a (user_id, audience, app_uid, id) index on
notification so the newest row of a slice is one seek.

* fix(puter-js): onPush starts its feed from the newest notification

A notifications listener started without catch-up asks for from: 'end'
and keeps the cursor it gets back instead of reading every stored
notification to mark them as seen. A server that ignores `from` answers
with the oldest page, and the listener reads through it as before.

* fix(events): read the newest notification id from the primary

* fix(events): mark the end cursor at the slice's latest replacement

The end cursor's replacement mark is the newest (replaced_at, id) in the
slice rather than the serving node's clock, since replaced_at is stamped
by whichever node wrote it.

* docs(events): describe the end cursor's replacement mark
2026-11-04 10:15:14 -05:00
Juan Fernando Castro 0a8b37ae56 fix(notifications): website apps are named by their hostname (#4468)
An app created for an origin keeps its uid as name and title, so its
notifications showed that uid as the sender. appMessageSender now names
such an app by the hostname of its index_url, and keeps the stored name
when index_url doesn't parse. The check AppDriver used for these rows
moves to util/appView so both share it.

The Settings app list shows a generic label for an app whose title is
still its uid.
2026-11-04 10:15:03 -05:00
Daniel Salazar 23782dec12 feat(events): inline handlers in the user's own scope for onSchedule and onPersistent (#4487)
* feat(events): schema for user-scope handlers

Add event_user_handlers and event_subscriptions.handler_scope in all three
dialects. event_handlers is unchanged.

* feat(events): scope the handler and durable subscription stores

Handler store methods take a scope (app, or a user's scope of an app) and read
the matching table. App-scope behavior, SQL and cache keys are unchanged; user
scopes get their own cache keys, a 20-handler cap, and the inline @<sha256>
name. Adds user-scope listing, counting, deletion and unbound-handler reads.
Durable subscriptions store and return handlerScope, and listByHandler
filters by scope and holder.

* fix(events): hold the user-scope handler cap under concurrency and drop all scope caches on app delete

A create that lands past the cap removes itself and is refused. deleteForApp
drops each holder's cached set in batches, here and on peers. Batch scans over
user scopes and unbound handlers take limit and cursor.

* feat(events): set a CPU limit on events worker deploys

* test(events): cover redeploy and resend of an evicted events script

* test(events): name the deploy-on-miss test for what it checks

* fix(events): keep a deploy-on-miss call inside the delivery lease

* feat(workers): cap rented units per plan and refuse only new ones

* feat(events): publish handlers into a user scope, with inline create

* test(events): cover user-scope publish, inline create and listing

* test(workers): name the credentials gate without a vendor

* feat(events): run user-scope handlers as the holder

A user scope deploys one script per handler under the holder's own
identity, throttled per scope, with an eager deploy after publish that
skips an upload another holder already made. Delivery and schedule runs
address the holder's handler, carry the holder on the invoke headers,
batch calls, the miss handler and the rehydrate route, and stay owed
while the flag is off. Suspensions of a user scope notify the holder,
teardown (app delete, consent revoke, account delete) drops the scope,
and a job deletes unbound handlers after a day.

* test(events): cover user-scope deploy, address, notices, teardown and the sweep

Runs an inline user-scope handler through the local worker runtime as its
holder, and covers the holder on the wire, the flag-off path, revoke and
account-delete teardown, and the unbound-handler sweep.

* test(events): a lagging replica read does not suspend a user-scope schedule run

* feat(puter-js): inline user-scope handlers and scope option for handlers and workers

* docs(events): document inline and user-scope handlers, with playground examples

Cover inline handlers on onSchedule and onPersistent, handlerScope, the
scope option on handlers and workers, the user-scope limits and rented
units, and the new error codes. Add six playground examples and pin the
new limit constants in limits.test.ts.

* feat(schedules): carry the handler scope on the schedule view

* docs(events): describe user-handler cleanup, schedule handlerScope and suspension notices

* fix(events): gate the rented-unit cap on user handlers and let an account session destroy its own user scope

* fix(events): keep user-scope rows off app handlers across mixed versions and bound their deploys and reads

Cached durable rows carry an explicit scope marker; an unmarked entry with a
handler is resolved from the table and rewritten. User-scope rows are parked
before credit checks, billing, client delivery and suspension while the flag
is off. Delivery-time handler checks and addressing share the replica cache
entry. Concurrent eager deploys of one user-scope script upload once.

* fix(events): keep a user scope's age across handler churn for rent

event_user_handlers.scope_since records when the scope first held a handler.
A new row takes the scope's existing minimum, so replacing code does not make
the scope look newer to the monthly sweep. The count uses it instead of the
oldest remaining handler's created_at.

* fix(events): drop cached entries no row backs; document account-session scope destroy

* test(jobs): list the user-handler sweep among the system jobs

* fix(events): let the user-handlers migration apply on its own

* fix(events): fail a cached-row read whose scope lookup fails instead of returning app-scope rows

* fix(events): touch a reused user-scope handler before binding so the sweep cannot take it

* fix(events): bound the deploy-on-miss wait by the call's deadline
2026-11-03 23:26:04 -07:00
Daniel Salazar 6a6cf78c03 fix(drivers): stream results with pipe so responses stay under the close-listener limit (#4485)
stream.pipeline adds seven close listeners to the response; with the ones
metering, compression and tracing add, every streamed /drivers/call passed
Node's ten-listener limit and logged MaxListenersExceededWarning. pipe plus
explicit error and close handling adds two, and keeps the same behavior: a
source failing mid-body aborts the response and logs once, and a client
hanging up stops the source.
2026-11-03 23:25:55 -07:00
Nariman Jelveh 8264837d9f feat(gui): type a path into the dashboard Files path bar (#4486)
* feat(gui): type a path into the dashboard Files path bar

Clicking the path bar anywhere but on a crumb, or typing /, swaps the
breadcrumbs for a text field. Enter opens the folder typed (or a file's
folder with the file selected); ~ is home and relative paths start from
the folder on screen. Escape or clicking away puts the crumbs back, and
a path to nothing shows an alert and keeps the field open.

* fix(gui): closing the path alert with Escape gives the path field back

UIAlert only settles when a button is clicked, so an alert closed with
Escape left the path field waiting forever: it no longer closed on blur
and never got the focus back. Wait for the alert window to close
instead.

* fix(gui): look a typed path up once, however often Enter is pressed

A second Enter (or a held key) during the lookup started another one,
and for a path to nothing stacked a second alert on the first.

* fix(gui): leave Enter and Escape to an IME composing in the path field

Picking a candidate with Enter while typing a name through an input
method went to the typed path, and Escape to cancel the composition
closed the field.

* fix(gui): arrow keys go on from rows the Files tab selects for you

Selecting a typed file, or the items just uploaded or pasted, left the
keyboard's starting row on whatever was selected before, so Down jumped
next to that row instead of the new selection.

* fix(gui): opening the path field no longer moves a narrow Files header

On phones and in narrow lists the field was shorter than the crumb row
it replaced and dropped its divider, so the buttons and the listing
jumped up as it opened. It now takes a crumb's height and keeps the
divider.

* fix(gui): space the path field from the Files toolbar buttons

Its border ran up against the sort button, a few pixels from the icon,
where the toolbar's other items keep 10px apart.

* fix(gui): phone keyboards leave a typed path as typed

iOS autocorrect rewrote folder names in the path field as they were
typed. Turn it off, as the GUI's other name fields do, and label the
keyboard's Enter key Go.

* fix(gui): a typed / or .. reaches the root folder

The root can't be stat'ed, so the path field reported "/" as not found,
even though Up goes there. Like Shared, it is now opened without a
lookup.

* fix(gui): Escape calls off a path lookup still running

Escape closed the path field, but the lookup behind it carried on: the
tab still went to the path once it answered, or alerted that it wasn't
found, and a path typed into the reopened field was ignored until then.
2026-11-03 17:52:23 -07:00
Daniel Salazar eb92421a59 fix: disallow puter emails from making new puter accounts (#4482) 2026-11-03 09:17:52 -07:00
Daniel Salazar e0af3361ed ci: add manual dev notify workflow (#4477)
* ci: add manual dev notify workflow

* ci: allow choosing puter ref

* ci: restrict dev notify to maintainers, scope dispatch token to an environment
2026-11-03 09:13:04 -07:00
Juan Fernando Castro 9bd7ba7079 fix(puter-js): push taps open the right page, once (#4472)
* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment
2026-11-03 10:06:16 -05:00
Juan Fernando Castro 8f3d4cc40f fix(puter-js): sign-out ends Puter's push subscription in this browser (#4471)
* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.
2026-11-03 10:06:15 -05:00
Juan Fernando Castro 96d91cd74e fix(puter-js): push syncs skip sign-in and follow disable() across tabs (#4470)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* test(puter-js): keep the push boot test comment to one line
2026-11-03 10:06:15 -05:00
sabrimete cdb3110857 fix(ui): honor the default font in showFontPicker on the desktop (#4340)
* fix(ui): honor the default font in showFontPicker on the desktop

String arguments are normalized to { defaultFont } in the IPC handler,
matching showColorPicker, instead of throwing on property assignment.
UIWindowFontPicker reads defaultValue/defaultFont/default case-
insensitively, and the SDK fallback and docs accept the same aliases.

* fix(ui): document the showFontPicker cancel result and alias order

The JSDoc gains overloads for the string and options call forms and
states that closing the picker resolves false in an app and null on a
website; the docs page and examples match. e2e coverage adds a
scroll-into-view check for the preselected font and a close-without-
selecting case.
2026-11-03 00:07:05 -07:00
Daniel Salazar d934a47c5e feat: jobs and schedules (#4467)
* feat(db): add nowMsExpr, the database clock in epoch milliseconds, and share the retry backoff

* feat(config): type the jobs config as an augmentable IJobsConfig and resolve its namespace in one place

* feat(jobs): add JobScheduler for fleet- and region-wide system jobs

* refactor(jobs): run core's periodic sweeps as system jobs, and publish the generation bumps of reaped subscriptions

* feat(events): seams for schedules run by an extension, with events_disabled where none is installed

* feat(notifications): add notification.send_key, the deferred mailbox write and notify({ sendAt }) through the schedules contract

* feat(puter.js): schedules, notify({ sendAt }) and notifications.cancel(), and treat a 503 events_disabled as final

* docs(events): document schedules, sendAt and cancel(), with recipes

* fix(events): stop resuming a holder's later pages once the credit sweep's signal aborts, and keep its checkpoint

* fix(events): pre-release fixes for jobs, schedule seams and delayed sends

- The delayed-send write is a field on the notifications driver, not a
  method, so /drivers/call can't reach it.
- A delayed send checks for its own earlier write by uid, so it takes no
  gap lock that every other notification insert waits on.
- The notification send_key index builds in place without locking writes.
- Reaps re-check expiry and suspension on the primary, so a row resumed
  since the replica read is kept.
- Revocation settles skip schedule rows they can't settle.
- An empty handler set read from a replica is re-read on the primary.
- One ended notice per settle, not one per page.
- Sweeps that run once per fleet take more batches per run.
- The global usage rate check keeps its 14-minute minimum window.
- shutdown() waits for prepare hooks a drain already started.

* fix(puter.js): quota refusals are final, and schedule error codes match the server

Schedule, subscription and handler quota 429s no longer go through the
generic retry. ScheduleErrorCode lists what the schedule methods throw,
and the notify({ sendAt }) overload has type tests.

* docs(events): schedule and delayed-send accuracy and style pass

Spread and send-time wording match the runner, the per-user runs-at-once
and rate-limited stored notification limits are published, and the new
pages follow the docs writing rules.

* docs(events): add the Change Data at a Set Time recipe

A one-time schedule that changes data when it runs, cancelling and moving
it, and a repeat-a-few-times variant with maxRuns.

* fix(events): delayed sends recheck the app's access when they fire, and a handler set missing the run's handler is re-read on the primary

Also stops job polling at drain start and marks replayable notification sends.

* feat(events): batch handler invocations to the events dispatcher

Invocations that come due within a few milliseconds go out as one
/invoke-batch request (up to events.invokeBatchSize, default 60). The
dispatcher runs a few at a time and streams a started line and an answer line
per call, so a slow handler delays no other call's answer. A call's timeout
runs from its started line, so time queued behind other calls is never the
handler's; queue time comes out of the connection-wait share instead, and a
call still queued past it is never started and fails as Puter's.

A lone call still uses /invoke, and a dispatcher without the batch route gets
single calls for five minutes before the batch route is tried again.
2026-11-02 23:48:35 -07:00
404oopsandClaude Fable 5.1 0128e8779d fix(ai): route per-breakpoint cache lifetimes through the Responses API
Chat Completions rejects `prompt_cache_breakpoint.ttl` whatever its
value, so a request whose content parts name a breakpoint lifetime is
handed to the Responses provider like a request-level `ttl: '12h'`. The
`/openai/v1/responses` translator keeps `prompt_cache_breakpoint` on
text, image and file parts instead of dropping it with the other unknown
fields.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-11-03 00:58:29 +01:00
404oopsandClaude Fable 5.1 89f070f7d0 fix(ai): meter OpenAI cache writes by lifetime
GPT-5.6 and later write prompt-cache entries with a 30m or 12h lifetime
and bill them at 1.25x and 2.5x input; every write was metered at the
30m rate. The OpenAI calculators now split `cache_write_tokens_details`
into `cache_write_tokens` (30m) and `cache_write_12h_tokens` (12h), the
GPT-5.6+ catalog carries the 12h rate, and `usageDetails` reports
`cacheWrite30mTokens` / `cacheWrite12hTokens` instead of labelling the
writes as 5m entries.

Callers pick the lifetime with `prompt_cache_options` ({ mode, ttl }) on
`puter.ai.chat`, `/openai/v1/responses` and `/openai/v1/chat/completions`.
The Chat Completions provider hands a 12h request to the Responses
provider, the only API that writes 12h entries. `prompt_cache_retention`
is forwarded on Chat Completions too, and the hyphenated `in-memory`
spelling maps to the `in_memory` value the upstreams accept.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-11-03 00:52:33 +01:00
Daniel Salazar 58456ae87c fix: signup gate order (#4474) 2026-11-02 13:27:01 -07:00
abb70c14a6 docs: complex fs recipe (PUT-1978) (#4093)
* docs: complex fs recipe (PUT-1978)

A drop folder that reacts to files landing in it, using `puter.events` with an
`fs:` subject instead of polling `readdir`.

One use case per the recipe guidance: watch, ignore your own writes, recover
from a gap, stop watching. The gap section works out what is already done from
the folder itself rather than a counter, so a dropped delivery costs a listing
and never a file.

* docs: rework fs recipe into a CSV importer (PUT-1978)

Renames the recipe to "Import Files From a Folder" and reframes it around
importing CSV files with puter.fs and onLocal(). Watches the app's own
inbox folder, imports files moved in from another folder, and catches up
on open and on gaps. Drops the event.self check, which skipped every file
the user added.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* refine 2

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-03 03:05:34 +07:00
Daniel Salazar 18fd655645 fix: sec improvements (#4469) 2026-11-02 12:11:38 -07:00
dependabot[bot] 480fa7b85b chore(deps): bump fast-uri from 3.1.8 to 3.1.9 (#4462)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.8 to 3.1.9.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.8...v3.1.9)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 00:05:11 -07:00
dependabot[bot] 47b54aacb4 chore(deps): bump postal-mime from 3.0.1 to 4.2.0 (#4461)
Bumps [postal-mime](https://github.com/postalsys/postal-mime) from 3.0.1 to 4.2.0.
- [Release notes](https://github.com/postalsys/postal-mime/releases)
- [Changelog](https://github.com/postalsys/postal-mime/blob/master/CHANGELOG.md)
- [Commits](https://github.com/postalsys/postal-mime/compare/v3.0.1...v4.2.0)

---
updated-dependencies:
- dependency-name: postal-mime
  dependency-version: 4.2.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 00:05:00 -07:00
dependabot[bot] dd3c0dd485 chore(deps): bump qs from 6.16.0 to 6.17.1 (#4460)
Bumps [qs](https://github.com/ljharb/qs) from 6.16.0 to 6.17.1.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.16.0...v6.17.1)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.17.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 00:04:37 -07:00
dependabot[bot] e0689986ed chore(deps): bump systeminformation from 5.33.1 to 5.33.19 (#4425)
Bumps [systeminformation](https://github.com/sebhildebrandt/systeminformation) from 5.33.1 to 5.33.19.
- [Release notes](https://github.com/sebhildebrandt/systeminformation/releases)
- [Changelog](https://github.com/sebhildebrandt/systeminformation/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sebhildebrandt/systeminformation/compare/v5.33.1...v5.33.19)

---
updated-dependencies:
- dependency-name: systeminformation
  dependency-version: 5.33.19
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:55:47 -07:00
dependabot[bot] 0f54588b31 chore(deps): bump qs from 6.16.0 to 6.17.1 in /src/puter-js (#4427)
Bumps [qs](https://github.com/ljharb/qs) from 6.16.0 to 6.17.1.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.16.0...v6.17.1)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.17.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:55:21 -07:00
dependabot[bot] 0cb9f7707d chore(deps): bump follow-redirects in /src/puter-js (#4424)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.16.1 to 1.16.2.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.16.1...v1.16.2)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:55:00 -07:00
dependabot[bot] 622b782f4e chore(deps): bump follow-redirects from 1.16.1 to 1.16.2 (#4426)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.16.1 to 1.16.2.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.16.1...v1.16.2)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:54:48 -07:00
Daniel Salazar efc0ab7126 docs: refresh KV sharing, Events and Notifications docs; fix router CORS and KV size error (#4445)
* docs: refresh KV sharing, Events and Notifications docs and recipes

Bring the KV, Events and Notifications reference pages and the recipes that
use them in line with current behavior, and tighten them.

- KV: share levels, re-share and recipient rules, handle scoping, list
  limits, the 399 KB value limit, and full error tables per method.
- Events: kv:<handle> subjects, what each way of losing access does to a
  subscription, fetch() notification types, handler ownership, and accurate
  anchor-deletion and deleted-app behavior.
- Notifications: where a push goes, desktop and website behavior, sensitive
  taps, share-notify limits, and enable()/status() errors.
- Recipes: fix broken code paths (undefined helpers, single-page share
  listings, read-modify-write races, oversized batch writes, missing value
  fallbacks), use events.update(), onlyIfAbsent and batch share recipients
  where they fit, and cut repetition.
- New recipes: Add a Notification Settings Switch, Clean Up Background Work.
- Rename push-when-closed to reach-a-user-when-your-app-is-closed, drop
  playground examples that need an app the reader owns, and redirect both.

* fix(workers): send CORS on router 404s and readable 500 bodies

Unmatched paths and methods now carry Access-Control-Allow-Origin like
every other router response, so a browser sees the 404 instead of a CORS
failure. A handler response with immutable headers is copied rather than
turned into a 500, and a thrown puter.js error object is sent as JSON.

* fix(kv): one error code for oversized values, and SDK types that match the store

The store now rejects a value over 399 KB with value_too_large, as the
SDK's own check and a growing path write already did. get() declares its
batch form, and the JSDoc gives the 399 KB limit and says watch shares
can list keys.

* docs(notifications): notifications sent within 250 ms reach live listeners as one

* test(events): give the per-handle subscription cap test room on a loaded runner

It fills the whole cap of 1,000 session subscriptions one add at a time,
which went past vitest's 5 s default on a busy CI shard.
2026-11-01 20:27:15 -07:00
Daniel Salazar ef99893fc8 fix(notifications): a browser removed in Puter stays off, and status() checks with Puter on websites (#4458) 2026-11-01 19:50:51 -07:00
Daniel Salazar f8dc10268e test(kv): give the batch-deadline retire test room on a loaded runner (#4444)
"carries on from its record when one run runs out of time" seeds and
retires a full batch of share handles. It takes under a second alone but
went past vitest's 5 s default on a busy CI shard. The timed-out body kept
running with Date.now moved ahead, and "runs once when two runners race
for it" then failed with it.
2026-10-31 14:52:26 -07:00
Daniel Salazar 5d34e803ff fix(events): delivery path fixes for queued deliveries, handler failures and per-invocation database load (#4443)
* fix(events): queue every row's single delivery before any handler runs

A dispatch queued each matched single row's event and then waited on that
row's hand-over, which can be a whole handler run, before queueing the next
row's. An event matching several rows queued the later ones only after the
earlier handlers finished, and lost them if the process stopped in between.
A batched filesystem write did the same across its entries.

The event is now queued in line and the hand-over runs detached; anything it
doesn't finish is retried by the pending sweep. A hand-over that throws no
longer raises the 'could not be queued and is lost' alarm, since the event
is still queued.

* perf(events): address an app's events worker from a cached handler set

Every handler invocation read the app's handler set (names and hashes)
from the primary database to work out which script to call. The delivery
path now reads it through a Redis cache. Publishing, updating or removing a
handler, and deleting the app's handlers, drop the entry on this node and
every peer, so only a read that raced one of those writes can address the
superseded script, for at most 30 seconds. An empty set isn't cached.
Deploys and the events-worker listing still read the primary.

* perf(events): skip leased queues without a database read, and sweep in parallel

The pending sweep read each queue's subscription row from the primary
database before finding out whether anything in the queue could be claimed.
A queue whose head is in flight or waiting out a retry now goes to the back
of the line on a Redis check alone; its row is read once the lease lapses.
The backstop re-check of a queued row's access waits for that lapse too, and
nothing is handed out in the meantime.

The pass also worked through its subscriptions one at a time, so one slow
handler run held up every queue behind it. It now works on up to ten at
once.

* fix(events): don't count a wait for a dispatcher connection as a handler timeout

Invocations share one pool of connections to the events dispatcher per
process, and the invoke timeout started when a call was made, not when it
got a connection. With the pool busy, a call could spend its timeout
waiting, be reported as a handler that ran past it, and count toward
suspending a subscription whose handler never ran.

A call now waits for a free connection before its timeout starts. A call
that waits more than half the timeout for one fails as Puter's
(unavailable): retried on the usual schedule and not counted, as the
documented failure rules already say. Wait and run together stay inside the
worker lease.

* fix(events): end a background row whose consent a delivery finds withdrawn

Withdrawing an app's events:background consent is settled by a listener on
the node that handled the revoke. If that settle never ran or failed, the
row stayed in service: every delivery found the consent gone, minted no
token and was deferred, so the backlog was retried forever and the row was
never suspended or reported.

When the delivery path's consent check answers no, it now runs the same
revocation settle (permission_revoked, backlog purged, holder told, worker
session revoked), one at a time per holder and app. A consent check that
throws still only defers the delivery.

* perf(auth): record no session or user activity for events handler calls

Every API call an events handler made authenticated through its worker
session and touched the session and the user, so background handler runs
wrote session and user activity rows on the primary database (each at most
once a minute), and replaced the session's last IP and user agent with the
worker runtime's.

Calls made with an events handler token no longer touch either. A handler
run isn't the user being active, and the handler session expires only when
it is revoked, so nothing reads its activity to keep it alive. The handler
session's last activity in the session list now reads as when it was
created.

* fix(events): settle withdrawn consent only on a fresh answer

The delivery path's consent check reads through the permission scan cache
and the cached user-to-app grants, which can trail a grant, for example on
a node that cached the app's grants before the consent was given. A stale
no then ended the subscription for good (permission_revoked, backlog
purged, holder told, worker session revoked), where before it only
deferred the delivery until the caches caught up.

Before settling, the no is now asked again with a new `primary` scan
option, which skips the scan cache and reads user-to-app grants from the
primary database. Only a fresh no settles. A fresh yes, or a check that
throws, leaves the delivery deferred.

* test(events): assert no forwarded delivery, not no traffic at all

The region's own connect announces presence to its peers on the forward
queue's next flush. Now that a dispatch returns before the hand-over, the
wait for the socket send can outlast that flush, and the test saw the
presence announce as forwarding.

* fix(events): budget the settles deliveries start for withdrawn consent

A settle a delivery starts when it finds background consent withdrawn ends
rows for good. If the permission check itself answered no wrongly, every
row a delivery reached would be suspended, its backlog purged and its
holder told, with nothing to bring it back.

Each process now settles at most 20 such pairs a minute. Past that, the
rest stay deferred, as a cached no always left them, and a warning alarm
says so. A spent budget skips the fresh read too. A pair a fresh read just
found consented isn't read again for a minute, so a stale cached no costs
one primary read rather than one per delivery. Each settle is logged.
2026-10-31 14:52:06 -07:00
Filip Kujundžić a40e4d984a Merge pull request #4339 from sabrimete/fix/alibaba-catalog-puter-ids 2026-10-31 21:09:05 +01:00
Daniel Salazar dd144b6b64 feat(notifications): notify through KV shares, group replacement, fewer push prompts, and events/worker fixes (#4432)
* feat(perms): add a pushNotifications resource to puter.perms.request

notifications:push gets a resource like backgroundEvents, so it can be
batched with other access behind one prompt. requestPermission() and
hasPermission() ask through it.

* fix(puter-js): read puter.gui_features once at load

The desktop's feature list was read from location.search on each call, so
an app whose router rewrote the URL lost it and enablePush() and the other
desktop dialogs answered as if the desktop lacked them.

* feat(gui): ask for notifications:push when an app calls enablePush without it

enable() and enablePush() inside the desktop answered unavailable for an
app without the push permission. They now show the permission dialog,
whose Allow also starts the browser registration, and answer with the
browser's push state once it settles, or off when denied. A browser that
blocks notifications or has no push service still answers without a
dialog.

* fix(kv-share): charge mint and grant budgets only for share calls that change something

A share call now spends the per-call mint budget and the per-recipient grant
windows after it has looked up which recipients already hold the share. A call
whose recipients all hold the share at the requested level and label spends
neither. Recipients without the share, including unknown usernames and refused
recipients, still spend a grant unit each. Every named recipient spends from a
new 1,200/min lookup window before any lookup runs.

* perf(events): skip the subject split for rows not made through a share handle

kvHandleFromSubject checks the stored subject's kv:kvh- prefix before
splitting it, so own-namespace and fs rows return null without parsing.

* feat(notifications): resolve enable() on a website before the confirmation push

enable() waited up to 30 seconds for the confirmation push, so a prompt
the same click started next had lost its activation. On a website it now
resolves once the browser answered and the server has the subscription,
records the opt-in, and finishes the confirmation in the background. A
failed confirmation goes to onPush's onError and the next load's resync
tries again. disable() and sign-out stop a confirmation still waiting.
The GUI keeps waiting, since it reads the registered device right after.

* fix(notifications): show a page's own notifications only while push is on

A hidden page with an onPush handler kept showing notifications after
disable(), and its receipts kept claiming showsOwn, which held pushes back
from the user's other devices. Both now need the user's opt-in, cached in
the page, refreshed on enable, disable, sign-out and each sync, and re-read
when another tab of the site changes it.

* fix(events): leave subscriptions suspended for good out of the durable quota

countForHolder no longer counts rows suspended with a reason that never
resumes (permission_revoked), in both the pre-insert check and the recount
after insert. Rows suspended for a resumable reason still count. The store now
owns the list of terminal reasons, and isResumable reads it.

listActiveForHolder with includeSuspended returns only rows suspended for a
resumable reason, so its quota-sized read still covers every row a revocation
has to settle. Each holder keeps at most 500 terminal rows; a subscribe past
that deletes the oldest ones.

* feat(notifications): add status() to read whether pushes reach this browser

status() resolves 'on', 'off', 'blocked' or 'unavailable' without asking
anything. On a website it reads the notification permission, this
browser's subscription and the user's opt-in. Inside the desktop a new
pushStatus IPC, advertised as push-status, answers the browser's push
state, 'on' only while the app holds notifications:push.

* feat(notifications): disable() inside the desktop revokes the app's push permission

disable() did nothing inside the desktop. A new disablePush IPC,
advertised as disable-push, revokes the calling app's notifications:push
grant through the same route Settings uses, so its notifications stop
going out as pushes and a later enable() shows the permission dialog
again.

* feat(notifications): allow @ in a notification group, up to 128 characters

A group is an opaque display tag, hashed for the Web Push topic and used
as the notification tag, so it can carry identifiers such as an email
address.

* docs(notifications): say when a page with an older Puter.js holds a push back

* fix(notifications): let disable() win over an enable() still in flight

An enable() whose subscription reached the server after a disable() went
on to record the opt-in, so the next page load turned pushes back on. It
now rejects with push_verification_canceled and records nothing.

* docs(notifications): list push_verification_canceled and reword sentences that led with code

* fix(events): carry KV values through a share handle only at read or write

The delivery re-check now reads the share's level off the live handle along
with whether it still holds. A row on a watch share loses its includeValue for
that delivery, so its events name the key without the value; the owner's own
rows on the handle keep theirs. Decided per delivery, so raising or lowering a
share applies to subscriptions already made, and subscribing with includeValue
at watch is still accepted.

The live-updates recipe now shares at read, which its events need to carry
the score.

* fix(events): retry worker runs Puter could not complete without counting them

Invocations that never reached the handler (transport and dispatcher errors,
unmarked answers, failed or throttled deploys, no configured runtime, a bad
invoke key or missing token) now report an 'unavailable' outcome. So does a
handler that throws after one of its own puter.js calls got a 5xx, 429 or
network error from Puter's API, which the events runtime reports as
'platform-error'. Those runs back off like retriable ones, honor the answer's
retry-after up to the backoff ceiling, and no longer count toward the
consecutive-failure suspension. A run past the invoke timeout still counts.

* test(notifications): cover the pushNotifications resource and status() in the API suites

* test(gui): Allow on a batch that includes notifications:push turns push on

* fix(events): hold a share-handle row's backlog when its share can't be checked

The share-handle delivery check now tells a share that is gone (handle not
live, not the row's holder, identity gone, grant withdrawn) from one it can't
check right now (a lookup that throws, an owner who is suspended). Only the
first settles a queued row as permission_revoked and purges its backlog. The
second delivers nothing, leaves the row in service with its backlog, and sends
it to the back of the sweeper's line, so delivery resumes once the lookup
recovers or the owner is unsuspended. A revocation settle skips such a row and
leaves it to the delivery check. kvSharedRegionAuthorized returns null when
the permission check throws, and that answer is not cached.

* feat(events): update a persistent subscription's context in place

POST /events/subscriptions/update replaces a durable row's context, with the
same validation and 4 KB cap as subscribe, keeping its subId and backlog. The
row's owner region cache is rebuilt from the table and the generation bumped so
every region delivers the new value. It spends the subscribe call budget.

puter.events.update(subId, { context }) wraps it; null clears the context, and
a function handler this client registered for the subscription gets the new
ctx.

* fix(events): serve a fan-out's holders in turns and cap values per delivery

#route now orders an event's candidate rows before any cap applies: the
keyspace owner's rows first, then the other holders taking turns, one row per
holder per round, starting from a random holder each event. The filter
evaluation cap, the delivery cap and the gap markers all count in that order.

Values no longer drop for every delivery once more than 128 rows match or the
filter evaluation stops early. The first 128 deliveries that asked for the
value carry it, in serving order, and later ones carry the key alone.
EVENTS_KV_VALUE_OMIT_MATCHED_SUBSCRIPTIONS becomes
EVENTS_KV_VALUE_DELIVERIES_PER_EVENT. The caps themselves are unchanged.

* fix(email): file inbox mail as a delivery, not the request actor's write

storeInboxMessage and copyIntoInbox now run their filesystem writes with a
platformWrite context flag, and fs event dispatch leaves actingUserId unset
under it. Mail filed into an account's inbox from inside that account's own
request (an app mailing the signed-in user) is delivered with self: false, the
same as external ingress. Handler depth and viaWorker are still carried from the
actor.

* feat(notifications): raise the default daily app message limit to 1000

The per-day default for `app.message` goes from 300 to 1000 per user and
app, matching the daily push cap. The per-minute default stays at 30, and
`notificationAppMessageLimits` still overrides both.

* feat(notifications): look up an unseen notification by group

`notification` gets a nullable `group_key` column, a sha256 hex digest of
the group a row was written with, indexed on (user_id, app_uid, group_key).
`NotificationStore.create` takes a `group`, and `getUnseenByGroup` returns
the newest row an app wrote for a user under that group that is neither
shown nor acknowledged. Rows written before the column exist never match.

`updateValue` takes `onlyUnseen`, which also skips a shown row, and
`refreshCreatedAt`, which dates the row to now.

Migrations: sqlite 0098, mysql_mig_52, postgres_mig_41.

* feat(notifications): replace an app's unseen message in the same group

An `app.message` with a `group` now replaces the app's earlier notification
to the same user with that group while it is neither shown nor
acknowledged. The row keeps its uid and takes the new value and time. With
no group, or once the earlier one was shown or dismissed, it is a new
notification as before.

A replacement spends a per-minute slot but no per-day slot. If the earlier
row is marked between the lookup and the rewrite, the message is written as
a new notification and spends a day slot then.

`app.message` is groupable, so the replacement goes through
`notifyUpdate`. For types kept off the desktop, `notifyUpdate` skips the
desktop wire and dispatches the rewrite again under the same uid, so app
subscribers and receipts see it, and `notif.written` carries `updated`.

The driver's write path takes the recipient id rather than the actor, so
rate limits key on the recipient.

* feat(workers): report the app a worker runs as

workers.create now returns app_uid and app_name for the app its token is bound
to (null for a user-scoped worker), and list/get items carry app_name beside
app_uid from the app rows they already load. The dev center workers table shows
an App column, and the SDK typedefs and docs describe the fields and that a
sandboxed worker's notifications and pushes come from its sandbox app.

* feat(events): refuse handlers with free variables on REST publish

POST /events/handlers/publish and /publishAll now run the SDK's own
scanHandlerSource on each source before publishing and answer 400
events_handler_free_variable, naming the identifier. The backend imports the
module from src/puter-js; the build marks dist/src/puter-js as an ES module
package so the compiled backend can require it. Sources over the per-handler cap
are left to the store's size refusal rather than tokenized.

* docs(events): say Puter-side retries never suspend and stay within backlog caps

* fix(events): redeploy an events worker that answers bad-key

A script deployed before the internal secret rotated keeps the old invoke key,
so every invocation answered bad-key and nothing redeployed it. A bad-key answer
now goes through the same miss handler as a dispatcher miss (the deployer, with
its hourly throttle) and is retried once with the deployed header. A throttled
or failed redeploy stays 'unavailable' and backs off without counting.

* perf(events): check the deploy throttle before reading handler sources

EventsWorkerDeployer now reads the app's handler names and hashes first,
answers no-handlers or stale from those, and checks the hourly deploy throttle
before loading the sources and generating the worker. A throttled retry costs a
hash read and a counter bump instead of a full source read and build, and a
stale request still does not spend the throttle. The generated set is checked
against the script again in case it moved between the two reads.

* fix(workers): move a redeployed worker's row to the app it now runs as

Redeploying an existing worker under a different app minted the new app's
token but left subdomains.app_owner on the old one, so list/get and hot-reload
kept naming and minting for the old app. The redeploy now sets app_owner to the
binding #resolveWorkerAppBinding authorized (null for a user-scoped worker)
through a new SubdomainStore.setAppOwner, since app_owner stays read-only to
update patches.

* fix(workers): revoke the old binding's worker session on a rebind

A redeploy that binds a worker to a different app (or to none) now deletes the
owner's worker sessions under that name other than the one for the new binding,
through the same deleteWorkerSessionsByName path worker deletion uses (which
also announces the revocation and drops access-token grants). SessionStore's
deleteWorkerSessions takes an exceptAppUid filter for it, null meaning the
user-scoped session. The app_owner move and the revocation now both run only
after the new script deployed successfully, so a failed redeploy leaves the
live script, its token and the row on the old binding.

* feat(kv-share): store a notify flag on share handles

kv_share_handles gains a notify column, off by default, so every existing
share reads as not allowing notifications (sqlite 0099, mysql_mig_53,
postgres_mig_42). The store reads it on every row, including the live
handle cache and the holder listing, writes it on mint, and changes it
through update().

* feat(kv-share): let the owner turn notifications on for a share

POST /kv/shares and puter.kv.share() take notify, off by default. Leaving it
out of a re-share keeps the stored value, and turning it on or off counts as a
change, so it spends the mint budget. Turning it on for a holder who now
refuses the owner's shares is refused, like a raise or a new label.

The kv.share.received notice is sent only when a share starts carrying
notify: a new share made with it, or an existing one switched on. Its hourly
bound and the app-less namespace exclusion are unchanged.

The share response, the owner's listing and the holder's listing report
notify.

* feat(kv-share): resolve the other party of a share for a notification

KvShareService.resolveNotifyRecipient(actor, handle) returns the recipient
user id, app uid and handle for a notification one party of a share sends
the other through its handle. The handle must be live with an unsuspended
owner, the caller must be its owner or holder acting from the app whose data
it is, the share must have notify on, the holder's grant must still stand,
and the recipient must not refuse shares from the sender.

An unknown, retired or foreign handle, a caller who is not a party, and a
caller in another app all get subject_does_not_exist. The account itself gets
forbidden, a share without notify kv_share_notify_not_enabled, and a lookup
that fails service_unavailable. Once every check passes it spends one unit
from KV_SHARE_NOTIFY_LIMITS (30/min, 500/day per sender, recipient and app),
peeking both windows first so a refusal spends nothing.

* feat(notifications): hand a replaced notification to listeners again

A replacement now records `replaced_at` (epoch ms) on its row, added to the
same unreleased migrations as `group_key` (the sqlite file is renamed to
0098_notification-groups.sql). Its live event and its fetched copy both
carry `replaced: true` and use that time as `ts`, so the pair versions the
uid identically on both paths.

`onNotification()` dedupes on uid plus that version: an original counts as
version 0 and a replacement as its `ts`. A newer replacement is handed over
again with `replaced: true`, while a copy already handed over, a replay, or
an older state is dropped. Listeners started with `catchUp: false` still
get replacements of what was already there.

* fix(notifications): replace only the newest notification in a group

The group lookup takes the newest row an app wrote under the group and
returns nothing when that row was shown or dismissed, instead of falling
back to an older row that is still unseen.

The docs state the rule as it is: a grouped notification replaces the
newest one in its group unless that was shown or dismissed. Puter.js
marks neither, and the only thing that does is a tap on the push inside
the Puter desktop, which dismisses it.

* feat(notifications): notify the other party of a KV share through its handle

`app.message` takes `to: { handle }`. The driver validates the message,
then `to` must be an object with only a non-empty string `handle`, else
400. `KvShareService.resolveNotifyRecipient` checks the share and spends
the pair limit, and the message is written for the share's other party in
the share's app. The recipient's per-app limits and group replacement
apply to their own rows, and the push is billed to them through
`notif.written`.

A call with `to` answers with `{ uid }` only, so nothing about the
recipient's row reaches the sender. puter.js gets the
`NotificationRecipient` type for `to`.

* docs(notifications): document notifying the other side of a KV share

notify.md covers `to: { handle }`: who can be reached, the errors, the
recipient's limits plus the per-pair share limit, the uid-only return and
that the recipient pays for pushes. push.md and rate-limits-and-quotas.md
point at the share path. The chat notification recipe sends through each
share's handle and keeps the per-member handler only for members who need
to reach each other; the alert recipe says when its handler is the right
fit.

* fix(notifications): reveal nothing to the sender of a notification through a share

A call with `to: { handle }` answers with an empty object, and puter.js
resolves it with no value: a uid would show whether the recipient's last
notification in a group was replaced. `notify()` gets one overload per
form, and `ShareNotificationInput` is exported.

Over the per-pair share limit or the recipient's own limits, such a call
gets the same 429 `too_many_requests` with the same generic message, so the
sender can't tell which limit it hit.

* fix(notifications): serialize grouped app messages per group

Grouped app.message sends take a Redis lock on (recipient, app, group) around the lookup and the replace-or-insert, so concurrent sends to an empty group write one row and spend one day slot. A lock still busy after the wait, or an unreachable Redis, sends unlocked as before. Sends without a group take no lock.

* fix(events): attribute Puter API failures to the invocation that saw them

The events worker runtime counted Puter API failures on the cached per-token client, so a concurrent invocation on the same token could answer platform-error for its own unrelated throw. A cached client now serves one invocation at a time and records failures and retry-after on that invocation; an invocation overlapping another on the same token gets its own client, and idle clients stay cached per token.

* fix(events): drop KV values from a queued backlog once its share is at watch

A queue delivery is stored with its value. Every hand-out of a stored
entry (backlog drain to a socket or another region, the worker, and a
declined delivery passed to the worker) now re-reads the share's level
and removes the value at watch. A share that can't be checked still
holds the backlog.

* fix(events): return notifications replaced behind the fetch cursor

The notification fetch cursor now holds the last row id and the last
(replaced_at, id) it returned. A read from it merges two indexed reads:
rows inserted after the id, and rows up to the id replaced after that
mark. A replacement made while a client was away therefore comes back
on its next catch-up. Adds idx_notification_replaced on (user_id,
audience, app_uid, replaced_at).
2026-10-31 12:21:18 -07:00
Daniel Salazar 4851df716c fix(kv): KV share lifecycle hardening (#4429)
* fix(kv): harden the KV share lifecycle

- Retire a deleted app's live shares before an origin app is registered
  again under the same uid. The retire runs before the app row exists and
  throws on failure, so a failed launch leaves no row and the next one
  retries.
- Record deleted-app retires and uninstall cleanups (retire, then purge) as
  reserved items before they run. A lease keeps one node on a job; a failed
  run raises an alarm and backs off, and a run that hits its time budget or
  is cut short by a restart resumes on the next check. Each job is bounded
  to handles made before it was recorded.
- Uninstalling with deleteData records the job and returns; the retire and
  purge run off the request. The purge checks whether the app was
  authorized again before deleting each page, so data written after that
  is kept.
- Take a per-region lock when minting and re-check for a live handle under
  it, so concurrent shares of one region make one handle.
- Retire the handles an account holds before the account is deleted, so the
  owners' subscriptions through them end instead of staying active.

* fix(events): bound share-handle session rows by the per-connection cap

The count across a holder's connections also counted connections that
had dropped but weren't reaped yet (a crashed node, a deploy, a network
blip the server hasn't noticed), so resubscribes after a reconnect could
be refused with `events_subscription_limit`. Rows made through a share
handle now count toward the 50-per-connection cap like any other, and
with the open-connections cap per account that bounds what one holder
can put in the owner's keyspace.

The handle index stays for ending rows on retire. Rows are added to it
with the row itself and taken out when the row is dropped, and members
whose row is gone are cleared when a retire reads it.

* fix(events): keep a retire's durable settle running if ending sessions fails

Ending a retired handle's session subscriptions reads Redis. A failure
there is now logged and the durable suspension that follows still runs,
so those rows' backlogs are still purged.

* perf(events): ask once per handle when a refresh screens rows, and share handle reads

A socket refresh asks whether a share handle still stands once per
handle, holder and app, rather than once per row: rows through one
handle share its grant. Concurrent cache misses on one handle share a
single read; a forget drops the shared read so later callers start a
fresh one.

* fix(permissions): chunk the exact-match subtree delete

The holder-scoped subtree delete names every matched grant, so a large
subtree could exceed an engine's limit on bound parameters. It now
deletes 500 grants per statement.

* fix(kv): give each holder a share of a namespace's handle budget

One holder could spend a namespace's whole handle budget and starve the
others. Each holder now gets at most 3,750 of the namespace's 15,000
units per 10 seconds, and the owner's own calls through a handle spend
none. The holder's budget is checked before the grant lookups, so
throttled retries cost none; the namespace's is only told to callers the
grant admits. Multi-unit charges use a new counted claim that keeps two
counters per key instead of a sorted-set member per unit.

* fix(kv): refuse handles older than their app and claim cleanup jobs globally

- A share handle created before its app row reads as gone: in the KV
  driver, in the handle cache, on the events subscribe path, and in the
  delegate implicator. An origin app registered again under a deleted
  app's uid can't reach its predecessor's shares, whatever created the row.
  The app-token launch no longer retires inline; it records a background
  retire and never waits on it or fails because of it. The clock-skew rule
  is shared with app sessions.
- Cleanup job records now carry their lease, attempts and retry time, and
  a node claims a job by compare-and-set on its record. A job stays with
  the deployment that recorded or last ran it, and another one adopts it
  only after it has gone unclaimed well past due.
- The job check pages through records from where the last look stopped, so
  waiting records don't hide due ones. After 10 failed runs a job stops
  retrying with a warning alarm, and its record expires after 30 days.
- A mint that waits out the region lock looks for the share again and
  returns it, or goes ahead, instead of failing with too_many_requests.
- The record-failure alarm dedups; an unfinished run is a log line.

* fix(kv): hide shares older than their app from re-shares and listings

After an app is registered again under a deleted app's uid, its
predecessor's live handles stayed visible: sharing the same region again
handed back the old, dead handle, and both sides' listings showed the old
handles and labels. findLive and both listing queries now leave out a live
handle created before its app row, the same rule handle resolution uses,
so a re-share makes a new handle and the listings, totals included, skip
the old ones until their retire lands.

A new share also retires, grants and all, the handles its owner gave the
same holder for the earlier app first, so withdrawing those later can't
take the new grant with it.

* fix(events): keep a loose cap on subscriptions through one share handle

Without a count across connections, a holder could put up to 20,000 rows
in the owner's keyspace (50 per connection, 400 connections), each one
evaluated on every owner write. The cap is back at 1,000 per holder per
handle, far above what connections that dropped without being reaped
leave behind. The index is only read row by row once it's past the cap.

* fix(kv): harden the share cleanup jobs

- Job records are keyed by domain, so a deployment that shares the table
  but not the database or broadcast (staging beside production) never
  runs another's jobs.
- A job whose record can't be written runs unrecorded instead of not at
  all, and a failed check for live handles records the job anyway so it
  retries. Failures that used to be swallowed are logged.
- Recording a job again keeps its attempts and backoff; one that gave up
  gets one more attempt.
- A system retire gives back the grant of a live share it doesn't cover
  inside a withdrawn region, and a share made under a reused uid grants
  again once its row stands, so a concurrent retire can't leave a live
  handle without its grant.

* fix(kv): don't cache a live handle a retire overtook during its app lookup

The live read decided whether it could cache before awaiting the app
lookup, so a retire's forget that landed during that lookup was
overwritten with the handle as live for up to 60 s. It's now checked
after the lookup.

* chore(kv): word the job key comment for any install
2026-10-31 12:12:29 -07:00
Daniel Salazar faa8cd99dd fix(kv): per-holder share of the KV handle budget (#4433)
* fix(permissions): chunk the exact-match subtree delete

The holder-scoped subtree delete names every matched grant, so a large
subtree could exceed an engine's limit on bound parameters. It now
deletes 500 grants per statement.

* fix(kv): give each holder a share of a namespace's handle budget

One holder could spend a namespace's whole handle budget and starve the
others. Each holder now gets at most 3,750 of the namespace's 15,000
units per 10 seconds, and the owner's own calls through a handle spend
none. The holder's budget is checked before the grant lookups, so
throttled retries cost none; the namespace's is only told to callers the
grant admits. Multi-unit charges use a new counted claim that keeps two
counters per key instead of a sorted-set member per unit.
2026-10-31 12:10:39 -07:00
Daniel Salazar 279bb8beb6 fix(events): KV share handle subscription follow-ups (#4434)
* fix(events): bound share-handle session rows by the per-connection cap

The count across a holder's connections also counted connections that
had dropped but weren't reaped yet (a crashed node, a deploy, a network
blip the server hasn't noticed), so resubscribes after a reconnect could
be refused with `events_subscription_limit`. Rows made through a share
handle now count toward the 50-per-connection cap like any other, and
with the open-connections cap per account that bounds what one holder
can put in the owner's keyspace.

The handle index stays for ending rows on retire. Rows are added to it
with the row itself and taken out when the row is dropped, and members
whose row is gone are cleared when a retire reads it.

* fix(events): keep a retire's durable settle running if ending sessions fails

Ending a retired handle's session subscriptions reads Redis. A failure
there is now logged and the durable suspension that follows still runs,
so those rows' backlogs are still purged.

* perf(events): ask once per handle when a refresh screens rows, and share handle reads

A socket refresh asks whether a share handle still stands once per
handle, holder and app, rather than once per row: rows through one
handle share its grant. Concurrent cache misses on one handle share a
single read; a forget drops the shared read so later callers start a
fresh one.

* fix(events): keep a loose cap on subscriptions through one share handle

Without a count across connections, a holder could put up to 20,000 rows
in the owner's keyspace (50 per connection, 400 connections), each one
evaluated on every owner write. The cap is back at 1,000 per holder per
handle, far above what connections that dropped without being reaped
leave behind. The index is only read row by row once it's past the cap.
2026-10-31 12:10:34 -07:00
Daniel Salazar 52aedde3c5 fix(kv): KV share authz and load hardening (#4430)
* fix(permissions): delete user-to-user grants by their exact spelling

MySQL compares the permission column case-insensitively, and LIKE folds
case on MySQL and SQLite, so withdrawing a grant could also remove one
spelled differently, which checks and the flat view treat as distinct.
The single, batch and subtree deletes for one holder now match the
permission byte for byte.

* fix(auth): refuse key-value share grants on /auth/revoke-user-user

Withdrawing a share's grant there left its handle live and listed. The
route now refuses kv-share permissions and points to puter.kv.unshare().

* perf(kv): ask whether a delegate app is authorized once per check

The app-delegate implicator re-read the app's authorization for every
option a refused check offered, one read per ancestor of a deep key.

* fix(puter-js): default a kvShare consent request to read

puter.kv.share() creates read shares by default, so a consent asked for
with the defaults (watch) didn't cover the share made with the defaults.

* fix(kv): treat a private entry as missing for a share holder's writes

Through a handle, a private entry reads as absent, but writes to it were
refused with an error naming it as private, and onlyIfAbsent answered
false. Deleting one through a handle now answers as deleting a missing
key does and leaves the entry in place. Any other write is refused with
kv_handle_access_denied, the same error as a call the share's level
doesn't allow, onlyIfAbsent included. Cross-app access is unchanged.

* fix(kv): cap handle traffic per owner namespace

Every call through a share handle lands on its owner's namespace, and
per-holder limits put no bound on what all holders together could send
there. Handle calls into one namespace now share a budget of 6,000 units
per 10 seconds, counted from the storage reads and writes each call
consumes (at least one), and are refused with too_many_requests past it.
2026-10-31 01:01:53 -07:00
Daniel Salazar 7f832ede43 fix(events): KV share handle subscription hardening (#4428)
* fix(events): keep a share's prefix out of handle subscribe errors and limits

A `kv:<handle>:<key>` pattern is now checked against the 256-character
limit as the holder wrote it, before it's composed onto the shared
prefix. The composed pattern is compiled with room for the longest
prefix a grant can carry, at subscribe and at dispatch, so neither the
limit nor its error depends on the prefix.

A handle subscription's `match` is read off the holder's own subject
instead of the stored composed pattern, so whether it's `null` no
longer depends on how deep or long the prefix is.

* fix(events): cap and end session subscriptions made through a share handle

Session subscriptions made through a share handle are indexed under the
handle in the owner's keyspace. A holder may keep 100 of them through
one handle across all their connections; past that, subscribe fails
with `events_subscription_limit`. The owner's own subscriptions through
the handle don't count.

Retiring a handle now ends its session subscriptions in the same
region and sends `events.ended` with `permission_revoked`, alongside
the existing durable settle. A socket refresh no longer renews a row
whose handle was retired or whose grant is gone: it ends it the same
way, which is how a retire made elsewhere reaches these rows.

* fix(kv): read a forgotten share handle from the primary

After a handle or its owner is forgotten, reads of that handle go to
the primary for the next minute, so a replica that hasn't caught up
can't put the old row back in the cache. A read that a forget lands on
while it's in flight is answered but not cached.

* fix(events): read a socket ref's subId after the last `|`

A KV token carries the subscribed key, which can contain `|`, so
splitting the ref at the second one cut the token short. A subscription
on such a key couldn't be unsubscribed, and a reap left its row behind
in the owner's keyspace. SubIds never contain `|`, so the ref is split
at the last one.
2026-10-31 00:58:42 -07:00
Daniel Salazar 58275ccd52 ci(puterjs): check out the coverage base leg from this repo (#4431)
The base leg checked out the base branch from the PR's head repo. For a
fork PR that is the fork's copy of main, which can predate the current
Node version and fail npm ci.
2026-10-31 00:42:37 -07:00
Daniel Salazar 11114c80d6 fix(auth): fail closed on otp-login token reuse check (#4421)
/login/otp and /login/recovery-code marked the otp-login token used
through a rate-limit check that admits on a store error, so while the
store was unavailable a captured token could complete another sign-in.
The check now fails closed.

Also drops the check-translations scripts and the AGENTS.md step that
runs them; the script they call was removed in #2873.
2026-10-31 00:26:56 -07:00
Daniel Salazar 5d907d18c9 fix(socket): bound acks a receipt ask leaves on sockets that never answer (#4422)
An acked broadcast leaves its callback in socket.io's per-socket ack map on every socket it reaches, and nodes other than the asker free it only when that socket answers or disconnects. Each socket's map is now a BoundedTtlMap capped at 64 entries and 60 s, so a socket that never answers holds a bounded number and a late answer is ignored. BoundedTtlMap drops expired entries on set so its TTL frees memory too.
2026-10-31 00:26:44 -07:00
Daniel Salazar 2d5e037599 fix(ai): bill realtime responses the upstream drops before reporting usage (#4423)
When the upstream realtime connection closes, for any reason, with a
started response that never sent response.done, the relay now records that
response at its credit hold (8k audio in + 4,096 audio out at the session
model's rates, through the AI cost factor) instead of releasing the hold
uncharged. The hold is released once the charge is recorded, and a warning
alarm is still raised.

Client conversation.item.retrieve events and MCP tools in session.update or
response.create are refused with an invalid_event error and a 1008 close.
2026-10-30 23:48:58 -07:00
8d3a1ccc2b feat(notifications): remove a notification from the dashboard history (#4412)
Each notification in the dashboard list gets a remove (✕) control next to
mark-as-read. It calls the new POST /notif/delete, which deletes the user's
own row and sends `notif.removed` to their other tabs so the entry and any
toast for it disappear everywhere.

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 22:52:27 -07:00
Daniel Salazar a1fe14cbd5 fix(ai): bound large AI request bodies per user and to chat (#4420)
Follow-up to #4408.

The 503 returned when a process holds too many large bodies is load
shedding, so it records an info alarm instead of a critical one.

One user can hold at most 128 MiB of large bodies at once; past that the
request gets 429 with Retry-After, so a single account can't take a
process's whole budget, e.g. with slow uploads.

Through /drivers/call only puter-chat-completion takes a body over 50 MiB;
other drivers keep the default limit and get 413. The check reads the
parsed body's interface, since route matching ignores case and trailing
slashes.
2026-10-30 21:16:53 -07:00
Filip KujundžićandDaniel Salazar a195605003 fix(ai): accept 128 MiB Anthropic message requests (#4408)
* fix(ai): accept larger Anthropic message requests

* fix(ai): accept 128 MiB bodies on /drivers/call and OpenAI routes

Extend the 128 MiB JSON limit to /drivers/call (puter.ai.*) and the
OpenAI-compatible routes, alongside the Anthropic ones.

A body over the default 50 MiB is read only after a header or cookie
credential resolves; otherwise the global parser rejects it with 413, so
unauthenticated callers can't make the server buffer and parse 128 MiB.
Large bodies skip the rawBody copy, and a per-process budget caps how many
bytes of them are held at once, returning 503 with Retry-After past it.

The prompt token estimate no longer splits text on whitespace, which took
seconds and allocated a string per word on large prompts. It counts
whitespace runs in a loop, scans at most 1 MiB per prompt and extrapolates
the measured density past that.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 20:52:00 -07:00
Daniel Salazar 1e39936d12 fix: custom domains (#4419) 2026-10-30 20:39:11 -07:00
John CostaandDaniel Salazar 1390a87280 fix(docker): hoist sharp so the backend can load it from dist (#4400)
The backend runs from dist/src/backend, so Node resolves modules from the
root node_modules, never src/backend/node_modules. Since sharp ^0.35.5
(#4081) only the backend asked for that version, so npm nested it under
src/backend while the root copy stayed miniflare's dev-only 0.35.4, which
the image doesn't ship. The 26.10.1 image then crash-loops on start with
"Cannot find module 'sharp'" from the Replicate image provider.

Declaring sharp ^0.35.5 at the root hoists the backend's version, the
same way other backend runtime deps are reachable from dist.

Fixes #4394

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 20:00:09 -07:00
dependabot[bot] 6d47db9358 chore(deps): bump nodemailer from 9.1.1 to 10.1.4 (#4386)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.1.1 to 10.1.4.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.1.1...v10.1.4)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 10.1.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 19:39:17 -07:00
mahilsaily 2e57214bb3 fix: update outdated HOME_URL in /etc/os-release (closes #4329) (#4392) 2026-10-30 19:37:22 -07:00
spacedandDevin AI 7ac2f2e247 feat(ai-chat): let requests opt out of provider fallback (#4343)
* ai-chat: fallback: false fails on the first route instead of rerouting

A chat request whose first provider fails is retried on up to two other
providers that serve the same model, and the caller pays for whichever
one answers. Callers who want one specific route, or who want to handle
a vendor failure themselves, had no way to turn that off.

Passing fallback: false (puter.js option, /drivers/call arg, and the
OpenAI-compat and Anthropic-compat request bodies) makes the first
attempt's failure final: the route is still marked unhealthy, and the
error is classified the same way an exhausted chain would be.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ai-chat: cover fallback: false in the SDK suite

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ai-chat: fallback: false rejects a pinned provider with no route for the model

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: plainer wording for the fallback option

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-30 19:35:45 -07:00
dafiliandDaniel Salazar 0982e696a5 docs(install): explain how to install Docker instead of stopping at a dead end (#4411)
* docs(install): explain how to install Docker instead of a dead end

The installer stopped with 'missing required command: docker' and nothing
else, which left self-hosters on a clean machine with no next step. Since
Puter self-hosting is a Docker Compose stack, that check is also the very
first thing a new user hits.

Print the official install links for macOS/Windows and Linux plus the
'docker compose version' sanity check, and cover the same ground in
doc/self-hosting.md under Requirements and Troubleshooting.

The installer still does not install Docker itself — per the maintainer's
note on #4346, pulling a remote script into a system package manager is
too intrusive. This only makes the failure actionable.

* fix: remove extra sh stuff already merged

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 19:33:01 -07:00
Juan Fernando Castro fea89c91b8 feat(puter-js)!: remove notifications.registerDevice (#4409)
* docs(notifications): registerDevice takes ios or android

* feat(puter-js)!: remove notifications.registerDevice
2026-10-30 19:31:30 -07:00
Rasad Regmi 489a7c0129 fix(install): improve Docker missing error with platform-specific instructions (#4384) 2026-10-30 19:29:52 -07:00
Juan Fernando Castro e5ba469ffc fix(notifications): keep app messages in their app, off the desktop (#4410)
* fix(notifications): keep app messages in their app, off the desktop

An app's notifications now reach only that app: its live feed on
notif:app-user and, through the app's own sockets, receipts and pushes.
The desktop's listing, unread replay, live wire and notif:app-user
subscription leave them out, the same way key-value share notices
already are, which covers rows stored before this change too. Puter's
own notifications and shares still show on the desktop.

The desktop and its notifications panel also ignore an app message
they are handed, and the desktop no longer notes one as received.
The notification docs now say where an app's notifications show.

* docs(notifications): say an app's notifications reach the app, not the panel

* test(notifications): wait for the unread replay instead of sleeping past it

* fix(gui): show notifications a frozen tab missed once it reconnects

A tab that was frozen or asleep keeps its socket registered until it
resumes, so a notification created meanwhile counts as shown and no
unread replay carries it: the tab never showed it, not even after a
reload.

On a socket reconnect the desktop and the dashboard now read the
undismissed notifications created since the page loaded and raise the
ones that page has not shown or seen dismissed, once. Every unread
replay is taken too, not only the first, and no longer only while the
events feed is down; a per-page log keeps either from raising anything
twice. App messages stay out.

* fix(gui): catch up after a reconnect without the page's clock

The reconnect catch-up compared each row's server time with the page's
own load time, so a page whose clock ran behind raised, on its first
reconnect, every undismissed notification from before it loaded.

A page now notes the undismissed notifications it knew at load (the
desktop from one listing, the dashboard from its first) and a reconnect
raises only those it neither knew then nor has raised since. The
dashboard reads them from the listing its reconnect refresh already
takes. The per-page log is bounded, and a caught-up notification shows
its title as the body when it has no text, as the replay does.

* style(gui): keep the reconnect comments to one line
2026-10-30 19:24:38 -07:00
Neal Shah 2a3c181657 DNS ui changes (#4417)
* DNS ui changes
2026-10-30 21:16:25 -04:00
Daniel Salazar 4881264c88 feat(ai): realtime reconnect notice, context restore and retry hardening (#4415)
* test(ai): end every realtime relay test's upstream before the next

The fake upstream now answers response.create with response.created first,
as OpenAI does, so closing those sessions no longer waits out the settle
window.

* feat(ai): ask realtime clients to reconnect before a restart

Open realtime sessions get a puter.reconnect event when the drain starts,
with close_in_ms set to the drain window, and new sessions are still refused
until the 1012 close at shutdown. A host extension can announce a restart
early by emitting serverRestartAnnounced; that only sends the notice, so a
false alarm leaves the server working normally. A session gets the notice at
most once a minute.

onServerDrainStart now receives the drain duration.

* fix(ai): retry realtime reconnects for about a minute and wait out offline browsers

Five attempts gave up after about 15 seconds, shorter than a server restart.
The conversation makes ten attempts with the backoff capped at 10 seconds,
and while the browser reports being offline it waits for the online event,
up to two minutes per reconnect, instead of spending attempts. close() ends
either wait and removes its listener.

* feat(ai): restore recent conversation context after a realtime reconnect

The provider starts a fresh session on every reconnect, so the conversation
lost all earlier items. The SDK now records finished items it can recreate
from the provider's own events: user text, user speech with a transcript,
assistant replies as text, and function calls with their outputs. After the
new session acknowledges its settings, and before ready fires, it adds back
the newest of them, up to 40 items and 48 KiB, which stays under the relay's
upload burst.

Assistant replies are cut to what the user saw or heard: the deltas shown
before an interruption, and the share of the transcript matching the audio
played before a truncation or a drop. Deleted items, unfinished replies,
images, speech without a transcript and incomplete function calls are left
out, as are function outputs whose call falls outside the window. Restored
copies take over their new item IDs. A refused item is reported once as
context_restore_failed without failing the reconnect.

* feat(ai): move a realtime conversation to a new session on the restart notice

On puter.reconnect the conversation waits until it is idle: no reply
requested, generating or playing, no user turn being spoken or transcribed,
no function call awaiting its output and no acknowledgment pending. Then it
reconnects through the usual path, emitting reconnecting (code 1012) and
ready with reconnected: true, without an error. Until then the session keeps
working, and if the server closes it first the normal 1012 reconnect takes
over. close() at any point stops the wait.

* docs(ai): realtime restart notice, context restore and reconnect limits

* test(ai): realtime reconnects against the local server

The mock provider answers with item events and whole replies like OpenAI.
New cases cover a dropped connection (cut through a TCP proxy so the client
sees 1006), a host-announced restart, and a draining server refusing the
reconnect; the first two check the provider receives the restored items
before the next turn. The browser media test waits for captured audio and
the reply instead of a fixed sleep. The live suite gains a tool and text
context round trip across a restart notice.

* ci: run the live realtime conversation tests with the AI provider suite

They need the OpenAI key, which only this workflow has, and skip themselves
without it.

* fix(ai): open the replacement before leaving a realtime session on the restart notice

The handover closed the current session and then reconnected, so every
notice cost a second or two in which sends failed and microphone audio was
dropped. On a single server that refuses new sessions while it drains, the
retries ran out during the drain and the conversation closed sooner than if
it had stayed. The conversation now opens a replacement session as soon as
the notice arrives and switches only once it is open and the conversation is
idle. A refused replacement is retried with backoff while the current
session keeps working; when the server closes it first, the held replacement
is used right away.

The idle check also missed some in-progress work:
- a reply created while the user was still speaking ended the user's turn;
  speech now lasts until its audio is committed or cleared, and a committed
  turn waits for the reply server turn detection creates
- only the newest function call was checked; every finished call since the
  last user message needs its output, and calls cut off by a cancellation
  don't count
- native session.update and conversation.item.create events the provider
  hadn't acknowledged could be lost; the switch waits for them, up to the
  acknowledgment timeout for items
- only the newest transcription was waited on; every committed item is
- clearing playback didn't recheck the switch

Restored items stay internal: the new session's copies of them no longer
reach listeners. At a reconnect, every queued item the user didn't hear is
cut, not just the one playing. A notice from a failed reconnect attempt is
dropped.

* test(ai): realtime replay after split settings, and a draining server refusing the replacement

The relay accepts the context replay right after two full-size settings
updates. Against the local server, a draining relay refuses the replacement
session while the current one keeps answering.

* ci: build the SDK in the live realtime step so it stands on its own

* fix(ai): a failed realtime reconnect attempt no longer poisons the next

An attempt that dropped after sending its session.update but before the
acknowledgment left that event ID queued, so the next attempt's
acknowledgment was matched to it and timed out, and so on until the
conversation closed. Turn state is now reset after each failed attempt.

* fix(ai): bound the realtime handover's waits and stop retries that can't succeed

- The first replacement opens within a randomized 3 seconds of the notice,
  so sessions notified together don't all open at once.
- A replacement refused with a code retrying can't fix, such as
  insufficient_funds, ends the attempts; the server's close still falls back
  to the usual reconnect.
- A held replacement that gets an error or its own server's notice is closed
  and replaced.
- A held replacement used after a drop gets a 3 second acknowledgment
  timeout, since its socket may have died with the network.
- close() during the switch closes the held replacement too.
- The reply to a committed turn and each transcription hold the switch for
  at most 10 and 30 seconds, like app items already did, and an app item is
  released only by an added item of its type, role and call ID.

* docs(ai): the realtime restart notice isn't guaranteed before every restart
2026-10-30 18:06:54 -07:00
c7c3f21e4b feat(ai): puter.ai.convo - Preliminary Realtime API implementation (#4401)
* feat(ai): relay OpenAI's Realtime API as puter.ai.assistant

Adds a WebSocket relay at /puterai/openai/v1/realtime on the api host,
one upstream OpenAI connection per client, frames passed through 1:1.
The client socket is accepted before auth so socket.io's 1 s
unhandled-upgrade timeout can't kill slow handshakes; failures arrive as
OpenAI-shaped error events and a 1008 close. Usage is metered per
response.done at the model's token rates, sessions are capped per user
(realtime.maxConnectionsPerUser), and each node logs counters,
event-loop delay and memory every 30 s.

puter.js gets puter.ai.assistant.connect(), returning a session with
send/on/off/close over the same OpenAI events.

realtime.upstreamUrl points the relay at a mock for load tests;
tools/realtime-stress has the mock upstream, a load generator with a
JSON report, and an agent runbook. The relay README lists its quirks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ai): backpressure and upload budget on the realtime relay

Neither direction applied backpressure: a client that stopped reading
grew a queue up to 64 MiB, and a client streaming 16 MiB frames was read
and forwarded as fast as it could send. A few such sockets pinned the
event loop and pushed RSS past 1 GB.

Each side now stops reading the other once 1 MiB is queued toward it,
and a client that keeps the upstream paused for 30 s is closed. Client
uploads are budgeted per session (1 MiB/s sustained, 16 MiB burst,
realtime.maxClientBytesPerSecond), pre-open buffering pauses reads at
1 MiB instead of rejecting at 32 MiB, and clients get a 30 s ping so
half-open sockets release their upstream and slot. Upstream frames use
a zero mask and skip UTF-8 validation, which the relay never needed.

The stats line gains elu. tools/realtime-stress gains abuse.mjs and a
runbook phase for event-loop abuse.

* fix(ai): drain, transcription metering and credential re-check on the realtime relay

Shutdown now drains: a new onServerDrainStart hook (run when SIGTERM
starts the drain, before the listener closes) makes the relay refuse new
sessions with 1012, open ones keep running through the drain window, and
at onServerPrepareShutdown each gets up to 5 s to finish an in-flight
response before closing with 1012 so the client reconnects elsewhere.

Input transcription is metered. The relay reads the type from each text
frame's first 96 bytes and, after forwarding, parses response.done,
input_audio_transcription.completed and session.created/updated (for the
transcription model) on setImmediate, so metering never holds up frames.
Unknown transcription models bill at the highest known rate.

Credentials are re-checked: a successful auth is cached in the kv.js
singleton for 60 s under a hash of the token, and a 15 s sweep
re-authenticates sessions whose entry expired, closing revoked or
suspended ones with invalid_api_key. Lookup errors keep the session and
retry on the next sweep.

* docs(ai): API shape proposal for puter.ai.conversation() across OpenAI, Gemini Live and xAI

* feat(ai): add composable realtime conversations with bounded transport

* fix(ai): realtime billing, subscription gate and conversation reconnects

Relay:
- Cancel and bill a response still generating when the client leaves,
  instead of cutting the upstream before response.done.
- Record realtime usage through the AI cost-factor hook.
- Drop the upstream byte and message caps; replies arrive faster than real
  time and tripping them closed sessions and skipped billing.
- Price usage as it arrives, before any forwarding decision, so a closing
  session never drops a usage event.
- Bill the model the session reports and refuse a session.update that
  changes the model.
- Require a paid plan.

SDK (puter.ai.convo):
- Reconnect after a dropped connection with the same settings, keeping
  listeners and media; emits reconnecting and ready { reconnected }.
- Errors reject only the operation they name; a failed send rejects only
  its own request.
- configure() sends only the given settings and settles on its own ack;
  commitAudio follows turn detection set natively.
- interrupt() right after sendText cancels the reply once it starts.
- Microphone: capture at the device rate, drop chunks instead of stopping
  on stalls or a congested socket, no timeout on the permission prompt,
  permission_denied on refusal.
- Speaker: 300 s playback bound without interrupting, played position
  accounts for output latency and gaps between chunks.
- Audio control events keep headroom over queued audio.
- Upgrade prompt on subscription_required / insufficient_funds.

Tests that need the built SDK and Chromium move to the puter.js suite.

* chore: remove stray node_modules symlinks

* feat(ai): retire assistant API and document realtime beta

* test(ai): wait for the matching realtime billing batch

* test(kv): exclude health probes from warm read measurements

* Revert "test(kv): exclude health probes from warm read measurements"

This reverts commit 631003f755.

* fix(ai): upstream message size, null usage, team 2FA and reconnect settings for realtime

Relay:
- Upstream messages may be up to 4 MiB; client messages stay at 64 KiB.
  An OpenAI event above 64 KiB (an echoed item near the client limit, a
  long reply, a retrieved audio item) closed the upstream before in-flight
  responses reported usage.
- An upstream that closes with a response still in flight raises a
  warning alarm.
- A null usage counts as none instead of closing the session.
- Seats whose team requires 2FA need it for realtime too, at connect and
  on the credential recheck.

SDK (puter.ai.convo):
- A reconnect reapplies every acknowledged setting, native session.update
  events included, split to fit the message limit.
- connect() during a reconnect settles with it.
- Reconnect delays are jittered, and closing cancels a pending delay.

The credential-recheck and browser worklet tests now fail when the
behavior they cover breaks.

* feat(ai): credit holds, attempt limits and late-response billing for realtime (#4407)

Relay:
- Each response holds 8,000 audio input and 4,096 audio output tokens of
  the account's budget, at the session model's rates, from
  response.created until its usage is recorded. A session opens only if
  one hold fits.
- Session attempts are limited to 60 per minute per credential, checked
  before authentication.
- A response the client requested that starts after the client leaves is
  cancelled and billed.
- Input transcription is priced at the costlier of the transcription
  models in force at commit and at completion.
- rate_limits.updated isn't forwarded, and provider organization IDs are
  removed from error messages.

SDK (puter.ai.convo):
- microphone.stop() during the permission prompt rejects the pending
  start() with session_closed.
- speaker.start() rejects with { code, message } for browser exceptions.
- connect() waits 20 seconds for the session, past the server's 15-second
  setup.
- Convenience events fill IDs the provider leaves out; types cover
  turn.done, error, and client events for send().

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 14:50:15 -07:00
Daniel Salazar 7401fe0b09 fix(notifications): receipts report what the user saw, sticky push opt-in, safer app pushes (#4413)
* fix(socket): count sockets that never acknowledge instead of asking them

askWithAck sent an acked broadcast to sockets outside the ack room only to
learn whether any were there. Those sockets never answer, so every node held
the ack callback until they disconnected. Each node now answers a server-side
count of them instead; a node that does not answer in time still reads as
holding one.

* feat(notifications): read focus and own display from receipt answers

A receipt answer may carry focused and showsOwn beside visibilityState; both
pass through to notif.delivery.settled when they are booleans. An answer with
received: false is not a responder, and an ask answered only that way settles
as unreceived.

* fix(notifications): name an app message's bad field before the size cap

An app message over the 16 KB value cap answered notification_value_too_large
even when one field was what broke a rule of its own. The fields are checked
first now, so the caller gets bad_request naming the field; the total cap still
applies after. Character counts skip the code point walk on strings that can't
fit.

* fix(push-sw): match the server's tags, keep taps, and show pages' notifications

- A push is tagged with the server's group, `<app uid>:<group>` or the uid, and
  alerts again when that replaces one showing. Handlers get the app's own group
  and the send time in ms from `ts`.
- The app's icon stands in when the notification has none.
- Default and relative tap URLs resolve against the worker's scope, still only
  on its origin.
- A tap posted to an open window waits briefly for the page to take it; without
  a click handler there, the window is taken to the URL, or one is opened.
- A page that got a notification live while away can ask the worker to show it;
  the worker holds it back while another window of the origin has focus.

* fix(notifications): answer every receipt ask

A page now answers each socket that asks about a notification, not only the
first, and answers { received: false } when the notification doesn't arrive
within the wait or the page holds too many asks, so the server's nodes can
let go of them. A received answer carries focused and showsOwn beside
visibilityState.

* fix(notifications): keep web push to users who turned it on, and show what a push would

Subscription:
- onPush() renews the subscription on load only for a user who turned push on
  on this origin with enable(); enable() records that, and disable(), sign-out
  and push_revoked forget it. One renewal runs per page and worker, however many
  onPush() calls there are, and push_disabled, push_revoked, push_unavailable
  and too_many_requests stay quiet there.
- puter.auth.signOut() on a website removes this browser's subscription with
  the leaving user's token and forgets the opt-in and kept taps, without
  holding up the sign-out.
- A subscription another service on the site made is left alone and rejected
  as push_subscription_conflict; one made with Puter's earlier key is replaced.
- A subscription refused with push_endpoint_revoked is replaced once, like
  push_subscription_invalid.
- Another worker holding the scope rejects push_worker_conflict, naming
  importScripts and the serviceWorker option, instead of waiting out the
  confirmation push.
- enable() rejects user_gesture_required outside a click instead of prompting,
  and a refused browser subscribe rejects push_subscribe_failed.

Live side:
- A page that is hidden or unfocused shows every app notification it receives,
  with or without a handler for its channel, through push-sw.js so a focused
  window of the app holds it back. It uses the server's tag, the app's icon as
  a fallback, the generic app line for sensitive ones, and the registration of
  its own worker.
- Page handlers get 5 seconds, as in the worker (push_handler_timeout).
- A feed stopped by a sign-out starts again on the next sign-in.
- onPush() still delivers live when IndexedDB is unavailable.
- onPush() rejects invalid_handler for a handler the worker could not rebuild
  from its source (methods, bound and built-in functions).
- Saving and removing a handler's source for the worker is ordered per key, so
  a quick unmount and remount keeps it.
- A tap posted by push-sw.js is acknowledged only when a click handler takes it.

* fix(gui): show app pushes under the app's name and Puter's icon

- On Puter's own origin, an app's push takes the app's name as its title, with
  the app's title and text in the body, and Puter's copy of the app's icon; an
  icon the app chose is never shown. A sensitive one keeps its generic line.
- A push's ts is its timestamp, and a newer push in a group alerts again.
- Re-registering this browser counts only apps that push to Puter's devices
  (puterDevices), so a website's Web Push confirmation alone never registers
  it.

* fix(gui): free the push dialog's button when registering fails late

A registration that failed after the dialog's deadline left the button
spinning and disabled. It now clears the attempt, settles on a state the
browser moved to meanwhile, or offers a retry.

* fix(permission): a user's own app grant clears a narrower via on the row

An app grant that already existed was skipped, so a row another path recorded
with extra.via kept it after the user approved the same permission in Puter.
A grant without via now rewrites such a row without it, keeping its other
extras; a grant with via still never touches an existing row.

* refactor(notifications): export the notification text cleaner

Extensions that word notifications can use the same cleaning as app messages.

* docs(notifications): describe when pushes go out, taps, worker rules and enable() errors

Pushes go out when no open page answers that it's visible and focused or
shows the notification itself. Documents the page-shown notification, the
sticky opt-in, tap routing, group and sensitive behavior, service worker
rules, requirements, every enable() and onPush() error, push caps for
unlisted apps and websites, and sending from workers and events handlers.

The push playground example is removed (the playground can't serve the
site's service worker) and redirects to the guide. The minimal page catches
enable() errors and sends after 5 seconds. The recipe no longer sends on
every load, and app.message is listed in events fetch and the catch-up
recipe.

* docs(notifications): per-app browser caps and when push_endpoint_revoked appears

* docs(notifications): the page's handler timeout

* fix(notifications): leave a site's own service worker alone when enabling pushes

Registering /puter-sw.js at a scope another worker holds queued it behind that worker, so enable() rejected with push_worker_conflict but swapped the site's worker out once its tabs closed. Now enable() checks the scope first and rejects without registering.

* docs(notifications): a background page shows score alerts itself
2026-10-30 14:43:48 -07:00
Daniel Salazar 61b713438d docs(notifications): push setup requirements, browser and device support, and push recipes (#4406)
Adds worker requirements, per-browser/OS support (iOS Home Screen, macOS/Windows/Android settings), and troubleshooting to the push setup page. Website taps now documented on on('click'). New recipes: notify chat members of new messages, alert users when shared data changes.
2026-10-30 09:12:19 -07:00
Daniel Salazar 7ceaf69651 docs(notifications): import push-sw.js from the root of js.puter.com (#4405)
The worker is published next to puter.js at https://js.puter.com/push-sw.js.
Docs and the worker's own header now point there. The js subdomain serves
/push-sw.js for self-hosted setups and keeps /v2/push-sw.js as an alias.
2026-10-29 20:10:27 -07:00
Juan Fernando CastroandDaniel Salazar c00f693ec6 feat(notifications): report how each live notification was received (#4403)
* feat(notifications): report how each live notif.message was received

SocketService.sendWithAck sends an event to rooms or sockets and waits,
up to a timeout, for an acknowledgement from each socket that listed the
event in its handshake (`auth.acks`); every other socket gets a plain
emit. It lists sockets across the cluster with fetchSockets and asks the
listed ones through the adapter's broadcast acks, and counts this node
alone when the adapter cannot list the others.

The emitting region now sends notif.message through it, with the window
from `notificationAckTimeoutMs` (default 5000), and emits the node-local
`notif.delivery.settled` with the outcome: received, unreceived,
no-socket or remote-only (sockets only in a peer region), whether any
socket there acknowledges, and each responder's visibilityState. Peer
regions keep sending their own copy as before; the GUI event carries
`region_delivered` so this region does not send it twice.

The desktop and dashboard list notif.message in their handshake and
answer its callback once with document.visibilityState. Nothing in core
acts on the settled event.

* fix(notifications): send live notif.message at once and report what the count covers

sendWithAck now emits to the whole room straight away as a broadcast with
ack, and lists the room's sockets alongside it only to fill in the
counts, so nobody waits on the listing. Whether the adapter can count
the cluster's servers is checked once; where it cannot, sends go out
per socket on this node with a one-time warning. The result carries
`complete`: false when only this node's sockets were counted, either
for that reason or because the listing failed or took over 1 s, which
is warned about at most once a minute.

notif.delivery.settled gains `sendId`, `type`, `complete` and
`peerRegions`. An incomplete count with no socket found settles as
unreceived rather than no-socket, and the peer-region read runs
alongside the send instead of after it. notif.written carries the same
`sendId`, minted per send (an update gets a new one) and passed through
notif.created to the socket adapter on the fold-in path.

The comment on revoking sockets no longer says fetchSockets is
unavailable with our Redis client; it works with ioredis cluster.

* fix(notifications): keep a failed server count from ending the process

socket.io awaits the adapter's serverCount without a catch on every
broadcast with an ack, and the redis-streams adapter's count rejects as
soon as one cluster node fails to answer. The count on the root
namespace's adapter now never rejects: a failure reports the last good
count (1 before any), and is warned about at most once a minute. A stale
count only lets an ack wait run to its timeout.

Whether sendWithAck takes the cluster path is no longer a one-shot check
awaited before the emit. Until a count has succeeded, sends go out per
socket on this node at once while a count is retried in the background,
at most every 30 s (SocketService.COUNT_RETRY_MS); the first success
switches to the cluster path for good.

* fix(notifications): ask only acknowledging sockets, through an ack room

A socket whose handshake lists an event in `acks` now joins
`<room>:acks:<event>` for each of its rooms and its own id.
sendWithAck sends the ordinary broadcast to the target rooms except
those ack rooms, which keeps it on the adapter's stream, and a broadcast
with an ack only to the ack rooms. Sockets that never acknowledge are no
longer asked, so other nodes hold no pending ack for them and a send
completes as soon as every acknowledging socket answered. The per-socket
listing that matched socket.data.acks is gone; the ack-capable count
comes from the room membership the listing already returns. The
per-socket path stays only for a node whose adapter has not counted
servers yet.

A failed server count during a send now marks its counts incomplete, so
a stale count can't make a listing that missed a node read as complete.
With no count ever succeeded (an in-memory Redis), the failure is warned
once per process instead of once a minute.

* fix(notifications): say whether every socket on a live notification acknowledges it

A tab on an older bundle beside one that acknowledges made the settle read
as ack-capable, so a listener could not tell that the older tab may have
shown the notification. notif.delivery.settled now carries everySocketAcks.

* feat(notifications): app pages acknowledge the live copy of an app message

NotificationService.appReceipt() asks an app's own sockets whether a
notification reached a page, and settles it as notif.delivery.settled with
appUid, the same outcomes as the desktop's. Callers ask only when the answer
decides something, since it is a round trip to every node.

- SocketService.askWithAck asks without listing sockets: sockets that don't
  acknowledge get an acked broadcast too, which can only time out, and that
  says whether an older client shares the target.
- puter.js declares notif.receipt on its filesystem and events sockets. A page
  answers once one of its notification feeds received that uid, before any
  handler runs, and never when no feed on the page got it.
- Docs describe the push route as "no page of the app received it live
  within 5 seconds".

* fix(notifications): count a socket that acknowledges other events as a current client

Every desktop page also holds puter.js's own socket, which never shows
notif.message, so "every socket acknowledges notif.message" never held there.
sendWithAck now reports `declaring`, the sockets that listed any event in
auth.acks, and everySocketAcks on the desktop's settle means no socket here
declares nothing.

* fix(puter-js): fall back to the Continue dialog when a popup can't be opened

Some browsers report user activation but still return null from
window.open (iOS home-screen web apps, Safari after awaited work), so
permission prompts, signIn, enablePush, and GUI action popups failed
without showing anything.

PuterDialog.launch() now owns the open-now-or-show-Continue decision for
every popup flow and shows the consent dialog when the direct open
returns null. On cross-origin-isolated pages signIn leaves settling to
the /login/wait relay on both launch paths.

* fix(notifications): show a live push when the page's window isn't focused

onPush showed the handler's result as a system notification only when
the tab was hidden. A visible tab in a window the user left showed
nothing. It now also shows when the document doesn't have focus. Inside
the Puter desktop the desktop's banner shows the notification, so an
app's frame still never does.

* build(puter-js): publish push-sw.js exactly as written

The build emits dist/push-sw.js next to puter.js, but terser minified it
along with the bundle. It is now marked as already minimized, so the file
apps import into their service worker is the source unchanged.

* fix(notifications): send live notif.message plainly and ask for receipts on request

Every live notif.message used to be sent with an ack, list the user's sockets
across the cluster (fetchSockets ships each socket's handshake between nodes)
and wait out the receipt window, whether or not anything would act on it.

notif.message is a plain send again. NotificationService.receipt() asks the
desktop's sockets, or with appUid an app's own, the way app receipts already
did: an ask with no socket listing, settled as notif.delivery.settled. A
caller asks only when the answer changes what it does.

- The desktop and dashboard sockets answer notif.receipt from the page's
  puter.js receipt log and note each live notif.message in it, so every socket
  on the page answers alike. A page whose puter.js has no log lists no acks.
- SocketService.sendWithAck, the socket listing and region_delivered are gone.
  askWithAck counts this node's sockets when the adapter cannot count servers.
- The send id stays on notif.written and the settle, and leaves the wire.

* fix(socket): join only the ack room receipts are asked in

A socket listing events in auth.acks joined an ack room beside each of its
rooms and its own id. Receipts are only asked in the user's room, or the app's
for an app socket, so that is the one ack room it joins.

* docs(notifications): publish the push limits of 12 a minute and 1,000 a day

Pushes are capped per user per app at 12 a minute and 1,000 a day, and at 6 a
minute and 200 a day while an app is new. The rate limits table and the
notification pages said one a minute and 100 a day.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-29 18:57:36 -07:00
Juan Fernando Castro dace77c212 fix(gui): a request the server no longer accepts signs the tab out locally (#4390)
A 401 outside the reauth flow used to run the full logout, which ends the
session on the server and takes every device registered under it. The tab
now drops its token and reloads instead; the session cookie signs it back
in when it still holds. A second 401 right after that reload falls back
to the full logout. A peer tab signing out is mirrored the same way.
2026-10-29 09:19:43 -04:00
Juan Fernando CastroandDaniel Salazar 8e1233d82a feat(push): notifications:push app permission (#4317)
* feat(push): notifications:push app permission

Apps need a per-app notifications:push grant for their notifications to
go out as pushes. hasPushConsent checks it as the app acting for the
user, so only that user's grant to that app answers yes, and a revoke
takes effect on the next check. The permission dialog describes it and
the perms docs list it.

* fix(push): canonical app uid in the push consent check

* fix(push): let users grant apps notifications:push

The grant route now only accepts permissions from a fixed set of
app-grantable families, which refused notifications:push, so an app could
never be given push consent. Register it next to events:background.

* refactor(permission): keep the notifications:push constant with the permission code

The consent check that reads it lives with the delivery code, outside
core. The grant flow test checks the grant through the permission
service directly.

* feat(notifications): node-local notif.written hook

NotificationService emits notif.written after each non-silent write and
after an in-place rewrite, on the node that wrote the row, so an
extension can act on new notifications without a second bus round trip.
It stays off the published event stream; notif.created carries the
notification itself.

* feat(puter-js): puter.notifications

An app notifies the user it runs for, reads its own notifications back
and follows new ones, hears clicks on them from the desktop, and asks
for or checks the notifications:push grant. Puter's own apps also
register, list and remove the user's devices.

* feat(puter-js): device calls report push_unavailable without a push service

A deployment with no push service has no /push routes, so core answers
the device calls with its catch-all 404. registerDevice, listDevices and
removeDevice turn that into push_unavailable, kept apart from a missing
device, which comes back as device_not_found.

* feat(homepage): serve the service worker at /sw.js

/sw.js serves the GUI's sw.js, preceded by one importScripts per script
an extension registered with registerServiceWorkerScript. Whether the
GUI registers the worker follows the pushEnabled gui param, which only
an extension sets. Notification reads also carry the row's app_uid, so
the GUI can route a tap to the app a notification is about.

* feat(gui): notifications settings for push devices and apps

Settings -> Notifications turns pushes on and off for this browser
through a push client an extension registers, lists the user's devices
and the apps allowed to push, and switches Puter's own pushes. The tab
shows only when the pushEnabled gui param is true. The sessions window
shares the user agent formatting with it.

* feat(gui): service worker that shows pushes and routes taps

sw.js shows each push as a notification and, on a tap, focuses or opens
a Puter window and hands it the tapped notification. The desktop routes
the tap to the app the notification names, opening it when needed, or
to the notifications panel. The worker is registered only when the
pushEnabled gui param is true.

* feat(gui): notifications settings read a missing push service as unavailable

Without a push extension the /push routes answer core's plain 404. The
GUI maps that to push_unavailable, so the tab says pushes aren't
available on this server instead of reporting a load failure, and a
device removal only counts as done when the device itself is missing.
The push client doc describes the payload sw.js reads and where the
push endpoints come from.

* fix(gui): keep the service worker on servers without push

Unregistering /sw.js wherever pushEnabled is unset ended the browser's
push subscription, so a single page load on such a server stopped pushes
from every server. Turning push off already ends the subscription.

* feat(gui): the service worker tells open tabs when the push subscription changes

* fix(gui): removing this browser in Settings also forgets its push registration

* fix(gui): removing this browser also deletes it on the server when its local record is gone

* fix(notifications): notif.written hands out a copy with the audience

Listeners get their own copy of the value, so what the socket carries is
what was written. Both emits name the audience, and an update names the
row's app when the row or the value says which.

* fix(notifications): read the app off the notification on an update

* fix(gui): route push taps through the stored notification

A tap from the worker names only a notification; the desktop and the
dashboard read the signed-in user's own row and route from its app,
target and data, as a tap from the URL already did. Without a row the
dashboard opens the list and the desktop does nothing.

* fix(gui): local banner tags carry the app

An app message's group is tagged under its app, as the server tags a
push, so two apps using the same group name keep separate banners.

* fix(gui): a tap delivered twice routes once

The page remembers the notifications it handed over in the last ten
seconds and drops a repeat, whichever way it arrives.

* fix(gui): the service worker never reuses cached imports

The worker is registered with updateViaCache set to none, so the scripts
it imports are fetched fresh on every update check.

* fix(gui): turn on can't run twice at once

The browser card's button stays out while the client is enabling or
disabling, through any redraw in between, and a status read begun before
the press no longer draws the card.

* feat(gui): turn on browser push when an app is allowed to notify

Allow on a permission request that includes notifications:push also
turns push on in this browser, from the same click, when it is off here.
The decision never depends on it. The permission popup starts the
service worker for such a request, answers the opener first and closes
once the registration is done. The Notifications tab records a browser
the user turned off, per user.

* feat(gui): register this browser for pushes again where it already allows them

On desktop and dashboard boot, and when an app that holds
notifications:push is launched, a browser whose notification permission
is already granted registers again without a click, unless the user
turned push off in this browser.

* feat(sdk): puter.notifications.enablePush() and the enable-push popup

An app on its own domain can offer to turn push on in the user's browser
from its own button: puter.notifications.enablePush() opens a popup on
the Puter origin whose single button supplies the click the browser's
prompt needs, and inside the desktop the same dialog is shown over IPC.
The popup hands the opener no token. Docs for the new call and for the
paths that now call the push client's enable().

* fix(gui): wait for the browser's prompt before closing the permission popup

The deadline on a registration begun from a click counts no time while
the browser's own notification prompt is open, in the permission popup
and in the enable-push dialog. The enable-push dialog keeps one
registration in flight, settles on whenever it resolves while open, and
offers a retry only after a real failure. A permission dialog that ends
in a denial after Allow began a registration turns push back off and
restores the opt-out it cleared.

* fix(gui): one enable-push dialog at a time, only for apps that may send pushes

The desktop and the popup show the enable-push dialog only when the
asking app holds notifications:push. The IPC branch answers a second
call unavailable while a dialog is up, as the feedback dialog does. The
silent registration on boot and app launch runs once at a time. A popup
whose opener cannot be attested answers the enable-push request before
it closes.

* feat(events): isOnline presence check for a user and app

* feat(notifications): onPush and enable with Web Push on the app's own origin

puter.notifications.onPush(key, handler) handles an app's notifications for
one channel: live over the events socket while the app is connected, and as
a browser push otherwise. enable() asks for notification permission on the
site's origin and subscribes through the site's worker, which imports
push-sw.js (served at js.puter.com/v2/push-sw.js). A new subscription is
confirmed by a challenge push the worker hands back to the page. disable()
removes it.

The GUI uses the same flow through /sw.js when the server publishes
pushWebPushKey.

* fix(notifications): visible sender for app messages; local banners follow push consent

An app title of only invisible or blank-filler characters no longer leaves
an app message without a sender; it falls back to the app's name, then its
uid. Hidden-tab banners now show only while push is on in this browser, and
for an app's message only when the app holds notifications:push.

* feat(notifications): enable() asks through Puter's prompt after a revoke

When the user turned an app's notifications off in Puter, the browser's
permission on the app's origin no longer re-enables them. enable() asks
again through Puter's permission prompt and retries once.

* fix(notifications): enable() rejects on a revoke, onPush() syncs quietly, one permission check per app a minute

enable() no longer opens Puter's prompt after a push_revoked error; apps call requestPermission() from a click and enable() again. onPush()'s page-load sync no longer reports push_unavailable or push_revoked. Hidden-tab banners for an app's messages check the app's push permission once a minute instead of once per message.

* docs(notifications): describe the confirmation step as what happens; one-line comments and neutral fixtures

* docs(notifications): guide for app push notifications, limits and prices

* fix(puter-js): subscribe once the worker is active, and retry a subscription the push service hasn't settled

* fix(gui): the desktop's hidden-tab banner is for Puter's own notifications only

* docs(notifications): app pushes at $2.50 per million

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-29 09:19:09 -04:00
Juan Castro 722c25e0dc Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-29 09:11:29 -04:00
Juan Fernando Castro 9e449e635b Merge pull request #4399 from HeyPuter/juancastro/put-2180-push-docs
docs(notifications): guide for app push notifications, limits and prices
2026-10-29 09:10:44 -04:00
Juan Fernando Castro ba7011b5bb Merge pull request #4391 from HeyPuter/juancastro/put-2258-push-from-app-subdomains
feat(gui): turn on browser push when an app is allowed to notify
2026-10-29 09:10:34 -04:00
Juan Castro 25d5843d03 Merge juancastro/put-2262-web-push into juancastro/put-2180-push-docs 2026-10-29 08:59:08 -04:00
Juan Castro 4e6d655f4c Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push 2026-10-29 08:59:06 -04:00
Juan Castro bd1dd6564e Merge juancastro/put-2172-notifications-settings into juancastro/put-2258-push-from-app-subdomains 2026-10-29 08:59:04 -04:00
Juan Castro 69a3e918a7 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-29 08:59:03 -04:00
Juan Castro aded055212 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-29 08:59:01 -04:00
Juan Castro e54750492f Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-29 08:58:59 -04:00
Juan Castro 406fe4dcfc Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-29 08:58:57 -04:00
Juan Castro 2e085c0b75 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-29 08:58:49 -04:00
Juan Castro b95fb870a5 Merge main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-29 08:58:39 -04:00
Daniel Salazar 28be1851f9 fix(workers): bind credentials and cleanup to the owning site (#4362)
* fix(workers): bind credentials and cleanup to the owning site

* test(workers): deploy valid source in the site-bound token test

Where workerd is installed the deploy starts the script, and the bare
identifier `src` threw at startup as an unhandled rejection.
2026-10-28 17:31:15 -07:00
Juan Castro ff83360272 docs(notifications): app pushes at $2.50 per million 2026-10-28 19:53:26 -04:00
Daniel Salazar b19fde242f fix(networking): recover shared relay clients and preserve write order (#4377)
* fix(networking): recover shared relay clients and preserve write order

* fix(networking): clear the flush flag with the last queue check

A write landing between the drained queue and the connect path resuming
was queued with nothing left to send it.

* fix(networking): keep a throwing socket listener out of connect and close

An open listener that threw was caught as a failed connect and could
retry onto a second stream; an error listener that threw stopped the
socket from ever closing. Both now rethrow on their own tick.
2026-10-28 16:05:54 -07:00
Daniel Salazar c2b38786f7 fix(ai): meter converted audio from reported output duration (#4366)
* fix(ai): meter converted audio from reported output duration

* fix(ai): size undocumented voice changer output formats as the default

* fix(ai): skip WAV headers when metering and renew streamed charge holds

A wav_* output's RIFF header was counted as samples, rounding a 3 s clip
up to 4 s. The streamed-charge hold now renews until the charge lands, so
a slow reader can't outlast it.

* fix(ai): time Opus output by its pages and renew holds through slow opens

Opus is variable bitrate, so a mostly silent clip billed far under its
length at the nominal rate; its length now comes from the last Ogg
page's granule position. The streamed-charge hold renews from before the
upstream opens, since opening can run as long as the hold's TTL, and is
released even if recording the charge throws. Speech-to-text extra_body
can no longer override response_format or stream, which billing reads.
2026-10-28 16:05:36 -07:00
Daniel Salazar 7c6c0f2d39 fix(ai): finish async media accounting and recheck output access (#4365)
* fix(ai): finish async media accounting and recheck output access

* fix(ai): settle late media jobs on their outcome and keep their slot

A job still running when the poll window closes no longer charges its
estimate up front. The caller still gets the 504; the job is followed in
the background, holding its credit, until it settles, and is metered from
the provider's outcome: what it reports, nothing for a failure that
reports no cost, the estimate only if it never settles.

The driver concurrency slot is now held until the handler and any work it
carries on after the response settle, so a disconnect no longer frees a
slot while polling continues.

A revoked puter_output_path is reported as metadata.outputSaved = false.

* fix(ai): bill a lost media job the same in and after the window

A poll error that isn't transient now ends the same way whether it hits
inside the window or after it: a client-error refusal is a failure the
provider doesn't bill, while auth, not-found and unclassified errors mean
we lost a job the provider still runs, charged its estimate. A Hailuo
clip already billed retries a transient file lookup instead of failing,
and a concurrency slot renews from the moment it is held.
2026-10-28 16:05:26 -07:00
Juan Castro 5ac41f90b6 Merge juancastro/put-2262-web-push into juancastro/put-2180-push-docs 2026-10-28 16:30:47 -04:00
Juan Castro c024e007aa Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push 2026-10-28 16:30:45 -04:00
Juan Castro eeedeb817f Merge juancastro/put-2172-notifications-settings into juancastro/put-2258-push-from-app-subdomains 2026-10-28 16:30:42 -04:00
Juan Castro ca79f01edd Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-28 16:30:40 -04:00
Juan Castro 55ba45e159 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-28 16:30:38 -04:00
Juan Castro e8688d5025 Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-28 16:30:35 -04:00
Juan Castro 10f10029c2 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-28 16:30:33 -04:00
Juan Castro 11a94ed608 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-28 16:30:30 -04:00
Juan Castro 5c35f8d053 Merge main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-28 16:30:10 -04:00
Juan Castro c5a4b1f3b5 Merge juancastro/put-2262-web-push into juancastro/put-2180-push-docs 2026-10-28 16:27:20 -04:00
Daniel Salazar 179ad990d6 feat(events): simplify onPersistent delivery (#4395)
* feat(events): simplify onPersistent delivery

Replace delivery/targets with a boolean single. Without single, every
event runs the app's published handler in its events worker and is also
sent to every connected client; the server no longer routes between the
two. single keeps leasing each event to one consumer that acks.

delivery: 'single' | 'broadcast' is still accepted and targets is
ignored. Stored rows are read by their delivery column alone.

* docs(events): compare onLocal, onPersistent and single

* feat(events): rename single to queue

onPersistent({ queue: true }) is the work-queue mode: each event goes to
one consumer at a time and is offered again until acknowledged.
delivery: 'single' is still accepted as an alias, and subscription
objects report queue instead of single.

The docs explain the three modes (onLocal, default onPersistent, queue)
and each events recipe says which one it uses and why.

* docs(events): retries are the only repeat for a queued event

* feat(events): queue deliveries fall through to the worker on decline

A queued delivery gets one socket attempt under a 15 s lease instead of
two 60 s ones. A client with no handler for it, or whose handler fails,
sends events.nack and the worker takes it without waiting for the lease.
Declines are relayed to the region holding the lease like acks are, and
a decline for a lease that is no longer the client's is ignored.

* feat(events): drop thrown errors, retry only temporary ones

A queued handler that throws now drops the event and counts one failure
toward the pause. Network errors, timeouts, rate limits, 5xx responses
and errors thrown with retry: true are retried with the existing
backoff. One classifier is shared by puter.js and the events worker
runtime, which inlines it at build time. A dropped event is billed once,
like a completed one. ack() remains as an optional early release.

* docs(events): queued events reach every tab with a handler
2026-10-28 12:58:51 -07:00
Juan Castro ff8dde6895 fix(gui): the desktop's hidden-tab banner is for Puter's own notifications only 2026-10-28 14:20:24 -04:00
Juan Castro d587ffa428 fix(puter-js): subscribe once the worker is active, and retry a subscription the push service hasn't settled 2026-10-28 13:50:19 -04:00
Juan Castro 0fe68dae30 docs(notifications): guide for app push notifications, limits and prices 2026-10-28 12:37:28 -04:00
Juan Castro eb1ad240f2 docs(notifications): describe the confirmation step as what happens; one-line comments and neutral fixtures 2026-10-28 10:06:55 -04:00
Juan Castro d40cc499f5 fix(notifications): enable() rejects on a revoke, onPush() syncs quietly, one permission check per app a minute
enable() no longer opens Puter's prompt after a push_revoked error; apps call requestPermission() from a click and enable() again. onPush()'s page-load sync no longer reports push_unavailable or push_revoked. Hidden-tab banners for an app's messages check the app's push permission once a minute instead of once per message.
2026-10-28 10:06:55 -04:00
Juan Castro 06ef9506fa Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push 2026-10-28 10:01:02 -04:00
Juan Castro 5513c23042 Merge juancastro/put-2172-notifications-settings into juancastro/put-2258-push-from-app-subdomains 2026-10-28 09:46:54 -04:00
Juan Castro 8aade99cc8 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-28 09:45:57 -04:00
Juan Castro 6f9719fd65 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-28 09:45:51 -04:00
Juan Castro 352a2fbabd Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-28 09:45:45 -04:00
Juan Castro 59700519a6 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-28 09:45:38 -04:00
Juan Castro 8482ff2167 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-28 09:45:31 -04:00
Juan Castro b42f2acf06 Merge main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-28 09:44:38 -04:00
Daniel Salazar 1b6de39a13 feat(notifications): enable() asks through Puter's prompt after a revoke
When the user turned an app's notifications off in Puter, the browser's
permission on the app's origin no longer re-enables them. enable() asks
again through Puter's permission prompt and retries once.
2026-10-28 01:17:00 -07:00
Daniel Salazar 1aa7caf44c fix(notifications): visible sender for app messages; local banners follow push consent
An app title of only invisible or blank-filler characters no longer leaves
an app message without a sender; it falls back to the app's name, then its
uid. Hidden-tab banners now show only while push is on in this browser, and
for an app's message only when the app holds notifications:push.
2026-10-28 01:06:54 -07:00
Daniel Salazar 40253ae12b feat(notifications): onPush and enable with Web Push on the app's own origin
puter.notifications.onPush(key, handler) handles an app's notifications for
one channel: live over the events socket while the app is connected, and as
a browser push otherwise. enable() asks for notification permission on the
site's origin and subscribes through the site's worker, which imports
push-sw.js (served at js.puter.com/v2/push-sw.js). A new subscription is
confirmed by a challenge push the worker hands back to the page. disable()
removes it.

The GUI uses the same flow through /sw.js when the server publishes
pushWebPushKey.
2026-10-28 01:05:36 -07:00
Daniel Salazar 3b4945f018 feat(events): isOnline presence check for a user and app 2026-10-28 00:53:28 -07:00
Juan Castro fe58213bb5 fix(gui): one enable-push dialog at a time, only for apps that may send pushes
The desktop and the popup show the enable-push dialog only when the
asking app holds notifications:push. The IPC branch answers a second
call unavailable while a dialog is up, as the feedback dialog does. The
silent registration on boot and app launch runs once at a time. A popup
whose opener cannot be attested answers the enable-push request before
it closes.
2026-10-27 16:48:07 -04:00
Juan Castro 6c35780d59 fix(gui): wait for the browser's prompt before closing the permission popup
The deadline on a registration begun from a click counts no time while
the browser's own notification prompt is open, in the permission popup
and in the enable-push dialog. The enable-push dialog keeps one
registration in flight, settles on whenever it resolves while open, and
offers a retry only after a real failure. A permission dialog that ends
in a denial after Allow began a registration turns push back off and
restores the opt-out it cleared.
2026-10-27 16:48:07 -04:00
Juan Castro 415c179770 feat(sdk): puter.notifications.enablePush() and the enable-push popup
An app on its own domain can offer to turn push on in the user's browser
from its own button: puter.notifications.enablePush() opens a popup on
the Puter origin whose single button supplies the click the browser's
prompt needs, and inside the desktop the same dialog is shown over IPC.
The popup hands the opener no token. Docs for the new call and for the
paths that now call the push client's enable().
2026-10-27 16:27:23 -04:00
Juan Castro feb0937b6d feat(gui): register this browser for pushes again where it already allows them
On desktop and dashboard boot, and when an app that holds
notifications:push is launched, a browser whose notification permission
is already granted registers again without a click, unless the user
turned push off in this browser.
2026-10-27 16:27:23 -04:00
Juan Castro f880544ebe feat(gui): turn on browser push when an app is allowed to notify
Allow on a permission request that includes notifications:push also
turns push on in this browser, from the same click, when it is off here.
The decision never depends on it. The permission popup starts the
service worker for such a request, answers the opener first and closes
once the registration is done. The Notifications tab records a browser
the user turned off, per user.
2026-10-27 16:27:23 -04:00
Neal Shah 25a574b57d GUI for custom domains (#4355)
* PUT-2226

* modifications'

* routing fixes

* secondary custom domain verification
2026-10-27 15:54:02 -04:00
Juan Castro 389fd26bee fix(gui): turn on can't run twice at once
The browser card's button stays out while the client is enabling or
disabling, through any redraw in between, and a status read begun before
the press no longer draws the card.
2026-10-27 14:20:17 -04:00
Juan Castro d77ebadfe6 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-27 14:18:31 -04:00
Juan Castro 7e78a4feef fix(gui): the service worker never reuses cached imports
The worker is registered with updateViaCache set to none, so the scripts
it imports are fetched fresh on every update check.
2026-10-27 14:18:07 -04:00
Juan Castro 7aa4d91ae9 fix(gui): a tap delivered twice routes once
The page remembers the notifications it handed over in the last ten
seconds and drops a repeat, whichever way it arrives.
2026-10-27 14:17:57 -04:00
Juan Castro 5f7ab80721 fix(gui): local banner tags carry the app
An app message's group is tagged under its app, as the server tags a
push, so two apps using the same group name keep separate banners.
2026-10-27 14:17:48 -04:00
Juan Castro b3dbfdbd7b fix(gui): route push taps through the stored notification
A tap from the worker names only a notification; the desktop and the
dashboard read the signed-in user's own row and route from its app,
target and data, as a tap from the URL already did. Without a row the
dashboard opens the list and the desktop does nothing.
2026-10-27 14:17:40 -04:00
Juan Castro 60dfff3b0b Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-27 14:11:58 -04:00
Juan Castro 5a83c4a045 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-27 14:11:58 -04:00
Juan Castro fe8acf007c fix(notifications): read the app off the notification on an update 2026-10-27 14:11:46 -04:00
Juan Castro 8010c9fc94 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-27 14:11:10 -04:00
Juan Castro bbf375db14 fix(notifications): notif.written hands out a copy with the audience
Listeners get their own copy of the value, so what the socket carries is
what was written. Both emits name the audience, and an update names the
row's app when the row or the value says which.
2026-10-27 14:10:56 -04:00
Juan Castro fe70e7c6ba Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-27 14:08:01 -04:00
Juan Castro f0d7a69dd4 fix(notifications): cap the sender name on app messages
The app's title or name is cut to the title length, counting code points,
before it is stored on the message.
2026-10-27 14:07:27 -04:00
Juan Castro 5ce131517b fix(notifications): refuse an app message before it spends the day window
The minute window is spent first, so a request the minute window refuses
between the read and the spend leaves the day window untouched.
2026-10-27 14:07:25 -04:00
Daniel Salazar 49b38f3482 fix: seat creation (#4387) 2026-10-27 12:49:31 -04:00
Nariman JelvehandClaude Opus 5.5 241fb7be8a feat(dashboard): simplify the share modal
- The access level and Share button appear beside the field only once
  someone is entered, instead of a full-width button that sat disabled.
- The access list sits directly under the field. Each person's menu ends
  with Remove access (or Cancel invitation), replacing the trash column;
  the inline confirmation stays.
- General access is one row: a lock or globe icon, the switch with its
  explanation beneath, and the level on the right.
- A footer holds Copy link and Done. Copy link is offered for any single
  item, since the link opens it for anyone who has access, and confirms
  in place. Status messages live in the footer; successes fade.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-27 09:08:44 -07:00
dependabot[bot] 0167a9d10e chore(deps): bump fast-uri from 3.1.7 to 3.1.9 in /src/puter-js (#4336)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.7 to 3.1.9.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.9)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-27 08:54:47 -07:00
dependabot[bot] 5a879a4346 chore(deps): bump smtp-server from 3.19.23 to 3.19.24 (#4383)
Bumps [smtp-server](https://github.com/nodemailer/smtp-server) from 3.19.23 to 3.19.24.
- [Release notes](https://github.com/nodemailer/smtp-server/releases)
- [Changelog](https://github.com/nodemailer/smtp-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/smtp-server/compare/v3.19.23...v3.19.24)

---
updated-dependencies:
- dependency-name: smtp-server
  dependency-version: 3.19.24
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-27 08:54:33 -07:00
Juan Castro c13bff0563 fix(gui): removing this browser also deletes it on the server when its local record is gone 2026-10-27 11:10:32 -04:00
Juan Castro 2f94ee1088 fix(gui): removing this browser in Settings also forgets its push registration 2026-10-27 11:07:23 -04:00
Juan Castro 3068059058 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-27 11:06:38 -04:00
Juan Castro 7ee0cbfac9 feat(gui): the service worker tells open tabs when the push subscription changes 2026-10-27 11:06:14 -04:00
Juan Castro dfa6f4effc Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-27 10:44:19 -04:00
Juan Castro 20c63b6282 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-27 10:44:19 -04:00
Juan Castro dd44c0704a Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-27 10:44:19 -04:00
Juan Castro 681b9cd463 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-27 10:44:19 -04:00
Juan Castro 9fb66137a3 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-27 10:44:14 -04:00
Juan Castro 057aa116a4 Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-27 10:44:11 -04:00
Daniel Salazar 1cea1b2a0a perf(thumbnails): keep read URLs stable within an hour (#4379) 2026-10-26 22:16:54 -07:00
Daniel Salazar 6c78d2f9e9 fix(ui): reject failed desktop IPC requests (#4378) 2026-10-26 22:14:19 -07:00
Daniel Salazar e8aa12ace5 fix(puter-js): replay requests with fresh body credentials (#4376) 2026-10-26 22:10:24 -07:00
Daniel Salazar f1964778b4 fix(puter-js): coordinate sign-in callbacks and filesystem sockets (#4375) 2026-10-26 22:09:52 -07:00
Daniel Salazar c8f7b1ceda fix(auth): support recovery codes when disabling two-factor auth (#4374) 2026-10-26 22:07:52 -07:00
Daniel Salazar 0411a92f4e perf(dashboard): share launch-app requests and size recent icons (#4373) 2026-10-26 22:07:33 -07:00
Daniel Salazar 5cfd4d1f53 fix(hosting): honor user-granted subdomain permissions (#4372) 2026-10-26 22:07:04 -07:00
Daniel Salazar e1a36215f6 fix(sharing): preserve team grants and scope link invalidation (#4371) 2026-10-26 22:06:16 -07:00
Daniel Salazar 6f13b1f457 fix(apps): compare normalized index URLs before update checks (#4370) 2026-10-26 22:05:21 -07:00
Daniel Salazar 996d85148e fix(fs): remove app data requested during uninstall (#4369) 2026-10-26 22:05:05 -07:00
Daniel Salazar 09da68424e fix(fs): preserve stored path spelling and app ACL identity (#4368) 2026-10-26 22:04:36 -07:00
Daniel Salazar 7bccde40a8 fix(hosting): use the current authenticated visitor for asset billing (#4364) 2026-10-26 22:04:17 -07:00
Daniel Salazar cf5d2c60a5 perf(permission): resolve default grants through full-access issuers (#4361) 2026-10-26 22:03:45 -07:00
Daniel Salazar b7a0276337 fix(fs): preserve paging and file read-copy contracts (#4367) 2026-10-26 22:03:09 -07:00
Daniel Salazar bbc056cc49 fix(hosting): track trashed roots and cached backing misses (#4363) 2026-10-26 21:58:05 -07:00
Daniel Salazar e2f6c2ed05 fix(cache): widen tombstone window to 5s (#4381) 2026-10-26 21:42:23 -07:00
Daniel Salazar 745b3a4318 perf(auth): coalesce session activity writes across processes (#4360) 2026-10-26 21:37:54 -07:00
Daniel Salazar 2dfd93687f perf(database): defer primary fallback for user lookups (#4359) 2026-10-26 21:37:06 -07:00
Daniel Salazar 91554a137e fix(apps): keep app provenance and permission caches consistent (#4357) 2026-10-26 21:28:40 -07:00
Daniel Salazar 52da7b571c perf(permission): read a large holder's user-to-user grants per permission (#4349)
A user-to-user grant check loaded the holder's whole row set, from one
Redis blob or one SELECT, and filtered it in JS. Every KV share handle a
user receives adds a row, so the cost of each check grew with incoming
shares.

Holders with up to 200 grants keep the whole-set cache. Past that the
cache holds a marker and each check reads only the permissions it names,
off a new (holder_user_id, permission) index. Matches stay exact in JS,
since MySQL compares `permission` case-insensitively. The batched
multi-holder read shares the same chunked query.

Migrations: mysql_mig_51, postgres_mig_40, sqlite 0097.
2026-10-26 20:51:46 -07:00
Nariman Jelveh 9e1ccf7ff3 fix(dashboard): ninth adversarial review of Files — bugs, UX, a11y, perf (#4356)
* fix(dashboard): keep Files keys away from the rows under a publish dialog

With Publish as website or Publish as Worker open and focus on one of its
buttons, the arrow keys moved the listing's selection behind the dialog,
Ctrl/Cmd+A selected every row, and Delete trashed whatever was selected.
The Files shortcuts already stood aside for the share and properties
modals; they now do for the dashboard dialog overlay too.

* fix(dashboard): let type-to-select in Files find names with spaces

Typing "my z" to reach "my zebra.txt" stopped at "my notes.txt": the space
went to the image preview instead of the name being typed, and the search
went on as "myz". A space typed while a name is being typed now belongs to
it; on its own, Space still opens and closes the preview.

* fix(gui): say so when a rename can't reach the server

A rename that failed on the network (offline, connection dropped) put the
old name back without a word, so in Files it looked as if the rename had
been ignored; only failures the server answered with a message were
reported. Every failure is now reported, with the unknown-cause message
when there is no other. The rename's own promise was also left to reject
unhandled on every failure, conflicts included; the error callback already
handles it.

* fix(dashboard): show a grid tile's full name on hover

A tile name cut short to fit shows in full on hover, but in the grid views
the hover never reached it: the badge layer is the size of the whole tile
and sat on top of the name. The name now sits above that layer, so its
tooltip shows; clicking and dragging by the name behave as before.

* fix(dashboard): make the empty-folder notice readable in dark mode

"No files in this directory." and "This folder couldn't be opened." were
styled inline with no color of their own and inherited the page default,
black, which on the dark theme's background could hardly be seen. Both now
share a class with the muted text color the Apps tab's empty state uses,
in either theme.

* fix(dashboard): let a folder be published again once its website is removed

Disassociating a folder's website in Properties took the globe badge off
its row, but the row's entry still said it had a website: its menu kept
Publish as website greyed out, and going into another folder and back
brought the badge back. The entry the menu and the listing read is now
updated with the row, and the sites cache is refreshed, as publishing does.

* fix(dashboard): keep a just-published worker's badge across navigation

Publishing a .js file as a worker badged its row, but the row's entry still
listed no worker. Going into a folder and back redrew the row from that
entry (the folder's cached listing), so the badge went away and the menu
offered Publish as Worker again, which would deploy the file a second time.
The entry now records the worker, as publishing a website records its site.

* fix(dashboard): badge a folder published from inside it once you go up

Publishing the folder on screen (from the listing's background menu) and
going Up showed its row without the globe badge, and with Publish as
website on offer again: the parent's listing came from puter.js's cache,
and adding a website sends no fs event to clear it. Removing a website from
inside the folder (Properties > Disassociate) left the badge behind the
same way. Both now drop the parent folder's cached listing.

* fix(dashboard): don't offer a second website for a published folder

A folder's own row greys out Publish as website once the folder has a
site, but its menu on the sidebar, a breadcrumb or the listing's
background kept offering it, and publishing again gave the folder a second
address. Those menus now check the user's sites as well.

* fix(dashboard): give Files' row menu button a finger-sized target on tablets

On a touch screen with room for the full list columns (an iPad in
landscape, say), a row's '⋯' was a 20x23px target right beside a row whose
tap opens the item, so a near miss opened the file instead of its menu.
On touch it now answers across the row's height and 44px wide; phones'
two-row layout keeps its 44px square, and mouse pointers are unchanged.

* fix(dashboard): keep Files' Back arrow clear of the phone menu button

On phones the breadcrumb row was shorter than the fixed menu button beside
it, so the button hung over the arrows below, and its enlarged tap area
took the top quarter of Back: a tap there opened the menu. The crumb row
is now as tall as the button, which also lines the crumbs up with it, and
on touch the crumbs themselves get a taller tap target.

* fix(dashboard): fit the Files image preview to the image and the listing

The Space preview was a strip the full width of the listing whatever the
image's size (a 64px icon sat in a card 900px wide), and at 3px from each
edge rather than the 20px it was given, its padding coming on top of its
width. In a short listing (a phone in landscape) it ran past the bottom of
the screen and cut off the file name. The card now wraps the image, stays
20px inside the listing both ways, and shrinks the image to keep the name
in view.

* fix(dashboard): let the last grid tile scroll clear of the phone footer

On a phone with a home indicator (a web app on the home screen, or Safari
with its bars collapsed) the Files footer grows by the bottom safe area.
The list makes room for that, but the grid views clear the footer with a
fixed 30px margin, so the bottom 17px or so of the last tile stayed behind
the footer even scrolled to the end. The grids' margin now grows with the
footer; select mode already made room for its bar and is unchanged.

* fix(dashboard): don't claim a large shared selection is restricted

Sharing more than 50 items skips listing who has access, but General
access still said "Restricted: only people with access can open these
items" without having looked, though some of them may be open to anyone
with the link. Choosing Restricted to close them did nothing either, since
it was already the selected option. It now says the link access wasn't
checked, and either choice applies to every item.

* fix(dashboard): don't let "Add to all" lower access on an item it couldn't list

Sharing several items lists who has access to each. When one item's
listing failed, the person's row counted that item as one they couldn't
reach and offered "Add to all", which granted it at the row's mode: someone
who could edit that item was quietly brought down to view. "Add to all"
now extends a grant only to items whose access was listed.

* fix(dashboard): let a click through as soon as a Files dialog closes

The share, properties and publish dialogs fade out for 200ms after they
close, and while they did, their invisible backdrop still took the
pointer: a click on a row right after Escape or Close was lost. The
backdrop now takes clicks only while the dialog is showing.

* fix(dashboard): don't make a removal under way look cancelled on Escape

In the share dialog, Escape on an open "Remove access" confirmation puts
the person's row back. Pressed after Remove was clicked, while the request
was still out, it did the same: the row came back as if the removal had
been called off, and then vanished when it finished. Escape now leaves a
removal that is under way alone.

* fix(dashboard): keep Share down while a share is on its way

Editing the name in the share dialog while a share was still being made
enabled the Share button again, so Enter sent a second share alongside
the first; the first one's end then took the spinner off while the second
was still running, and the two results raced for the status line. The
button now stays down until the share in flight is done.

* fix(gui): find the web address of files inside a published folder again

Files under a folder served as a website carry a link badge and their
address on the site. Working that out read `dir_path` and `address` off
each of the user's sites, which the hosting list no longer returns, so no
file ever had one. In Files the badges publishing just put up were gone at
the next listing. The folder and address are now read from the site's
root_dir, where the list carries them.

* fix(dashboard): show the link badge in the dark list view

In dark mode the list view drew a file's link badge (it is served by a
published website) as its black glyph on the dark row, where it couldn't
be seen. The grid view already gave it a light disc; the list view now
draws the glyph light.

* fix(dashboard): let Undo take back a paste of something cut from its menu

A row menu's Cut puts the item's bare path on the clipboard, where the
Files keys and the selection bar put { path, uid }. Pasting it moved the
item but recorded nothing for Undo (and didn't point Back/Forward at a
moved folder), so Ctrl/Cmd+Z then undid whatever came before, a rename
say. A bare path is now taken as an item like the rest.

* fix(gui): undo one action per Ctrl/Cmd+Z

The Undo shortcut sits in a handler bound to both keydown and keyup, and
ran on each: letting Z go before Ctrl/Cmd, as one usually does, undid a
second action straight after the first. In Files that could put back a
rename or delete an upload the user never meant to touch. It now runs on
the key going down only.

* fix(dashboard): re-sort and re-icon a Files row renamed elsewhere

A file or folder renamed in another tab, on another device or in a desktop
window took its new name in the listing but kept the place its old name
sorted to, and the icon of its old type (notes.txt renamed to notes.png
still showed a text icon). A renamed folder also left Back/Forward on its
old path. Such a row is now handled as a rename made in Files is.

* fix(dashboard): stop failed and cancelled Files uploads throwing uncaught

An upload from Files reports a failure in its progress window, through the
error callback, but the upload's promise also rejected with nobody
listening, so every failed or cancelled upload logged an uncaught error. It
is now handled there.
2026-10-26 17:21:35 -07:00
Filip Kujundžić 8f94514692 feat(ai): add GPT-6.1 Terra to the OpenAI catalog (#4350) 2026-10-26 16:01:44 -07:00
Juan Castro 2ad49e1040 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-26 18:50:14 -04:00
Juan Castro 57349a6982 fix(gui): keep the service worker on servers without push
Unregistering /sw.js wherever pushEnabled is unset ended the browser's
push subscription, so a single page load on such a server stopped pushes
from every server. Turning push off already ends the subscription.
2026-10-26 18:50:12 -04:00
Juan Castro ac0de9df44 feat(gui): notifications settings read a missing push service as unavailable
Without a push extension the /push routes answer core's plain 404. The
GUI maps that to push_unavailable, so the tab says pushes aren't
available on this server instead of reporting a load failure, and a
device removal only counts as done when the device itself is missing.
The push client doc describes the payload sw.js reads and where the
push endpoints come from.
2026-10-26 16:45:28 -04:00
Juan Castro 7d8711bf5b feat(gui): service worker that shows pushes and routes taps
sw.js shows each push as a notification and, on a tap, focuses or opens
a Puter window and hands it the tapped notification. The desktop routes
the tap to the app the notification names, opening it when needed, or
to the notifications panel. The worker is registered only when the
pushEnabled gui param is true.
2026-10-26 16:45:28 -04:00
Juan Castro af13c33678 feat(gui): notifications settings for push devices and apps
Settings -> Notifications turns pushes on and off for this browser
through a push client an extension registers, lists the user's devices
and the apps allowed to push, and switches Puter's own pushes. The tab
shows only when the pushEnabled gui param is true. The sessions window
shares the user agent formatting with it.
2026-10-26 16:45:28 -04:00
Juan Castro 5cf13547f2 feat(homepage): serve the service worker at /sw.js
/sw.js serves the GUI's sw.js, preceded by one importScripts per script
an extension registered with registerServiceWorkerScript. Whether the
GUI registers the worker follows the pushEnabled gui param, which only
an extension sets. Notification reads also carry the row's app_uid, so
the GUI can route a tap to the app a notification is about.
2026-10-26 16:45:28 -04:00
Juan Castro e181a25f95 feat(puter-js): device calls report push_unavailable without a push service
A deployment with no push service has no /push routes, so core answers
the device calls with its catch-all 404. registerDevice, listDevices and
removeDevice turn that into push_unavailable, kept apart from a missing
device, which comes back as device_not_found.
2026-10-26 16:36:46 -04:00
Juan Castro 5e376bc0c8 feat(puter-js): puter.notifications
An app notifies the user it runs for, reads its own notifications back
and follows new ones, hears clicks on them from the desktop, and asks
for or checks the notifications:push grant. Puter's own apps also
register, list and remove the user's devices.
2026-10-26 16:36:42 -04:00
Juan Castro fceaee6372 feat(notifications): node-local notif.written hook
NotificationService emits notif.written after each non-silent write and
after an in-place rewrite, on the node that wrote the row, so an
extension can act on new notifications without a second bus round trip.
It stays off the published event stream; notif.created carries the
notification itself.
2026-10-26 16:34:18 -04:00
Juan Castro 47a6122c37 refactor(permission): keep the notifications:push constant with the permission code
The consent check that reads it lives with the delivery code, outside
core. The grant flow test checks the grant through the permission
service directly.
2026-10-26 16:30:05 -04:00
Juan Castro 9fabd1d5b3 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-26 16:28:52 -04:00
Juan Castro 32b28c1994 Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-26 16:28:48 -04:00
Neal Shah 21625de22e Merge pull request #4315 from HeyPuter/codex/kimi-k3-1-pro
Codex/kimi k3 1 pro
2026-10-26 21:01:04 +01:00
Juan Castro c79b3edf4f Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-26 13:12:24 -04:00
Juan Castro d366861321 Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-26 13:12:23 -04:00
Daniel Salazar f850155411 KV share handles: hardening, higher limits, batch share, uninstall data deletion, account-deletion cleanup (#4344)
* fix(kv): sign share handle ids and drop retired handles from every node's cache

New handle ids carry a short HMAC (kvh-<uuid>.<mac>); an id whose mac
does not verify is refused before any lookup. Unsigned ids keep working
but no longer fall back to the primary on a replica miss.

Retiring or changing a handle now broadcasts outer.pubsub.kv.share.forget
so every node drops its cached row. Event subscribes and delivery
rechecks refuse a handle whose owner is suspended, as the driver does.

* fix(kv): an app its user removed stops reaching shares

An app acting for a user exercises a share grant only while the user
still has the app authorized: at least one user-to-app permission row,
which every app token path leaves and revoke-user-app '*' removes. The
share itself stays; authorizing the app again restores access. The
answer rides the permission cache, keyed by the app-under-user
generation the revoke bumps, so live subscriptions go quiet too.

* feat(kv): list keys through a watch-level share handle

list with as: 'keys' through a handle needs only watch, which already
learns every key written in the region. Values and entries still need
read.

* feat(kv): filter share listings by app and by state

GET /kv/shares and GET /kv/shares/with-me take an optional appUid: an
app uid, or null for the app-less store. An account session may name
any app; an app is refused any namespace but its own. GET /kv/shares
also takes state=live|revoked|all, all by default.

* feat(puter.js): appUid and state options on the KV share listings

listShared() and listSharedByMe() pass appUid (null for the app-less
store), and listSharedByMe() passes state. Docs cover the new options
and that a watch share can list keys without their values.

* feat(kv): indexes and store counts for share-handle quotas and retention

Adds covering indexes for the owner's live count, the owner-holder pair,
app-filtered listings and retired rows, plus store methods to count a
pair's live handles and delete rows retired before a cutoff. The live
handle cache holds up to 50,000 entries; the miss cache stays at 10,000.

* feat(kv): higher share-handle caps, a per-holder cap, and sharing with several recipients

Live share handles per account go to 25,000 paid / 5,000 free, and per
app to 10,000 / 2,000. One owner may hold out 256 live handles to one
holder on every plan, which bounds the rows a revoke or unshare walks.
Re-sharing an existing region takes no slot and is never refused.

POST /kv/shares takes recipients (1 to 50) as an alternative to
recipient: one region, level and label for all. Caps and a new
per-recipient grant budget (300/min, 2,000/hour) are checked for the
whole batch up front; failures about one recipient are reported on its
item. The single-recipient request and response are unchanged and run
through the same code path.

* feat(kv): delete share handles 30 days after they stop

A daily job deletes retired share-handle rows older than 30 days, in
batches of 500. One node runs it per day; it stops on shutdown.

* feat(puter.js): share a KV prefix with several users in one call

puter.kv.share() takes an array of recipients, positionally or as
recipients in the options form, and resolves to one item per recipient:
the share, or an error with its message and code. The single-recipient
forms are unchanged.

* feat(kv): delete an app's data on uninstall, and close account-deletion gaps

Uninstall:
- POST /auth/revoke-user-app takes an optional `deleteData` boolean, only
  with `permission: '*'` (400 otherwise). AuthService.uninstallApp revokes
  every grant, deletes the app's worker sessions, then awaits
  `app.uninstalled` { userId, userUuid, appUid, deleteData }.
- With deleteData, KvShareService retires the handles the user shares out
  of that app's namespace (as the platform, in batches) and then empties
  the namespace in the background, 5,000 items per step, stopping on
  shutdown. Handles shared with the user are not touched.
- SystemKVStore.purgeNamespace(namespace, { maxItems }) deletes up to
  maxItems items of one `v1:<uuid>:<appUid>` namespace, reading keys only,
  with no mutation events or usage, and reports `done` once it is empty.
  It refuses the system namespace and anything else. flush shares its
  page-delete path.
- puter.perms.revokeApp(appUid, '*', { deleteData: true }); the desktop's
  uninstall dialog has an unchecked "Also delete this app's data" option.

Account deletion:
- EventsService settles each of a deleted account's apps on `user.delete`
  (they cascade without `app.changed`) and drops the owner's subscription
  keys.
- The flat permission entries the account held are purged in pages.

Suspension:
- `user.suspended` / `user.unsuspended` events; the share-handle store
  drops cached rows owned by that user here and on other nodes.

* fix(permission): await a peer's flat-permission deletes so emitAndWait sees them applied

* fix(kv): strongly consistent purge pages, and an uninstall purge stops once the app is authorized again

A stale empty page could end a purge as done while items remained. The
background purge after an uninstall now checks before each step that the
user hasn't authorized the app again, so data written after that is kept.

* fix(kv): flush keeps eventually consistent page reads; only a purge reads strongly consistent

* fix(user): emit user.suspended and user.unsuspended when the user store changes an account's suspension
2026-10-26 09:43:55 -07:00
Juan Castro c8998075ea fix(push): let users grant apps notifications:push
The grant route now only accepts permissions from a fixed set of
app-grantable families, which refused notifications:push, so an app could
never be given push consent. Register it next to events:background.
2026-10-26 11:55:37 -04:00
Juan Castro 759f1a798d Merge origin/juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-26 11:30:50 -04:00
Juan Castro b57fef8c9d Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-26 11:29:13 -04:00
Reynaldi Chernando 7c0cf6f6a1 docs(fs): drop batch-write fallback notes from puter.fs.upload() (#4332) 2026-10-26 17:25:23 +07:00
Nariman Jelveh 7f93166560 fix(hosting): disconnect the directory when update gets a null root_dir (#4345)
puter.hosting.update(name, null) sends root_dir: null, which the driver
stringified into a lookup for a folder named "null" and rejected, so
disassociating a website from a folder's properties always failed.
2026-10-25 23:28:45 -07:00
Nariman Jelveh 21846192d1 feat(dashboard): publish websites and workers from Files in dashboard dialogs (#4342)
* feat(dashboard): publish websites and workers from Files in dashboard dialogs

The Files tab's Publish as website and Publish as Worker actions open
dashboard form dialogs instead of desktop windows: the address field shows
the domain inline and shapes the typed name into a valid one, workers get a
Sandboxed switch, and success shows the live address with copy and open.
Published rows get their badges right away.

The form dialog gains suffix and switch fields and a shared copy row, which
the API token dialog now uses too. The save-account / confirm-email gate
before publishing moves to one helper shared by both flows.

* fix(gui): ask a just-saved account to confirm its email before publishing

Saving a temporary account can leave its email unconfirmed when signup
doesn't require confirmation. The publish gate returned right after the
save, skipping the confirm-email step the old inline checks still ran.

* fix(dashboard): link a published site to the address the server reports

The success screen built https://<name>.<hosting domain>, which is wrong on
a self-hosted instance serving sites over http. Use the address the create
response carries, the one the rest of the GUI shows, and fall back to the
built one only when it's missing.

* fix(dashboard): dim a suffixed field as one box while the form submits

Only the input inside it dimmed, leaving the domain after the name at full
contrast while the field was locked.

* fix(dashboard): point a taken or reserved publish name at the address field

The server's reason showed in the status line, but the field stayed
unmarked and focus stayed on Publish, so changing the name took an extra
click. Mark the field invalid and focus it for the rejections that mean
"pick another name".

* fix(dashboard): type into a suffixed field when its suffix is pressed

The domain after the name is drawn inside the field's box but was a plain
span, so pressing it focused nothing. A press there now focuses the field
with the caret at the end, and the suffix shows a text cursor.

* chore(dashboard): drop a phone width rule the icon buttons never used

The form dialog's phone flex rule sizes these buttons, so the width never
applied; touch screens get 44px from the coarse-pointer min-width.

* fix(dashboard): read an underscore in a website address as a word break

A website name can't hold an underscore, so typing My_Cool Site gave
mycool-site. Underscores now become a hyphen, as spaces do; a worker
name keeps them.

* fix(dashboard): keep the letter when an accented one is typed into a publish address

Café became caf: an accented letter isn't a DNS character, so it was
dropped whole. The accent comes off and the letter stays.

* fix(dashboard): cap a website address at a hostname label's 63 characters

The server takes a 64-character subdomain, but DNS caps a label at 63,
so the site it publishes can't be reached. The field now stops at 63,
as the worker one does.

* fix(dashboard): let a publish address finish composing before shaping it

The address was reshaped on every input, composing included: a dead
key's accent turned into a hyphen before its letter arrived, and IME
text was rewritten under the composition. Composed text is now shaped
once it's in.
2026-10-25 21:34:39 -07:00
Nariman JelvehandClaude Opus 5.5 b9f0c8601d fix(dashboard): eighth adversarial review of Files — bugs, UX, a11y, perf (#4330)
* fix(dashboard): don't preview a file named after an image type, or a failed image

Space on a file named "png" (no extension) opened the image preview over a
broken image: the type was read from the last dot-separated part of the
name, which for an extensionless name is the whole name. It's now read from
the extension.

A preview whose image URL couldn't be fetched left the previous image up
under the newly selected row and raised an unhandled rejection. It now
closes instead.

* fix(dashboard): sort Files names by number, and break size/date ties by name

Names were compared as plain text, so "file10" sorted before "file2" and
"dir10" before "dir2". They now compare as a file manager does, with runs
of digits by value.

Sorting by size or date left rows with the same value (every folder, when
sorting by size) in whatever order the server listed them, and a row added
afterwards (a new folder, a rename) landed after its equals instead of where
a fresh listing would put it. Ties now fall back to the name, in the full
sort and in the single-row insert alike.

The sort also parsed each trashed item's metadata on every comparison; it
now reads each entry's sort key once.

* fix(dashboard): ask about "this item" when permanently deleting one Trash item

The Delete key and the phone selection bar's Delete Permanently asked
"Are you sure you want to permanently delete these items?" for a single
item too. One item now gets the single-item question, as from its own
menu. The three copies of the confirm-and-delete step (Delete key,
selection bar, multi-select menu) are now one.

* fix(dashboard): don't let saved column widths squeeze out an auto-width name

Saved widths too wide for the listing fall back to the defaults, but only
when the Name column had a saved width too. Beside an auto-width name, a
Size or Modified column widened on a large screen kept its width on a
smaller one and squeezed the names to nothing, with the row scrolling
sideways. The check now counts an auto-width name at its 100px minimum.

* fix(dashboard): give Files names a phone's full row, not a desktop column width

A narrow listing drops the columns for two-row rows, where each name has
the row's width. Names were still cut to the Name column's saved width
whenever the saved columns happened to fit, so a 120px column set on a
desktop cut names to about 100px on a phone with 280px to spare.

* fix(fs): refuse to replace a folder that contains the item being moved or copied

Moving or copying an item onto a name taken by one of its own parent
folders, with overwrite, removed that folder recursively first, and the
item went with it. Cutting the inner "photos" out of photos/photos, pasting
it next to the outer one and choosing Replace deleted both folders and
everything in them. Such an overwrite is now refused before anything is
removed.

* fix(dashboard): keep the Files selection when an upload lands in another folder

An upload into a folder other than the one on screen (a drop on a folder
row, a sidebar folder or a crumb, or Upload from a folder's menu) still
listed the folder on screen again when it finished. That cleared whatever
was selected there and left an open menu acting on rows no longer on the
page. Only an upload into the folder on screen refreshes it now.

The Upload picker ran its own copy of the upload flow; it now goes through
the same one as drops.

* fix(dashboard): keep Files rows up while a refresh reads web link and app icons

A refresh of the folder on screen keeps its rows until the new listing is
in, so it doesn't flash blank. But it cleared them before reading the
icons, and a web link's or .app's icon is read from the file over the
network, so in such a folder every re-sort, upload or return to the tab
blanked the pane for those reads. The icons are now read before the rows
are swapped.

* fix(dashboard): Ctrl/Cmd-drag a Files selection as a whole

Pressing Ctrl (Cmd on a Mac) on a row that was already selected took it out
of the selection at once, so a drag starting there collapsed the selection
to that one row: a Ctrl-drag of three files to copy them copied one, under
a ghost that still read "3". The row now leaves the selection when the
button is released without a drag, as in a desktop file manager, and a
drag carries the whole selection.

* fix(dashboard): let Enter open the row the arrow keys moved to in Files

Clicking a sidebar folder, crumb or toolbar button leaves it focused, and
the arrow keys and type-to-select still move the row selection from there.
Enter, Space and the menu key then went to that focused control: after
clicking Documents and pressing Down twice, Enter opened Documents again
and dropped the selection instead of opening the highlighted row. The
listing now takes the focus when these keys move the selection, so the
keys after them act on the rows, and Shift+Tab goes back to the toolbar.

* fix(dashboard): let Files' image preview be cancelled while it loads, and close it with the tab

The preview only counted as open once its image URL had come back (two
round-trips), so nothing could cancel it before then: Escape cleared the
selection and the image popped up anyway, a second Space opened it instead
of closing it, and opening a folder left the old image to appear over the
new one. It now counts as open from the start, so each of those cancels
it.

Browser Back/Forward to another dashboard tab left the preview floating
over that tab, where Escape and Space no longer reach it. A route change
away from Files now closes it, as it does Share and Properties.

* fix(dashboard): keep Files keys off a new folder until the server has made it

A new folder's row goes up with the name editor open before mkdir answers.
Clicks and menus already wait on it, but the keyboard didn't: pressing
Escape and then Delete on a slow connection sent a move for the row's
placeholder and showed "Moving New Folder: Subject does not exist", and
Enter opened the predicted path before it existed. Enter, Delete, Ctrl/Cmd+C
and Ctrl/Cmd+X now skip such a row, as do Cut and Delete in the
multi-select menu and the phone's selection bar.

* fix(dashboard): explain a Files action that fails on the network

A request that fails at the network level is rejected with no message.
New Folder then took its row back with no alert at all, and Restore and
the New menu's files showed "[object XMLHttpRequest]". A failed move from
the clipboard named the item and nothing else. Each now falls back to
"An unknown error occurred."

* fix(dashboard): don't offer Trash folders, or view-only shares' crumbs, as Files drop targets

Inside the Trash a folder row lit up as a drop target and then refused the
drop with an untranslated "Cannot move items into a deleted folder.", and a
crumb inside the Trash took the move, though only Restore may take an item
out of the Trash. A crumb of a folder shared with the user to view only
lit up too, and the drop ended in one refusal per item from the server.

Such folders no longer light up, and a drop on one goes back, as a drop on
a view-only folder row already did. A view-only crumb still opens on a
dwell.

* fix(dashboard): count a new folder, not the old selection, in the Files footer

New Folder selects the new row, but the footer was updated before that and
kept describing what was selected before ("3 items selected"), as did the
phone's selection bar, whose buttons by then acted on the new folder. A
Shift+click also still extended from the old anchor. The footer now
follows the new selection, and the new row is the anchor.

* fix(dashboard): update the Files footer when Enter ends a rename

Ending a rename with Enter selects the row, and then called the desktop
Explorer's footer update, which finds no footer in the dashboard. Renaming
a row from its menu while another was selected left the footer at
"1 item selected" with two rows selected.

* fix(dashboard): stop Files' image preview from running out of read URLs

The preview fetched a read URL for every image it showed, and read URLs
share the access-token allowance of 20 an hour. Stepping through a folder
of images with the arrow keys spent it in seconds, after which the preview
(and any app asking for a read URL) failed for the rest of the hour. The
preview now reads the image directly and shows it from memory, freed when
it closes or moves on, and a held arrow key loads only the image it stops
on.

* fix(dashboard): don't offer to publish a folder the user may not share from Files' folder menus

The background, crumb and sidebar menus offered Publish as Website for any
folder, so in a folder shared with the user to view or edit it opened the
publish dialog only for the server to refuse. Publishing takes the right
sharing does; the item now stays disabled without it, as on a folder row's
own menu.

* fix(dashboard): open a Files folder under its new name when it's opened mid-rename

Double-clicking a folder right after typing its new name (the first click
sends the rename) opened it under the old path, which the rename then
took away: the view sat in a folder that no longer existed, with crumbs
and Back/Forward disagreeing about its name, and New Folder or Upload
there failed. Opening a row (double-click, tap, Enter, or Open from its
menu) now waits for a rename still on its way, and goes by one path.

* fix(dashboard): keep Files' Back/Forward in step with folders moved or trashed

Back and Forward kept a folder's old path after it was moved or deleted,
here or from elsewhere: going into a folder, back out, and dragging it
somewhere else (or deleting it) left Forward leading to "This folder
couldn't be opened", with New Folder and Upload gone. History entries now
follow a moved folder and what is inside it, and a trashed or deleted
one's entries are dropped.

* fix(dashboard): keep screen readers from reading Files' icon file names

The crumbs' separators, the folder sidebar's icons and every row's icon
were images with no alt text, so a screen reader announced each one, by
file name in some ("triangle-right.svg"), between every crumb and before
every name. They are decorative beside the text they sit next to and are
now marked so. The crumb of the folder on screen is marked as the current
page.

* fix(dashboard): make double-click on a Files column divider fit the column

Double-clicking a column divider measured the text in system-ui, but the
listing renders in Inter, and took the cells' padding as 8px a side where
Size and Modified have 10. Right after "fit to content", the longest name
was still cut in the middle and every Modified cell ended in an ellipsis
("40 minutes a…"). Each column is now measured in its cells' own font and
padding.

* fix(dashboard): keep a renamed item among its sharer's in Files → Shared

With Shared sorted by "Shared by", renaming an item shared with the user
to edit placed its row by name alone, so it could land among another
person's items until the listing was loaded again. A row put back in
place now goes by its sharer first there, as the listing does.

* fix(dashboard): keep every item, and empty folders' places, in a downloaded zip

Downloading several items as one zip keyed each by its name, so two items
with the same name (in Files → Shared, two people can each share a
"Notes.txt") came out as one file, the other silently gone. Such names
now get " (1)" and so on in the zip, as a folder does for a duplicate.

Empty folders were put at the top of the zip by their own name, not where
they sat: zipping a folder holding x/y gave "y/" next to "x/". They now
keep their place.

dedupedName moves to helpers/, now that the zip uses it as well.

* fix(dashboard): give Files grid tiles both lines of their name

A tile has room for two lines of name, but names were cut to fit one:
"Quarterly financial report final version.pdf" showed as
"Quarterl…l version.pdf" over an empty second line. A name now shows whole
when it wraps into the two lines, and is cut in the middle, keeping its
extension, only as far as it has to be to fit them.

Where the browser wraps a name can't be told from text widths alone, so
the tiles are written and read back together, one layout a round rather
than one a tile.

* fix(dashboard): keep the Files listing when a refresh can't reach the server

A refresh of the folder on screen that failed for any reason replaced the
listing with "This folder couldn't be opened." and took New Folder and
Upload away. Returning to the tab triggers one, so opening a laptop before
its Wi-Fi reconnected, or a moment's lost connection during a re-sort, left
a folder that was fine looking broken until it was opened again. A refresh
that doesn't reach the server now leaves the rows as they are; one the
server answers with an error (the folder is gone) still says so.

* fix(dashboard): don't drop files added to a Files folder while a new folder is made

While New Folder (or New › file) waited for the server, every item added
to the folder on screen was dropped, not only the one being made, so a
file uploaded from another tab or device in that moment didn't show up
until the folder was listed again. Such items are now held until the
create is done and added then; the create's own comes through as an
update of the row already drawn for it.

* perf(dashboard): refresh Files on return to the browser tab only when Files is showing

Coming back to the browser tab listed the Files folder again even with
another dashboard tab in front, and in Files → Shared that meant every
share all over again. The refresh now waits until Files is next shown.

* fix(dashboard): move the keyboard focus off a Files arrow that disables itself

Pressing Enter on Back to reach the first folder disables Back while it has
the focus, and a disabled arrow is drawn at 10% opacity, focus ring
included, so the keyboard user lost track of where they were. The focus
now moves on to an arrow that still works, or to the listing.

* fix(dashboard): show "can't drop here" when dragging files from the computer where Files takes none

Files dragged in from the computer showed the copy pointer everywhere in
Files, then dropping on the Trash, in Shared, on a folder shared to view
only, or while a folder was loading did nothing. The pointer now shows the
drop isn't possible there.

* fix(dashboard): let Files' type-to-select find names with "i" in a Turkish browser

Typed letters were lower-cased for the browser's language and the names
they're matched against without one. In Turkish, "I" lower-cases to "ı",
so typing it never reached "Invoices". Both are now lower-cased alike.

* fix(dashboard): bring Files' Back/Forward along when a folder rename is undone

Renaming a folder points Back and Forward at its new name, but undoing the
rename left them there: Forward then led to "This folder couldn't be
opened" for a name the folder no longer had. Undo now takes them back to
the old name too.

* fix(dashboard): don't shrink Files → Shared's name column when it's dragged wider

In a narrow Shared listing, "Shared by" is laid out narrower beside a name
column with no set width. Dragging the name's divider gave it a width, at
which "Shared by" sprang back to its own and pushed the name narrower than
it started: dragging right made it 30px smaller, and there it stuck.
Sizing the name now keeps "Shared by" at the width it had on screen.

* docs(fs): note that move and copy never overwrite a folder holding the source

* fix(dashboard): keep the Files drop zone after files dragged in cross a folder row

Leaving a folder row while dragging files in from the computer also took
the highlight off the listing around it, as the row's notice bubbled up to
the listing's own. With the "can't drop here" pointer going by that
highlight, a drop on the listing after crossing a folder row was refused.
The row's notices stay with the row, and the listing goes by whether the
folder takes uploads.

* fix(dashboard): don't let a Files folder wait for ever on a rename to open

Opening a row waits for a rename on its way, which settles in the last
step of the rename's handling. A failure after the rename itself (such as
refreshing the folder's websites) never got there, and the folder then
wouldn't open until it was listed again. Opening waits five seconds at
most.

* docs(fs): drop the note on overwriting a folder that holds the source

The refusal is an edge case, and its error message already explains it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-25 18:59:24 -07:00
sabrimete 18f5c9f2c7 fix(ai-chat): admit only chat-capable models from the Alibaba catalog
The feed admission gate now applies uniformly: declared capabilities
must include a chat capability (TG, Reasoning, VU), request modality
must include Text, and response modality must include Text. Signal-less
entries are admitted only when they match a curated model.
2026-10-25 21:07:31 +00:00
sabrimete f0f97a08cf fix(ai-chat): give uncurated Alibaba catalog models a puterId
Uncurated entries admitted from the DashScope catalog feed previously
surfaced with their raw feed id. They now get an
alibaba:<vendor>/<model> puterId, vendor-prefixed ids keep their org,
and others infer the org from the curated family sibling. Entries whose
synthesized puterId collides with a curated entry are skipped so the
model is not listed twice.
2026-10-25 21:04:28 +00:00
Daniel Salazar 24aac2f6d2 fix: app token (#4338) 2026-10-25 11:50:25 -07:00
Daniel Salazar 1eb4e36cf4 fix: add beta flag to kv share docs to match kvhandle beta status (#4337) 2026-10-25 09:17:12 -07:00
Daniel Salazar 3160b99f11 chore: bump ver (#4335) 2026-10-25 08:24:40 -07:00
Daniel Salazar 589a98b12c feat(kv): share KV prefixes with other users at watch/read/write, and bound what apps can be granted (#4331)
* feat(kv): share key-value prefixes with other users, and bound what apps can be granted

- KV share handles move out of Events into KvShareService, served at /kv/shares
  (the /events/kv-handles routes stay as an alias).
- Shares have watch, read and write levels, encoded in the permission root
  (kv-share, kv-share-read, kv-share-write) with write implying read and watch.
- Every puter.kv method takes optConfig.handle; keys are relative to the shared
  prefix, cursors are sealed, and private entries stay hidden.
- New puter.kv.share, unshare, listShared and listSharedByMe; optional label;
  kv.share.received notice in the app's mailbox only; recipient blocks apply.
- puter.kv.set gains onlyIfAbsent.
- fs and kv events gain viaWorker, and kv events report self from the writer.
- /auth/grant-user-app only accepts canonical permissions from a fixed set of
  families; the consent dialog, puter.perms and docs follow.
- Migrations: kv_share_handles gains revoked_by, label, and grantee and app
  indexes (mysql_mig_49, postgres_mig_38, sqlite 0095).

* fix(kv): never name a private key in a cross-app or handle cursor, and bound flat-permission scans

- Listings of another namespace (cross-app or through a handle) resume after
  the last entry they returned, and their cursors are sealed, bound to the
  namespace or handle, and padded to one length. Old plain cross-app cursors
  are still read when they name the target namespace; other namespaces get 400.
- PermissionStore scans flat permissions a few roots at a time and skips
  roots nested under another root in the same call.
2026-10-25 08:16:19 -07:00
Toshit ChawdaandReynaldi Chernando ec3ad1cb2f docs: easy/medium/hard networking recipes (#4040)
* add net category + http request recipes

* add tcp/tls socket recipes

* add complex recipes

* minor refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-25 19:22:45 +07:00
Reynaldi ChernandoandDaniel Salazar c7cc64e45c docs: medium events recipes (PUT-1971) (#4333)
* docs: easy events recipe (PUT-1961)

* docs: catch-up-on-notifications events recipe (PUT-1961)

* docs: events recipe corrections from a run against a live server (PUT-1961)

* docs: medium events recipes (PUT-1971)

* docs: simpler wording in medium events recipes (PUT-1971)

* refine

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-25 19:18:55 +07:00
Nariman Jelveh 0c67499dc2 fix(dashboard): serve stale profile pictures to every caller, skip detached roots
While a picture refreshed after its TTL, only the first caller got the
stale answer; every later one waited on the network, so in Files → Shared
one row showed the picture at once and the rest fell back to initials.
paintAvatars now also folds a frame's roots into one pass and skips roots
re-rendered away before the frame, so they fire no lookups.
2026-10-24 19:05:54 -07:00
Nariman Jelveh 34d74d36bd feat(dashboard): show profile pictures on people avatars (#4328)
* feat(dashboard): show profile pictures on people avatars

Avatars that name a person now show that person's profile picture once
it loads, and keep the initial-and-color avatar when there is none:

- Files → Shared: the "Shared by" cell and the people filter chips.
- Share dialog: the owner row and each person with access. Teams,
  invited emails and link shares keep their initial.
- Properties: a new Owner row with avatar for items someone else owns,
  and avatars beside each version's author.

Lookups go through one cache: each person is looked up once and shared
by every avatar of theirs, answers (including "no picture") are kept
for five minutes, at most 100 are kept, and at most four run at once.
Calls made while a listing renders row by row fold into one pass per
frame. The signed-in user's own picture is read from their loaded
profile, so it takes no request and follows changes in the Account tab.
The picture is set as an image's src property, never through markup.

* fix(dashboard): profile picture review — stale refresh, picker avatars, limiter

- Serve a stale picture while a fresh one loads, so a listing re-rendered
  after the cache expires no longer drops everyone back to initials.
- Show pictures on people in the share recipient suggestions, matching the
  rows of the dialog they open from.
- Free a lookup slot when a lookup throws before returning a promise.
- Move profilePictures.js to helpers/, now that a shared helper uses it.
2026-10-24 19:01:05 -07:00
Nariman Jelveh 0bdc2197b0 feat(dashboard): show who shared each item in Files → Shared (#4327)
* feat(dashboard): show who shared each item in the Files tab's Shared view

The Shared view now names the person behind every item:

- List view gets a sortable, resizable "Shared by" column with the
  share modal's initial-and-color avatar. The tooltip names the owner
  too when someone else re-shared the item.
- Grid tiles show the sharer under the item name.
- Narrow listings lead each row's second line with the sharer.
- A people filter above the listing (one chip per sharer, with counts)
  narrows the view to one person. It shows once two or more people have
  shared something, and scrolls sideways on small screens.

Sorting by "Shared by" applies only in the Shared view; other folders
keep sorting by name while it is the saved sort.

* fix(dashboard): keep the name column readable beside "Shared by"

In a listing between the phone layout and about 700px wide, the fixed
160px "Shared by" column squeezed an auto-width name down to a few
characters. Beside an auto-width name, "Shared by" now takes at most 40%
of the room the two split, and resizing it stops at the same width.

* fix(dashboard): keep the "Shared by" header on one line

Narrowed, its two-word label wrapped onto a second line that spilled out
of the 32px header row. It now stays on one line and ends in an
ellipsis, as longer translations need too.

* fix(dashboard): fit "Shared by" to its longest name on double-click

Auto-fitting the column measured names in a font the cell doesn't use
and left out part of its padding, so the longest name still ended in an
ellipsis. It now reads the font, padding, avatar and gap off a rendered
cell.

* fix(dashboard): keep the people filter on coming back to Shared

Opening a folder from a filtered Shared view and going Back reset the
filter to Everyone, so going through one person's folders meant picking
them again each time. The filter now lasts while the user browses what
was shared with them, and clears once they go back to their own files.

* fix(dashboard): update Shared after Remove from Shared

Remove from Shared only took the row away, leaving the people filter
counting the item. Removing the last item of the person filtered to left
a blank pane under a chip that still claimed one. Shared is now listed
again afterwards, which updates the filter, the empty notice and the
footer.

* fix(dashboard): show a notified share past the people filter

Opening a "shared with you" notification, or a share link, while Shared
was filtered to someone else picked out nothing: the item was filtered
away. The filter now gives way to everyone's when it hides an item the
notification or link names.

* fix(dashboard): truncate a long sharer name on phone-width rows

In the two-line row layout, a long username grew the details line past
the row, pushing the date off screen and under the ⋯ button. The name
now ends in an ellipsis, and the line keeps clear of the button as the
name line does.

* perf(dashboard): sort the Shared view by sharer with one collator

Each sharer comparison called localeCompare with options, which builds
a collator per call: sorting 2,000 rows took about 25ms in Chrome. One
shared Intl.Collator orders them the same in under 1ms.

* fix(dashboard): make share avatar initials readable for every name

The initial-and-color avatar drew its letter at too little contrast for
nearly half the hues a username can map to: yellows and greens came out
near 3:1 in light mode, and lower still on a selected row. The letter is
now darker in light mode and lighter in dark mode, reaching 4.5:1 for
every hue on the page, selected rows and the share suggestions' hover
row. The share dialog and the Shared listing change together, so a
person keeps one look.

* fix(dashboard): show every folder when a drag springs into Shared

Dragging an item out of a shared folder and holding it over Up, Back or
the Shared crumb opens Shared to drop into a folder there. The people
filter, which now lasts while browsing shared items, could hide those
folders, and its chips can't be pressed mid-drag. A drag arriving in
Shared now sees everyone's folders.

* fix(dashboard): keep filtering Shared when it can't be listed again

A chip click lists Shared again so its rows are current. When that
request failed, the listing was replaced with "This folder couldn't be
opened" and the chips disappeared, though only a filter had changed. A
filter change now applies to the rows already loaded when the listing
can't be fetched.

* fix(dashboard): return a drag let go over Shared to where it began

A drag that springs into Shared and is let go over the listing rather
than a folder tried to move the items into Shared itself, which is a
query and not a folder: nothing moved, nothing said so, and the user was
left in Shared. Let go over a listing that takes no items, the drag now
goes back to the folder it started in, as the cancel zone does.

* fix(dashboard): don't offer read-only shared folders as drop targets

Folders shared read-only lit up under a drag and took the drop, which
the server then refused with "Forbidden". They no longer light up, and
a drop on one sends the drag back where it began; they still open on a
dwell.

* fix(dashboard): slide a drag back when a read-only folder refuses it

A drop a read-only shared folder refused left the drag chip to vanish
where it was let go, as if something had happened. It now slides back
to where the drag began, as any drop that doesn't land does.
2026-10-24 18:10:27 -07:00
Nariman Jelveh 7759f7afe9 fix(dashboard): seventh adversarial review — bugs, UX, a11y, perf (#4326)
* fix(dashboard): show notification times in Brazilian Portuguese, Ukrainian and Traditional Chinese

The notification panel and toasts handed Puter's language code straight to
Intl, and a few of those codes aren't BCP 47 tags. Brazilian Portuguese
(`br`) is Breton to Intl, so times read "5 munut zo"; Ukrainian (`ua`)
fell back to the browser's default language; and `zhtw` threw, so Taiwan
users got English. Those codes are now mapped to pt-BR, uk and zh-TW
before formatting.

* fix(dashboard): count "yesterday" and "N days ago" in calendar days

Notification times rounded the elapsed hours to whole days, so at 23:30
something from 00:45 the same day read "yesterday", and on Wednesday
morning something from Monday night (34 hours earlier) read "yesterday"
too. Days are now counted from midnight to midnight in the reader's time
zone; anything from today stays in hours.

* fix(dashboard): don't let deploy results crowd notifications out of the panel

The panel listed the 30 newest notification rows and then dropped the
ones it never shows, a worker's deploy results among them. A developer
with 30 deploys since their last share saw "No notifications" and no
badge while the share sat behind them. The listing now asks for up to
200 rows and keeps the newest 30 the panel shows.

* fix(dashboard): keep every unread notification in the panel

The socket replays up to 200 unread notifications on connect, but the
panel's listing held only the 30 newest, and the first refresh after that
dropped the rest: 45 unread read "45" until the panel was opened, then
"30", and "Mark all as read" left the other 15 unread on the server for
the next visit. The panel now keeps unread notifications past the newest
30, up to the 200 a listing returns.

* fix(dashboard): let Undo take back a new Files folder or file

The Files tab creates folders and New › files itself rather than through
the desktop's helpers, and never recorded them for Undo. Undo then
reversed whatever came before: upload notes.txt, create a folder, press
Ctrl+Z, and notes.txt was deleted for good (undoing an upload deletes it
outright, not to Trash) while the new folder stayed. Both now record
themselves, so Undo removes what was just created, as on the desktop.

* fix(dashboard): let Undo take back a Files cut and paste

Pasting cut items in the Files tab moves them itself rather than through
the desktop's move_items, and recorded nothing for Undo. Undo after a
paste reversed the action before it instead, which deletes an upload for
good. The paste now records the move as move_items does, so Undo puts
the items back where they were cut from.

* fix(dashboard): don't drop a Files selection onto one of its own folders

With folders A and B selected, dragging A onto B moved A into B and then
stopped on "Cannot move a directory into itself" for B. Hovering B also
sprang it open mid-drag. A folder row that is part of the drag is no
longer a drop target, so the drag reverts as it does over any other
non-target. A drop elsewhere (a sidebar folder, say) also leaves out the
target folder itself and any folder that holds it.

* fix(dashboard): don't drop a Files refresh asked for during another

A refresh of the folder on screen was ignored while another refresh of
it was loading. Drop two files: when the second finished during the
first's refresh, its own refresh was dropped, the earlier listing (read
before the file existed) was drawn, and the file's item.added was
ignored meanwhile too, so it stayed missing until a manual refresh.
Paste, Refresh and returning to the browser tab could lose items the
same way. A refresh now takes over from one in flight, as opening
another folder already did; a navigation in progress still holds.

* fix(dashboard): keep the phone selection bar after cancelling a delete

In select mode on a phone, tapping Delete permanently and then Cancel
left every item selected but hid the floating action bar, the only way
to act on the selection, until the selection next changed. The same
happened when nothing in the selection could be moved to Trash. The bar
now goes away only once something is actually deleted or trashed.

* fix(dashboard): show the phone selection bar when Select starts with a selection

Uploaded or pasted items arrive selected. Tapping Select on a phone then
turned select mode on with them selected but no floating action bar,
until a row was toggled. Entering select mode now brings the bar up for
the selection it starts with.

* fix(dashboard): give Files footer sizes in the rows' format

The footer used the desktop's byte formatter while the rows use their
own, so a 5-byte file read "5 B" in its row and "5.00 Bytes" below it,
1.5 KB read "1.50 KB", and a folder of folders read "0 Byte". The footer
now formats sizes as the rows do.

* fix(dashboard): title a trashed folder's Properties with its name

Properties from the breadcrumb of a folder opened inside Trash was
titled with the folder's uid, the last segment of its trashed path,
while the crumb itself showed the original name. It now uses that name.

* fix(dashboard): show delete and log-out confirmations without entities

The Files tab's "delete these items permanently?" confirmation and the
dashboard's "close your apps and log out?" one passed HTML-encoded i18n
strings to UIAlert, which encodes its message and button labels again.
In Igbo both questions read "Ị ji n&#039;aka …". These alerts, and the
Files tab's name-conflict and folder-depth ones, now pass plain strings.

* fix(gui): read the Trash's state fresh when updating its icon

refresh_trash_state stats the Trash to pick the empty or full icon, and
its comment calls for a strong read, but it asked for an eventual one.
Once a listing of Home had cached the Trash entry (without `is_empty`),
the stat came back without it: permanently deleting the last items from
Trash in Files left the full-Trash icon up, and other tabs were told the
same. The stat is now strong, as intended.

* fix(dashboard): don't add a Files row to a folder opened meanwhile

A file added from elsewhere (an upload, another tab, an app) is checked
against the folder on screen and then drawn once its icon resolves. A web
link's or app's icon is read from the file, and opening another folder
during that read put the row into the new folder's listing, where it
stayed and opened the file from the old one. The checks now run again
once the icon is in.

* fix(dashboard): let a focused folder or crumb take the Menu key

The Files tab answered the Menu key and Shift+F10 with the selection's
menu wherever focus was. With a file selected and a sidebar folder, a
breadcrumb or Home's Shared row focused, the key opened the file's menu;
with nothing selected it cancelled the browser's contextmenu, so the
focused element opened none. A focused control now gets the browser's
contextmenu event, and its own menu, as with a right-click.

* fix(dashboard): don't drop New Folder during a Files refresh

New Folder did nothing while the folder on screen was refreshing (after
an upload or paste, or on returning to the browser tab). A refresh keeps
the rows up with no spinner, so the click just looked ignored. It now
takes over the refresh and makes the folder once the listing is in;
during a navigation to another folder it is still dropped, as before.

* fix(dashboard): step through Files names on a repeated letter

Type-to-select meant to move past the selected row to the next name with
the same start, but returned early whenever the selected row matched, so
the stepping never ran: with readme, report-a and report-b, pressing "r"
again stayed on readme, and "rr" matched nothing. A repeated letter now
steps through the names it starts, wrapping around, as file managers do;
a longer prefix still stays on a selected row it matches.

* fix(dashboard): open one touch menu per long-press

A long-press on Android fires both contextmenu and taphold. The Files
rows already ignore the second, but a breadcrumb, a sidebar folder,
Home's Shared and Trash rows and the listing's empty space each opened a
second sheet over the first; picking an item or tapping the backdrop
then left the other one up. The touch menu now doesn't open while
another is on screen.

* fix(dashboard): treat Ctrl+click on a Mac as a right-click in Files

On a Mac, Ctrl+click opens the context menu, but the Files rows read it
as a Ctrl-toggle first: Ctrl+clicking one of three selected files
deselected it and the menu then acted on that file alone, and on an
unselected file it was added to the selection. On a Mac it now leaves
the selection alone, as a right-click does; Cmd+click still toggles.

* fix(dashboard): keep arrow keys in the open phone drawer on Apps

With the drawer open over the Apps tab, an arrow key moved focus out of
it to the first app tile behind the scrim, and the next Tab then walked
the page behind it. The open drawer now keeps the arrow keys, as the
notifications panel and dialogs already do.

* fix(dashboard): act once on a held Enter or Space in Apps

Holding Enter on an Apps folder opened it, moved focus to its first app,
and the key's repeats then launched that app; held on the "+" tile, the
repeats picked "Browse the App Center" in the dialog it opened, and held
in the search box they launched the first match again and again. Key
repeats now do nothing on a tile, in the search box or on the add-app
dialog's options.

* fix(dashboard): don't blank the Apps grid when it re-flows on resize

When a resize changed how many columns or rows fit (resizing the window,
collapsing the sidebar, zooming, an Android keyboard), the grid was
rebuilt behind the first load's fade, which waits on page one's icons.
With an icon host that was slow or unreachable the grid went blank for
up to 1.5s each time. A grid already on screen now re-flows at once;
one still loading keeps its fade.

* fix(dashboard): open an app from the add-app dialog at once during a search

"Browse the App Center" and "Build with AI" play an intro from the
app's tile, and wait up to 3s for that tile to appear. With a search
that leaves the app out (the case the "+" tile stays for: a search that
found nothing), the tile never appears, so the app opened only after the
full 3s. The intro now gives up as soon as the search is what hides the
tile, and the app opens straight away.

* fix(dashboard): make the Apps search's clear control a button

The "×" that clears the Apps search was a bare SVG with a click handler:
Tab couldn't reach it, screen readers had nothing to announce, and on a
phone it was a 16px target inside the field. It is now a labelled button
after the field in the Tab order, with a focus ring, and on touch it
hit-tests at 44x44 while keeping its look.

* fix(dashboard): tell screen readers when an Apps search finds nothing

Typing in the Apps search re-renders the grid silently, and the "No apps
match your search" line is drawn into it fresh each time, which screen
readers don't announce. A search that matched nothing gave no feedback
at all. A persistent status region now says so.

* fix(dashboard): keep keyboard focus on an Apps tile across a re-render

The grid is rebuilt wholesale whenever its data or size changes: the
refresh that follows opening the tab, a resize or zoom that changes how
many columns fit, an uninstall that fails. A keyboard user who had
arrowed onto a tile lost focus to <body>, so Enter did nothing and the
next arrow started over from the first tile. Focus now moves to the
same app's or folder's new tile.

* fix(dashboard): don't offer Change username to a temporary account

A temporary account's Account tab offered Change username, whose dialog
asks for an account password the account doesn't have: left empty it
said "All fields are required", and anything typed got a 403 from the
server, which refuses temporary accounts. The button is now left out
until the account is saved, as the Password card already is; saving it
re-renders the tab with the button.

* fix(dashboard): don't offer Change email to a team seat

A team seat's Account tab offered Change email, and after the user
filled in the new address and their password the server refused it:
"Your team set this address. Ask a team admin to change it." The Email
card now shows that note in place of the button, as the Username card
already does for seats.

* fix(dashboard): don't let a slow picture removal undo a newer picture

Saving a profile picture is ordered so an older save can't put its
picture back over a newer one, but Remove picture stood outside that
order. Clicking Remove on a slow connection and then saving a new photo
before the removal answered left the default avatar on screen, and
Remove hidden, while the account had the new photo. A removal now
yields to a picture saved after it.

* fix(dashboard): show placeholders in the Usage headers until they load

The Storage and Resources headers started out as " ·  used of " with the
numbers left blank, and stayed that way when reading the storage space
or the month's usage failed. They now start at "--% · -- used of --",
as Home's cards start at "--", and keep that if the read fails.

* fix(dashboard): write Home's trial dates in the dashboard's language

The free-trial note on Home formats its dates with the browser's
language, not the one chosen in Puter, so with Puter in French and the
browser in English it read "…le Mar 5", and the other way round "First
charge on 5 mars". The dates now follow the dashboard's language. The
notification panel's mapping of Puter's language codes to Intl tags
(Brazilian Portuguese, Ukrainian, Traditional Chinese) moves to
i18n/intlLocale.js so both use it.

* fix(dashboard): write credit and dollar amounts in the dashboard's language

Credits were always formatted with English separators and dollar amounts
with "$" and English separators, so in German Home read "5,000
monatliche Credits", which a German reader takes as five, and Usage
showed "2.5" and "$1,234.50". Both now use the dashboard's language
("5.000", "2,5", "1.234,50 $"); English is unchanged.

* fix(dashboard): say why Create token is off where everyone can read it

When the account's email isn't confirmed, Create token is disabled and
the reason ("Verify your email address to create an API token.") lived
only in a hover tooltip: a disabled button isn't reachable by Tab, a
phone has no hover, and screen readers don't read a plain span's title,
so those users met a dead button with no explanation. The reason now
replaces the card's description while the button is off. The tooltip,
which encoded the already-encoded text again, now uses it as is.

* fix(dashboard): announce that a new API token is shown only once

After a token was created, focus moved to Done and a screen reader read
"Done, button" and nothing else: the done state arrives already filled
in a status region, which isn't announced, and Done is described by the
message only for plain-text results. The token dialog's warning ("Copy
this token now — it won't be shown again…") now describes Done, so it
is read out when focus lands there.

* fix(dashboard): show an email change confirmed in another tab

Confirming a change of email from its link sets the new address and
marks it confirmed, but sends no socket event, unlike confirming a code.
Back in the dashboard the user was re-read on returning to the browser
tab, yet nothing redrew: Account kept the old address under the
username, Create token stayed disabled and Security had no 2FA card
until a reload. Account and Security now redraw after that re-read when
the address or its confirmation changed.

* fix(dashboard): close the share dialog after leaving a share

Someone the owner let share could remove their own row in the share
dialog to leave. The removal went through, but the dialog then reloaded
its access list with the access just given up: the server refused it,
the success line gave way to "Subject does not exist" over an empty
list, and the item stayed in the Shared view until a reload. Leaving
now closes the dialog and refreshes the folder on screen, which drops
the item, as "Remove from Shared" does.

* fix(dashboard): say the access list couldn't load, not that sharing failed

When the share dialog couldn't read who has access (a network error,
which carries no message of its own), it said "Could not share this
item." though nothing had been shared or attempted. It now says "Could
not load who has access."

* fix(dashboard): don't pull focus back after a share change saves

When a change in the share dialog finished saving (a level change, Add
to all, a removal, a share), focus was put back on that row's control or
the add field, wherever the user had gone meanwhile. Someone who changed
a person's level and then started typing a name had the next keystrokes
land on that person's level select, where a matching letter picked
another level and granted it. Focus now goes back only when the save
took it away; focus the user moved elsewhere stays there.

* refactor(gui): move the localized time formatting to i18n/formatTime.js

formatRelativeTime and formatAbsoluteTime lived in the notification
center's module; Files and Properties are about to use them too.

* fix(dashboard): write Files and Properties dates in the dashboard's language

The Files tab's Modified column and Properties' Modified, Created and
version times came from timeago, which has no language but English
registered, so a German dashboard read "3 hours ago" beside "Geändert".
They now use the same localized formatter as the notification panel
("vor 3 Stunden"), which also gives a date past a week rather than
"3 weeks ago". Its formatters are now made once per language, since a
listing formats one date per row.

* fix(dashboard): style the Usage table's Show less button in dark mode

The button's dark variant was written for an `html.dark-mode` class that
nothing sets, so in dark mode it kept its light look: grey text at 3.4:1
on the dark card, a light border, and a light chip on hover. Its rules
move under prefers-color-scheme, as the table's fade overlay already
did.

* fix(dashboard): show the Files breadcrumb separators in dark mode

The chevrons between breadcrumbs are a black stroke at 20% opacity,
which vanishes on the dark header: a nested folder read "Home Documents
Work" with nothing between the names. In dark mode they are now
inverted, like the toolbar's icons.

* fix(dashboard): show the website-link badge on dark Files tiles

In grid view a file inside a published folder carries a link badge: a
black glyph on a disc of the page's background color. In dark mode that
is black on near-black and can't be seen. The badge now gets the white
disc the shortcut and worker badges already have.

* fix(dashboard): skip disabled Back, Forward and Up when tabbing in Files

The Files header's arrows stayed in the Tab order while disabled, drawn
at 10% opacity, which takes their focus ring with them. Tabbing into
Files, the first two stops (Back and Forward, disabled on arrival)
looked like nothing had focus. A disabled arrow now leaves the Tab
order, as a disabled button does, and returns once it can be used.

* fix(dashboard): give the Delete account button an AA-contrast red

The Account tab's Delete account button used the desktop's danger red
(#f00808), which puts its white label at 4.4:1, under the 4.5:1 that
WCAG AA asks of text. It now uses the dashboard dialogs' danger red
(#d90707, 5.3:1) and their darker hover, so the two match.

* fix(dashboard): label collapsed sidebar items for keyboard users too

With the sidebar collapsed, each item shows only its icon, and its name
appeared in a tooltip on mouse hover alone. Tabbing through the sidebar
showed a row of icons with no names. The tooltip now also shows while
an item has keyboard focus.

* fix(dashboard): keep Share and Properties inside the visible screen

Both dialogs capped their height at 85vh (88vh as phone sheets), and vh
is the viewport with the browser's bars hidden. With the bars showing,
on a phone in landscape say, the card plus the overlay's padding was
taller than the screen, and the overlay centers without scrolling: the
title and the buttons were cut off with no way to reach them. Each card
is now also held within its overlay, as the dashboard's dialogs are,
and its body scrolls instead.

* fix(dashboard): outline the off state of the 2FA switch

Off, the switch was a light grey track (#e0e0e0) with a white thumb,
1.3:1 against a white card and 1.2:1 against the amber card the 2FA
toggle sits on while 2FA is off, so the control could barely be seen
where it matters most. WCAG asks 3:1 for a control's boundary. The off
track now carries a 1px slate outline (4.3:1 on the amber card); the
on state is unchanged.

* fix(dashboard): make the keyboard focus ring visible on sidebar, tiles and lists

The sidebar items, the Files folders, crumbs and arrows, the app tiles,
Home's cards and recent apps, the notification rows and the Usage sort
buttons show keyboard focus with a 2px outline and nothing else, drawn
in --select-ring: the selection color at 40% (50% dark), about 1.6:1
against the surfaces it sits on. Keyboard users could hardly tell where
focus was; a focus indicator needs 3:1. Those outlines now use the solid
selection color (3.7:1 on white), as the switch and pager dots already
do. Field halos, which come with a colored border, are unchanged.

* fix(dashboard): bring text in the selection blue up to AA contrast

The selection blue is 3.7:1 against white, and a few places set text in
it or white text on it, under the 4.5:1 that text needs: the active
folder in the Files sidebar, the share dialog's "Add to all" link, the
add-app dialog's Send button, the notification count badge, the drag
count badge, and a breadcrumb whose menu is open. They now use the same
hue darkened to 45% (5.3:1; a lighter one for text in dark mode), from
two new tokens, --select-text and --select-fill.

* fix(dashboard): give the Apps page dots a 24px touch target

The Apps pager's dots, and a folder's, are 17px buttons side by side,
under the 24px a target needs when it has neighbors, so on a phone or
tablet a tap meant for one page often opened the next. On a touch
screen each dot is now 24px around the same 7px dot; mouse layouts are
unchanged.

* fix(dashboard): return from a drag-opened folder while it is still loading

Hovering a dragged file over a folder springs it open. Cancelling the
drag (Escape, the Cancel zone, or a drop outside the listing) is meant
to go back where the drag started, but while the sprung folder was still
loading the return was dropped like any navigation during another, and
the user was left in the folder they had backed out of. The return now
takes over from that load.

* fix(dashboard): drop into a drag-opened folder while it is still loading

When a dragged file springs a folder open, the drop anywhere in the
listing moves it into the folder now shown. Until that folder's listing
arrived, though, the Files tab still counted the folder the drag left as
the one on screen, so a drop in that time was taken as a move to where
the file already was and did nothing. It now goes to the folder being
opened.

* fix(dashboard): say "Incorrect password" when the account password is wrong

The account dialogs showed the server's message as is. A wrong current
password in Change password read "Password mismatch", in English in
every language, beside a dialog that says "New Password and Confirm New
Password do not match" when those two differ, so users corrected the
wrong fields. A wrong code read "Incorrect code." and too many attempts
"Too many requests.", also always in English. These now show the
translated messages the GUI already has, and a wrong password puts the
status on the password field, focused for retyping.

* fix(dashboard): don't send an account change to the value it already has

Change username with the account's own name got "This username is
already taken." and used up one of the hour's password attempts. Change
email with the current address was accepted: it mailed a confirmation
and a change notice to the same inbox, and following the link unlinked
any sign-in provider. Change password to the current password signed
out every other session and mailed a "password changed" notice. Each
dialog now says the value is already the account's and sends nothing.

* fix(dashboard): ask for the new password after a provider-only account sets one

An account that signs in only through a linked provider can set a
password with Change password, after which the server treats it as a
password account. This session's copy still said provider-only, so
Change username, Change email and Delete account went on hiding the
password field and sending the user through the provider's popup until
a reload. The copy now changes with the password.

* fix(dashboard): let Escape cancel a session rename without closing Manage sessions

In Manage sessions (Security), Escape in a session's rename field
cancelled the rename and then closed the whole modal, and Escape in the
search box closed it too, search and all. Escape now cancels the rename
or empties the search, and only an Escape with neither closes the
modal.

* fix(dashboard): show the server's message when a session can't be revoked

Manage sessions put a refused request's raw response body in its alert,
so revoking a session already signed out from another tab read
{"message":"Session not found","code":"not_found"}, and the stale row
stayed listed; a failed rename and "Revoke all other sessions" did the
same. The alerts now show the message itself, and a session found gone
reloads the list.

* fix(dashboard): show 2FA as it is when its dialog closes without success

The Security tab's 2FA toggle assumed nothing had changed whenever its
dialog closed without success. With 2FA turned on in another tab, the
setup here refused ("2FA is already enabled."), the toggle went back to
off, and every retry got the same refusal, with Disable never offered;
a code accepted just as the user closed the setup dialog left the same
state. The tab now re-reads the account and shows its real 2FA state.

* fix(gui): leave Ctrl/Cmd+Z to the text field being typed in

Ctrl/Cmd+Z ran the file Undo wherever focus was, and cancelled the
field's own text undo. Typing a new name (F2) or anything in another
field and pressing it undid the last file action instead; when that was
an upload, the uploaded file was deleted for good. Undo now leaves text
fields alone.

* fix(gui): let Escape on a context menu close only the menu

Escape removed any open context menu and then went on to the Files tab's
own Escape handling, which, finding no menu left, cleared the selection
and closed the image preview. Select three files, open their menu,
press Escape: the menu closed and the selection was gone. When Escape
closes a menu, nothing else now acts on it; a second Escape clears the
selection as before.

* fix(dashboard): close the Files image preview when an app opens over it

The image preview sits above every window, and only a double-click
closed it on the way to an app. Opening the image with Enter, or with
Open or Open With from its menu, started the app underneath the
preview, which went on covering it while keyboard focus moved into the
app, so Space and Escape no longer reached the preview. It now closes
whenever an app window comes on screen.

* fix(dashboard): close the Files image preview when its file goes away

Deleting the previewed image (the Delete key, the menu, the phone
selection bar), moving it, or having it removed from another tab took
its row out of the listing but left the preview up, still showing the
image. The preview now closes with its row; a refresh of the folder,
which redraws the row, keeps it.

* fix(dashboard): download one selected file as itself from the phone selection bar

In select mode, the floating bar's Download zipped whatever was
selected, so a single photo arrived as "photo.jpg.zip" where its own
menu's Download gives "photo.jpg". One selected file now downloads as
itself; folders and several items are still zipped.

* fix(dashboard): don't offer New Folder, Upload or Paste in a view-only share

Inside a folder shared with the user to view only, the Files tab still
showed New Folder and Upload, offered New and Paste in its menu, and
took files dropped on the listing or on a folder in it; each then
failed: an upload window ending in an error, or a new folder row with
its name field open, then taken back with a Forbidden alert. Those are
now left out there, as in Trash. A share whose level isn't known yet
stays as before, the server deciding.

* fix(dashboard): move through Files with Home, End, Page Up and Page Down

The Files listing took the arrow keys but not Home, End, Page Up or Page
Down, and as the listing itself never has focus the browser didn't
scroll it on them either: in a folder of hundreds of items the keyboard
could only step through them one row at a time. Home and End now select
the first and last item, Page Up and Page Down move a screenful, and
Shift extends the selection as with the arrows. With a menu open they
are left to it, as the arrows are.

* fix(dashboard): apply the hidden-files setting to a renamed Files row

With Show hidden off, renaming notes.txt to .notes.txt left the row in
the listing until the folder was next loaded, and with it on the row
wasn't dimmed as other dot-files are; renaming .env to env left it
dimmed. A rename made in another tab did the same. A renamed row now
follows the setting as listing the folder would.

* fix(dashboard): open a recent app once on a double-click

Double-clicking an app under Home's "Recently used" opened it in two new
browser tabs, one per click. The second click of a double-click is now
ignored, as on the Apps tab's tiles.

* fix(dashboard): don't say "No recent apps yet" before Home's list has loaded

Home's "Recently used" card was drawn from the list as it stood when the
page opened, which is empty until the request answers: every visit
first read "No recent apps yet / Apps you use will appear here" and
then swapped in the apps, and if the request failed the card kept
saying there were none. It now stays blank until the list arrives, and
a first load that fails says the apps couldn't be loaded; a list
already shown stays when a later refresh fails.

* fix(dashboard): drop "Save session" when the account was saved in another tab

A temporary account saved in one browser tab stayed temporary-looking
in its other dashboard tabs: Home kept its "Save session" button, which
opened Save Account for an account already saved, and Account and
Security kept their temporary-account layout. Coming back to such a tab
re-reads the account; that now removes the button and redraws the two
tabs when the account is no longer temporary.

* fix(dashboard): let go of the phone drawer when the window widens

With the sidebar open as a drawer below 768px, widening the window past
it (an iPad turned to landscape) drew the sidebar as the desktop one but
left it marked open: Tab stayed trapped in it, so the content couldn't
be reached by keyboard, and the Apps grid ignored the arrow keys, until
an Escape or a click in the content. The drawer now closes when the
window widens, leaving focus where it is.

* fix(dashboard): don't let a #files/<path> link pull Files back to its folder

Files doesn't keep the folder on screen in the URL, but a link like
/dashboard#files/alice/Documents stayed in the address bar once opened.
After the user moved on to Pictures, went to Home and pressed Back, the
browser returned to that link and Files jumped back to Documents; a
reload did the same, and Files' own Back/Forward lost track, as the jump
wasn't recorded in it. The link's path is now consumed once opened, as
a share link's is, and a folder opened from such a link is a step in
Files' history.

* fix(dashboard): close the drawer and Files' dialogs on browser Back and Forward

Browser Back or Forward switched tabs but left whatever was open on
screen: on a phone, Android's Back changed the tab under the open menu
drawer and left it open; in Files, the Share or Properties dialog
stayed up over Home, still acting on Files rows; an open context menu
stayed too. These now close with the tab, as with a sidebar click.

* fix(dashboard): fill Home's storage and usage meters without waiting on the plan

Home read the plan first and only then asked for storage and the month's
usage, so every visit paid for one more round-trip before either meter
filled in. The plan request had no timeout either: one that never
answered kept both meters at "--" and, as later loads wait for the one
in flight, held every later refresh of the cards for the rest of the
session. The three now load side by side, and the plan read gives up
after 15 seconds.

* fix(dashboard): link Home's card headers to their tabs

The headers of Home's cards (Recently used, Usage, Your Plan, Your
Storage, Resources) linked to "#", so opening one in a new browser tab
(middle-click, the browser's menu) showed Apps rather than the tab it
names, and Cmd/Ctrl+click switched tabs in place. They now link to the
tab they open, and a modified click is left to the browser, as on the
sidebar.

* fix(dashboard): prefill the username when saving a session from Home

Home's "Save session" button opened Save Account with an empty username,
so a temporary user had to make one up, while the same action in the
user menu fills in the account's current name and starts at the email
field. Both now open it the same way.

* fix(dashboard): measure Home's storage meter by the amount it shows

On a self-hosted install the storage card names the disk's use (Puter's
files and everything else on it) but filled its meter, and gave its
percentage, from Puter's files alone, so it could read "93 GB used ·
0% of 466 GB". The meter and percentage now come from the same amount
the card shows.

* fix(dashboard): refresh Home's recently used apps on returning to the tab

Recent apps open in a new browser tab, and coming back to the dashboard
refreshed the usage cards but not "Recently used", whose comment said
it followed the same triggers: the app just opened only moved up the
list after switching away from Home and back. Returning to the tab now
re-reads the list too.

* fix(dashboard): leave IME keystrokes to the IME in Apps

Typing Japanese or Chinese in the Apps search or a folder's name, the
keys that drive the input method reached the tab's own handlers:
ArrowDown to pick a candidate moved focus to the first tile and cut the
conversion short, Enter to confirm one named the folder half-written or
launched the first match, and Escape put the old name back. The search
box's Enter meant to skip composition, but read isComposing off jQuery's
event, which never carries it. Keys during composition are now left to
the IME.

* fix(dashboard): leave IME keystrokes to the IME in the share recipient field

With a Japanese or Chinese input method on, even a Latin username is
composed, and the share dialog's recipient field took the keys that
drive it: the arrows moved through its suggestions instead of the IME's
candidates, and Enter to confirm the text picked a suggestion instead.
Keys during composition are now left to the IME.

* fix(dashboard): re-read the Apps layout on returning to its browser tab

The Apps tab read its order and folders only when it was opened, and
saves write its whole copy of them. With the dashboard open in two
browser tabs, a folder made in one was wiped as soon as the other,
showing Apps since before, changed anything (a reorder, a new folder, a
rename), as was the order arranged in the first. The tab now reads the
layout again when its browser tab comes back into view.

* fix(dashboard): undo a drag out of an Apps folder when the drag is cancelled

Dragging an app past the edge of an open folder takes it out, closes the
folder and carries the drag on over the grid, and the removal was saved
right then. Pressing Escape, or the window losing focus, cancels the
drag but left the app out; out of a two-app folder that dissolved the
folder for good, its name with it (dragging the app back made a new
"Folder"). A cancelled drag now puts the folder and the order back.

* fix(gui): show the 2FA setup's confirmations without entities

The 2FA setup window, which the dashboard's Security tab opens, encoded
already-encoded translations for its labels, so in French the two
checkboxes before enabling read "J&apos;ai enregistré mes codes…" and
"Je suis prêt à activer l&apos;A2F". The labels, and the window's other
translated text set the same way, are now encoded once.

* fix(gui): don't leave the 2FA dialogs stuck when a request can't be sent

Turning 2FA on or off from the dashboard's Security tab while offline,
or on a dropped connection, left the dialog busy for good: enabling kept
its spinner up with the code boxes disabled, and disabling kept its
button and password field disabled, with nothing said either way. A
request that gets no answer is now reported like a refused one, and the
dialog can be tried again.

* fix(gui): stop disabling 2FA when the provider sign-in doesn't happen

For an account that signs in through a linked provider, turning 2FA off
opens the provider's sign-in popup. When the popup was closed or
blocked, the dialog showed why for a moment and then sent the request
anyway, whose refusal ("OIDC revalidation required") replaced the
reason. It now stops at the popup's error.
2026-10-24 15:05:56 -07:00
404oopsandClaude Fable 5.1 68249028ef test(ai): assert replayed Kimi tool arguments reach the wire decoded once
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:59:04 +02:00
404oops cbcbdc2599 Merge branch 'fix/replayed-tool-call-arguments' into codex/kimi-k3-1-pro 2026-10-24 22:57:54 +02:00
404oopsandClaude Fable 5.1 3296c1b0d6 fix(ai): bill Kimi K3.1 Pro from reported usage at the published rates
Cache writes now meter under `cache_write_tokens` at the model's write
rate, split out of `prompt_tokens` by the shared usage splitter, so K3 gets
the same breakdown. The credit hold reserves the prompt and any replayed
reasoning at the input rate, and a stream that ends without a usage chunk
is charged the estimate like every other provider. The per-TTL response
headers, the 502 accounting error and the one-hour write prices are gone:
the cache is scoped to the account and nothing here requests a one-hour
TTL, so those writes never occur and Moonshot publishes no rate for them.

The catalog entries carry the release date so responses normalize by
default, the provider suite no longer imports the SDK, and the docs cover
the reasoning efforts, replay and cache usage keys in their existing
sections.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:55:10 +02:00
404oopsandClaude Fable 5.1 a10a9d9b93 fix(ai): keep replayed tool-call arguments serialized once
An OpenAI-shaped `tool_calls` entry normalizes with its `arguments` still a
JSON string, and hoisting it back into `tool_calls` stringified that string
again, so every OpenAI-compatible provider received a JSON string literal
instead of the call's arguments. Hoisting now sends a string `input` as is
and only serializes object inputs.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:54:39 +02:00
Filip KujundžićandClaude Fable 5.1 bbb780eb31 feat(ai): MiniMax video generation (H3 and Hailuo) (#4309)
* feat(ai): MiniMax video generation (H3 and Hailuo)

Adds a minimax-video-generation provider to the video driver. H3 models go
through MiniMax's v2 task API and are billed per second of output; Hailuo
models go through the v1 API, exchange a file id for a download URL, and
are billed per clip. The shared `minimax` chat key is reused, with the
chat block's `/v1` base URL reduced to its host.

WIDTHxHEIGHT sizes now map onto tiers for catalogs that include a 2k or
4k tier.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai): validate MiniMax video inputs and resolution tiers

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-10-24 13:49:21 -07:00
Juan Castro ad5f9b6196 fix(push): canonical app uid in the push consent check 2026-10-24 16:03:05 -04:00
Juan Castro 65b966c264 feat(push): notifications:push app permission
Apps need a per-app notifications:push grant for their notifications to
go out as pushes. hasPushConsent checks it as the app acting for the
user, so only that user's grant to that app answers yes, and a revoke
takes effect on the next check. The permission dialog describes it and
the perms docs list it.
2026-10-24 16:03:05 -04:00
Juan Castro c701b8d7c0 fix(notifications): show the sending app on app messages
App messages carry the sending app's uid and name (and its icon when it
is an https URL), set by the server; the desktop toast and the dashboard
panel show the name above the title. Title and text drop invisible
formatting marks and control characters, with newlines kept in text and
whitespace-only text left out. The icon is stored as its normalized URL,
and a target may not hold dot segments or non-ASCII characters. Both
rate-limit windows are read before either is spent, so a refused call
counts against neither. The docs say what ttl means for push delivery.
2026-10-24 16:02:54 -04:00
Juan Castro 4b41b9cd54 feat(notifications): apps can notify the user they act for
Adds the app.message type (audience app-user). An app, its workers and
the tokens it issues can create it through puter-notifications create,
only for their own user and under their own app. The value is validated
against a fixed format, the write is not silent, and each app has a
per-user budget of 30 per minute and 300 per day, configurable with
notificationAppMessageLimits. Holders keep creating the other types
silently and cannot create app.message.
2026-10-24 16:02:54 -04:00
Nariman Jelveh 8a55debdae fix(dashboard): sixth adversarial review — bugs, UX, a11y, perf (#4314)
* fix(dashboard): render in the user's saved language

The dashboard never read the account's preferences: only the desktop did,
on boot. Everyone saw it in English whatever language they had chosen,
until the Files tab's Show hidden toggle switched a running session over,
to the browser's language when none was saved, since that was the
fallback the toggle wrote back.

The desktop's preference loading moves into load_user_preferences, which
the dashboard now runs in the background. The first paint uses the
language this browser last loaded, so nothing waits on the read; a saved
language that differs reloads into it, and a changed Show hidden setting
refreshes the open folder. The tab names are read when the sidebar is
drawn rather than when the modules load. A failed read keeps the cached
preferences instead of resetting them to English.

* fix(dashboard): label credit amounts with a key meant for them

Home's allowance and the Usage tab's total and Credits column used the
`credits` key, which the translations render in its acknowledgments sense:
"5,000 Mitwirkende" (contributors) in German, "Авторы" in Russian,
"特别鸣谢" in Chinese. They now use `usage_credits`, which falls back to
English until it is translated, as `monthly_credits` already does.

* fix(dashboard): enable the token and 2FA cards once the email is confirmed

The dashboard had no handler for the `user.email_confirmed` socket event,
and Account and Security pick the token button's state and whether to
offer 2FA when they render. A user who confirmed their email in another
tab, or in this one through the confirmation prompt, kept a disabled
"Create token" and no 2FA card until a reload.

The dashboard now refreshes the user on that event and renders both tabs
again, as it already does after a temporary account is saved.

* fix(dashboard): save only the profile fields that were edited

Edit profile sent both the display name and the bio on every save, taken
from this session's copy of the profile. That copy comes from the cached
user and can be stale (the profile changed on another device) or empty
(the profile read failed), so changing only the bio wrote the old display
name back, and a save from an empty prefill erased the bio.

The dialog now sends just the fields whose value changed from what it
opened with, and closes without a request when nothing did.

* fix(dashboard): don't show storage as 100% while there is room left

Home and Usage rounded the storage share to a whole percent, so an
account at 99.5% or more read "100%" while uploads still fit. The usage
meter already holds at 99% until the allowance is spent; that rule moves
into meterPercent and the storage meters use it too.

* fix(dashboard): keep keyboard focus inside the open phone drawer

Below 768px the sidebar is a modal drawer over a scrim, but opening it
left focus on the menu button, and Tab past its last entry walked into
the dimmed content behind the scrim. Opening it now focuses the current
page's entry, and Tab and Shift+Tab cycle within it while it is open.

* fix(dashboard): tapping into a Files name field edits the name

On a phone or tablet, tapping the name field of a new folder or a rename
(which iOS needs before it shows the keyboard) reached the row's tap
handler, which opened the folder or launched the file and threw the edit
away. A long-press in the field opened the touch sheet instead of
selecting text, and on desktop a right-click in it showed the file menu
instead of the browser's Cut/Copy/Paste.

The row's tap and menu handlers now leave the name field alone.

* fix(dashboard): don't offer Cut on items the user can't move

A file's menu offered Cut on anything that wasn't immutable, and Shared
rows aren't marked immutable. Cutting a file someone shared with you, or
anything inside a read-only shared folder, then pasting it, ended in a
"Forbidden" alert and an emptied clipboard. Cut now needs the same
permission as Delete in that menu, as Ctrl+X and the selection's Cut
already do.

* fix(dashboard): don't paste or upload into the folder being left

Opening a folder clears the listing at once but only moves currentPath
when the new listing arrives. Ctrl+V, a file dropped from the computer
onto the pane, or Paste from the background menu during that wait went
into the folder the user had just left, and since the refresh then
showed the new one, the items seemed to vanish.

Those three now refuse while another folder is loading, as New Folder
already did.

* fix(dashboard): clear the drop highlight when Escape cancels a drag

Escape ends a Files drag with a synthetic mouseup, which jQuery UI
treats as a release outside every target: it fires no `out`, so the
folder row under the pointer kept its dashed drop outline and tint (and
a sidebar folder or breadcrumb its highlight) after the drag was gone.
The Escape branch now takes those off.

* fix(gui): show the "can't rename in Trash" alert without entities

Pressing F2 on a trashed item (on the desktop or in the dashboard's
Files tab) showed "This item can&apos;t be renamed because it&apos;s in
the trash": the message was encoded by i18n and then again by UIAlert.

* fix(gui): show the failed-upload title without entities

The upload progress window writes its error title with innerText, but
was handed the HTML-encoded i18n string, so French showed "Échec de
l&apos;importation des fichiers". The Files tab's two upload paths and
the desktop's now pass the plain string.

* feat(dashboard): open a Files row's menu with the Menu key or Shift+F10

Share, Download, Open With and Properties were reachable only by
pointer: the ⋯ button isn't focusable, and the keyboard's menu keys
fired a contextmenu on <body>, which the listing ignores. Both keys now
open the selection's menu at its row's ⋯ button, where arrow keys and
Enter already work.

* fix(dashboard): let Shift+Arrow shrink a Files selection

Shift+Arrow only ever added the next row, so after Shift+Down twice,
Shift+Up left all three rows selected instead of going back to two. A
run of Shift+Arrows now selects the range from the row it started on to
the cursor, on top of whatever was selected before the run, so moving
back shrinks it. A click or a plain arrow starts a new run.

* fix(dashboard): name folders in the Back/Forward history menus as the crumbs do

Holding Back or Forward listed each folder by its path's basename: `/`
was a blank entry, the Shared view read "shared", Trash stayed in
English, and a trashed folder showed its uid. The menus now use the same
names the breadcrumbs show.

* fix(dashboard): show a shortened Files name in full on hover

Grid tiles and a resized Name column shorten long names in the middle
("Quarterl…draft v2.xlsx"), and nothing showed the rest short of opening
Rename or Properties. A shortened name now carries the full one as its
tooltip.

* fix(dashboard): keep keyboard focus on the Files breadcrumbs

Every listing rebuilds the breadcrumbs, which took a keyboard-focused
crumb with it: pressing Enter on one, or any refresh of the folder while
a crumb had focus, left focus on <body>, and the next Tab started over
from the top of the page. Focus now moves to the same folder's crumb,
or to the one for the folder just opened.

* fix(dashboard): reach Home's Shared and Trash rows from the keyboard

At 480px wide or less (a phone, or a desktop browser zoomed in) the
folder sidebar is hidden and Home lists Shared and Trash as rows instead.
Those rows only had a click handler, so a keyboard user couldn't get to
either. They are now focusable links that open on Enter or Space, with
the sidebar's focus ring.

* fix(dashboard): keep the Files selection across a sort or view change

Sorting by a column, switching between list and grid, or toggling Show
hidden lists the folder again, and the rebuild dropped the selection and
the shift anchor: select three files, sort by Size, and nothing was
selected. Only the refresh on returning to the browser tab carried them
across. That logic is now refreshKeepingSelection, and those three use
it too.

* fix(dashboard): don't drop a folder open during a Files refresh

A refresh of the folder on screen (after an upload or paste, Refresh, a
sort, or returning to the browser tab) set the same "already rendering"
flag that a navigation sets, so double-clicking a folder, a sidebar
folder, a crumb or Back during that round-trip did nothing, with no
spinner to explain it.

Opening another folder now takes over from a refresh in flight; the
refresh notices it was superseded and drops its result. A navigation
during another navigation is still ignored, as before.

* fix(dashboard): keep the Files marquee when an item arrives mid-drag

Adding a row clears the empty-folder notice by removing every non-row
child of the listing, and the selection marquee is one. A file added
from elsewhere (an upload, another tab, an app) while the user was
drawing a selection made the rectangle vanish under the pointer.

* perf(dashboard): don't rewrite unchanged Files names while resizing

Every width change (a window resize, collapsing the sidebar) re-fits the
item names, at most once a frame, and wrote each name's text back even
when it came out the same. Each write invalidated layout for the whole
list. With 300 rows that took ~6.4ms a frame in list view and ~4.9ms in
grid; skipping the unchanged writes brings it to ~0.7ms and ~3.2ms.

* fix(dashboard): tell assistive tech the Files Select toggle's state

The Select button only toggled a class, so a screen reader announced a
plain button with no state; it now carries aria-pressed. Sort and View
open menus and now say so with aria-haspopup.

* fix(dashboard): show a person's strongest grant in the share list

When someone held two grants on one item (the owner gave them view
access, and someone the owner let share gave them edit), the share
dialog showed whichever grant the listing returned first, so their row
could read "Can view" while they could edit. A second grant on the same
item now counts toward the row with the stronger of the two levels.

* fix(dashboard): only offer share controls on grants the user can change

The backend lets an item's owner change or withdraw any share of it,
and anyone else only the grants they issued (or their own access). The
share dialog offered the level select and Remove on every row. Someone
the owner let share who removed a person the owner had added got
"Removed bob" while nothing changed, and lowering their level added a
second grant that changed nothing either.

The access list now records who issued each row's grants. A user who
doesn't own the selection gets the controls only on rows whose grants
they issued all of, plus Remove on their own row to leave; other rows
show their level as a tag.

* fix(dashboard): update Files' shared markers after the share dialog closes

The share dialog marks the Files rows it covers as shared or not from
the listings it reloads after each change, and skipped that whole reload
once it was closed. Closing it while a share or removal was still saving
left the row without its shared marker (or with a stale one) until the
folder was reloaded. The markers are now updated either way; only the
dialog's own list is skipped once it's gone.

* fix(dashboard): mark rows shared after sharing more than 50 at once

The share dialog updates the Files rows' shared markers from the share
listings it reloads after a change, and doesn't reload them for more
than 50 items. Sharing such a selection, with a person or by link,
reported "Shared with bob on 55 items" while no row showed the marker
until the folder was reloaded. Those grants now mark their rows
directly.

* fix(gui): unbind each context menu's mousemove handler with the menu

The submenu-aim code bound a mousemove handler on the document for every
menu and submenu and never removed it. Each one kept its closed menu and
items alive and ran on every mouse move for the rest of the session; ten
right-clicks in Files left five such handlers behind. The handler now
comes off when its menu is removed. Moving diagonally into a submenu
still keeps it open.

* fix(gui): ignore clicks on a context menu's dividers

A divider is an <li> like the items, so a click on one (they're 11px
tall, easy to hit between two items) ran the item handler with no
action behind it and threw "Cannot read properties of undefined". The
item handler now skips dividers; the menu stays open, as a native
menu's separator does.

* fix(dashboard): warn that removing a team removes everyone in it

Removing a team from a single item's share dialog asked "Remove Design's
access to this item?", the wording used for one person. The desktop's
share window adds "Everyone in the team loses it."; the dashboard now
uses that confirmation for teams too.

* fix(dashboard): count folders, not items, in "via N folders"

A person who reaches a multi-item selection through more than one shared
ancestor gets a "via N folders" note in the share dialog, and N was the
number of items reached that way: five files under two shared folders
read "via 5 folders". It is now the number of distinct folders.

* fix(dashboard): don't treat a long tail after a dot as a file extension

Files middle-truncates long names and keeps everything after the last
dot whole as the extension. For "Report v1.2 final draft for the board
meeting" that kept ".2 final draft for the board meeting", so the tile
read "….2 final draft for the board meeting" and lost the start of the
name. Only a short tail without spaces now counts as an extension; this
one shortens to "Report v1.…d meeting".

* fix(dashboard): keep Tab inside the touch context menu

The touch menu sheet is modal, but Tab and Shift+Tab left it for the
page behind the backdrop while it stayed open; closing it then no
longer returned focus, since focus was elsewhere. Tab and Shift+Tab now
move between its items like the arrow keys, wrapping at the ends.

* fix(dashboard): say "the other item" when one item is missing a grant

The share dialog's "Add to all" link told screen readers "Give bob
access to the other 1 items" when the person was missing from a single
item of the selection. That case now has its own singular string.

* fix(dashboard): announce a failed Properties load, on both tabs

When Properties couldn't read the item, the General list showed the
error but stayed aria-busy, so screen readers kept it unannounced, and
the Versions tab stayed blank. The list is no longer busy once the
error shows, and Versions shows the same error.

* perf(dashboard): launch the add-app modal's apps without reloading the grid

"Browse the App Center" and "Build with AI" go through the same intro
as a deep link, which first awaited loadApps(). With the grid already
loaded that started a fresh fetch of every app list and the saved
layout, and the launch waited on it: with an 800ms connection, ~1.35s
before anything opened instead of ~0.1s. The intro now uses the grid
already loaded, and only joins or starts a load when there is none.

* fix(dashboard): don't hide the Apps grid on every search keystroke

Search re-rendered the grid behind the first load's fade-in, which waits
for page one's icons. Each keystroke made fresh <img> elements, so an
icon that was slow or failing (an unreachable icon host) blanked the
whole grid until it settled, about a second per keystroke with a 1s
icon and up to 1.5s if it never answers. Search, its clear button and
entering reorder mode now render the results at once.

* fix(dashboard): stop the Apps pager's dots jumping back mid-flip

A page arrow, a dot, or the wheel starts a smooth scroll and marks the
target page at once, but the scroll handler then read the pages passed
over on the way as the current one. The active dot went forward, back
to the old page, and forward again, and a second click on the arrow
added to the stale page: two quick clicks showed 0 → 1 → 0 → 2 → 1 → 2.

While such a scroll is on its way, the pager (and the open folder's)
now ignores the positions in between, until it arrives or a second has
passed. The user's own scrolling still moves the dots.

* fix(dashboard): keep paging while a dragged app is held at the edge

Holding a dragged tile still at the Apps grid's edge (or an open
folder's) turned one page and stopped: the next dwell only started on
another pointer move, so a mouse held steady stayed on the second page
of three. Once a flip settles, the dwell starts again if the pointer is
still in the edge zone.

* fix(dashboard): don't turn Apps pages on a trackpad pinch

A pinch on a trackpad reaches the page as wheel events with ctrlKey
set, and the Apps grid (and an open folder) read them as a wheel flick
and turned the page while the user zoomed. Those events are now left
alone.

The handlers also called preventDefault() from a listener the app makes
passive, which does nothing and logged a console warning on every wheel
tick; that call is gone, as the scroller has no vertical scroll to stop.

* feat(dashboard): open the first Apps search result on Enter

Typing in the Apps search and pressing Enter did nothing; reaching a
result took ArrowDown and then Enter. Enter in a non-empty search now
opens the first match (or a folder that matches), as launchers do. It
does nothing while an IME composition is open, or when nothing matches.

* fix(dashboard): include an open folder's page dots in its Tab cycle

The open folder keeps Tab inside it, but its list of stops left out the
page dots, which are buttons. Tab never reached them, and once one had
focus (clicking it focuses it) the next Tab walked out of the modal
folder to the page behind. The dots are now part of the cycle.

* feat(dashboard): open an app tile's menu with the Menu key or Shift+F10

An app or folder tile's menu (Open in New Tab, Copy Link, Uninstall,
Remove from Folder; a folder's Rename and Ungroup) opened only on a
right-click or long-press, and browsers don't all turn the keyboard's
menu keys into a contextmenu event (macOS has none). The Menu key and
Shift+F10 on a focused tile now open its menu at the tile, where arrow
keys and Enter work.

* fix(dashboard): keep keyboard focus on the grid after Ungroup

Ungrouping a folder from its menu re-rendered the grid without the
folder's tile, so a keyboard user who had it focused was left on
<body> and the next Tab started from the top of the page. Focus now
moves to the folder's first app, which takes the folder's place.

* fix(dashboard): give links AA contrast in the light theme

Link-colored text (Home's "View more" and "Upgrade", the notification
panel's "Mark all read" and "Retry", Properties' website links, the
token dialog's sessions link) used #5271ff, 4.07:1 on white and 3.97:1
on the content background, under the 4.5:1 AA minimum for text that
size. The light theme's link is now #3d5bd9 (5.7:1), with a darker
hover. Dark mode already passed.

* fix(dashboard): give success text AA contrast in the light theme

Success text (the 2FA "enabled" line, Home's active-plan badge, the
share dialog's confirmations) used #03933a: 4.01:1 on white and 3.80:1
on its green tint, under the 4.5:1 minimum. It is now #15803d (5.02:1
and 4.75:1); the success check icons that share the token darken with
it. Dark mode already passed.

* fix(dashboard): give danger buttons and error text AA contrast

White text on the red fills fell short of 4.5:1: the Apps uninstall
confirm (#e53935, 4.23:1), the delete-account button (#f00808, 4.43:1)
and the Files drag "Cancel" zone (#ef4444, 3.76:1). So did light-theme
danger text on its pink tint (#dc2626 on #fef2f2, 4.41:1), used for
form, share, crop and notification errors and the past-due badge.

The fills are now #d32f2f, #d90707 and #dc2626 (4.8:1 to 5.3:1), each
with a darker hover, and danger text is #b91c1c (5.9:1 on the tint).

* fix(dashboard): give the app feedback dialog's Send button AA contrast

The feedback dialog's primary button (Send, and Close once sent) put
white text on a light-blue gradient, 2.66:1 at the top and 3.42:1 at the
bottom. It now uses the dashboard's primary button colors, which meet
4.5:1 in both themes, like the dialogs around it.

* fix(dashboard): show the drop outline and dwell fill on grid folders

Dragging files over a folder in grid view only tinted it like a
selection: the grid's own border, hover and selected rules out-ranked
the shared drop-target and spring-load rules, and the hover rule's
background shorthand wiped the dwell fill. The folder then opened after
700ms with no sign it was about to. Grid and compact-grid folders now
get the list's dashed outline and filling dwell, with the padding
trimmed to match so the tile doesn't shift.

* fix(dashboard): keep the Files preview and drag ghost above the window

The Space-bar image preview (z-index 9999) and the drag ghost (10000)
are appended to <body>, beside the dashboard window. On a touch-only
tablet with a keyboard every window is pinned to 9999999, so the preview
opened invisibly behind it from the first press; on desktop the
window's z-index climbs with each click and overtakes both in a long
session. Both now sit in the band the sidebar tooltip and the toasts
already use.

* fix(dashboard): give small dashboard controls 44px touch targets

Several controls a phone or tablet user taps were under the 44px
minimum touch size, with nothing enlarging them on a coarse pointer:
the Files toolbar buttons (32px), the touch menu's rows (36px tall),
the drawer's close button (32px), the notifications close (30px), the
Properties close (32px), the feedback dialog's ✕ (30px), the add-app
Back arrow (26px), the Apps modal buttons (34px tall) and the share
dialog's Copy link (38px tall).

Following the earlier touch fixes, the icon buttons keep their look
and get a 44x44 hit area, and the text buttons and menu rows grow to
44px. Mouse and trackpad layouts are unchanged.

* fix(dashboard): keep a landscape phone's notch clear of the dashboard

A phone held sideways and wider than 768px (an iPhone Pro Max, a large
Android) gets the desktop layout, and the page asks for the full screen
with viewport-fit=cover. Nothing in that layout read the side safe-area
insets, so with the notch on the left the sidebar's icons sat under it
(x=24 against a 47px inset), and with it on the right the content's
edge and the toasts did.

The sidebar (open or collapsed) now adds the left inset, and the content
area, including Apps and Files, and the toasts add the right one. The
insets are 0 everywhere else, so other layouts don't change.

* fix(dashboard): size the touch menu and bottom sheets to the visible height

The touch context menu, the app feedback dialog and the profile-photo
crop sheet capped their height in vh. On phones 100vh is the viewport
with the browser bars hidden, so while they show these could be taller
than the screen. The touch menu, which ContextMenu.js keeps inside the
visible height, then ran its last rows off the bottom on a landscape
phone, and the bottom sheets could push their header off the top. They
now cap in dvh, as style.css already does.

* fix(gui): move keyboard focus into the Manage sessions dialog

The sessions dialog is modal but never took focus: opening it from the
keyboard left focus on the button behind the backdrop, and Tab walked
the page underneath instead of the session list. It now focuses its
close button on open, keeps Tab inside, and hands focus back to the
control that opened it when it closes.

* fix(dashboard): return focus after the Security tab's dialogs close

Closing Blocked people, or the 2FA setup or disable dialog opened from
the switch, left keyboard focus on <body>, so the next Tab started at
the top of the page. The 2FA switch also disables itself while its
dialog is up, which drops focus even before then. Focus now goes back
to the button or switch that opened the dialog once it has gone.
2026-10-24 08:30:57 -07:00
Filip Kujundžić ce5f28e844 Validate Kimi completion accounting and preserve reasoning request semantics 2026-10-24 16:02:17 +02:00
Daniel SalazarandReynaldi Chernando f3e8539eaf docs: easy events recipe (PUT-1961) (#3998)
* docs: easy events recipe (PUT-1961)

* docs: catch-up-on-notifications events recipe (PUT-1961)

* docs: notification examples use notif:app-user, which apps can read

* docs: events recipe corrections from a run against a live server (PUT-1961)

* refine and update agents md

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-24 15:26:07 +07:00
Filip Kujundžić 97beffb572 Make completed Kimi accounting failures terminal without estimated billing 2026-10-23 21:05:39 +02:00
Filip Kujundžić 03eab894a0 fix(ai): reserve Kimi cache writes and preserve tool arguments 2026-10-23 20:56:42 +02:00
Filip Kujundžić 53c77c8333 feat(ai): support Kimi K3.1 Pro models 2026-10-23 20:43:35 +02:00
913 changed files with 138176 additions and 15988 deletions

No files matched your search

@@ -47,3 +47,13 @@ jobs:
PUTER_TEST_AI_MOONSHOT_API_KEY: ${{ secrets.PUTER_TEST_AI_MOONSHOT_API_KEY }}
PUTER_TEST_AI_ZAI_API_KEY: ${{ secrets.PUTER_TEST_AI_ZAI_API_KEY }}
PUTER_TEST_AI_ELEVENLABS_API_KEY: ${{ secrets.PUTER_TEST_AI_ELEVENLABS_API_KEY }}
# Realtime conversations through the SDK against live OpenAI. Builds
# the SDK bundle these tests load itself, so it runs even when the
# step above fails; the tests skip themselves without the key.
- name: Run live realtime conversation tests
if: ${{ !cancelled() }}
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/convoLive.test.ts
env:
CI: 'true'
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
+48
View File
@@ -0,0 +1,48 @@
name: Notify HeyPuter (dev)
on:
workflow_dispatch:
inputs:
puter_ref:
description: "puter branch, tag or SHA"
type: string
default: "main"
permissions:
contents: read
jobs:
authorize:
if: github.repository == 'HeyPuter/puter'
runs-on: ubuntu-latest
steps:
- name: Require maintain or admin
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ACTOR: ${{ github.triggering_actor }}
run: |
ROLE=$(gh api "repos/$GITHUB_REPOSITORY/collaborators/$ACTOR/permission" --jq .role_name)
if [ "$ROLE" != "maintain" ] && [ "$ROLE" != "admin" ]; then
echo "::error::$ACTOR has role '$ROLE'; maintain or admin required"
exit 1
fi
notify:
needs: authorize
runs-on: ubuntu-latest
# Holds HEYPUTER_DISPATCH_TOKEN; limited to main so branch copies of this file can't read it.
environment: heyputer-dispatch
steps:
- name: Trigger heyputer dev deploy
env:
PUTER_REF: ${{ inputs.puter_ref }}
TOKEN: ${{ secrets.HEYPUTER_DISPATCH_TOKEN }}
run: |
jq -nc --arg ref "$PUTER_REF" \
'{event_type: "puter-dev-requested", client_payload: {puter_ref: $ref}}' |
curl -fsS -X POST \
-H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/HeyPuter/heyputer/dispatches \
-d @-
echo "Requested \`$PUTER_REF\`" >> "$GITHUB_STEP_SUMMARY"
+2 -1
View File
@@ -8,6 +8,7 @@ on:
jobs:
notify:
runs-on: ubuntu-latest
environment: heyputer-dispatch
steps:
- name: Trigger heyputer build
run: |
@@ -15,4 +16,4 @@ jobs:
-H "Authorization: token ${{ secrets.HEYPUTER_DISPATCH_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/HeyPuter/heyputer/dispatches \
-d '{"event_type":"puter-main-updated","client_payload":{"puter_ref":"main"}}'
-d '{"event_type":"puter-main-updated","client_payload":{"puter_ref":"main"}}'
+5 -1
View File
@@ -118,9 +118,13 @@ jobs:
fail-fast: false
matrix:
include:
# The base branch lives in this repo; for a fork PR, the head
# repo's copy of it can be arbitrarily stale.
- ref: ${{ github.base_ref }}
repository: ${{ github.repository }}
artifact: base
- ref: ${{ github.head_ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
artifact: pr
steps:
@@ -129,7 +133,7 @@ jobs:
uses: actions/checkout@v7
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
repository: ${{ matrix.repository }}
- name: Set up Node.js
if: needs.changes.outputs.puterjs == 'true'
+31 -2
View File
@@ -161,11 +161,38 @@ An overview page answers "what is this API and is it what I need?", and a functi
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
### Function page layout
A function page opens with one sentence that says what the function does, in its own paragraph, such as "Given a prompt, generate an image using AI." Put any more detail in the paragraphs after it.
After the intro, a function page has these `##` sections, in this order:
1. Syntax
2. Parameters
3. Return value
4. Errors, if the function has any
5. Sections for specific topics, if the page needs them, with `## Model-specific features` last
6. Examples
[chat.md](src/docs/src/AI/chat.md) follows this layout.
- **Options as a table.** Document an options object as a table with Option, Type, and Description columns. Give each option its own row, even when two options go together, such as `width` and `height`. Order the rows by how often developers use them, most common first, without group rows. In chat.md that's the core options, then reasoning, tools, response handling, request control, and output.
- **Common options in the table, model-specific behavior in its own section.** A page whose function works with many models or providers, such as [chat.md](src/docs/src/AI/chat.md) and [txt2img.md](src/docs/src/AI/txt2img.md), splits its options this way:
- **The options table** under Parameters lists the options that work with every model, and describes each one the way it works for every model.
- **`## Model-specific features`**, the last section before Examples, covers everything else. It has a `###` for each model family or provider, named after it, such as "OpenAI models" or "Cloudflare models". Under it go the options only some models accept, and the common options a model handles differently, such as a narrower range, other allowed values, or an option it ignores.
- **Write there only what's different.** A `###` lists an option only when that model or provider adds it or handles it differently. An option that works the same everywhere stays out, even if the provider supports it, and a model or provider with nothing different gets no `###`.
- **Model-specific features is the only place for model-specific details.** Every limit, format, allowed value, default, price, or quirk that applies to some models goes there, in that model's or provider's `###`, and nowhere else. The options table and topic sections (such as Image inputs or Video input) describe what's true for every model and link to Model-specific features for the rest. Don't add a table or sentence there that compares providers or repeats a model's limits. Each copy is another source of truth to keep in sync, and copies drift when a model changes.
- **Link the two.** Directly under the options table, put `Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).` A row whose option behaves differently on some models can also end with `See [Model-specific features](#model-specific-features).`
- **A long topic gets its own `##` section.** When explaining an option or feature takes more than a sentence or two, such as function calling or output dimensions, give it a section between Errors and Examples and link to it from the options table. This keeps the reference sections short.
- **Usage patterns are examples.** A way of using the API, such as stopping a stream, goes in a runnable example that the relevant section links to ("See the [stop a stream example](/playground/ai-chat-stop-stream/)."), not in a section of its own.
- **Each example has a `###` heading** under `## Examples`, so every example shows in the table of contents.
### Writing docs
These apply to reference pages and recipes alike.
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
- **Document Puter as it runs on puter.com.** The docs are for developers building on production Puter. Leave out behavior that only happens on a self-hosted or differently configured server, such as a provider that isn't set up there.
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
@@ -184,10 +211,12 @@ Recipes are tested, copy-pasteable patterns for a task, one layer above the per-
- **Description.** Written for someone who has never heard of Puter. Start with "Learn how to", put the action first, and say what the reader's app gains: "Learn how to add file sharing to your app with Puter.js, so users can share files in their Puter account with other people." No API names, no mechanism, no list of example cases, and no "not just …" contrast.
- **Intro.** Explain the situation and why you'd do this, tied to something the reader already knows ("`puter.kv` is your database in Puter.js"). No "Use this when" bullet list.
- **Structure.** Get to the first working call as early as possible, then add validation, rate limits, and sign-in after it. Explain a Puter concept the first time it appears, such as what `me.puter` is or that `puter.workers.exec()` opens the Puter sign-in window. Split recipes by what the reader needs, not by technique, and give a family of recipes a shared tag.
- **Reading order.** A recipe reads top to bottom. Each code block uses only functions and variables from blocks above it, so the reader never has to search further down the page to understand it. Start with a small version that works, and have each section add one case. When a section changes a function from earlier, show the whole new version of that function there. If the code is built up over several sections, end with a `## Full Code` block that has all of it.
- **Link to the reference for full option lists.** Mention only the options the recipe uses, then link to the function page for the rest ("All the sort options are listed on [`puter.fs.readdir()`](/FS/readdir/)."). A copied list of options, rules, or scopes goes stale when the reference changes.
- **Limits.** Mention a limit only when it's the reason for a pattern the recipe uses, such as one wildcard subscription instead of one per key, and link to [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) or the function page for the number. Leave out limits that code following the recipe never reaches.
- **Code.** Platform-neutral: just the code, with top-level `await`. No `<script>` or `<html>` wrapper, no IIFE.
- **Tradeoffs.** Justify a choice with one scenario the reader can picture right away, in about two short paragraphs, or fold each point into the section where it matters. The description states the benefit only.
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit goes in the section it applies to; other facts belong on the linked reference page.
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit the recipe needs goes in the section it applies to, and other facts belong on the linked reference page.
---
@@ -199,6 +228,6 @@ Recipes are tested, copy-pasteable patterns for a task, one layer above the per-
- **Reuse existing UI primitives before writing new ones.** Windows and dialogs are `UIWindow*` functions in [src/gui/src/UI/](src/gui/src/UI/); generic pieces already exist (`UIAlert`, `UIPrompt`, `UIContextMenu`, `UINotification`, `UIPopover`, widgets in [UI/Components/](src/gui/src/UI/Components/)). A new window should read like its neighbors: an async function taking an options object, composing an HTML string, wiring behavior with jQuery, delegating to `UIWindow(...)`.
- **Match the visual language.** Use existing CSS classes (`button`, `button-primary`, window chrome, form styles) and copy the layout patterns of neighboring windows; new styles go in [src/gui/src/css/](src/gui/src/css/) following existing conventions. Verify anything positional (menus, overlays, z-index) on both desktop and mobile viewports.
- **i18n every user-facing string.** No hardcoded UI text — use `i18n('key')` and add the key to [src/gui/src/i18n/translations/en.js](src/gui/src/i18n/translations/en.js). Run `npm run check-translations` before opening the PR.
- **i18n every user-facing string.** No hardcoded UI text — use `i18n('key')` and add the key to [src/gui/src/i18n/translations/en.js](src/gui/src/i18n/translations/en.js).
- **Shared logic goes in helpers/services.** Reusable non-UI logic belongs in [src/gui/src/helpers/](src/gui/src/helpers/) or [src/gui/src/services/](src/gui/src/services/), not copy-pasted between windows.
- **Tests.** Vitest is wired for the GUI (`src/**/*.test.js`; see [appOrder.test.js](src/gui/src/UI/Dashboard/appOrder.test.js) for the shape). Extract pure logic into functions and test those. Desktop behavior driven through puter.js (`puter.ui.*`) is covered by the Playwright harness in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/) — add a spec there when you change how the desktop renders SDK-driven UI.
+7 -1
View File
@@ -73,6 +73,12 @@ COPY --from=build --chown=node:node /opt/puter .
RUN mkdir -p /etc/puter /var/puter && \
chown -R node:node /etc/puter /var/puter
# Fix outdated HOME_URL in /etc/os-release (see issue #4329)
# The base image may contain a reference to the old phoenix repository.
RUN if [ -f /etc/os-release ]; then \
sed -i 's|HOME_URL="https://github.com/HeyPuter/phoenix"|HOME_URL="https://github.com/HeyPuter/puter"|' /etc/os-release; \
fi
# Self-hosters mount their override at this exact path. The v2 loader
# deep-merges it over config.default.json (see backend/index.ts).
ENV PUTER_CONFIG_PATH=/etc/puter/config.json
@@ -87,4 +93,4 @@ USER node
HEALTHCHECK --interval=30s --timeout=3s --start-period=30s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
CMD ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/index.js"]
CMD ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/index.js"]
+5
View File
@@ -350,6 +350,9 @@
// not, a row past this is swept. Off by default; any value above 0 arms
// the sweep.
// "notificationRetentionDays": 14,
//
// Ms a live notification waits for open tabs to acknowledge it.
// "notificationAckTimeoutMs": 5000,
// ── Alarms / alerting ───────────────────────────────────────────────
// Where system alarms go. Severity is the routing decision — each
@@ -433,6 +436,8 @@
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
// MiniMax. One key powers chat and video generation (H3, Hailuo);
// video calls go to the same host, beside the chat API's `/v1`.
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
+1 -1
View File
@@ -47,7 +47,7 @@ Both are supported ways to define an API. **Prefer a controller when you need fi
Provider sign-in requires the account's authenticator or a recovery code when 2FA is enabled, including signup callbacks that resolve an existing account. The five-minute challenge uses the same single-use `/login/otp` and `/login/recovery-code` verification as password sign-in. The GUI carries it in a URL fragment and removes it before initialization. A session and successful popup-return proof are issued only after verification; cancelling retains the popup's signed retry context without marking it authenticated.
Disabling 2FA through `/user-protected/disable-2fa` requires both password or OIDC revalidation and the current six-digit authenticator `code`. Missing, malformed, or incorrect codes leave the secret and recovery codes intact. A team's requirement for 2FA still prevents disabling it.
Disabling 2FA through `/user-protected/disable-2fa` requires both password or OIDC revalidation and either the current six-digit authenticator `code` or an unused recovery code in `code`. Recovery codes are consumed atomically. Missing or incorrect codes leave the secret and remaining recovery codes intact. A team's requirement for 2FA still prevents disabling it.
Email changes through `/user-protected/change-email` require the password or OIDC revalidation cookie, plus a six-digit `code` from the authenticator when 2FA is enabled. Missing or malformed codes return `400 bad_request`; incorrect codes return `401 code_mismatch`. Both include `factor: 'otp'`. Confirmation links are bound to that verified request. A 2FA-enabled account must request a new link if its existing link was issued without second-factor verification.
+105
View File
@@ -0,0 +1,105 @@
# Embedded app browser
`GET /embed/apps` on the deployment's main GUI origin renders the same app browser
as the dashboard's Apps tab, without the desktop or dashboard navigation.
It supports search, paging, saved folders, and saved ordering. It uses the same
installed-app and recommendation data as the dashboard, including the user's
removed recommendations. Editing, adding, reordering, renaming, and uninstalling
are disabled. Selecting an app opens it in a new browser tab at its own URL, the
same way the dashboard does, and the host page stays open. The app's token isn't
put in the URL: the app's SDK asks the embed for it once the tab loads, so the
embed must still be open then. If the iframe is removed first, the app falls back
to its normal sign-in. Apps that need the desktop (built-in and godmode apps,
private apps the user can't open, or a failed token request) open `/app/<name>`
on the GUI origin, cut off from the embed. Website shortcuts open their external
destination with no opener.
Apps opened from the embed carry `utm_source=puter.com&utm_medium=referral&utm_campaign=app_browser_embed`;
the desktop and dashboard send `utm_campaign=desktop` and `utm_campaign=dashboard`.
## Session and framing
Only show the iframe after the user has signed in. The embed reads the existing
GUI session from local storage on its own origin. It does not create accounts,
show a login dialog, or accept a token from its parent. A missing session or
failed load displays an unavailable/error state. Being signed in on a subdomain
alone does not populate the main GUI origin's storage; the main GUI session must
already exist.
The route's `Content-Security-Policy: frame-ancestors` allows the GUI origin and
the embedding page's origin when it is a subdomain of `config.domain` with the
scheme and port from `config.origin`, and not under one of the user-content
hosting domains (`static_hosting_domain`, `private_app_hosting_domain`, and their
`_alt` variants). For example, `https://puter.com` permits
`https://spreadsheet.puter.com`, and `http://puter.localhost:4100` permits
`http://spreadsheet.puter.localhost:4100` but not
`http://someone.site.puter.localhost:4100`. The embedding page is read from the
iframe request's referrer, so the host must not suppress it (the browser default
`strict-origin-when-cross-origin` is enough). All ancestors in a nested iframe
chain must be allowed. Other GUI routes retain `X-Frame-Options: SAMEORIGIN`.
The embed accepts no query parameters. Requests with a query string redirect to
the same embed path so SDK bootstrap parameters cannot override the session or API
origin. The host needs to allow the GUI origin in its own `frame-src` CSP if it
sets one. If sandboxing the iframe, allow scripts, same-origin storage, popups,
and popups to escape the sandbox.
## Host integration
Create the iframe when “More apps” is opened, and remove it on close so reopening
loads current apps and preferences. The host owns the modal and close control;
there is no parent messaging API and no app list or credential is sent to it.
The app browser fills its iframe and responds to its size.
This example assumes `puterOrigin` comes from the host app's deployment settings
and the caller has already established the GUI session. Translate the button and
frame labels using the host app's localization system.
```html
<button id="more-apps" type="button">More apps</button>
<dialog id="apps-dialog" aria-label="More apps">
<button id="close-apps" type="button">Close</button>
<div id="apps-frame"></div>
</dialog>
<style>
#apps-dialog { padding: 12px; width: min(900px, 90vw); }
#apps-frame iframe { width: 100%; height: min(650px, 75vh); border: 0; }
</style>
<script type="module">
const puterOrigin = 'https://puter.com'; // Use this deployment's configured origin.
const dialog = document.querySelector('#apps-dialog');
const container = document.querySelector('#apps-frame');
const trigger = document.querySelector('#more-apps');
trigger.addEventListener('click', () => {
const frame = document.createElement('iframe');
frame.title = 'Your apps';
frame.src = new URL('/embed/apps', puterOrigin).href;
container.replaceChildren(frame);
dialog.showModal();
});
document.querySelector('#close-apps').addEventListener('click', () => dialog.close());
dialog.addEventListener('close', () => {
container.replaceChildren();
trigger.focus();
});
</script>
```
## Implementation and checks
`createAppBrowser` in `src/gui/src/UI/Dashboard/AppBrowser.js` owns the shared UI
and data behavior. `TabApps.js` injects desktop actions. `appsEmbed.js` supplies
new-tab launching through `launchAppInTab` in `src/gui/src/helpers/appTabLaunch.js`,
which builds the same app URL as `launch_app`, and loads only the embed entry, its
SDK, and shared styles.
The normal GUI webpack build emits `bundle.min.js` and `apps-embed.min.js`.
No puter.js API or app-list endpoint changes are needed.
Run the GUI component/browser tests (with the SDK bundle built and Playwright
Chromium installed), and the backend route tests:
```sh
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
```
+11 -1
View File
@@ -51,11 +51,21 @@ two pages can't be decoded into how fast the table grows; `openCursor` still
reads a plain cursor. What a cursor wraps is an implementation detail per
store:
- DynamoDB-backed lists wrap `LastEvaluatedKey`.
- DynamoDB-backed lists wrap `LastEvaluatedKey`. A kv listing of another
namespace (cross-app or through a share handle) resumes after the last entry
it returned instead, since the scan can stop on a private one, and its cursor
is sealed to that listing and padded to one length.
- SQL-backed lists wrap a keyset position — `(sortValue, id)` of the last
row — and the query seeks past it (`WHERE (col, id) > (?, ?) ORDER BY col,
id`). Any SQL list gaining a cursor must have a deterministic `ORDER BY`
ending in a unique tiebreaker (`id`).
- The notification catch-up (`GET /events/fetch`) holds two keysets in one
sealed cursor: the last `id` it returned, and the last `(replaced_at, id)`
among rows up to that id. Each is its own indexed read, merged into one
page, so a row replaced after a client read it comes back from that cursor.
`from=end` with no cursor answers an empty page whose cursor holds the
slice's newest `id` and the slice's latest `(replaced_at, id)` as the
replacement mark.
Cursors that carry a sort also pin it: a request that passes a cursor plus a
conflicting sort is rejected with 400.
+80
View File
@@ -0,0 +1,80 @@
# Browser push client
Puter's desktop shows pushes through `/sw.js`. Core shows Settings → Notifications only when the `pushEnabled` gui param is `true`; core never sets it. The worker is registered only when push is turned on in the browser, and a load removes one left without a subscription. A backend extension that delivers pushes sets it with `services.homepage.setGuiParam('pushEnabled', true)` and can load its own code into the worker with `services.homepage.registerServiceWorkerScript(path)`.
Core subscribes the browser to a push service only when the server also publishes `pushWebPushKey` (see [Built-in Web Push client](#built-in-web-push-client)), and it has no `/push/*` routes of its own. The device, app, preference and subscription endpoints below come from that extension. On a deployment without one, such as most self-hosted servers, they answer core's plain 404 (`code: 'not_found'`), which `puter.notifications` reports as `push_unavailable` and the Notifications tab shows as unavailable. A device the extension can't find is `device_not_found`.
## Push payload
`sw.js` reads the push's JSON body. The message is its `puter` field, or `data.puter` for services that only carry string data, as an object or a JSON string:
```ts
{ uid?: string; title?: string; body?: string; icon?: string; group?: string;
target?: { app?: string; path?: string }; data?: unknown; channel?: string;
type?: 'challenge'; challenge?: string }
```
`uid` is the notification's uid, a later push with the same `group` replaces the one showing, and `target` and `data` reach the app when the user taps it.
When the browser replaces the subscription (`pushsubscriptionchange`), `sw.js` posts `{ type: 'puter-push-subscription-changed' }` to every open window on Puter's origin. The worker has no token or session, so the page's push client registers the new subscription.
## Receipts
Before a notification is pushed, the server asks the open clients whether they received it live. It sends `notif.receipt` with `{ uid }` on the user's sockets for the app that sent it, or on the desktop's sockets for Puter's own notifications. The SDK remembers the uids its page received through `onNotification()`, `onPush()` or the desktop's feed, and answers once its page has that notification, or with `{ received: false }` when it never gets it. The server waits up to 5 seconds (`notificationAckTimeoutMs`).
An answer carries `visibilityState` and `focused` as they were when the page received the notification. Pages never show a browser notification themselves; pushes do.
- An answer from a page that was visible and focused stops the push (`handled` on `notif.delivery.settled`). An older client that doesn't send `focused` counts while visible.
- Otherwise the notification is pushed, to every browser the user subscribed. A website's pushes go only to the browsers that subscribed through its own worker, never to Puter's `/sw.js`. An app inside the desktop pushes through `/sw.js`, which shows the app's name and icon.
- A client that can't answer, such as an older puter.js, doesn't hold the push back.
The ask reaches only the sockets in the asking region. So that other regions hear about a page too, a page that receives a notification live while visible and focused also sends `notif.handled` with `{ uid }` on the socket it came on. Core relays it as the node-local `notif.handled` event with the socket's user and app, at most 60 a minute per user; an extension that delivers pushes decides what to do with it.
## Registering
The registry is in [src/gui/src/helpers/pushClient.js](../src/gui/src/helpers/pushClient.js). A service script registers its client like this:
```js
service_script(async (api) => {
api.use()['push.registerPushClient']?.(client);
});
```
`use(id)` throws when nothing is registered under `id`, so the form above also runs on a Puter version without the registry and does nothing there. Any script that runs after the GUI bundle can call `use()['push.registerPushClient']?.(client)` the same way. Registering again replaces the client. Registering dispatches `push-client-registered` on `window`, and an open Notifications tab redraws.
With no client registered, the tab says push notifications aren't available in this browser.
### Built-in Web Push client
When the server also sets the `pushWebPushKey` gui param (a VAPID public key), core registers its own client ([webPushClient.js](../src/gui/src/helpers/webPushClient.js)) unless an extension registered one first. It uses `puter.notifications.enable()` / `disable()`, the same standard Web Push flow apps use on their own origins:
1. `GET /push/web/key` for the key, then `pushManager.subscribe()` on the `/sw.js` registration.
2. `POST /push/web/subscriptions { subscription }`. A subscription the server already has answers `{ status: 'active' }`. A new one answers `{ status: 'pending' }`, and the server sends it a push with `{ type: 'challenge', challenge }`.
3. The worker shows that push and posts `{ type: 'puter-push-challenge', challenge }` to the origin's windows; the page sends it to `POST /push/web/subscriptions/verify`. The device is registered once the page hands the confirmation back.
4. `DELETE /push/web/subscriptions { endpoint }` removes it.
Apps on their own origin do the same through `push-sw.js`, which they import into their own worker. A subscription made with an app token is the app's: its pushes are that app's notifications only.
## Interface
```ts
interface PushClient {
enable(): Promise<{ deviceUid: string }>;
disable(): Promise<void>;
status(): Promise<{
permission: 'default' | 'granted' | 'denied';
registered: boolean;
deviceUid?: string;
}>;
}
```
- `enable()` may ask for notification permission, so core calls it from a click: Turn on in the Notifications tab, Allow on a permission prompt that includes `notifications:push` while this browser is off, and the "Turn on notifications" button of the `/action/enable-push` popup and the matching desktop dialog (`UIEnablePushDialog`). Core also calls it without a click, from `helpers/pushSync.js`, when `Notification.permission` is already `granted`, `status()` reports the browser as not registered, the user has not turned push off in this browser, and an app holds `notifications:push`; the browser does not prompt then. The built-in client runs `puter.notifications.enable()`, which subscribes with the `/sw.js` registration and registers the browser through the Web Push flow above (`platform: 'webpush'`). It then resolves the `uid` of this browser's `webpush` device from `listDevices()`. It rejects when permission is refused or the browser wasn't registered.
- `disable()` runs `puter.notifications.disable()`, which sends `DELETE /push/web/subscriptions { endpoint }` and ends the browser's subscription. The tab calls it for Turn off, and for Remove on the device marked as this browser. Remove on any other device is `DELETE /push/devices/:uid`.
- `status()` reports the state without prompting. `permission` is `Notification.permission`. `registered` is true when this browser has a device registered for the signed-in user, and `deviceUid` is that device's `uid`.
After `enable()` resolves, the tab calls `POST /push/devices/:uid/test` and asks the user whether a notification showed up.
The tab also reads `GET /push/apps` and `GET`/`POST /push/preferences`.
Core keeps its own record of a browser the user turned off, `puter.push.browser-off:<user uuid>` in `localStorage`, written by Turn off and by Remove on this browser's device, and cleared when `enable()` resolves. The silent registration above skips a browser with that record, so a device removed here stays removed while one removed from another device, or a new sign-in in a browser that already allows notifications, registers again.
+19 -1
View File
@@ -12,7 +12,17 @@ Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker
## Requirements
- **Docker** with the `compose` plugin.
- **Docker** with the `compose` plugin. This is the one thing you have to install
yourself — the installer checks for it and stops with an explanation rather
than installing it for you.
- **macOS / Windows:** [Docker Desktop](https://docs.docker.com/get-docker/).
Both ship the `compose` plugin.
- **Linux:** [Docker Engine](https://docs.docker.com/engine/install/) plus the
[`docker-compose-plugin`](https://docs.docker.com/compose/install/) package.
- Confirm before running anything else: `docker compose version` should print a
version, not `command not found`.
- If Docker is installed but the shell can't see it, make sure your user is in
the `docker` group (Linux) or that Docker Desktop is actually running.
- A **domain** with DNS access — you need a wildcard record (`*.your-domain.com` → server IP). Puter routes by subdomain (`api.<domain>`, `site.<domain>`, `app.<domain>`).
- Optional: **TLS certs** (or `certbot` to grab them — see Step 3).
@@ -657,6 +667,14 @@ Migrations re-apply idempotently across pulls. Volumes are preserved.
## Troubleshooting
**`missing required command: docker` — the installer stops immediately.**
Docker isn't on your `PATH` yet. Puter self-hosting is a Docker Compose stack, so
nothing can run without it. Install [Docker Desktop](https://docs.docker.com/get-docker/)
on macOS/Windows or [Docker Engine](https://docs.docker.com/engine/install/) plus
`docker-compose-plugin` on Linux, then confirm `docker compose version` works in
the same shell and re-run the installer. On Linux, a fresh Docker install usually
needs a new login session before your user picks up the `docker` group.
**Site loads but I get a 502 / "Bad Gateway" from Caddy.**
The puter container failed to come up. `docker compose logs puter` will tell you which dependency rejected it (most often DB password mismatch between `.env` and `config.json`).
+35
View File
@@ -314,6 +314,41 @@ describe('thumbnails extension — handleThumbnailRead', () => {
await server?.shutdown();
});
it('keeps read URLs stable within an hour and rotates them at the next hour', async () => {
const key = mintedKey();
const signedAt = Date.UTC(2026, 9, 26, 10);
const read = async () => {
const entry: Record<string, unknown> = {
thumbnail: `s3://${BUCKET}/${key}`,
};
await handleThumbnailRead(entry, {
s3,
s3Presign,
bucketName: BUCKET,
bucketEndpoint: BUCKET_ENDPOINT,
fsEntry: server.stores.fsEntry,
});
return entry.thumbnail as string;
};
vi.useFakeTimers({ toFake: ['Date'] });
try {
vi.setSystemTime(signedAt + 1000);
const first = await read();
vi.setSystemTime(signedAt + 3599_000);
expect(await read()).toBe(first);
expect(new URL(first).searchParams.get('X-Amz-Date')).toBe(
'20261026T100000Z',
);
expect(new URL(first).searchParams.get('X-Amz-Expires')).toBe(
'604800',
);
vi.setSystemTime(signedAt + 3600_000);
expect(await read()).not.toBe(first);
} finally {
vi.useRealTimers();
}
});
// Thumbnails stored before keys were bound to entries keep displaying.
it.each([
['an entry-bound key', () => mintedKey()],
+5 -2
View File
@@ -307,6 +307,9 @@ export const handleThumbnailRead = async (
const thumb = entry.thumbnail;
if (typeof thumb !== 'string' || !thumb) return;
const presignClient = deps.s3Presign;
const signingDate = new Date(
Math.floor(Date.now() / 3_600_000) * 3_600_000,
);
if (isStoragePointer(thumb, deps.bucketEndpoint)) {
const key = resolveThumbnailKey(thumb);
@@ -319,7 +322,7 @@ export const handleThumbnailRead = async (
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
{ expiresIn: 604800 },
{ expiresIn: 604800, signingDate },
);
} else if (thumb.startsWith('data')) {
// Move an inline thumbnail into storage. The row only takes the new
@@ -373,7 +376,7 @@ export const handleThumbnailRead = async (
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
{ expiresIn: 604800 },
{ expiresIn: 604800, signingDate },
);
}
};
+21 -1
View File
@@ -53,9 +53,29 @@ need() {
command -v "$1" >/dev/null 2>&1 || die "missing required command: $1"
}
check_docker() {
if command -v docker >/dev/null 2>&1; then
return 0
fi
echo >&2 ""
echo >&2 "[puter-install] missing required command: docker"
echo >&2 ""
echo >&2 "Docker is required to run Puter self-hosted."
echo >&2 ""
echo >&2 "Installation instructions:"
echo >&2 " - macOS (Apple Silicon & Intel): https://docs.docker.com/desktop/setup/install/mac-install/"
echo >&2 " (Docker Desktop is recommended. Alternatives: OrbStack, Colima)"
echo >&2 " - Linux: https://docs.docker.com/engine/install/"
echo >&2 " - Windows: https://docs.docker.com/desktop/setup/install/windows-install/"
echo >&2 ""
echo >&2 "After installing Docker, make sure it's running and try again."
echo >&2 ""
exit 1
}
# ── Step 1: dependency check ────────────────────────────────────────
log "checking dependencies"
need docker
check_docker
need curl
need openssl
docker compose version >/dev/null 2>&1 \
+786 -768
View File
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -54,7 +54,6 @@
"dev": "npm start",
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
"check-translations": "node tools/check-translations.js",
"prepare": "husky",
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
"check:puterjs:types": "node tools/checkPuterjsTypes.mjs",
@@ -79,7 +78,8 @@
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.15",
"miniflare": "^4.20260617.1",
"open": "^10.1.0"
"open": "^10.1.0",
"sharp": "^0.35.5"
},
"engines": {
"node": ">=24.0.0"
@@ -108,6 +108,25 @@ describe('AbstractDatabaseClient — unimplemented surface', () => {
});
describe('AbstractDatabaseClient — replica-aware reads', () => {
it('avoids a primary read when the replica has rows in sequential mode', async () => {
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
expect(
await client.tryHardRead('SELECT 1', [], { race: false }),
).toEqual([{ id: 1 }]);
expect(client.preads).toEqual([]);
});
it.each([[], new Error('replica unavailable')])(
'falls back once in sequential mode',
async (replica) => {
const client = new FixtureDatabaseClient(replica, [{ id: 2 }]);
expect(
await client.tryHardRead('SELECT 1', [], { race: false }),
).toEqual([{ id: 2 }]);
expect(client.preads).toEqual(['SELECT 1']);
},
);
it('returns replica rows without waiting on the primary', async () => {
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
@@ -256,6 +275,28 @@ describe('AbstractDatabaseClient — SQL generation', () => {
expect(postgres().timestampFromUnixSql()).toBe('to_timestamp(?)');
});
it('reads a timestamp column back as unix seconds in each engine', () => {
expect(sqlite().unixSecondsSql('`t`')).toBe(
"CAST(strftime('%s', `t`) AS INTEGER)",
);
expect(mysql().unixSecondsSql('`t`')).toBe('UNIX_TIMESTAMP(`t`)');
expect(postgres().unixSecondsSql('`t`')).toBe(
'EXTRACT(EPOCH FROM `t`)::bigint',
);
});
it('reads the database clock as epoch milliseconds in each dialect', () => {
expect(sqlite().nowMsExpr()).toBe(
"CAST((julianday('now') - 2440587.5) * 86400000 AS INTEGER)",
);
expect(mysql().nowMsExpr()).toBe(
"(TIMESTAMPDIFF(MICROSECOND, '1970-01-01 00:00:00', UTC_TIMESTAMP(6)) DIV 1000)",
);
expect(postgres().nowMsExpr()).toBe(
'CAST(EXTRACT(EPOCH FROM clock_timestamp()) * 1000 AS BIGINT)',
);
});
it('describes a stored timestamp the way each driver reads it back', () => {
const seconds = Date.UTC(2026, 9, 11, 8, 5, 9) / 1000;
expect(sqlite().timestampValue(seconds)).toBe('2026-10-11 08:05:09');
@@ -277,3 +318,13 @@ describe('AbstractDatabaseClient — SQL generation', () => {
);
});
});
it('builds an insert for batchWrite and shares it with insert', async () => {
const client = new FixtureDatabaseClient([]);
const entry = client.insertEntry('widget', { name: "reader's", enabled: true });
expect(entry.values).toEqual(["reader's", true]);
expect(entry.statement).toContain('VALUES (?, ?)');
await client.insert('widget', { name: "reader's", enabled: true });
expect(client.writes).toEqual([{ query: entry.statement, params: entry.values }]);
});
+51 -12
View File
@@ -110,14 +110,23 @@ export class AbstractDatabaseClient extends PuterClient {
tableName: string,
data: Record<string, unknown>,
): Promise<WriteResult> {
const entry = this.insertEntry(tableName, data);
return this.write(
entry.statement + this.returningIdClause(),
entry.values,
);
}
/** An insert that can join other writes in a transaction. */
insertEntry(tableName: string, data: Record<string, unknown>): BatchEntry {
const cols = Object.keys(data);
const values = Object.values(data);
const sql =
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
return {
statement:
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`,
values: Object.values(data),
};
}
/** Whether `read()` goes to a separate replica rather than the primary. */
@@ -138,25 +147,27 @@ export class AbstractDatabaseClient extends PuterClient {
/**
* Like `read()`, but an empty or failed replica read falls back to the
* primary, so replication lag can't hide a row that was just written. The
* primary read starts alongside the replica one rather than after it.
* Without a replica this is a single `read()`.
* primary, so replication lag can't hide a row that was just written. By
* default the primary read starts alongside the replica one; `race: false`
* reads the primary only after the replica returns empty or fails. Without
* a replica this is a single `read()`.
*/
@Span('db.tryHardRead', (query: string) => ({ 'db.statement': query }))
async tryHardRead(
query: string,
params: unknown[] = [],
{ race = true }: { race?: boolean } = {},
): Promise<Record<string, unknown>[]> {
if (!this.hasReadReplica()) return this.read(query, params);
const primary = settle(this.pread(query, params));
const primary = race ? settle(this.pread(query, params)) : null;
try {
const rows = await this.replicaRead(query, params);
if (rows.length > 0) return rows;
} catch {
// replica failed — fall through to primary
}
const result = await primary;
const result = await (primary ?? settle(this.pread(query, params)));
if ('error' in result) throw result.error;
return result.value;
}
@@ -386,6 +397,34 @@ export class AbstractDatabaseClient extends PuterClient {
});
}
/**
* A timestamp column as unix seconds, computed by the database: the mysql
* driver reads a stored UTC timestamp as local time, so parsing it in JS
* can be off by the server's UTC offset.
*/
unixSecondsSql(column: string): string {
return this.case({
postgres: `EXTRACT(EPOCH FROM ${column})::bigint`,
mysql: `UNIX_TIMESTAMP(${column})`,
otherwise: `CAST(strftime('%s', ${column}) AS INTEGER)`,
});
}
/**
* The database's own clock as epoch milliseconds, so nodes whose clocks
* disagree still agree on it. The mysql form ignores the session time zone,
* so a DST change can't shift it.
*/
nowMsExpr(): string {
return this.case({
sqlite: "CAST((julianday('now') - 2440587.5) * 86400000 AS INTEGER)",
postgres:
'CAST(EXTRACT(EPOCH FROM clock_timestamp()) * 1000 AS BIGINT)',
otherwise:
"(TIMESTAMPDIFF(MICROSECOND, '1970-01-01 00:00:00', UTC_TIMESTAMP(6)) DIV 1000)",
});
}
/**
* What a read of a timestamp column holding `seconds` returns: sqlite gives
* back the UTC text it stored, mysql2 and pg a `Date` read in node's time
@@ -217,7 +217,8 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
return this.configuration === Configuration.REPLICA;
}
// No failover: `tryHardRead` already has a primary read in flight.
// No failover: `tryHardRead` falls back to the primary on an empty result
// or error (raced alongside by default, after the replica with `race: false`).
protected override async replicaRead(
query: string,
params: unknown[],
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 90;
const CURRENT_SCHEMA_VERSION = 98;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -673,6 +673,38 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
});
});
it('adds kv_share_handles.notify from 0099, off for every existing share', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-notify-'));
const path = join(dir, 'puter.sqlite');
try {
const before = new SqliteDatabaseClient(
sqliteConfig({ inMemory: false, path, targetVersion: 94 }),
);
await before.onServerStart();
await before.write(
'INSERT INTO `kv_share_handles` (`handle`, `owner_user_id`, ' +
'`grantee_user_id`, `app_uid`, `key_prefix`, `permission`, ' +
'`created_at`) VALUES (?, 1, 1, ?, ?, ?, 0)',
['kvh-before-notify', 'os-global', 'a:', 'kv-share:x'],
);
before.onServerShutdown();
const after = new SqliteDatabaseClient(
sqliteConfig({ inMemory: false, path }),
);
await after.onServerStart();
await expect(
after.read(
'SELECT `notify` FROM `kv_share_handles` WHERE `handle` = ?',
['kvh-before-notify'],
),
).resolves.toEqual([{ notify: 0 }]);
after.onServerShutdown();
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('indexes notification.created_at for the retention sweep', async () => {
const rows = await client.read('PRAGMA index_list(`notification`)');
expect(rows.map((r) => String(r.name))).toContain(
@@ -680,6 +712,97 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
);
});
it('indexes notification.replaced_at per mailbox slice for catch-up', async () => {
const columns = await client.read(
'PRAGMA index_info(`idx_notification_replaced`)',
);
expect(columns.map((c) => String(c.name))).toEqual([
'user_id',
'audience',
'app_uid',
'replaced_at',
]);
});
it('keeps notification.send_key unique', async () => {
const rows = await client.read('PRAGMA index_list(`notification`)');
expect(
rows.find((r) => r.name === 'idx_notification_send_key'),
).toMatchObject({ unique: 1 });
const columns = await client.read(
'PRAGMA index_info(`idx_notification_send_key`)',
);
expect(columns.map((c) => String(c.name))).toEqual(['send_key']);
});
it('creates event_user_handlers with its unique and holder indexes', async () => {
const columns = await client.read(
'PRAGMA table_info(`event_user_handlers`)',
);
expect(columns.map((c) => String(c.name))).toEqual([
'id',
'app_uid',
'holder_user_id',
'name',
'source',
'source_hash',
'created_at',
'updated_at',
'scope_since',
]);
const indexes = await client.read(
'PRAGMA index_list(`event_user_handlers`)',
);
expect(
indexes.find(
(i) => i.name === 'idx_event_user_handlers_app_holder_name',
),
).toMatchObject({ unique: 1 });
expect(
indexes.find((i) => i.name === 'idx_event_user_handlers_holder_app'),
).toMatchObject({ unique: 0 });
const columnsOf = async (index: string) =>
(await client.read(`PRAGMA index_info(\`${index}\`)`)).map((c) =>
String(c.name),
);
expect(await columnsOf('idx_event_user_handlers_app_holder_name')).toEqual(
['app_uid', 'holder_user_id', 'name'],
);
expect(await columnsOf('idx_event_user_handlers_holder_app')).toEqual([
'holder_user_id',
'app_uid',
]);
});
it('leaves event_handlers as it was', async () => {
const columns = await client.read('PRAGMA table_info(`event_handlers`)');
expect(columns.map((c) => String(c.name))).toEqual([
'id',
'app_uid',
'name',
'source',
'source_hash',
'created_at',
'updated_at',
]);
});
it('adds event_subscriptions.handler_scope, app scope for existing rows', async () => {
const columns = await client.read(
'PRAGMA table_info(`event_subscriptions`)',
);
expect(columns.find((c) => c.name === 'handler_scope')).toMatchObject({
notnull: 1,
dflt_value: '0',
});
const indexes = await client.read(
'PRAGMA index_list(`event_subscriptions`)',
);
expect(indexes.map((i) => String(i.name))).toContain(
'idx_event_subscriptions_holder',
);
});
it('leaves an already-migrated database untouched on a second boot', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
const path = join(dir, 'nested', 'puter.sqlite');
@@ -129,6 +129,14 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[87, ['0092_share-entry-owner.sql']],
[88, ['0093_fsentries-parent-uid-index.sql']],
[89, ['0094_deleted-apps.sql']],
[90, ['0095_kv-share-handle-levels.sql']],
[91, ['0096_kv-share-handle-quota-indexes.sql']],
[92, ['0097_user-to-user-holder-permission-index.sql']],
[93, ['0098_notification-groups.sql']],
[94, ['0099_kv-share-handle-notify.sql']],
[95, ['0100_notification-send-key.sql']],
[96, ['0101_user-scoped-handlers.sql']],
[97, ['0102_notification-scope-id-index.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,50 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share levels on `kv_share_handles`. See
-- sqlite/0095_kv-share-handle-levels.sql for the rationale. No per-file
-- applied-state tracking, so every step is guarded and a replay does nothing.
CALL _puter_add_col('kv_share_handles', 'revoked_by', '`revoked_by` varchar(8) DEFAULT NULL');
CALL _puter_add_col('kv_share_handles', 'label', '`label` varchar(512) DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_kv_share_handle_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_grantee'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_grantee` (`grantee_user_id`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_app'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_app` (`app_uid`);
END IF;
END//
DELIMITER ;
CALL _puter_add_kv_share_handle_indexes();
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_indexes;
@@ -0,0 +1,72 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share-handle quota, listing and purge indexes. See
-- sqlite/0096_kv-share-handle-quota-indexes.sql for what each serves. No
-- per-file applied-state tracking, so every index is guarded and a replay does
-- nothing.
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_quota_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_kv_share_handle_quota_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_owner_live'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_owner_live` (`owner_user_id`, `revoked_at`, `app_uid`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_owner_app'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_owner_app` (`owner_user_id`, `app_uid`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_grantee_app'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_grantee_app` (`grantee_user_id`, `app_uid`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_pair'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_pair` (`owner_user_id`, `grantee_user_id`, `revoked_at`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'kv_share_handles'
AND INDEX_NAME = 'idx_kv_share_handles_revoked'
) THEN
ALTER TABLE `kv_share_handles` ADD INDEX `idx_kv_share_handles_revoked` (`revoked_at`);
END IF;
END//
DELIMITER ;
CALL _puter_add_kv_share_handle_quota_indexes();
DROP PROCEDURE IF EXISTS _puter_add_kv_share_handle_quota_indexes;
@@ -0,0 +1,38 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Lets a holder's grant check read only the permissions it names. Guarded, so
-- a replay does nothing.
DROP PROCEDURE IF EXISTS _puter_add_u2u_holder_permission_index;
DELIMITER //
CREATE PROCEDURE _puter_add_u2u_holder_permission_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_to_user_permissions'
AND INDEX_NAME = 'idx_user_to_user_permissions_holder_permission'
) THEN
ALTER TABLE `user_to_user_permissions` ADD INDEX `idx_user_to_user_permissions_holder_permission` (`holder_user_id`, `permission`);
END IF;
END//
DELIMITER ;
CALL _puter_add_u2u_holder_permission_index();
DROP PROCEDURE IF EXISTS _puter_add_u2u_holder_permission_index;
@@ -0,0 +1,53 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `notification`.`group_key` and `replaced_at`. See
-- sqlite/0098_notification-groups.sql for the rationale. The key is a hex
-- digest, so ascii and compared byte for byte. No per-file applied-state
-- tracking, so every step is guarded and a replay does nothing.
CALL _puter_add_col('notification', 'group_key', '`group_key` char(64) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL');
CALL _puter_add_col('notification', 'replaced_at', '`replaced_at` bigint DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_notification_group_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_group_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_group'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_group`
(`user_id`, `app_uid`, `group_key`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_replaced'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_replaced`
(`user_id`, `audience`, `app_uid`, `replaced_at`);
END IF;
END//
DELIMITER ;
CALL _puter_add_notification_group_indexes();
DROP PROCEDURE IF EXISTS _puter_add_notification_group_indexes;
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Notifications through a share handle. See
-- sqlite/0099_kv-share-handle-notify.sql for the column. No per-file
-- applied-state tracking, so it goes through _puter_add_col and a replay does
-- nothing.
CALL _puter_add_col('kv_share_handles', 'notify', '`notify` tinyint(1) NOT NULL DEFAULT 0');
@@ -0,0 +1,43 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `notification`.`send_key`. See sqlite/0100_notification-send-key.sql for the
-- rationale. The key is a uuid, so ascii and compared byte for byte. No
-- per-file applied-state tracking, so every step is guarded and a replay does
-- nothing.
CALL _puter_add_col('notification', 'send_key', '`send_key` char(36) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_notification_send_key_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_send_key_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_send_key'
) THEN
ALTER TABLE `notification` ADD UNIQUE KEY `idx_notification_send_key`
(`send_key`), ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_notification_send_key_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_send_key_index;
@@ -0,0 +1,66 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Handlers an app's user published into their own scope, kept apart from
-- `event_handlers` so code that predates them never reads them. See
-- sqlite/0101_user-scoped-handlers.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- and a guarded column add. There is no per-file applied-state tracking, so a
-- replay has to be a no-op.
CREATE TABLE IF NOT EXISTS `event_user_handlers` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`holder_user_id` int unsigned NOT NULL,
`name` varchar(128) NOT NULL,
`source` mediumtext NOT NULL,
`source_hash` char(64) NOT NULL,
`created_at` bigint NOT NULL,
`updated_at` bigint NOT NULL,
-- When the scope first held a handler; kept across handler churn, so the
-- rent sweep sees the scope's age, not its oldest remaining handler's.
`scope_since` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_user_handlers_app_holder_name` (`app_uid`, `holder_user_id`, `name`),
KEY `idx_event_user_handlers_holder_app` (`holder_user_id`, `app_uid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Guarded here rather than through `_puter_add_col`, so the file also applies
-- on its own to a database that never ran the earlier migrations.
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_handler_scope;
DELIMITER //
CREATE PROCEDURE _puter_add_event_subscriptions_handler_scope()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'handler_scope'
) THEN
ALTER TABLE `event_subscriptions`
ADD COLUMN `handler_scope` tinyint unsigned NOT NULL DEFAULT 0,
ALGORITHM=INSTANT;
END IF;
END//
DELIMITER ;
CALL _puter_add_event_subscriptions_handler_scope();
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_handler_scope;
@@ -0,0 +1,41 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `notification` (`user_id`, `audience`, `app_uid`, `id`). See
-- sqlite/0102_notification-scope-id-index.sql for the rationale. No per-file
-- applied-state tracking, so the index is added only when missing and a
-- replay does nothing.
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_id_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_scope_id_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_scope_id'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_scope_id`
(`user_id`, `audience`, `app_uid`, `id`), ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_notification_scope_id_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_id_index;
@@ -0,0 +1,29 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share levels on kv_share_handles. See
-- sqlite/0095_kv-share-handle-levels.sql for the rationale. Idempotent via
-- IF NOT EXISTS; there is no per-file applied-state tracking.
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS revoked_by VARCHAR(8) DEFAULT NULL;
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS label VARCHAR(512) DEFAULT NULL;
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_grantee
ON kv_share_handles (grantee_user_id, id);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_app
ON kv_share_handles (app_uid);
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Share-handle quota, listing and purge indexes. See
-- sqlite/0096_kv-share-handle-quota-indexes.sql for what each serves.
-- Idempotent via IF NOT EXISTS; there is no per-file applied-state tracking.
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_owner_live
ON kv_share_handles (owner_user_id, revoked_at, app_uid);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_owner_app
ON kv_share_handles (owner_user_id, app_uid, id);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_grantee_app
ON kv_share_handles (grantee_user_id, app_uid, id);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_pair
ON kv_share_handles (owner_user_id, grantee_user_id, revoked_at);
CREATE INDEX IF NOT EXISTS idx_kv_share_handles_revoked
ON kv_share_handles (revoked_at);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Lets a holder's grant check read only the permissions it names.
CREATE INDEX IF NOT EXISTS idx_user_to_user_permissions_holder_permission
ON user_to_user_permissions (holder_user_id, permission);
@@ -0,0 +1,29 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- notification.group_key and replaced_at. See
-- sqlite/0098_notification-groups.sql for the rationale. Idempotent via IF NOT
-- EXISTS.
ALTER TABLE notification ADD COLUMN IF NOT EXISTS group_key varchar(64) DEFAULT NULL;
ALTER TABLE notification ADD COLUMN IF NOT EXISTS replaced_at bigint DEFAULT NULL;
CREATE INDEX IF NOT EXISTS idx_notification_group
ON notification (user_id, app_uid, group_key);
CREATE INDEX IF NOT EXISTS idx_notification_replaced
ON notification (user_id, audience, app_uid, replaced_at);
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Notifications through a share handle. See
-- sqlite/0099_kv-share-handle-notify.sql for the column. Idempotent via
-- IF NOT EXISTS; there is no per-file applied-state tracking.
ALTER TABLE kv_share_handles ADD COLUMN IF NOT EXISTS notify smallint NOT NULL DEFAULT 0;
@@ -0,0 +1,24 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- notification.send_key. See sqlite/0100_notification-send-key.sql for the
-- rationale. Idempotent via IF NOT EXISTS.
ALTER TABLE notification ADD COLUMN IF NOT EXISTS send_key char(36) DEFAULT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_notification_send_key
ON notification (send_key);
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Handlers an app's user published into their own scope. See
-- sqlite/0101_user-scoped-handlers.sql for the column rationale.
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS event_user_handlers (
id BIGSERIAL PRIMARY KEY,
app_uid VARCHAR(40) NOT NULL,
holder_user_id INTEGER NOT NULL,
name VARCHAR(128) NOT NULL,
source TEXT NOT NULL,
source_hash VARCHAR(64) NOT NULL,
created_at BIGINT NOT NULL,
updated_at BIGINT NOT NULL,
scope_since BIGINT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_event_user_handlers_app_holder_name
ON event_user_handlers (app_uid, holder_user_id, name);
CREATE INDEX IF NOT EXISTS idx_event_user_handlers_holder_app
ON event_user_handlers (holder_user_id, app_uid);
ALTER TABLE event_subscriptions ADD COLUMN IF NOT EXISTS handler_scope smallint NOT NULL DEFAULT 0;
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- notification (user_id, audience, app_uid, id). See
-- sqlite/0102_notification-scope-id-index.sql for the rationale. Idempotent via
-- IF NOT EXISTS.
CREATE INDEX IF NOT EXISTS idx_notification_scope_id
ON notification (user_id, audience, app_uid, id);
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- What share levels need from `kv_share_handles`. The level itself is not a
-- column: it is the root of the stored `permission`.
--
-- - `revoked_by` : `owner`, `holder` or `system` (the namespace app was
-- deleted). Null while live, and on rows retired before
-- this existed.
-- - `label` : optional owner-set text the holder is shown.
-- - the grantee index serves the holder's listing and their unshare, and
-- the app index retiring a deleted app's handles.
ALTER TABLE `kv_share_handles` ADD COLUMN `revoked_by` TEXT DEFAULT NULL;
ALTER TABLE `kv_share_handles` ADD COLUMN `label` TEXT DEFAULT NULL;
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_grantee`
ON `kv_share_handles` (`grantee_user_id`, `id`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_app`
ON `kv_share_handles` (`app_uid`);
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Indexes behind the share-handle quotas, the app-filtered listings and the
-- retention purge on `kv_share_handles`.
--
-- - owner_live : the owner's live count, in total and per app, read off
-- the index alone.
-- - owner_app : the owner's listing narrowed to one app, in id order.
-- - grantee_app : the holder's listing narrowed to one app, in id order.
-- - pair : live handles one owner holds out to one holder, counted at
-- mint and walked on revoke and unshare.
-- - revoked : retired rows past retention, for the purge.
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_owner_live`
ON `kv_share_handles` (`owner_user_id`, `revoked_at`, `app_uid`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_owner_app`
ON `kv_share_handles` (`owner_user_id`, `app_uid`, `id`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_grantee_app`
ON `kv_share_handles` (`grantee_user_id`, `app_uid`, `id`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_pair`
ON `kv_share_handles` (`owner_user_id`, `grantee_user_id`, `revoked_at`);
CREATE INDEX IF NOT EXISTS `idx_kv_share_handles_revoked`
ON `kv_share_handles` (`revoked_at`);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Lets a holder's grant check read only the permissions it names.
CREATE INDEX IF NOT EXISTS `idx_user_to_user_permissions_holder_permission`
ON `user_to_user_permissions` (`holder_user_id`, `permission`);
@@ -0,0 +1,38 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- What app message groups need from `notification`:
--
-- - `group_key` : a sha256 hex digest of an app message's `group`, so the
-- notification a newer one replaces is found by index.
-- Null on every other row, and on rows written before
-- this, which therefore never match.
-- - `replaced_at` : epoch milliseconds of the last replacement, null for a
-- row never replaced. Live and fetched copies carry it as
-- their version, so a client can tell a replacement from a
-- copy it already has. Indexed per mailbox slice, so a
-- catch-up reads the rows replaced since its cursor
-- without scanning the ones that never were.
ALTER TABLE `notification` ADD COLUMN `group_key` TEXT DEFAULT NULL;
ALTER TABLE `notification` ADD COLUMN `replaced_at` INTEGER DEFAULT NULL;
CREATE INDEX IF NOT EXISTS `idx_notification_group`
ON `notification` (`user_id`, `app_uid`, `group_key`);
CREATE INDEX IF NOT EXISTS `idx_notification_replaced`
ON `notification` (`user_id`, `audience`, `app_uid`, `replaced_at`);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Whether the two parties of a share may send each other notifications through
-- its handle. Off by default, so every existing share reads as not allowing
-- them.
ALTER TABLE `kv_share_handles`
ADD COLUMN `notify` INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,27 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `notification`.`send_key`: the key of the delayed send (a `deferred`
-- schedule) that wrote this row or last replaced it, derived from the
-- schedule and its slot. Unique, so one slot can never write two rows, and a
-- send that crashed after writing finds its row again by it. Null on every row
-- a delayed send never touched.
ALTER TABLE `notification` ADD COLUMN `send_key` TEXT DEFAULT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_notification_send_key`
ON `notification` (`send_key`);
@@ -0,0 +1,48 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Handlers an app's user published into their own scope, in a table of their
-- own so `event_handlers` stays the app owner's set. Same columns as
-- `event_handlers` (see 0076_event-handlers.sql) plus the holder.
--
-- - `holder_user_id` : the user whose scope this is. A name is unique per
-- (app, holder), so the same name can live in the app's
-- set and in each holder's.
-- - `(holder_user_id, app_uid)` : a holder's scopes, for listing and for the
-- rent count.
--
-- `event_subscriptions.handler_scope` says which table a row's `handler_name`
-- refers to: 0 the app's set, 1 the row holder's scope.
CREATE TABLE IF NOT EXISTS `event_user_handlers` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"app_uid" TEXT NOT NULL,
"holder_user_id" INTEGER NOT NULL,
"name" TEXT NOT NULL,
"source" TEXT NOT NULL,
"source_hash" TEXT NOT NULL,
"created_at" INTEGER NOT NULL, -- unix seconds
"updated_at" INTEGER NOT NULL,
"scope_since" INTEGER NOT NULL -- unix seconds; the scope's first handler, kept across churn
);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_event_user_handlers_app_holder_name`
ON `event_user_handlers` (`app_uid`, `holder_user_id`, `name`);
CREATE INDEX IF NOT EXISTS `idx_event_user_handlers_holder_app`
ON `event_user_handlers` (`holder_user_id`, `app_uid`);
ALTER TABLE `event_subscriptions` ADD COLUMN `handler_scope` INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index a mailbox slice by row id, so a fetch finds the newest row of a slice,
-- and pages forward through one, with a seek rather than a scan of the slice.
CREATE INDEX IF NOT EXISTS `idx_notification_scope_id`
ON `notification` (`user_id`, `audience`, `app_uid`, `id`);
@@ -188,6 +188,47 @@ describe('DDBClient — item operations', () => {
).toBeUndefined();
});
it('puts only while its condition holds', async () => {
const absent = {
expression: 'attribute_not_exists(#pk)',
names: { '#pk': 'pk' },
};
await client.put(
TABLE,
{ pk: 'u7', sk: 'once', v: 1 },
{
condition: absent,
},
);
await expect(
client.put(
TABLE,
{ pk: 'u7', sk: 'once', v: 2 },
{
condition: absent,
},
),
).rejects.toMatchObject({ name: 'ConditionalCheckFailedException' });
expect(
(await client.get(TABLE, { pk: 'u7', sk: 'once' })).Item?.v,
).toBe(1);
await client.put(
TABLE,
{ pk: 'u7', sk: 'once', v: 3 },
{
condition: {
expression: '#v = :held',
names: { '#v': 'v' },
values: { ':held': 1 },
},
},
);
expect(
(await client.get(TABLE, { pk: 'u7', sk: 'once' })).Item?.v,
).toBe(3);
});
it('fetches keys from several tables in one batch', async () => {
await client.put(TABLE, { pk: 'b1', sk: 'a', v: 1 });
await client.put(TABLE, { pk: 'b1', sk: 'b', v: 2 });
+39 -1
View File
@@ -252,7 +252,19 @@ export class DDBClient extends PuterClient {
}
@Span('ddb.put', (table: string) => ({ 'db.table': table }))
async put<T extends Record<string, unknown>>(table: string, item: T) {
async put<T extends Record<string, unknown>>(
table: string,
item: T,
opts?: {
/** Refused (`ConditionalCheckFailedException`) unless it holds. */
condition?: {
expression: string;
names?: Record<string, string>;
values?: Record<string, unknown>;
};
},
) {
const condition = opts?.condition;
const client = await this.#getDocumentClient();
return sendRepairingValues(
item,
@@ -262,6 +274,23 @@ export class DDBClient extends PuterClient {
TableName: table,
Item: itemToWrite,
ReturnConsumedCapacity: 'TOTAL',
...(condition
? {
ConditionExpression: condition.expression,
...(condition.names
? {
ExpressionAttributeNames:
condition.names,
}
: {}),
...(condition.values
? {
ExpressionAttributeValues:
condition.values,
}
: {}),
}
: {}),
}),
),
() => `a put to ${table}`,
@@ -505,6 +534,8 @@ export class DDBClient extends PuterClient {
scanIndexForward?: boolean;
beginsWith?: { key: string; value: string };
select?: 'COUNT';
/** Attributes to return instead of the whole item. */
projection?: string[];
filter?: {
expression: string;
values?: Record<string, unknown>;
@@ -545,6 +576,10 @@ export class DDBClient extends PuterClient {
Object.assign(expressionAttributeValues, options.filter.values);
Object.assign(expressionAttributeNames, options.filter.names);
}
const projection = (options?.projection ?? []).map((name) => {
expressionAttributeNames[`#p_${name}`] = name;
return `#p_${name}`;
});
const command = new QueryCommand({
TableName: table,
@@ -559,6 +594,9 @@ export class DDBClient extends PuterClient {
? { FilterExpression: options.filter.expression }
: {}),
...(options?.select ? { Select: options.select } : {}),
...(projection.length > 0
? { ProjectionExpression: projection.join(', ') }
: {}),
...(options?.scanIndexForward !== undefined
? { ScanIndexForward: options.scanIndexForward }
: {}),
+175 -9
View File
@@ -75,6 +75,15 @@ export type EventMap = {
serverStart: Record<string, never>;
serverPrepareShutdown: Record<string, never>;
serverShutdown: Record<string, never>;
/**
* Emitted by a host extension that knows this server will restart soon.
* Long-lived connections ask their clients to reconnect; nothing is refused
* or closed, so a false alarm is harmless.
*/
serverRestartAnnounced: {
/** How long until the server stops, when known. */
closeInMs?: number;
};
// ---- AI ----
'ai.prompt.validate': {
@@ -108,7 +117,7 @@ export type EventMap = {
model_used: string;
service_used: string;
intended_service: string;
model_details: {
model_details?: {
id: string;
provider: string;
input_cost_key: string;
@@ -137,6 +146,17 @@ export type EventMap = {
};
'app.new-icon': { app_uid: string; data_url: string };
'app.opened': { app_uid: string; user_id: number; ts: number };
/**
* A user removed an app: its grants and worker sessions are gone. With
* `deleteData`, the user asked for the app's data in their account to go
* too. Awaited by the emitter.
*/
'app.uninstalled': {
userId: number;
userUuid: string;
appUid: string;
deleteData: boolean;
};
'app.rename': {
app_uid: string;
old_name: string;
@@ -180,6 +200,16 @@ export type EventMap = {
};
// ---- Auth / signup ----
/**
* Which signup methods are open, emitted via `emitAndWait` before every
* signup and whenever the GUI reads the provider list. A listener sets
* `oidcOnly` to refuse form and temp signups, leaving OIDC and team
* provisioning open; `message` is what a refused form signup is told.
*/
'puter.signup.policy': {
oidcOnly: boolean;
message: string | null;
};
'puter.signup.validate': {
allow: boolean;
email?: string;
@@ -197,7 +227,7 @@ export type EventMap = {
*/
is_temp?: boolean;
ip?: string | null;
source?: 'oidc';
source?: 'oidc' | 'team_provision';
req?: unknown;
data?: unknown;
abuse?: unknown;
@@ -257,7 +287,7 @@ export type EventMap = {
ip?: string | null;
fingerprint?: string | null;
reputation?: number | null;
source?: 'oidc';
source?: 'oidc' | 'team_provision';
requires_phone_verification: boolean;
requires_card_verification: boolean;
accepted: boolean;
@@ -303,6 +333,8 @@ export type EventMap = {
// the abuse extension cap sends per device (across accounts) so a
// device farm can't dodge the per-account cap with fresh signups.
device_fingerprint: string | null;
// Browser-signals dispatch id the client sent, or null.
dispatch_id: string | null;
allowed: boolean;
reason: string | null;
// With `allowed` false, offer the card fallback in place of SMS. Only
@@ -389,6 +421,9 @@ export type EventMap = {
new_username: string;
};
'user.email-changed': { user_id: number; new_email: string };
/** An account was suspended, or its suspension lifted. Local events. */
'user.suspended': { userId: number };
'user.unsuspended': { userId: number };
// Fired after an account is torn down (self-serve, admin, or temp-user
// logout cleanup). Listeners purge external state tied to the account —
// e.g. the marketplace extension cancels the user's Stripe subscriptions.
@@ -397,6 +432,8 @@ export type EventMap = {
user_id: number;
user_uuid?: string;
stripe_customer_id?: string | null;
/** Apps the account owned, which cascaded away with it. */
appUids?: string[];
};
// ---- Team billing ---- the trigger, never the charge ----
@@ -490,15 +527,43 @@ export type EventMap = {
noShareKeys?: string[];
/** Events handler runs behind the write; see `Actor.handlerDepth`. */
handlerDepth?: number;
/**
* The user who made the change. Differs from `userId` when a share
* holder wrote into the owner's namespace.
*/
actingUserId?: number;
/** Made by a worker session or an events handler; see `viaWorkerOf`. */
viaWorker?: true;
};
/**
* A whole namespace was emptied. Namespace-level on purpose: `flush`'s own
* key enumeration is truncated for a large namespace, so the keys it saw
* are not the keys it removed.
*/
'kv.flushed': { namespace: string; userId: number };
'kv.flushed': {
namespace: string;
userId: number;
actingUserId?: number;
viaWorker?: true;
};
/**
* Share handles stopped being addresses, whoever retired them. `userId` is
* one user who could have subscribed through them, their owner or their
* holder. Listeners end what was made through them even where another grant
* still covers the region. Local, like `permission.revoked`: one region
* settling covers every other.
*/
'kv.share.retired': { handles: string[]; userId: number };
// ---- Notifications ----
/** An app message was acknowledged after its state committed. */
'notif.acknowledged': {
userId: number;
uid: string;
type: string;
audience: string;
appUid: string;
};
/**
* A notification row exists. Emitted after the insert commits — nothing
* pushes a uid before there is a row to name — and carries the scope tuple
@@ -513,9 +578,32 @@ export type EventMap = {
appUid: string | null;
value: Record<string, unknown>;
createdAt: number;
/** An earlier notification replaced in place; `createdAt` is when. */
replaced?: true;
/** Events handler runs behind whatever created it. */
handlerDepth?: number;
};
/**
* Every non-silent write or update, only on the node that wrote it. The
* value is the listener's own copy of what the socket carries.
*/
'notif.written': {
userId: number;
uid: string;
type: string;
audience: string;
/** The row's app; null when it names none or an update could not tell. */
appUid: string | null;
value: Record<string, unknown>;
/** An earlier notification rewritten in place. */
updated?: true;
/** Replaced an earlier notification in its group. */
replaced?: true;
/** Names this one send, for a receipt asked about it. */
sendId: string;
/** The same `sendId` may be emitted again, as a delayed send is. */
replayable?: true;
};
/**
* One notification, addressed at whatever sockets each region holds for the
* recipient. `outer.*` reaches peer regions and never sibling nodes, so
@@ -524,9 +612,43 @@ export type EventMap = {
*/
'outer.notif.delivery': {
userId: number;
wire: 'notif.message' | 'notif.unreads' | 'notif.ack';
wire: 'notif.message' | 'notif.unreads' | 'notif.ack' | 'notif.removed';
response: unknown;
};
/**
* How one live notification went, on the node that asked for its receipt:
* from the desktop's sockets, or with `appUid` the app's own.
*/
'notif.delivery.settled': {
sendId?: string;
uid: string;
userId: number;
type: string;
/** The app whose own sockets were asked; absent for the desktop's. */
appUid?: string;
outcome: NotifDeliveryOutcome;
/** A socket in this region acknowledges the receipt. */
ackCapable: boolean;
/** No socket here is an older client that acknowledges nothing. */
everySocketAcks: boolean;
/** Every node here answered the ask, not just this one. */
complete: boolean;
/** A peer region holds a socket for the user. */
peerRegions: boolean;
responders: NotifDeliveryResponder[];
/** A page here got it live while visible and focused. */
handled: boolean;
};
/**
* A page reported that it got a notification live while visible and
* focused. Node-local; `appUid` is the reporting socket's app, absent for
* the desktop.
*/
'notif.handled': {
userId: number;
appUid?: string;
uid: string;
};
// ---- Metering ----
// Recurring charges are pure mechanism here: the metering service knows
@@ -576,9 +698,18 @@ export type EventMap = {
// and going away (1→0, whether by removing the last handler or by the
// destroy route). `actor.user` is the app's owner, not the caller — a
// developer session publishing for an app it owns is the common case, but
// billing follows ownership.
'events.worker.create': { actor: Actor; appUid: string };
'events.worker.destroy': { actor: Actor; appUid: string };
// billing follows ownership. For `scope: 'user'` the unit is one user's
// handlers in one app instead, and `actor.user` is that user.
'events.worker.create': {
actor: Actor;
appUid: string;
scope: 'app' | 'user';
};
'events.worker.destroy': {
actor: Actor;
appUid: string;
scope: 'app' | 'user';
};
// ---- Outer / GUI broadcast ----
'outer.cacheUpdate': {
@@ -643,6 +774,13 @@ export type EventMap = {
* reason as the events generation bump: a per-process cache.
*/
'outer.pubsub.share.ownerShared': { ownerUserId: number };
/**
* Share handles were retired or changed, so every process must drop its
* cached copy. `outer.pubsub.*` because the cache is a per-process map.
*/
'outer.pubsub.kv.share.forget': { handles: string[] };
/** The same, for every cached handle one owner holds out. */
'outer.pubsub.kv.share.forgetOwner': { ownerUserId: number };
'outer.fs.write-hash': { hash: string; uuid: string };
/**
* Cache keys the KV read cache must stop serving, because the entries
@@ -661,6 +799,7 @@ export type EventMap = {
'outer.gui.item.renamed': GuiEvent;
'outer.gui.notif.ack': GuiEvent<{ uid: string }>;
'outer.gui.notif.message': GuiEvent<{ uid: string; notification: unknown }>;
'outer.gui.notif.removed': GuiEvent<{ uid: string }>;
'outer.gui.notif.unreads': GuiEvent<{
unreads: { uid: string; notification: unknown }[];
}>;
@@ -668,7 +807,7 @@ export type EventMap = {
// ---- Subdomains ----
// `uid` is the row uuid — for `delete` it is the only surviving handle a
// listener can use to find state keyed to the row after it is gone.
'subdomain.delete': { subdomain: string; uid?: string };
'subdomain.delete': { subdomain: string; uid?: string; userId: number };
'subdomain.update': { subdomain: string; uid?: string };
'site.htmlServed': {
subdomain: string;
@@ -734,6 +873,12 @@ export type EventMap = {
/** One access token was revoked; only its own connections should drop. */
'auth.access-token.revoked': { token_uid: string };
/** Same-cluster eviction for raw realtime WebSockets. */
'pubsub.realtime.auth-revoked': {
sessionUids?: string[];
tokenUid?: string;
};
/**
* A grant was withdrawn, so whatever was standing on it has to be settled
* rather than left to fail its next check.
@@ -856,6 +1001,9 @@ export type AiCostFactorEvent = {
*/
export type LifecyclePhase = 'before' | 'after' | 'error' | 'reject';
/** Status a vetoing `before` listener may answer with. */
export type LifecycleRejectStatus = 403 | 429;
/**
* Payload for `driver.<iface>.<method>.<phase>` events.
*
@@ -885,6 +1033,8 @@ export type DriverMethodLifecycleEvent = {
allow?: boolean;
/** Optional human-readable reason surfaced to the caller when vetoed. */
rejectReason?: string;
/** Status for a veto: 403 (default) or 429 for a throttle. */
rejectStatus?: LifecycleRejectStatus;
};
/**
@@ -919,6 +1069,8 @@ export type RouteLifecycleEvent = {
allow?: boolean;
/** Optional human-readable reason surfaced to the caller when vetoed. */
rejectReason?: string;
/** Status for a veto: 403 (default) or 429 for a throttle. */
rejectStatus?: LifecycleRejectStatus;
};
export type EventKey = keyof EventMap & string;
@@ -940,6 +1092,20 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
? Extract<EventKey, `${Prefix}.${string}`>
: P & EventKey;
/** Acked, not acked, no socket anywhere, or sockets only in a peer region. */
export type NotifDeliveryOutcome =
| 'received'
| 'unreceived'
| 'no-socket'
| 'remote-only';
/** One socket's receipt for a live notification. */
export interface NotifDeliveryResponder {
visibilityState: 'visible' | 'hidden' | 'unknown';
/** The page's window has focus. Absent from older clients. */
focused?: boolean;
}
export type EventMetadata = {
from_outside?: boolean;
/** Set by `emitAndWait` when a listener threw, so a verdict hook can refuse. */
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,85 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
describe('auth elevation cookies over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
it.each(['bearer', 'cookie'])(
'API host returns a usable token without setting cookies (%s auth)',
async (credential) => {
const { token, password, username } = env.users.user;
const res = await fetch(new URL('/auth/elevate', env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
...(credential === 'cookie'
? { cookie: `puter_auth_token=${token}` }
: { authorization: `Bearer ${token}` }),
},
body: JSON.stringify({ password }),
});
expect(res.status).toBe(200);
expect(res.headers.get('set-cookie')).toBeNull();
const body = (await res.json()) as {
elevated: boolean;
token: string;
};
expect(body.elevated).toBe(true);
expect(typeof body.token).toBe('string');
const user = await env.server.stores.user.getByUsername(username);
const { verifyStepUpSession } =
await import('../../core/http/middleware/stepUpSession.js');
expect(
verifyStepUpSession(
{
headers: { 'x-puter-elevation': body.token },
actor: { user: { uuid: user!.uuid } },
} as never,
{ tokenService: env.server.services.token },
),
).toBe(true);
},
);
it('GUI host sets the elevation cookie and omits the raw token for cookie auth', async () => {
const { token, password } = env.users.user;
const res = await fetch(new URL('/auth/elevate', env.origin), {
method: 'POST',
headers: {
'content-type': 'application/json',
cookie: `puter_auth_token=${token}`,
},
body: JSON.stringify({ password }),
});
expect(res.status).toBe(200);
expect(res.headers.get('set-cookie')).toContain('puter_elevated=');
expect(await res.json()).toEqual({ elevated: true });
});
});
@@ -42,6 +42,10 @@ import {
adminOnlyGate,
requireUserActorGate,
} from '../../core/http/middleware/gates.js';
import {
configureRateLimit,
listConfiguredRateLimitBackends,
} from '../../core/http/middleware/rateLimit.js';
import {
ROUTES_METADATA_KEY,
type CollectedRoute,
@@ -49,8 +53,11 @@ import {
} from '../../core/http/types.js';
import { PuterServer } from '../../server.js';
import { createSecret as otpCreateSecret } from '../../services/auth/OTPUtil.js';
import { kvSharePermission } from '../../services/events/kvShares.js';
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
import { kvSharePermission } from '../../services/kvShare/kvShares.js';
import {
FULL_API_ACCESS,
NOTIFICATIONS_PUSH_PERMISSION,
} from '../../services/permission/consts.js';
import { setupTestServer } from '../../testUtil.js';
import { resetCardVerificationStatusCache } from '../../util/cardFallback.js';
import { FS_READ_LIMIT } from '../fs/limits.js';
@@ -1750,6 +1757,40 @@ describe('AuthController.handleLoginOtp + handleLoginRecoveryCode', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('refuses the sign-in while the used-token store is down', async () => {
const { user, secret, otpToken } = await startOtpLogin();
const token = await otpToken();
const { default: backend } = listConfiguredRateLimitBackends();
configureRateLimit({
default: 'redis',
redis: {
multi: () => {
throw new Error('store down');
},
},
});
const logged = vi.spyOn(console, 'error').mockImplementation(() => {});
const res = makeRes();
try {
await expect(
controller.handleLoginOtp(
makeReq({ token, code: liveTotp(user.username, secret) }),
res,
),
).rejects.toThrow('store down');
} finally {
logged.mockRestore();
configureRateLimit({
default: backend,
redis: server.clients.redis,
kv: server.stores.kv,
metering: server.services.metering,
});
}
expect(res.body).toBeUndefined();
});
it('a wrong code leaves the token usable for a retry', async () => {
const { user, secret, otpToken } = await startOtpLogin();
const token = await otpToken();
@@ -2151,7 +2192,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = `service:tg-app:ii:read`;
const permission = `events:background`;
// The controller's grant call delegates through PermissionService
// (which uses ALS Context.set), so wrap in runWithContext.
const res = makeRes();
@@ -2179,6 +2220,78 @@ describe('AuthController grant flows', () => {
).toContain(permission);
});
it('grant-user-app: grants and revokes notifications:push for an app', async () => {
const app = await server.stores.app.create(
{
name: `tgp-${uuidv4()}`,
title: 'TestGrantPushApp',
index_url: 'https://example.test/index.html',
},
{ ownerUserId: issuer.id },
);
const user = await server.stores.user.getById(issuer.id);
const appActor = makeActor({
user: user!,
app: { uid: app.uid, id: app.id },
});
const hasPush = () =>
server.services.permission.check(
appActor,
NOTIFICATIONS_PUSH_PERMISSION,
);
expect(await hasPush()).toBe(false);
// Anything past the bare permission names nothing a prompt describes.
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: app.uid,
permission: `${NOTIFICATIONS_PUSH_PERMISSION}:extra`,
},
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'permission_not_grantable',
});
expect(await hasPush()).toBe(false);
const res = makeRes();
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: app.uid,
permission: NOTIFICATIONS_PUSH_PERMISSION,
},
{ actor: issuerActor },
),
res,
),
);
expect(res.body).toEqual({});
expect(await hasPush()).toBe(true);
await inCtx(issuerActor, () =>
controller.handleRevokeUserApp(
makeReq(
{
app_uid: app.uid,
permission: NOTIFICATIONS_PUSH_PERMISSION,
},
{ actor: issuerActor },
),
makeRes(),
),
);
expect(await hasPush()).toBe(false);
});
it('grant-user-app: resolves `origin` to the app when app_uid is omitted', async () => {
// The GUI permission dialog and puter.perms.grantOrigin() identify
// third-party sites by origin rather than app uid.
@@ -2192,7 +2305,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = `service:tg-origin-app:ii:read`;
const permission = `events:background`;
const res = makeRes();
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
@@ -2229,7 +2342,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = `service:tr-origin-app:ii:read`;
const permission = `events:background`;
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
@@ -2294,7 +2407,7 @@ describe('AuthController grant flows', () => {
await inCtx(issuerActor, () =>
controller.handleRevokeUserApp(
makeReq(
{ app_uid: app.uid, permission: 'service:x:ii:read' },
{ app_uid: app.uid, permission: 'events:background' },
{ actor: issuerActor },
),
makeRes(),
@@ -2318,12 +2431,12 @@ describe('AuthController grant flows', () => {
it('grant-user-app: 400 on non-string or oversized origin/app_uid/permission', async () => {
const cases = [
{ origin: { host: 'evil' }, permission: 'service:x:ii:read' },
{ origin: 'https://a.test', permission: ['service:x:ii:read'] },
{ app_uid: 12345, permission: 'service:x:ii:read' },
{ origin: { host: 'evil' }, permission: 'events:background' },
{ origin: 'https://a.test', permission: ['events:background'] },
{ app_uid: 12345, permission: 'events:background' },
{
origin: `https://${'a'.repeat(5000)}.test`,
permission: 'service:x:ii:read',
permission: 'events:background',
},
];
for (const body of cases) {
@@ -2354,7 +2467,7 @@ describe('AuthController grant flows', () => {
{ ownerUserId: target.id },
);
const permission = 'service:squat:ii:read';
const permission = 'events:background';
for (const handler of [
'handleGrantUserApp',
'handleRevokeUserApp',
@@ -2398,7 +2511,7 @@ describe('AuthController grant flows', () => {
{ ownerUserId: target.id },
);
const permission = 'service:squat-beside:ii:read';
const permission = 'events:background';
for (const handler of [
'handleGrantUserApp',
'handleRevokeUserApp',
@@ -2451,7 +2564,7 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const permission = 'service:tp-origin:ii:read';
const permission = 'events:background';
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
@@ -2487,7 +2600,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = 'service:tm-app:ii:read';
const permission = 'events:background';
for (const body of [
{ app_uid: app.uid, permission, meta: 'nope' },
{ app_uid: app.uid, permission, meta: [1, 2] },
@@ -2528,7 +2641,7 @@ describe('AuthController grant flows', () => {
},
{ ownerUserId: issuer.id },
);
const permission = 'service:tb-app:ii:read';
const permission = 'events:background';
const oversized = { blob: 'z'.repeat(5000) };
for (const body of [
{ app_uid: app.uid, permission, extra: oversized },
@@ -2605,26 +2718,278 @@ describe('AuthController grant flows', () => {
expect(res.body).toEqual({});
});
it('grant-user-app: 400 on a `permission` wider than the column it lands in', async () => {
// 300 chars is under the 4096 input cap but over `varchar(255)`, so it
// used to reach the INSERT and fault on MySQL/Postgres. The check runs
// after the rewrite, so this has to be a permission no rewriter
// shortens — and it has to reject before the app is resolved, since
// this uid names no app.
await expect(
it('grant-user-app: grants read and write share delegations on a region, never on the namespace', async () => {
const appName = `tkvl-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvShareLevelsApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const root of ['kv-share-read', 'kv-share-write']) {
const namespace = `manage:${root}:${issuer.uuid}:${app.uid}`;
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission: namespace },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
const res = makeRes();
await inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission: `${namespace}:team` },
{ actor: issuerActor },
),
res,
),
);
expect(res.body).toEqual({});
}
const rows = await server.clients.db.read(
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ? ORDER BY `permission`',
[issuer.id, app.id],
);
expect(rows.map((r: { permission: string }) => r.permission)).toEqual([
`manage:kv-share-read:${issuer.uuid}:${app.uid}:team`,
`manage:kv-share-write:${issuer.uuid}:${app.uid}:team`,
]);
});
it('grant-user-app: refuses a share delegation whose region a new share could not name, writing nothing', async () => {
const appName = `tkvc-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvShareCharsApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const region of ['Team', 'team:Abc', 'caf\u00E9', 'a b'])
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: app.uid,
permission: `manage:kv-share-read:${issuer.uuid}:${app.uid}:${region}`,
},
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
const rows = await server.clients.db.read(
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
[issuer.id, app.id],
);
expect(rows).toEqual([]);
});
it('grant-user-app: refuses a permission outside the app-grantable families, writing nothing', async () => {
const appName = `tng-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestNotGrantableApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
const other = await server.stores.app.create(
{
name: `tng-other-${uuidv4()}`,
title: 'TestNotGrantableOther',
index_url: `https://tng-other-${uuidv4()}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
const refused = [
['driver:puter-image-generation:generate'],
[`user:${target.uuid}:email:read`],
// One bad entry refuses the batch.
['events:background', 'random-string'],
// A share delegation names the app's own namespace only.
[`manage:kv-share:${issuer.uuid}:${other.uid}:rooms`],
];
for (const permissions of refused) {
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permissions },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'permission_not_grantable',
});
}
const rows = await server.clients.db.read(
'SELECT 1 FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
[issuer.id, app.id],
);
expect(rows).toEqual([]);
});
it('grant-user-app: refuses a non-canonical spelling of a grantable permission', async () => {
const appName = `tnc-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestNonCanonicalApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const permission of [
`f\\s:/${issuer.username}/Documents:read`,
`app-root-di\\r:${app.uid}:read`,
`manag\\e:kv-share:${issuer.uuid}:${app.uid}:rooms`,
]) {
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'permission_not_grantable' });
}
});
it('grant-user-app: refuses a share delegation over a prefix no share could be made of, as a share would be', async () => {
const appName = `tkvp-${uuidv4().slice(0, 8)}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvSharePrefixApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
for (const root of ['kv-share', 'kv-share-read', 'kv-share-write']) {
// A share grant one character wider than the column.
const head = `${root}:${issuer.uuid}:${app.uid}:`;
const region = 'a'.repeat(256 - head.length);
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: app.uid,
permission: `manage:${head}${region}`,
},
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ legacyCode: 'invalid_kv_share_prefix' });
}
});
it('grant-user-app: a share delegation must name the app the request names, by uid, name or origin', async () => {
const appName = `tkvn-${uuidv4()}`;
const origin = `https://${appName}.example.test`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestKvShareNamedApp',
index_url: `${origin}/index.html`,
},
{ ownerUserId: issuer.id },
);
const other = await server.stores.app.create(
{
name: `tkvn-other-${uuidv4()}`,
title: 'TestKvShareNamedOther',
index_url: `https://tkvn-other-${uuidv4()}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
const grant = (requester: Record<string, string>, namespace: string) =>
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{
app_uid: `app-${uuidv4()}`,
permission: `service:${'a'.repeat(300)}:ii:read`,
...requester,
permission: `manage:kv-share:${issuer.uuid}:${namespace}:rooms`,
},
{ actor: issuerActor },
),
makeRes(),
),
);
for (const requester of [{ app_uid: app.name }, { origin }]) {
await grant(requester, app.uid);
await expect(grant(requester, other.uid)).rejects.toMatchObject({
legacyCode: 'permission_not_grantable',
});
}
// The app's name is not its namespace.
await expect(
grant({ app_uid: app.uid }, app.name),
).rejects.toMatchObject({ legacyCode: 'permission_not_grantable' });
const rows = await server.clients.db.read(
'SELECT `permission` FROM `user_to_app_permissions` WHERE `user_id` = ? AND `app_id` = ?',
[issuer.id, app.id],
);
expect(rows.map((r: { permission: string }) => r.permission)).toEqual([
`manage:kv-share:${issuer.uuid}:${app.uid}:rooms`,
]);
});
it('grant-user-app: 400 on a `permission` wider than the column it lands in', async () => {
// Under the 4096 input cap but over `varchar(255)`, so it would reach
// the INSERT and fault on MySQL/Postgres. The check runs after the
// rewrite, so this has to be a grantable permission no rewriter
// shortens.
const appName = `tw-${uuidv4()}`;
const app = await server.stores.app.create(
{
name: appName,
title: 'TestWidthApp',
index_url: `https://${appName}.example.test/index.html`,
},
{ ownerUserId: issuer.id },
);
// The shares it delegates fit, at 255 characters; the consent's own
// `manage:` arm takes it past the column.
const head = `kv-share:${issuer.uuid}:${app.uid}:`;
const permission = `manage:${head}${'a'.repeat(255 - head.length)}`;
await expect(
inCtx(issuerActor, () =>
controller.handleGrantUserApp(
makeReq(
{ app_uid: app.uid, permission },
{ actor: issuerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({ statusCode: 400 });
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
message: 'Invalid `permission`',
});
});
it('grant-user-app: an oversized permission that a rewriter shortens is accepted', async () => {
@@ -2636,6 +3001,8 @@ describe('AuthController grant flows', () => {
// on "please try again" for any deeply nested file. Exercised here
// through a rewriter of our own, since the real fs rewriter resolves
// the path through the fsentry store and this suite has no entries.
// It rewrites a share delegation only because that is a grantable
// shape no production rewriter touches.
const appName = `tlp-${uuidv4()}`;
const app = await server.stores.app.create(
{
@@ -2646,13 +3013,15 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const prefix = `tlprw-${uuidv4()}`;
const longPermission = `${prefix}:${'a'.repeat(300)}:read`;
const shortPermission = `${prefix}:${uuidv4()}:read`;
// Over the column only by its `manage:` arm, so the shares it
// delegates still fit.
const head = `kv-share:${issuer.uuid}:${app.uid}:`;
const longPermission = `manage:${head}${'a'.repeat(255 - head.length)}`;
const shortPermission = `manage:${head}${uuidv4()}`;
expect(longPermission.length).toBeGreaterThan(255);
expect(shortPermission.length).toBeLessThanOrEqual(255);
server.services.permission.registerRewriter({
id: `test-shorten-${prefix}`,
id: `test-shorten-${app.uid}`,
matches: (permission: string) => permission === longPermission,
rewrite: async () => shortPermission,
});
@@ -2712,7 +3081,8 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const prefix = `conc-${uuidv4()}`;
// A grantable shape no production rewriter touches.
const prefix = `manage:kv-share:${issuer.uuid}:${app.uid}:conc-${uuidv4()}`;
let release: () => void;
let entered: () => void;
const held = new Promise<void>((r) => {
@@ -2740,7 +3110,7 @@ describe('AuthController grant flows', () => {
makeReq(
{
app_uid: app.uid,
permission: `${prefix}:${app.uid}:write`,
permission: `${prefix}:write`,
},
{ actor: issuerActor },
),
@@ -2781,8 +3151,8 @@ describe('AuthController grant flows', () => {
{ ownerUserId: issuer.id },
);
const prefix = `tardrw-${uuidv4()}`;
const pseudoPermission = `${prefix}:${app.uid}:write`;
const prefix = `manage:kv-share:${issuer.uuid}:${app.uid}:tardrw-${uuidv4()}`;
const pseudoPermission = `${prefix}:write`;
const resolvedPermission = `fs:${uuidv4()}:write`;
const NOTHING = 'nothing-in-particular';
server.services.permission.registerRewriter({
@@ -3023,7 +3393,7 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
// phase A's pre-flight passes, phase C's re-check then fails, so the
// leaf (and its auto-created parent) already exist by the time the
// request unwinds.
const prefix = `rbparent-${uuidv4()}`;
const prefix = `manage:kv-share:${user.uuid}:${appUid}:rbparent-${uuidv4()}`;
let calls = 0;
server.services.permission.registerRewriter({
id: prefix,
@@ -3189,13 +3559,13 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
});
it('does not apply `create` to a `manage:` grant — 404 unchanged, nothing created', async () => {
it('refuses a `manage:` grant outright, creating nothing', async () => {
const p = path(`manage-missing-${rand()}`);
await expect(
grant({ permission: `manage:fs:${p}:write` }),
).rejects.toMatchObject({
statusCode: 404,
legacyCode: 'subject_does_not_exist',
statusCode: 400,
legacyCode: 'permission_not_grantable',
});
expect(await server.stores.fsEntry.getEntryByPath(p)).toBeFalsy();
});
@@ -3285,7 +3655,7 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
// (so nothing is rejected before anything is created) but has grown
// too wide by the time phase C re-runs it — modelling a grant that
// only fails once the fs path from the same batch already exists.
const prefix = `rbtest-${uuidv4()}`;
const prefix = `manage:kv-share:${user.uuid}:${appUid}:rbtest-${uuidv4()}`;
let calls = 0;
server.services.permission.registerRewriter({
id: `test-rollback-${prefix}`,
@@ -3326,6 +3696,37 @@ describe('AuthController.handleGrantUserApp `create` flag', () => {
);
});
it('check-permissions answers an app asking about itself through ACL, so a folder grant covers what is inside', async () => {
const dir = path(`acl-self-${rand()}`);
const child = `${dir}/inner`;
await server.services.fs.mkdir(user.id, {
path: child,
createMissingParents: true,
});
expect(await checkAsApp(`fs:${child}:read`)).toBe(false);
await grant({ permission: `fs:${dir}:write`, create: false });
expect(await checkAsApp(`fs:${child}:read`)).toBe(true);
expect(await checkAsApp(`fs:${child}:write`)).toBe(true);
// Another user's home stays out of reach.
expect(await checkAsApp(`fs:/${target.username}/Documents:read`)).toBe(
false,
);
});
it('check-permissions answers an app asking about itself through ACL for a uid too', async () => {
const dir = path(`acl-self-uid-${rand()}`);
const child = await server.services.fs.mkdir(user.id, {
path: `${dir}/inner`,
createMissingParents: true,
});
expect(await checkAsApp(`fs:${child.uuid}:read`)).toBe(false);
await grant({ permission: `fs:${dir}:read`, create: false });
expect(await checkAsApp(`fs:${child.uuid}:read`)).toBe(true);
expect(await checkAsApp(`fs:${child.uuid}:write`)).toBe(false);
});
it('check-permissions on a missing fs path answers false rather than 404ing, without poisoning a mixed batch', async () => {
const missingPath = path(`.checkmissing-${rand()}`);
// A held fs grant: the owning user holds it implicitly (is-owner), so
@@ -4152,9 +4553,9 @@ describe('AuthController.handleCreateAccessToken + handleRevokeAccessToken', ()
['an oversized permission string', ['x'.repeat(4097)]],
[
'an oversized `extra`',
[['service:x:ii:read', { a: 'y'.repeat(5000) }]],
[['events:background', { a: 'y'.repeat(5000) }]],
],
['a non-object `extra`', [['service:x:ii:read', 'nope']]],
['a non-object `extra`', [['events:background', 'nope']]],
['an empty permission string', ['']],
])('rejects %s with 400', async (_label, permissions) => {
await expect(
@@ -4731,6 +5132,26 @@ describe('AuthController phone verification — staging & reuse', () => {
expect(staged).toBe('+14155550123');
});
it('makes an email-gate-last account verify its card before its phone', async () => {
const { user, actor } = await makeUserAndActor({
requires_phone_verification: 1,
requires_card_verification: 1,
});
await server.stores.user.updateMetadata(user.id, {
emailGateLast: true,
});
const prelude = stubPrelude();
await withClients({ prelude }, async () => {
await expect(
controller.handleSendConfirmPhone(
makeReq({ phone: '+14155550123' }, { actor }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 409 });
});
expect(prelude.createVerification).not.toHaveBeenCalled();
});
it('forwards an abuse-extension veto as 429 with the opaque reason, and sends nothing', async () => {
const { user, actor } = await makeUserAndActor();
const emitAndWait = vi.fn(
@@ -4796,7 +5217,7 @@ describe('AuthController phone verification — staging & reuse', () => {
);
});
it('carries the client dispatch id on the sent signal', async () => {
it('carries the client dispatch id on the check and sent events', async () => {
const { actor } = await makeUserAndActor();
const emit = vi.fn();
const emitAndWait = vi.fn(async () => {});
@@ -4817,6 +5238,11 @@ describe('AuthController phone verification — staging & reuse', () => {
expect.objectContaining({ dispatch_id: 'disp-123' }),
expect.anything(),
);
expect(emitAndWait).toHaveBeenCalledWith(
'puter.phone-verification.check',
expect.objectContaining({ dispatch_id: 'disp-123' }),
expect.anything(),
);
});
it('does NOT emit the sent signal when the send fails upstream', async () => {
@@ -7324,6 +7750,46 @@ describe('AuthController permission revokes', () => {
);
expect(res.body).toEqual({});
});
it('revoke-user-user: refuses a key-value share grant and leaves it standing', async () => {
const { actor: ownerActor, user: owner } = await makeUserAndActor();
const { user: holder } = await makeUserAndActor();
const permissions = ['kv-share', 'kv-share-read', 'kv-share-write'].map(
(root) => `${root}:${owner.uuid}:os-global:notes:${uuidv4()}`,
);
for (const permission of permissions)
await server.stores.permission.upsertUserUserPerm(
holder.id,
owner.id,
permission,
{},
);
for (const permission of permissions)
await expect(
inCtx(ownerActor, () =>
controller.handleRevokeUserUser(
makeReq(
{ target_username: holder.username, permission },
{ actor: ownerActor },
),
makeRes(),
),
),
).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('puter.kv.unshare()'),
});
expect(
(
await server.stores.permission.findUserUserPermsByIssuer(
holder.id,
owner.id,
permissions,
)
).sort(),
).toEqual([...permissions].sort());
});
});
// ── Permission checks + listing ────────────────────────────────────
@@ -7365,6 +7831,94 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
]);
});
it('check-permissions: answers false for share grants, which name what a holder is never told, but not for their manage arm', async () => {
const { user: owner, actor: ownerActor } = await makeUserAndActor();
const { user: holder, actor: holderActor } = await makeUserAndActor();
const region = `notes-${uuidv4().slice(0, 8)}`;
const grant = `kv-share-read:${owner.uuid}:os-global:${region}`;
await inCtx(ownerActor, () =>
server.services.permission.grantUserUserPermission(
ownerActor,
holder.username,
grant,
),
);
expect(await server.services.permission.check(holderActor, grant)).toBe(
true,
);
const asked = [
grant,
`kv-share:${owner.uuid}:os-global:${region}:inner`,
`kv-share-write:${owner.uuid}:os-global:${region}`,
];
const res = makeRes();
await inCtx(holderActor, () =>
controller.handleCheckPermissions(
makeReq({ permissions: asked }, { actor: holderActor }),
res,
),
);
expect(res.body).toEqual({
permissions: Object.fromEntries(asked.map((p) => [p, false])),
});
// What `puter.perms.check('kvShare', …)` asks.
const manage = `manage:kv-share-read:${owner.uuid}:os-global:${region}`;
const ownerRes = makeRes();
await inCtx(ownerActor, () =>
controller.handleCheckPermissions(
makeReq({ permissions: [manage] }, { actor: ownerActor }),
ownerRes,
),
);
expect(ownerRes.body).toEqual({ permissions: { [manage]: true } });
});
it('check-permissions: answers false for share grants asked by an app actor too', async () => {
const { user: owner, actor: ownerActor } = await makeUserAndActor();
const { user: holder, actor: holderActor } = await makeUserAndActor();
const app = await server.stores.app.create(
{
name: `cpk-${uuidv4()}`,
title: 'TestCheckPermsKvApp',
index_url: 'https://check-perms-kv.example.test/index.html',
},
{ ownerUserId: holder.id },
);
const grant = `kv-share-read:${owner.uuid}:${app.uid}:notes-${uuidv4().slice(0, 8)}`;
await inCtx(ownerActor, () =>
server.services.permission.grantUserUserPermission(
ownerActor,
holder.username,
grant,
),
);
// What launching the app records for the holder.
await server.services.permission.grantUserAppPermission(
holderActor,
app.uid,
'flag:app-is-authenticated',
);
const appActor = makeActor({
user: holderActor.user,
app: { id: app.id, uid: app.uid },
});
// The holder's app in the share's namespace holds it.
expect(await server.services.permission.check(appActor, grant)).toBe(
true,
);
const res = makeRes();
await inCtx(appActor, () =>
controller.handleCheckPermissions(
makeReq({ permissions: [grant] }, { actor: appActor }),
res,
),
);
expect(res.body).toEqual({ permissions: { [grant]: false } });
});
// What lets a permission request settle without a prompt. The second string
// is a consent scope nothing implies, so the first's `true` is the grant.
it('check-permissions: an app-under-user actor sees a grant made to that app', async () => {
@@ -7579,7 +8133,7 @@ describe('AuthController.handleCheckPermissions + handleListPermissions', () =>
},
{ ownerUserId: user.id },
);
const permission = 'service:tl-app:ii:read';
const permission = 'events:background';
await inCtx(actor, () =>
controller.handleGrantUserApp(
makeReq({ app_uid: app.uid, permission, extra: {} }, { actor }),
@@ -9288,6 +9842,70 @@ describe('AuthController grant/revoke additional branches', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
it('revoke-user-app: 400 on `deleteData` without "*" or as a non-boolean', async () => {
const { actor } = await makeUserAndActor();
const app = await server.stores.app.create(
{
name: `tr-delete-data-${uuidv4()}`,
title: 'TestDeleteDataApp',
index_url: `https://tr-delete-data-${uuidv4()}.example.test/`,
},
{ ownerUserId: actor.user.id },
);
for (const body of [
{ permission: 'events:background', deleteData: true },
{ permission: '*', deleteData: 'yes' },
]) {
await expect(
controller.handleRevokeUserApp(
makeReq({ app_uid: app.uid, ...body }, { actor }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
}
});
it('revoke-user-app `*`: announces the uninstall, with `deleteData` as sent', async () => {
const { actor } = await makeUserAndActor();
const app = await server.stores.app.create(
{
name: `tr-uninstall-${uuidv4()}`,
title: 'TestUninstallApp',
index_url: `https://tr-uninstall-${uuidv4()}.example.test/`,
},
{ ownerUserId: actor.user.id },
);
const seen: unknown[] = [];
const listener = (_key: string, data: unknown) => {
seen.push(data);
};
server.clients.event.on('app.uninstalled', listener);
try {
for (const deleteData of [undefined, true]) {
await inCtx(actor, () =>
controller.handleRevokeUserApp(
makeReq(
{ app_uid: app.uid, permission: '*', deleteData },
{ actor },
),
makeRes(),
),
);
}
} finally {
server.clients.event.off('app.uninstalled', listener);
}
const expected = {
userId: actor.user.id,
userUuid: actor.user.uuid,
appUid: app.uid,
};
expect(seen).toEqual([
{ ...expected, deleteData: false },
{ ...expected, deleteData: true },
]);
});
it('revoke-user-app: 400 when permission is "*" but app_uid is missing', async () => {
const { actor } = await makeUserAndActor();
await expect(
@@ -10215,7 +10833,7 @@ describe('AuthController — app-data grants', () => {
await expect(
post('handleGrantUserApp', {
app_uid: grantee.uid,
permission: 'app-data:app-nope/:kv:read',
permission: `app-data:app-${uuidv4()}:kv:read`,
}),
).rejects.toMatchObject({ statusCode: 404 });
});
@@ -10282,7 +10900,7 @@ describe('AuthController — app-data grants', () => {
it('leaves unrelated permissions untouched by the new validation', async () => {
const grantee = await makeAppRow();
const permission = 'service:unrelated:ii:read';
const permission = 'events:background';
await post('handleGrantUserApp', {
app_uid: grantee.uid,
permission,
+174 -36
View File
@@ -33,7 +33,7 @@ import {
hasVerifiedPhone,
} from '../../core/http/middleware/gates.js';
import type { Actor } from '../../core/actor.js';
import { isPlainUserActor, makeActor } from '../../core/actor.js';
import { isAppActor, isPlainUserActor, makeActor } from '../../core/actor.js';
import {
checkRateLimit,
consumeRouteRateLimit,
@@ -66,6 +66,7 @@ import {
isCardFallbackEnabled,
phoneAttemptsKey,
} from '../../util/cardFallback.js';
import { isEmailGateLast } from '../../util/verificationOrder.js';
import { sessionCookieFlags } from '../../util/cookieFlags.js';
import { cleanEmail, isStorableEmail } from '../../util/email.js';
import { isGodmodeApp } from '../../util/godmodeApps.js';
@@ -88,13 +89,16 @@ import {
import { promoteToVerifiedGroup } from '../../util/userProvisioning.js';
import {
assertBoundedManageGrant,
assertNewShareSegments,
assertPrefixFitsAccess,
isKvSharePermission,
} from '../../services/events/kvShares.js';
} from '../../services/kvShare/kvShares.js';
import {
APP_DATA_PERMISSION_PREFIX,
appDataSharingAllowed,
parseAppDataPermission,
} from '../../services/permission/appDataScopes.js';
import { parseAppGrantablePermission } from '../../services/permission/appGrantable.js';
import {
assertCreatablePath,
isOwnHomePath,
@@ -1217,6 +1221,18 @@ export class AuthController extends PuterController {
}
}
/** The email-gate-last confirmation email; a failure never fails the gate. */
private async sendDeferredConfirmation(userId: number): Promise<void> {
try {
await this.services.signup.sendDeferredConfirmationEmail(userId);
} catch (e) {
console.warn(
'[verification] deferred confirmation email failed:',
e,
);
}
}
private async isCardFallbackEligible(user: {
id: number;
requires_phone_verification?: boolean | number | null;
@@ -1262,6 +1278,12 @@ export class AuthController extends PuterController {
throw new HttpError(403, 'Account suspended.', {
legacyCode: 'account_suspended',
});
if (isEmailGateLast(user) && user.requires_card_verification)
throw new HttpError(
409,
'Card verification must be completed first.',
{ legacyCode: 'conflict' },
);
if (!this.clients.prelude?.isConfigured())
throw await this.smsSendError(
503,
@@ -1332,6 +1354,7 @@ export class AuthController extends PuterController {
user_uid: user.uuid,
phone: parsed.e164,
device_fingerprint: req.deviceFingerprint ?? null,
dispatch_id: dispatchId ?? null,
allowed: true,
reason: null as string | null,
card_fallback_instead: false,
@@ -1570,6 +1593,7 @@ export class AuthController extends PuterController {
requires_phone_verification: 0,
phone: pendingPhone,
});
await this.sendDeferredConfirmation(user.id);
// Run the verified-event listeners synchronously (emitAndWait, not
// fire-and-forget emit) so a carrier-based card-verification waiver in
@@ -1650,7 +1674,11 @@ export class AuthController extends PuterController {
res.json({ card_verified: true });
return;
}
if (user.requires_phone_verification && !fallbackEligible)
if (
user.requires_phone_verification &&
!fallbackEligible &&
!isEmailGateLast(user)
)
throw new HttpError(
409,
'Phone verification must be completed first.',
@@ -1713,6 +1741,7 @@ export class AuthController extends PuterController {
await this.stores.user.update(user.id, {
requires_card_verification: 0,
});
await this.sendDeferredConfirmation(user.id);
res.json({ card_verified: true, disabled: true });
return;
}
@@ -1772,7 +1801,11 @@ export class AuthController extends PuterController {
res.json({ card_verified: true });
return;
}
if (user.requires_phone_verification && !fallbackEligible)
if (
user.requires_phone_verification &&
!fallbackEligible &&
!isEmailGateLast(user)
)
throw new HttpError(
409,
'Phone verification must be completed first.',
@@ -1813,6 +1846,7 @@ export class AuthController extends PuterController {
await this.stores.user.update(user.id, {
requires_card_verification: 0,
});
await this.sendDeferredConfirmation(user.id);
res.json({ card_verified: true, disabled: true });
return;
}
@@ -1837,6 +1871,7 @@ export class AuthController extends PuterController {
requires_card_verification: 0,
...(clearedPhoneGate ? { requires_phone_verification: 0 } : {}),
});
await this.sendDeferredConfirmation(user.id);
// Awaited like `user.phone-verified`, so whatever a listener grants on
// verification is in place before the client refreshes. emitAndWait
@@ -2997,6 +3032,45 @@ export class AuthController extends PuterController {
return typeof permission === 'string' && permission ? [permission] : [];
}
/**
* Refuses any entry outside the families an app can be granted. Anything
* else is a row the backend never checks for an app, or one wider than the
* consent prompt can describe.
*/
async #assertAppGrantable(
actor: Actor,
appIdentifier: string,
list: string[],
): Promise<void> {
const refuse = (entry: string) =>
new HttpError(
400,
`Permission cannot be granted to an app: ${entry}`,
{ legacyCode: 'permission_not_grantable' },
);
let appUid: string | undefined;
for (const entry of list) {
// A delegation over a whole key-value namespace has no bounded
// description; refused with its own code.
assertBoundedManageGrant(entry);
const parsed = parseAppGrantablePermission(entry, actor.user!.uuid);
if (!parsed) throw refuse(entry);
if (parsed.family !== 'kvShare') continue;
// Held to what a new share's prefix may be, so a consent never
// covers shares that could not be made.
assertNewShareSegments(parsed.segments);
assertPrefixFitsAccess(
parsed.ownerUuid,
parsed.appUid,
parsed.segments.join(':'),
parsed.level,
);
// An app shares regions of its own namespace only.
appUid ??= (await this.stores.app.resolveApp(appIdentifier))?.uid;
if (parsed.appUid !== appUid) throw refuse(entry);
}
}
/**
* Gate a cross-app data grant: the target must exist, must not have opted
* out of sharing, and must be named. Also creates the target's AppData
@@ -3175,6 +3249,7 @@ export class AuthController extends PuterController {
}
// 400 on a bad value, before anything else runs. Absent means create.
const create = parseCreateFlag(req.body?.create);
await this.#assertAppGrantable(req.actor!, app_uid, list);
// Validate every entry before writing or creating anything, so a bad
// one in the list cannot leave a partially-granted set behind: the
@@ -3187,10 +3262,6 @@ export class AuthController extends PuterController {
kind: FsCreateKind;
}> = [];
for (const entry of list) {
// A delegation over a whole key-value namespace has no bounded
// description, so it is refused where it is asked for rather than
// prompted for and then refused at use.
assertBoundedManageGrant(entry);
await this.#prepareAppDataGrant(req.actor!, entry);
const plan = await this.#planFsPathCreate(
req.actor!,
@@ -3265,6 +3336,14 @@ export class AuthController extends PuterController {
{ legacyCode: 'bad_request' },
);
}
// Only unsharing retires the handle along with its grant; withdrawing
// the grant alone would leave the share listed and dead.
if (typeof permission === 'string' && isKvSharePermission(permission))
throw new HttpError(
400,
'Stop a key-value share with `puter.kv.unshare()` (`DELETE /kv/shares/:handle`)',
{ legacyCode: 'bad_request' },
);
await this.services.permission.revokeUserUserPermission(
req.actor!,
target_username,
@@ -3288,7 +3367,8 @@ export class AuthController extends PuterController {
})
async handleRevokeUserApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body ?? {};
const { origin, permission, permissions, meta } = req.body ?? {};
const { origin, permission, permissions, meta, deleteData } =
req.body ?? {};
this.#validateAppPermissionParams({
app_uid,
origin,
@@ -3304,20 +3384,25 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
}
if (deleteData !== undefined && typeof deleteData !== 'boolean') {
throw new HttpError(400, '`deleteData` must be a boolean', {
legacyCode: 'bad_request',
});
}
if (deleteData && permission !== '*') {
throw new HttpError(
400,
'`deleteData` only applies when `permission` is "*"',
{ legacyCode: 'bad_request' },
);
}
// Deliberately not gated by the target's sharing flag: a user must
// always be able to withdraw a grant, whatever the target now says.
if (permission === '*') {
await this.services.permission.revokeUserAppAll(
req.actor!,
app_uid,
meta ?? undefined,
);
// Withdrawing everything also retires the workers the app deployed
// under this user; their tokens would otherwise act as the app.
await this.services.auth.deleteWorkerSessionsForApp(
req.actor!.user!.id as number,
app_uid,
);
await this.services.auth.uninstallApp(req.actor!, app_uid, {
deleteData: deleteData === true,
meta: meta ?? undefined,
});
} else {
for (const entry of list) {
await this.services.permission.revokeUserAppPermission(
@@ -3397,11 +3482,28 @@ export class AuthController extends PuterController {
const unique = [...new Set(permissions)] as string[];
const result: Record<string, boolean> = {};
// `fs:` reaches down a tree, and only ACL walks that.
// A share grant names its owner and prefix, which its holder is never
// told, so asking about one would confirm a guess. `manage:` arms are
// the app's own consent and stay answerable.
const answerable: string[] = [];
for (const permission of unique) {
if (
typeof permission === 'string' &&
isKvSharePermission(permission)
)
result[permission] = false;
else answerable.push(permission);
}
// `fs:` reaches down a tree, and only ACL walks that. An app asking
// about itself gets the same answer, so a grant on a parent folder
// doesn't read as missing and prompt again.
const viaAcl = new Set(
app_uid ? unique.filter((p) => aclAnswerableFsPermission(p)) : [],
app_uid || isAppActor(actor)
? answerable.filter((p) => aclAnswerableFsPermission(p))
: [],
);
const viaScan = unique.filter((p) => !viaAcl.has(p));
const viaScan = answerable.filter((p) => !viaAcl.has(p));
let granted: Map<string, boolean>;
try {
@@ -3748,8 +3850,9 @@ export class AuthController extends PuterController {
let app = await this.stores.app.getByUid(app_uid);
if (!app && resolvedFromOrigin) {
// Hosted-subdomain origins get the site owner stamped as the
// app's creator at bootstrap; external origins stay unowned.
// Hosted-subdomain and custom-domain origins get the site owner
// stamped as the app's creator at bootstrap; external origins
// stay unowned.
// Canonical origin only, so alternate hosts share one row.
const canonicalOrigin =
this.services.auth.canonicalizeOrigin(origin);
@@ -3765,9 +3868,10 @@ export class AuthController extends PuterController {
// An origin's uid is a deterministic uuidv5, so a deleted app
// reappears here under the identical uid. Withdraw any cross-app
// data grants left pointing at it before this new row can inherit
// consent the user gave its predecessor. Only *this* path can reuse
// a uid: `AppStore.create` mints a random uuid4, which no deleted
// app can ever hold again.
// consent the user gave its predecessor. Only `createFromOrigin`
// reuses a uid, here or wherever else an origin app is bootstrapped:
// `AppStore.create` mints a random uuid4, which no deleted app can
// ever hold again.
//
// Called directly rather than through `app.changed`: the token is
// issued below, so this has to be able to stop that, and
@@ -3778,6 +3882,9 @@ export class AuthController extends PuterController {
app_uid,
'uid reused by a new app',
);
// Key-value shares the predecessor left read as gone already,
// being older than this row; this only cleans them up.
this.services.kvShare.retireSharesOlderThanApp(app);
}
if (!app) {
throw new HttpError(404, `App ${app_uid} does not exist`, {
@@ -4213,7 +4320,27 @@ export class AuthController extends PuterController {
);
}
this.#requireAccountTwoFactor(user, req.body?.code);
const code = req.body?.code;
if (
user.otp_enabled &&
typeof code === 'string' &&
/^[A-Z0-9]{8}$/.test(code)
) {
await this.#assertSecondFactorAttemptsRemain(user.uuid);
const consumed = await this.stores.user.consumeOtpRecoveryCode(
user.id,
hashRecoveryCode(code),
);
if (!consumed) {
await this.#recordSecondFactorFailure(user.uuid);
throw new HttpError(401, 'Incorrect code.', {
legacyCode: 'code_mismatch',
fields: { factor: 'otp' },
});
}
} else {
this.#requireAccountTwoFactor(user, code);
}
await this.stores.user.update(user.id, {
otp_enabled: false,
@@ -4423,11 +4550,14 @@ export class AuthController extends PuterController {
}
const token = signStepUpToken(this.services.token, user as never);
res.cookie(
STEP_UP_COOKIE_NAME,
token,
stepUpCookieOptions(this.config),
);
const isApiHost = req.subdomains?.at(-1) === 'api';
if (!isApiHost) {
res.cookie(
STEP_UP_COOKIE_NAME,
token,
stepUpCookieOptions(this.config),
);
}
// A browser reads its elevation back from the httpOnly cookie and never
// needs the raw value; handing it to page JS would put the second factor
@@ -4436,7 +4566,9 @@ export class AuthController extends PuterController {
// password/TOTP — this only avoids needless exposure, it isn't a gate.
const cookieName = this.config.cookie_name ?? 'puter_token';
const usedSessionCookie =
!!req.token && req.token === req.cookies?.[cookieName];
!isApiHost &&
!!req.token &&
req.token === req.cookies?.[cookieName];
res.json(
usedSessionCookie ? { elevated: true } : { elevated: true, token },
);
@@ -4549,13 +4681,19 @@ export class AuthController extends PuterController {
await checkRateLimit(`${scope}:${userUid}`, limit, window);
}
/** An `otp-login` JWT completes one sign-in; a second redemption is refused. */
/**
* An `otp-login` JWT completes one sign-in; a second redemption is refused.
* Fails closed: while the store can't say whether the token was used, the
* sign-in is refused rather than letting a captured token replay.
*/
async #consumeOtpLoginToken(token: string): Promise<void> {
const digest = crypto.createHash('sha256').update(token).digest('hex');
const firstUse = await checkRateLimit(
`login-otp-token-used:${digest}`,
1,
OTP_LOGIN_TOKEN_USED_WINDOW_MS,
undefined,
{ failOpen: false },
);
if (!firstUse) {
throw new HttpError(400, 'Invalid token.', {
@@ -0,0 +1,89 @@
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { hashRecoveryCode } from '../../services/auth/OTPUtil.js';
import {
createTestUser,
setupPuterTestEnv,
type PuterTestEnv,
} from '../../testUtil.js';
describe('2FA after signing in with a recovery code', () => {
let env: PuterTestEnv;
const password = 'lost-authenticator-password';
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const post = (
path: string,
body: Record<string, unknown>,
cookie?: string,
) =>
fetch(new URL(path, env.origin), {
method: 'POST',
headers: {
'content-type': 'application/json',
...(cookie ? { cookie } : {}),
},
body: JSON.stringify(body),
});
it('allows an unused recovery code to disable 2FA after password revalidation', async () => {
const username = `lost2fa_${uuidv4().slice(0, 8)}`;
await createTestUser(env.server, { username, password });
const user = (await env.server.stores.user.getByUsername(username))!;
await env.server.stores.user.update(user.id, {
email: `${username}@example.com`,
email_confirmed: true,
otp_enabled: true,
otp_secret: 'TESTSECRETBASE32',
otp_recovery_codes: ['RECOVER1', 'RECOVER2', 'RECOVER3']
.map(hashRecoveryCode)
.join(','),
});
// Sign in with a recovery code: works, and yields a web session.
const login = await post('/login', { username, password });
expect(login.status).toBe(202);
const { otp_jwt_token } = await login.json();
const recovered = await post('/login/recovery-code', {
token: otp_jwt_token,
code: 'RECOVER1',
});
expect((await recovered.json()).next_step).toBe('complete');
const cookie = recovered.headers
.getSetCookie()
.find((value) => value.startsWith('puter_auth_token='))!
.split(';')[0]!;
const reused = await post(
'/user-protected/disable-2fa',
{ password, code: 'RECOVER1' },
cookie,
);
expect(reused.status).toBe(401);
expect(
(await env.server.stores.user.getById(user.id, { force: true }))!
.otp_enabled,
).toBeTruthy();
const disable = await post(
'/user-protected/disable-2fa',
{ password, code: 'RECOVER2' },
cookie,
);
expect(disable.status).toBe(200);
expect(await disable.json()).toMatchObject({ success: true });
const after = await env.server.stores.user.getById(user.id, {
force: true,
});
expect(after!.otp_enabled).toBeFalsy();
expect(after!.otp_secret).toBeNull();
expect(after!.otp_recovery_codes).toBeNull();
});
});
@@ -19,6 +19,7 @@
import type { Request, Response } from 'express';
import { Controller, Post } from '../../core/http/decorators.js';
import { schedulesUnavailable } from '../../core/schedules.js';
import type { BroadcastService } from '../../services/broadcast/BroadcastService.js';
import type { ForwardBatch } from '../../services/events/forwardQueue.js';
import { PuterController } from '../types.js';
@@ -108,6 +109,42 @@ export class BroadcastController extends PuterController {
);
res.status(200).json({ ok: true, ...reply });
}
/**
* Schedule changes a region made for schedules this one owns, handed to
* `services.schedules`; `events_disabled` on a server without it.
*/
@Post('/events/schedules', { subdomain: '*' })
async schedules(req: Request, res: Response): Promise<void> {
const broadcast = this.services.broadcast as unknown as
BroadcastService | undefined;
if (!broadcast) {
res.status(503).json({
error: { message: 'Broadcast service not registered' },
});
return;
}
const verified = await broadcast.verifySignedRequest(
req.rawBody,
broadcastHeaders(req),
);
if (!verified.ok) {
res.status(verified.status ?? 400).json({
error: { message: verified.message ?? 'Bad request' },
});
return;
}
if (verified.info?.ignored) {
res.status(200).json({ ok: true, ...verified.info });
return;
}
const schedules = this.services.schedules;
if (!schedules) throw schedulesUnavailable();
const reply = await schedules.receivePeerChanges(req.body);
res.status(200).json({ ok: true, ...reply });
}
}
const broadcastHeaders = (req: Request) => {
@@ -21,7 +21,7 @@
import { EventEmitter } from 'node:events';
import type { Request, RequestHandler, Response } from 'express';
import { beforeEach, describe, expect, it } from 'vitest';
import { runWithContext } from '../../core/context.js';
import { Context, runWithContext } from '../../core/context.js';
import { isHttpError } from '../../core/http/HttpError.js';
import { configureRateLimit } from '../../core/http/middleware/rateLimit.js';
import { DriverController } from './DriverController.js';
@@ -278,6 +278,86 @@ describe('DriverController — concurrent acquire/release', () => {
expect(res2.body).toMatchObject({ success: true, result: 'pong' });
});
// A driver that keeps working after the caller hangs up (async video
// polling) must keep counting against the cap until it settles.
it('holds the slot while the driver still works after a disconnect', async () => {
let finish = () => {};
const driver = {
...makeSyntheticDriver(),
ping: () =>
new Promise<string>((resolve) => {
finish = () => resolve('pong');
}),
};
const handler = captureCallHandler(buildController(driver));
const req = () =>
makeReq({ interface: 'test-iface', method: 'ping' }, 'working');
const res1 = new StubRes();
const first = callInScope(handler, req(), res1 as unknown as Response);
await new Promise((r) => setImmediate(r));
res1.emit('close');
await new Promise((r) => setImmediate(r));
await expect(
callInScope(handler, req(), new StubRes() as unknown as Response),
).rejects.toSatisfy(
(e) =>
isHttpError(e) &&
(e as { statusCode: number }).statusCode === 429,
);
finish();
await first;
await new Promise((r) => setImmediate(r));
const res2 = new StubRes();
const second = callInScope(handler, req(), res2 as unknown as Response);
await new Promise((r) => setImmediate(r));
finish();
await second;
res2.emit('finish');
expect(res2.body).toMatchObject({ success: true, result: 'pong' });
});
it('holds the slot for work a driver keeps past its response', async () => {
let settleWork = () => {};
const driver = {
...makeSyntheticDriver(),
ping: async () => {
Context.get('keepDriverSlot')?.(
new Promise<void>((resolve) => {
settleWork = resolve;
}),
);
return 'pong';
},
};
const handler = captureCallHandler(buildController(driver));
const req = () =>
makeReq({ interface: 'test-iface', method: 'ping' }, 'kept');
const res1 = new StubRes();
await callInScope(handler, req(), res1 as unknown as Response);
res1.emit('finish');
await new Promise((r) => setImmediate(r));
await expect(
callInScope(handler, req(), new StubRes() as unknown as Response),
).rejects.toSatisfy(
(e) =>
isHttpError(e) &&
(e as { statusCode: number }).statusCode === 429,
);
settleWork();
await new Promise((r) => setImmediate(r));
const res2 = new StubRes();
await callInScope(handler, req(), res2 as unknown as Response);
res2.emit('finish');
settleWork();
expect(res2.body).toMatchObject({ success: true, result: 'pong' });
});
it('does not attach release listeners when the driver declares no concurrent config', async () => {
// With no spec there is nothing to release, so the gate must not
// wire `finish`/`close` release listeners. The client-disconnect
@@ -736,6 +736,42 @@ describe('DriverController driver-method lifecycle events', () => {
rejectReason: 'blocked in test',
});
});
it('answers 429 when a before listener vetoes with rejectStatus 429', async () => {
const actor = await makeUserActor();
await grantKvPerm(actor);
const key = `throttle-${uuidv4()}`;
server.clients.event.on(
'driver.puter-kvstore.set.before',
(_k, data) => {
if ((data.args as { key?: string })?.key === key) {
data.allow = false;
data.rejectStatus = 429;
}
},
);
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key, value: 'v' },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 429,
legacyCode: 'too_many_requests',
});
});
});
describe('DriverController.#handleListInterfaces', () => {
@@ -31,6 +31,7 @@ import {
checkDriverRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { LifecycleRejectStatus } from '../../clients/event/types.js';
import type { DriverMeta } from '../../drivers/meta.js';
import {
isCreditExhaustion,
@@ -50,6 +51,7 @@ import { PermissionUtil } from '../../services/permission/permissionUtil.js';
import type { WithLifecycle } from '../../types';
import { withSpan } from '../../util/span.js';
import { PuterController } from '../types.js';
import { CallSlot } from './callSlot.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
@@ -259,6 +261,17 @@ export class DriverController extends PuterController {
// -- Handlers ----------------------------------------------------
#handleCall = async (req: Request, res: Response): Promise<void> => {
const slot = new CallSlot();
const call = this.#call(req, res, slot);
slot.keep(call);
await call;
};
#call = async (
req: Request,
res: Response,
slot: CallSlot,
): Promise<void> => {
const {
interface: ifaceName,
method,
@@ -322,7 +335,8 @@ export class DriverController extends PuterController {
if (req.actor) {
const permService = this.services.permission as unknown as
PermissionService | undefined;
| PermissionService
| undefined;
if (permService) {
// Build via PermissionUtil.join so any `:` in a driver or
// interface name is escaped — raw interpolation would let a
@@ -395,10 +409,9 @@ export class DriverController extends PuterController {
// Per-method rate-limit and concurrent specs both live on the
// driver's resolved meta. Rate-limit is single-shot; concurrent
// acquires a slot that must be released when the response is done
// — we hook `res.finish` / `res.close` for that so streamed
// responses hold their slot until the stream drains, and aborted
// requests still give the slot back.
// acquires a slot held until the response is done and the call's
// work has settled, so streamed responses hold it until the stream
// drains and work that continues after a disconnect still counts.
const rateLimitSpec = resolvePerMethod(driverMeta?.rateLimit, method);
if (
!(await checkDriverRateLimit(req, ifaceName, method, rateLimitSpec))
@@ -430,18 +443,8 @@ export class DriverController extends PuterController {
legacyCode: 'too_many_requests',
});
}
let released = false;
const release = () => {
if (released) return;
released = true;
void handle.release();
};
// If the handler throws before responding, the express error
// handler will eventually send a response — `finish` fires then,
// so we still release. `close` covers client aborts.
res.once('finish', release);
res.once('close', release);
if (res.destroyed || res.writableFinished) release();
slot.attach(handle, res);
Context.set('keepDriverSlot', (work) => slot.keep(work));
}
// Stash the requested driver name in Context so multi-provider
@@ -474,6 +477,7 @@ export class DriverController extends PuterController {
args,
allow: true as boolean,
rejectReason: undefined as string | undefined,
rejectStatus: undefined as LifecycleRejectStatus | undefined,
};
await this.clients.event?.emitAndWait(
`driver.${ifaceName}.${method}.before`,
@@ -495,11 +499,12 @@ export class DriverController extends PuterController {
},
{},
);
const throttled = beforeEvent.rejectStatus === 429;
throw new HttpError(
403,
throttled ? 429 : 403,
beforeEvent.rejectReason ??
`Blocked by policy: ${ifaceName}:${method}`,
{ legacyCode: 'forbidden' },
{ legacyCode: throttled ? 'too_many_requests' : 'forbidden' },
);
}
@@ -0,0 +1,73 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { EventEmitter } from 'node:events';
import type { Response } from 'express';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { CallSlot } from './callSlot.js';
const handle = () => ({
release: vi.fn(() => Promise.resolve()),
renew: vi.fn(() => Promise.resolve()),
});
const deferred = () => {
let resolve = () => {};
const promise = new Promise<void>((r) => {
resolve = r;
});
return { promise, resolve };
};
afterEach(() => {
vi.useRealTimers();
});
describe('CallSlot', () => {
it('renews a slot held by a long call and stops once it is released', async () => {
vi.useFakeTimers();
const slot = new CallSlot();
const slotHandle = handle();
const res = new EventEmitter();
const call = deferred();
slot.keep(call.promise);
slot.attach(slotHandle, res as unknown as Response);
res.emit('close');
await vi.advanceTimersByTimeAsync(25 * 60 * 1000);
expect(slotHandle.renew).toHaveBeenCalledTimes(2);
expect(slotHandle.release).not.toHaveBeenCalled();
call.resolve();
await vi.advanceTimersByTimeAsync(0);
expect(slotHandle.release).toHaveBeenCalledOnce();
await vi.advanceTimersByTimeAsync(25 * 60 * 1000);
expect(slotHandle.renew).toHaveBeenCalledTimes(2);
});
it('releases once the response is done if no work is outstanding', async () => {
const slot = new CallSlot();
const slotHandle = handle();
const res = new EventEmitter();
slot.attach(slotHandle, res as unknown as Response);
res.emit('finish');
expect(slotHandle.release).toHaveBeenCalledOnce();
expect(slotHandle.renew).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,80 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Response } from 'express';
// Under the concurrency store's one-hour orphan TTL.
const SLOT_RENEW_MS = 10 * 60 * 1000;
interface SlotHandle {
release(): Promise<void>;
renew?(): Promise<void>;
}
/**
* A driver call's concurrency slot, held until the response is done and the
* call's work has settled, including work a driver carries on after the caller
* hung up or was answered.
*/
export class CallSlot {
#handle?: SlotHandle;
#work = new Set<Promise<unknown>>();
#responded = false;
#released = false;
#renew?: NodeJS.Timeout;
attach(handle: SlotHandle, res: Response): void {
this.#handle = handle;
const responded = () => {
this.#responded = true;
this.#maybeRelease();
};
res.once('finish', responded);
res.once('close', responded);
if (res.destroyed || res.writableFinished) responded();
if (!this.#released && this.#work.size > 0) this.#startRenewing();
}
keep(work: Promise<unknown>): void {
if (this.#released) return;
this.#work.add(work);
if (this.#handle) this.#startRenewing();
const settled = () => {
this.#work.delete(work);
this.#maybeRelease();
};
work.then(settled, settled);
}
#startRenewing(): void {
if (this.#renew) return;
this.#renew = setInterval(() => {
void this.#handle?.renew?.();
}, SLOT_RENEW_MS);
this.#renew.unref?.();
}
#maybeRelease(): void {
if (this.#released || !this.#handle) return;
if (!this.#responded || this.#work.size > 0) return;
this.#released = true;
clearInterval(this.#renew);
void this.#handle.release();
}
}
@@ -25,6 +25,11 @@ import { bodyRecord } from '../../core/http/requestBody.js';
import { DURABLE_LIST_LIMIT_CAP } from '../../stores/events/DurableSubscriptionStore.js';
import { EVENTS_WORKERS_LIST_LIMIT_CAP } from '../../stores/events/EventHandlerStore.js';
import { KV_HANDLE_LIST_LIMIT_CAP } from '../../stores/events/KvShareHandleStore.js';
import {
SCHEDULE_LIST_LIMIT_CAP,
schedulesUnavailable,
type ScheduleServiceContract,
} from '../../core/schedules.js';
import { normalizeLimit } from '../../util/pagination.js';
import { PuterController } from '../types.js';
import { EVENTS_WORKER_PREFIX } from '../../services/events/workerRuntime.js';
@@ -59,6 +64,8 @@ const EVENTS_SCRIPT_REGEX = new RegExp(`^${EVENTS_WORKER_PREFIX}[a-f0-9]{32}$`);
/** Matches the edge dispatcher's own validation of the app uid it forwards. */
const APP_UID_REGEX = /^app-[A-Za-z0-9_-]{1,64}$/;
/** A user uuid. */
const HOLDER_REGEX = /^[A-Za-z0-9-]{1,64}$/;
@Controller('/events')
export class EventsController extends PuterController {
@@ -104,12 +111,31 @@ export class EventsController extends PuterController {
});
}
/**
* POST /events/subscriptions/update — replace one subscription's `context`
* in place. An id the caller does not hold reads as absent.
*/
@Post('/subscriptions/update', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async update(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const { sub } = await this.services.events.updateDurable(
actor,
bodyRecord(req),
);
res.json(sub);
}
/**
* GET /events/fetch — what the caller missed, one page at a time.
*
* Stateless: no subscription, no stored position, nothing written. The
* cursor comes back to the caller and goes out again as `cursor` (or
* `after`, the name it first shipped under).
* `after`, the name it first shipped under). `from=end` with no cursor
* starts at the newest row instead of the oldest.
*/
@Get('/fetch', {
subdomain: 'api',
@@ -120,9 +146,16 @@ export class EventsController extends PuterController {
async fetch(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const from = query.from as 'start' | 'end' | undefined;
if (from !== undefined && from !== 'start' && from !== 'end') {
throw new HttpError(400, '`from` must be `start` or `end`', {
legacyCode: 'bad_request',
});
}
const page = await this.services.events.fetchMissed(actor, {
subject: typeof query.subject === 'string' ? query.subject : '',
from,
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
after: typeof query.after === 'string' ? query.after : undefined,
limit: normalizeLimit(query.limit, {
@@ -151,11 +184,105 @@ export class EventsController extends PuterController {
res.json({});
}
// -- Schedules ---------------------------------------------------
//
// An app's handler run on a timer, served by `services.schedules` when a
// server has it and answered `events_disabled` when it doesn't. Creating
// one needs an app context; listing, reading and removing are scoped the
// way the durable verbs are. The create and remove verbs spend the
// subscribe and unsubscribe budgets in the service, shared with those
// verbs.
#schedules(): ScheduleServiceContract {
const schedules = this.services.schedules;
if (!schedules) throw schedulesUnavailable();
return schedules;
}
/** POST /events/schedules — create a schedule for the calling app. */
@Post('/schedules', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async createSchedule(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(await this.#schedules().createFor(actor, bodyRecord(req)));
}
/** GET /events/schedules — the caller's schedules, one page at a time. */
@Get('/schedules', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async listSchedules(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.#schedules().list(actor, {
limit: normalizeLimit(query.limit, {
cap: SCHEDULE_LIST_LIMIT_CAP,
}),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
});
res.json({
items: page.items,
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
@Post('/schedules/update', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async updateSchedule(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const { scheduleId, changes } = bodyRecord(req);
res.json(await this.#schedules().update(actor, scheduleId, changes));
}
/**
* POST /events/schedules/remove — end one schedule. An id the caller does
* not hold reads as absent.
*/
@Post('/schedules/remove', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async removeSchedule(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
await this.#schedules().remove(actor, bodyRecord(req).scheduleId);
res.json({});
}
/**
* GET /events/schedules/:scheduleId — one schedule, ended or not. An id the
* caller does not hold reads as absent.
*/
@Get('/schedules/:scheduleId', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async getSchedule(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(await this.#schedules().get(actor, req.params.scheduleId));
}
// -- Cross-user key-value handles --------------------------------
/**
* POST /events/kv-handles — hand another user a watchable region of this
* account's key-value data.
* POST /events/kv-handles — hand another user a region of this account's
* key-value data, `watch` unless the body asks for more. The older name for
* `POST /kv/shares`, keeping its own body and response shapes.
*
* An account session, or an app session holding a `manage:` delegation on
* the region — the service tells those apart and gates each rather than the
@@ -167,12 +294,32 @@ export class EventsController extends PuterController {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
requireVerified: true,
})
async mintKvHandle(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.mintKvHandle(actor, bodyRecord(req)),
const body = bodyRecord(req);
const share = await this.services.kvShare.mintKvHandle(
actor,
{
prefix: body.prefix,
recipient: {
username:
typeof body.granteeUsername === 'string'
? body.granteeUsername
: undefined,
uid:
typeof body.granteeUid === 'string'
? body.granteeUid
: undefined,
},
access: body.access,
appUid: body.appUid,
},
// Callers of this route mint again to get the same handle back.
{ defaultAccess: 'watch', keepLevelUnlessNamed: true },
);
res.json({ handle: share.handle, prefix: share.prefix });
}
/**
@@ -189,7 +336,7 @@ export class EventsController extends PuterController {
const actor = this.#requireActor(req);
const query = (req.query ?? {}) as Record<string, unknown>;
const page = await this.services.events.listKvHandles(actor, {
const page = await this.services.kvShare.listKvHandles(actor, {
limit: normalizeLimit(query.limit, {
cap: KV_HANDLE_LIST_LIMIT_CAP,
}),
@@ -198,7 +345,10 @@ export class EventsController extends PuterController {
});
res.json({
items: page.items,
items: page.items.map(({ holderUsername, ...row }) => ({
...row,
granteeUsername: holderUsername,
})),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
@@ -217,7 +367,10 @@ export class EventsController extends PuterController {
async revokeKvHandle(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.events.revokeKvHandle(actor, req.params.handle),
await this.services.kvShare.revokeKvHandle(
actor,
req.params.handle,
),
);
}
@@ -270,6 +423,7 @@ export class EventsController extends PuterController {
handlers: await this.services.events.listHandlers(actor, {
appUid:
typeof query.appUid === 'string' ? query.appUid : undefined,
scope: query.scope,
}),
});
}
@@ -368,6 +522,15 @@ export class EventsController extends PuterController {
throw new HttpError(400, 'Missing or invalid `script`/`appUid`', {
legacyCode: 'bad_request',
});
// Absent or empty is the app's own scope.
const holder = body.holder === undefined ? '' : body.holder;
if (
typeof holder !== 'string' ||
!(holder === '' || HOLDER_REGEX.test(holder))
)
throw new HttpError(400, 'Invalid `holder`', {
legacyCode: 'bad_request',
});
const deployer = this.#eventsWorkerDeployer();
if (!deployer.enabled) {
@@ -375,15 +538,25 @@ export class EventsController extends PuterController {
return;
}
const outcome = await deployer.ensure(appUid, script);
const scope = await deployer.scopeFor(appUid, holder);
if (!scope) {
res.status(404).json({ deployed: false, reason: 'no-holder' });
return;
}
const outcome = await deployer.ensure(scope, script);
if (outcome === 'deployed') {
res.json({ deployed: true });
return;
}
// `failed` and `throttled` are worth another attempt: the rest say
// this script is not what the app's handlers currently are.
// `failed`, `throttled` and `busy` are worth another attempt: the rest
// say this script is not what the app's handlers currently are.
res.status(
outcome === 'failed' || outcome === 'throttled' ? 502 : 404,
outcome === 'failed' ||
outcome === 'throttled' ||
outcome === 'busy'
? 502
: 404,
).json({
deployed: false,
reason: outcome,
@@ -404,14 +577,22 @@ export class EventsController extends PuterController {
override onServerStart(): void {
const deployer = this.#eventsWorkerDeployer();
if (!deployer.enabled) return;
this.services.events.useEagerDeploy((scope, script) =>
deployer.deployEagerly(scope, script),
);
if (this.config.workers?.localServer) {
this.services.events.useWorkerTransport(
new LocalEventsInvokeTransport(this.services, deployer),
);
return;
}
this.services.events.useWorkerMissHandler((appUid, script) =>
deployer.ensure(appUid, script),
this.services.events.useWorkerMissHandler(
async (appUid, script, options) => {
const scope = await deployer.scopeFor(appUid, options.holder);
return scope
? deployer.ensure(scope, script, options)
: 'no-holder';
},
);
}
+73 -11
View File
@@ -34,23 +34,46 @@ import {
EVENTS_DURABLE_SUBSCRIPTIONS_MAX,
EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP,
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER,
EVENTS_HANDLER_PUBLISH_LIMIT,
EVENTS_HANDLER_SOURCE_MAX_BYTES,
EVENTS_HANDLERS_PER_APP,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_USER,
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT,
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT,
EVENTS_SINGLE_DELIVERY_LIMIT,
EVENTS_USER_HANDLERS_PER_SCOPE,
EVENTS_USER_SCOPE_SOURCE_MAX_BYTES,
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_INVOCATION_LIMIT,
EVENTS_WORKER_SOURCE_MAX_BYTES,
RENTED_UNITS_PER_USER,
limitFor,
tiered,
type TieredLimit,
} from './limits.js';
const tiers: Array<[string, TieredLimit]> = [
['durable subscriptions per account', EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER],
[
'durable subscriptions per account',
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER,
],
['durable subscriptions per app', EVENTS_DURABLE_SUBSCRIPTIONS_PER_APP],
['live share handles per account', EVENTS_KV_HANDLES_PER_USER],
['live share handles per app', EVENTS_KV_HANDLES_PER_APP],
];
describe('tiered', () => {
it('gives paid plans the base, and the free and temporary tiers their own', () => {
const limit = tiered(30, 20, 10);
expect(limitFor(limit, null)).toBe(30);
expect(limitFor(limit, 'some_plan')).toBe(30);
expect(limitFor(limit, DEFAULT_FREE_SUBSCRIPTION)).toBe(20);
expect(limitFor(limit, DEFAULT_TEMP_SUBSCRIPTION)).toBe(10);
});
});
describe('the tiered subscription quotas', () => {
it.each(tiers)('%s never lets a free tier exceed paid', (_name, tier) => {
for (const n of Object.values(tier.bySubscription))
@@ -105,9 +128,9 @@ describe('the tiered subscription quotas', () => {
});
it('holds an unrecognised plan to the paid base', () => {
expect(limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER, 'some_plan')).toBe(
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit,
);
expect(
limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER, 'some_plan'),
).toBe(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit);
expect(limitFor(EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER, null)).toBe(
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER.limit,
);
@@ -131,22 +154,22 @@ describe('the delivery budgets', () => {
expect(
limitFor(EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT, plan),
).toBe(matched);
expect(
limitFor(EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT, plan),
).toBe(evaluated);
expect(limitFor(EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT, plan)).toBe(
evaluated,
);
expect(evaluated).toBeGreaterThanOrEqual(matched);
}
});
it('pins the share-handle quotas by plan', () => {
expect(EVENTS_KV_HANDLES_PER_USER.limit).toBe(512);
expect(EVENTS_KV_HANDLES_PER_APP.limit).toBe(512);
expect(EVENTS_KV_HANDLES_PER_USER.limit).toBe(25_000);
expect(EVENTS_KV_HANDLES_PER_APP.limit).toBe(10_000);
expect(
limitFor(EVENTS_KV_HANDLES_PER_USER, DEFAULT_FREE_SUBSCRIPTION),
).toBe(200);
).toBe(5_000);
expect(
limitFor(EVENTS_KV_HANDLES_PER_APP, DEFAULT_FREE_SUBSCRIPTION),
).toBe(128);
).toBe(2_000);
});
it('keeps `single` well under broadcast — each one costs far more', () => {
@@ -169,3 +192,42 @@ describe('the delivery budgets', () => {
}
});
});
describe('the handler limits', () => {
it('pins the app-scope numbers the quotas page lists', () => {
expect(EVENTS_HANDLERS_PER_APP).toBe(100);
expect(EVENTS_HANDLER_SOURCE_MAX_BYTES).toBe(64 * 1024);
expect(EVENTS_WORKER_SOURCE_MAX_BYTES).toBe(5 * 1024 * 1024);
expect(EVENTS_WORKER_DEPLOYS_PER_HOUR).toBe(30);
expect(EVENTS_HANDLER_PUBLISH_LIMIT.limit).toBe(60);
expect(EVENTS_HANDLER_PUBLISH_LIMIT.window).toBe(60_000);
});
it('pins the user-scope numbers the quotas page lists', () => {
expect(EVENTS_USER_HANDLERS_PER_SCOPE).toBe(20);
expect(EVENTS_USER_SCOPE_SOURCE_MAX_BYTES).toBe(1024 * 1024);
expect(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR).toBe(10);
});
it('keeps a user scope tighter than an app scope', () => {
expect(EVENTS_USER_HANDLERS_PER_SCOPE).toBeLessThan(
EVENTS_HANDLERS_PER_APP,
);
expect(EVENTS_USER_SCOPE_SOURCE_MAX_BYTES).toBeLessThan(
EVENTS_WORKER_SOURCE_MAX_BYTES,
);
expect(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR).toBeLessThan(
EVENTS_WORKER_DEPLOYS_PER_HOUR,
);
});
it('pins the rented-unit cap by plan', () => {
expect(RENTED_UNITS_PER_USER.limit).toBe(100);
expect(limitFor(RENTED_UNITS_PER_USER, DEFAULT_FREE_SUBSCRIPTION)).toBe(
16,
);
expect(limitFor(RENTED_UNITS_PER_USER, DEFAULT_TEMP_SUBSCRIPTION)).toBe(
16,
);
});
});
+123 -28
View File
@@ -53,7 +53,12 @@ export interface TieredLimit {
bySubscription: Record<string, number>;
}
const tiered = (paid: number, free: number, temp: number): TieredLimit => ({
/** Paid, free and temporary caps as a {@link TieredLimit}. Called by extensions. */
export const tiered = (
paid: number,
free: number,
temp: number,
): TieredLimit => ({
limit: paid,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: free,
@@ -93,6 +98,18 @@ export interface TieredQuota {
*/
export const EVENTS_SESSION_SUBSCRIPTIONS_PER_SOCKET = 50;
/**
* Live subscriptions one user may hold through a share handle someone else
* minted for them, across all of their connections.
*
* Those rows sit in the owner's keyspace, where each one costs a filter
* evaluation on the owner's writes. The count also takes in connections that
* dropped and haven't been reaped yet, so the cap sits far above what
* reconnects leave behind. The owner's own rows through the handle don't
* count.
*/
export const EVENTS_SESSION_SUBSCRIPTIONS_PER_KV_HANDLE = 1000;
/**
* Durable subscriptions one account may hold, across every app.
*
@@ -125,6 +142,13 @@ export const EVENTS_DURABLE_SUBSCRIPTIONS_MAX =
*/
export const SUSPENDED_ROW_TTL_DAYS = 30;
/**
* Subscriptions stopped by a revoked grant that one account keeps for `list`.
* They hold no quota slot, so past this the oldest are deleted when the account
* subscribes again, keeping its rows bounded.
*/
export const EVENTS_REVOKED_SUBSCRIPTIONS_KEPT = 500;
/**
* Subscribe calls per minute, per user.
*
@@ -155,6 +179,13 @@ export const EVENTS_LIST_LIMIT = userWindow('events:list', 120);
*/
export const EVENTS_ACK_LIMIT = userWindow('events:ack', 600);
/**
* How long a connected client holds a queued delivery before the events worker
* may take it. A client that has no handler for it, or whose handler fails,
* declines it at once; this only bounds one that never answers.
*/
export const EVENTS_SOCKET_LEASE_MS = 15_000;
/**
* Share-handle mint + revoke calls per minute, per user.
*
@@ -164,22 +195,67 @@ export const EVENTS_ACK_LIMIT = userWindow('events:ack', 600);
export const EVENTS_KV_HANDLE_LIMIT = userWindow('events:kvHandles', 60);
/**
* Live share handles one account may hold out at a time, across every app.
*
* Each is a standing grant on part of the account's data, and revoking marks
* rather than deletes, so without a ceiling the rate limit alone lets the rows
* grow forever. A temporary account holds none: the grant outlives the session
* that made it, and no account is left to take it back.
* `kv.share.received` notices per hour, per owner and holder. A share past it
* is still made, without a notice.
*/
export const EVENTS_KV_HANDLES_PER_USER = tiered(512, 200, 0);
export const KV_SHARE_NOTICE_LIMIT = userWindow(
'kvShare:notices',
5,
60 * 60_000,
);
/**
* Live share handles one app may hold out for one account.
* Live share handles one account may hold out at a time, across every app.
*
* Separately bounded from the account cap so one namespace cannot consume the
* whole budget on free accounts.
* A backstop above the per-app cap: every count reads a covering index, so the
* bound is on standing grants, not query cost. A temporary account holds none:
* the grant outlives the session that made it, and no account is left to take
* it back.
*/
export const EVENTS_KV_HANDLES_PER_APP = tiered(512, 128, 0);
export const EVENTS_KV_HANDLES_PER_USER = tiered(25_000, 5_000, 0);
/**
* Live share handles one app may hold out for one account, so one namespace
* cannot consume the whole account budget.
*/
export const EVENTS_KV_HANDLES_PER_APP = tiered(10_000, 2_000, 0);
/**
* Live share handles one owner may hold out to one holder, on every plan.
* Bounds the rows a revoke or unshare walks to find the narrower handles inside
* one.
*/
export const EVENTS_KV_HANDLES_PER_HOLDER = 256;
/**
* Recipients one account may share with, per minute and per hour. A call spends
* one per recipient that doesn't already hold the share, on top of the per-call
* mint budget.
*/
export const KV_SHARE_GRANT_LIMITS = [
userWindow('kvShare:grants:minute', 300),
userWindow('kvShare:grants:hour', 2_000, 60 * 60_000),
] as const;
/**
* Recipients one account may name in share calls per minute, existing shares
* included. Each one is looked up before the call knows whether it changes
* anything, so a client that shares again on every load is bounded here.
*/
export const KV_SHARE_LOOKUP_LIMIT = userWindow('kvShare:lookups', 1_200);
/** Recipients one `POST /kv/shares` call may name. */
export const KV_SHARE_BATCH_MAX = 50;
/**
* Notifications one party of a share may send the other through share handles,
* per minute and per day, in one app. Keeps one share from spending the
* recipient's whole notification allowance.
*/
export const KV_SHARE_NOTIFY_LIMITS = [
userWindow('kvShare:notify:minute', 30),
userWindow('kvShare:notify:day', 500, 24 * 60 * 60_000),
] as const;
// -- Handler surface -------------------------------------------------
@@ -193,6 +269,12 @@ export const EVENTS_KV_HANDLES_PER_APP = tiered(512, 128, 0);
*/
export const EVENTS_HANDLERS_PER_APP = 100;
/**
* Handlers one user may hold in their own scope of one app. Each is a script of
* its own, so the cap is tighter than the app's.
*/
export const EVENTS_USER_HANDLERS_PER_SCOPE = 20;
/** Longest a serialized handler may be. */
export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
@@ -204,6 +286,9 @@ export const EVENTS_HANDLER_SOURCE_MAX_BYTES = 64 * 1024;
*/
export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
/** Source one user scope may hold, all its handlers combined. */
export const EVENTS_USER_SCOPE_SOURCE_MAX_BYTES = 1024 * 1024;
/**
* Deploys one app's events worker may trigger per hour.
*
@@ -213,6 +298,16 @@ export const EVENTS_WORKER_SOURCE_MAX_BYTES = 5 * 1024 * 1024;
*/
export const EVENTS_WORKER_DEPLOYS_PER_HOUR = 30;
/** Deploys one user scope may trigger per hour. */
export const EVENTS_USER_WORKER_DEPLOYS_PER_HOUR = 10;
/**
* Units one user may rent at once: ordinary workers, user scopes, and the
* events workers of apps they own. Only creating a unit is refused; holdings
* over the cap are kept.
*/
export const RENTED_UNITS_PER_USER = tiered(100, 16, 16);
/**
* Handlers one `publishAll` may carry. A build step publishes its whole set in
* one call, and the set is capped by what an app may hold anyway.
@@ -277,8 +372,11 @@ export const EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT = tiered(512, 128, 128);
*/
export const EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT = tiered(2048, 512, 512);
/** Omit all inline KV values when the matched audience exceeds this count. */
export const EVENTS_KV_VALUE_OMIT_MATCHED_SUBSCRIPTIONS = 128;
/**
* Deliveries of one key-value change that carry its value inline, in the order
* the fan-out serves them. Later ones carry the key alone.
*/
export const EVENTS_KV_VALUE_DELIVERIES_PER_EVENT = 128;
/**
* Largest inline KV value in serialized bytes. Larger values are omitted;
@@ -370,16 +468,20 @@ export const EVENTS_REGION_PENDING_CEILING = 1_000_000;
// a handler that answers "no" is not. Retrying on a fixed cadence turns one
// broken deploy into a permanent load on whatever is failing, so the wait
// doubles per attempt up to a ceiling, and a run of failures stops the
// subscription rather than retrying it forever.
// subscription rather than retrying it forever. A run Puter could not start or
// finish backs off the same way but is not the handler's failure, so it never
// counts toward that run.
/** Wait before the first retry of a delivery a handler could not take. */
export const EVENTS_RETRY_BASE_MS = 2_000;
/** Longest wait between retries, however many have failed. */
export const EVENTS_RETRY_MAX_MS = 5 * 60 * 1000;
// Wait before the first retry of a delivery a handler could not take, the
// longest wait, and the doubling between them. Shared with system jobs.
export {
deliveryBackoffMs,
EVENTS_RETRY_BASE_MS,
EVENTS_RETRY_MAX_MS,
} from '../../util/retryBackoff.js';
/**
* Failures in a row before a subscription is suspended. Counted per
* Handler failures in a row before a subscription is suspended. Counted per
* subscription and reset by the first delivery a handler takes, so an
* occasional failure never accumulates into one.
*/
@@ -392,13 +494,6 @@ export const EVENTS_CONSECUTIVE_FAILURES = 5;
*/
export const EVENTS_FAILURE_COUNTER_TTL_MS = 60 * 60 * 1000;
/** How long to hold a delivery whose handler has failed `attempts` times. */
export const deliveryBackoffMs = (attempts: number): number =>
Math.min(
EVENTS_RETRY_MAX_MS,
EVENTS_RETRY_BASE_MS * 2 ** Math.max(0, attempts - 1),
);
// -- Suspended backlog -----------------------------------------------
//
// A suspended subscription stops metering but keeps what it is owed, and that
@@ -0,0 +1,289 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* The schedule routes are thin delegates: without `services.schedules` every
* one answers 503 `events_disabled`; with it, each hands its request over and
* answers what it returns.
*/
import type { Request, Response } from 'express';
import {
afterAll,
afterEach,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { makeActor, type Actor } from '../../core/actor.js';
import { isHttpError } from '../../core/http/HttpError.js';
import type {
ScheduleServiceContract,
ScheduleView,
} from '../../core/schedules.js';
import type { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { BroadcastController } from '../broadcast/BroadcastController.js';
import type { EventsController } from './EventsController.js';
let server: PuterServer;
let events: EventsController;
let broadcast: BroadcastController;
let actor: Actor;
let saved: unknown;
const SCHEDULE_ID = '0b3a3c9e-9f61-4f0e-8d2a-2f1b6f8f2c11';
const view = {
scheduleId: SCHEDULE_ID,
kind: 'recurring',
appUid: 'app-x',
} as ScheduleView;
const slot = () => server.services as unknown as { schedules?: unknown };
/** Put `schedules` in the slot, or leave the slot empty. */
const setSchedules = (schedules: unknown): void => {
if (schedules === undefined) delete slot().schedules;
else slot().schedules = schedules;
};
const fakeSchedules = () =>
({
onSubscriptionsChanged: vi.fn(async () => {}),
createFor: vi.fn(async () => view),
list: vi.fn(async () => ({
items: [view],
cursor: 'next',
total: 3,
})),
get: vi.fn(async () => view),
update: vi.fn(async () => view),
remove: vi.fn(async () => {}),
receivePeerChanges: vi.fn(async () => ({ applied: 2 })),
prepareSendAt: vi.fn(),
createDeferredAppMessage: vi.fn(),
}) satisfies ScheduleServiceContract;
const req = (init: {
body?: unknown;
query?: Record<string, unknown>;
params?: Record<string, string>;
}): Request =>
({
actor,
body: init.body ?? {},
query: init.query ?? {},
params: init.params ?? {},
headers: {},
rawBody: Buffer.from(JSON.stringify(init.body ?? {})),
}) as unknown as Request;
const res = () => {
const sent: { status: number; body: unknown } = {
status: 200,
body: undefined,
};
const response = {
json: vi.fn((body: unknown) => {
sent.body = body;
return response;
}),
status: vi.fn((code: number) => {
sent.status = code;
return response;
}),
setHeader: vi.fn(() => response),
};
return { response: response as unknown as Response, sent };
};
/** Each route, called as the router would. */
const routes: Array<[string, (response: Response) => Promise<void>]> = [
[
'POST /events/schedules',
(r) => events.createSchedule(req({ body: { every: 60_000 } }), r),
],
[
'GET /events/schedules',
(r) => events.listSchedules(req({ query: { limit: '5' } }), r),
],
[
'GET /events/schedules/:scheduleId',
(r) =>
events.getSchedule(req({ params: { scheduleId: SCHEDULE_ID } }), r),
],
[
'POST /events/schedules/update',
(r) => events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes: { every: '2h' } } }), r),
],
[
'POST /events/schedules/remove',
(r) =>
events.removeSchedule(
req({ body: { scheduleId: SCHEDULE_ID } }),
r,
),
],
[
'POST /broadcast/events/schedules',
(r) =>
broadcast.schedules(
req({ body: { namespace: 'n', changes: [] } }),
r,
),
],
];
beforeAll(async () => {
server = await setupTestServer();
events = server.controllers.events as unknown as EventsController;
broadcast = server.controllers.broadcast as unknown as BroadcastController;
actor = makeActor({
user: { id: 1, uuid: 'u-1', username: 'someone' },
} as never);
saved = slot().schedules;
vi.spyOn(
server.services.broadcast,
'verifySignedRequest',
).mockResolvedValue({ ok: true, peerId: 'peer' } as never);
});
afterEach(() => {
setSchedules(saved);
});
afterAll(async () => {
vi.restoreAllMocks();
await server?.shutdown();
});
describe('without services.schedules', () => {
it.each(routes)('%s answers 503 events_disabled', async (_name, call) => {
setSchedules(undefined);
const { response } = res();
let thrown: unknown;
try {
await call(response);
} catch (err) {
thrown = err;
}
expect(isHttpError(thrown)).toBe(true);
expect(thrown).toMatchObject({
statusCode: 503,
legacyCode: 'events_disabled',
// The designed answer, so it never pages.
noAlarm: true,
});
});
it('refuses an unsigned peer batch before saying schedules are off', async () => {
setSchedules(undefined);
vi.spyOn(
server.services.broadcast,
'verifySignedRequest',
).mockResolvedValueOnce({
ok: false,
status: 401,
message: 'bad signature',
} as never);
const { response, sent } = res();
await broadcast.schedules(
req({ body: { namespace: 'n', changes: [] } }),
response,
);
expect(sent.status).toBe(401);
});
});
describe('with services.schedules', () => {
it('creates through createFor and answers its view', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
await events.createSchedule(req({ body: { every: 60_000 } }), response);
expect(schedules.createFor).toHaveBeenCalledWith(actor, {
every: 60_000,
});
expect(sent.body).toEqual(view);
});
it('lists with the limit capped, the cursor and includeTotal', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
await events.listSchedules(
req({
query: { limit: '5000', cursor: 'c', includeTotal: 'true' },
}),
response,
);
expect(schedules.list).toHaveBeenCalledWith(actor, {
limit: 200,
cursor: 'c',
includeTotal: true,
});
expect(sent.body).toEqual({ items: [view], cursor: 'next', total: 3 });
});
it('reads one through get', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
await events.getSchedule(
req({ params: { scheduleId: SCHEDULE_ID } }),
response,
);
expect(schedules.get).toHaveBeenCalledWith(actor, SCHEDULE_ID);
expect(sent.body).toEqual(view);
});
it('updates through update and answers its view', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
const changes = { every: '2h' };
await events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes } }), response);
expect(schedules.update).toHaveBeenCalledWith(actor, SCHEDULE_ID, changes);
expect(sent.body).toEqual(view);
});
it('removes through remove and answers {}', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
await events.removeSchedule(
req({ body: { scheduleId: SCHEDULE_ID } }),
response,
);
expect(schedules.remove).toHaveBeenCalledWith(actor, SCHEDULE_ID);
expect(sent.body).toEqual({});
});
it('hands a signed peer batch to receivePeerChanges', async () => {
const schedules = fakeSchedules();
setSchedules(schedules);
const { response, sent } = res();
const body = { namespace: 'n', changes: [] };
await broadcast.schedules(req({ body }), response);
expect(schedules.receivePeerChanges).toHaveBeenCalledWith(body);
expect(sent).toEqual({ status: 200, body: { ok: true, applied: 2 } });
});
});
@@ -0,0 +1,77 @@
import type { Request, Response } from 'express';
import { describe, expect, it } from 'vitest';
import { makeActor } from '../../core/actor.js';
import type {
ScheduleServiceContract,
ScheduleView,
} from '../../core/schedules.js';
import { EventsController } from './EventsController.js';
const actor = makeActor({
user: { id: 1, uuid: 'user-1', username: 'user1' } as never,
});
const request = (body: unknown) => ({ actor, body }) as Request;
const controller = (schedules?: Pick<ScheduleServiceContract, 'update'>) =>
new EventsController(
{} as never,
{} as never,
{} as never,
{ schedules } as never,
{} as never,
);
describe('update schedule route contract', () => {
it('answers events_disabled when schedules are unavailable', async () => {
await expect(
controller().updateSchedule(
request({ scheduleId: 'id', changes: {} }),
{} as Response,
),
).rejects.toMatchObject({
legacyCode: 'events_disabled',
statusCode: 503,
noAlarm: true,
});
});
it('passes the actor, id and changes through and returns the updated view', async () => {
const view = {
scheduleId: 'id',
kind: 'recurring',
state: 'suspended',
} as ScheduleView;
const changes = { cron: '0 9 * * *', tz: 'UTC' };
const calls: unknown[][] = [];
const events = controller({
update: async (...args) => {
calls.push(args);
return view;
},
});
let sent: unknown;
await events.updateSchedule(request({ scheduleId: 'id', changes }), {
json: (value: unknown) => {
sent = value;
},
} as Response);
expect(calls).toEqual([[actor, 'id', changes]]);
expect(sent).toBe(view);
});
it('preserves validation and sending errors from the implementation', async () => {
const error = {
code: 'events_schedule_sending',
message: 'Already sending',
};
await expect(
controller({
update: async () => {
throw error;
},
}).updateSchedule(
request({ scheduleId: 'id', changes: {} }),
{} as Response,
),
).rejects.toBe(error);
});
});
@@ -25,9 +25,35 @@
import MockRedis from 'ioredis-mock';
import { describe, expect, it } from 'vitest';
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
import { EventsWorkerDeployer } from './workerDeploy.js';
import { eventsWorkerScript } from '../../services/events/workerRuntime.js';
import type { fetch as undiciFetch } from 'undici';
import {
DispatcherInvokeTransport,
EVENTS_DEPLOYED_HEADER,
EVENTS_DISPATCH_ERROR_HEADER,
EVENTS_HANDLED_HEADER,
EventsWorkerInvokerClient,
type EventsInvokeCall,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import { INTERNAL_DEPLOY_TARGET } from '../../drivers/workers/WorkerDriver.js';
import {
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_DEPLOYS_PER_HOUR,
} from './limits.js';
import {
EventsWorkerDeployer,
LocalEventsInvokeTransport,
type EventsDeployOptions,
type EventsDeployOutcome,
} from './workerDeploy.js';
import {
eventsWorkerScript,
userHandlerScript,
} from '../../services/events/workerRuntime.js';
import {
appHandlerScope,
hashContent,
type HandlerScope,
} from '../../stores/events/EventHandlerStore.js';
import { handlerSetHash } from '../../services/events/workerSource.js';
import type { IConfig } from '../../types.js';
@@ -46,12 +72,22 @@ interface Owner {
type Layers = ConstructorParameters<typeof EventsWorkerDeployer>[0];
/** Stub layers whose only real dependency is a fresh redis mock. */
const makeLayers = (owners: Record<string, Owner | null>) => {
const makeLayers = (
owners: Record<string, Owner | null>,
events: Record<string, unknown> = {},
) => {
const redis = new MockRedis.Cluster(['redis://localhost:7001']);
const created: string[] = [];
const targets: Array<Record<string, unknown>> = [];
/** Apps whose handler sources were read, once per read. */
const sourceReads: string[] = [];
const layers = {
config: {
events: { workerRuntime: true, internalSecret: 'sekret' },
events: {
workerRuntime: true,
internalSecret: 'sekret',
...events,
},
// `#namespace()` needs a local-worker shape to resolve without
// throwing — this stub never reaches a real Cloudflare or
// Miniflare deploy either way.
@@ -60,16 +96,20 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
clients: { redis },
stores: {
eventHandler: {
allForApp: async (appUid: string) => [
{
appUid,
name: 'h',
source: 'async () => {}',
sourceHash: SOURCE_HASH,
createdAt: 0,
updatedAt: 0,
},
],
setForApp: async () => [{ name: 'h', sourceHash: SOURCE_HASH }],
allForApp: async ({ appUid }: { appUid: string }) => {
sourceReads.push(appUid);
return [
{
appUid,
name: 'h',
source: 'async () => {}',
sourceHash: SOURCE_HASH,
createdAt: 0,
updatedAt: 0,
},
];
},
},
app: {
getByUid: async (appUid: string) => {
@@ -80,6 +120,18 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
},
},
user: {
getByUuid: async (uuid: string) => {
const owner = Object.values(owners).find(
(o) => `uuid-${o?.ownerUserId}` === uuid,
);
if (!owner) return null;
return {
id: owner.ownerUserId,
uuid,
username: `user-${owner.ownerUserId}`,
suspended: owner.suspended ?? false,
};
},
getById: async (id: number) => {
const owner = Object.values(owners).find(
(o) => o?.ownerUserId === id,
@@ -97,14 +149,24 @@ const makeLayers = (owners: Record<string, Owner | null>) => {
services: {},
drivers: {
workers: {
create: async (args: { workerName: string }) => {
create: async (args: {
workerName: string;
[INTERNAL_DEPLOY_TARGET]?: Record<string, unknown>;
}) => {
created.push(args.workerName);
targets.push(args[INTERNAL_DEPLOY_TARGET] ?? {});
return { success: true, errors: [] };
},
},
},
};
return { layers: layers as unknown as Layers, created, redis };
return {
layers: layers as unknown as Layers,
created,
targets,
redis,
sourceReads,
};
};
describe('EventsWorkerDeployer.ensure', () => {
@@ -120,15 +182,17 @@ describe('EventsWorkerDeployer.ensure', () => {
const deployer = new EventsWorkerDeployer(layers);
const [a, b] = await Promise.all([
deployer.ensure('app-a', SCRIPT),
deployer.ensure('app-b', SCRIPT),
deployer.ensure(appHandlerScope('app-a'), SCRIPT),
deployer.ensure(appHandlerScope('app-b'), SCRIPT),
]);
expect(a).toBe('deployed');
expect(b).toBe('no-owner');
});
it('refuses once the app has deployed past the hourly cap', async () => {
const { layers, redis } = makeLayers({ 'app-c': { ownerUserId: 3 } });
it('refuses once the app has deployed past the hourly cap, before reading or building any source', async () => {
const { layers, redis, sourceReads, created } = makeLayers({
'app-c': { ownerUserId: 3 },
});
const deployer = new EventsWorkerDeployer(layers);
const hour = new Date().toISOString().slice(0, 13);
await redis.set(
@@ -136,12 +200,697 @@ describe('EventsWorkerDeployer.ensure', () => {
String(EVENTS_WORKER_DEPLOYS_PER_HOUR),
);
expect(await deployer.ensure('app-c', SCRIPT)).toBe('throttled');
expect(await deployer.ensure(appHandlerScope('app-c'), SCRIPT)).toBe(
'throttled',
);
expect(sourceReads).toEqual([]);
expect(created).toEqual([]);
});
it('answers a stale script without spending the hourly cap', async () => {
const { layers, redis, sourceReads } = makeLayers({
'app-e': { ownerUserId: 5 },
});
const deployer = new EventsWorkerDeployer(layers);
const stale = eventsWorkerScript(
handlerSetHash([{ name: 'h', sourceHash: 'b'.repeat(64) }]),
'',
);
expect(await deployer.ensure(appHandlerScope('app-e'), stale)).toBe(
'stale',
);
expect(sourceReads).toEqual([]);
const hour = new Date().toISOString().slice(0, 13);
expect(await redis.get(`ev:deploys:{app-e}:${hour}`)).toBeNull();
});
it('passes events.workerCpuMs to the deploy target', async () => {
const { layers, targets } = makeLayers(
{ 'app-f': { ownerUserId: 6 } },
{ workerCpuMs: 50 },
);
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure(appHandlerScope('app-f'), SCRIPT)).toBe(
'deployed',
);
expect(targets[0]!.cpuMs).toBe(50);
});
it('passes no cpuMs when events.workerCpuMs is unset', async () => {
const { layers, targets } = makeLayers({ 'app-g': { ownerUserId: 7 } });
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure(appHandlerScope('app-g'), SCRIPT)).toBe(
'deployed',
);
expect(targets[0]!.cpuMs).toBeUndefined();
});
it('deploys normally under the cap', async () => {
const { layers } = makeLayers({ 'app-d': { ownerUserId: 4 } });
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.ensure('app-d', SCRIPT)).toBe('deployed');
expect(await deployer.ensure(appHandlerScope('app-d'), SCRIPT)).toBe(
'deployed',
);
});
});
/** A server's schedules with a deploy gate that records and can refuse. */
const withGate = (
layers: Layers,
refuse: (capped: boolean) => boolean = () => false,
) => {
const gated: Array<{ script: string; capped: boolean }> = [];
const gate = {
gateDeploy: async (
script: string,
options: { capped: boolean },
deploy: () => Promise<EventsDeployOutcome>,
): Promise<EventsDeployOutcome> => {
gated.push({ script, capped: options.capped });
return refuse(options.capped) ? 'busy' : deploy();
},
};
const deployer = new EventsWorkerDeployer({
...layers,
services: { ...layers.services, schedules: gate },
} as unknown as Layers);
return { deployer, gated };
};
describe('the schedules` deploy gate', () => {
it('is skipped without schedules, or with schedules that have no gate', async () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
await expect(
new EventsWorkerDeployer(layers).ensure(
appHandlerScope('app-x'),
SCRIPT,
{
lane: 'schedule',
},
),
).resolves.toBe('deployed');
const gateless = new EventsWorkerDeployer({
...layers,
services: { ...layers.services, schedules: {} },
} as unknown as Layers);
await expect(
gateless.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
}),
).resolves.toBe('deployed');
expect(created).toEqual([SCRIPT, SCRIPT]);
});
it('takes every deploy, capped only for a schedule run', async () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer, gated } = withGate(layers);
await expect(
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
).resolves.toBe('deployed');
await expect(
deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
}),
).resolves.toBe('deployed');
expect(gated).toEqual([
{ script: SCRIPT, capped: false },
{ script: SCRIPT, capped: true },
]);
expect(created).toEqual([SCRIPT, SCRIPT]);
});
it('answers a refused schedule run busy, and a delivery still deploys', async () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer } = withGate(layers, (capped) => capped);
await expect(
deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
}),
).resolves.toBe('busy');
await expect(
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
).resolves.toBe('deployed');
expect(created).toEqual([SCRIPT]);
});
it('lets deliveries that join a schedule run`s refused deploy deploy once', async () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer, gated } = withGate(layers, (capped) => capped);
const scheduled = deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
});
const deliveries = [
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
deployer.ensure(appHandlerScope('app-x'), SCRIPT),
];
await expect(scheduled).resolves.toBe('busy');
await expect(Promise.all(deliveries)).resolves.toEqual([
'deployed',
'deployed',
]);
expect(created).toEqual([SCRIPT]);
expect(gated.map((g) => g.capped)).toEqual([true, false]);
});
it('lets a schedule run join a delivery`s deploy in flight', async () => {
const { layers, created } = makeLayers({ 'app-x': { ownerUserId: 1 } });
const { deployer, gated } = withGate(layers, (capped) => capped);
const delivery = deployer.ensure(appHandlerScope('app-x'), SCRIPT);
const scheduled = deployer.ensure(appHandlerScope('app-x'), SCRIPT, {
lane: 'schedule',
});
await expect(Promise.all([delivery, scheduled])).resolves.toEqual([
'deployed',
'deployed',
]);
expect(gated).toEqual([{ script: SCRIPT, capped: false }]);
expect(created).toEqual([SCRIPT]);
});
});
describe('LocalEventsInvokeTransport', () => {
it('renders each request as it goes out, after a cold deploy', async () => {
const bodies: string[] = [];
let resident = false;
const transport = new LocalEventsInvokeTransport(
{
localworkerservice: {
dispatchEventsWorker: async (request: Request) => {
bodies.push(await request.text());
if (!resident) return null;
return new Response(null, {
status: 200,
headers: { 'x-puter-events-handled': '1' },
});
},
},
} as never,
{
ensure: async () => {
resident = true;
return 'deployed';
},
scopeFor: async (appUid: string) => appHandlerScope(appUid),
} as unknown as EventsWorkerDeployer,
);
let renders = 0;
const call: EventsInvokeCall = {
script: SCRIPT,
appUid: 'app-x',
key: 'k',
body: '{"n":0}',
timeoutMs: 5000,
render: () => `{"n":${++renders}}`,
};
await expect(transport.send(call)).resolves.toMatchObject({
status: 200,
});
expect(bodies).toEqual(['{"n":1}', '{"n":2}']);
});
});
describe('deploy-on-miss of an evicted script', () => {
it('redeploys the evicted script, then resends the run with the deployed marker', async () => {
const { layers, created } = makeLayers({ 'app-m': { ownerUserId: 8 } });
// The namespace only has the script once the deploy lands: before
// that the dispatcher answers a miss.
let resident = false;
layers.drivers.workers.create = (async (args: {
workerName: string;
}) => {
created.push(args.workerName);
resident = true;
return { success: true, errors: [] };
}) as never;
const deployer = new EventsWorkerDeployer(layers);
const sent: boolean[] = [];
const fetchImpl = (async (_url: string, init?: RequestInit) => {
const headers = (init?.headers ?? {}) as Record<string, string>;
sent.push(headers[EVENTS_DEPLOYED_HEADER] === '1');
if (resident)
return new Response(null, {
status: 200,
headers: { [EVENTS_HANDLED_HEADER]: '1' },
});
return new Response(null, {
status: 404,
headers: { [EVENTS_DISPATCH_ERROR_HEADER]: 'missing' },
});
}) as unknown as typeof undiciFetch;
const client = new EventsWorkerInvokerClient({
events: { workerRuntime: true },
} as IConfig);
client.setTransport(
new DispatcherInvokeTransport('http://dispatcher', 's', {
fetchImpl,
}),
);
client.setMissHandler((appUid, script, options) =>
deployer.ensure(appHandlerScope(appUid), script, options),
);
const result = await client.invoke({
script: SCRIPT,
appUid: 'app-m',
handler: 'h',
token: 't',
key: 'k',
event: {},
ctx: {},
});
expect(result).toEqual({ outcome: 'settled', status: 200 });
expect(created).toEqual([SCRIPT]);
// One miss, then the resend carrying the deployed marker.
expect(sent).toEqual([false, true]);
});
});
// The redis mock is shared across tests, so each one uses its own app and code.
let fresh = 0;
const nextApp = () => `app-u${++fresh}`;
const userScope = (holder: number, appUid: string) => ({ appUid, holder });
const sourceN = (label: string) =>
`async () => { await ${label}${++fresh}(); }`;
const scriptOf = (name: string, source: string) =>
userHandlerScript({ name, sourceHash: hashContent(source) }, '');
type Sets = Record<number, Array<{ name: string; source: string }>>;
/** Layers whose handler sets are per holder (0 is the app's own). */
const withSets = (layers: Layers, sets: Sets) => {
const handlersOf = (scope: { appUid: string; holder: number }) =>
(sets[scope.holder] ?? []).map((h) => ({
appUid: scope.appUid,
name: h.name,
source: h.source,
sourceHash: hashContent(h.source),
createdAt: 0,
updatedAt: 0,
}));
const store = layers.stores.eventHandler as unknown as Record<
string,
unknown
>;
store.setForApp = async (scope: { appUid: string; holder: number }) =>
handlersOf(scope).map(({ name, sourceHash }) => ({ name, sourceHash }));
store.allForApp = async (scope: { appUid: string; holder: number }) =>
handlersOf(scope);
};
const holders = (...ids: number[]) =>
Object.fromEntries(
ids.map((id) => [`holder-${id}`, { ownerUserId: id }]),
) as Record<string, Owner>;
describe('EventsWorkerDeployer in a user scope', () => {
it('deploys the one handler whose script was asked for, as the holder', async () => {
const app = nextApp();
const a = sourceN('a');
const b = sourceN('b');
const { layers, created, targets } = makeLayers(holders(10));
withSets(layers, {
10: [
{ name: 'a', source: a },
{ name: 'b', source: b },
],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('b', b);
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script]);
const target = targets[0]!;
expect(target.tags).toEqual([`events-app:${app}`, 'user:uuid-10']);
// One handler's script: the other one's code is not in it.
expect(target.source).toContain(b);
expect(target.source).not.toContain(a);
});
it('answers a script no handler of the scope matches as stale, before the throttle', async () => {
const app = nextApp();
const { layers, redis, created } = makeLayers(holders(10));
withSets(layers, { 10: [{ name: 'a', source: sourceN('a') }] });
const deployer = new EventsWorkerDeployer(layers);
expect(
await deployer.ensure(
userScope(10, app),
scriptOf('b', sourceN('b')),
),
).toBe('stale');
expect(created).toEqual([]);
const hour = new Date().toISOString().slice(0, 13);
expect(await redis.get(`ev:deploys:{${app}}:10:${hour}`)).toBeNull();
});
it('answers a scope with no handlers as no-handlers', async () => {
const { layers } = makeLayers(holders(10));
withSets(layers, {});
const deployer = new EventsWorkerDeployer(layers);
expect(
await deployer.ensure(
userScope(10, nextApp()),
scriptOf('a', sourceN('a')),
),
).toBe('no-handlers');
});
it('does not deploy for a holder that is suspended or gone', async () => {
const source = sourceN('a');
const { layers, created } = makeLayers({
'holder-10': { ownerUserId: 10, suspended: true },
});
withSets(layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
const app = nextApp();
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'no-owner',
);
expect(await deployer.ensure(userScope(11, app), script)).toBe(
'no-owner',
);
expect(created).toEqual([]);
});
it('gives each scope its own hourly cap, apart from the app scope', async () => {
const app = nextApp();
const source = sourceN('a');
const { layers, created } = makeLayers({
...holders(10, 11),
[app]: { ownerUserId: 12 },
});
withSets(layers, {
0: [{ name: 'a', source }],
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
for (let i = 0; i < EVENTS_USER_WORKER_DEPLOYS_PER_HOUR; i++)
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'deployed',
);
expect(await deployer.ensure(userScope(10, app), script)).toBe(
'throttled',
);
expect(created).toHaveLength(EVENTS_USER_WORKER_DEPLOYS_PER_HOUR);
// Another holder of the same app, and the app's own scope, still go.
expect(await deployer.ensure(userScope(11, app), script)).toBe(
'deployed',
);
expect(await deployer.ensure(appHandlerScope(app), script)).toBe(
'deployed',
);
});
it('collapses concurrent deploys of one script across holders into one upload', async () => {
const app = nextApp();
const source = sourceN('a');
const { layers, created } = makeLayers(holders(10, 11));
withSets(layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
await Promise.all([
deployer.ensure(userScope(10, app), script),
deployer.ensure(userScope(10, app), script),
deployer.ensure(userScope(11, app), script),
deployer.ensure(userScope(11, nextApp()), script),
]);
expect(created).toHaveLength(1);
});
it('names a script for identical code the same under any holder and app', async () => {
const source = sourceN('a');
const { layers, created } = makeLayers(holders(10, 11));
withSets(layers, {
10: [{ name: '@x', source }],
11: [{ name: '@x', source }],
});
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('@x', source);
expect(await deployer.ensure(userScope(10, nextApp()), script)).toBe(
'deployed',
);
expect(await deployer.ensure(userScope(11, nextApp()), script)).toBe(
'deployed',
);
expect(created).toEqual([script, script]);
});
});
describe('the deployed marker', () => {
const setup = () => {
const source = sourceN('a');
const made = makeLayers(holders(10, 11));
withSets(made.layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
return {
...made,
deployer: new EventsWorkerDeployer(made.layers),
script: scriptOf('a', source),
app: nextApp(),
};
};
it('is left by a user-scope deploy with a ttl under a day', async () => {
const { deployer, redis, script, app } = setup();
await deployer.ensure(userScope(10, app), script);
const ttl = await redis.ttl(`ev:deployed:${script}`);
expect(ttl).toBeGreaterThan(0);
expect(ttl).toBeLessThan(24 * 60 * 60);
});
it('lets an eager deploy skip the upload another holder already made', async () => {
const { deployer, created, script, app } = setup();
expect(await deployer.deployEagerly(userScope(10, app), script)).toBe(
'deployed',
);
expect(await deployer.deployEagerly(userScope(11, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script]);
});
it('spends no throttle on a skipped eager deploy', async () => {
const { deployer, redis, script, app } = setup();
await deployer.deployEagerly(userScope(10, app), script);
await deployer.deployEagerly(userScope(11, app), script);
const hour = new Date().toISOString().slice(0, 13);
expect(await redis.get(`ev:deploys:{${app}}:11:${hour}`)).toBeNull();
});
it('never skips a deploy-on-miss', async () => {
const { deployer, created, script, app } = setup();
await deployer.deployEagerly(userScope(10, app), script);
expect(await deployer.ensure(userScope(11, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script, script]);
});
it('is not left by an app-scope deploy', async () => {
const app = nextApp();
const source = sourceN('a');
const { layers, redis, created } = makeLayers({
[app]: { ownerUserId: 4 },
});
withSets(layers, { 0: [{ name: 'a', source }] });
const deployer = new EventsWorkerDeployer(layers);
const script = scriptOf('a', source);
await deployer.ensure(appHandlerScope(app), script);
expect(created).toEqual([script]);
expect(await redis.get(`ev:deployed:${script}`)).toBeNull();
});
it('still deploys eagerly when the marker cannot be read', async () => {
const { deployer, redis, created, script, app } = setup();
await deployer.deployEagerly(userScope(10, app), script);
const get = redis.get.bind(redis);
redis.get = (async (key: string) => {
if (key.startsWith('ev:deployed:')) throw new Error('down');
return get(key);
}) as never;
expect(await deployer.deployEagerly(userScope(11, app), script)).toBe(
'deployed',
);
expect(created).toEqual([script, script]);
});
});
describe('scopeFor', () => {
it('reads no holder as the app scope, and resolves a uuid to the user scope', async () => {
const { layers } = makeLayers(holders(10));
const deployer = new EventsWorkerDeployer(layers);
expect(await deployer.scopeFor('app-u')).toEqual({
appUid: 'app-u',
holder: 0,
});
expect(await deployer.scopeFor('app-u', '')).toEqual({
appUid: 'app-u',
holder: 0,
});
expect(await deployer.scopeFor('app-u', 'uuid-10')).toEqual({
appUid: 'app-u',
holder: 10,
});
expect(await deployer.scopeFor('app-u', 'uuid-99')).toBeNull();
});
});
describe('LocalEventsInvokeTransport in a user scope', () => {
it('deploys the holder`s script on a miss and refuses an unknown holder', async () => {
const source = sourceN('a');
const { layers, created } = makeLayers(holders(10));
withSets(layers, { 10: [{ name: 'a', source }] });
const deployer = new EventsWorkerDeployer(layers);
let resident = false;
const transport = new LocalEventsInvokeTransport(
{
localworkerservice: {
dispatchEventsWorker: async () => {
if (!resident) return null;
return new Response(null, {
status: 200,
headers: { 'x-puter-events-handled': '1' },
});
},
},
} as never,
{
ensure: async (
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
) => {
const outcome = await deployer.ensure(
scope,
script,
options,
);
resident = outcome === 'deployed';
return outcome;
},
scopeFor: (appUid: string, holder?: string) =>
deployer.scopeFor(appUid, holder),
} as unknown as EventsWorkerDeployer,
);
const script = scriptOf('a', source);
const call: EventsInvokeCall = {
script,
appUid: nextApp(),
holder: 'uuid-10',
key: 'k',
body: '{}',
timeoutMs: 5000,
};
await expect(transport.send(call)).resolves.toMatchObject({
status: 200,
});
expect(created).toEqual([script]);
resident = false;
await expect(
transport.send({ ...call, holder: 'uuid-99' }),
).resolves.toMatchObject({ status: null, error: 'deploy: no-holder' });
expect(created).toEqual([script]);
});
});
describe('concurrent eager deploys of one script', () => {
const setup = () => {
const source = sourceN('a');
const made = makeLayers(holders(10, 11));
withSets(made.layers, {
10: [{ name: 'a', source }],
11: [{ name: 'a', source }],
});
return {
...made,
script: scriptOf('a', source),
app: nextApp(),
};
};
it('upload once when two holders publish the same new code together', async () => {
const { layers, created, script, app } = setup();
const deployer = new EventsWorkerDeployer(layers);
const outcomes = await Promise.all([
deployer.deployEagerly(userScope(10, app), script),
deployer.deployEagerly(userScope(11, app), script),
]);
expect(outcomes).toEqual(['deployed', 'deployed']);
expect(created).toEqual([script]);
});
it('skip the upload when the marker appears while they wait on the gate', async () => {
const { layers, redis, created, script, app } = setup();
// Another node's deploy of the script finishes before the gate opens.
const gated = new EventsWorkerDeployer({
...layers,
services: {
schedules: {
gateDeploy: async (
_script: string,
_options: { capped: boolean },
deploy: () => Promise<EventsDeployOutcome>,
) => {
await redis.set(`ev:deployed:${script}`, '1');
return deploy();
},
},
},
} as unknown as Layers);
expect(
await gated.ensure(userScope(10, app), script, { eager: true }),
).toBe('deployed');
expect(created).toEqual([]);
// A miss still uploads, whatever the marker says.
expect(await gated.ensure(userScope(11, app), script)).toBe('deployed');
expect(created).toEqual([script]);
});
it('make a miss that joins an eager deploy deploy after it', async () => {
const { layers, redis, created, script, app } = setup();
const deployer = new EventsWorkerDeployer(layers);
await redis.set(`ev:deployed:${script}`, '1');
const eager = deployer.ensure(userScope(10, app), script, {
eager: true,
});
const miss = deployer.ensure(userScope(11, app), script);
expect(await eager).toBe('deployed');
expect(await miss).toBe('deployed');
// The eager one skipped on the marker; the miss uploaded.
expect(created).toEqual([script]);
});
});
+310 -72
View File
@@ -26,50 +26,100 @@ import {
INTERNAL_ADMISSION_BYPASS,
INTERNAL_DEPLOY_TARGET,
} from '../../drivers/workers/WorkerDriver.js';
import type { EventsInvokeCall } from '../../clients/events/EventsWorkerInvokerClient.js';
import type {
EventsInvokeAnswer,
EventsInvokeCall,
InvokeLane,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import {
EVENTS_HANDLED_HEADER,
EVENTS_INVOKE_PATH,
answerFrom,
failedAnswer,
} from '../../clients/events/EventsWorkerInvokerClient.js';
import type { puterServices } from '../../services/index.js';
import type { IExtensionServiceInstances } from '../../services/types.js';
import {
eventsInvokeKey,
eventsWorkerScope,
eventsWorkerScript,
userHandlerScript,
} from '../../services/events/workerRuntime.js';
import { generateEventsWorkerSource } from '../../services/events/workerSource.js';
import {
generateEventsWorkerSource,
handlerSetHash,
} from '../../services/events/workerSource.js';
import {
appHandlerScope,
type HandlerScope,
} from '../../stores/events/EventHandlerStore.js';
import type { puterStores } from '../../stores/index.js';
import type { IConfig, LayerInstances } from '../../types.js';
import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
import {
EVENTS_USER_WORKER_DEPLOYS_PER_HOUR,
EVENTS_WORKER_DEPLOYS_PER_HOUR,
} from './limits.js';
/**
* Deploys an app's events worker from its published handlers.
*
* Deployed the first time a delivery needs it and again on eviction — a publish
* only writes rows. Scripts carry no worker token and no `subdomains` row: not
* one of the owner's workers, not addressable from the internet, not listed or
* deleted as if it were. A republished set leaves the old script behind rather
* than deleting it; redeploying a still-live one is always safe.
* only writes rows (a user-scope publish also deploys eagerly). An app's scope
* is one script for the whole set; a user scope is one script per handler,
* deployed as the holder. Scripts carry no worker token and no `subdomains`
* row: not one of the owner's workers, not addressable from the internet, not
* listed or deleted as if it were. A republished set leaves the old script
* behind rather than deleting it; redeploying a still-live one is always safe.
*/
/** Why a deploy could not happen, for the callers that answer differently. */
export type EventsDeployOutcome =
'deployed' | 'stale' | 'no-handlers' | 'no-owner' | 'throttled' | 'failed';
| 'deployed'
| 'stale'
| 'no-handlers'
| 'no-owner'
| 'throttled'
| 'failed'
| 'busy';
/** Whose call asked for the deploy: a schedule run's is held to a share. */
export interface EventsDeployOptions {
lane?: InvokeLane;
/**
* A user-scope publish's deploy, which skips the upload when the script's
* deployed marker is already set. A deploy-on-miss is never eager.
*/
eager?: boolean;
}
interface DeployLayers {
config: IConfig;
clients: LayerInstances<typeof puterClients>;
stores: LayerInstances<typeof puterStores>;
services: LayerInstances<typeof puterServices>;
services: LayerInstances<typeof puterServices> &
Pick<IExtensionServiceInstances, 'schedules'>;
drivers: LayerInstances<typeof puterDrivers>;
}
/**
* `ev:deploys:{<appUid>}:<hour>` bucket TTL — comfortably past the hour it
* counts.
* `ev:deploys:{<appUid>}[:<holder>]:<hour>` bucket TTL — comfortably past the
* hour it counts.
*/
const DEPLOY_THROTTLE_TTL_SECONDS = 2 * 60 * 60;
/** How long `ev:deployed:<script>` says a script was uploaded. Under a day. */
const DEPLOYED_MARKER_TTL_SECONDS = 20 * 60 * 60;
const deployedMarkerKey = (script: string): string => `ev:deployed:${script}`;
const isUserScope = (scope: HandlerScope): boolean => scope.holder !== 0;
interface InFlightDeploy {
/** A schedule run's, which the region's deploy slots may refuse. */
capped: boolean;
eager: boolean;
promise: Promise<EventsDeployOutcome>;
}
interface DeployResult {
success?: boolean;
errors?: unknown[];
@@ -78,7 +128,7 @@ interface DeployResult {
export class EventsWorkerDeployer {
readonly #layers: DeployLayers;
/** Collapses the burst of misses one cold app produces into one deploy. */
readonly #inFlight = new Map<string, Promise<EventsDeployOutcome>>();
readonly #inFlight = new Map<string, InFlightDeploy>();
constructor(layers: DeployLayers) {
this.#layers = layers;
@@ -93,39 +143,187 @@ export class EventsWorkerDeployer {
}
/**
* Deploy `script`, which must be what the app's handlers currently hash to.
* A request for anything else is `stale` — the set moved on, and the next
* delivery resolves the new script rather than this one being revived.
* Deploy `script`, which must be what the scope's handlers currently hash
* to: the whole set for an app, one handler for a user scope. A request for
* anything else is `stale` — the handlers moved on, and the next delivery
* resolves the new script rather than this one being revived. Always
* uploads; a deploy-on-miss means whatever said it was there is wrong.
*/
ensure(appUid: string, script: string): Promise<EventsDeployOutcome> {
const key = `${appUid}|${script}`;
ensure(
scope: HandlerScope,
script: string,
options: EventsDeployOptions = {},
): Promise<EventsDeployOutcome> {
// A user scope's script is the same bytes whoever holds it, so
// concurrent deploys of it from any holder or app are one.
const key = isUserScope(scope)
? `user|${script}`
: `${scope.appUid}|${scope.holder}|${script}`;
const capped = options.lane === 'schedule';
const existing = this.#inFlight.get(key);
if (existing) return existing;
const started = this.#deploy(appUid, script).finally(() =>
this.#inFlight.delete(key),
if (!existing) return this.#start(key, scope, script, options);
// An eager deploy may have skipped the upload, which a miss must not
// take for done.
if (existing.eager && !options.eager)
return this.#afterEager(
existing.promise,
key,
scope,
script,
options,
);
if (capped || !existing.capped) return existing.promise;
return this.#afterCapped(existing.promise, key, scope, script, options);
}
/**
* {@link ensure} for the deploy a user-scope publish starts, which skips the
* upload when another holder's deploy of the same script left its marker.
* Only this path reads the marker.
*/
async deployEagerly(
scope: HandlerScope,
script: string,
): Promise<EventsDeployOutcome> {
if (await this.#markedDeployed(script)) return 'deployed';
return this.ensure(scope, script, { eager: true });
}
/**
* The scope a script request names: the app's own, or the user scope of
* `holder` (a user uuid). Null for a holder that does not exist.
*/
async scopeFor(
appUid: string,
holder?: string | null,
): Promise<HandlerScope | null> {
if (!holder) return appHandlerScope(appUid);
const user = await this.#layers.stores.user.getByUuid(holder);
if (!user?.id) return null;
return { appUid, holder: Number(user.id) };
}
#start(
key: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
const entry: InFlightDeploy = {
capped: options.lane === 'schedule',
eager: options.eager === true,
promise: this.#gated(scope, script, options).finally(() => {
if (this.#inFlight.get(key) === entry)
this.#inFlight.delete(key);
}),
};
this.#inFlight.set(key, entry);
return entry.promise;
}
/** A miss that found an eager deploy in flight deploys after it. */
async #afterEager(
eager: Promise<EventsDeployOutcome>,
key: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
await eager;
const current = this.#inFlight.get(key);
if (current && !current.eager) return current.promise;
return this.#start(key, scope, script, options);
}
/**
* A delivery that joined a schedule run's deploy: `busy` means no deploy
* started, and a delivery is never refused a slot, so it asks again.
*/
async #afterCapped(
capped: Promise<EventsDeployOutcome>,
key: string,
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
const outcome = await capped;
if (outcome !== 'busy') return outcome;
const current = this.#inFlight.get(key);
if (current && !current.capped) return current.promise;
return this.#start(key, scope, script, options);
}
/**
* Every deploy goes through the schedules' deploy gate when a server has
* one: a per-script lock, so a burst of runs of an evicted script deploys
* it once, and a region-wide cap that a schedule run's deploy (`capped`)
* may be refused with `busy`. A delivery's deploy is never refused.
*/
#gated(
scope: HandlerScope,
script: string,
options: EventsDeployOptions,
): Promise<EventsDeployOutcome> {
const deploy = () =>
this.#deploy(scope, script, options.eager === true);
return (
this.#layers.services.schedules?.gateDeploy?.(
script,
{ capped: options.lane === 'schedule' },
deploy,
) ?? deploy()
);
this.#inFlight.set(key, started);
return started;
}
async #deploy(
appUid: string,
handlerScope: HandlerScope,
script: string,
eager: boolean,
): Promise<EventsDeployOutcome> {
const { appUid } = handlerScope;
const { stores, drivers } = this.#layers;
const backend = eventsWorkerScope(this.#layers.config);
const userScope = isUserScope(handlerScope);
const handlers = await stores.eventHandler.allForApp(appUid);
if (handlers.length === 0) return 'no-handlers';
const generated = generateEventsWorkerSource(handlers);
// Names and hashes are enough to tell a stale request, so the set's
// sources are read and built only once the throttle has let it through.
const set = await stores.eventHandler.setForApp(handlerScope);
if (set.length === 0) return 'no-handlers';
// Two apps that published byte-identical sets share one script, which
// is sound: the source is the same, and everything per-app arrives with
// the invocation. Whoever deploys first owns it.
// the invocation. Whoever deploys first owns it. A user scope's script
// is one handler's, so it is shared the same way across holders.
const wanted = userScope
? set.find((entry) => userHandlerScript(entry, backend) === script)
: undefined;
if (
eventsWorkerScript(
generated.setHash,
eventsWorkerScope(this.#layers.config),
) !== script
userScope
? !wanted
: eventsWorkerScript(handlerSetHash(set), backend) !== script
)
return 'stale';
// Asked again inside the gate, where another deploy of this script may
// have just finished.
if (eager && userScope && (await this.#markedDeployed(script)))
return 'deployed';
if (await this.#tooManyDeploysThisHour(handlerScope))
return 'throttled';
const all = await stores.eventHandler.allForApp(handlerScope);
const handlers = wanted
? all.filter(
(handler) =>
handler.name === wanted.name &&
handler.sourceHash === wanted.sourceHash,
)
: all;
const generated = generateEventsWorkerSource(handlers);
// The set can move between the two reads.
if (
handlers.length === 0 ||
eventsWorkerScript(generated.setHash, backend) !== script
)
return 'stale';
if (generated.tooLarge) {
@@ -140,9 +338,7 @@ export class EventsWorkerDeployer {
generated.broken.join(', '),
);
if (await this.#tooManyDeploysThisHour(appUid)) return 'throttled';
const owner = await this.#owner(appUid);
const owner = await this.#owner(handlerScope);
if (!owner) return 'no-owner';
const secret = this.#layers.config.events?.internalSecret;
@@ -171,6 +367,7 @@ export class EventsWorkerDeployer {
runtime: 'events',
source: generated.source,
omitOwnerToken: true,
cpuMs: this.#layers.config.events?.workerCpuMs,
// What the script compares an invocation against.
// Derived from the script name, so a redeploy of
// the same set binds the same key.
@@ -201,34 +398,74 @@ export class EventsWorkerDeployer {
);
return 'failed';
}
if (userScope) await this.#markDeployed(script);
return 'deployed';
}
/**
* Whether this app has already deployed `EVENTS_WORKER_DEPLOYS_PER_HOUR`
* times this hour. A flapping set (publish, remove, republish) provisions
* upstream on every miss, and this is the ceiling on that loop.
*/
async #tooManyDeploysThisHour(appUid: string): Promise<boolean> {
const redis = this.#layers.clients.redis;
const hour = new Date().toISOString().slice(0, 13);
const key = `ev:deploys:{${appUid}}:${hour}`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
return count > EVENTS_WORKER_DEPLOYS_PER_HOUR;
async #markedDeployed(script: string): Promise<boolean> {
try {
return !!(await this.#layers.clients.redis.get(
deployedMarkerKey(script),
));
} catch (err) {
console.warn('[events] deployed marker unreadable', err);
return false;
}
}
/** The app owner, as an actor the deploy can run as. */
async #markDeployed(script: string): Promise<void> {
try {
await this.#layers.clients.redis.set(
deployedMarkerKey(script),
'1',
'EX',
DEPLOYED_MARKER_TTL_SECONDS,
);
} catch (err) {
console.warn('[events] could not mark a script deployed', err);
}
}
/**
* Whether this scope has already deployed its hourly cap this hour. A
* flapping set (publish, remove, republish) provisions upstream on every
* miss, and this is the ceiling on that loop. An app's counter is per app;
* a user scope has its own, so one user cannot spend another's.
*/
async #tooManyDeploysThisHour(scope: HandlerScope): Promise<boolean> {
const redis = this.#layers.clients.redis;
const hour = new Date().toISOString().slice(0, 13);
const userScope = isUserScope(scope);
const key = userScope
? `ev:deploys:{${scope.appUid}}:${scope.holder}:${hour}`
: `ev:deploys:{${scope.appUid}}:${hour}`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, DEPLOY_THROTTLE_TTL_SECONDS);
return (
count >
(userScope
? EVENTS_USER_WORKER_DEPLOYS_PER_HOUR
: EVENTS_WORKER_DEPLOYS_PER_HOUR)
);
}
/**
* The user a scope's deploy runs as, as an actor: the app's owner, or the
* holder of a user scope.
*/
async #owner(
appUid: string,
scope: HandlerScope,
): Promise<
| (Actor & { user: { id: number; uuid: string; username: string } })
| null
> {
const { stores } = this.#layers;
const app = await stores.app.getByUid(appUid);
const ownerUserId = Number(app?.owner_user_id);
if (!app || !Number.isFinite(ownerUserId)) return null;
let ownerUserId = scope.holder;
if (!isUserScope(scope)) {
const app = await stores.app.getByUid(scope.appUid);
ownerUserId = Number(app?.owner_user_id);
if (!app || !Number.isFinite(ownerUserId)) return null;
}
const user = await stores.user.getById(ownerUserId);
if (!user?.id || !user.uuid || !user.username || user.suspended)
@@ -289,11 +526,9 @@ export class LocalEventsInvokeTransport {
this.#deployer = deployer;
}
async send(call: EventsInvokeCall): Promise<{
status: number | null;
handled?: boolean;
error?: string;
}> {
async send(call: EventsInvokeCall): Promise<EventsInvokeAnswer> {
// Built per attempt, so a send time stamped into the body is taken
// after a cold deploy rather than before it.
const request = (): Request =>
new Request(
`http://${call.script}.workers.puter.localhost${EVENTS_INVOKE_PATH}`,
@@ -303,7 +538,7 @@ export class LocalEventsInvokeTransport {
'content-type': 'application/json',
'x-puter-events-key': call.key,
},
body: call.body,
body: call.render ? call.render() : call.body,
},
);
@@ -311,22 +546,27 @@ export class LocalEventsInvokeTransport {
const resident = await this.#dispatch(request(), call.timeoutMs);
if (resident !== null) return resident;
const outcome = await this.#deployer.ensure(
const scope = await this.#deployer.scopeFor(
call.appUid,
call.script,
call.holder,
);
if (!scope) return { status: null, error: 'deploy: no-holder' };
const outcome = await this.#deployer.ensure(scope, call.script, {
lane: call.lane,
});
if (outcome !== 'deployed')
return { status: null, error: `deploy: ${outcome}` };
return {
status: null,
error: `deploy: ${outcome}`,
...(outcome === 'busy' ? { deployBusy: true } : {}),
};
const retried = await this.#dispatch(request(), call.timeoutMs);
if (retried === null)
return { status: null, error: 'script not resident' };
return retried;
} catch (err) {
return {
status: null,
error: err instanceof Error ? err.message : String(err),
};
return failedAnswer(err);
}
}
@@ -337,7 +577,7 @@ export class LocalEventsInvokeTransport {
async #dispatch(
request: Request,
timeoutMs: number,
): Promise<{ status: number; handled: boolean } | null> {
): Promise<EventsInvokeAnswer | null> {
let timer: ReturnType<typeof setTimeout> | undefined;
const timeout = new Promise<'timeout'>((resolve) => {
timer = setTimeout(() => resolve('timeout'), timeoutMs);
@@ -348,12 +588,10 @@ export class LocalEventsInvokeTransport {
this.#services.localworkerservice.dispatchEventsWorker(request),
timeout,
]);
if (response === 'timeout') throw new Error('invocation timed out');
if (response === 'timeout')
throw new DOMException('invocation timed out', 'TimeoutError');
if (response === null) return null;
return {
status: response.status,
handled: response.headers.get(EVENTS_HANDLED_HEADER) === '1',
};
return answerFrom(response);
} finally {
clearTimeout(timer);
}
@@ -0,0 +1,80 @@
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
describe('recursive readdir with offset', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const post = async (path: string, token: string, body: unknown) =>
fetch(new URL(path, env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
it('skips entries before applying the recursive page limit', async () => {
const { username, token } = env.users.user;
const base = `/${username}/Documents/rec-offset`;
for (const path of [base, `${base}/a`, `${base}/b`, `${base}/c`]) {
expect((await post('/fs/mkdir', token, { path })).status).toBe(200);
}
const list = async (body: Record<string, unknown>) => {
const response = await post('/fs/readdir', token, {
path: base,
recursive: true,
...body,
});
expect(response.status).toBe(200);
const page = (await response.json()) as {
items: Array<{ name: string }>;
};
return page.items.map((item) => item.name);
};
expect(await list({ limit: 3 })).toEqual(['a', 'b', 'c']);
// Non-recursive honours offset.
const flat = await post('/fs/readdir', token, {
path: base,
limit: 1,
offset: 2,
});
expect(
((await flat.json()) as Array<{ name: string }>).map((e) => e.name),
).toEqual(['c']);
expect(await list({ limit: 1, offset: 2 })).toEqual(['c']);
expect(await list({ offset: 2 })).toEqual(['c']);
expect(await list({ offset: 20 })).toEqual([]);
expect(await list({ limit: 1, offset: 1, sortOrder: 'desc' })).toEqual([
'b',
]);
const first = await post('/fs/readdir', token, {
path: base,
recursive: true,
limit: 1,
offset: 1,
});
const page = await first.json();
expect(page.items.map((item: { name: string }) => item.name)).toEqual([
'b',
]);
expect(page.cursor).toBeTypeOf('string');
expect(await list({ limit: 1, cursor: page.cursor })).toEqual(['c']);
// A cursor wins over offset, and an invalid offset is ignored, as
// recursive listings did before they honoured offset.
expect(
await list({ limit: 1, cursor: page.cursor, offset: 1 }),
).toEqual(['c']);
expect(await list({ limit: 1, offset: -1 })).toEqual(['a']);
});
});
@@ -1335,6 +1335,7 @@ export class FSController extends PuterController {
parent.path,
{
limit,
offset,
cursor:
typeof body.cursor === 'string'
? body.cursor
@@ -0,0 +1,114 @@
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { makeActor } from '../../core/actor.js';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
describe('/auth/check-app-acl', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const post = (path: string, body: unknown, token: string) =>
fetch(new URL(path, env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
const makeDirAndApp = async () => {
const { username, token } = env.users.user;
const server = env.server;
const user = (await server.stores.user.getByUsername(username))!;
const path = `/${username}/Documents/cacl-${uuidv4()}`;
expect((await post('/fs/mkdir', { path }, token)).status).toBe(200);
const entry = (await server.stores.fsEntry.getEntryByPath(path))!;
const app = await (
server.stores.app.create as unknown as (
f: Record<string, unknown>,
o: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `cacl-${uuidv4()}`,
title: 'Check app acl',
index_url: `https://cacl-${uuidv4()}.test/`,
},
{ ownerUserId: user.id },
);
return { user, path, entry, app };
};
it('allows a granted file without poisoning the app permission cache', async () => {
const { token } = env.users.user;
const server = env.server;
const { user, path, entry, app } = await makeDirAndApp();
const grant = await post(
'/auth/grant-user-app',
{ app_uid: app.uid, permission: `fs:${entry.uuid}:read` },
token,
);
expect(grant.status).toBe(200);
const res = await post(
'/auth/check-app-acl',
{ subject: { path }, app: app.uid, mode: 'read' },
token,
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ allowed: true });
const asApp = makeActor({ user, app: { uid: app.uid, id: app.id } });
const descriptor = () => server.services.acl.fsDescriptor(path);
expect(
await server.services.acl.check(asApp, descriptor(), 'read'),
).toBe(true);
});
it('denies an app with no grant', async () => {
const { path, app } = await makeDirAndApp();
const res = await post(
'/auth/check-app-acl',
{ subject: { path }, app: app.uid, mode: 'read' },
env.users.user.token,
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ allowed: false });
});
it('refuses app and access tokens', async () => {
const { token, apiToken } = env.users.user;
const { path, entry, app } = await makeDirAndApp();
const grant = await post(
'/auth/grant-user-app',
{ app_uid: app.uid, permission: `fs:${entry.uuid}:read` },
token,
);
expect(grant.status).toBe(200);
const minted = await post(
'/auth/get-user-app-token',
{ app_uid: app.uid },
token,
);
expect(minted.status).toBe(200);
const appToken = ((await minted.json()) as { token: string }).token;
for (const credential of [appToken, apiToken]) {
const res = await post(
'/auth/check-app-acl',
{ subject: { path }, app: app.uid, mode: 'read' },
credential,
);
expect(res.status).toBe(403);
}
});
});
@@ -0,0 +1,204 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const post = (path: string, body: unknown) =>
fetch(new URL(path, env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${env.users.user.token}`,
},
body: JSON.stringify(body),
});
const write = (path: string, content: string) =>
post('/fs/write', {
fileMetadata: { path, size: content.length, contentType: 'text/plain' },
fileContent: content,
});
const read = async (path: string) => {
const url = new URL('/read', env.apiOrigin);
url.searchParams.set('file', path);
const response = await fetch(url, {
headers: { Authorization: `Bearer ${env.users.user.token}` },
});
return response.text();
};
const makeDir = async (name: string) => {
const path = `/${env.users.user.username}/Documents/${name}-${randomUUID().slice(0, 8)}`;
expect((await post('/fs/mkdir', { path })).status).toBe(200);
return path;
};
describe('legacy /copy onto an existing file', () => {
it('replaces the file with overwrite', async () => {
const dir = await makeDir('copy-onto');
expect((await write(`${dir}/a.txt`, 'aaa')).status).toBe(200);
expect((await write(`${dir}/b.txt`, 'bbbbbb')).status).toBe(200);
const copy = await post('/copy', {
source: `${dir}/a.txt`,
destination: `${dir}/b.txt`,
overwrite: true,
});
expect(copy.status).toBe(200);
const [result] = (await copy.json()) as Array<{
copied: { name: string; path: string };
overwritten?: { name: string };
}>;
expect(result!.copied.name).toBe('b.txt');
expect(result!.overwritten?.name).toBe('b.txt');
expect(await read(`${dir}/b.txt`)).toBe('aaa');
expect(await read(`${dir}/a.txt`)).toBe('aaa');
});
it('refuses without overwrite and leaves the file alone', async () => {
const dir = await makeDir('copy-onto-refused');
await write(`${dir}/a.txt`, 'aaa');
await write(`${dir}/b.txt`, 'bbbbbb');
const copy = await post('/copy', {
source: `${dir}/a.txt`,
destination: `${dir}/b.txt`,
});
expect(copy.status).toBe(409);
expect(((await copy.json()) as { code: string }).code).toBe(
'item_with_same_name_exists',
);
expect(await read(`${dir}/b.txt`)).toBe('bbbbbb');
});
it('still copies into a directory destination under the same name', async () => {
const dir = await makeDir('copy-into');
await write(`${dir}/a.txt`, 'aaa');
expect((await post('/fs/mkdir', { path: `${dir}/sub` })).status).toBe(
200,
);
const copy = await post('/copy', {
source: `${dir}/a.txt`,
destination: `${dir}/sub`,
});
expect(copy.status).toBe(200);
expect(await read(`${dir}/sub/a.txt`)).toBe('aaa');
});
it('refuses to overwrite a file with itself', async () => {
const dir = await makeDir('copy-self');
await write(`${dir}/a.txt`, 'aaa');
for (const destination of [`${dir}/a.txt`, dir]) {
const copy = await post('/copy', {
source: `${dir}/a.txt`,
destination,
overwrite: true,
});
expect(copy.status).toBe(400);
}
expect(await read(`${dir}/a.txt`)).toBe('aaa');
});
});
describe('signed folder listing', () => {
const signedListing = async (path: string) => {
const signed = await post('/sign', {
items: [{ path, action: 'read' }],
});
expect(signed.status).toBe(200);
const readUrl = new URL(
(await signed.json()).signatures[0].read_url as string,
);
return fetch(
new URL(`${readUrl.pathname}${readUrl.search}`, env.apiOrigin),
);
};
it('lists every child of a folder larger than one query', async () => {
const dir = await makeDir('big');
const parent = (await env.server.stores.fsEntry.getEntryByPath(dir))!;
const total = 10_050;
const now = Math.floor(Date.now() / 1000);
for (let start = 0; start < total; start += 500) {
const rows: unknown[][] = [];
for (let i = start; i < Math.min(total, start + 500); i++) {
const name = `f${String(i).padStart(5, '0')}.txt`;
rows.push([
randomUUID(),
parent.userId,
parent.id,
parent.uuid,
name,
`${dir}/${name}`,
0,
now,
now,
now,
0,
]);
}
await env.server.clients.db.write(
`INSERT INTO fsentries (uuid, user_id, parent_id, parent_uid, name, path, is_dir, modified, created, accessed, size) VALUES ${rows.map(() => '(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)').join(', ')}`,
rows.flat(),
);
}
const response = await signedListing(dir);
expect(response.status).toBe(200);
expect(response.headers.get('content-type')).toMatch(
/^application\/json/,
);
const children = (await response.json()) as Array<{
fsentry_name: string;
write_url?: string;
}>;
expect(children).toHaveLength(total);
expect(children[0]!.fsentry_name).toBe('f00000.txt');
expect(children.at(-1)!.fsentry_name).toBe('f10049.txt');
expect(new Set(children.map((c) => c.fsentry_name)).size).toBe(total);
expect(children.some((c) => c.write_url)).toBe(false);
}, 120_000);
it('answers a small folder in one body', async () => {
const dir = await makeDir('small');
await write(`${dir}/one.txt`, '1');
const response = await signedListing(dir);
expect(response.status).toBe(200);
expect(response.headers.get('content-length')).not.toBeNull();
const children = (await response.json()) as Array<{
fsentry_name: string;
}>;
expect(children.map((c) => c.fsentry_name)).toEqual(['one.txt']);
});
});
@@ -153,6 +153,34 @@ describe('legacy file downloads over HTTP', () => {
},
);
it.each(routes)(
'GET %s answers a ranged read of an empty file with an empty 200',
async (route) => {
const file = await writeFile(
`empty-${route.replace(/\W/g, '')}.bin`,
Buffer.alloc(0),
);
const ranged = await requestFor(route, file, {
headers: { Range: 'bytes=0-511' },
});
expect(ranged.status).toBe(200);
expect(await ranged.text()).toBe('');
},
);
it('GET /read answers byte_count on an empty file with an empty 200', async () => {
const file = await writeFile('empty-byte-count.bin', Buffer.alloc(0));
for (const query of ['byte_count=512', 'offset=0']) {
const url = new URL(`/read?${query}`, env.apiOrigin);
url.searchParams.set('file', file.path);
const response = await fetch(url, {
headers: { Authorization: `Bearer ${env.users.user.token}` },
});
expect(response.status).toBe(200);
expect(await response.text()).toBe('');
}
});
// Storage hands over one chunk and then goes quiet, as a large object
// does while the client is still reading.
const stallReadsOf = (objectKey: string) => {
@@ -0,0 +1,160 @@
import type { Request, RequestHandler, Response } from 'express';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import type { LegacyFSController } from './LegacyFSController.js';
const PNG_2X2 =
'iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAYAAABytg0kAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAAEUlEQVR4nGP4z8DwH4QZYAwAR8oH+WdZbrcAAAAASUVORK5CYII=';
let server: PuterServer;
let handler: RequestHandler;
type Driver = {
create: (a: Record<string, unknown>) => Promise<Record<string, unknown>>;
};
beforeAll(async () => {
server = await setupTestServer({
api_base_url: 'http://api.test.local',
static_hosting_domain: 'site.test',
} as unknown as Parameters<typeof setupTestServer>[0]);
await server.services.appIcon.ensureIconsDirectory();
const controller = server.controllers
.legacyFs as unknown as LegacyFSController;
const router = new PuterRouter();
controller.registerRoutes(router);
handler = router.routes.find(
(r) =>
r.path === '/get-launch-apps' &&
String(r.method).toLowerCase() === 'get',
)!.handler as RequestHandler;
}, 60_000);
afterAll(async () => {
await server?.shutdown();
});
const makeActor = async (): Promise<{ actor: Actor; id: number }> => {
const username = `v13-${Math.random().toString(36).slice(2, 10)}`;
const u = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const r = (await server.stores.user.getById(u.id))!;
return {
id: r.id,
actor: {
user: {
id: r.id,
uuid: r.uuid,
username: r.username,
email: r.email ?? null,
email_confirmed: true,
},
} as unknown as Actor,
};
};
const createViaDriver = async (actor: Actor, icon: string) =>
runWithContext({ actor }, () =>
(server.drivers.apps as unknown as Driver).create({
object: {
name: `v13-${Math.random().toString(36).slice(2, 10)}`,
title: 'V',
index_url: `https://v13-${Math.random().toString(36).slice(2, 10)}.test/`,
icon,
},
}),
);
const launchApps = async (actor: Actor, appUid: string, userId: number) => {
await server.clients.db.write(
'INSERT INTO `app_opens` (`app_uid`, `user_id`, `ts`) VALUES (?, ?, ?)',
[appUid, userId, Math.floor(Date.now() / 1000)],
);
let body: { recent: Array<Record<string, unknown>> } | undefined;
const res = {
json: (v: unknown) => {
body = v as typeof body;
return res;
},
status: () => res,
setHeader: () => res,
} as unknown as Response;
await handler(
{
query: { icon_size: '64' },
headers: {},
body: {},
actor,
} as unknown as Request,
res,
() => {},
);
return body!.recent.find((e) => e.uuid === appUid)!;
};
describe('/get-launch-apps recent icon, verified', () => {
it('a valid data-URL icon written through the driver is migrated to a short URL', async () => {
const { actor, id } = await makeActor();
const app = await createViaDriver(
actor,
`data:image/png;base64,${PNG_2X2}`,
);
await vi.waitFor(
async () => {
const row = await server.stores.app.getByUid(app.uid as string);
expect(String(row?.icon).slice(0, 80)).toMatch(
/^https?:\/\/.*\/app-icon\/app-/,
);
},
{ timeout: 10_000 },
);
const entry = await launchApps(actor, app.uid as string, id);
expect(String(entry.icon).length).toBeLessThan(200);
expect(entry.iconCdnUrl).toMatch(/-64\.png$/);
}, 30_000);
it('returns an icon endpoint when migration cannot decode the stored icon', async () => {
const { actor, id } = await makeActor();
// PNG signature passes the write-path sniff; sharp rejects the rest.
const corrupt = Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
Buffer.alloc(1024 * 1024, 0x41),
]).toString('base64');
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
let app: Record<string, unknown>;
try {
app = await createViaDriver(
actor,
`data:image/png;base64,${corrupt}`,
);
// Give the background pipeline time to try and fail.
await vi.waitFor(
() =>
expect(
warn.mock.calls.some(
([m]) => m === '[app-icon] icon processing failed',
),
).toBe(true),
{ timeout: 10_000 },
);
} finally {
warn.mockRestore();
}
const row = await server.stores.app.getByUid(app!.uid as string);
expect(String(row?.icon).startsWith('data:image/png')).toBe(true);
const entry = await launchApps(actor, app!.uid as string, id);
expect(entry.icon).toBe(
`http://api.test.local/app-icon/${app!.uid}/64`,
);
}, 30_000);
});
@@ -5293,4 +5293,39 @@ describe('LegacyFSController GET /get-launch-apps', () => {
expect(returned).toEqual(recentUids);
});
it('skips the recent opens for recent=0 and keeps recommended', async () => {
const { actor, userId } = await makeUser();
const app = await makeHostedApp(userId, 'unregistered-skip');
await recordOpen(userId, app.uid);
const call = async (query: Record<string, unknown>) => {
const { res, captured } = makeRes();
await withActor(actor, async () => {
await getLaunchAppsHandler()(
makeReq({ actor, query }),
res,
() => {
throw new Error('handler called next() unexpectedly');
},
);
});
return captured.body as {
recommended: unknown[];
recent: unknown[];
};
};
const full = await call({});
expect(full.recent.length).toBeGreaterThan(0);
const opens = vi.spyOn(server.stores.app, 'getRecentAppOpens');
try {
const skipped = await call({ recent: '0' });
expect(skipped.recent).toEqual([]);
expect(skipped.recommended).toEqual(full.recommended);
expect(opens).not.toHaveBeenCalled();
} finally {
opens.mockRestore();
}
});
});
@@ -26,6 +26,7 @@ import {
assertResolvedActor,
isAccessTokenActor,
isAppActor,
isPlainUserActor,
makeActor,
} from '../../core/actor.js';
import { Context } from '../../core/context.js';
@@ -95,6 +96,7 @@ import {
assertAccess,
assertCanCreate,
assertCanMoveInto,
contextBoundStream,
emitGuiItemEvent,
expandClientPath,
findOverwriteTarget,
@@ -102,6 +104,7 @@ import {
getString,
type GuiItemEventName,
loadLegacyAssociatedApps,
rangeForEntry,
replacedByMove,
resolveReadRange,
resolveV1Selector,
@@ -381,7 +384,9 @@ export class LegacyFSController extends PuterController {
let recent: unknown[] = [];
const userId = req.actor?.user?.id;
if (userId) {
// `recent=0`: a caller that only shows `recommended`.
const wantsRecent = req.query.recent !== '0';
if (userId && wantsRecent) {
const recentUids = await this.stores.app.getRecentAppOpens(
userId,
{ limit: 10 },
@@ -418,9 +423,9 @@ export class LegacyFSController extends PuterController {
return {
...toAppSummary(app, {
config: this.config,
apiBaseUrl: this.config.api_base_url,
iconSize,
}),
icon: app.icon ?? null,
index_url: backingGone ? null : app.index_url,
...(backingGone
? { privateAccess: buildHostedBackingDenial() }
@@ -801,10 +806,23 @@ export class LegacyFSController extends PuterController {
this.stores.fsEntry,
body.source,
);
const destinationParent = await resolveV1Selector(
const destination = await resolveV1Selector(
this.stores.fsEntry,
body.destination,
);
// A file destination names the copy itself, replaced only with
// `overwrite`.
let newName = getString(body, 'new_name');
let destinationParent = destination;
if (!destination.isDir && !newName && destination.parentUid) {
const parent = await this.stores.fsEntry.getEntryByUuid(
destination.parentUid,
);
if (parent) {
destinationParent = parent;
newName = destination.name;
}
}
await assertAccess(
this.services.acl,
@@ -829,14 +847,14 @@ export class LegacyFSController extends PuterController {
? await findOverwriteTarget(
this.stores.fsEntry,
destinationParent,
getString(body, 'new_name') ?? source.name,
newName ?? source.name,
)
: null;
const copy = await this.services.fs.copy(userId, {
source,
destinationParent,
newName: getString(body, 'new_name'),
newName,
overwrite: overwriteRequested,
dedupeName: getBoolean(body, 'dedupe_name', 'change_name') ?? false,
});
@@ -1289,7 +1307,10 @@ export class LegacyFSController extends PuterController {
});
}
const range = resolveReadRange(req.headers.range, query);
const range = rangeForEntry(
entry,
resolveReadRange(req.headers.range, query),
);
const download = await this.services.fs.readContent(entry, {
range,
});
@@ -1790,19 +1811,50 @@ export class LegacyFSController extends PuterController {
// owner, so the layout it reveals is theirs to reveal, and the child
// signatures expire; the paths are the only part that outlives them.
if (entry.isDir) {
const children = await this.services.fs.listDirectory(entry.uuid);
const signedChildren = children.map((child) => {
const signChild = (child: FSEntry) => {
const { write_url: _, ...rest } = signEntry(child, signingCfg, {
ttlSeconds: NON_OWNER_SIGNATURE_TTL_SECONDS,
});
return { ...rest, path: maskEntryPath(child) };
});
res.json(signedChildren);
};
const pages = this.services.fs.listDirectoryPages(entry.uuid);
const first = await pages.next();
const firstPage = first.done ? [] : first.value;
const second = await pages.next();
if (second.done) {
res.json(firstPage.map(signChild));
return;
}
// Past one page the listing streams, so a large folder isn't held
// in memory whole. A failure partway cuts the response off.
const secondPage = second.value;
const serialize = (child: FSEntry) =>
JSON.stringify(signChild(child));
async function* allPages() {
yield firstPage;
yield secondPage;
yield* pages;
}
async function* body() {
let separator = '[';
for await (const page of allPages()) {
if (page.length === 0) continue;
yield separator + page.map(serialize).join(',');
separator = ',';
}
yield separator === '[' ? '[]' : ']';
}
res.type('application/json');
await pipeToResponse(contextBoundStream(body()), res);
return;
}
// File: stream bytes with Range support.
const range = resolveReadRange(req.headers.range, query);
const range = rangeForEntry(
entry,
resolveReadRange(req.headers.range, query),
);
const download = await this.services.fs.readContent(entry, {
range,
});
@@ -1992,6 +2044,11 @@ export class LegacyFSController extends PuterController {
*/
checkAppAcl = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
// Sessions only: for an app, this would be a window onto its neighbours' grants.
if (!isPlainUserActor(actor))
throw new HttpError(403, 'actor must be a user', {
legacyCode: 'forbidden',
});
const body = asRecord(req.body);
const subjectRef = body.subject;
@@ -2022,7 +2079,7 @@ export class LegacyFSController extends PuterController {
legacyCode: 'subject_does_not_exist',
});
let app: { uid: string } | null = null;
let app: { uid: string; id: number } | null = null;
if (typeof appRef === 'string') {
app =
(await this.stores.app.getByUid(appRef)) ??
@@ -2036,7 +2093,7 @@ export class LegacyFSController extends PuterController {
// Build an actor-under-user shape for the check.
const actorForApp = makeActor({
user: actor.user,
app: { uid: (app as { uid: string }).uid },
app: { uid: app.uid, id: app.id },
});
const allowed = await acl.check(actorForApp, descriptor, mode);
res.json({ allowed });
@@ -2084,10 +2141,12 @@ export class LegacyFSController extends PuterController {
// by default, so opt in by naming the downloader as who pays.
req.egressActor = actor;
const range =
const range = rangeForEntry(
entry,
typeof req.headers.range === 'string'
? req.headers.range
: undefined;
: undefined,
);
const download = await this.services.fs.readContent(entry, {
range,
});
@@ -17,15 +17,19 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Writable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { Context, runWithContext } from '../../core/context.js';
import type { PuterServer } from '../../server.js';
import { createTestUser, setupTestServer } from '../../testUtil.js';
import {
asRecord,
contextBoundStream,
fsEntryMimeType,
getBoolean,
getString,
loadLegacyAssociatedApps,
rangeForEntry,
resolveReadRange,
signEntry,
signEntryThumbnail,
@@ -705,3 +709,50 @@ describe('resolveReadRange', () => {
).toThrow('Invalid byte_count');
});
});
describe('rangeForEntry', () => {
it('drops the range for an empty file only', () => {
const entry = (size: number) => ({ size }) as FSEntry;
expect(rangeForEntry(entry(0), 'bytes=0-511')).toBeUndefined();
expect(rangeForEntry(entry(10), 'bytes=0-4')).toBe('bytes=0-4');
});
});
describe('contextBoundStream', () => {
it("reads every chunk in the caller's context", async () => {
// A sink slower than the source that completes writes from outside
// any request, so later chunks are asked for from its drain.
const pending: Array<() => void> = [];
const flush = runWithContext({}, () =>
setInterval(() => pending.splice(0).forEach((done) => done()), 5),
);
const sink = new Writable({
highWaterMark: 1,
write(_chunk, _encoding, done) {
pending.push(done);
},
});
const seen: unknown[] = [];
try {
await runWithContext({ actor: 'caller' as never }, async () => {
async function* chunks() {
for (let i = 0; i < 40; i++) {
await new Promise((resolve) => setTimeout(resolve, 1));
seen.push(Context.get('actor'));
yield 'x';
}
}
await new Promise((resolve, reject) => {
contextBoundStream(chunks())
.pipe(sink)
.on('finish', resolve)
.on('error', reject);
});
});
} finally {
clearInterval(flush);
}
expect(seen).toHaveLength(40);
expect(seen.every((actor) => actor === 'caller')).toBe(true);
});
});
@@ -17,7 +17,9 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { AsyncResource } from 'node:async_hooks';
import { posix as pathPosix } from 'node:path';
import { Readable } from 'node:stream';
import { contentType as contentTypeFromMime } from 'mime-types';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import type { FSEntryStore } from '../../stores/fs/FSEntryStore.js';
@@ -150,6 +152,29 @@ export function resolveReadRange(
return undefined;
}
/**
* A response body read from `chunks` for `pipeToResponse`. Each chunk is pulled
* in the caller's async context, which share masking reads, not in whichever
* socket event asked for more.
*/
export function contextBoundStream(
chunks: AsyncGenerator<string, void, undefined>,
): Readable {
const next = AsyncResource.bind(() => chunks.next());
const stop = AsyncResource.bind(() => chunks.return(undefined));
return Readable.from({
[Symbol.asyncIterator]: () => ({ next, return: stop }),
});
}
/** No range is satisfiable on an empty file, so it reads whole: empty, 200. */
export function rangeForEntry(
entry: FSEntry,
range: string | undefined,
): string | undefined {
return entry.size === 0 ? undefined : range;
}
// Accepts either `{ path }` or `{ uid }` or `{ id }` or `{ parent, name }`
// from a legacy body field. Returns a resolved entry (throwing 404 if not
// found or 400 if no usable ref is present).
@@ -17,6 +17,9 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { mkdir, mkdtemp, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import path from 'node:path';
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
@@ -466,3 +469,63 @@ describe('HomepageController GET /show/*splat', () => {
expect(html).toContain('/alice/Documents');
});
});
// ── /sw.js ──────────────────────────────────────────────────────────
describe('HomepageController GET /sw.js', () => {
const headersOf = (res: Response): Record<string, string> =>
Object.fromEntries(
(res.set as ReturnType<typeof vi.fn>).mock.calls.map(
([key, value]) => [String(key).toLowerCase(), String(value)],
),
);
it('serves the worker with root scope, as JavaScript, revalidated each time', async () => {
const root = await mkdtemp(path.join(tmpdir(), 'puter-sw-route-'));
await mkdir(path.join(root, 'dist'));
await writeFile(path.join(root, 'dist', 'sw.js'), '// worker\n');
const config = server.services.homepage.config as {
gui_assets_root?: string;
env?: string;
};
const prev = { root: config.gui_assets_root, env: config.env };
config.gui_assets_root = root;
config.env = 'prod';
const { res, captured } = makeRes();
try {
await callRoute('get', '/sw.js', makeReq({ path: '/sw.js' }), res);
} finally {
config.gui_assets_root = prev.root;
config.env = prev.env;
}
expect(captured.statusCode).toBe(200);
expect(captured.body).toBe('// worker\n');
expect(headersOf(res)).toMatchObject({
'content-type': 'text/javascript; charset=utf-8',
'cache-control': 'no-cache',
'service-worker-allowed': '/',
});
});
it('is a 404 when there is no worker to serve', async () => {
const config = server.services.homepage.config as {
gui_assets_root?: string;
};
const prev = config.gui_assets_root;
config.gui_assets_root = path.join(tmpdir(), 'no-such-gui-root');
const { res, captured } = makeRes();
try {
await callRoute('get', '/sw.js', makeReq({ path: '/sw.js' }), res);
} finally {
config.gui_assets_root = prev;
}
expect(captured.statusCode).toBe(404);
});
it('is registered for the root subdomain only', () => {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === '/sw.js',
);
expect(route?.options.subdomain).toBe('');
});
});
@@ -22,6 +22,7 @@ import path from 'node:path';
import { PuterController } from '../types.js';
import { toAppShellView } from '../../util/appShellView.js';
import type { PuterRouter } from '../../core/http/PuterRouter';
import type { IConfig } from '../../types';
import type {
PuterHomepageService,
PageMeta,
@@ -43,9 +44,9 @@ const APP_LANDING_LIMIT = {
};
/**
* App descriptions can run long; search snippets and social cards want a
* short blurb. Collapses whitespace, then cuts at a word boundary with an
* ellipsis so the result never exceeds `APP_META_DESCRIPTION_MAX`.
* App descriptions can run long; search snippets and social cards want a short
* blurb. Collapses whitespace, then cuts at a word boundary with an ellipsis so
* the result never exceeds `APP_META_DESCRIPTION_MAX`.
*/
export function appMetaDescription(text: string): string {
const clean = text.replace(/\s+/g, ' ').trim();
@@ -56,6 +57,47 @@ export function appMetaDescription(text: string): string {
return `${cut.replace(/[\s,;:.!?…—-]+$/, '')}…`;
}
/**
* `frame-ancestors` for the app browser embed. A wildcard over `*.<domain>`
* would also admit user-hosted sites whenever the hosting domains sit under
* it, so the embedding page is named exactly, from the request's referrer.
*/
export function embedFrameAncestors(
config: Pick<
IConfig,
| 'origin'
| 'domain'
| 'static_hosting_domain'
| 'static_hosting_domain_alt'
| 'private_app_hosting_domain'
| 'private_app_hosting_domain_alt'
>,
referer: string | undefined,
): string {
const gui = new URL(config.origin ?? `https://${config.domain}`);
let parent: URL;
try {
parent = new URL(referer ?? '');
} catch {
return "'self'";
}
const host = parent.hostname.toLowerCase();
const hostingDomains = [
config.static_hosting_domain,
config.static_hosting_domain_alt,
config.private_app_hosting_domain,
config.private_app_hosting_domain_alt,
]
.map((d) => (d ?? '').toLowerCase().split(':')[0])
.filter(Boolean);
const allowed =
parent.protocol === gui.protocol &&
parent.port === gui.port &&
host.endsWith(`.${config.domain.toLowerCase()}`) &&
!hostingDomains.some((d) => host === d || host.endsWith(`.${d}`));
return allowed ? `'self' ${parent.origin}` : "'self'";
}
/**
* Routes that render the Puter GUI shell, plus a catch-all static fallback
* under `<gui_assets_root>/src` for non-dist/src paths (images, fonts, lib
@@ -105,6 +147,21 @@ export class HomepageController extends PuterController {
router.get('/', {}, (req, res) => sendShell(req, res));
router.get('/embed/apps', {}, (req, res) => {
// SDK bootstrap parameters must not change this page's account or API origin.
if (Object.keys(req.query).length) {
res.redirect('/embed/apps');
return;
}
res.removeHeader('X-Frame-Options');
res.setHeader(
'Content-Security-Policy',
`frame-ancestors ${embedFrameAncestors(this.config, req.get('referer'))}`,
);
res.setHeader('Cache-Control', 'no-store');
res.send(homepage.renderAppBrowserEmbed());
});
router.get('/settings', {}, (req, res) => sendShell(req, res));
router.get('/settings/*splat', {}, (req, res) => sendShell(req, res));
@@ -198,6 +255,20 @@ export class HomepageController extends PuterController {
return sendShell(req, res, {}, launch);
});
// -- /sw.js - the service worker, scoped to the whole origin --
router.get('/sw.js', { subdomain: '' }, async (_req, res) => {
const source = await homepage.serviceWorkerSource();
if (source === null) {
res.status(404).send('Not found');
return;
}
res.set('Content-Type', 'text/javascript; charset=utf-8');
res.set('Cache-Control', 'no-cache');
res.set('Service-Worker-Allowed', '/');
res.send(source);
});
// -- Fallback static mount -----------------------------------
// Serves lingering GUI assets (images, fonts, lib files, etc.)
// out of <gui_assets_root>/src. Falls through to the 404 handler
@@ -198,7 +198,7 @@ describe('HostingController /delete-site', () => {
);
expect(emitSpy).toHaveBeenCalledWith(
'subdomain.delete',
{ subdomain: site.subdomain, uid: String(site.uuid) },
{ subdomain: site.subdomain, uid: String(site.uuid), userId },
{},
);
} finally {
+2
View File
@@ -27,6 +27,7 @@ import { EventsController } from './events/EventsController.js';
import { FSController } from './fs/FSController.js';
import { HomepageController } from './homepage/HomepageController.js';
import { HostingController } from './hosting/HostingController.js';
import { KvShareController } from './kvShare/KvShareController.js';
import { LegacyFSController } from './fs/LegacyFSController.js';
import { NotificationController } from './notification/NotificationController.js';
import { OIDCController } from './oidc/OIDCController.js';
@@ -58,6 +59,7 @@ export const puterControllers = {
broadcast: BroadcastController,
notification: NotificationController,
events: EventsController,
kvShare: KvShareController,
share: ShareController,
webdav: WebDAVController,
oidc: OIDCController,
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,272 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import type { Actor } from '../../core/actor.js';
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { bodyRecord } from '../../core/http/requestBody.js';
import {
assertShareableAppUid,
parseKvShareRecipient,
} from '../../services/kvShare/kvShares.js';
import type {
KvShareHandleListOptions,
KvShareOwnerListOptions,
KvShareOwnerListState,
} from '../../stores/events/KvShareHandleStore.js';
import {
KV_HANDLE_LIST_LIMIT_CAP,
KV_SHARE_OWNER_LIST_STATES,
} from '../../stores/events/KvShareHandleStore.js';
import { KV_GLOBAL_APP_KEY } from '../../stores/systemKv/SystemKVStore.js';
import { normalizeLimit } from '../../util/pagination.js';
import type { KvShareView } from '../../services/kvShare/KvShareService.js';
import { EVENTS_LIST_LIMIT, KV_SHARE_BATCH_MAX } from '../events/limits.js';
import { PuterController } from '../types.js';
/** The app-less namespace is no app, so the wire says `null` for it. */
const publicAppUid = (appUid: string): string | null =>
appUid === KV_GLOBAL_APP_KEY ? null : appUid;
/**
* The namespace a listing's `appUid` names, in the form responses use: an app
* uid, or `null` for the app-less namespace.
*/
const listAppUid = (value: unknown): string | undefined => {
if (value === undefined || value === '') return undefined;
if (typeof value !== 'string')
throw new HttpError(400, '`appUid` must be a string', {
legacyCode: 'bad_request',
});
return value === 'null' ? KV_GLOBAL_APP_KEY : assertShareableAppUid(value);
};
const badRequest = (message: string): HttpError =>
new HttpError(400, message, { legacyCode: 'bad_request' });
/** A share as `POST /kv/shares` reports it. */
const shareBody = (share: KvShareView) => ({
handle: share.handle,
prefix: share.prefix,
access: share.access,
appUid: publicAppUid(share.appUid),
holder: share.holderUsername,
label: share.label,
notify: share.notify,
createdAt: share.createdAt,
});
/** `recipients` when the body names it: 1 to `KV_SHARE_BATCH_MAX` entries. */
const batchRecipients = (
body: Record<string, unknown>,
): unknown[] | undefined => {
const { recipient, recipients } = body;
if (recipients === undefined) return undefined;
if (recipient !== undefined)
throw badRequest('Send either `recipient` or `recipients`, not both');
if (
!Array.isArray(recipients) ||
recipients.length === 0 ||
recipients.length > KV_SHARE_BATCH_MAX
)
throw badRequest(
`\`recipients\` must be an array of 1 to ${KV_SHARE_BATCH_MAX} recipients`,
);
return recipients;
};
const listState = (value: unknown): KvShareOwnerListState | undefined => {
if (value === undefined || value === '') return undefined;
if (!(KV_SHARE_OWNER_LIST_STATES as readonly unknown[]).includes(value))
throw new HttpError(400, '`state` must be `live`, `revoked` or `all`', {
legacyCode: 'bad_request',
});
return value as KvShareOwnerListState;
};
const pageOptions = (req: Request): KvShareHandleListOptions => {
const query = (req.query ?? {}) as Record<string, unknown>;
return {
limit: normalizeLimit(query.limit, { cap: KV_HANDLE_LIST_LIMIT_CAP }),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
appUid: listAppUid(query.appUid),
};
};
const ownerPageOptions = (req: Request): KvShareOwnerListOptions => ({
...pageOptions(req),
state: listState((req.query ?? {}).state),
});
/**
* Sharing a region of one account's key-value data with another account.
*
* Gated like the events surface: `subdomain: 'api'` plus `allowAccessToken`,
* with app-ness decided from `effectiveApp` in the service, which is where the
* owner, holder, delegation and namespace rules live. Only sharing asks for a
* confirmed email; listing, revoking and unsharing only narrow or report.
*/
@Controller('/kv')
export class KvShareController extends PuterController {
/**
* POST /kv/shares — share a region with another account, `read` unless the
* body asks otherwise. Sharing the same region with the same account again
* changes its level, label or `notify` flag in place. `recipients` shares
* it with several at once and reports each one on its own item.
*/
@Post('/shares', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
requireVerified: true,
})
async share(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = bodyRecord(req);
const fields = {
prefix: body.prefix,
access: body.access,
appUid: body.appUid,
label: body.label,
notify: body.notify,
};
const recipients = batchRecipients(body);
if (recipients) {
const items = await this.services.kvShare.mintKvHandles(
actor,
{ ...fields, recipients },
{ defaultAccess: 'read' },
);
res.json({
items: items.map((item) =>
item.share
? {
recipient: item.recipient,
...shareBody(item.share),
}
: {
recipient: item.recipient,
error: {
message: item.error?.message ?? 'Not shared',
code:
item.error?.legacyCode ??
item.error?.code ??
'internal_error',
},
},
),
});
return;
}
const share = await this.services.kvShare.mintKvHandle(
actor,
{ ...fields, recipient: parseKvShareRecipient(body.recipient) },
{ defaultAccess: 'read' },
);
res.json(shareBody(share));
}
/**
* GET /kv/shares — what this account has shared out, retired included
* unless `state` narrows it.
*/
@Get('/shares', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async listSharedByMe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const page = await this.services.kvShare.listKvHandles(
actor,
ownerPageOptions(req),
);
res.json({
items: page.items.map((row) => ({
handle: row.handle,
prefix: row.prefix,
access: row.access,
appUid: publicAppUid(row.appUid),
holder: row.holderUsername,
label: row.label,
notify: row.notify,
createdAt: row.createdAt,
revokedAt: row.revokedAt,
revokedBy: row.revokedBy,
})),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
/** GET /kv/shares/with-me — live shares this account holds. */
@Get('/shares/with-me', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
rateLimit: EVENTS_LIST_LIMIT,
})
async listSharedWithMe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const page = await this.services.kvShare.listKvSharesWithMe(
actor,
pageOptions(req),
);
res.json({
items: page.items.map((row) => ({
handle: row.handle,
access: row.access,
appUid: publicAppUid(row.appUid),
label: row.label,
notify: row.notify,
createdAt: row.createdAt,
})),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
/**
* DELETE /kv/shares/:handle — the owner revokes, the holder gives it up,
* and anyone else is told it is not there.
*/
@Delete('/shares/:handle', {
subdomain: 'api',
requireAuth: true,
allowAccessToken: true,
})
async unshare(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
res.json(
await this.services.kvShare.removeKvShare(actor, req.params.handle),
);
}
#requireActor(req: Request): Actor {
const actor = req.actor;
if (!actor?.user)
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
return actor;
}
}
@@ -21,9 +21,9 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
/**
* Route-level coverage for the notification endpoint. Dismissing is the only
* mark the desktop performs over HTTP, so this suite is what catches the route
* going missing or a gate turning a legitimate dismiss away.
* Route-level coverage for the notification endpoints the desktop calls over
* HTTP, so this suite is what catches a route going missing or a gate turning
* a legitimate dismiss or removal away.
*/
describe('notification endpoints over HTTP', () => {
let env: PuterTestEnv;
@@ -85,4 +85,34 @@ describe('notification endpoints over HTTP', () => {
});
expect(after?.acknowledged).toBeFalsy();
});
it('removes a notification', async () => {
const owner = env.users.user;
const { userId, uid } = await seedNotification(owner.username);
const res = await post('/notif/delete', owner.token, { uid });
expect(res.status).toBe(200);
await expect(res.json()).resolves.toEqual({});
const after = await env.server.stores.notification.getByUid(uid, {
userId,
});
expect(after).toBeNull();
});
it('refuses an anonymous removal', async () => {
const { userId, uid } = await seedNotification(env.users.other.username);
const res = await fetch(new URL('/notif/delete', env.apiOrigin), {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ uid }),
});
expect(res.status).toBeGreaterThanOrEqual(400);
const after = await env.server.stores.notification.getByUid(uid, {
userId,
});
expect(after?.uid).toBe(uid);
});
});
@@ -183,3 +183,74 @@ describe('NotificationController.markAck', () => {
expect(after?.acknowledged).toBeFalsy();
});
});
// ── /notif/delete ───────────────────────────────────────────────────
describe('NotificationController.remove', () => {
it('deletes the underlying notification row', async () => {
const { actor, userId } = await makeUser();
const created = await server.stores.notification.create({
userId,
value: { title: 't' },
});
const { res, captured } = makeRes();
await controller.remove(
makeReq({ body: { uid: created.uid }, actor }),
res,
);
expect(captured.body).toEqual({});
const after = await server.stores.notification.getByUid(
created.uid as string,
{ userId },
);
expect(after).toBeNull();
});
it('succeeds for a uid that names no row', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
await controller.remove(
makeReq({ body: { uid: 'no-such-uid' }, actor }),
res,
);
expect(captured.body).toEqual({});
});
it('rejects a missing uid with 400', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
controller.remove(makeReq({ body: {}, actor }), res),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 401 when there is no actor on the request', async () => {
const { res } = makeRes();
await expect(
controller.remove(makeReq({ body: { uid: 'whatever' } }), res),
).rejects.toMatchObject({ statusCode: 401 });
});
it("does not delete another user's notification", async () => {
const a = await makeUser();
const b = await makeUser();
const created = await server.stores.notification.create({
userId: a.userId,
value: {},
});
const { res } = makeRes();
await controller.remove(
makeReq({ body: { uid: created.uid }, actor: b.actor }),
res,
);
const after = await server.stores.notification.getByUid(
created.uid as string,
{ userId: a.userId },
);
expect(after?.uid).toBe(created.uid);
});
});
@@ -20,16 +20,44 @@
import type { Request, Response } from 'express';
import { Controller, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { NotificationService } from '../../services/notification/NotificationService.js';
import { PuterController } from '../types.js';
/** Both routes fire per interaction; the ceiling catches a loop, not a user. */
const MARK_ROUTE_OPTIONS = {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: {
scope: 'notification-mark',
limit: 300,
window: 60_000,
key: 'user',
},
} as const;
/** The uid and owner a mark names, or the 400/401 the request earns. */
const markTarget = (req: Request): { uid: string; userId: number } => {
const uid = req.body?.uid;
if (typeof uid !== 'string' || uid.length === 0) {
throw new HttpError(400, '`uid` must be a non-empty string', {
legacyCode: 'bad_request',
});
}
const userId = req.actor?.user?.id;
if (!userId)
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
return { uid, userId };
};
/**
* GUI-facing notification endpoint. Supplements the `puter-notifications`
* driver (which handles CRUD via `/drivers/call`) with the one mutation route
* the puter desktop client calls directly.
* GUI-facing notification endpoints. Supplement the `puter-notifications`
* driver (which handles reads via `/drivers/call`) with the mutations the puter
* desktop client calls directly.
*
* It emits `outer.gui.notif.ack` via the NotificationService so other open tabs
* for the same user see the state change immediately.
* Each goes through the NotificationService, which tells the user's other open
* tabs (`notif.ack`, `notif.removed`) so they update immediately.
*/
@Controller('/notif')
export class NotificationController extends PuterController {
@@ -37,50 +65,21 @@ export class NotificationController extends PuterController {
* POST /notif/mark-ack — user dismissed a notification. Sets `acknowledged`
* timestamp; pushes ack event to sockets.
*/
@Post('/mark-ack', {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
// Fires per notification interaction, so the ceiling stays
// generous — it is here to catch a loop, not to pace a user.
rateLimit: {
scope: 'notification-mark',
limit: 300,
window: 60_000,
key: 'user',
},
})
@Post('/mark-ack', MARK_ROUTE_OPTIONS)
async markAck(req: Request, res: Response): Promise<void> {
const uid = req.body?.uid;
if (typeof uid !== 'string' || uid.length === 0) {
throw new HttpError(400, '`uid` must be a non-empty string', {
legacyCode: 'bad_request',
});
}
const userId = req.actor?.user?.id;
if (!userId)
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
const { uid, userId } = markTarget(req);
await this.services.notification.markAcknowledged(uid, userId);
res.json({});
}
const notifService = this.services.notification as unknown as
| NotificationService
| undefined;
if (notifService?.markAcknowledged) {
await notifService.markAcknowledged(uid, userId);
} else {
// Fallback: direct store call if service isn't wired
await (
this.stores as Record<string, unknown> as {
notification: {
markAcknowledged: (
uid: string,
userId: number,
) => Promise<boolean>;
};
}
).notification.markAcknowledged(uid, userId);
}
/**
* POST /notif/delete — user removed a notification from their history. A
* uid naming none of their rows succeeds too: it is gone either way.
*/
@Post('/delete', MARK_ROUTE_OPTIONS)
async remove(req: Request, res: Response): Promise<void> {
const { uid, userId } = markTarget(req);
await this.services.notification.remove(uid, userId);
res.json({});
}
}
@@ -244,7 +244,10 @@ describe('OIDCController GET /auth/oidc/providers', () => {
await callRoute('get', '/auth/oidc/providers', makeReq({}), res);
// We configured a single `custom` provider; the live service
// walks the config and returns its id.
expect(captured.body).toEqual({ providers: ['custom'] });
expect(captured.body).toEqual({
providers: ['custom'],
signupOidcOnly: false,
});
});
});
@@ -396,9 +396,12 @@ export class OIDCController extends PuterController {
},
},
async (_req: Request, res: Response) => {
const providers =
await this.services.oidc.getEnabledProviderIds();
res.json({ providers });
const [providers, policy] = await Promise.all([
this.services.oidc.getEnabledProviderIds(),
this.services.signup.signupPolicy(),
]);
// The signup window drops its email form when this is set.
res.json({ providers, signupOidcOnly: policy.oidcOnly });
},
);
@@ -201,6 +201,7 @@ describe('PuterAIController.openaiResponses parameter forwarding', () => {
context_management: [{ type: 'compaction' }],
prompt_cache_key: 'ck',
prompt_cache_retention: '24h',
prompt_cache_options: { ttl: '12h', prewarm: true },
store: true,
truncation: 'auto',
service_tier: 'default',
@@ -226,6 +227,9 @@ describe('PuterAIController.openaiResponses parameter forwarding', () => {
context_management: [{ type: 'compaction' }],
prompt_cache_key: 'ck',
prompt_cache_retention: '24h',
// Forwarded as sent; the provider keeps only the fields it
// accepts.
prompt_cache_options: { ttl: '12h', prewarm: true },
// forced false regardless of the client's request.
store: false,
truncation: 'auto',
@@ -359,13 +363,18 @@ describe('PuterAIController.openaiResponses input normalization', () => {
role: 'user',
content: [
'a bare string part',
{ type: 'input_text', text: 'typed text' },
{
type: 'input_text',
text: 'typed text',
prompt_cache_breakpoint: { mode: 'explicit', ttl: '12h' },
},
{ type: 'output_text', text: 'echoed text' },
{
type: 'input_image',
detail: 'high',
image_url: 'https://img.test/a.png',
file_id: 'file_img',
prompt_cache_breakpoint: { mode: 'explicit' },
},
{ type: 'input_audio', input_audio: { data: 'AAA' } },
{
@@ -386,13 +395,18 @@ describe('PuterAIController.openaiResponses input normalization', () => {
role: 'user',
content: [
{ type: 'text', text: 'a bare string part' },
{ type: 'text', text: 'typed text' },
{
type: 'text',
text: 'typed text',
prompt_cache_breakpoint: { mode: 'explicit', ttl: '12h' },
},
{ type: 'text', text: 'echoed text' },
{
type: 'image_url',
detail: 'high',
image_url: { url: 'https://img.test/a.png' },
file_id: 'file_img',
prompt_cache_breakpoint: { mode: 'explicit' },
},
{ type: 'input_audio', input_audio: { data: 'AAA' } },
{
@@ -639,6 +653,33 @@ describe('PuterAIController.openaiResponses output items', () => {
// -- /openai/v1/chat/completions edges -------------------------------
describe('PuterAIController.openaiChatCompletions translation edges', () => {
it('forwards the prompt-cache fields to the driver', async () => {
const completeSpy = stubChatComplete({
message: { role: 'assistant', content: 'ok' },
});
const { res } = makeRes();
await controller.openaiChatCompletions(
makeReq({
body: {
model: 'gpt-test',
messages: [{ role: 'user', content: 'hi' }],
prompt_cache_key: 'ck',
prompt_cache_retention: 'in_memory',
prompt_cache_options: { ttl: '30m' },
},
}),
res,
);
const args = completeSpy.mock.calls[0]![0] as Record<string, unknown>;
expect(args).toMatchObject({
prompt_cache_key: 'ck',
prompt_cache_retention: 'in_memory',
prompt_cache_options: { ttl: '30m' },
});
});
it('derives tool_calls from tool_use content blocks when the message has none', async () => {
stubChatComplete({
message: {
@@ -511,6 +511,31 @@ describe('PuterAIController.openaiChatCompletions', () => {
const completeArgs = completeSpy.mock.calls[0]![0];
expect(completeArgs.provider).toBe('openai-completion');
expect(completeArgs.fallback).toBeUndefined();
});
it('forwards fallback: false so the driver stops at the first route', async () => {
const completeSpy = stubChatComplete({
message: { role: 'assistant', content: 'hi there' },
finish_reason: 'stop',
usage: { prompt_tokens: 4, completion_tokens: 2 },
});
const { res } = makeRes();
await controller.openaiChatCompletions(
makeReq({
body: {
model: 'gpt-test',
provider: 'openai-completion',
fallback: false,
messages: [{ role: 'user', content: 'hi' }],
},
actor: makeUserActor(),
}),
res,
);
expect(completeSpy.mock.calls[0]![0].fallback).toBe(false);
});
it('still pins OpenAI when the caller names no model, so the default model does not move', async () => {
@@ -447,7 +447,9 @@ export class PuterAIController extends PuterController {
body.reasoning_effort as ICompleteArguments['reasoning_effort'],
}
: {}),
...promptCacheArgsFromBody(body),
...openaiCompatProvider(body),
...(body.fallback === false ? { fallback: false } : {}),
};
const result = await this.#complete(res, completeArgs);
@@ -545,8 +547,7 @@ export class PuterAIController extends PuterController {
}
} else if (ev.type === 'usage') {
usageDetails = ev.usageDetails as
| UsageDetails
| undefined;
UsageDetails | undefined;
finishReason = ev.finish_reason as string | undefined;
} else if (ev.type === 'error') {
res.write(
@@ -649,6 +650,7 @@ export class PuterAIController extends PuterController {
: {}),
...(finiteMaxTokens(body.max_tokens) ?? {}),
...openaiCompatProvider(body),
...(body.fallback === false ? { fallback: false } : {}),
};
const completionId = `cmpl-${randomId()}`;
@@ -694,8 +696,7 @@ export class PuterAIController extends PuterController {
sendChunk(ev.text);
} else if (ev.type === 'usage') {
usageDetails = ev.usageDetails as
| UsageDetails
| undefined;
UsageDetails | undefined;
finishReason = ev.finish_reason as string | undefined;
} else if (ev.type === 'error') {
res.write(
@@ -749,8 +750,7 @@ export class PuterAIController extends PuterController {
text: extractTextContent(
(
messageResult.message as
| Record<string, unknown>
| undefined
Record<string, unknown> | undefined
)?.content,
),
index: 0,
@@ -843,15 +843,7 @@ export class PuterAIController extends PuterController {
body.compaction as ICompleteArguments['compaction'],
}
: {}),
...(body.prompt_cache_key
? { prompt_cache_key: String(body.prompt_cache_key) }
: {}),
...(body.prompt_cache_retention
? {
prompt_cache_retention:
body.prompt_cache_retention as ICompleteArguments['prompt_cache_retention'],
}
: {}),
...promptCacheArgsFromBody(body),
// forced off regardless of what the caller sent — OpenAI
// defaults `store` to true, which would persist state server-side
// on OpenAI's end that this route gives no way to retrieve.
@@ -869,6 +861,7 @@ export class PuterAIController extends PuterController {
}
: {}),
provider: providerName,
...(body.fallback === false ? { fallback: false } : {}),
};
const responseId = generateId('resp');
@@ -1343,6 +1336,33 @@ const openaiCompatProvider = (
return { provider: DEFAULTS.openaiChat };
};
/**
* The prompt-cache fields both OpenAI-shaped routes forward as sent; the
* provider checks the values.
*/
const promptCacheArgsFromBody = (
body: Record<string, unknown>,
): Pick<
ICompleteArguments,
'prompt_cache_key' | 'prompt_cache_retention' | 'prompt_cache_options'
> => ({
...(body.prompt_cache_key
? { prompt_cache_key: String(body.prompt_cache_key) }
: {}),
...(body.prompt_cache_retention
? {
prompt_cache_retention:
body.prompt_cache_retention as ICompleteArguments['prompt_cache_retention'],
}
: {}),
...(body.prompt_cache_options
? {
prompt_cache_options:
body.prompt_cache_options as ICompleteArguments['prompt_cache_options'],
}
: {}),
});
const randomId = (): string => crypto.randomUUID().replace(/-/g, '');
const generateId = (prefix: string): string => `${prefix}_${randomId()}`;
@@ -1590,12 +1610,24 @@ const parseJsonMaybe = (value: unknown): unknown => {
}
};
/** An OpenAI `prompt_cache_breakpoint` marker, kept on the translated part. */
const cacheBreakpointOf = (
p: Record<string, unknown>,
): { prompt_cache_breakpoint?: unknown } =>
p.prompt_cache_breakpoint && typeof p.prompt_cache_breakpoint === 'object'
? { prompt_cache_breakpoint: p.prompt_cache_breakpoint }
: {};
const normalizeContentPart = (part: unknown): Record<string, unknown> => {
if (typeof part === 'string') return { type: 'text', text: part };
if (!part || typeof part !== 'object') return { type: 'text', text: '' };
const p = part as Record<string, unknown>;
if (p.type === 'input_text' || p.type === 'output_text') {
return { type: 'text', text: String(p.text ?? '') };
return {
type: 'text',
text: String(p.text ?? ''),
...cacheBreakpointOf(p),
};
}
if (p.type === 'input_image') {
return {
@@ -1603,6 +1635,7 @@ const normalizeContentPart = (part: unknown): Record<string, unknown> => {
...(p.detail ? { detail: p.detail } : {}),
...(p.image_url ? { image_url: { url: p.image_url } } : {}),
...(p.file_id ? { file_id: p.file_id } : {}),
...cacheBreakpointOf(p),
};
}
if (p.type === 'input_audio')
@@ -1614,6 +1647,7 @@ const normalizeContentPart = (part: unknown): Record<string, unknown> => {
...(p.file_id ? { file_id: p.file_id } : {}),
...(p.file_url ? { file_url: p.file_url } : {}),
...(p.filename ? { filename: p.filename } : {}),
...cacheBreakpointOf(p),
};
}
return p;
@@ -324,6 +324,18 @@ describe('parseAnthropicRequest', () => {
expect(args.provider).toBe('claude');
expect(args.normalize).toBe(false);
expect(args.streamToolInput).toBe(true);
expect(args.fallback).toBeUndefined();
});
it('forwards fallback: false and nothing else for fallback', () => {
expect(
parseAnthropicRequest({ messages: [], fallback: false }, headers)
.fallback,
).toBe(false);
expect(
parseAnthropicRequest({ messages: [], fallback: 'no' }, headers)
.fallback,
).toBeUndefined();
});
it('maps thinking, tool_choice, stop_sequences, top_p/top_k', () => {
@@ -431,6 +431,7 @@ export const parseAnthropicRequest = (
: {}),
...(betas.length ? { anthropicBetas: betas } : {}),
provider: toStringOrEmpty(body.provider) || 'claude',
...(body.fallback === false ? { fallback: false } : {}),
};
// Silently ignored: metadata, service_tier, inference_geo, container,
// fallbacks, diagnostics.
@@ -19,8 +19,9 @@
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import type { AddressInfo } from 'node:net';
import path from 'node:path';
import type { Request, Response } from 'express';
import express, { type Request, type Response } from 'express';
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import type { RouteDescriptor } from '../../core/http/types.js';
@@ -111,6 +112,7 @@ const callGetHandler = async (
let captured: { filename?: string; root?: string } = {};
const req = {} as Request;
const res = {
setHeader: vi.fn(),
sendFile: vi.fn((filename: string, opts: { root: string }) => {
captured = { filename, root: opts.root };
}),
@@ -151,6 +153,12 @@ describe('StaticAssetsController client_libs_root', () => {
const sentPutility = await callGetHandler(router, '/putility/v1');
expect(sentPutility).toEqual({ filename: 'putility.js/v1.js', root });
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
expect(findRoute(router, 'get', route)?.options.subdomain).toBe('js');
const sentWorker = await callGetHandler(router, route);
expect(sentWorker).toEqual({ filename: 'puter.js/push-sw.js', root });
}
});
it('does not register the client_libs routes when the root is unset', () => {
@@ -186,6 +194,11 @@ describe('StaticAssetsController puterjs_root', () => {
expect(v1.root).toBe(root);
const bareV1 = await callGetHandler(router, '/v1');
expect(bareV1.filename).toBe('puter.js');
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
const worker = await callGetHandler(router, route);
expect(worker).toEqual({ filename: 'push-sw.js', root });
}
});
it('serves puter.dev.js when present (dev webpack build)', async () => {
@@ -227,6 +240,48 @@ describe('StaticAssetsController puterjs_root', () => {
});
});
// ── push-sw.js caching ──────────────────────────────────────────────
/** Serves the route's handler over real HTTP and returns the response. */
const fetchThroughExpress = async (router: PuterRouter, routePath: string) => {
const route = findRoute(router, 'get', routePath);
if (!route) throw new Error(`No GET ${routePath} registered`);
const app = express();
app.get(routePath, (req, res, next) => route.handler(req, res, next));
const listener = app.listen(0);
await new Promise((resolve) => listener.once('listening', resolve));
try {
const { port } = listener.address() as AddressInfo;
return await fetch(`http://127.0.0.1:${port}${routePath}`);
} finally {
listener.close();
}
};
describe('StaticAssetsController push-sw.js', () => {
it.each([
['client_libs_root', 'puter.js/push-sw.js'],
['puterjs_root', 'push-sw.js'],
] as const)(
'lets the CDN keep it for five minutes from %s',
async (configKey, file) => {
const root = makeTempDir();
mkdirSync(path.dirname(path.join(root, file)), { recursive: true });
writeFileSync(path.join(root, file), '/* worker */');
const router = buildController({ [configKey]: root });
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
const response = await fetchThroughExpress(router, route);
expect(response.status).toBe(200);
expect(response.headers.get('cache-control')).toBe(
'public, max-age=0, s-maxage=300',
);
expect(await response.text()).toBe('/* worker */');
}
},
);
});
// ── gui_assets_root ─────────────────────────────────────────────────
describe('StaticAssetsController gui_assets_root', () => {
@@ -23,12 +23,15 @@ import path from 'node:path';
import { PuterController } from '../types.js';
import type { PuterRouter } from '../../core/http/PuterRouter';
// Sites import push-sw.js by URL, so the CDN refetches it within five minutes.
const PUSH_WORKER_CACHE_CONTROL = 'public, max-age=0, s-maxage=300';
/**
* Static asset routes.
*
* /puter.js/v1, /puter.js/v2 → any subdomain /v1, /v2, /putility/v1 → js
* subdomain /sdk/* → root subdomain — puter-js bundle /dist/_, /src/_,
* /assets/* → root subdomain
* /puter.js/v1, /puter.js/v2 → any subdomain /v1, /v2, /push-sw.js,
* /putility/v1 → js subdomain /sdk/* → root subdomain — puter-js bundle
* /dist/_, /src/_, /assets/* → root subdomain
*
* Each block depends on its config root (`client_libs_root`, `gui_assets_root`,
* `puterjs_root`). When unset, that block is skipped — deployments that don't
@@ -55,6 +58,13 @@ export class StaticAssetsController extends PuterController {
router.get('/putility/v1', { subdomain: 'js' }, (_req, res) => {
res.sendFile('putility.js/v1.js', { root });
});
// `/v2/push-sw.js` is the path older docs gave.
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
router.get(route, { subdomain: 'js' }, (_req, res) => {
res.setHeader('Cache-Control', PUSH_WORKER_CACHE_CONTROL);
res.sendFile('puter.js/push-sw.js', { root });
});
}
}
// puter-js SDK mount. GUI loads it at `/sdk/puter.dev.js`; the
@@ -100,6 +110,15 @@ export class StaticAssetsController extends PuterController {
router.get('/v2', { subdomain: 'js' }, (_req, res) => {
res.sendFile(puterJsFile, { root: puterjsRoot });
});
for (const route of ['/push-sw.js', '/v2/push-sw.js']) {
router.get(route, { subdomain: 'js' }, (_req, res) => {
res.setHeader(
'Cache-Control',
PUSH_WORKER_CACHE_CONTROL,
);
res.sendFile('push-sw.js', { root: puterjsRoot });
});
}
}
}
@@ -0,0 +1,146 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { randomUUID } from 'node:crypto';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { runWithContext } from '../../core/context.js';
import {
setupPuterTestEnv,
TEST_OTHER_USER_CREDENTIALS,
TEST_USER_CREDENTIALS,
type PuterTestEnv,
} from '../../testUtil.js';
/** PROPFIND has no paging, so a folder must list whole however large. */
describe('PROPFIND on a folder larger than one query', () => {
let env: PuterTestEnv;
const total = 10_050;
let dir: string;
let dirUuid: string;
beforeAll(async () => {
env = await setupPuterTestEnv();
const { username } = env.users.user;
dir = `/${username}/Documents/big-${randomUUID().slice(0, 8)}`;
const owner = (await env.server.stores.user.getByUsername(username))!;
const parent = (await env.server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
))!;
dirUuid = randomUUID();
const now = Math.floor(Date.now() / 1000);
const insert = (rows: unknown[][]) =>
env.server.clients.db.write(
`INSERT INTO fsentries (uuid, user_id, parent_id, parent_uid, name, path, is_dir, modified, created, accessed, size) VALUES ${rows.map(() => '(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)').join(', ')}`,
rows.flat(),
);
await insert([
[
dirUuid,
owner.id,
parent.id,
parent.uuid,
dir.split('/').at(-1),
dir,
1,
now,
now,
now,
0,
],
]);
const folder = (await env.server.stores.fsEntry.getEntryByPath(dir))!;
for (let start = 0; start < total; start += 500) {
const rows: unknown[][] = [];
for (let i = start; i < Math.min(total, start + 500); i++) {
const name = `f${String(i).padStart(5, '0')}.txt`;
rows.push([
randomUUID(),
owner.id,
folder.id,
folder.uuid,
name,
`${dir}/${name}`,
0,
now,
now,
now,
0,
]);
}
await insert(rows);
}
const actor = { user: owner };
await runWithContext({ actor: actor as never }, () =>
env.server.services.share.share(actor as never, {
uid: dirUuid,
recipient: { username: env.users.other.username },
mode: 'read',
}),
);
}, 180_000);
afterAll(async () => {
await env?.shutdown();
});
const propfind = async (
path: string,
credentials: { username: string; password: string },
) => {
const port = new URL(env.apiOrigin).port;
const auth = Buffer.from(
`${credentials.username}:${credentials.password}`,
).toString('base64');
const response = await fetch(
`http://dav.puter.localhost:${port}${encodeURI(path)}`,
{
method: 'PROPFIND',
headers: { Authorization: `Basic ${auth}`, Depth: '1' },
},
);
return { status: response.status, body: await response.text() };
};
const hrefsOf = (xml: string) =>
[...xml.matchAll(/<D:href>([^<]*)<\/D:href>/g)].map((m) => m[1]!);
it('lists every child', async () => {
const { status, body } = await propfind(dir, TEST_USER_CREDENTIALS);
expect(status).toBe(207);
expect(body.endsWith('</D:multistatus>')).toBe(true);
const hrefs = hrefsOf(body);
expect(hrefs).toHaveLength(1 + total);
expect(new Set(hrefs).size).toBe(1 + total);
expect(body).toContain(`${dir}/f10049.txt`);
}, 60_000);
it("masks every page of a share recipient's listing", async () => {
const masked = `/${env.users.user.username}/${dirUuid}/${dir.split('/').at(-1)}`;
const { status, body } = await propfind(
masked,
TEST_OTHER_USER_CREDENTIALS,
);
expect(status).toBe(207);
const hrefs = hrefsOf(body);
expect(hrefs).toHaveLength(1 + total);
expect(hrefs.every((href) => href.startsWith(masked))).toBe(true);
expect(body).not.toContain('/Documents/');
}, 60_000);
});
@@ -24,6 +24,7 @@ import { posix as pathPosix } from 'node:path';
import { pipeline } from 'node:stream/promises';
import type { Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { pipeToResponse } from '../../core/http/pipeToResponse.js';
import {
assertNotSuspended,
assertTeam2fa,
@@ -58,6 +59,7 @@ import {
import { Context } from '../../core/context.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import {
contextBoundStream,
emitGuiItemEvent,
type GuiItemEventName,
} from '../fs/legacyFsHelpers.js';
@@ -600,19 +602,41 @@ export class WebDAVController extends PuterController {
propfindEntry(maskEntryPath(entry), entry, entry.isDir),
];
if (depth !== '0' && entry.isDir) {
const children = await this.services.fs.listDirectory(
entry.uuid,
{},
);
for (const child of children) {
responses.push(
propfindEntry(maskEntryPath(child), child, child.isDir),
);
}
if (depth === '0' || !entry.isDir) {
this.#sendMultistatus(res, responses);
return;
}
this.#sendMultistatus(res, responses);
const toXml = (children: FSEntry[]) =>
children
.map((child) =>
propfindEntry(maskEntryPath(child), child, child.isDir),
)
.join('\n');
const pages = this.services.fs.listDirectoryPages(entry.uuid);
const first = await pages.next();
if (!first.done && first.value.length > 0)
responses.push(toXml(first.value));
const second = await pages.next();
if (second.done) {
this.#sendMultistatus(res, responses);
return;
}
// Past one page the listing streams, so a large folder isn't held in
// memory whole. A failure partway cuts the response off.
const secondPage = second.value;
async function* body() {
yield `${MULTISTATUS_OPEN}${responses.join('\n')}`;
yield `\n${toXml(secondPage)}`;
for await (const page of pages)
if (page.length > 0) yield `\n${toXml(page)}`;
yield MULTISTATUS_CLOSE;
}
res.status(207).set({
'Content-Type': 'application/xml; charset=utf-8',
});
await pipeToResponse(contextBoundStream(body()), res);
}
/**
@@ -1289,8 +1313,12 @@ function escapeXml(text: string): string {
.replace(/'/g, '&#39;');
}
const MULTISTATUS_OPEN =
'<?xml version="1.0" encoding="utf-8"?>\n<D:multistatus xmlns:D="DAV:">\n';
const MULTISTATUS_CLOSE = '\n</D:multistatus>';
function wrapMultistatus(inner: string): string {
return `<?xml version="1.0" encoding="utf-8"?>\n<D:multistatus xmlns:D="DAV:">\n${inner}\n</D:multistatus>`;
return `${MULTISTATUS_OPEN}${inner}${MULTISTATUS_CLOSE}`;
}
/**
+26
View File
@@ -23,6 +23,7 @@ import {
makeActor,
SYSTEM_ACTOR,
userRelatedActor,
viaWorkerOf,
type Actor,
} from './actor';
@@ -111,3 +112,28 @@ describe('assertResolvedActor', () => {
);
});
});
describe('viaWorkerOf', () => {
const user = { uuid: 'u-1', id: 1 };
it('reads a worker session or an events handler as server-side', () => {
expect(
viaWorkerOf(
makeActor({ user, session: { uid: 's-1', kind: 'worker' } }),
),
).toBe(true);
expect(viaWorkerOf(makeActor({ user, handlerDepth: 1 }))).toBe(true);
});
it('reads a browser session, an app or no actor at all as not', () => {
expect(
viaWorkerOf(
makeActor({ user, session: { uid: 's-1', kind: 'web' } }),
),
).toBe(false);
expect(viaWorkerOf(makeActor({ user, app: { uid: 'app-1' } }))).toBe(
false,
);
expect(viaWorkerOf(undefined)).toBe(false);
});
});
+7
View File
@@ -239,3 +239,10 @@ export const userRelatedActor = (actor: Actor): Actor => {
/** How many events handler runs led to a write by this actor. */
export const handlerDepthOf = (actor: Actor | undefined | null): number =>
actor?.handlerDepth ?? 0;
/**
* Whether a write by this actor was made by server-side code running as the
* account: a worker session, or an events handler.
*/
export const viaWorkerOf = (actor: Actor | undefined | null): boolean =>
actor?.session?.kind === 'worker' || handlerDepthOf(actor) > 0;
Loaded 100 of 913 files, more files were not shown because too many files have changed in this diff. Show more