Compare commits

..
Author SHA1 Message Date
jelveh 1f75ee51e0 Test native app hosts with apps still served natively
editor, markus and pdf moved to standard host routing, but the host
redirect and embed framing tests still used editor as their native app,
so they failed. They now use about, and a test checks that editor,
markus and pdf are passed through even with a folder under the native
apps root.
2026-11-07 01:27:03 -07:00
Nariman Jelveh cb214f28cc Open Contact Us as a dashboard dialog in the dashboard (#4533)
The dashboard's Contact Us now uses the dashboard form dialog instead of a
UIWindow: a centered card on desktop and a bottom sheet on phones. Files can
be picked, dropped anywhere over the dialog, or pasted into the message, and
images show a preview. Cmd/Ctrl+Enter sends. The desktop keeps
UIWindowFeedback.
2026-11-07 00:40:45 -07:00
Nariman Jelveh 8ed439e897 Stop the signup screen redrawing when it opens (#4532)
The signup screen showed its email form, then looked up the identity
providers and folded the form away behind "Sign up with email" once they
arrived, a round trip later. Every screen made its own lookup, so moving
from login to signup redrew the card each time.

The provider lookup now happens once per auth flow, so moving between
login and signup draws the providers before the next paint. When signup
is the first screen, its email form stays hidden until the lookup
answers, or for 1.5 seconds if it is slow, and shows when no providers
are offered or the form was opened prefilled.
2026-11-06 23:58:44 -07:00
jelveh 672a704193 Shorten popup Puter branding message
Updates the English translation for `popup_opener_uses_puter` to a shorter single-line message. This removes the extra explanatory text about AI/cloud features and using a Puter account, likely for a cleaner popup prompt.
2026-11-06 23:41:00 -07:00
jelveh 35ae464d86 Tighten signup terms typography
Reduce the signup terms text in the auth dialog to match the tighter dashboard layout. This is a small styling adjustment for the dialog copy to keep it balanced and readable.
2026-11-06 23:37:37 -07:00
jelveh 79f24ed3df Hide the opener notice on the popup login screen
Someone logging in already has a Puter account, so like the account
picker the login screen no longer explains that the site uses Puter.
The signup screen keeps it.
2026-11-06 23:31:49 -07:00
Nariman Jelveh f30d2134b1 Keep profile pictures in the saved account list (#4531)
The account picker reads pictures from the logged_in_users list in
localStorage, but update_auth_data saved that list with an empty profile
and only added the picture in memory once the profile loaded. A picture
reached storage only when a later account switch or sign-out rewrote the
list, so the active account usually showed just its initial.

update_auth_data now keeps the account's saved profile until the fresh
one loads, then saves the user and the account list again. The profile
loads after puter.js holds the new account's token, so a switch can no
longer fetch the previous account's profile, and a profile that arrives
after another account signed in is dropped.

The first-visit migration in globals.js now stores the account list as
JSON instead of "[object Object]".
2026-11-06 23:22:11 -07:00
Nariman Jelveh c24c8589cb Hide the opener notice on the popup account picker (#4530)
Someone choosing from accounts they've already signed in with doesn't need
to be told the site uses Puter. The login and signup screens keep it.
2026-11-06 23:02:18 -07:00
Nariman Jelveh 2f8f81e621 Bind the app browser embed to the host's user and add <puter-user-menu> (#4528)
* Show the app browser embed's session only to the host's signed-in user

/embed/apps now reads the host page's signed-in user from its fragment
(#user=<uuid>) and asks /whoami which user the GUI session belongs to.
It shows the user's apps only when the two match. A host that names no
user, a malformed one, or another user, and a session the API refuses,
get the signed-out view of recommended apps with no SDK and no app
tokens. A failed /whoami check shows the unavailable state. Changing the
fragment reloads the embed.

Hosts pass the uuid from puter.auth.getUser() and remove the iframe when
their own sign-in changes; the doc's example does both.

* Add <puter-user-menu> for first-party apps on Puter subdomains

puter.js registers a <puter-user-menu> element that follows the page's
own puter.js sign-in. Signed out it shows Sign in; signed in it shows the
user's profile picture, or their initial when there is none, with a menu
holding the username and Sign out. On a page one label under the Puter
origin, an Apps button opens /embed/apps in a modal with the signed-in
user named in the fragment, so the app browser shows the Puter session's
apps only to that user. Any change to the page's sign-in closes the app
browser. The element renders nothing outside a top-level page.

Its labels can be set with apps-label, sign-in-label, sign-out-label,
close-label and account-label, and it fires signin and signout events.
It is documented internally in components.md and the embed doc, and is
left out of the public docs because the app browser only works on Puter
subdomains.

* Show <puter-user-menu>'s app browser in a panel under its button

The Apps view and the account menu are now panels that hang under their
button with a caret, closing on Escape, an outside press or the button.
They use the top layer where the browser has one, so a host's transform,
overflow or z-index can't clip or cover them. On screens up to 600px wide
the Apps panel spans the width below the button.

Closing the Apps panel keeps the app browser loaded; a change to the
page's sign-in still discards it. A spinner shows until it loads, and its
colour scheme follows the component's theme, which follows the OS unless
the theme attribute is set.

A temporary account counts as signed out: it gets Sign in, no profile
picture request, and an app browser with no user named. The signin event
carries the signed-in user, and the component exposes the parts
apps-button, avatar, sign-in, menu and apps-panel for host styling.

* Read the user <puter-user-menu> signs in as after the SDK starts its whoami

setAuthToken notifies auth listeners before it starts the new token's
/whoami, so a sign-in on a loaded page showed '?' and opened the app
browser without naming the user.

* Close <puter-user-menu>'s account menu when Tab leaves it

Focus moved on to the page with the menu still open, where Escape no
longer reached it. Tab now closes the menu and moves on from its button.

* Move focus to Sign in after signing out from <puter-user-menu>

The focused menu and avatar disappear on sign-out, which dropped focus
to the page body.

* Show the app browser in the theme <puter-user-menu> forces

Chromium doesn't pass an iframe's color-scheme on to the page inside it,
so a forced theme that differed from the OS showed the app browser in
the OS's theme inside a panel in the forced one. The component now
passes a forced theme as theme= in the fragment, and the embed applies
it to its color-scheme rules.

* Show <puter-user-menu>'s account before its profile picture loads

The avatar and Apps button waited for the /profile request as well as
the user, so a slow profile kept them hidden. The picture is now added
when it arrives.

* Open one sign-in popup at a time from <puter-user-menu>

Clicking Sign in again while its popup is still open opened another popup each time. Further clicks are ignored until the sign-in finishes or its popup is closed. A cross-origin isolated page can't see its popup close, so it keeps the old behavior rather than locking the button.

* Keep <puter-user-menu>'s Apps button unshaded after a tap closes it

Touch screens keep :hover on the last tapped element, so tapping the Apps button to close the panel left it shaded as if the panel were still open. Hover shading now applies only where the device can hover; the open state still shades the button everywhere.

* Show <puter-user-menu>'s Sign in pressed and its account hovered

Sign in had no pressed state and the account button no hover state, so neither responded to the pointer. Sign in now takes the pressed look of Puter's other primary buttons, and the account button shows its open-menu ring on hover where the device can hover.

* Show a spinner in the app browser embed until its apps arrive

The embed stayed blank while it checked the session with /whoami and loaded the SDK. <puter-user-menu> hides its own spinner once the embed's page loads, so its panel went blank for that whole time. The empty embed now shows the same spinner in the same place until it renders its apps or its unavailable message.
2026-11-06 22:18:38 -07:00
Nariman Jelveh 7d02c7f622 Offer a custom domain when publishing a folder from the dashboard (#4527)
* Offer a custom domain when publishing a folder from the dashboard

The Files tab's Publish Website dialog has a "Use a custom domain" switch.
It appears only when the server has custom domains. Turned on, it asks for
the domain, with the monthly price and suggestions from the domains already
on the account. Picking one of those moves it to the new site instead of
adding it again.

Publishing creates the site, then adds the domain. The done screen shows the
live address and the domain's status with the DNS records to set, each with
a Copy button, and a Check Status button. A failure after the site is live
keeps the site: the address locks, and the button retries only the domain.
A domain held by another account shows its ownership record to verify.

On a plan without custom domains, the switch shows an upgrade prompt in
place of the field and opens the upgrade flow where the deployment has one.
Once the plan changes, the field unlocks in place.

The /custom-domains request, error wording, price format and status-check
merge move to helpers/customDomains.js, shared with the desktop's attach
dialog.

* Name the served and forwarding domains the right way round in the redirect note

The note reads "served on {first}, forwarded from {second}", but both the
publish dialog and the attach dialog passed the root domain first.

* Fit DNS record labels of any length in the publish dialog

The label column was a fixed 44px, so a longer translation ("Pangalan")
ran under the field beside it. Each record card is now one grid whose label
column grows to its longest label, keeping the name and value aligned.

* Run one domain action at a time on the publish done screen

Check Status and the switch to www each disabled only themselves, so both
could be in flight and a slow status check could redraw the root domain
over the www one it had just become.

* Keep focus on the custom domain switch when it shows the upgrade prompt

Turning the switch on moved focus to Upgrade, so a second Space meant to
turn it back off opened the upgrade flow instead. The prompt follows the
switch in tab order.

* Finish with the live site when the plan can't add its domain

Once the site was live, a 402 from adding the domain left the button
reading Add Domain, and pressing it said to turn the switch off "to publish
without one". The button now reads Done and finishes with the site; an
upgrade brings the add back.

* Keep the ownership record and the Verify step in step while typing

Any edit hid the claim's TXT record for good, but the button went back to
Verify and Add as soon as the claimed hostname was typed again, asking to
verify a record no longer on screen. Both now follow the typed hostname.

* Keep the publish done screen up through a stray backdrop click when it lists DNS records

The records to set at the DNS provider appear only on this screen; a click
outside the card while switching to the provider's tab threw them away.
Done, Escape and the close button still close it.

* Listen for plan changes only once the publish dialog has offered the upgrade

Every website publish dialog added a window listener that held its closed
dialog until the next plan change, which may never come. The dialog is
modal, so the plan can only change after its Upgrade button opened the flow;
the listener is added then.

* Keep the upgrade prompt's text beside its icon on phones

Wrapping put the text on its own line under the icon, leaving the icon
alone on the first row; now only the button wraps.

* Drop "After publishing" from the domain hint once the site is live

When adding the domain needed another try, the hint still promised DNS
records "after publishing" for a site already published; it now keeps
only the price.

* Announce publish domain checks in a status region that stays on screen

Each check redrew the domain section, status line included, so its result
landed in a brand-new live region that screen readers don't announce, while
the redrawn records sat in the done screen's status region. The summary is
now redrawn on its own, outside the announcements.
2026-11-06 20:06:30 -07:00
Nariman Jelveh d2bbe2b5e2 Ignore Explorer responses for a folder the window has left (#4526)
Each refresh of an item container now takes an id, and only the newest
refresh may write to the container or its window. Before, a slow stat for
the folder a window had left could point the window back at it, so New
folder, Paste and Upload went to the old folder while the window showed
the new one. A listing that arrived just before navigating away added its
rows to the new folder, and returning to a folder while its first listing
was in flight rendered every row twice. The stat in update_window_path
also skips its writes once the window has moved to another path.
2026-11-06 18:28:25 -07:00
Nariman Jelveh 3ff70ef97f Remove web apps from native host redirects
Allow the `editor`, `markus`, and `pdf` subdomains to use standard host routing.
2026-11-06 17:20:22 -07:00
Nariman Jelveh f791024b2e Support dark mode in the desktop context menu (#4525)
The desktop context menu (UIContextMenu) stayed white with dark text when
the OS was in dark mode, even though the dashboard around it turns dark.
It now follows prefers-color-scheme like the dashboard: a dark translucent
surface, light text, and dark variants for dividers, the open-submenu and
blurred highlights, the submenu chevron, monochrome icons, and the mobile
action sheet.
2026-11-06 16:01:08 -07:00
Nariman Jelveh cce9718b45 Stop the sign-in page jumping when the iOS keyboard opens
A sign-in page already fills the screen, so it no longer shrinks itself
to the area above an on-screen keyboard. On iOS that resize fought the
browser's own pan to the focused field and the form moved up and down
as the keyboard opened. Dialogs still fit above the keyboard.
2026-11-06 15:40:36 -07:00
Nariman Jelveh 9758ada36e Keep the wallpaper off sign-in pages and drop the OIDC-only signup notice
Signed out, /action/login, /action/signup, /action/password-recovery and
the sign-in page for returning visitors, app landings and share links no
longer load the wallpaper behind them, and the page covers the screen at
once instead of fading in over whatever was there.

The signup screen no longer explains that only providers are offered when
email signup is closed; it just shows the provider buttons.
2026-11-06 15:30:57 -07:00
Nariman Jelveh 37dcee78f0 Move login, signup, account picker, password recovery and 2FA to dashboard-style dialogs (#4522)
* Move login, signup, account picker, password recovery and 2FA to dashboard-style dialogs

The sign-in surfaces were separate UIWindows that opened over each other
(login, then a recovery window over it, then a 2FA window over that). They
are now screens of one dashboard-style card built on mountDashboardOverlay,
the same shell, fields, buttons and status line as the dashboard's account
dialogs. Screens replace each other in place, with Back, and keep what was
typed.

- Page presentation for cover pages and sign-in popups: an opaque page with
  a centered card, filling the screen on phones and in popups. Dialog
  presentation elsewhere: dimmed backdrop, centered card, bottom sheet on
  phones.
- Each call site keeps whether it could be dismissed, its promise contract,
  OIDC gating in popups, Turnstile, the honeypot, bonus codes, the
  post-signup verification gates, reauth auth_id, and redirect handling.
- Keys typed in the card no longer reach the desktop's document shortcuts.
- A second sign-in prompt joins the open flow instead of stacking a card.
- Server messages render as text; prefilled values are encoded.

* Show the signup screen's errors above the provider buttons

The status line lived inside the email form, which hides behind the
provider buttons (or for good, when only provider signups are open), so
an error the screen opened with, like a refused provider signup, was
never shown.

* Cover the page with a new account's checks when signup was a page

Signup as a page (a sign-in popup, /action/signup) closes its card for
the email, phone and card checks; they opened as floating windows over
an empty backdrop instead of covering the page as they did before.

* Give the auth card's Back and close bar a background once it scrolls

On a short screen the card's content scrolled under the bare Back and
close buttons, which then sat on top of it: the Back chevron covered the
edge of a provider button and took its taps. Once the showing screen is
scrolled, the bar turns opaque with a divider, as the footer has; at
rest nothing changes.

* Raise a failed session-cookie sync alert above the sign-in card

Signing in waits on this alert with the auth card still up. The card
sits above every ordinary window, so the alert opened underneath it and
the login spun forever. It now opens in the stay-on-top band.

* Keep focus where it is when a second sign-in prompt joins the card

A prompt that joins the open auth card focused the screen's first
field, so one arriving while the user typed their password sent the
rest of it into the username field. It now only brings focus back when
it has left the card.

* Keep focus in the auth card while its form is busy

Submitting disabled the focused field, which dropped focus to the page,
so keys pressed while the request was in flight (Delete, Enter, letters)
reached the desktop's document shortcuts.

* Keep focus in the auth card when its backdrop is pressed

A press on the dimmed backdrop (or a page's background) that didn't
dismiss the card moved focus to the page, so the next keys went to the
desktop's document shortcuts and Tab could leave the card.

* Point the Turnstile migration note at the new signup screen

UIWindowSignup no longer exists; the widget renders in UI/Auth/UIAuthSignup.js.

* Reload after sign-in when only the URL's hash would change

On a dashboard tab (/#files) the redirect after a login or signup was
the same page with a hash, so location.replace only scrolled: the new
account was signed in but the page never reloaded, and the card stayed
up, busy, until closed.

* Span the screen with the auth bottom sheet on wider phones

The card's 400px cap outranked the dashboard sheet's full width, so on
phones between 401 and 600px wide the sheet left gaps on both sides.

* Return focus to the auth form when a request fails

A busy card parks focus on itself while its fields are locked. When the
server refused a signup, focus stayed there, so Enter did nothing and the
user had to tab back in. Ending busy now hands focus back to the control
that had it, once that control is enabled again.

* Keep the current account when a login is dismissed mid-request

Closing the auth card while its login request was out still signed the
browser in to the other account and reloaded once the server answered,
though the caller had already been told the user dismissed it. A login
that answers after the card closed is now dropped.

* Make the auth card's logo a link where it leads somewhere

On a cover page the Puter logo was an image with a click handler: no
keyboard access, no name for screen readers, no middle-click. It is now a
link named Puter. Everywhere else the logo is marked decorative, since the
title beside it names the screen.

* Return to the login screen when signing in fails after a 2FA code

If storing the new session failed after a verified code, the error went
to the hidden login screen while the code screen stayed up, busy, with
its boxes locked. The card now goes back to the login screen first.

* Hand the session cookie back when a login is dismissed mid-request

The server sets the session cookie as soon as it accepts a login or a
second-factor code. Dropping an answer that arrived after the card closed
kept the page on its account while the cookie had moved to the other one,
so cookie-gated actions (password, 2FA) applied to that account and the
next load picked up its token. The cookie is now synced back to the
account the page is signed in as.

* Keep padded notices inside the auth page's column

On a phone-width page or in a sign-in popup, every direct child of the
auth body is stretched to the column's width. The overlay sits outside
.dashboard, so it isn't border-box, and the padded error line and bonus
notice came out wider than the fields and scrolled the page sideways.

* Keep the recovery-code field one size when it takes focus

On phones the generic dialog-input and phone input rules outranked the
recovery-code field's style until it was focused, so tapping it grew the
text from 16-17px to 20px and shifted the screen. The field's rule now
carries the same weight as theirs, like the code boxes beside it.

* Show the provider buttons' hover in dark mode

The hover shade was the sidebar background, which in dark mode is one
step off the card's, so Google, Apple and the other sign-in buttons gave
no feedback under the pointer. Dark mode now uses the hover shade.

* Theme the signup bonus notice with the auth card

The notice kept the old signup window's fixed pastel colors, which stood
out as a light box on the card in dark mode. It now uses the dashboard's
success and danger colors, which follow the color scheme.

* Keep a new account's email check from closing without an answer

A signup that needs its email confirmed takes the auth card down and
waits on the confirmation window, whose ✕ closed it without ever
answering. In a sign-in popup reached from the account picker that left
an empty popup with nothing to pick. The check now stays until the code
is confirmed, like the phone and card checks and the boot-time email
check.
2026-11-06 15:20:08 -07:00
Daniel Salazar 162398a827 test: fix coverage line mapping, puter.js coverage OOM, and flaky backend tests (#4523)
- lowerDecoratorsPlugin dropped esbuild's source map, so coverage and
  stack traces reported transformed lines. Pass the external map through.
- puter.js node runner kept a ~6 MB `__coverage__` per vm context and ran
  out of heap. Contexts now share one object; workerd merges as it goes.
  report-coverage skips its comparison when the PR leg failed.
- CI-only test/hook timeouts of 20 s / 60 s.
- setupTestServer disables the database-liveness health check, whose
  `SELECT 1` landed in query-counting tests.
- LocalWorkerService.shutdown asserts the timer shutdown arms instead of
  the global fake-timer count.
- userHandlers worker test waits for posts by label.
- AppStore stats test pins the clock so rows don't cross midnight UTC.
- Realtime backpressure test waits up to 5 s for holds to settle.
2026-11-06 12:30:36 -07:00
jelveh 6785b4d9cc Reorder recommended apps and drop terminal
Updates `RECOMMENDED_APP_NAMES` to prioritize core productivity apps (`word-processor`, `spreadsheet`, `presentation`, `whiteboard`, `pdf-editor`) immediately after `builder`, and removes `terminal` from the recommended list.
2026-11-06 09:05:33 -07:00
Nariman Jelveh d2acdafaed Show recommended apps in the app browser embed without a session (#4517)
* Show recommended apps in the app browser embed without a session

/embed/apps only worked for a signed-in user; without a session it showed
an unavailable state. The route now puts the recommended apps
(RecommendedAppsService) in the page's config as guestApps, and when the
embed finds no session it shows those, without loading puter.js or reading
installed apps, folders, or saved order. Selecting one opens it at the
same URL a signed-in launch uses, with no opener and no token on offer, so
the app signs in on its own; apps that need the desktop open /app/<name>.
Signed-in behavior is unchanged.

createAppBrowser takes a guestApps option for this, and appTabLaunch's
URL building is shared between launchAppInTab and the new
guestAppLaunchUrl.

* Show the app browser embed's error state when the session lookup fails

Only a 401 from /get-gui-token now means there is no session. A rate
limit or server error shows the unavailable state, as before, instead of
the signed-out list of recommended apps.

* Let the app browser embed find no session when storage is refused

Reading local storage can throw when the browser blocks site data. The
embed now falls back to the session cookie, as it does for empty
storage, so a visitor without a session sees the recommended apps
instead of the unavailable state.

* Fix the app browser embed doc's broken sentence about puterOrigin
2026-11-06 07:48:11 -07:00
Reynaldi Chernando 6a46363d6d reorder api sections (#4519) 2026-11-06 17:30:22 +07:00
Reynaldi Chernando 2de9d509e8 docs layout cleanup (#4518) 2026-11-06 16:22:48 +07:00
Nariman Jelveh 763742dbc0 fix(embed): read the app browser's session from the cookie when its storage is empty (#4516)
Safari keys a framed origin's local storage by the top-level origin, so the
app browser embedded on a sibling subdomain finds no auth_token_v2 and shows
"Apps are unavailable". The session cookie is still sent there, so the embed
now falls back to /get-gui-token and keeps the token in memory.
2026-11-05 23:41:38 -07:00
Daniel Salazar 7f0cd1689b fix(embed): only sibling subdomains not served by core can frame the app browser (#4515)
/embed/apps now names the referring origin in frame-ancestors only when it
is exactly one label under the domain and that label is not one the server
answers itself (api, dav, js, www, native-app hosts) or the host of
api_base_url. Nested subdomains are refused too.

The query-string redirect now checks the raw URL instead of the parsed
query, so every request carrying a query string is redirected.
2026-11-05 23:10:34 -07:00
Daniel Salazar 9d22f39928 Ask for notification permission from Puter's dialog when the click has expired (#4514)
On a website, puter.notifications.enable() threw user_gesture_required
without prompting once the calling click's activation was gone: after
puter.auth.signIn() or requestPermission() opened a popup in the same
click, after a few seconds of awaited work, or when called on load. Apps
that ignored the error showed nothing.

It now shows Puter's consent dialog in that case and asks the browser
from its Continue click; Cancel resolves false. Calls made while an ask
is open share it. PuterDialog gets an onContinue option and a show()
that launch() also uses. Inside Puter's own desktop page the call still
throws, since it always runs from a click.

The push guide also notes that Chrome stops showing the prompt after
three dismissals and that its quieter messaging shows a bell icon.
2026-11-05 23:10:06 -07:00
Nariman Jelveh ef185c04ca Open apps from the dashboard's Home tab at their own URL (#4513)
Recently used apps on the Home tab opened /app/<name> in a new tab, which
loads the desktop and launches the app from there. They now open the way
Apps tab tiles do: a new tab at the app's index_url through launch_app's
newTab, or in this page when the browser refuses the tab. The Apps tab's
open-window lookup moves to helpers/focusAppWindow.js so both share it,
and the shared launch lives in helpers/dashboardAppLaunch.js, outside the
app browser so the embed bundle doesn't pull in launch_app.
2026-11-05 21:30:52 -07:00
Nariman Jelveh dec7d1fd96 Load the app browser embed's script from beside the GUI bundle (#4511)
The embed page always loaded /dist/apps-embed.min.js from the server,
while the GUI bundle and the embed build are published to gui_bundle's
location (a CDN in production). The server's /dist copy can be stale, so
the embed ran an old build. It now loads apps-embed.min.js from
gui_bundle's directory, keeping /dist when gui_bundle isn't set.
2026-11-05 21:07:05 -07:00
Nariman Jelveh fb9e7d70c6 Open apps from the embedded app browser at their own URL (#4510)
* Open apps from the embedded app browser at their own URL

Selecting an app in the /embed/apps browser now opens it in a new tab at
its index_url, with the same launch params the dashboard's tab launch
uses, instead of the GUI's /app/<name> landing. The tab is opened inside
the click and without noopener; the app's SDK asks the embed for its
token through the opener, so the token stays out of the URL.

Apps that need the desktop (built-in and godmode apps, private apps the
user can't open, or a failed token request) still open /app/<name>, with
the tab's opener cleared. Website shortcuts are unchanged.

The app URL, the app token request and the /rao report move out of
launch_app into helpers without desktop globals (appLaunchUrl.js,
appLaunchApi.js), so launch_app and the embed build the same URL.
uuidv4 moves to a helper the embed can import.

* Tag app launches with utm_medium and a utm_campaign naming the launch surface

Apps now get utm_medium=referral and utm_campaign=desktop, dashboard, or
app_browser_embed alongside utm_source=puter.com, so their analytics can
tell where in Puter they were opened from.
2026-11-05 20:31:30 -07:00
Nariman Jelveh 819e77bb4c Open dashboard apps and files in a new tab at the app's index_url (#4508)
* Open dashboard app tiles in a new tab at the app's index_url

Clicking an app tile in the dashboard now opens the app in a new browser
tab at its index_url, with the same query params (puter.app_instance_id,
puter.app.*, puter.auth.token, puter.domain, puter.api_origin,
puter.locale, puter.args, ...) its in-page iframe gets.

launch_app takes a newTab option: it runs the normal launch (private
access, file signing, token) and sends the caller's tab to the built URL
instead of opening a window. The tile click opens the blank tab inside
the click so popup blockers allow it, and closes it if the launch fails.

* Hand an app tab its token through the opener, not the URL

An app opened in its own tab no longer gets puter.auth.token in its URL,
which kept the token in browser history, and which puter.js ignored anyway
outside an iframe.

The GUI now holds the token for the tab it opened. Once the app loads,
puter.js (standalone, with puter.app_instance_id in its URL and an opener)
asks its opener for it at the configured GUI origin, takes only a reply
from that window at that origin, and then clears window.opener. The GUI
answers once, only to that tab at the origin it sent the tab to, and
drops the offer after two minutes. Sign-in and implicit auth wait for the
handoff, then fall back to the normal sign-in flow when no token came
(opener closed, slow load, redirect to another origin).

Built-in apps join explorer in opening in-page, since they belong to the
desktop.

* Open files from the dashboard Files tab in a new tab

Opening a file in the Files tab (double-click, Enter, tap, or the Open
menu item) now opens its app in a new browser tab, the same way app
tiles do. The tab is opened inside the gesture and handed to open_item,
which passes it to launch_app or closes it when the file opens no app.
Web links reuse the tab, cut off from the dashboard. A second file of a
multi-select Enter, whose tab the popup blocker refuses, opens in-page.

The opened file's signed read/write URLs don't expire, so a tab gets
them through the opener handoff with its token instead of in its URL;
the URL keeps only puter.item.uid as a marker. puter.js delivers the
handed-over item to onLaunchedWithItems / onItemsOpened, and
wasLaunchedWithItems answers true while it's on its way.

launch_app now closes a tab it was given whenever the launch fails, so
callers don't each have to.

* Open "Open With" apps from the Files tab in a new tab

Picking an app under Open With in the dashboard Files tab now opens it
in a new browser tab, like opening the row. The tab is opened in the
menu click, or, when the "always open with" prompt comes first, right
after it's answered, since that click counts as a fresh user gesture.
If the popup blocker refuses the tab, the app opens in-page. The desktop
keeps opening in-page.

* Wait for a GUI-opened tab's token in getUser, whoami and getProfile

A tab the GUI opens for an app gets its token over postMessage moments
after puter.js loads. These calls short-circuited on a missing token, so
an app reading the user at startup failed with a 401 (or a null profile)
even though the token was on its way. They now wait for the handoff
first, the same way sign-in already does.

* Close an app's blank tab before alerting that its launch failed

A launch denied for a private app, or one whose token couldn't be
fetched, awaited its alert while the blank tab it was given stayed in
front. The alert renders on the dashboard behind it, so the user was
left on an empty tab until they went looking. The tab is now closed
before the alert shows.

* Launch an app tile in-page when the browser refuses its tab

When window.open returned null (a popup-blocking setting, or a context
that can't open tabs), clicking an app tile did nothing at all. It now
falls back to the in-page launch it had before, as opening a file from
the Files tab and Open With already do: an open window is surfaced,
otherwise the app opens maximized with the tile's open morph.

* Open no tab for a file opened from the Trash

Opening a file in the Trash only explains that it can't be opened, but
the Files tab still opened a blank tab for it first, which then closed
again: a flash of a new tab before the alert. Trashed files (by the
path a shortcut resolves to) now get no tab.

* Don't count a tab launch as one that can report interactivity

launch.ipc_tracked on app-is-ready is the denominator for the
app-interactive series: launches that could produce one. An app opened
in a new tab never signals over IPC, yet was recorded as tracked, so
every tab launch counted as an interactive span that never came. Tab
launches are now recorded as untracked.

* Leave openerLaunch_ unset when there is no GUI opener to ask

requestLaunchFromOpener resolved null asynchronously even when there was
no opener at all, so openerLaunch_ was briefly set on every boot with an
app instance id in the URL. A reloaded app tab (opener already cut off,
URL still marking the opened item with puter.item.uid) reported
wasLaunchedWithItems() as true to code running in the same task as the
boot, though no item would ever arrive. With no opener to ask it now
returns null synchronously and nothing is left pending.

* Call an app tab launch off when the user closed its tab

Closing the blank tab while the launch's fetches were in flight didn't
stop the launch: it still navigated the closed tab, offered it a token,
and recorded the app as opened (recent apps and /rao). The launch now
checks the tab before sending the app there and ends as not launched,
without an alert, when the user has closed it.

* Keep a same-origin opener instead of asking it for a GUI launch

Any top-level page with an app instance id and an opener asked that
opener for its launch, waited out the timeout when no answer came, and
then nulled window.opener. A page an app opened from its own origin (a
pop-out of itself, say) can't have the GUI as its opener, yet it waited
five seconds before its launch item and sign-in, and lost its link to
the app window that opened it. An opener readable from this page, on an
origin other than the GUI's, is now left alone and not asked.
2026-11-05 19:27:46 -07:00
Daniel Salazar 7355eaa77b feat(realtime): run the host prompt check per response and log its cost (#4507)
Each response that starts runs `ai.prompt.validate` on the text the client
sent since the previous check (instructions, message text and transcripts,
function call output), and its usage follows as `ai.prompt.cost-calculated`
under the same completion id. A refusal closes the session with
`permission_denied`. Nothing runs when no host listens for the check.
2026-11-05 18:05:43 -07:00
Nariman Jelveh d55aac1f98 Add embeddable Puter toolbar and shared app browser (#4019)
* Add embeddable Puter toolbar and shared app browser

Provide deployment-scoped iframe routes, a responsive host helper, read-only app browsing, saved-account switching, and new-tab account and upgrade links. Validate account sessions before switching and keep credentials inside the GUI origin.

Verified with 182 targeted tests, production GUI build, backend type checking, lint, and a real-backend browser smoke test. Translation checker could not run because tools/check-translations.js is missing.

* Prevent embedded toolbar from scrolling when menus open

* Stop double-encoding translated toolbar labels

i18n() already HTML-encodes its result, so wrapping it in escape() again
turned accented labels into literal entities (e.g. "Schlie&szlig;en") for
screen readers, and .text() did the same for the error message.

* Load only the preferred language in the toolbar and app browser embeds

Importing i18n.js pulled every translation into each embed, making the
toolbar bundle 1.1 MB. The embeds now bundle English and fetch the user's
language on demand (toolbar 1.12 MB -> 179 KB, app browser 1.18 MB -> 237 KB).
The desktop bundle keeps all translations.

* Keep toolbar buttons in place when a menu opens near the viewport edge

On phones the expanded frame was clamped to an 8px viewport margin, which
moved the whole toolbar row (by the header padding minus 8px) under the
user's finger. The frame now shrinks to fit beside and below the collapsed
toolbar, and only falls back to shifting when there isn't enough room.

* Don't pull focus back into the toolbar when the host page closes its menu

Clicking a host text field while a toolbar menu was open closed the menu and
then refocused the toolbar button, so the host field lost focus and typing
went nowhere. Focus is now restored only while the toolbar document has it.

* Close toolbar menus when the empty part of the expanded frame is pressed

While a menu is open the toolbar frame covers host content beside the
buttons; presses there reached neither the host nor the toolbar's outside
click handling, so the menu stayed open. They now close it.

* Omit the empty email line in the toolbar account menu

Temporary accounts and saved accounts without an email rendered an empty
element, adding a stray gap under the username.

* Stop user-hosted sites from framing the toolbar and app browser embeds

frame-ancestors allowed every *.<domain> subdomain, which on the default
self-hosted layout includes user sites (*.site.<domain>) and private apps
(*.app.<domain>), letting any hosted page clickjack sign-out and account
switching. The policy now names the embedding page exactly, from the
request's referrer, and refuses the user-content hosting domains.

* Use the dashboard danger color for the toolbar sign-out confirmation

The hardcoded red had 3:1 contrast on the dark panel, below WCAG AA for
14px text. The dashboard's danger token is 4.8:1 in light and 6:1 in dark.

* Drop the dashboard's hamburger clearance from the embedded app browser

At narrow widths the dashboard pads the search 60px down to clear its fixed
menu button. The embeds have no such button, so the toolbar's app panel
lost 90px of a ~500px frame to empty space above the search.

* Keep the toolbar's Upgrade button legible in dark mode

Upgrade was filled with the link color, which dark mode lightens to
#6b8cff: 3.1:1 under white text. Use the dashboard's primary-button fill
(4.6:1 in both themes) and its hover color, and drop the button classes
nothing styles on this page.

* Keep the toolbar's apps icon legible on hosts in the other theme

The toolbar follows the OS theme but draws on the host page, so in dark
mode on a light host its icon was #d4d4d4 on white (and the reverse on a
dark host). Give the apps toggle a surface of its own, like the avatar
beside it, and the session-unavailable notice one too.

* Let the toolbar sign out of a session that already ended

When the session was ended elsewhere (revoked, or signed out in another
tab), the anti-CSRF and logout requests answer 401 and sign-out failed
on every retry, stranding the stale token. Treat 401 as already signed
out and finish clearing it locally, as the desktop does; other failures
still keep the session and show the error.

* Keep the toolbar's slot following the host layout while a menu is open

The helper reserved the toolbar's space with a span frozen at the
frame's resolved size and margins, so a host that right-aligns it with
margin-left: auto (or sizes it in percentages or media queries) left the
open menu anchored to a stale spot, overflowing the header, after a
resize or rotation. Reserve it with an empty clone of the frame instead,
which the page's own rules keep laying out; if they target
data-puter-toolbar, which the clone drops so hosts still find one frame,
hold the frame's measured size.

* Reuse isOrgSeat for the toolbar's Upgrade check

The toolbar re-implemented the dashboard's organization-seat test
inline. Use the shared helper so both surfaces agree on who sees an
upgrade offer.

* Point comments at AppBrowser.js, where the Apps tab code now lives

The constants and methods these comments name moved out of TabApps.js
when the app browser became shared.

* Restore the license header on the new embed and toolbar files

TabApps.js lost its AGPL header when its code moved to AppBrowser.js,
and the new embed, toolbar, and host-helper modules were added without
one; every other GUI source file carries it.

* Remove the embeddable toolbar, keeping the standalone app browser

Drop /embed/toolbar, its page and host helper, their styles, strings and
tests, and the dashboard's ?upgrade=1 deep link that only the toolbar's
Upgrade button used. /embed/apps no longer allows the GUI origin to frame
it with a wildcard over deployment subdomains, which existed only for the
toolbar's nested app browser, and its bootstrap config no longer carries
the domain the toolbar read.
2026-11-05 17:19:05 -07:00
Daniel Salazar c6987d9606 feat(notifications): push unless a page is visible and focused (#4506)
A push is skipped only when a page that got the notification live was
visible and focused. Hidden, unfocused or unanswering pages no longer hold
it back, and pages no longer show browser notifications themselves, so the
push reaches every subscribed browser.

- Receipts carry the page's visibility and focus as of arrival; the settle
  gains `handled`. `showsOwn` is gone.
- Pages send `notif.handled { uid }` on the socket a notification arrived on
  when they got it while visible and focused. Core relays it as the
  node-local `notif.handled` event (60 a minute per user) for extensions.
- puter.js drops the page-shown notification path; push-sw.js drops its
  page-show message. Handlers still run live in the page.
- The desktop registers /sw.js only when push is turned on, updates it when
  subscribed, and removes one left without a subscription. Its hidden-tab
  banners are gone.
- Docs describe the focus rule.
2026-11-05 14:43:33 -07:00
Filip Kujundžić 796b42c74d Merge pull request #4475 from HeyPuter/fix/openai-cache-write-tiers 2026-11-05 21:27:46 +01:00
404oopsandClaude Opus 5.5 e2043a5e58 Merge remote-tracking branch 'origin/main' into fix/openai-cache-write-tiers
Main's docs cleanup (#4504) moved the prompt caching docs under
per-provider sections; the OpenAI paragraphs now live under
OpenAI models > Prompt Caching.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 19:55:55 +01:00
Filip Kujundžić 8b98c7ada8 Merge pull request #4503 from HeyPuter/codex/openai-image-default
feat(ai): add GPT Image 2.6 and default to Flare
2026-11-05 19:53:05 +01:00
Filip Kujundžić b27e84d181 Merge pull request #4457 from HeyPuter/fix/convo-bounded-recovery
Bound conversation recovery and report omitted replay context
2026-11-05 19:46:10 +01:00
404oopsandClaude Opus 5.5 add0e7be7d fix(convo): report routine context trimming as context_trimmed
A reconnect that left older items out of the 40-item, 48 KiB restore window
fired an error event with context_restore_failed, the code a provider
refusal uses, with the count only in the message. Every reconnect of a long
conversation raised an error that apps treating error as fatal would surface.

Trimming now fires a nonfatal context_trimmed event, before ready, with the
count in omittedItems. context_restore_failed is kept for a provider refusal
of a restored item or a failed send of the replay.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 19:28:02 +01:00
404oops b5d221ede6 fix(convo): bound recovery attempts and report omitted context 2026-11-05 19:28:02 +01:00
404oops 3137a631a7 fix(realtime): reuse settled credit for post-record continuation 2026-11-05 19:28:02 +01:00
404oopsandClaude Opus 5.5 57706d8093 fix(realtime): bound unacknowledged audio by duration, not a byte cap
The per-commit cap of 1,280,000 bytes assumed G.711, so a PCM16 session
(48,000 bytes/s, the SDK's format) with transcription on was closed with
1013 after about 27 seconds without a commit: silence under server VAD,
one long utterance, or a long assistant reply.

Unacknowledged audio is now a duration at the provider-confirmed input
format (the lower rate while a requested format is unconfirmed). Without
turn detection it counts all audio since the last commit or clear; with
server VAD only speech from speech_started, plus the buffer after an
explicit commit, since the provider drops silence. Each started 160 s
claims one spare item as its hold instead of closing the session; a
commit, clear or input_audio_buffer_commit_empty returns the holds to the
pool. If the upstream closes first, every claimed item is billed. The
settle no longer commits silence.

The relay test file now expires the in-memory per-IP pre-auth window
after each test; every test connects from loopback and the added tests
pushed the file past 120 upgrades a minute.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 19:27:54 +01:00
Reynaldi ChernandoandClaude Opus 5.5 fec1d1916f Merge remote-tracking branch 'upstream/main' into codex/openai-image-default
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-06 01:03:08 +07:00
Juan Fernando CastroandReynaldi Chernando da369039e0 docs: complex teams recipe (PUT-1986) (#4095)
* docs: complex teams recipe (PUT-1986)

Showing which teammates have a file open, built from a folder shared with the
team plus `puter.events` watching it. There is no presence API, so the recipe
is about composing the two.

One use case per the recipe guidance. Uses the masked path from
`listShared()` so members address the owner's folder, keys presence files on
`uuid` rather than username, and re-reads the folder on each event so a missed
delivery cannot leave the list wrong.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-11-06 00:53:25 +07:00
404oopsandClaude Opus 5.5 4f36152850 fix(realtime): hold transcription credit per item at the selected model
Input transcription reserved 64 items up front at the costliest model's
price and largest cost factor, about $7 before factors per session. Ordinary
subscribers couldn't enable it, and concurrent sessions were refused while
balance was left.

Each item now holds its own credit at the selected transcription model's
budget and cost factor. Two spare items are reserved before configuration or
audio is forwarded and refilled as the provider commits items, within the
64-item pending cap. A commit with no spare reserves its own item. A model
change releases the spares and reserves new ones at the new price; committed
items keep their hold. When the account can't cover another item, client
reads pause while the session's own holds may still free credit, and the
session is refused with insufficient_funds once it holds none.

Audio awaiting a commit acknowledgement is now tracked separately from pool
capacity and reset by each acknowledged commit or clear. If the upstream
closes before that acknowledgement, the audio bills as one item. Previously
the fallback required an untouched window, so a lost acknowledgement after an
earlier completed item went unbilled; the cumulative byte count could not
simply be reused without charging earlier work twice. The uncommitted-audio
cap is now 160 seconds of audio instead of a per-window total.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:49 -07:00
404oops 86db96d587 fix(realtime): reserve and settle pending transcription work 2026-11-05 10:23:49 -07:00
404oops 214ffb6605 fix(realtime): evict revoked credentials and recheck paid eligibility 2026-11-05 10:23:48 -07:00
404oops 871cc2cf87 fix(realtime): bound upgrades before credential admission 2026-11-05 10:23:48 -07:00
404oops ba002be0f7 fix(convo): reject interrupt readiness failures asynchronously 2026-11-05 10:23:47 -07:00
404oops 2832926f8d test(realtime): isolate credential attempt budgets between cases 2026-11-05 10:23:47 -07:00
404oops 1ab1c04760 fix(realtime): redact structured provider diagnostics 2026-11-05 10:23:47 -07:00
404oopsandClaude Opus 5.5 63a98f89ff test(metering): find the actor key in any meterIncr argument
The buffered write now passes the claim key too, which moves the counter
key's position. The image provider's write-ordering test matched it by
index and stopped seeing the write.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oops d2cef27085 perf(metering): avoid retries for growing buffers 2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 74b60c1227 fix(metering): type the array read's retry callback
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 2ed3e78dfa fix(metering): keep buffered and store reads from missing or doubling usage
Three reads still got a flush wrong:

- A write re-sent after its reply was lost, found already applied, left
  the claim without touching a warm cached base, so balance reads stayed
  short of what the store held. The claim and the base are now dropped
  together, and the next read seeds a fresh base from the store.
- A read with no cached base took the buffer, then the store; a flush
  that wrote a batch and left its claim in between had it counted twice.
- An exact read added another deployment's claim but not the usage that
  arrived behind it, which that read can't flush, so a charge crossing
  the allowance went entirely to the allowance.

Reads that go to the store now take everything buffered (delta and
claim), read the store consistently, and read the buffer again, retrying
when it shrank in between. Single and array reads and exact reads all
use it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 22ebd353b8 fix(metering): read the claim before the store in an exact read
`readExact` read the store and the outstanding claim side by side. When
a flush wrote a batch and took it off its claim between the two, the
store read had the counter from before the write and the claim read had
nothing, so the batch was missed: 10 units recorded, 0 read.

The claim is now read first. An amount gone from it by then was written
before the store read, which is consistent, so the read has it. When the
claim changes during the store read, the batch may be in both, so the
read is retried, up to three times.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 7551e12370 fix(metering): count claims on the write path and read missing bases exactly
The running total `incr` returns and `readExact` also missed usage a
flush had claimed but not yet written. Both feed how a charge is split
between the allowance and purchased credit, so an account could be
charged to its allowance for usage that had already used it up. Both
now add the counter's outstanding claim.

A read or seed with no cached base asked the store with an ordinary
read, which could miss a write a settle had just made; it now reads
consistently. A write-through after a cache failure drops the cached
base, which doesn't have the write, so the next read reseeds it.

The dirty-set reconcile test no longer depends on how many flush cycles
earlier tests ran.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 6e29e32fe9 fix(metering): count usage mid-flush in balance reads
A flush takes a counter's buffered amounts aside before writing them
onward, and reads only looked at the live delta and the cached base.
From the claim until the settle replaced the base, those amounts were
missing from every read: one store write normally, up to the orphan
sweep if the write failed. Balance checks, credit reservations among
them, could see an account as having more than it does.

Reads now add the counter's outstanding claim. A batch written onward
leaves the claim and joins the cached base in one step, so it stays
counted until the settle replaces the base with the store's view.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:46 -07:00
404oopsandClaude Opus 5.5 54bfc0ffe5 test(puter-js): prove microphone chunks hit the closing-socket gap
The browser test now counts the chunks the transport refused while the
socket read as closing, and requires at least one, so it can't pass
without exercising the gap. The socket stays closing until the
connection drops instead of going back to open.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:45 -07:00
404oopsandClaude Opus 5.5 c27883134b fix(puter-js): keep microphone capture when the socket closes first
A conversation still reads as connected for a moment after its socket
starts closing. A microphone chunk sent in that moment got
`session_closed`, and the capture handler treated that like a real
media failure: it stopped the tracks and the AudioContext, so the
replacement session after the reconnect had no microphone.

The built-in microphone now drops a chunk the transport refuses with
`session_closed` or `session_not_ready`, as it already did for
`queue_full`, and keeps capturing. A reconnect resumes sending; a final
close still stops capture through the conversation, and processor or
other errors still release it. Dropped chunks aren't replayed, and
public sends keep reporting the transport error.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:45 -07:00
404oopsandClaude Opus 5.5 cf4c1f6b64 fix(puter-js): end the conversation on an oversized event mid-reconnect
A server event over the 4 MiB limit closes the socket with 4009, which
ends a connected conversation. During a reconnect attempt the same close
was reported as `session_closed`, which is retried, so the conversation
tried ten more times and finally closed with the original drop code.
A replacement session opened after a restart notice was reopened again
and again for the same reason.

A 4009 close now ends the conversation during a reconnect too, reporting
4009, and a replacement session that hits it isn't reopened. Tests cover
both, a multibyte event of exactly 4 MiB, and a whole retrieved item
through the relay.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 305d20ab1c fix(puter-js): accept provider events up to the relay's 4 MiB cap
The conversation transport refused any server event over 64 KiB with a
terminal 4009 close, the limit meant for what the SDK sends. The relay
forwards provider events up to 4 MiB, and a session.updated echo or a
whole audio item easily passes 64 KiB, so valid conversations ended.

Incoming events now have their own 4 MiB limit, matching the relay,
counted in UTF-8 bytes (a string short enough that it can't exceed the
limit isn't encoded to check). An event over it still closes with 4009
and doesn't reconnect. Uploads keep their 64 KiB cap, and malformed
events are still ignored.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 0ce6b0646a perf(metering): stop releases from churning the reservation version
Every release bumped the balance version, so an account's own short-lived
holds (each realtime session probes one and releases it at once) could
keep its other reservations reading a stale version until they were
refused. The usage a hold stands for already bumps the version when it's
recorded, before the hold is released, and a release without usage
leaves the balance as it was, so releases no longer bump it.

The admission read also uses the cached add-ons record when it shows no
purchased credit left, which can only understate the balance; it reads
consistently only while credit remains.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 6b5c4f3d9e fix(metering): refuse a reservation whose balance never holds still
A reservation that found the balance changed three times in a row used
to skip the version check on its fourth attempt, which brought back the
race the check closes; an account could provoke that itself. It now
waits a short randomized time between attempts and refuses after five.

A missing version key now starts from a random value, so a version read
before the key expired can't match its successor. Credit grants,
allowance changes, manual usage corrections and a usage write that
fails partway also bump the version, and the store refuses an amount
it can't hold instead of storing a negative hold.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 d6fdc5f268 fix(metering): reserve AI credit only when it fits, in one step
`reserveAiCredits` checked the balance and then took an unconditional
hold, so concurrent requests could all pass the check before any hold
was visible; two 16.2M µ¢ holds were admitted against a 20M µ¢ balance.

The hold store now takes a hold only if it fits beside the live holds,
in one Lua script on the actor's hold key. A per-user version, bumped
when usage is recorded and when a hold is released, lets the script
refuse a balance read that a settlement has since overtaken; the
service reads the balance again and retries, and skips the version
check on its last attempt rather than refuse an account whose balance
keeps moving. Both keys share the actor's hash tag, so the script runs
on Redis Cluster. The balance read for this check reads purchased
credit consistently.

`reserveCreditsIfAffordable` carries this through the service, and the
AI facade calls it once with the cost factor applied. `reserveCredits`
keeps its unconditional semantics. When the cache is unreachable, the
balance alone decides and nothing is held, as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:44 -07:00
404oopsandClaude Opus 5.5 4f3f946941 fix(realtime): judge a refusal by the plan it was decided on
A plan or funds refusal was classified by a plan lookup made after the
decision. A fallback plan is cached for only a few seconds, so when it
ran out in between, the refusal was judged against the recovered plan
and still ended the session with a terminal 1008. The plan is now also
checked just before each decision, and a refusal is retryable if either
check found a fallback.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 4e36672b6b test(puter-js): fail credential lookups until the relay hits one
The reconnect test made the next credential lookup fail once, but the
SDK's own socket.io connection could take that failure before the
relay's reconnect attempt did, so the relay never saw a failed lookup
and the test failed in CI. Lookups now fail until the relay has refused
an attempt on one, then recover.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 6b0b530ab1 fix(realtime): retry instead of refusing when the plan lookup fails
When the billing lookup failed, metering answered with a provisional
fallback plan, so the relay refused the session with
`subscription_required` or, against the fallback's allowance,
`insufficient_funds`, both terminal 1008 closes, and the SDK showed the
upgrade prompt to a paying account. A plan or funds refusal decided on a
provisional plan now sends `internal_error` and closes with 1011, at
connection and mid-session, so the client retries once the lookup
recovers.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 68b8f341e9 test(realtime): cover a revoked session credential
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 c3756d3ac3 fix(realtime): tell transient server failures apart from bad credentials
A credential lookup that threw (a database or cache outage) was treated
as no actor, so the session got `invalid_api_key` and a 1008 close. That
is terminal for the SDK, which then ended an established conversation
that only needed to reconnect. A failed lookup now gets `internal_error`
and closes with 1011, which the SDK retries; missing, invalid and
revoked credentials keep `invalid_api_key` and 1008. Nothing is cached
for the failed lookup, and the cause is logged server-side only.

A session whose usage records pile up behind metering now closes with
1013 instead of 1008, still with `rate_limit_exceeded`. Queued usage is
still recorded and its holds released. An unexpected setup failure now
sends `internal_error` before its 1011 close.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:43 -07:00
404oopsandClaude Opus 5.5 0c927ac5a4 fix(realtime): listen for client socket errors before refusing a session
A session refused while the server drains returned before its client
error and close listeners were registered, so a frame that errored the
socket during the close handshake (an oversized one, for example)
emitted an unhandled `error` and ended the process. The listeners now
go on as soon as the session exists, ahead of every refusal.

The 404 answer to an upgrade for another host also ends a raw socket
that has no error listener once Node hands it over; it now has one.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 10:23:42 -07:00
Neal Shah 64c3b3c0ce subdomain and worker recipes complex, writing full stack applications with both (#4206) 2026-11-05 21:33:21 +07:00
Reynaldi Chernando e9a4c7a461 ai docs cleanup (#4504)
* ai docs cleanup

* fixes
2026-11-05 19:15:32 +07:00
Filip Kujundžić be9cd9eb89 feat(ai): add GPT Image 2.6 and default to Flare 2026-11-05 11:13:43 +01:00
Daniel Salazar 8b64cc5076 docs(notifications): recipes and format pass for dismissal, priority, buttons and schedule updates (#4502)
- dismiss.md is the one place that explains closing a notification on other
  devices; on/onNotification/push link to it instead of repeating it.
- Pages follow the house format: section order, error tables, playground
  examples for dismiss() and markShown(), sidebar and index listings.
- updateSchedule.md describes what a developer sees instead of internals.
- New recipes: mark notifications as read, incoming calls (high priority,
  short ttl, buttons), unread counts on the app icon.
- Existing recipes move reminders, snoozes and recurring summaries with
  updateSchedule() instead of cancel-and-recreate; the buttons recipe follows
  the recipe format.
2026-11-05 00:32:53 -07:00
Daniel Salazar d3869b7846 feat(notifications): dismissal across devices, high priority, buttons, schedule updates, docs fixes (#4501)
- Docs: status(), browser limits, events worker fetch/ack, update() across
  pages, stored-notification retries, device removal, push delivery, queue
  coalescing, existing service workers and missing error codes match the code.
- puter.notifications.markShown(uid) / dismiss(uid).
- Dismissals reach the app's live feed (on('dismiss')); pages close the
  notification on screen, and a later page load closes ones already dismissed.
  The desktop closes its own copies on notif.ack.
- Live notification postings are forwarded to sessions in other regions.
- notify({ priority: 'high' }).
- onPush() results accept requireInteraction and actions; clicks carry action.
- replaced reaches service-worker onPush() handlers.
- puter.events.updateSchedule(scheduleId, changes), with the invoker
  re-checking a schedule run's guard right before it is sent.
2026-11-05 00:17:27 -07:00
Daniel Salazar 91230697cd fix(backend): shutdown hooks don't skip each other, and final flushes wait for in-flight work (#4499) (#4500)
A throwing prepare/shutdown hook stopped the loop, so shutdown() skipped
closeAllConnections and every later hook, including the final metering
flush; each hook now runs and failures are logged. Broadcast, events
forwarding and share digests joined nothing already in flight and could
drop what was queued behind it; their final flushes now wait for it, ship
the tail, and drain once more at shutdown. Local worker disposal and the
OpenAI image metering call are awaited. settleWithin and
FINAL_FLUSH_TIMEOUT_MS bound each final flush.
2026-11-04 21:47:35 -07:00
Daniel Salazar 56e4856518 feat(backend): drain-start hooks run concurrently and can hold the listener open (#4498)
The listener closes once every onServerDrainStart hook settles, and the
drain window now starts at the signal, so time a hook spends holding the
listener open counts against drainMs instead of extending shutdown. A
failing hook no longer skips the others.
2026-11-04 19:40:54 -07:00
Reynaldi Chernando 8acd73675c use gpt 6 luna as example in docs (#4497) 2026-11-05 08:20:12 +07:00
Nariman Jelveh fc4af97885 fix(gui): the custom domain empty state holds its add button
With no domains, the only way on was the add button in the footer, which
the full-page popup pins to the bottom edge, far from the empty state the
eye lands on. The button now sits in the empty state under its hint, and
the footer keeps only Cancel until there's a domain to list.
2026-11-04 17:08:06 -07:00
Nariman Jelveh 7cec710014 feat(gui): redesign the custom domain dialog, full page in its popup (#4496)
* feat(gui): redesign the custom domain dialog, full page in its popup

- Domains list as cards with a status badge, trash icon, and inline
  Cancel/Confirm for remove and move
- DNS records as cards with copy buttons that confirm with a check
- Status card on the records step; a green check once the domain serves
  the site, after which the dialog closes
- Spinners on the button that's working, skeleton while loading, empty
  and unavailable states
- Pasted URLs are trimmed to their hostname before adding
- In the attach-custom-domain popup the dialog fills the page, with the
  actions pinned to the bottom

* fix(gui): Escape backs out of removing the chosen custom domain

Escape already put back a list row's pending confirmation. On the records
step, it closed the dialog while Remove was asking to confirm; it now puts
the button back first, like the rows.

* fix(gui): each custom domain step opens scrolled to the top

Switching steps reset the scroll of the dialog's body, which only scrolls
when the dialog fills a popup. The floating modal scrolls itself, so a
step opened from far down the list started scrolled past its status card.

* fix(gui): keyboard focus stays in the custom domain dialog

Focus on a control that got hidden or disabled fell to the page behind the
dialog, so Tab started over from there. Switching steps now focuses the
dialog, backing out of a row's confirmation returns to the button that
asked, a request gives focus back to the button that started it, and
removing a row moves focus to the next one.

* fix(gui): a removed custom domain row takes no input as it fades

Its buttons came back on once the delete finished, so a second click in
the fade sent the delete again, and Cancel brought back Use for a domain
that no longer exists. The row is inert until it's gone.

* fix(gui): DNS record copy buttons work without the clipboard API

On an origin with no navigator.clipboard (a Puter served over plain http)
or when the browser refuses the write, the copy buttons did nothing. They
now select the value and try the legacy copy, like the dashboard's form
dialog, and leave it selected to copy by hand if that fails too.

* fix(gui): custom domain rows keep the icon beside the name on phones

The name was sized to the row less 46px for the icon and gap, but the icon
is 36px wide with its border, so the row overflowed by 2px and every name
dropped below its icon. Only the actions wrap now; the name takes the rest
of the first line.

* fix(gui): hostnames in the custom domain dialog wrap whole

Every bold hostname broke at any letter, so one that would fit on the
next line was split mid-word at the end of this one ("puter.lo / calhost").
It now moves down whole and breaks only when it's wider than the line.

* fix(gui): readable Remove and Confirm buttons in dark mode

The red fill under their white labels used the danger text color, which
dark mode lightens to #f87171: 2.8:1 against white. Fills now use their
own token that stays #dc2626 (4.8:1); light mode is unchanged.

* fix(gui): the switch-to-www link shows when it's working

The link that switches a root domain to www had no disabled or loading
style, so while its request ran it still looked clickable and showed
nothing, while every other control dimmed. It now dims like them when
waiting and shows a spinner when it's the one working.
2026-11-04 16:45:56 -07:00
Daniel Salazar 48cd4ea355 fix: oidc toggle (#4494) 2026-11-04 13:13:15 -07:00
Neal ShahandClaude Opus 5.5 a110be0340 feat(gui): remove a custom domain from the attach dialog (#4493)
Each domain in the list gets a Remove button, confirmed in place like Move,
and the records pane can remove the domain just added. A removed domain is
never handed back as the dialog's answer.

Also fixes two strings that went stale: the claim prompt (an active domain
can be claimed too now) and the redirect note (root domains get two A
records).

PUT-2338

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 13:23:04 -05:00
Juan Fernando Castro 93798cba93 fix(puter-js): saved push handlers stay while any tab uses them, and the worker can skip them (#4481)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment

* test(puter-js): keep the push boot test comment to one line

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.

* fix(static): push-sw.js refreshes at the CDN within five minutes

* feat(puter-js): the push worker can skip saved handlers

* fix(puter-js): off() keeps a saved push handler another tab still uses

* docs(notifications): self-hosting the push worker

* docs(notifications): name the pinned push worker version as a placeholder

* fix(puter-js): pages with push handlers can enter the back/forward cache

* test(puter-js): a restored page saves its push handler again

* docs(notifications): pin the push worker only with a pinned puter.js
2026-11-04 10:15:36 -05:00
Juan Fernando Castro 848e336455 fix(puter-js): page loads check push status instead of registering again (#4480)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment

* test(puter-js): keep the push boot test comment to one line

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.

* fix(puter-js): a page load checks push status instead of registering again

* fix(puter-js): subscriptions made with an earlier Puter key are replaced

* fix(puter-js): a replaced browser subscription is picked up on the next visit

* fix(puter-js): a page load that only checks push status still looks for a newer worker

* fix(puter-js): the worker subscribes again only where a Puter page used it

* docs(puter-js): soften the note on subscriptions Puter made earlier

* fix(puter-js): a status answer on load leaves a subscription another tab just turned on
2026-11-04 10:15:25 -05:00
Juan Fernando Castro a9a8df6409 feat(events): fetch can start at the newest notification, and the feed uses it (#4479)
* feat(events): fetch can start from the newest event

GET /events/fetch takes from=end: with no cursor it answers an empty page
whose cursor sits at the newest row of the slice, so later reads return
only rows created or replaced after it. puter.events.fetch() passes
`from` through. Adds a (user_id, audience, app_uid, id) index on
notification so the newest row of a slice is one seek.

* fix(puter-js): onPush starts its feed from the newest notification

A notifications listener started without catch-up asks for from: 'end'
and keeps the cursor it gets back instead of reading every stored
notification to mark them as seen. A server that ignores `from` answers
with the oldest page, and the listener reads through it as before.

* fix(events): read the newest notification id from the primary

* fix(events): mark the end cursor at the slice's latest replacement

The end cursor's replacement mark is the newest (replaced_at, id) in the
slice rather than the serving node's clock, since replaced_at is stamped
by whichever node wrote it.

* docs(events): describe the end cursor's replacement mark
2026-11-04 10:15:14 -05:00
Juan Fernando Castro 0a8b37ae56 fix(notifications): website apps are named by their hostname (#4468)
An app created for an origin keeps its uid as name and title, so its
notifications showed that uid as the sender. appMessageSender now names
such an app by the hostname of its index_url, and keeps the stored name
when index_url doesn't parse. The check AppDriver used for these rows
moves to util/appView so both share it.

The Settings app list shows a generic label for an app whose title is
still its uid.
2026-11-04 10:15:03 -05:00
Daniel Salazar 23782dec12 feat(events): inline handlers in the user's own scope for onSchedule and onPersistent (#4487)
* feat(events): schema for user-scope handlers

Add event_user_handlers and event_subscriptions.handler_scope in all three
dialects. event_handlers is unchanged.

* feat(events): scope the handler and durable subscription stores

Handler store methods take a scope (app, or a user's scope of an app) and read
the matching table. App-scope behavior, SQL and cache keys are unchanged; user
scopes get their own cache keys, a 20-handler cap, and the inline @<sha256>
name. Adds user-scope listing, counting, deletion and unbound-handler reads.
Durable subscriptions store and return handlerScope, and listByHandler
filters by scope and holder.

* fix(events): hold the user-scope handler cap under concurrency and drop all scope caches on app delete

A create that lands past the cap removes itself and is refused. deleteForApp
drops each holder's cached set in batches, here and on peers. Batch scans over
user scopes and unbound handlers take limit and cursor.

* feat(events): set a CPU limit on events worker deploys

* test(events): cover redeploy and resend of an evicted events script

* test(events): name the deploy-on-miss test for what it checks

* fix(events): keep a deploy-on-miss call inside the delivery lease

* feat(workers): cap rented units per plan and refuse only new ones

* feat(events): publish handlers into a user scope, with inline create

* test(events): cover user-scope publish, inline create and listing

* test(workers): name the credentials gate without a vendor

* feat(events): run user-scope handlers as the holder

A user scope deploys one script per handler under the holder's own
identity, throttled per scope, with an eager deploy after publish that
skips an upload another holder already made. Delivery and schedule runs
address the holder's handler, carry the holder on the invoke headers,
batch calls, the miss handler and the rehydrate route, and stay owed
while the flag is off. Suspensions of a user scope notify the holder,
teardown (app delete, consent revoke, account delete) drops the scope,
and a job deletes unbound handlers after a day.

* test(events): cover user-scope deploy, address, notices, teardown and the sweep

Runs an inline user-scope handler through the local worker runtime as its
holder, and covers the holder on the wire, the flag-off path, revoke and
account-delete teardown, and the unbound-handler sweep.

* test(events): a lagging replica read does not suspend a user-scope schedule run

* feat(puter-js): inline user-scope handlers and scope option for handlers and workers

* docs(events): document inline and user-scope handlers, with playground examples

Cover inline handlers on onSchedule and onPersistent, handlerScope, the
scope option on handlers and workers, the user-scope limits and rented
units, and the new error codes. Add six playground examples and pin the
new limit constants in limits.test.ts.

* feat(schedules): carry the handler scope on the schedule view

* docs(events): describe user-handler cleanup, schedule handlerScope and suspension notices

* fix(events): gate the rented-unit cap on user handlers and let an account session destroy its own user scope

* fix(events): keep user-scope rows off app handlers across mixed versions and bound their deploys and reads

Cached durable rows carry an explicit scope marker; an unmarked entry with a
handler is resolved from the table and rewritten. User-scope rows are parked
before credit checks, billing, client delivery and suspension while the flag
is off. Delivery-time handler checks and addressing share the replica cache
entry. Concurrent eager deploys of one user-scope script upload once.

* fix(events): keep a user scope's age across handler churn for rent

event_user_handlers.scope_since records when the scope first held a handler.
A new row takes the scope's existing minimum, so replacing code does not make
the scope look newer to the monthly sweep. The count uses it instead of the
oldest remaining handler's created_at.

* fix(events): drop cached entries no row backs; document account-session scope destroy

* test(jobs): list the user-handler sweep among the system jobs

* fix(events): let the user-handlers migration apply on its own

* fix(events): fail a cached-row read whose scope lookup fails instead of returning app-scope rows

* fix(events): touch a reused user-scope handler before binding so the sweep cannot take it

* fix(events): bound the deploy-on-miss wait by the call's deadline
2026-11-03 23:26:04 -07:00
Daniel Salazar 6a6cf78c03 fix(drivers): stream results with pipe so responses stay under the close-listener limit (#4485)
stream.pipeline adds seven close listeners to the response; with the ones
metering, compression and tracing add, every streamed /drivers/call passed
Node's ten-listener limit and logged MaxListenersExceededWarning. pipe plus
explicit error and close handling adds two, and keeps the same behavior: a
source failing mid-body aborts the response and logs once, and a client
hanging up stops the source.
2026-11-03 23:25:55 -07:00
Nariman Jelveh 8264837d9f feat(gui): type a path into the dashboard Files path bar (#4486)
* feat(gui): type a path into the dashboard Files path bar

Clicking the path bar anywhere but on a crumb, or typing /, swaps the
breadcrumbs for a text field. Enter opens the folder typed (or a file's
folder with the file selected); ~ is home and relative paths start from
the folder on screen. Escape or clicking away puts the crumbs back, and
a path to nothing shows an alert and keeps the field open.

* fix(gui): closing the path alert with Escape gives the path field back

UIAlert only settles when a button is clicked, so an alert closed with
Escape left the path field waiting forever: it no longer closed on blur
and never got the focus back. Wait for the alert window to close
instead.

* fix(gui): look a typed path up once, however often Enter is pressed

A second Enter (or a held key) during the lookup started another one,
and for a path to nothing stacked a second alert on the first.

* fix(gui): leave Enter and Escape to an IME composing in the path field

Picking a candidate with Enter while typing a name through an input
method went to the typed path, and Escape to cancel the composition
closed the field.

* fix(gui): arrow keys go on from rows the Files tab selects for you

Selecting a typed file, or the items just uploaded or pasted, left the
keyboard's starting row on whatever was selected before, so Down jumped
next to that row instead of the new selection.

* fix(gui): opening the path field no longer moves a narrow Files header

On phones and in narrow lists the field was shorter than the crumb row
it replaced and dropped its divider, so the buttons and the listing
jumped up as it opened. It now takes a crumb's height and keeps the
divider.

* fix(gui): space the path field from the Files toolbar buttons

Its border ran up against the sort button, a few pixels from the icon,
where the toolbar's other items keep 10px apart.

* fix(gui): phone keyboards leave a typed path as typed

iOS autocorrect rewrote folder names in the path field as they were
typed. Turn it off, as the GUI's other name fields do, and label the
keyboard's Enter key Go.

* fix(gui): a typed / or .. reaches the root folder

The root can't be stat'ed, so the path field reported "/" as not found,
even though Up goes there. Like Shared, it is now opened without a
lookup.

* fix(gui): Escape calls off a path lookup still running

Escape closed the path field, but the lookup behind it carried on: the
tab still went to the path once it answered, or alerted that it wasn't
found, and a path typed into the reopened field was ignored until then.
2026-11-03 17:52:23 -07:00
Daniel Salazar eb92421a59 fix: disallow puter emails from making new puter accounts (#4482) 2026-11-03 09:17:52 -07:00
Daniel Salazar e0af3361ed ci: add manual dev notify workflow (#4477)
* ci: add manual dev notify workflow

* ci: allow choosing puter ref

* ci: restrict dev notify to maintainers, scope dispatch token to an environment
2026-11-03 09:13:04 -07:00
Juan Fernando Castro 9bd7ba7079 fix(puter-js): push taps open the right page, once (#4472)
* fix(puter-js): a tap without a URL opens the site the push came from

A page that calls onPush() or enable() saves where taps resolve for its
worker: the worker's folder when the page is inside it, else the site's
root. push-sw.js and the notifications a page shows itself read relative
URLs from it, and fall back to the worker's scope when nothing is saved.

* fix(puter-js): push taps go to top-level windows only

push-sw.js routes taps, and holds back a page's notification for a focused
window, among top-level windows only, as the GUI's worker does. Challenges
and subscription changes still reach every window of the origin.

* fix(puter-js): deliver a push tap to the page once

push-sw.js keeps a tap before posting it to an open page. The page runs
its click handler only after taking that copy out of IndexedDB in one
readwrite transaction, and acknowledges only then. When no
acknowledgement comes in time, the worker takes the copy itself and opens
the tap's URL only if the page hadn't. The wait starts at the post and
ends at most 800 ms after the click, so openWindow still runs in time.
Pages and workers without this keep today's behaviour.

* fix(puter-js): clear stored push taps that nobody reads

push-sw.js removes taps kept over five minutes on every push and tap, and
a page sweeps them once when onPush() syncs. A page opened for a tap
ignores a kept tap older than that, and removes it when no click handler
is added within 10 seconds of loading.

* fix(puter-js): a late page already at the tap's URL still gets the tap

When no acknowledgement comes in time and the window is already at the
tap's URL, push-sw.js leaves the copy it kept for that page instead of
taking it back, so the page's click handler still runs when it answers
late. A copy nobody takes is removed with the other old taps.

* fix(puter-js): a push tap runs at most once when IndexedDB fails

A page that can't take a kept tap out of IndexedDB leaves it to the
worker instead of running it, and a worker whose claim fails leaves it
to the page instead of opening the tap's URL.

* fix(puter-js): take a tap back and keep it for the next page in one step

When no acknowledgement comes in time, push-sw.js takes its copy back and
keeps it, without the id, for the page it opens in one readwrite
transaction, leaving more of Firefox's window-opening allowance for
openWindow.

* test(puter-js): shorten the fake IndexedDB comment
2026-11-03 10:06:16 -05:00
Juan Fernando Castro 8f3d4cc40f fix(puter-js): sign-out ends Puter's push subscription in this browser (#4471)
* fix(puter-js): sign-out and disable() remove only Puter's push subscription

Removal now looks at the subscription's application server key and ends
it only when it is Puter's: the key stored at the last subscribe, or the
server's key when none is stored. Any other subscription on the site is
left as it is and the server is not told about it. Without a named
worker, as at sign-out or disable() without serviceWorker, every
registration on the origin is checked, so a subscription under a folder
worker is the one removed.

* fix(puter-js): sign-out and a user change end this browser's pushes

Sign-out stops this tab's syncs, tells the site's other tabs, and inside
the push lock forgets every opt-in and tap, ends Puter's subscription in
the browser, then sends the removal with the leaving user's token as a
keepalive request so it is still sent if the page reloads.
puter.auth.signOut() returns a promise that resolves once that is done;
the token is still discarded at once.

A different user signing in over a session, or after the token was
dropped, gets the same cleanup with the token of the user signed in
before. A new token for the same user, or a dropped token, keeps
everything.

A page load finding a Puter subscription with no opt-in for the
signed-in user, or with nobody signed in, ends it in the browser. Opt-in
records that can't be read leave it as it is.

* fix(puter-js): record a push opt-in before other tabs can check it

enable() now writes the user's opt-in inside the push lock, right after
the server has the subscription and before the lock is released. A tab
loading at the same time waits for the lock, reads the opt-in, and keeps
the new subscription. A failed announce, or push turned off while it
ran, records nothing.

* fix(puter-js): push cleanup runs without the lock where Web Locks refuse

In a frame where navigator.locks.request rejects, such as a sandboxed
frame, sign-out, disable(), the load-time check and syncs now run their
step without the lock instead of rejecting, so signOut() still resolves
and the subscription is still ended.

* fix(puter-js): send the sign-out removal again when keepalive is refused

A keepalive request that needs a CORS preflight is refused by some
browsers. The removal at sign-out is then sent once more as a normal
request with the same token and without a sign-in prompt, after the
browser has already ended the subscription.

* fix(puter-js): a user change finishes its push cleanup before enable()

enable() waits for the cleanup a user change started. When the token
was dropped and a different user signs in from enable(), the pushes of
the user signed in before end first, and the new user keeps their new
opt-in and subscription. A token that sign-out already cleaned up is
not cleaned up again when the next user signs in.

* fix(puter-js): a user signing back in with the same token gets the user-change cleanup again

The token that sign-out already cleaned up is skipped only for the next
user change. Once the same token is set again, a later sign-in by a
different user ends its pushes as usual.
2026-11-03 10:06:15 -05:00
Juan Fernando Castro 96d91cd74e fix(puter-js): push syncs skip sign-in and follow disable() across tabs (#4470)
* fix(puter-js): background push syncs never prompt for sign-in

The load-time sync and the resync after the browser replaces a
subscription ask for the key and announce the subscription without
opening Puter's sign-in; only enable() may. The confirmation and the
removal never open it, and looking up the user for the opt-in uses a
non-interactive /whoami. A rejected token during a background sync is
not reported as a sync failure.

* fix(puter-js): turning push off in one tab stops other tabs' syncs

Tabs of a site take a shared Web Lock to subscribe and announce, and
again to confirm, without holding it while the confirmation push is on
its way. A load sync re-reads the opt-in inside the lock and stops when
it is gone. disable() tells the other tabs first, so their syncs and any
enable() in flight stop, then forgets the opt-ins and removes the
subscription inside the same lock. Without Web Locks the re-reads and
the message still apply.

* fix(puter-js): wait for a push confirmation only once the lock is held

A sync queued behind another tab's lock started its 30 second
confirmation timer and listener before it got the lock, so a long wait
failed the sync or left the timeout unhandled. The listener now starts
inside the locked step, right before the subscription is announced.

* test(puter-js): reach the push sync in the boot sign-in e2e

The push page now has the user's opt-in and cached user from an earlier
load, starts its sync before boot's own requests come back, and gets a
rejected token from the push routes, so the spec covers the key request
that must not open a sign-in.

* test(puter-js): run the push boot test on full Chromium and skip it without push support

The headless shell keeps Notification.permission at 'denied' after the
permission is granted, so onPush() never synced there and the test saw
no key request. The test now launches full Chromium itself, since a
channel can't be set per test, and is skipped on WebKit and wherever the
page has no PushManager or notification permission.

* test(puter-js): keep the push boot test comment to one line
2026-11-03 10:06:15 -05:00
sabrimete cdb3110857 fix(ui): honor the default font in showFontPicker on the desktop (#4340)
* fix(ui): honor the default font in showFontPicker on the desktop

String arguments are normalized to { defaultFont } in the IPC handler,
matching showColorPicker, instead of throwing on property assignment.
UIWindowFontPicker reads defaultValue/defaultFont/default case-
insensitively, and the SDK fallback and docs accept the same aliases.

* fix(ui): document the showFontPicker cancel result and alias order

The JSDoc gains overloads for the string and options call forms and
states that closing the picker resolves false in an app and null on a
website; the docs page and examples match. e2e coverage adds a
scroll-into-view check for the preselected font and a close-without-
selecting case.
2026-11-03 00:07:05 -07:00
Daniel Salazar d934a47c5e feat: jobs and schedules (#4467)
* feat(db): add nowMsExpr, the database clock in epoch milliseconds, and share the retry backoff

* feat(config): type the jobs config as an augmentable IJobsConfig and resolve its namespace in one place

* feat(jobs): add JobScheduler for fleet- and region-wide system jobs

* refactor(jobs): run core's periodic sweeps as system jobs, and publish the generation bumps of reaped subscriptions

* feat(events): seams for schedules run by an extension, with events_disabled where none is installed

* feat(notifications): add notification.send_key, the deferred mailbox write and notify({ sendAt }) through the schedules contract

* feat(puter.js): schedules, notify({ sendAt }) and notifications.cancel(), and treat a 503 events_disabled as final

* docs(events): document schedules, sendAt and cancel(), with recipes

* fix(events): stop resuming a holder's later pages once the credit sweep's signal aborts, and keep its checkpoint

* fix(events): pre-release fixes for jobs, schedule seams and delayed sends

- The delayed-send write is a field on the notifications driver, not a
  method, so /drivers/call can't reach it.
- A delayed send checks for its own earlier write by uid, so it takes no
  gap lock that every other notification insert waits on.
- The notification send_key index builds in place without locking writes.
- Reaps re-check expiry and suspension on the primary, so a row resumed
  since the replica read is kept.
- Revocation settles skip schedule rows they can't settle.
- An empty handler set read from a replica is re-read on the primary.
- One ended notice per settle, not one per page.
- Sweeps that run once per fleet take more batches per run.
- The global usage rate check keeps its 14-minute minimum window.
- shutdown() waits for prepare hooks a drain already started.

* fix(puter.js): quota refusals are final, and schedule error codes match the server

Schedule, subscription and handler quota 429s no longer go through the
generic retry. ScheduleErrorCode lists what the schedule methods throw,
and the notify({ sendAt }) overload has type tests.

* docs(events): schedule and delayed-send accuracy and style pass

Spread and send-time wording match the runner, the per-user runs-at-once
and rate-limited stored notification limits are published, and the new
pages follow the docs writing rules.

* docs(events): add the Change Data at a Set Time recipe

A one-time schedule that changes data when it runs, cancelling and moving
it, and a repeat-a-few-times variant with maxRuns.

* fix(events): delayed sends recheck the app's access when they fire, and a handler set missing the run's handler is re-read on the primary

Also stops job polling at drain start and marks replayable notification sends.

* feat(events): batch handler invocations to the events dispatcher

Invocations that come due within a few milliseconds go out as one
/invoke-batch request (up to events.invokeBatchSize, default 60). The
dispatcher runs a few at a time and streams a started line and an answer line
per call, so a slow handler delays no other call's answer. A call's timeout
runs from its started line, so time queued behind other calls is never the
handler's; queue time comes out of the connection-wait share instead, and a
call still queued past it is never started and fails as Puter's.

A lone call still uses /invoke, and a dispatcher without the batch route gets
single calls for five minutes before the batch route is tried again.
2026-11-02 23:48:35 -07:00
404oopsandClaude Fable 5.1 0128e8779d fix(ai): route per-breakpoint cache lifetimes through the Responses API
Chat Completions rejects `prompt_cache_breakpoint.ttl` whatever its
value, so a request whose content parts name a breakpoint lifetime is
handed to the Responses provider like a request-level `ttl: '12h'`. The
`/openai/v1/responses` translator keeps `prompt_cache_breakpoint` on
text, image and file parts instead of dropping it with the other unknown
fields.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-11-03 00:58:29 +01:00
404oopsandClaude Fable 5.1 89f070f7d0 fix(ai): meter OpenAI cache writes by lifetime
GPT-5.6 and later write prompt-cache entries with a 30m or 12h lifetime
and bill them at 1.25x and 2.5x input; every write was metered at the
30m rate. The OpenAI calculators now split `cache_write_tokens_details`
into `cache_write_tokens` (30m) and `cache_write_12h_tokens` (12h), the
GPT-5.6+ catalog carries the 12h rate, and `usageDetails` reports
`cacheWrite30mTokens` / `cacheWrite12hTokens` instead of labelling the
writes as 5m entries.

Callers pick the lifetime with `prompt_cache_options` ({ mode, ttl }) on
`puter.ai.chat`, `/openai/v1/responses` and `/openai/v1/chat/completions`.
The Chat Completions provider hands a 12h request to the Responses
provider, the only API that writes 12h entries. `prompt_cache_retention`
is forwarded on Chat Completions too, and the hyphenated `in-memory`
spelling maps to the `in_memory` value the upstreams accept.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-11-03 00:52:33 +01:00
Daniel Salazar 58456ae87c fix: signup gate order (#4474) 2026-11-02 13:27:01 -07:00
abb70c14a6 docs: complex fs recipe (PUT-1978) (#4093)
* docs: complex fs recipe (PUT-1978)

A drop folder that reacts to files landing in it, using `puter.events` with an
`fs:` subject instead of polling `readdir`.

One use case per the recipe guidance: watch, ignore your own writes, recover
from a gap, stop watching. The gap section works out what is already done from
the folder itself rather than a counter, so a dropped delivery costs a listing
and never a file.

* docs: rework fs recipe into a CSV importer (PUT-1978)

Renames the recipe to "Import Files From a Folder" and reframes it around
importing CSV files with puter.fs and onLocal(). Watches the app's own
inbox folder, imports files moved in from another folder, and catches up
on open and on gaps. Drops the event.self check, which skipped every file
the user added.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* refine 2

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-03 03:05:34 +07:00
Daniel Salazar 18fd655645 fix: sec improvements (#4469) 2026-11-02 12:11:38 -07:00
dependabot[bot] 480fa7b85b chore(deps): bump fast-uri from 3.1.8 to 3.1.9 (#4462)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.8 to 3.1.9.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.8...v3.1.9)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 00:05:11 -07:00
dependabot[bot] 47b54aacb4 chore(deps): bump postal-mime from 3.0.1 to 4.2.0 (#4461)
Bumps [postal-mime](https://github.com/postalsys/postal-mime) from 3.0.1 to 4.2.0.
- [Release notes](https://github.com/postalsys/postal-mime/releases)
- [Changelog](https://github.com/postalsys/postal-mime/blob/master/CHANGELOG.md)
- [Commits](https://github.com/postalsys/postal-mime/compare/v3.0.1...v4.2.0)

---
updated-dependencies:
- dependency-name: postal-mime
  dependency-version: 4.2.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 00:05:00 -07:00
dependabot[bot] dd3c0dd485 chore(deps): bump qs from 6.16.0 to 6.17.1 (#4460)
Bumps [qs](https://github.com/ljharb/qs) from 6.16.0 to 6.17.1.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.16.0...v6.17.1)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.17.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 00:04:37 -07:00
dependabot[bot] e0689986ed chore(deps): bump systeminformation from 5.33.1 to 5.33.19 (#4425)
Bumps [systeminformation](https://github.com/sebhildebrandt/systeminformation) from 5.33.1 to 5.33.19.
- [Release notes](https://github.com/sebhildebrandt/systeminformation/releases)
- [Changelog](https://github.com/sebhildebrandt/systeminformation/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sebhildebrandt/systeminformation/compare/v5.33.1...v5.33.19)

---
updated-dependencies:
- dependency-name: systeminformation
  dependency-version: 5.33.19
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:55:47 -07:00
dependabot[bot] 0f54588b31 chore(deps): bump qs from 6.16.0 to 6.17.1 in /src/puter-js (#4427)
Bumps [qs](https://github.com/ljharb/qs) from 6.16.0 to 6.17.1.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.16.0...v6.17.1)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.17.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:55:21 -07:00
dependabot[bot] 0cb9f7707d chore(deps): bump follow-redirects in /src/puter-js (#4424)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.16.1 to 1.16.2.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.16.1...v1.16.2)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:55:00 -07:00
dependabot[bot] 622b782f4e chore(deps): bump follow-redirects from 1.16.1 to 1.16.2 (#4426)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.16.1 to 1.16.2.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.16.1...v1.16.2)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-01 23:54:48 -07:00
Daniel Salazar efc0ab7126 docs: refresh KV sharing, Events and Notifications docs; fix router CORS and KV size error (#4445)
* docs: refresh KV sharing, Events and Notifications docs and recipes

Bring the KV, Events and Notifications reference pages and the recipes that
use them in line with current behavior, and tighten them.

- KV: share levels, re-share and recipient rules, handle scoping, list
  limits, the 399 KB value limit, and full error tables per method.
- Events: kv:<handle> subjects, what each way of losing access does to a
  subscription, fetch() notification types, handler ownership, and accurate
  anchor-deletion and deleted-app behavior.
- Notifications: where a push goes, desktop and website behavior, sensitive
  taps, share-notify limits, and enable()/status() errors.
- Recipes: fix broken code paths (undefined helpers, single-page share
  listings, read-modify-write races, oversized batch writes, missing value
  fallbacks), use events.update(), onlyIfAbsent and batch share recipients
  where they fit, and cut repetition.
- New recipes: Add a Notification Settings Switch, Clean Up Background Work.
- Rename push-when-closed to reach-a-user-when-your-app-is-closed, drop
  playground examples that need an app the reader owns, and redirect both.

* fix(workers): send CORS on router 404s and readable 500 bodies

Unmatched paths and methods now carry Access-Control-Allow-Origin like
every other router response, so a browser sees the 404 instead of a CORS
failure. A handler response with immutable headers is copied rather than
turned into a 500, and a thrown puter.js error object is sent as JSON.

* fix(kv): one error code for oversized values, and SDK types that match the store

The store now rejects a value over 399 KB with value_too_large, as the
SDK's own check and a growing path write already did. get() declares its
batch form, and the JSDoc gives the 399 KB limit and says watch shares
can list keys.

* docs(notifications): notifications sent within 250 ms reach live listeners as one

* test(events): give the per-handle subscription cap test room on a loaded runner

It fills the whole cap of 1,000 session subscriptions one add at a time,
which went past vitest's 5 s default on a busy CI shard.
2026-11-01 20:27:15 -07:00
Daniel Salazar ef99893fc8 fix(notifications): a browser removed in Puter stays off, and status() checks with Puter on websites (#4458) 2026-11-01 19:50:51 -07:00
Daniel Salazar f8dc10268e test(kv): give the batch-deadline retire test room on a loaded runner (#4444)
"carries on from its record when one run runs out of time" seeds and
retires a full batch of share handles. It takes under a second alone but
went past vitest's 5 s default on a busy CI shard. The timed-out body kept
running with Date.now moved ahead, and "runs once when two runners race
for it" then failed with it.
2026-10-31 14:52:26 -07:00
Daniel Salazar 5d34e803ff fix(events): delivery path fixes for queued deliveries, handler failures and per-invocation database load (#4443)
* fix(events): queue every row's single delivery before any handler runs

A dispatch queued each matched single row's event and then waited on that
row's hand-over, which can be a whole handler run, before queueing the next
row's. An event matching several rows queued the later ones only after the
earlier handlers finished, and lost them if the process stopped in between.
A batched filesystem write did the same across its entries.

The event is now queued in line and the hand-over runs detached; anything it
doesn't finish is retried by the pending sweep. A hand-over that throws no
longer raises the 'could not be queued and is lost' alarm, since the event
is still queued.

* perf(events): address an app's events worker from a cached handler set

Every handler invocation read the app's handler set (names and hashes)
from the primary database to work out which script to call. The delivery
path now reads it through a Redis cache. Publishing, updating or removing a
handler, and deleting the app's handlers, drop the entry on this node and
every peer, so only a read that raced one of those writes can address the
superseded script, for at most 30 seconds. An empty set isn't cached.
Deploys and the events-worker listing still read the primary.

* perf(events): skip leased queues without a database read, and sweep in parallel

The pending sweep read each queue's subscription row from the primary
database before finding out whether anything in the queue could be claimed.
A queue whose head is in flight or waiting out a retry now goes to the back
of the line on a Redis check alone; its row is read once the lease lapses.
The backstop re-check of a queued row's access waits for that lapse too, and
nothing is handed out in the meantime.

The pass also worked through its subscriptions one at a time, so one slow
handler run held up every queue behind it. It now works on up to ten at
once.

* fix(events): don't count a wait for a dispatcher connection as a handler timeout

Invocations share one pool of connections to the events dispatcher per
process, and the invoke timeout started when a call was made, not when it
got a connection. With the pool busy, a call could spend its timeout
waiting, be reported as a handler that ran past it, and count toward
suspending a subscription whose handler never ran.

A call now waits for a free connection before its timeout starts. A call
that waits more than half the timeout for one fails as Puter's
(unavailable): retried on the usual schedule and not counted, as the
documented failure rules already say. Wait and run together stay inside the
worker lease.

* fix(events): end a background row whose consent a delivery finds withdrawn

Withdrawing an app's events:background consent is settled by a listener on
the node that handled the revoke. If that settle never ran or failed, the
row stayed in service: every delivery found the consent gone, minted no
token and was deferred, so the backlog was retried forever and the row was
never suspended or reported.

When the delivery path's consent check answers no, it now runs the same
revocation settle (permission_revoked, backlog purged, holder told, worker
session revoked), one at a time per holder and app. A consent check that
throws still only defers the delivery.

* perf(auth): record no session or user activity for events handler calls

Every API call an events handler made authenticated through its worker
session and touched the session and the user, so background handler runs
wrote session and user activity rows on the primary database (each at most
once a minute), and replaced the session's last IP and user agent with the
worker runtime's.

Calls made with an events handler token no longer touch either. A handler
run isn't the user being active, and the handler session expires only when
it is revoked, so nothing reads its activity to keep it alive. The handler
session's last activity in the session list now reads as when it was
created.

* fix(events): settle withdrawn consent only on a fresh answer

The delivery path's consent check reads through the permission scan cache
and the cached user-to-app grants, which can trail a grant, for example on
a node that cached the app's grants before the consent was given. A stale
no then ended the subscription for good (permission_revoked, backlog
purged, holder told, worker session revoked), where before it only
deferred the delivery until the caches caught up.

Before settling, the no is now asked again with a new `primary` scan
option, which skips the scan cache and reads user-to-app grants from the
primary database. Only a fresh no settles. A fresh yes, or a check that
throws, leaves the delivery deferred.

* test(events): assert no forwarded delivery, not no traffic at all

The region's own connect announces presence to its peers on the forward
queue's next flush. Now that a dispatch returns before the hand-over, the
wait for the socket send can outlast that flush, and the test saw the
presence announce as forwarding.

* fix(events): budget the settles deliveries start for withdrawn consent

A settle a delivery starts when it finds background consent withdrawn ends
rows for good. If the permission check itself answered no wrongly, every
row a delivery reached would be suspended, its backlog purged and its
holder told, with nothing to bring it back.

Each process now settles at most 20 such pairs a minute. Past that, the
rest stay deferred, as a cached no always left them, and a warning alarm
says so. A spent budget skips the fresh read too. A pair a fresh read just
found consented isn't read again for a minute, so a stale cached no costs
one primary read rather than one per delivery. Each settle is logged.
2026-10-31 14:52:06 -07:00
Filip Kujundžić a40e4d984a Merge pull request #4339 from sabrimete/fix/alibaba-catalog-puter-ids 2026-10-31 21:09:05 +01:00
Daniel Salazar dd144b6b64 feat(notifications): notify through KV shares, group replacement, fewer push prompts, and events/worker fixes (#4432)
* feat(perms): add a pushNotifications resource to puter.perms.request

notifications:push gets a resource like backgroundEvents, so it can be
batched with other access behind one prompt. requestPermission() and
hasPermission() ask through it.

* fix(puter-js): read puter.gui_features once at load

The desktop's feature list was read from location.search on each call, so
an app whose router rewrote the URL lost it and enablePush() and the other
desktop dialogs answered as if the desktop lacked them.

* feat(gui): ask for notifications:push when an app calls enablePush without it

enable() and enablePush() inside the desktop answered unavailable for an
app without the push permission. They now show the permission dialog,
whose Allow also starts the browser registration, and answer with the
browser's push state once it settles, or off when denied. A browser that
blocks notifications or has no push service still answers without a
dialog.

* fix(kv-share): charge mint and grant budgets only for share calls that change something

A share call now spends the per-call mint budget and the per-recipient grant
windows after it has looked up which recipients already hold the share. A call
whose recipients all hold the share at the requested level and label spends
neither. Recipients without the share, including unknown usernames and refused
recipients, still spend a grant unit each. Every named recipient spends from a
new 1,200/min lookup window before any lookup runs.

* perf(events): skip the subject split for rows not made through a share handle

kvHandleFromSubject checks the stored subject's kv:kvh- prefix before
splitting it, so own-namespace and fs rows return null without parsing.

* feat(notifications): resolve enable() on a website before the confirmation push

enable() waited up to 30 seconds for the confirmation push, so a prompt
the same click started next had lost its activation. On a website it now
resolves once the browser answered and the server has the subscription,
records the opt-in, and finishes the confirmation in the background. A
failed confirmation goes to onPush's onError and the next load's resync
tries again. disable() and sign-out stop a confirmation still waiting.
The GUI keeps waiting, since it reads the registered device right after.

* fix(notifications): show a page's own notifications only while push is on

A hidden page with an onPush handler kept showing notifications after
disable(), and its receipts kept claiming showsOwn, which held pushes back
from the user's other devices. Both now need the user's opt-in, cached in
the page, refreshed on enable, disable, sign-out and each sync, and re-read
when another tab of the site changes it.

* fix(events): leave subscriptions suspended for good out of the durable quota

countForHolder no longer counts rows suspended with a reason that never
resumes (permission_revoked), in both the pre-insert check and the recount
after insert. Rows suspended for a resumable reason still count. The store now
owns the list of terminal reasons, and isResumable reads it.

listActiveForHolder with includeSuspended returns only rows suspended for a
resumable reason, so its quota-sized read still covers every row a revocation
has to settle. Each holder keeps at most 500 terminal rows; a subscribe past
that deletes the oldest ones.

* feat(notifications): add status() to read whether pushes reach this browser

status() resolves 'on', 'off', 'blocked' or 'unavailable' without asking
anything. On a website it reads the notification permission, this
browser's subscription and the user's opt-in. Inside the desktop a new
pushStatus IPC, advertised as push-status, answers the browser's push
state, 'on' only while the app holds notifications:push.

* feat(notifications): disable() inside the desktop revokes the app's push permission

disable() did nothing inside the desktop. A new disablePush IPC,
advertised as disable-push, revokes the calling app's notifications:push
grant through the same route Settings uses, so its notifications stop
going out as pushes and a later enable() shows the permission dialog
again.

* feat(notifications): allow @ in a notification group, up to 128 characters

A group is an opaque display tag, hashed for the Web Push topic and used
as the notification tag, so it can carry identifiers such as an email
address.

* docs(notifications): say when a page with an older Puter.js holds a push back

* fix(notifications): let disable() win over an enable() still in flight

An enable() whose subscription reached the server after a disable() went
on to record the opt-in, so the next page load turned pushes back on. It
now rejects with push_verification_canceled and records nothing.

* docs(notifications): list push_verification_canceled and reword sentences that led with code

* fix(events): carry KV values through a share handle only at read or write

The delivery re-check now reads the share's level off the live handle along
with whether it still holds. A row on a watch share loses its includeValue for
that delivery, so its events name the key without the value; the owner's own
rows on the handle keep theirs. Decided per delivery, so raising or lowering a
share applies to subscriptions already made, and subscribing with includeValue
at watch is still accepted.

The live-updates recipe now shares at read, which its events need to carry
the score.

* fix(events): retry worker runs Puter could not complete without counting them

Invocations that never reached the handler (transport and dispatcher errors,
unmarked answers, failed or throttled deploys, no configured runtime, a bad
invoke key or missing token) now report an 'unavailable' outcome. So does a
handler that throws after one of its own puter.js calls got a 5xx, 429 or
network error from Puter's API, which the events runtime reports as
'platform-error'. Those runs back off like retriable ones, honor the answer's
retry-after up to the backoff ceiling, and no longer count toward the
consecutive-failure suspension. A run past the invoke timeout still counts.

* test(notifications): cover the pushNotifications resource and status() in the API suites

* test(gui): Allow on a batch that includes notifications:push turns push on

* fix(events): hold a share-handle row's backlog when its share can't be checked

The share-handle delivery check now tells a share that is gone (handle not
live, not the row's holder, identity gone, grant withdrawn) from one it can't
check right now (a lookup that throws, an owner who is suspended). Only the
first settles a queued row as permission_revoked and purges its backlog. The
second delivers nothing, leaves the row in service with its backlog, and sends
it to the back of the sweeper's line, so delivery resumes once the lookup
recovers or the owner is unsuspended. A revocation settle skips such a row and
leaves it to the delivery check. kvSharedRegionAuthorized returns null when
the permission check throws, and that answer is not cached.

* feat(events): update a persistent subscription's context in place

POST /events/subscriptions/update replaces a durable row's context, with the
same validation and 4 KB cap as subscribe, keeping its subId and backlog. The
row's owner region cache is rebuilt from the table and the generation bumped so
every region delivers the new value. It spends the subscribe call budget.

puter.events.update(subId, { context }) wraps it; null clears the context, and
a function handler this client registered for the subscription gets the new
ctx.

* fix(events): serve a fan-out's holders in turns and cap values per delivery

#route now orders an event's candidate rows before any cap applies: the
keyspace owner's rows first, then the other holders taking turns, one row per
holder per round, starting from a random holder each event. The filter
evaluation cap, the delivery cap and the gap markers all count in that order.

Values no longer drop for every delivery once more than 128 rows match or the
filter evaluation stops early. The first 128 deliveries that asked for the
value carry it, in serving order, and later ones carry the key alone.
EVENTS_KV_VALUE_OMIT_MATCHED_SUBSCRIPTIONS becomes
EVENTS_KV_VALUE_DELIVERIES_PER_EVENT. The caps themselves are unchanged.

* fix(email): file inbox mail as a delivery, not the request actor's write

storeInboxMessage and copyIntoInbox now run their filesystem writes with a
platformWrite context flag, and fs event dispatch leaves actingUserId unset
under it. Mail filed into an account's inbox from inside that account's own
request (an app mailing the signed-in user) is delivered with self: false, the
same as external ingress. Handler depth and viaWorker are still carried from the
actor.

* feat(notifications): raise the default daily app message limit to 1000

The per-day default for `app.message` goes from 300 to 1000 per user and
app, matching the daily push cap. The per-minute default stays at 30, and
`notificationAppMessageLimits` still overrides both.

* feat(notifications): look up an unseen notification by group

`notification` gets a nullable `group_key` column, a sha256 hex digest of
the group a row was written with, indexed on (user_id, app_uid, group_key).
`NotificationStore.create` takes a `group`, and `getUnseenByGroup` returns
the newest row an app wrote for a user under that group that is neither
shown nor acknowledged. Rows written before the column exist never match.

`updateValue` takes `onlyUnseen`, which also skips a shown row, and
`refreshCreatedAt`, which dates the row to now.

Migrations: sqlite 0098, mysql_mig_52, postgres_mig_41.

* feat(notifications): replace an app's unseen message in the same group

An `app.message` with a `group` now replaces the app's earlier notification
to the same user with that group while it is neither shown nor
acknowledged. The row keeps its uid and takes the new value and time. With
no group, or once the earlier one was shown or dismissed, it is a new
notification as before.

A replacement spends a per-minute slot but no per-day slot. If the earlier
row is marked between the lookup and the rewrite, the message is written as
a new notification and spends a day slot then.

`app.message` is groupable, so the replacement goes through
`notifyUpdate`. For types kept off the desktop, `notifyUpdate` skips the
desktop wire and dispatches the rewrite again under the same uid, so app
subscribers and receipts see it, and `notif.written` carries `updated`.

The driver's write path takes the recipient id rather than the actor, so
rate limits key on the recipient.

* feat(workers): report the app a worker runs as

workers.create now returns app_uid and app_name for the app its token is bound
to (null for a user-scoped worker), and list/get items carry app_name beside
app_uid from the app rows they already load. The dev center workers table shows
an App column, and the SDK typedefs and docs describe the fields and that a
sandboxed worker's notifications and pushes come from its sandbox app.

* feat(events): refuse handlers with free variables on REST publish

POST /events/handlers/publish and /publishAll now run the SDK's own
scanHandlerSource on each source before publishing and answer 400
events_handler_free_variable, naming the identifier. The backend imports the
module from src/puter-js; the build marks dist/src/puter-js as an ES module
package so the compiled backend can require it. Sources over the per-handler cap
are left to the store's size refusal rather than tokenized.

* docs(events): say Puter-side retries never suspend and stay within backlog caps

* fix(events): redeploy an events worker that answers bad-key

A script deployed before the internal secret rotated keeps the old invoke key,
so every invocation answered bad-key and nothing redeployed it. A bad-key answer
now goes through the same miss handler as a dispatcher miss (the deployer, with
its hourly throttle) and is retried once with the deployed header. A throttled
or failed redeploy stays 'unavailable' and backs off without counting.

* perf(events): check the deploy throttle before reading handler sources

EventsWorkerDeployer now reads the app's handler names and hashes first,
answers no-handlers or stale from those, and checks the hourly deploy throttle
before loading the sources and generating the worker. A throttled retry costs a
hash read and a counter bump instead of a full source read and build, and a
stale request still does not spend the throttle. The generated set is checked
against the script again in case it moved between the two reads.

* fix(workers): move a redeployed worker's row to the app it now runs as

Redeploying an existing worker under a different app minted the new app's
token but left subdomains.app_owner on the old one, so list/get and hot-reload
kept naming and minting for the old app. The redeploy now sets app_owner to the
binding #resolveWorkerAppBinding authorized (null for a user-scoped worker)
through a new SubdomainStore.setAppOwner, since app_owner stays read-only to
update patches.

* fix(workers): revoke the old binding's worker session on a rebind

A redeploy that binds a worker to a different app (or to none) now deletes the
owner's worker sessions under that name other than the one for the new binding,
through the same deleteWorkerSessionsByName path worker deletion uses (which
also announces the revocation and drops access-token grants). SessionStore's
deleteWorkerSessions takes an exceptAppUid filter for it, null meaning the
user-scoped session. The app_owner move and the revocation now both run only
after the new script deployed successfully, so a failed redeploy leaves the
live script, its token and the row on the old binding.

* feat(kv-share): store a notify flag on share handles

kv_share_handles gains a notify column, off by default, so every existing
share reads as not allowing notifications (sqlite 0099, mysql_mig_53,
postgres_mig_42). The store reads it on every row, including the live
handle cache and the holder listing, writes it on mint, and changes it
through update().

* feat(kv-share): let the owner turn notifications on for a share

POST /kv/shares and puter.kv.share() take notify, off by default. Leaving it
out of a re-share keeps the stored value, and turning it on or off counts as a
change, so it spends the mint budget. Turning it on for a holder who now
refuses the owner's shares is refused, like a raise or a new label.

The kv.share.received notice is sent only when a share starts carrying
notify: a new share made with it, or an existing one switched on. Its hourly
bound and the app-less namespace exclusion are unchanged.

The share response, the owner's listing and the holder's listing report
notify.

* feat(kv-share): resolve the other party of a share for a notification

KvShareService.resolveNotifyRecipient(actor, handle) returns the recipient
user id, app uid and handle for a notification one party of a share sends
the other through its handle. The handle must be live with an unsuspended
owner, the caller must be its owner or holder acting from the app whose data
it is, the share must have notify on, the holder's grant must still stand,
and the recipient must not refuse shares from the sender.

An unknown, retired or foreign handle, a caller who is not a party, and a
caller in another app all get subject_does_not_exist. The account itself gets
forbidden, a share without notify kv_share_notify_not_enabled, and a lookup
that fails service_unavailable. Once every check passes it spends one unit
from KV_SHARE_NOTIFY_LIMITS (30/min, 500/day per sender, recipient and app),
peeking both windows first so a refusal spends nothing.

* feat(notifications): hand a replaced notification to listeners again

A replacement now records `replaced_at` (epoch ms) on its row, added to the
same unreleased migrations as `group_key` (the sqlite file is renamed to
0098_notification-groups.sql). Its live event and its fetched copy both
carry `replaced: true` and use that time as `ts`, so the pair versions the
uid identically on both paths.

`onNotification()` dedupes on uid plus that version: an original counts as
version 0 and a replacement as its `ts`. A newer replacement is handed over
again with `replaced: true`, while a copy already handed over, a replay, or
an older state is dropped. Listeners started with `catchUp: false` still
get replacements of what was already there.

* fix(notifications): replace only the newest notification in a group

The group lookup takes the newest row an app wrote under the group and
returns nothing when that row was shown or dismissed, instead of falling
back to an older row that is still unseen.

The docs state the rule as it is: a grouped notification replaces the
newest one in its group unless that was shown or dismissed. Puter.js
marks neither, and the only thing that does is a tap on the push inside
the Puter desktop, which dismisses it.

* feat(notifications): notify the other party of a KV share through its handle

`app.message` takes `to: { handle }`. The driver validates the message,
then `to` must be an object with only a non-empty string `handle`, else
400. `KvShareService.resolveNotifyRecipient` checks the share and spends
the pair limit, and the message is written for the share's other party in
the share's app. The recipient's per-app limits and group replacement
apply to their own rows, and the push is billed to them through
`notif.written`.

A call with `to` answers with `{ uid }` only, so nothing about the
recipient's row reaches the sender. puter.js gets the
`NotificationRecipient` type for `to`.

* docs(notifications): document notifying the other side of a KV share

notify.md covers `to: { handle }`: who can be reached, the errors, the
recipient's limits plus the per-pair share limit, the uid-only return and
that the recipient pays for pushes. push.md and rate-limits-and-quotas.md
point at the share path. The chat notification recipe sends through each
share's handle and keeps the per-member handler only for members who need
to reach each other; the alert recipe says when its handler is the right
fit.

* fix(notifications): reveal nothing to the sender of a notification through a share

A call with `to: { handle }` answers with an empty object, and puter.js
resolves it with no value: a uid would show whether the recipient's last
notification in a group was replaced. `notify()` gets one overload per
form, and `ShareNotificationInput` is exported.

Over the per-pair share limit or the recipient's own limits, such a call
gets the same 429 `too_many_requests` with the same generic message, so the
sender can't tell which limit it hit.

* fix(notifications): serialize grouped app messages per group

Grouped app.message sends take a Redis lock on (recipient, app, group) around the lookup and the replace-or-insert, so concurrent sends to an empty group write one row and spend one day slot. A lock still busy after the wait, or an unreachable Redis, sends unlocked as before. Sends without a group take no lock.

* fix(events): attribute Puter API failures to the invocation that saw them

The events worker runtime counted Puter API failures on the cached per-token client, so a concurrent invocation on the same token could answer platform-error for its own unrelated throw. A cached client now serves one invocation at a time and records failures and retry-after on that invocation; an invocation overlapping another on the same token gets its own client, and idle clients stay cached per token.

* fix(events): drop KV values from a queued backlog once its share is at watch

A queue delivery is stored with its value. Every hand-out of a stored
entry (backlog drain to a socket or another region, the worker, and a
declined delivery passed to the worker) now re-reads the share's level
and removes the value at watch. A share that can't be checked still
holds the backlog.

* fix(events): return notifications replaced behind the fetch cursor

The notification fetch cursor now holds the last row id and the last
(replaced_at, id) it returned. A read from it merges two indexed reads:
rows inserted after the id, and rows up to the id replaced after that
mark. A replacement made while a client was away therefore comes back
on its next catch-up. Adds idx_notification_replaced on (user_id,
audience, app_uid, replaced_at).
2026-10-31 12:21:18 -07:00
Daniel Salazar 4851df716c fix(kv): KV share lifecycle hardening (#4429)
* fix(kv): harden the KV share lifecycle

- Retire a deleted app's live shares before an origin app is registered
  again under the same uid. The retire runs before the app row exists and
  throws on failure, so a failed launch leaves no row and the next one
  retries.
- Record deleted-app retires and uninstall cleanups (retire, then purge) as
  reserved items before they run. A lease keeps one node on a job; a failed
  run raises an alarm and backs off, and a run that hits its time budget or
  is cut short by a restart resumes on the next check. Each job is bounded
  to handles made before it was recorded.
- Uninstalling with deleteData records the job and returns; the retire and
  purge run off the request. The purge checks whether the app was
  authorized again before deleting each page, so data written after that
  is kept.
- Take a per-region lock when minting and re-check for a live handle under
  it, so concurrent shares of one region make one handle.
- Retire the handles an account holds before the account is deleted, so the
  owners' subscriptions through them end instead of staying active.

* fix(events): bound share-handle session rows by the per-connection cap

The count across a holder's connections also counted connections that
had dropped but weren't reaped yet (a crashed node, a deploy, a network
blip the server hasn't noticed), so resubscribes after a reconnect could
be refused with `events_subscription_limit`. Rows made through a share
handle now count toward the 50-per-connection cap like any other, and
with the open-connections cap per account that bounds what one holder
can put in the owner's keyspace.

The handle index stays for ending rows on retire. Rows are added to it
with the row itself and taken out when the row is dropped, and members
whose row is gone are cleared when a retire reads it.

* fix(events): keep a retire's durable settle running if ending sessions fails

Ending a retired handle's session subscriptions reads Redis. A failure
there is now logged and the durable suspension that follows still runs,
so those rows' backlogs are still purged.

* perf(events): ask once per handle when a refresh screens rows, and share handle reads

A socket refresh asks whether a share handle still stands once per
handle, holder and app, rather than once per row: rows through one
handle share its grant. Concurrent cache misses on one handle share a
single read; a forget drops the shared read so later callers start a
fresh one.

* fix(permissions): chunk the exact-match subtree delete

The holder-scoped subtree delete names every matched grant, so a large
subtree could exceed an engine's limit on bound parameters. It now
deletes 500 grants per statement.

* fix(kv): give each holder a share of a namespace's handle budget

One holder could spend a namespace's whole handle budget and starve the
others. Each holder now gets at most 3,750 of the namespace's 15,000
units per 10 seconds, and the owner's own calls through a handle spend
none. The holder's budget is checked before the grant lookups, so
throttled retries cost none; the namespace's is only told to callers the
grant admits. Multi-unit charges use a new counted claim that keeps two
counters per key instead of a sorted-set member per unit.

* fix(kv): refuse handles older than their app and claim cleanup jobs globally

- A share handle created before its app row reads as gone: in the KV
  driver, in the handle cache, on the events subscribe path, and in the
  delegate implicator. An origin app registered again under a deleted
  app's uid can't reach its predecessor's shares, whatever created the row.
  The app-token launch no longer retires inline; it records a background
  retire and never waits on it or fails because of it. The clock-skew rule
  is shared with app sessions.
- Cleanup job records now carry their lease, attempts and retry time, and
  a node claims a job by compare-and-set on its record. A job stays with
  the deployment that recorded or last ran it, and another one adopts it
  only after it has gone unclaimed well past due.
- The job check pages through records from where the last look stopped, so
  waiting records don't hide due ones. After 10 failed runs a job stops
  retrying with a warning alarm, and its record expires after 30 days.
- A mint that waits out the region lock looks for the share again and
  returns it, or goes ahead, instead of failing with too_many_requests.
- The record-failure alarm dedups; an unfinished run is a log line.

* fix(kv): hide shares older than their app from re-shares and listings

After an app is registered again under a deleted app's uid, its
predecessor's live handles stayed visible: sharing the same region again
handed back the old, dead handle, and both sides' listings showed the old
handles and labels. findLive and both listing queries now leave out a live
handle created before its app row, the same rule handle resolution uses,
so a re-share makes a new handle and the listings, totals included, skip
the old ones until their retire lands.

A new share also retires, grants and all, the handles its owner gave the
same holder for the earlier app first, so withdrawing those later can't
take the new grant with it.

* fix(events): keep a loose cap on subscriptions through one share handle

Without a count across connections, a holder could put up to 20,000 rows
in the owner's keyspace (50 per connection, 400 connections), each one
evaluated on every owner write. The cap is back at 1,000 per holder per
handle, far above what connections that dropped without being reaped
leave behind. The index is only read row by row once it's past the cap.

* fix(kv): harden the share cleanup jobs

- Job records are keyed by domain, so a deployment that shares the table
  but not the database or broadcast (staging beside production) never
  runs another's jobs.
- A job whose record can't be written runs unrecorded instead of not at
  all, and a failed check for live handles records the job anyway so it
  retries. Failures that used to be swallowed are logged.
- Recording a job again keeps its attempts and backoff; one that gave up
  gets one more attempt.
- A system retire gives back the grant of a live share it doesn't cover
  inside a withdrawn region, and a share made under a reused uid grants
  again once its row stands, so a concurrent retire can't leave a live
  handle without its grant.

* fix(kv): don't cache a live handle a retire overtook during its app lookup

The live read decided whether it could cache before awaiting the app
lookup, so a retire's forget that landed during that lookup was
overwritten with the handle as live for up to 60 s. It's now checked
after the lookup.

* chore(kv): word the job key comment for any install
2026-10-31 12:12:29 -07:00
Daniel Salazar faa8cd99dd fix(kv): per-holder share of the KV handle budget (#4433)
* fix(permissions): chunk the exact-match subtree delete

The holder-scoped subtree delete names every matched grant, so a large
subtree could exceed an engine's limit on bound parameters. It now
deletes 500 grants per statement.

* fix(kv): give each holder a share of a namespace's handle budget

One holder could spend a namespace's whole handle budget and starve the
others. Each holder now gets at most 3,750 of the namespace's 15,000
units per 10 seconds, and the owner's own calls through a handle spend
none. The holder's budget is checked before the grant lookups, so
throttled retries cost none; the namespace's is only told to callers the
grant admits. Multi-unit charges use a new counted claim that keeps two
counters per key instead of a sorted-set member per unit.
2026-10-31 12:10:39 -07:00
Daniel Salazar 279bb8beb6 fix(events): KV share handle subscription follow-ups (#4434)
* fix(events): bound share-handle session rows by the per-connection cap

The count across a holder's connections also counted connections that
had dropped but weren't reaped yet (a crashed node, a deploy, a network
blip the server hasn't noticed), so resubscribes after a reconnect could
be refused with `events_subscription_limit`. Rows made through a share
handle now count toward the 50-per-connection cap like any other, and
with the open-connections cap per account that bounds what one holder
can put in the owner's keyspace.

The handle index stays for ending rows on retire. Rows are added to it
with the row itself and taken out when the row is dropped, and members
whose row is gone are cleared when a retire reads it.

* fix(events): keep a retire's durable settle running if ending sessions fails

Ending a retired handle's session subscriptions reads Redis. A failure
there is now logged and the durable suspension that follows still runs,
so those rows' backlogs are still purged.

* perf(events): ask once per handle when a refresh screens rows, and share handle reads

A socket refresh asks whether a share handle still stands once per
handle, holder and app, rather than once per row: rows through one
handle share its grant. Concurrent cache misses on one handle share a
single read; a forget drops the shared read so later callers start a
fresh one.

* fix(events): keep a loose cap on subscriptions through one share handle

Without a count across connections, a holder could put up to 20,000 rows
in the owner's keyspace (50 per connection, 400 connections), each one
evaluated on every owner write. The cap is back at 1,000 per holder per
handle, far above what connections that dropped without being reaped
leave behind. The index is only read row by row once it's past the cap.
2026-10-31 12:10:34 -07:00
Daniel Salazar 52aedde3c5 fix(kv): KV share authz and load hardening (#4430)
* fix(permissions): delete user-to-user grants by their exact spelling

MySQL compares the permission column case-insensitively, and LIKE folds
case on MySQL and SQLite, so withdrawing a grant could also remove one
spelled differently, which checks and the flat view treat as distinct.
The single, batch and subtree deletes for one holder now match the
permission byte for byte.

* fix(auth): refuse key-value share grants on /auth/revoke-user-user

Withdrawing a share's grant there left its handle live and listed. The
route now refuses kv-share permissions and points to puter.kv.unshare().

* perf(kv): ask whether a delegate app is authorized once per check

The app-delegate implicator re-read the app's authorization for every
option a refused check offered, one read per ancestor of a deep key.

* fix(puter-js): default a kvShare consent request to read

puter.kv.share() creates read shares by default, so a consent asked for
with the defaults (watch) didn't cover the share made with the defaults.

* fix(kv): treat a private entry as missing for a share holder's writes

Through a handle, a private entry reads as absent, but writes to it were
refused with an error naming it as private, and onlyIfAbsent answered
false. Deleting one through a handle now answers as deleting a missing
key does and leaves the entry in place. Any other write is refused with
kv_handle_access_denied, the same error as a call the share's level
doesn't allow, onlyIfAbsent included. Cross-app access is unchanged.

* fix(kv): cap handle traffic per owner namespace

Every call through a share handle lands on its owner's namespace, and
per-holder limits put no bound on what all holders together could send
there. Handle calls into one namespace now share a budget of 6,000 units
per 10 seconds, counted from the storage reads and writes each call
consumes (at least one), and are refused with too_many_requests past it.
2026-10-31 01:01:53 -07:00
Daniel Salazar 7f832ede43 fix(events): KV share handle subscription hardening (#4428)
* fix(events): keep a share's prefix out of handle subscribe errors and limits

A `kv:<handle>:<key>` pattern is now checked against the 256-character
limit as the holder wrote it, before it's composed onto the shared
prefix. The composed pattern is compiled with room for the longest
prefix a grant can carry, at subscribe and at dispatch, so neither the
limit nor its error depends on the prefix.

A handle subscription's `match` is read off the holder's own subject
instead of the stored composed pattern, so whether it's `null` no
longer depends on how deep or long the prefix is.

* fix(events): cap and end session subscriptions made through a share handle

Session subscriptions made through a share handle are indexed under the
handle in the owner's keyspace. A holder may keep 100 of them through
one handle across all their connections; past that, subscribe fails
with `events_subscription_limit`. The owner's own subscriptions through
the handle don't count.

Retiring a handle now ends its session subscriptions in the same
region and sends `events.ended` with `permission_revoked`, alongside
the existing durable settle. A socket refresh no longer renews a row
whose handle was retired or whose grant is gone: it ends it the same
way, which is how a retire made elsewhere reaches these rows.

* fix(kv): read a forgotten share handle from the primary

After a handle or its owner is forgotten, reads of that handle go to
the primary for the next minute, so a replica that hasn't caught up
can't put the old row back in the cache. A read that a forget lands on
while it's in flight is answered but not cached.

* fix(events): read a socket ref's subId after the last `|`

A KV token carries the subscribed key, which can contain `|`, so
splitting the ref at the second one cut the token short. A subscription
on such a key couldn't be unsubscribed, and a reap left its row behind
in the owner's keyspace. SubIds never contain `|`, so the ref is split
at the last one.
2026-10-31 00:58:42 -07:00
Daniel Salazar 58275ccd52 ci(puterjs): check out the coverage base leg from this repo (#4431)
The base leg checked out the base branch from the PR's head repo. For a
fork PR that is the fork's copy of main, which can predate the current
Node version and fail npm ci.
2026-10-31 00:42:37 -07:00
Daniel Salazar 11114c80d6 fix(auth): fail closed on otp-login token reuse check (#4421)
/login/otp and /login/recovery-code marked the otp-login token used
through a rate-limit check that admits on a store error, so while the
store was unavailable a captured token could complete another sign-in.
The check now fails closed.

Also drops the check-translations scripts and the AGENTS.md step that
runs them; the script they call was removed in #2873.
2026-10-31 00:26:56 -07:00
Daniel Salazar 5d907d18c9 fix(socket): bound acks a receipt ask leaves on sockets that never answer (#4422)
An acked broadcast leaves its callback in socket.io's per-socket ack map on every socket it reaches, and nodes other than the asker free it only when that socket answers or disconnects. Each socket's map is now a BoundedTtlMap capped at 64 entries and 60 s, so a socket that never answers holds a bounded number and a late answer is ignored. BoundedTtlMap drops expired entries on set so its TTL frees memory too.
2026-10-31 00:26:44 -07:00
Daniel Salazar 2d5e037599 fix(ai): bill realtime responses the upstream drops before reporting usage (#4423)
When the upstream realtime connection closes, for any reason, with a
started response that never sent response.done, the relay now records that
response at its credit hold (8k audio in + 4,096 audio out at the session
model's rates, through the AI cost factor) instead of releasing the hold
uncharged. The hold is released once the charge is recorded, and a warning
alarm is still raised.

Client conversation.item.retrieve events and MCP tools in session.update or
response.create are refused with an invalid_event error and a 1008 close.
2026-10-30 23:48:58 -07:00
8d3a1ccc2b feat(notifications): remove a notification from the dashboard history (#4412)
Each notification in the dashboard list gets a remove (✕) control next to
mark-as-read. It calls the new POST /notif/delete, which deletes the user's
own row and sends `notif.removed` to their other tabs so the entry and any
toast for it disappear everywhere.

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 22:52:27 -07:00
Daniel Salazar a1fe14cbd5 fix(ai): bound large AI request bodies per user and to chat (#4420)
Follow-up to #4408.

The 503 returned when a process holds too many large bodies is load
shedding, so it records an info alarm instead of a critical one.

One user can hold at most 128 MiB of large bodies at once; past that the
request gets 429 with Retry-After, so a single account can't take a
process's whole budget, e.g. with slow uploads.

Through /drivers/call only puter-chat-completion takes a body over 50 MiB;
other drivers keep the default limit and get 413. The check reads the
parsed body's interface, since route matching ignores case and trailing
slashes.
2026-10-30 21:16:53 -07:00
Filip KujundžićandDaniel Salazar a195605003 fix(ai): accept 128 MiB Anthropic message requests (#4408)
* fix(ai): accept larger Anthropic message requests

* fix(ai): accept 128 MiB bodies on /drivers/call and OpenAI routes

Extend the 128 MiB JSON limit to /drivers/call (puter.ai.*) and the
OpenAI-compatible routes, alongside the Anthropic ones.

A body over the default 50 MiB is read only after a header or cookie
credential resolves; otherwise the global parser rejects it with 413, so
unauthenticated callers can't make the server buffer and parse 128 MiB.
Large bodies skip the rawBody copy, and a per-process budget caps how many
bytes of them are held at once, returning 503 with Retry-After past it.

The prompt token estimate no longer splits text on whitespace, which took
seconds and allocated a string per word on large prompts. It counts
whitespace runs in a loop, scans at most 1 MiB per prompt and extrapolates
the measured density past that.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 20:52:00 -07:00
Daniel Salazar 1e39936d12 fix: custom domains (#4419) 2026-10-30 20:39:11 -07:00
John CostaandDaniel Salazar 1390a87280 fix(docker): hoist sharp so the backend can load it from dist (#4400)
The backend runs from dist/src/backend, so Node resolves modules from the
root node_modules, never src/backend/node_modules. Since sharp ^0.35.5
(#4081) only the backend asked for that version, so npm nested it under
src/backend while the root copy stayed miniflare's dev-only 0.35.4, which
the image doesn't ship. The 26.10.1 image then crash-loops on start with
"Cannot find module 'sharp'" from the Replicate image provider.

Declaring sharp ^0.35.5 at the root hoists the backend's version, the
same way other backend runtime deps are reachable from dist.

Fixes #4394

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 20:00:09 -07:00
dependabot[bot] 6d47db9358 chore(deps): bump nodemailer from 9.1.1 to 10.1.4 (#4386)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.1.1 to 10.1.4.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.1.1...v10.1.4)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 10.1.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 19:39:17 -07:00
mahilsaily 2e57214bb3 fix: update outdated HOME_URL in /etc/os-release (closes #4329) (#4392) 2026-10-30 19:37:22 -07:00
spacedandDevin AI 7ac2f2e247 feat(ai-chat): let requests opt out of provider fallback (#4343)
* ai-chat: fallback: false fails on the first route instead of rerouting

A chat request whose first provider fails is retried on up to two other
providers that serve the same model, and the caller pays for whichever
one answers. Callers who want one specific route, or who want to handle
a vendor failure themselves, had no way to turn that off.

Passing fallback: false (puter.js option, /drivers/call arg, and the
OpenAI-compat and Anthropic-compat request bodies) makes the first
attempt's failure final: the route is still marked unhealthy, and the
error is classified the same way an exhausted chain would be.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ai-chat: cover fallback: false in the SDK suite

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ai-chat: fallback: false rejects a pinned provider with no route for the model

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: plainer wording for the fallback option

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-30 19:35:45 -07:00
dafiliandDaniel Salazar 0982e696a5 docs(install): explain how to install Docker instead of stopping at a dead end (#4411)
* docs(install): explain how to install Docker instead of a dead end

The installer stopped with 'missing required command: docker' and nothing
else, which left self-hosters on a clean machine with no next step. Since
Puter self-hosting is a Docker Compose stack, that check is also the very
first thing a new user hits.

Print the official install links for macOS/Windows and Linux plus the
'docker compose version' sanity check, and cover the same ground in
doc/self-hosting.md under Requirements and Troubleshooting.

The installer still does not install Docker itself — per the maintainer's
note on #4346, pulling a remote script into a system package manager is
too intrusive. This only makes the failure actionable.

* fix: remove extra sh stuff already merged

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 19:33:01 -07:00
Juan Fernando Castro fea89c91b8 feat(puter-js)!: remove notifications.registerDevice (#4409)
* docs(notifications): registerDevice takes ios or android

* feat(puter-js)!: remove notifications.registerDevice
2026-10-30 19:31:30 -07:00
Rasad Regmi 489a7c0129 fix(install): improve Docker missing error with platform-specific instructions (#4384) 2026-10-30 19:29:52 -07:00
Juan Fernando Castro e5ba469ffc fix(notifications): keep app messages in their app, off the desktop (#4410)
* fix(notifications): keep app messages in their app, off the desktop

An app's notifications now reach only that app: its live feed on
notif:app-user and, through the app's own sockets, receipts and pushes.
The desktop's listing, unread replay, live wire and notif:app-user
subscription leave them out, the same way key-value share notices
already are, which covers rows stored before this change too. Puter's
own notifications and shares still show on the desktop.

The desktop and its notifications panel also ignore an app message
they are handed, and the desktop no longer notes one as received.
The notification docs now say where an app's notifications show.

* docs(notifications): say an app's notifications reach the app, not the panel

* test(notifications): wait for the unread replay instead of sleeping past it

* fix(gui): show notifications a frozen tab missed once it reconnects

A tab that was frozen or asleep keeps its socket registered until it
resumes, so a notification created meanwhile counts as shown and no
unread replay carries it: the tab never showed it, not even after a
reload.

On a socket reconnect the desktop and the dashboard now read the
undismissed notifications created since the page loaded and raise the
ones that page has not shown or seen dismissed, once. Every unread
replay is taken too, not only the first, and no longer only while the
events feed is down; a per-page log keeps either from raising anything
twice. App messages stay out.

* fix(gui): catch up after a reconnect without the page's clock

The reconnect catch-up compared each row's server time with the page's
own load time, so a page whose clock ran behind raised, on its first
reconnect, every undismissed notification from before it loaded.

A page now notes the undismissed notifications it knew at load (the
desktop from one listing, the dashboard from its first) and a reconnect
raises only those it neither knew then nor has raised since. The
dashboard reads them from the listing its reconnect refresh already
takes. The per-page log is bounded, and a caught-up notification shows
its title as the body when it has no text, as the replay does.

* style(gui): keep the reconnect comments to one line
2026-10-30 19:24:38 -07:00
Neal Shah 2a3c181657 DNS ui changes (#4417)
* DNS ui changes
2026-10-30 21:16:25 -04:00
Daniel Salazar 4881264c88 feat(ai): realtime reconnect notice, context restore and retry hardening (#4415)
* test(ai): end every realtime relay test's upstream before the next

The fake upstream now answers response.create with response.created first,
as OpenAI does, so closing those sessions no longer waits out the settle
window.

* feat(ai): ask realtime clients to reconnect before a restart

Open realtime sessions get a puter.reconnect event when the drain starts,
with close_in_ms set to the drain window, and new sessions are still refused
until the 1012 close at shutdown. A host extension can announce a restart
early by emitting serverRestartAnnounced; that only sends the notice, so a
false alarm leaves the server working normally. A session gets the notice at
most once a minute.

onServerDrainStart now receives the drain duration.

* fix(ai): retry realtime reconnects for about a minute and wait out offline browsers

Five attempts gave up after about 15 seconds, shorter than a server restart.
The conversation makes ten attempts with the backoff capped at 10 seconds,
and while the browser reports being offline it waits for the online event,
up to two minutes per reconnect, instead of spending attempts. close() ends
either wait and removes its listener.

* feat(ai): restore recent conversation context after a realtime reconnect

The provider starts a fresh session on every reconnect, so the conversation
lost all earlier items. The SDK now records finished items it can recreate
from the provider's own events: user text, user speech with a transcript,
assistant replies as text, and function calls with their outputs. After the
new session acknowledges its settings, and before ready fires, it adds back
the newest of them, up to 40 items and 48 KiB, which stays under the relay's
upload burst.

Assistant replies are cut to what the user saw or heard: the deltas shown
before an interruption, and the share of the transcript matching the audio
played before a truncation or a drop. Deleted items, unfinished replies,
images, speech without a transcript and incomplete function calls are left
out, as are function outputs whose call falls outside the window. Restored
copies take over their new item IDs. A refused item is reported once as
context_restore_failed without failing the reconnect.

* feat(ai): move a realtime conversation to a new session on the restart notice

On puter.reconnect the conversation waits until it is idle: no reply
requested, generating or playing, no user turn being spoken or transcribed,
no function call awaiting its output and no acknowledgment pending. Then it
reconnects through the usual path, emitting reconnecting (code 1012) and
ready with reconnected: true, without an error. Until then the session keeps
working, and if the server closes it first the normal 1012 reconnect takes
over. close() at any point stops the wait.

* docs(ai): realtime restart notice, context restore and reconnect limits

* test(ai): realtime reconnects against the local server

The mock provider answers with item events and whole replies like OpenAI.
New cases cover a dropped connection (cut through a TCP proxy so the client
sees 1006), a host-announced restart, and a draining server refusing the
reconnect; the first two check the provider receives the restored items
before the next turn. The browser media test waits for captured audio and
the reply instead of a fixed sleep. The live suite gains a tool and text
context round trip across a restart notice.

* ci: run the live realtime conversation tests with the AI provider suite

They need the OpenAI key, which only this workflow has, and skip themselves
without it.

* fix(ai): open the replacement before leaving a realtime session on the restart notice

The handover closed the current session and then reconnected, so every
notice cost a second or two in which sends failed and microphone audio was
dropped. On a single server that refuses new sessions while it drains, the
retries ran out during the drain and the conversation closed sooner than if
it had stayed. The conversation now opens a replacement session as soon as
the notice arrives and switches only once it is open and the conversation is
idle. A refused replacement is retried with backoff while the current
session keeps working; when the server closes it first, the held replacement
is used right away.

The idle check also missed some in-progress work:
- a reply created while the user was still speaking ended the user's turn;
  speech now lasts until its audio is committed or cleared, and a committed
  turn waits for the reply server turn detection creates
- only the newest function call was checked; every finished call since the
  last user message needs its output, and calls cut off by a cancellation
  don't count
- native session.update and conversation.item.create events the provider
  hadn't acknowledged could be lost; the switch waits for them, up to the
  acknowledgment timeout for items
- only the newest transcription was waited on; every committed item is
- clearing playback didn't recheck the switch

Restored items stay internal: the new session's copies of them no longer
reach listeners. At a reconnect, every queued item the user didn't hear is
cut, not just the one playing. A notice from a failed reconnect attempt is
dropped.

* test(ai): realtime replay after split settings, and a draining server refusing the replacement

The relay accepts the context replay right after two full-size settings
updates. Against the local server, a draining relay refuses the replacement
session while the current one keeps answering.

* ci: build the SDK in the live realtime step so it stands on its own

* fix(ai): a failed realtime reconnect attempt no longer poisons the next

An attempt that dropped after sending its session.update but before the
acknowledgment left that event ID queued, so the next attempt's
acknowledgment was matched to it and timed out, and so on until the
conversation closed. Turn state is now reset after each failed attempt.

* fix(ai): bound the realtime handover's waits and stop retries that can't succeed

- The first replacement opens within a randomized 3 seconds of the notice,
  so sessions notified together don't all open at once.
- A replacement refused with a code retrying can't fix, such as
  insufficient_funds, ends the attempts; the server's close still falls back
  to the usual reconnect.
- A held replacement that gets an error or its own server's notice is closed
  and replaced.
- A held replacement used after a drop gets a 3 second acknowledgment
  timeout, since its socket may have died with the network.
- close() during the switch closes the held replacement too.
- The reply to a committed turn and each transcription hold the switch for
  at most 10 and 30 seconds, like app items already did, and an app item is
  released only by an added item of its type, role and call ID.

* docs(ai): the realtime restart notice isn't guaranteed before every restart
2026-10-30 18:06:54 -07:00
c7c3f21e4b feat(ai): puter.ai.convo - Preliminary Realtime API implementation (#4401)
* feat(ai): relay OpenAI's Realtime API as puter.ai.assistant

Adds a WebSocket relay at /puterai/openai/v1/realtime on the api host,
one upstream OpenAI connection per client, frames passed through 1:1.
The client socket is accepted before auth so socket.io's 1 s
unhandled-upgrade timeout can't kill slow handshakes; failures arrive as
OpenAI-shaped error events and a 1008 close. Usage is metered per
response.done at the model's token rates, sessions are capped per user
(realtime.maxConnectionsPerUser), and each node logs counters,
event-loop delay and memory every 30 s.

puter.js gets puter.ai.assistant.connect(), returning a session with
send/on/off/close over the same OpenAI events.

realtime.upstreamUrl points the relay at a mock for load tests;
tools/realtime-stress has the mock upstream, a load generator with a
JSON report, and an agent runbook. The relay README lists its quirks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ai): backpressure and upload budget on the realtime relay

Neither direction applied backpressure: a client that stopped reading
grew a queue up to 64 MiB, and a client streaming 16 MiB frames was read
and forwarded as fast as it could send. A few such sockets pinned the
event loop and pushed RSS past 1 GB.

Each side now stops reading the other once 1 MiB is queued toward it,
and a client that keeps the upstream paused for 30 s is closed. Client
uploads are budgeted per session (1 MiB/s sustained, 16 MiB burst,
realtime.maxClientBytesPerSecond), pre-open buffering pauses reads at
1 MiB instead of rejecting at 32 MiB, and clients get a 30 s ping so
half-open sockets release their upstream and slot. Upstream frames use
a zero mask and skip UTF-8 validation, which the relay never needed.

The stats line gains elu. tools/realtime-stress gains abuse.mjs and a
runbook phase for event-loop abuse.

* fix(ai): drain, transcription metering and credential re-check on the realtime relay

Shutdown now drains: a new onServerDrainStart hook (run when SIGTERM
starts the drain, before the listener closes) makes the relay refuse new
sessions with 1012, open ones keep running through the drain window, and
at onServerPrepareShutdown each gets up to 5 s to finish an in-flight
response before closing with 1012 so the client reconnects elsewhere.

Input transcription is metered. The relay reads the type from each text
frame's first 96 bytes and, after forwarding, parses response.done,
input_audio_transcription.completed and session.created/updated (for the
transcription model) on setImmediate, so metering never holds up frames.
Unknown transcription models bill at the highest known rate.

Credentials are re-checked: a successful auth is cached in the kv.js
singleton for 60 s under a hash of the token, and a 15 s sweep
re-authenticates sessions whose entry expired, closing revoked or
suspended ones with invalid_api_key. Lookup errors keep the session and
retry on the next sweep.

* docs(ai): API shape proposal for puter.ai.conversation() across OpenAI, Gemini Live and xAI

* feat(ai): add composable realtime conversations with bounded transport

* fix(ai): realtime billing, subscription gate and conversation reconnects

Relay:
- Cancel and bill a response still generating when the client leaves,
  instead of cutting the upstream before response.done.
- Record realtime usage through the AI cost-factor hook.
- Drop the upstream byte and message caps; replies arrive faster than real
  time and tripping them closed sessions and skipped billing.
- Price usage as it arrives, before any forwarding decision, so a closing
  session never drops a usage event.
- Bill the model the session reports and refuse a session.update that
  changes the model.
- Require a paid plan.

SDK (puter.ai.convo):
- Reconnect after a dropped connection with the same settings, keeping
  listeners and media; emits reconnecting and ready { reconnected }.
- Errors reject only the operation they name; a failed send rejects only
  its own request.
- configure() sends only the given settings and settles on its own ack;
  commitAudio follows turn detection set natively.
- interrupt() right after sendText cancels the reply once it starts.
- Microphone: capture at the device rate, drop chunks instead of stopping
  on stalls or a congested socket, no timeout on the permission prompt,
  permission_denied on refusal.
- Speaker: 300 s playback bound without interrupting, played position
  accounts for output latency and gaps between chunks.
- Audio control events keep headroom over queued audio.
- Upgrade prompt on subscription_required / insufficient_funds.

Tests that need the built SDK and Chromium move to the puter.js suite.

* chore: remove stray node_modules symlinks

* feat(ai): retire assistant API and document realtime beta

* test(ai): wait for the matching realtime billing batch

* test(kv): exclude health probes from warm read measurements

* Revert "test(kv): exclude health probes from warm read measurements"

This reverts commit 631003f755.

* fix(ai): upstream message size, null usage, team 2FA and reconnect settings for realtime

Relay:
- Upstream messages may be up to 4 MiB; client messages stay at 64 KiB.
  An OpenAI event above 64 KiB (an echoed item near the client limit, a
  long reply, a retrieved audio item) closed the upstream before in-flight
  responses reported usage.
- An upstream that closes with a response still in flight raises a
  warning alarm.
- A null usage counts as none instead of closing the session.
- Seats whose team requires 2FA need it for realtime too, at connect and
  on the credential recheck.

SDK (puter.ai.convo):
- A reconnect reapplies every acknowledged setting, native session.update
  events included, split to fit the message limit.
- connect() during a reconnect settles with it.
- Reconnect delays are jittered, and closing cancels a pending delay.

The credential-recheck and browser worklet tests now fail when the
behavior they cover breaks.

* feat(ai): credit holds, attempt limits and late-response billing for realtime (#4407)

Relay:
- Each response holds 8,000 audio input and 4,096 audio output tokens of
  the account's budget, at the session model's rates, from
  response.created until its usage is recorded. A session opens only if
  one hold fits.
- Session attempts are limited to 60 per minute per credential, checked
  before authentication.
- A response the client requested that starts after the client leaves is
  cancelled and billed.
- Input transcription is priced at the costlier of the transcription
  models in force at commit and at completion.
- rate_limits.updated isn't forwarded, and provider organization IDs are
  removed from error messages.

SDK (puter.ai.convo):
- microphone.stop() during the permission prompt rejects the pending
  start() with session_closed.
- speaker.start() rejects with { code, message } for browser exceptions.
- connect() waits 20 seconds for the session, past the server's 15-second
  setup.
- Convenience events fill IDs the provider leaves out; types cover
  turn.done, error, and client events for send().

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-30 14:50:15 -07:00
Daniel Salazar 7401fe0b09 fix(notifications): receipts report what the user saw, sticky push opt-in, safer app pushes (#4413)
* fix(socket): count sockets that never acknowledge instead of asking them

askWithAck sent an acked broadcast to sockets outside the ack room only to
learn whether any were there. Those sockets never answer, so every node held
the ack callback until they disconnected. Each node now answers a server-side
count of them instead; a node that does not answer in time still reads as
holding one.

* feat(notifications): read focus and own display from receipt answers

A receipt answer may carry focused and showsOwn beside visibilityState; both
pass through to notif.delivery.settled when they are booleans. An answer with
received: false is not a responder, and an ask answered only that way settles
as unreceived.

* fix(notifications): name an app message's bad field before the size cap

An app message over the 16 KB value cap answered notification_value_too_large
even when one field was what broke a rule of its own. The fields are checked
first now, so the caller gets bad_request naming the field; the total cap still
applies after. Character counts skip the code point walk on strings that can't
fit.

* fix(push-sw): match the server's tags, keep taps, and show pages' notifications

- A push is tagged with the server's group, `<app uid>:<group>` or the uid, and
  alerts again when that replaces one showing. Handlers get the app's own group
  and the send time in ms from `ts`.
- The app's icon stands in when the notification has none.
- Default and relative tap URLs resolve against the worker's scope, still only
  on its origin.
- A tap posted to an open window waits briefly for the page to take it; without
  a click handler there, the window is taken to the URL, or one is opened.
- A page that got a notification live while away can ask the worker to show it;
  the worker holds it back while another window of the origin has focus.

* fix(notifications): answer every receipt ask

A page now answers each socket that asks about a notification, not only the
first, and answers { received: false } when the notification doesn't arrive
within the wait or the page holds too many asks, so the server's nodes can
let go of them. A received answer carries focused and showsOwn beside
visibilityState.

* fix(notifications): keep web push to users who turned it on, and show what a push would

Subscription:
- onPush() renews the subscription on load only for a user who turned push on
  on this origin with enable(); enable() records that, and disable(), sign-out
  and push_revoked forget it. One renewal runs per page and worker, however many
  onPush() calls there are, and push_disabled, push_revoked, push_unavailable
  and too_many_requests stay quiet there.
- puter.auth.signOut() on a website removes this browser's subscription with
  the leaving user's token and forgets the opt-in and kept taps, without
  holding up the sign-out.
- A subscription another service on the site made is left alone and rejected
  as push_subscription_conflict; one made with Puter's earlier key is replaced.
- A subscription refused with push_endpoint_revoked is replaced once, like
  push_subscription_invalid.
- Another worker holding the scope rejects push_worker_conflict, naming
  importScripts and the serviceWorker option, instead of waiting out the
  confirmation push.
- enable() rejects user_gesture_required outside a click instead of prompting,
  and a refused browser subscribe rejects push_subscribe_failed.

Live side:
- A page that is hidden or unfocused shows every app notification it receives,
  with or without a handler for its channel, through push-sw.js so a focused
  window of the app holds it back. It uses the server's tag, the app's icon as
  a fallback, the generic app line for sensitive ones, and the registration of
  its own worker.
- Page handlers get 5 seconds, as in the worker (push_handler_timeout).
- A feed stopped by a sign-out starts again on the next sign-in.
- onPush() still delivers live when IndexedDB is unavailable.
- onPush() rejects invalid_handler for a handler the worker could not rebuild
  from its source (methods, bound and built-in functions).
- Saving and removing a handler's source for the worker is ordered per key, so
  a quick unmount and remount keeps it.
- A tap posted by push-sw.js is acknowledged only when a click handler takes it.

* fix(gui): show app pushes under the app's name and Puter's icon

- On Puter's own origin, an app's push takes the app's name as its title, with
  the app's title and text in the body, and Puter's copy of the app's icon; an
  icon the app chose is never shown. A sensitive one keeps its generic line.
- A push's ts is its timestamp, and a newer push in a group alerts again.
- Re-registering this browser counts only apps that push to Puter's devices
  (puterDevices), so a website's Web Push confirmation alone never registers
  it.

* fix(gui): free the push dialog's button when registering fails late

A registration that failed after the dialog's deadline left the button
spinning and disabled. It now clears the attempt, settles on a state the
browser moved to meanwhile, or offers a retry.

* fix(permission): a user's own app grant clears a narrower via on the row

An app grant that already existed was skipped, so a row another path recorded
with extra.via kept it after the user approved the same permission in Puter.
A grant without via now rewrites such a row without it, keeping its other
extras; a grant with via still never touches an existing row.

* refactor(notifications): export the notification text cleaner

Extensions that word notifications can use the same cleaning as app messages.

* docs(notifications): describe when pushes go out, taps, worker rules and enable() errors

Pushes go out when no open page answers that it's visible and focused or
shows the notification itself. Documents the page-shown notification, the
sticky opt-in, tap routing, group and sensitive behavior, service worker
rules, requirements, every enable() and onPush() error, push caps for
unlisted apps and websites, and sending from workers and events handlers.

The push playground example is removed (the playground can't serve the
site's service worker) and redirects to the guide. The minimal page catches
enable() errors and sends after 5 seconds. The recipe no longer sends on
every load, and app.message is listed in events fetch and the catch-up
recipe.

* docs(notifications): per-app browser caps and when push_endpoint_revoked appears

* docs(notifications): the page's handler timeout

* fix(notifications): leave a site's own service worker alone when enabling pushes

Registering /puter-sw.js at a scope another worker holds queued it behind that worker, so enable() rejected with push_worker_conflict but swapped the site's worker out once its tabs closed. Now enable() checks the scope first and rejects without registering.

* docs(notifications): a background page shows score alerts itself
2026-10-30 14:43:48 -07:00
Daniel Salazar 61b713438d docs(notifications): push setup requirements, browser and device support, and push recipes (#4406)
Adds worker requirements, per-browser/OS support (iOS Home Screen, macOS/Windows/Android settings), and troubleshooting to the push setup page. Website taps now documented on on('click'). New recipes: notify chat members of new messages, alert users when shared data changes.
2026-10-30 09:12:19 -07:00
Daniel Salazar 7ceaf69651 docs(notifications): import push-sw.js from the root of js.puter.com (#4405)
The worker is published next to puter.js at https://js.puter.com/push-sw.js.
Docs and the worker's own header now point there. The js subdomain serves
/push-sw.js for self-hosted setups and keeps /v2/push-sw.js as an alias.
2026-10-29 20:10:27 -07:00
Juan Fernando CastroandDaniel Salazar c00f693ec6 feat(notifications): report how each live notification was received (#4403)
* feat(notifications): report how each live notif.message was received

SocketService.sendWithAck sends an event to rooms or sockets and waits,
up to a timeout, for an acknowledgement from each socket that listed the
event in its handshake (`auth.acks`); every other socket gets a plain
emit. It lists sockets across the cluster with fetchSockets and asks the
listed ones through the adapter's broadcast acks, and counts this node
alone when the adapter cannot list the others.

The emitting region now sends notif.message through it, with the window
from `notificationAckTimeoutMs` (default 5000), and emits the node-local
`notif.delivery.settled` with the outcome: received, unreceived,
no-socket or remote-only (sockets only in a peer region), whether any
socket there acknowledges, and each responder's visibilityState. Peer
regions keep sending their own copy as before; the GUI event carries
`region_delivered` so this region does not send it twice.

The desktop and dashboard list notif.message in their handshake and
answer its callback once with document.visibilityState. Nothing in core
acts on the settled event.

* fix(notifications): send live notif.message at once and report what the count covers

sendWithAck now emits to the whole room straight away as a broadcast with
ack, and lists the room's sockets alongside it only to fill in the
counts, so nobody waits on the listing. Whether the adapter can count
the cluster's servers is checked once; where it cannot, sends go out
per socket on this node with a one-time warning. The result carries
`complete`: false when only this node's sockets were counted, either
for that reason or because the listing failed or took over 1 s, which
is warned about at most once a minute.

notif.delivery.settled gains `sendId`, `type`, `complete` and
`peerRegions`. An incomplete count with no socket found settles as
unreceived rather than no-socket, and the peer-region read runs
alongside the send instead of after it. notif.written carries the same
`sendId`, minted per send (an update gets a new one) and passed through
notif.created to the socket adapter on the fold-in path.

The comment on revoking sockets no longer says fetchSockets is
unavailable with our Redis client; it works with ioredis cluster.

* fix(notifications): keep a failed server count from ending the process

socket.io awaits the adapter's serverCount without a catch on every
broadcast with an ack, and the redis-streams adapter's count rejects as
soon as one cluster node fails to answer. The count on the root
namespace's adapter now never rejects: a failure reports the last good
count (1 before any), and is warned about at most once a minute. A stale
count only lets an ack wait run to its timeout.

Whether sendWithAck takes the cluster path is no longer a one-shot check
awaited before the emit. Until a count has succeeded, sends go out per
socket on this node at once while a count is retried in the background,
at most every 30 s (SocketService.COUNT_RETRY_MS); the first success
switches to the cluster path for good.

* fix(notifications): ask only acknowledging sockets, through an ack room

A socket whose handshake lists an event in `acks` now joins
`<room>:acks:<event>` for each of its rooms and its own id.
sendWithAck sends the ordinary broadcast to the target rooms except
those ack rooms, which keeps it on the adapter's stream, and a broadcast
with an ack only to the ack rooms. Sockets that never acknowledge are no
longer asked, so other nodes hold no pending ack for them and a send
completes as soon as every acknowledging socket answered. The per-socket
listing that matched socket.data.acks is gone; the ack-capable count
comes from the room membership the listing already returns. The
per-socket path stays only for a node whose adapter has not counted
servers yet.

A failed server count during a send now marks its counts incomplete, so
a stale count can't make a listing that missed a node read as complete.
With no count ever succeeded (an in-memory Redis), the failure is warned
once per process instead of once a minute.

* fix(notifications): say whether every socket on a live notification acknowledges it

A tab on an older bundle beside one that acknowledges made the settle read
as ack-capable, so a listener could not tell that the older tab may have
shown the notification. notif.delivery.settled now carries everySocketAcks.

* feat(notifications): app pages acknowledge the live copy of an app message

NotificationService.appReceipt() asks an app's own sockets whether a
notification reached a page, and settles it as notif.delivery.settled with
appUid, the same outcomes as the desktop's. Callers ask only when the answer
decides something, since it is a round trip to every node.

- SocketService.askWithAck asks without listing sockets: sockets that don't
  acknowledge get an acked broadcast too, which can only time out, and that
  says whether an older client shares the target.
- puter.js declares notif.receipt on its filesystem and events sockets. A page
  answers once one of its notification feeds received that uid, before any
  handler runs, and never when no feed on the page got it.
- Docs describe the push route as "no page of the app received it live
  within 5 seconds".

* fix(notifications): count a socket that acknowledges other events as a current client

Every desktop page also holds puter.js's own socket, which never shows
notif.message, so "every socket acknowledges notif.message" never held there.
sendWithAck now reports `declaring`, the sockets that listed any event in
auth.acks, and everySocketAcks on the desktop's settle means no socket here
declares nothing.

* fix(puter-js): fall back to the Continue dialog when a popup can't be opened

Some browsers report user activation but still return null from
window.open (iOS home-screen web apps, Safari after awaited work), so
permission prompts, signIn, enablePush, and GUI action popups failed
without showing anything.

PuterDialog.launch() now owns the open-now-or-show-Continue decision for
every popup flow and shows the consent dialog when the direct open
returns null. On cross-origin-isolated pages signIn leaves settling to
the /login/wait relay on both launch paths.

* fix(notifications): show a live push when the page's window isn't focused

onPush showed the handler's result as a system notification only when
the tab was hidden. A visible tab in a window the user left showed
nothing. It now also shows when the document doesn't have focus. Inside
the Puter desktop the desktop's banner shows the notification, so an
app's frame still never does.

* build(puter-js): publish push-sw.js exactly as written

The build emits dist/push-sw.js next to puter.js, but terser minified it
along with the bundle. It is now marked as already minimized, so the file
apps import into their service worker is the source unchanged.

* fix(notifications): send live notif.message plainly and ask for receipts on request

Every live notif.message used to be sent with an ack, list the user's sockets
across the cluster (fetchSockets ships each socket's handshake between nodes)
and wait out the receipt window, whether or not anything would act on it.

notif.message is a plain send again. NotificationService.receipt() asks the
desktop's sockets, or with appUid an app's own, the way app receipts already
did: an ask with no socket listing, settled as notif.delivery.settled. A
caller asks only when the answer changes what it does.

- The desktop and dashboard sockets answer notif.receipt from the page's
  puter.js receipt log and note each live notif.message in it, so every socket
  on the page answers alike. A page whose puter.js has no log lists no acks.
- SocketService.sendWithAck, the socket listing and region_delivered are gone.
  askWithAck counts this node's sockets when the adapter cannot count servers.
- The send id stays on notif.written and the settle, and leaves the wire.

* fix(socket): join only the ack room receipts are asked in

A socket listing events in auth.acks joined an ack room beside each of its
rooms and its own id. Receipts are only asked in the user's room, or the app's
for an app socket, so that is the one ack room it joins.

* docs(notifications): publish the push limits of 12 a minute and 1,000 a day

Pushes are capped per user per app at 12 a minute and 1,000 a day, and at 6 a
minute and 200 a day while an app is new. The rate limits table and the
notification pages said one a minute and 100 a day.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-29 18:57:36 -07:00
Juan Fernando Castro dace77c212 fix(gui): a request the server no longer accepts signs the tab out locally (#4390)
A 401 outside the reauth flow used to run the full logout, which ends the
session on the server and takes every device registered under it. The tab
now drops its token and reloads instead; the session cookie signs it back
in when it still holds. A second 401 right after that reload falls back
to the full logout. A peer tab signing out is mirrored the same way.
2026-10-29 09:19:43 -04:00
Juan Fernando CastroandDaniel Salazar 8e1233d82a feat(push): notifications:push app permission (#4317)
* feat(push): notifications:push app permission

Apps need a per-app notifications:push grant for their notifications to
go out as pushes. hasPushConsent checks it as the app acting for the
user, so only that user's grant to that app answers yes, and a revoke
takes effect on the next check. The permission dialog describes it and
the perms docs list it.

* fix(push): canonical app uid in the push consent check

* fix(push): let users grant apps notifications:push

The grant route now only accepts permissions from a fixed set of
app-grantable families, which refused notifications:push, so an app could
never be given push consent. Register it next to events:background.

* refactor(permission): keep the notifications:push constant with the permission code

The consent check that reads it lives with the delivery code, outside
core. The grant flow test checks the grant through the permission
service directly.

* feat(notifications): node-local notif.written hook

NotificationService emits notif.written after each non-silent write and
after an in-place rewrite, on the node that wrote the row, so an
extension can act on new notifications without a second bus round trip.
It stays off the published event stream; notif.created carries the
notification itself.

* feat(puter-js): puter.notifications

An app notifies the user it runs for, reads its own notifications back
and follows new ones, hears clicks on them from the desktop, and asks
for or checks the notifications:push grant. Puter's own apps also
register, list and remove the user's devices.

* feat(puter-js): device calls report push_unavailable without a push service

A deployment with no push service has no /push routes, so core answers
the device calls with its catch-all 404. registerDevice, listDevices and
removeDevice turn that into push_unavailable, kept apart from a missing
device, which comes back as device_not_found.

* feat(homepage): serve the service worker at /sw.js

/sw.js serves the GUI's sw.js, preceded by one importScripts per script
an extension registered with registerServiceWorkerScript. Whether the
GUI registers the worker follows the pushEnabled gui param, which only
an extension sets. Notification reads also carry the row's app_uid, so
the GUI can route a tap to the app a notification is about.

* feat(gui): notifications settings for push devices and apps

Settings -> Notifications turns pushes on and off for this browser
through a push client an extension registers, lists the user's devices
and the apps allowed to push, and switches Puter's own pushes. The tab
shows only when the pushEnabled gui param is true. The sessions window
shares the user agent formatting with it.

* feat(gui): service worker that shows pushes and routes taps

sw.js shows each push as a notification and, on a tap, focuses or opens
a Puter window and hands it the tapped notification. The desktop routes
the tap to the app the notification names, opening it when needed, or
to the notifications panel. The worker is registered only when the
pushEnabled gui param is true.

* feat(gui): notifications settings read a missing push service as unavailable

Without a push extension the /push routes answer core's plain 404. The
GUI maps that to push_unavailable, so the tab says pushes aren't
available on this server instead of reporting a load failure, and a
device removal only counts as done when the device itself is missing.
The push client doc describes the payload sw.js reads and where the
push endpoints come from.

* fix(gui): keep the service worker on servers without push

Unregistering /sw.js wherever pushEnabled is unset ended the browser's
push subscription, so a single page load on such a server stopped pushes
from every server. Turning push off already ends the subscription.

* feat(gui): the service worker tells open tabs when the push subscription changes

* fix(gui): removing this browser in Settings also forgets its push registration

* fix(gui): removing this browser also deletes it on the server when its local record is gone

* fix(notifications): notif.written hands out a copy with the audience

Listeners get their own copy of the value, so what the socket carries is
what was written. Both emits name the audience, and an update names the
row's app when the row or the value says which.

* fix(notifications): read the app off the notification on an update

* fix(gui): route push taps through the stored notification

A tap from the worker names only a notification; the desktop and the
dashboard read the signed-in user's own row and route from its app,
target and data, as a tap from the URL already did. Without a row the
dashboard opens the list and the desktop does nothing.

* fix(gui): local banner tags carry the app

An app message's group is tagged under its app, as the server tags a
push, so two apps using the same group name keep separate banners.

* fix(gui): a tap delivered twice routes once

The page remembers the notifications it handed over in the last ten
seconds and drops a repeat, whichever way it arrives.

* fix(gui): the service worker never reuses cached imports

The worker is registered with updateViaCache set to none, so the scripts
it imports are fetched fresh on every update check.

* fix(gui): turn on can't run twice at once

The browser card's button stays out while the client is enabling or
disabling, through any redraw in between, and a status read begun before
the press no longer draws the card.

* feat(gui): turn on browser push when an app is allowed to notify

Allow on a permission request that includes notifications:push also
turns push on in this browser, from the same click, when it is off here.
The decision never depends on it. The permission popup starts the
service worker for such a request, answers the opener first and closes
once the registration is done. The Notifications tab records a browser
the user turned off, per user.

* feat(gui): register this browser for pushes again where it already allows them

On desktop and dashboard boot, and when an app that holds
notifications:push is launched, a browser whose notification permission
is already granted registers again without a click, unless the user
turned push off in this browser.

* feat(sdk): puter.notifications.enablePush() and the enable-push popup

An app on its own domain can offer to turn push on in the user's browser
from its own button: puter.notifications.enablePush() opens a popup on
the Puter origin whose single button supplies the click the browser's
prompt needs, and inside the desktop the same dialog is shown over IPC.
The popup hands the opener no token. Docs for the new call and for the
paths that now call the push client's enable().

* fix(gui): wait for the browser's prompt before closing the permission popup

The deadline on a registration begun from a click counts no time while
the browser's own notification prompt is open, in the permission popup
and in the enable-push dialog. The enable-push dialog keeps one
registration in flight, settles on whenever it resolves while open, and
offers a retry only after a real failure. A permission dialog that ends
in a denial after Allow began a registration turns push back off and
restores the opt-out it cleared.

* fix(gui): one enable-push dialog at a time, only for apps that may send pushes

The desktop and the popup show the enable-push dialog only when the
asking app holds notifications:push. The IPC branch answers a second
call unavailable while a dialog is up, as the feedback dialog does. The
silent registration on boot and app launch runs once at a time. A popup
whose opener cannot be attested answers the enable-push request before
it closes.

* feat(events): isOnline presence check for a user and app

* feat(notifications): onPush and enable with Web Push on the app's own origin

puter.notifications.onPush(key, handler) handles an app's notifications for
one channel: live over the events socket while the app is connected, and as
a browser push otherwise. enable() asks for notification permission on the
site's origin and subscribes through the site's worker, which imports
push-sw.js (served at js.puter.com/v2/push-sw.js). A new subscription is
confirmed by a challenge push the worker hands back to the page. disable()
removes it.

The GUI uses the same flow through /sw.js when the server publishes
pushWebPushKey.

* fix(notifications): visible sender for app messages; local banners follow push consent

An app title of only invisible or blank-filler characters no longer leaves
an app message without a sender; it falls back to the app's name, then its
uid. Hidden-tab banners now show only while push is on in this browser, and
for an app's message only when the app holds notifications:push.

* feat(notifications): enable() asks through Puter's prompt after a revoke

When the user turned an app's notifications off in Puter, the browser's
permission on the app's origin no longer re-enables them. enable() asks
again through Puter's permission prompt and retries once.

* fix(notifications): enable() rejects on a revoke, onPush() syncs quietly, one permission check per app a minute

enable() no longer opens Puter's prompt after a push_revoked error; apps call requestPermission() from a click and enable() again. onPush()'s page-load sync no longer reports push_unavailable or push_revoked. Hidden-tab banners for an app's messages check the app's push permission once a minute instead of once per message.

* docs(notifications): describe the confirmation step as what happens; one-line comments and neutral fixtures

* docs(notifications): guide for app push notifications, limits and prices

* fix(puter-js): subscribe once the worker is active, and retry a subscription the push service hasn't settled

* fix(gui): the desktop's hidden-tab banner is for Puter's own notifications only

* docs(notifications): app pushes at $2.50 per million

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-29 09:19:09 -04:00
Juan Castro 722c25e0dc Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-29 09:11:29 -04:00
Juan Fernando Castro 9e449e635b Merge pull request #4399 from HeyPuter/juancastro/put-2180-push-docs
docs(notifications): guide for app push notifications, limits and prices
2026-10-29 09:10:44 -04:00
Juan Fernando Castro ba7011b5bb Merge pull request #4391 from HeyPuter/juancastro/put-2258-push-from-app-subdomains
feat(gui): turn on browser push when an app is allowed to notify
2026-10-29 09:10:34 -04:00
Juan Castro 25d5843d03 Merge juancastro/put-2262-web-push into juancastro/put-2180-push-docs 2026-10-29 08:59:08 -04:00
Juan Castro 4e6d655f4c Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push 2026-10-29 08:59:06 -04:00
Juan Castro bd1dd6564e Merge juancastro/put-2172-notifications-settings into juancastro/put-2258-push-from-app-subdomains 2026-10-29 08:59:04 -04:00
Juan Castro 69a3e918a7 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-29 08:59:03 -04:00
Juan Castro aded055212 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-29 08:59:01 -04:00
Juan Castro e54750492f Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-29 08:58:59 -04:00
Juan Castro 406fe4dcfc Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-29 08:58:57 -04:00
Juan Castro 2e085c0b75 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-29 08:58:49 -04:00
Juan Castro b95fb870a5 Merge main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-29 08:58:39 -04:00
Daniel Salazar 28be1851f9 fix(workers): bind credentials and cleanup to the owning site (#4362)
* fix(workers): bind credentials and cleanup to the owning site

* test(workers): deploy valid source in the site-bound token test

Where workerd is installed the deploy starts the script, and the bare
identifier `src` threw at startup as an unhandled rejection.
2026-10-28 17:31:15 -07:00
Juan Castro ff83360272 docs(notifications): app pushes at $2.50 per million 2026-10-28 19:53:26 -04:00
Daniel Salazar b19fde242f fix(networking): recover shared relay clients and preserve write order (#4377)
* fix(networking): recover shared relay clients and preserve write order

* fix(networking): clear the flush flag with the last queue check

A write landing between the drained queue and the connect path resuming
was queued with nothing left to send it.

* fix(networking): keep a throwing socket listener out of connect and close

An open listener that threw was caught as a failed connect and could
retry onto a second stream; an error listener that threw stopped the
socket from ever closing. Both now rethrow on their own tick.
2026-10-28 16:05:54 -07:00
Daniel Salazar c2b38786f7 fix(ai): meter converted audio from reported output duration (#4366)
* fix(ai): meter converted audio from reported output duration

* fix(ai): size undocumented voice changer output formats as the default

* fix(ai): skip WAV headers when metering and renew streamed charge holds

A wav_* output's RIFF header was counted as samples, rounding a 3 s clip
up to 4 s. The streamed-charge hold now renews until the charge lands, so
a slow reader can't outlast it.

* fix(ai): time Opus output by its pages and renew holds through slow opens

Opus is variable bitrate, so a mostly silent clip billed far under its
length at the nominal rate; its length now comes from the last Ogg
page's granule position. The streamed-charge hold renews from before the
upstream opens, since opening can run as long as the hold's TTL, and is
released even if recording the charge throws. Speech-to-text extra_body
can no longer override response_format or stream, which billing reads.
2026-10-28 16:05:36 -07:00
Daniel Salazar 7c6c0f2d39 fix(ai): finish async media accounting and recheck output access (#4365)
* fix(ai): finish async media accounting and recheck output access

* fix(ai): settle late media jobs on their outcome and keep their slot

A job still running when the poll window closes no longer charges its
estimate up front. The caller still gets the 504; the job is followed in
the background, holding its credit, until it settles, and is metered from
the provider's outcome: what it reports, nothing for a failure that
reports no cost, the estimate only if it never settles.

The driver concurrency slot is now held until the handler and any work it
carries on after the response settle, so a disconnect no longer frees a
slot while polling continues.

A revoked puter_output_path is reported as metadata.outputSaved = false.

* fix(ai): bill a lost media job the same in and after the window

A poll error that isn't transient now ends the same way whether it hits
inside the window or after it: a client-error refusal is a failure the
provider doesn't bill, while auth, not-found and unclassified errors mean
we lost a job the provider still runs, charged its estimate. A Hailuo
clip already billed retries a transient file lookup instead of failing,
and a concurrency slot renews from the moment it is held.
2026-10-28 16:05:26 -07:00
Juan Castro 5ac41f90b6 Merge juancastro/put-2262-web-push into juancastro/put-2180-push-docs 2026-10-28 16:30:47 -04:00
Juan Castro c024e007aa Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push 2026-10-28 16:30:45 -04:00
Juan Castro eeedeb817f Merge juancastro/put-2172-notifications-settings into juancastro/put-2258-push-from-app-subdomains 2026-10-28 16:30:42 -04:00
Juan Castro ca79f01edd Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-28 16:30:40 -04:00
Juan Castro 55ba45e159 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-28 16:30:38 -04:00
Juan Castro e8688d5025 Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-28 16:30:35 -04:00
Juan Castro 10f10029c2 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-28 16:30:33 -04:00
Juan Castro 11a94ed608 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-28 16:30:30 -04:00
Juan Castro 5c35f8d053 Merge main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-28 16:30:10 -04:00
Juan Castro c5a4b1f3b5 Merge juancastro/put-2262-web-push into juancastro/put-2180-push-docs 2026-10-28 16:27:20 -04:00
Daniel Salazar 179ad990d6 feat(events): simplify onPersistent delivery (#4395)
* feat(events): simplify onPersistent delivery

Replace delivery/targets with a boolean single. Without single, every
event runs the app's published handler in its events worker and is also
sent to every connected client; the server no longer routes between the
two. single keeps leasing each event to one consumer that acks.

delivery: 'single' | 'broadcast' is still accepted and targets is
ignored. Stored rows are read by their delivery column alone.

* docs(events): compare onLocal, onPersistent and single

* feat(events): rename single to queue

onPersistent({ queue: true }) is the work-queue mode: each event goes to
one consumer at a time and is offered again until acknowledged.
delivery: 'single' is still accepted as an alias, and subscription
objects report queue instead of single.

The docs explain the three modes (onLocal, default onPersistent, queue)
and each events recipe says which one it uses and why.

* docs(events): retries are the only repeat for a queued event

* feat(events): queue deliveries fall through to the worker on decline

A queued delivery gets one socket attempt under a 15 s lease instead of
two 60 s ones. A client with no handler for it, or whose handler fails,
sends events.nack and the worker takes it without waiting for the lease.
Declines are relayed to the region holding the lease like acks are, and
a decline for a lease that is no longer the client's is ignored.

* feat(events): drop thrown errors, retry only temporary ones

A queued handler that throws now drops the event and counts one failure
toward the pause. Network errors, timeouts, rate limits, 5xx responses
and errors thrown with retry: true are retried with the existing
backoff. One classifier is shared by puter.js and the events worker
runtime, which inlines it at build time. A dropped event is billed once,
like a completed one. ack() remains as an optional early release.

* docs(events): queued events reach every tab with a handler
2026-10-28 12:58:51 -07:00
Juan Castro ff8dde6895 fix(gui): the desktop's hidden-tab banner is for Puter's own notifications only 2026-10-28 14:20:24 -04:00
Juan Castro d587ffa428 fix(puter-js): subscribe once the worker is active, and retry a subscription the push service hasn't settled 2026-10-28 13:50:19 -04:00
Juan Castro 0fe68dae30 docs(notifications): guide for app push notifications, limits and prices 2026-10-28 12:37:28 -04:00
Juan Castro eb1ad240f2 docs(notifications): describe the confirmation step as what happens; one-line comments and neutral fixtures 2026-10-28 10:06:55 -04:00
Juan Castro d40cc499f5 fix(notifications): enable() rejects on a revoke, onPush() syncs quietly, one permission check per app a minute
enable() no longer opens Puter's prompt after a push_revoked error; apps call requestPermission() from a click and enable() again. onPush()'s page-load sync no longer reports push_unavailable or push_revoked. Hidden-tab banners for an app's messages check the app's push permission once a minute instead of once per message.
2026-10-28 10:06:55 -04:00
Juan Castro 06ef9506fa Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push 2026-10-28 10:01:02 -04:00
Juan Castro 5513c23042 Merge juancastro/put-2172-notifications-settings into juancastro/put-2258-push-from-app-subdomains 2026-10-28 09:46:54 -04:00
Juan Castro 8aade99cc8 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-28 09:45:57 -04:00
Juan Castro 6f9719fd65 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-28 09:45:51 -04:00
Juan Castro 352a2fbabd Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-28 09:45:45 -04:00
Juan Castro 59700519a6 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-28 09:45:38 -04:00
Juan Castro 8482ff2167 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-28 09:45:31 -04:00
Juan Castro b42f2acf06 Merge main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-28 09:44:38 -04:00
Daniel Salazar 1b6de39a13 feat(notifications): enable() asks through Puter's prompt after a revoke
When the user turned an app's notifications off in Puter, the browser's
permission on the app's origin no longer re-enables them. enable() asks
again through Puter's permission prompt and retries once.
2026-10-28 01:17:00 -07:00
Daniel Salazar 1aa7caf44c fix(notifications): visible sender for app messages; local banners follow push consent
An app title of only invisible or blank-filler characters no longer leaves
an app message without a sender; it falls back to the app's name, then its
uid. Hidden-tab banners now show only while push is on in this browser, and
for an app's message only when the app holds notifications:push.
2026-10-28 01:06:54 -07:00
Daniel Salazar 40253ae12b feat(notifications): onPush and enable with Web Push on the app's own origin
puter.notifications.onPush(key, handler) handles an app's notifications for
one channel: live over the events socket while the app is connected, and as
a browser push otherwise. enable() asks for notification permission on the
site's origin and subscribes through the site's worker, which imports
push-sw.js (served at js.puter.com/v2/push-sw.js). A new subscription is
confirmed by a challenge push the worker hands back to the page. disable()
removes it.

The GUI uses the same flow through /sw.js when the server publishes
pushWebPushKey.
2026-10-28 01:05:36 -07:00
Daniel Salazar 3b4945f018 feat(events): isOnline presence check for a user and app 2026-10-28 00:53:28 -07:00
Juan Castro fe58213bb5 fix(gui): one enable-push dialog at a time, only for apps that may send pushes
The desktop and the popup show the enable-push dialog only when the
asking app holds notifications:push. The IPC branch answers a second
call unavailable while a dialog is up, as the feedback dialog does. The
silent registration on boot and app launch runs once at a time. A popup
whose opener cannot be attested answers the enable-push request before
it closes.
2026-10-27 16:48:07 -04:00
Juan Castro 6c35780d59 fix(gui): wait for the browser's prompt before closing the permission popup
The deadline on a registration begun from a click counts no time while
the browser's own notification prompt is open, in the permission popup
and in the enable-push dialog. The enable-push dialog keeps one
registration in flight, settles on whenever it resolves while open, and
offers a retry only after a real failure. A permission dialog that ends
in a denial after Allow began a registration turns push back off and
restores the opt-out it cleared.
2026-10-27 16:48:07 -04:00
Juan Castro 415c179770 feat(sdk): puter.notifications.enablePush() and the enable-push popup
An app on its own domain can offer to turn push on in the user's browser
from its own button: puter.notifications.enablePush() opens a popup on
the Puter origin whose single button supplies the click the browser's
prompt needs, and inside the desktop the same dialog is shown over IPC.
The popup hands the opener no token. Docs for the new call and for the
paths that now call the push client's enable().
2026-10-27 16:27:23 -04:00
Juan Castro feb0937b6d feat(gui): register this browser for pushes again where it already allows them
On desktop and dashboard boot, and when an app that holds
notifications:push is launched, a browser whose notification permission
is already granted registers again without a click, unless the user
turned push off in this browser.
2026-10-27 16:27:23 -04:00
Juan Castro f880544ebe feat(gui): turn on browser push when an app is allowed to notify
Allow on a permission request that includes notifications:push also
turns push on in this browser, from the same click, when it is off here.
The decision never depends on it. The permission popup starts the
service worker for such a request, answers the opener first and closes
once the registration is done. The Notifications tab records a browser
the user turned off, per user.
2026-10-27 16:27:23 -04:00
Neal Shah 25a574b57d GUI for custom domains (#4355)
* PUT-2226

* modifications'

* routing fixes

* secondary custom domain verification
2026-10-27 15:54:02 -04:00
Juan Castro 389fd26bee fix(gui): turn on can't run twice at once
The browser card's button stays out while the client is enabling or
disabling, through any redraw in between, and a status read begun before
the press no longer draws the card.
2026-10-27 14:20:17 -04:00
Juan Castro d77ebadfe6 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-27 14:18:31 -04:00
Juan Castro 7e78a4feef fix(gui): the service worker never reuses cached imports
The worker is registered with updateViaCache set to none, so the scripts
it imports are fetched fresh on every update check.
2026-10-27 14:18:07 -04:00
Juan Castro 7aa4d91ae9 fix(gui): a tap delivered twice routes once
The page remembers the notifications it handed over in the last ten
seconds and drops a repeat, whichever way it arrives.
2026-10-27 14:17:57 -04:00
Juan Castro 5f7ab80721 fix(gui): local banner tags carry the app
An app message's group is tagged under its app, as the server tags a
push, so two apps using the same group name keep separate banners.
2026-10-27 14:17:48 -04:00
Juan Castro b3dbfdbd7b fix(gui): route push taps through the stored notification
A tap from the worker names only a notification; the desktop and the
dashboard read the signed-in user's own row and route from its app,
target and data, as a tap from the URL already did. Without a row the
dashboard opens the list and the desktop does nothing.
2026-10-27 14:17:40 -04:00
Juan Castro 60dfff3b0b Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-27 14:11:58 -04:00
Juan Castro 5a83c4a045 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-27 14:11:58 -04:00
Juan Castro fe8acf007c fix(notifications): read the app off the notification on an update 2026-10-27 14:11:46 -04:00
Juan Castro 8010c9fc94 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-27 14:11:10 -04:00
Juan Castro bbf375db14 fix(notifications): notif.written hands out a copy with the audience
Listeners get their own copy of the value, so what the socket carries is
what was written. Both emits name the audience, and an update names the
row's app when the row or the value says which.
2026-10-27 14:10:56 -04:00
Juan Castro fe70e7c6ba Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-27 14:08:01 -04:00
Juan Castro f0d7a69dd4 fix(notifications): cap the sender name on app messages
The app's title or name is cut to the title length, counting code points,
before it is stored on the message.
2026-10-27 14:07:27 -04:00
Juan Castro 5ce131517b fix(notifications): refuse an app message before it spends the day window
The minute window is spent first, so a request the minute window refuses
between the read and the spend leaves the day window untouched.
2026-10-27 14:07:25 -04:00
Daniel Salazar 49b38f3482 fix: seat creation (#4387) 2026-10-27 12:49:31 -04:00
Nariman JelvehandClaude Opus 5.5 241fb7be8a feat(dashboard): simplify the share modal
- The access level and Share button appear beside the field only once
  someone is entered, instead of a full-width button that sat disabled.
- The access list sits directly under the field. Each person's menu ends
  with Remove access (or Cancel invitation), replacing the trash column;
  the inline confirmation stays.
- General access is one row: a lock or globe icon, the switch with its
  explanation beneath, and the level on the right.
- A footer holds Copy link and Done. Copy link is offered for any single
  item, since the link opens it for anyone who has access, and confirms
  in place. Status messages live in the footer; successes fade.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-27 09:08:44 -07:00
dependabot[bot] 0167a9d10e chore(deps): bump fast-uri from 3.1.7 to 3.1.9 in /src/puter-js (#4336)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.7 to 3.1.9.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.9)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-27 08:54:47 -07:00
dependabot[bot] 5a879a4346 chore(deps): bump smtp-server from 3.19.23 to 3.19.24 (#4383)
Bumps [smtp-server](https://github.com/nodemailer/smtp-server) from 3.19.23 to 3.19.24.
- [Release notes](https://github.com/nodemailer/smtp-server/releases)
- [Changelog](https://github.com/nodemailer/smtp-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/smtp-server/compare/v3.19.23...v3.19.24)

---
updated-dependencies:
- dependency-name: smtp-server
  dependency-version: 3.19.24
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-27 08:54:33 -07:00
Juan Castro c13bff0563 fix(gui): removing this browser also deletes it on the server when its local record is gone 2026-10-27 11:10:32 -04:00
Juan Castro 2f94ee1088 fix(gui): removing this browser in Settings also forgets its push registration 2026-10-27 11:07:23 -04:00
Juan Castro 3068059058 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-27 11:06:38 -04:00
Juan Castro 7ee0cbfac9 feat(gui): the service worker tells open tabs when the push subscription changes 2026-10-27 11:06:14 -04:00
Juan Castro dfa6f4effc Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications 2026-10-27 10:44:19 -04:00
Juan Castro 20c63b6282 Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook 2026-10-27 10:44:19 -04:00
Juan Castro dd44c0704a Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-27 10:44:19 -04:00
Juan Castro 681b9cd463 Merge juancastro/put-2170-puterjs-notifications into juancastro/put-2171-push-service-worker 2026-10-27 10:44:19 -04:00
Juan Castro 9fb66137a3 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-27 10:44:14 -04:00
Juan Castro 057aa116a4 Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-27 10:44:11 -04:00
Daniel Salazar 1cea1b2a0a perf(thumbnails): keep read URLs stable within an hour (#4379) 2026-10-26 22:16:54 -07:00
Daniel Salazar 6c78d2f9e9 fix(ui): reject failed desktop IPC requests (#4378) 2026-10-26 22:14:19 -07:00
Daniel Salazar e8aa12ace5 fix(puter-js): replay requests with fresh body credentials (#4376) 2026-10-26 22:10:24 -07:00
Daniel Salazar f1964778b4 fix(puter-js): coordinate sign-in callbacks and filesystem sockets (#4375) 2026-10-26 22:09:52 -07:00
Daniel Salazar c8f7b1ceda fix(auth): support recovery codes when disabling two-factor auth (#4374) 2026-10-26 22:07:52 -07:00
Daniel Salazar 0411a92f4e perf(dashboard): share launch-app requests and size recent icons (#4373) 2026-10-26 22:07:33 -07:00
Daniel Salazar 5cfd4d1f53 fix(hosting): honor user-granted subdomain permissions (#4372) 2026-10-26 22:07:04 -07:00
Daniel Salazar e1a36215f6 fix(sharing): preserve team grants and scope link invalidation (#4371) 2026-10-26 22:06:16 -07:00
Daniel Salazar 6f13b1f457 fix(apps): compare normalized index URLs before update checks (#4370) 2026-10-26 22:05:21 -07:00
Daniel Salazar 996d85148e fix(fs): remove app data requested during uninstall (#4369) 2026-10-26 22:05:05 -07:00
Daniel Salazar 09da68424e fix(fs): preserve stored path spelling and app ACL identity (#4368) 2026-10-26 22:04:36 -07:00
Daniel Salazar 7bccde40a8 fix(hosting): use the current authenticated visitor for asset billing (#4364) 2026-10-26 22:04:17 -07:00
Daniel Salazar cf5d2c60a5 perf(permission): resolve default grants through full-access issuers (#4361) 2026-10-26 22:03:45 -07:00
Daniel Salazar b7a0276337 fix(fs): preserve paging and file read-copy contracts (#4367) 2026-10-26 22:03:09 -07:00
Daniel Salazar bbc056cc49 fix(hosting): track trashed roots and cached backing misses (#4363) 2026-10-26 21:58:05 -07:00
Daniel Salazar e2f6c2ed05 fix(cache): widen tombstone window to 5s (#4381) 2026-10-26 21:42:23 -07:00
Daniel Salazar 745b3a4318 perf(auth): coalesce session activity writes across processes (#4360) 2026-10-26 21:37:54 -07:00
Daniel Salazar 2dfd93687f perf(database): defer primary fallback for user lookups (#4359) 2026-10-26 21:37:06 -07:00
Daniel Salazar 91554a137e fix(apps): keep app provenance and permission caches consistent (#4357) 2026-10-26 21:28:40 -07:00
Daniel Salazar 52da7b571c perf(permission): read a large holder's user-to-user grants per permission (#4349)
A user-to-user grant check loaded the holder's whole row set, from one
Redis blob or one SELECT, and filtered it in JS. Every KV share handle a
user receives adds a row, so the cost of each check grew with incoming
shares.

Holders with up to 200 grants keep the whole-set cache. Past that the
cache holds a marker and each check reads only the permissions it names,
off a new (holder_user_id, permission) index. Matches stay exact in JS,
since MySQL compares `permission` case-insensitively. The batched
multi-holder read shares the same chunked query.

Migrations: mysql_mig_51, postgres_mig_40, sqlite 0097.
2026-10-26 20:51:46 -07:00
Nariman Jelveh 9e1ccf7ff3 fix(dashboard): ninth adversarial review of Files — bugs, UX, a11y, perf (#4356)
* fix(dashboard): keep Files keys away from the rows under a publish dialog

With Publish as website or Publish as Worker open and focus on one of its
buttons, the arrow keys moved the listing's selection behind the dialog,
Ctrl/Cmd+A selected every row, and Delete trashed whatever was selected.
The Files shortcuts already stood aside for the share and properties
modals; they now do for the dashboard dialog overlay too.

* fix(dashboard): let type-to-select in Files find names with spaces

Typing "my z" to reach "my zebra.txt" stopped at "my notes.txt": the space
went to the image preview instead of the name being typed, and the search
went on as "myz". A space typed while a name is being typed now belongs to
it; on its own, Space still opens and closes the preview.

* fix(gui): say so when a rename can't reach the server

A rename that failed on the network (offline, connection dropped) put the
old name back without a word, so in Files it looked as if the rename had
been ignored; only failures the server answered with a message were
reported. Every failure is now reported, with the unknown-cause message
when there is no other. The rename's own promise was also left to reject
unhandled on every failure, conflicts included; the error callback already
handles it.

* fix(dashboard): show a grid tile's full name on hover

A tile name cut short to fit shows in full on hover, but in the grid views
the hover never reached it: the badge layer is the size of the whole tile
and sat on top of the name. The name now sits above that layer, so its
tooltip shows; clicking and dragging by the name behave as before.

* fix(dashboard): make the empty-folder notice readable in dark mode

"No files in this directory." and "This folder couldn't be opened." were
styled inline with no color of their own and inherited the page default,
black, which on the dark theme's background could hardly be seen. Both now
share a class with the muted text color the Apps tab's empty state uses,
in either theme.

* fix(dashboard): let a folder be published again once its website is removed

Disassociating a folder's website in Properties took the globe badge off
its row, but the row's entry still said it had a website: its menu kept
Publish as website greyed out, and going into another folder and back
brought the badge back. The entry the menu and the listing read is now
updated with the row, and the sites cache is refreshed, as publishing does.

* fix(dashboard): keep a just-published worker's badge across navigation

Publishing a .js file as a worker badged its row, but the row's entry still
listed no worker. Going into a folder and back redrew the row from that
entry (the folder's cached listing), so the badge went away and the menu
offered Publish as Worker again, which would deploy the file a second time.
The entry now records the worker, as publishing a website records its site.

* fix(dashboard): badge a folder published from inside it once you go up

Publishing the folder on screen (from the listing's background menu) and
going Up showed its row without the globe badge, and with Publish as
website on offer again: the parent's listing came from puter.js's cache,
and adding a website sends no fs event to clear it. Removing a website from
inside the folder (Properties > Disassociate) left the badge behind the
same way. Both now drop the parent folder's cached listing.

* fix(dashboard): don't offer a second website for a published folder

A folder's own row greys out Publish as website once the folder has a
site, but its menu on the sidebar, a breadcrumb or the listing's
background kept offering it, and publishing again gave the folder a second
address. Those menus now check the user's sites as well.

* fix(dashboard): give Files' row menu button a finger-sized target on tablets

On a touch screen with room for the full list columns (an iPad in
landscape, say), a row's '⋯' was a 20x23px target right beside a row whose
tap opens the item, so a near miss opened the file instead of its menu.
On touch it now answers across the row's height and 44px wide; phones'
two-row layout keeps its 44px square, and mouse pointers are unchanged.

* fix(dashboard): keep Files' Back arrow clear of the phone menu button

On phones the breadcrumb row was shorter than the fixed menu button beside
it, so the button hung over the arrows below, and its enlarged tap area
took the top quarter of Back: a tap there opened the menu. The crumb row
is now as tall as the button, which also lines the crumbs up with it, and
on touch the crumbs themselves get a taller tap target.

* fix(dashboard): fit the Files image preview to the image and the listing

The Space preview was a strip the full width of the listing whatever the
image's size (a 64px icon sat in a card 900px wide), and at 3px from each
edge rather than the 20px it was given, its padding coming on top of its
width. In a short listing (a phone in landscape) it ran past the bottom of
the screen and cut off the file name. The card now wraps the image, stays
20px inside the listing both ways, and shrinks the image to keep the name
in view.

* fix(dashboard): let the last grid tile scroll clear of the phone footer

On a phone with a home indicator (a web app on the home screen, or Safari
with its bars collapsed) the Files footer grows by the bottom safe area.
The list makes room for that, but the grid views clear the footer with a
fixed 30px margin, so the bottom 17px or so of the last tile stayed behind
the footer even scrolled to the end. The grids' margin now grows with the
footer; select mode already made room for its bar and is unchanged.

* fix(dashboard): don't claim a large shared selection is restricted

Sharing more than 50 items skips listing who has access, but General
access still said "Restricted: only people with access can open these
items" without having looked, though some of them may be open to anyone
with the link. Choosing Restricted to close them did nothing either, since
it was already the selected option. It now says the link access wasn't
checked, and either choice applies to every item.

* fix(dashboard): don't let "Add to all" lower access on an item it couldn't list

Sharing several items lists who has access to each. When one item's
listing failed, the person's row counted that item as one they couldn't
reach and offered "Add to all", which granted it at the row's mode: someone
who could edit that item was quietly brought down to view. "Add to all"
now extends a grant only to items whose access was listed.

* fix(dashboard): let a click through as soon as a Files dialog closes

The share, properties and publish dialogs fade out for 200ms after they
close, and while they did, their invisible backdrop still took the
pointer: a click on a row right after Escape or Close was lost. The
backdrop now takes clicks only while the dialog is showing.

* fix(dashboard): don't make a removal under way look cancelled on Escape

In the share dialog, Escape on an open "Remove access" confirmation puts
the person's row back. Pressed after Remove was clicked, while the request
was still out, it did the same: the row came back as if the removal had
been called off, and then vanished when it finished. Escape now leaves a
removal that is under way alone.

* fix(dashboard): keep Share down while a share is on its way

Editing the name in the share dialog while a share was still being made
enabled the Share button again, so Enter sent a second share alongside
the first; the first one's end then took the spinner off while the second
was still running, and the two results raced for the status line. The
button now stays down until the share in flight is done.

* fix(gui): find the web address of files inside a published folder again

Files under a folder served as a website carry a link badge and their
address on the site. Working that out read `dir_path` and `address` off
each of the user's sites, which the hosting list no longer returns, so no
file ever had one. In Files the badges publishing just put up were gone at
the next listing. The folder and address are now read from the site's
root_dir, where the list carries them.

* fix(dashboard): show the link badge in the dark list view

In dark mode the list view drew a file's link badge (it is served by a
published website) as its black glyph on the dark row, where it couldn't
be seen. The grid view already gave it a light disc; the list view now
draws the glyph light.

* fix(dashboard): let Undo take back a paste of something cut from its menu

A row menu's Cut puts the item's bare path on the clipboard, where the
Files keys and the selection bar put { path, uid }. Pasting it moved the
item but recorded nothing for Undo (and didn't point Back/Forward at a
moved folder), so Ctrl/Cmd+Z then undid whatever came before, a rename
say. A bare path is now taken as an item like the rest.

* fix(gui): undo one action per Ctrl/Cmd+Z

The Undo shortcut sits in a handler bound to both keydown and keyup, and
ran on each: letting Z go before Ctrl/Cmd, as one usually does, undid a
second action straight after the first. In Files that could put back a
rename or delete an upload the user never meant to touch. It now runs on
the key going down only.

* fix(dashboard): re-sort and re-icon a Files row renamed elsewhere

A file or folder renamed in another tab, on another device or in a desktop
window took its new name in the listing but kept the place its old name
sorted to, and the icon of its old type (notes.txt renamed to notes.png
still showed a text icon). A renamed folder also left Back/Forward on its
old path. Such a row is now handled as a rename made in Files is.

* fix(dashboard): stop failed and cancelled Files uploads throwing uncaught

An upload from Files reports a failure in its progress window, through the
error callback, but the upload's promise also rejected with nobody
listening, so every failed or cancelled upload logged an uncaught error. It
is now handled there.
2026-10-26 17:21:35 -07:00
Filip Kujundžić 8f94514692 feat(ai): add GPT-6.1 Terra to the OpenAI catalog (#4350) 2026-10-26 16:01:44 -07:00
Juan Castro 2ad49e1040 Merge juancastro/put-2171-push-service-worker into juancastro/put-2172-notifications-settings 2026-10-26 18:50:14 -04:00
Juan Castro 57349a6982 fix(gui): keep the service worker on servers without push
Unregistering /sw.js wherever pushEnabled is unset ended the browser's
push subscription, so a single page load on such a server stopped pushes
from every server. Turning push off already ends the subscription.
2026-10-26 18:50:12 -04:00
Juan Castro ac0de9df44 feat(gui): notifications settings read a missing push service as unavailable
Without a push extension the /push routes answer core's plain 404. The
GUI maps that to push_unavailable, so the tab says pushes aren't
available on this server instead of reporting a load failure, and a
device removal only counts as done when the device itself is missing.
The push client doc describes the payload sw.js reads and where the
push endpoints come from.
2026-10-26 16:45:28 -04:00
Juan Castro 7d8711bf5b feat(gui): service worker that shows pushes and routes taps
sw.js shows each push as a notification and, on a tap, focuses or opens
a Puter window and hands it the tapped notification. The desktop routes
the tap to the app the notification names, opening it when needed, or
to the notifications panel. The worker is registered only when the
pushEnabled gui param is true.
2026-10-26 16:45:28 -04:00
Juan Castro af13c33678 feat(gui): notifications settings for push devices and apps
Settings -> Notifications turns pushes on and off for this browser
through a push client an extension registers, lists the user's devices
and the apps allowed to push, and switches Puter's own pushes. The tab
shows only when the pushEnabled gui param is true. The sessions window
shares the user agent formatting with it.
2026-10-26 16:45:28 -04:00
Juan Castro 5cf13547f2 feat(homepage): serve the service worker at /sw.js
/sw.js serves the GUI's sw.js, preceded by one importScripts per script
an extension registered with registerServiceWorkerScript. Whether the
GUI registers the worker follows the pushEnabled gui param, which only
an extension sets. Notification reads also carry the row's app_uid, so
the GUI can route a tap to the app a notification is about.
2026-10-26 16:45:28 -04:00
Juan Castro e181a25f95 feat(puter-js): device calls report push_unavailable without a push service
A deployment with no push service has no /push routes, so core answers
the device calls with its catch-all 404. registerDevice, listDevices and
removeDevice turn that into push_unavailable, kept apart from a missing
device, which comes back as device_not_found.
2026-10-26 16:36:46 -04:00
Juan Castro 5e376bc0c8 feat(puter-js): puter.notifications
An app notifies the user it runs for, reads its own notifications back
and follows new ones, hears clicks on them from the desktop, and asks
for or checks the notifications:push grant. Puter's own apps also
register, list and remove the user's devices.
2026-10-26 16:36:42 -04:00
Juan Castro fceaee6372 feat(notifications): node-local notif.written hook
NotificationService emits notif.written after each non-silent write and
after an in-place rewrite, on the node that wrote the row, so an
extension can act on new notifications without a second bus round trip.
It stays off the published event stream; notif.created carries the
notification itself.
2026-10-26 16:34:18 -04:00
Juan Castro 47a6122c37 refactor(permission): keep the notifications:push constant with the permission code
The consent check that reads it lives with the delivery code, outside
core. The grant flow test checks the grant through the permission
service directly.
2026-10-26 16:30:05 -04:00
Juan Castro 9fabd1d5b3 Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-26 16:28:52 -04:00
Juan Castro 32b28c1994 Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-26 16:28:48 -04:00
Neal Shah 21625de22e Merge pull request #4315 from HeyPuter/codex/kimi-k3-1-pro
Codex/kimi k3 1 pro
2026-10-26 21:01:04 +01:00
Juan Castro c79b3edf4f Merge juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-26 13:12:24 -04:00
Juan Castro d366861321 Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-26 13:12:23 -04:00
Daniel Salazar f850155411 KV share handles: hardening, higher limits, batch share, uninstall data deletion, account-deletion cleanup (#4344)
* fix(kv): sign share handle ids and drop retired handles from every node's cache

New handle ids carry a short HMAC (kvh-<uuid>.<mac>); an id whose mac
does not verify is refused before any lookup. Unsigned ids keep working
but no longer fall back to the primary on a replica miss.

Retiring or changing a handle now broadcasts outer.pubsub.kv.share.forget
so every node drops its cached row. Event subscribes and delivery
rechecks refuse a handle whose owner is suspended, as the driver does.

* fix(kv): an app its user removed stops reaching shares

An app acting for a user exercises a share grant only while the user
still has the app authorized: at least one user-to-app permission row,
which every app token path leaves and revoke-user-app '*' removes. The
share itself stays; authorizing the app again restores access. The
answer rides the permission cache, keyed by the app-under-user
generation the revoke bumps, so live subscriptions go quiet too.

* feat(kv): list keys through a watch-level share handle

list with as: 'keys' through a handle needs only watch, which already
learns every key written in the region. Values and entries still need
read.

* feat(kv): filter share listings by app and by state

GET /kv/shares and GET /kv/shares/with-me take an optional appUid: an
app uid, or null for the app-less store. An account session may name
any app; an app is refused any namespace but its own. GET /kv/shares
also takes state=live|revoked|all, all by default.

* feat(puter.js): appUid and state options on the KV share listings

listShared() and listSharedByMe() pass appUid (null for the app-less
store), and listSharedByMe() passes state. Docs cover the new options
and that a watch share can list keys without their values.

* feat(kv): indexes and store counts for share-handle quotas and retention

Adds covering indexes for the owner's live count, the owner-holder pair,
app-filtered listings and retired rows, plus store methods to count a
pair's live handles and delete rows retired before a cutoff. The live
handle cache holds up to 50,000 entries; the miss cache stays at 10,000.

* feat(kv): higher share-handle caps, a per-holder cap, and sharing with several recipients

Live share handles per account go to 25,000 paid / 5,000 free, and per
app to 10,000 / 2,000. One owner may hold out 256 live handles to one
holder on every plan, which bounds the rows a revoke or unshare walks.
Re-sharing an existing region takes no slot and is never refused.

POST /kv/shares takes recipients (1 to 50) as an alternative to
recipient: one region, level and label for all. Caps and a new
per-recipient grant budget (300/min, 2,000/hour) are checked for the
whole batch up front; failures about one recipient are reported on its
item. The single-recipient request and response are unchanged and run
through the same code path.

* feat(kv): delete share handles 30 days after they stop

A daily job deletes retired share-handle rows older than 30 days, in
batches of 500. One node runs it per day; it stops on shutdown.

* feat(puter.js): share a KV prefix with several users in one call

puter.kv.share() takes an array of recipients, positionally or as
recipients in the options form, and resolves to one item per recipient:
the share, or an error with its message and code. The single-recipient
forms are unchanged.

* feat(kv): delete an app's data on uninstall, and close account-deletion gaps

Uninstall:
- POST /auth/revoke-user-app takes an optional `deleteData` boolean, only
  with `permission: '*'` (400 otherwise). AuthService.uninstallApp revokes
  every grant, deletes the app's worker sessions, then awaits
  `app.uninstalled` { userId, userUuid, appUid, deleteData }.
- With deleteData, KvShareService retires the handles the user shares out
  of that app's namespace (as the platform, in batches) and then empties
  the namespace in the background, 5,000 items per step, stopping on
  shutdown. Handles shared with the user are not touched.
- SystemKVStore.purgeNamespace(namespace, { maxItems }) deletes up to
  maxItems items of one `v1:<uuid>:<appUid>` namespace, reading keys only,
  with no mutation events or usage, and reports `done` once it is empty.
  It refuses the system namespace and anything else. flush shares its
  page-delete path.
- puter.perms.revokeApp(appUid, '*', { deleteData: true }); the desktop's
  uninstall dialog has an unchecked "Also delete this app's data" option.

Account deletion:
- EventsService settles each of a deleted account's apps on `user.delete`
  (they cascade without `app.changed`) and drops the owner's subscription
  keys.
- The flat permission entries the account held are purged in pages.

Suspension:
- `user.suspended` / `user.unsuspended` events; the share-handle store
  drops cached rows owned by that user here and on other nodes.

* fix(permission): await a peer's flat-permission deletes so emitAndWait sees them applied

* fix(kv): strongly consistent purge pages, and an uninstall purge stops once the app is authorized again

A stale empty page could end a purge as done while items remained. The
background purge after an uninstall now checks before each step that the
user hasn't authorized the app again, so data written after that is kept.

* fix(kv): flush keeps eventually consistent page reads; only a purge reads strongly consistent

* fix(user): emit user.suspended and user.unsuspended when the user store changes an account's suspension
2026-10-26 09:43:55 -07:00
Juan Castro c8998075ea fix(push): let users grant apps notifications:push
The grant route now only accepts permissions from a fixed set of
app-grantable families, which refused notifications:push, so an app could
never be given push consent. Register it next to events:background.
2026-10-26 11:55:37 -04:00
Juan Castro 759f1a798d Merge origin/juancastro/put-2251-let-apps-send-notifications-to-their-own-user into juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes 2026-10-26 11:30:50 -04:00
Juan Castro b57fef8c9d Merge origin/main into juancastro/put-2251-let-apps-send-notifications-to-their-own-user 2026-10-26 11:29:13 -04:00
Reynaldi Chernando 7c0cf6f6a1 docs(fs): drop batch-write fallback notes from puter.fs.upload() (#4332) 2026-10-26 17:25:23 +07:00
Nariman Jelveh 7f93166560 fix(hosting): disconnect the directory when update gets a null root_dir (#4345)
puter.hosting.update(name, null) sends root_dir: null, which the driver
stringified into a lookup for a folder named "null" and rejected, so
disassociating a website from a folder's properties always failed.
2026-10-25 23:28:45 -07:00
Nariman Jelveh 21846192d1 feat(dashboard): publish websites and workers from Files in dashboard dialogs (#4342)
* feat(dashboard): publish websites and workers from Files in dashboard dialogs

The Files tab's Publish as website and Publish as Worker actions open
dashboard form dialogs instead of desktop windows: the address field shows
the domain inline and shapes the typed name into a valid one, workers get a
Sandboxed switch, and success shows the live address with copy and open.
Published rows get their badges right away.

The form dialog gains suffix and switch fields and a shared copy row, which
the API token dialog now uses too. The save-account / confirm-email gate
before publishing moves to one helper shared by both flows.

* fix(gui): ask a just-saved account to confirm its email before publishing

Saving a temporary account can leave its email unconfirmed when signup
doesn't require confirmation. The publish gate returned right after the
save, skipping the confirm-email step the old inline checks still ran.

* fix(dashboard): link a published site to the address the server reports

The success screen built https://<name>.<hosting domain>, which is wrong on
a self-hosted instance serving sites over http. Use the address the create
response carries, the one the rest of the GUI shows, and fall back to the
built one only when it's missing.

* fix(dashboard): dim a suffixed field as one box while the form submits

Only the input inside it dimmed, leaving the domain after the name at full
contrast while the field was locked.

* fix(dashboard): point a taken or reserved publish name at the address field

The server's reason showed in the status line, but the field stayed
unmarked and focus stayed on Publish, so changing the name took an extra
click. Mark the field invalid and focus it for the rejections that mean
"pick another name".

* fix(dashboard): type into a suffixed field when its suffix is pressed

The domain after the name is drawn inside the field's box but was a plain
span, so pressing it focused nothing. A press there now focuses the field
with the caret at the end, and the suffix shows a text cursor.

* chore(dashboard): drop a phone width rule the icon buttons never used

The form dialog's phone flex rule sizes these buttons, so the width never
applied; touch screens get 44px from the coarse-pointer min-width.

* fix(dashboard): read an underscore in a website address as a word break

A website name can't hold an underscore, so typing My_Cool Site gave
mycool-site. Underscores now become a hyphen, as spaces do; a worker
name keeps them.

* fix(dashboard): keep the letter when an accented one is typed into a publish address

Café became caf: an accented letter isn't a DNS character, so it was
dropped whole. The accent comes off and the letter stays.

* fix(dashboard): cap a website address at a hostname label's 63 characters

The server takes a 64-character subdomain, but DNS caps a label at 63,
so the site it publishes can't be reached. The field now stops at 63,
as the worker one does.

* fix(dashboard): let a publish address finish composing before shaping it

The address was reshaped on every input, composing included: a dead
key's accent turned into a hyphen before its letter arrived, and IME
text was rewritten under the composition. Composed text is now shaped
once it's in.
2026-10-25 21:34:39 -07:00
Nariman JelvehandClaude Opus 5.5 b9f0c8601d fix(dashboard): eighth adversarial review of Files — bugs, UX, a11y, perf (#4330)
* fix(dashboard): don't preview a file named after an image type, or a failed image

Space on a file named "png" (no extension) opened the image preview over a
broken image: the type was read from the last dot-separated part of the
name, which for an extensionless name is the whole name. It's now read from
the extension.

A preview whose image URL couldn't be fetched left the previous image up
under the newly selected row and raised an unhandled rejection. It now
closes instead.

* fix(dashboard): sort Files names by number, and break size/date ties by name

Names were compared as plain text, so "file10" sorted before "file2" and
"dir10" before "dir2". They now compare as a file manager does, with runs
of digits by value.

Sorting by size or date left rows with the same value (every folder, when
sorting by size) in whatever order the server listed them, and a row added
afterwards (a new folder, a rename) landed after its equals instead of where
a fresh listing would put it. Ties now fall back to the name, in the full
sort and in the single-row insert alike.

The sort also parsed each trashed item's metadata on every comparison; it
now reads each entry's sort key once.

* fix(dashboard): ask about "this item" when permanently deleting one Trash item

The Delete key and the phone selection bar's Delete Permanently asked
"Are you sure you want to permanently delete these items?" for a single
item too. One item now gets the single-item question, as from its own
menu. The three copies of the confirm-and-delete step (Delete key,
selection bar, multi-select menu) are now one.

* fix(dashboard): don't let saved column widths squeeze out an auto-width name

Saved widths too wide for the listing fall back to the defaults, but only
when the Name column had a saved width too. Beside an auto-width name, a
Size or Modified column widened on a large screen kept its width on a
smaller one and squeezed the names to nothing, with the row scrolling
sideways. The check now counts an auto-width name at its 100px minimum.

* fix(dashboard): give Files names a phone's full row, not a desktop column width

A narrow listing drops the columns for two-row rows, where each name has
the row's width. Names were still cut to the Name column's saved width
whenever the saved columns happened to fit, so a 120px column set on a
desktop cut names to about 100px on a phone with 280px to spare.

* fix(fs): refuse to replace a folder that contains the item being moved or copied

Moving or copying an item onto a name taken by one of its own parent
folders, with overwrite, removed that folder recursively first, and the
item went with it. Cutting the inner "photos" out of photos/photos, pasting
it next to the outer one and choosing Replace deleted both folders and
everything in them. Such an overwrite is now refused before anything is
removed.

* fix(dashboard): keep the Files selection when an upload lands in another folder

An upload into a folder other than the one on screen (a drop on a folder
row, a sidebar folder or a crumb, or Upload from a folder's menu) still
listed the folder on screen again when it finished. That cleared whatever
was selected there and left an open menu acting on rows no longer on the
page. Only an upload into the folder on screen refreshes it now.

The Upload picker ran its own copy of the upload flow; it now goes through
the same one as drops.

* fix(dashboard): keep Files rows up while a refresh reads web link and app icons

A refresh of the folder on screen keeps its rows until the new listing is
in, so it doesn't flash blank. But it cleared them before reading the
icons, and a web link's or .app's icon is read from the file over the
network, so in such a folder every re-sort, upload or return to the tab
blanked the pane for those reads. The icons are now read before the rows
are swapped.

* fix(dashboard): Ctrl/Cmd-drag a Files selection as a whole

Pressing Ctrl (Cmd on a Mac) on a row that was already selected took it out
of the selection at once, so a drag starting there collapsed the selection
to that one row: a Ctrl-drag of three files to copy them copied one, under
a ghost that still read "3". The row now leaves the selection when the
button is released without a drag, as in a desktop file manager, and a
drag carries the whole selection.

* fix(dashboard): let Enter open the row the arrow keys moved to in Files

Clicking a sidebar folder, crumb or toolbar button leaves it focused, and
the arrow keys and type-to-select still move the row selection from there.
Enter, Space and the menu key then went to that focused control: after
clicking Documents and pressing Down twice, Enter opened Documents again
and dropped the selection instead of opening the highlighted row. The
listing now takes the focus when these keys move the selection, so the
keys after them act on the rows, and Shift+Tab goes back to the toolbar.

* fix(dashboard): let Files' image preview be cancelled while it loads, and close it with the tab

The preview only counted as open once its image URL had come back (two
round-trips), so nothing could cancel it before then: Escape cleared the
selection and the image popped up anyway, a second Space opened it instead
of closing it, and opening a folder left the old image to appear over the
new one. It now counts as open from the start, so each of those cancels
it.

Browser Back/Forward to another dashboard tab left the preview floating
over that tab, where Escape and Space no longer reach it. A route change
away from Files now closes it, as it does Share and Properties.

* fix(dashboard): keep Files keys off a new folder until the server has made it

A new folder's row goes up with the name editor open before mkdir answers.
Clicks and menus already wait on it, but the keyboard didn't: pressing
Escape and then Delete on a slow connection sent a move for the row's
placeholder and showed "Moving New Folder: Subject does not exist", and
Enter opened the predicted path before it existed. Enter, Delete, Ctrl/Cmd+C
and Ctrl/Cmd+X now skip such a row, as do Cut and Delete in the
multi-select menu and the phone's selection bar.

* fix(dashboard): explain a Files action that fails on the network

A request that fails at the network level is rejected with no message.
New Folder then took its row back with no alert at all, and Restore and
the New menu's files showed "[object XMLHttpRequest]". A failed move from
the clipboard named the item and nothing else. Each now falls back to
"An unknown error occurred."

* fix(dashboard): don't offer Trash folders, or view-only shares' crumbs, as Files drop targets

Inside the Trash a folder row lit up as a drop target and then refused the
drop with an untranslated "Cannot move items into a deleted folder.", and a
crumb inside the Trash took the move, though only Restore may take an item
out of the Trash. A crumb of a folder shared with the user to view only
lit up too, and the drop ended in one refusal per item from the server.

Such folders no longer light up, and a drop on one goes back, as a drop on
a view-only folder row already did. A view-only crumb still opens on a
dwell.

* fix(dashboard): count a new folder, not the old selection, in the Files footer

New Folder selects the new row, but the footer was updated before that and
kept describing what was selected before ("3 items selected"), as did the
phone's selection bar, whose buttons by then acted on the new folder. A
Shift+click also still extended from the old anchor. The footer now
follows the new selection, and the new row is the anchor.

* fix(dashboard): update the Files footer when Enter ends a rename

Ending a rename with Enter selects the row, and then called the desktop
Explorer's footer update, which finds no footer in the dashboard. Renaming
a row from its menu while another was selected left the footer at
"1 item selected" with two rows selected.

* fix(dashboard): stop Files' image preview from running out of read URLs

The preview fetched a read URL for every image it showed, and read URLs
share the access-token allowance of 20 an hour. Stepping through a folder
of images with the arrow keys spent it in seconds, after which the preview
(and any app asking for a read URL) failed for the rest of the hour. The
preview now reads the image directly and shows it from memory, freed when
it closes or moves on, and a held arrow key loads only the image it stops
on.

* fix(dashboard): don't offer to publish a folder the user may not share from Files' folder menus

The background, crumb and sidebar menus offered Publish as Website for any
folder, so in a folder shared with the user to view or edit it opened the
publish dialog only for the server to refuse. Publishing takes the right
sharing does; the item now stays disabled without it, as on a folder row's
own menu.

* fix(dashboard): open a Files folder under its new name when it's opened mid-rename

Double-clicking a folder right after typing its new name (the first click
sends the rename) opened it under the old path, which the rename then
took away: the view sat in a folder that no longer existed, with crumbs
and Back/Forward disagreeing about its name, and New Folder or Upload
there failed. Opening a row (double-click, tap, Enter, or Open from its
menu) now waits for a rename still on its way, and goes by one path.

* fix(dashboard): keep Files' Back/Forward in step with folders moved or trashed

Back and Forward kept a folder's old path after it was moved or deleted,
here or from elsewhere: going into a folder, back out, and dragging it
somewhere else (or deleting it) left Forward leading to "This folder
couldn't be opened", with New Folder and Upload gone. History entries now
follow a moved folder and what is inside it, and a trashed or deleted
one's entries are dropped.

* fix(dashboard): keep screen readers from reading Files' icon file names

The crumbs' separators, the folder sidebar's icons and every row's icon
were images with no alt text, so a screen reader announced each one, by
file name in some ("triangle-right.svg"), between every crumb and before
every name. They are decorative beside the text they sit next to and are
now marked so. The crumb of the folder on screen is marked as the current
page.

* fix(dashboard): make double-click on a Files column divider fit the column

Double-clicking a column divider measured the text in system-ui, but the
listing renders in Inter, and took the cells' padding as 8px a side where
Size and Modified have 10. Right after "fit to content", the longest name
was still cut in the middle and every Modified cell ended in an ellipsis
("40 minutes a…"). Each column is now measured in its cells' own font and
padding.

* fix(dashboard): keep a renamed item among its sharer's in Files → Shared

With Shared sorted by "Shared by", renaming an item shared with the user
to edit placed its row by name alone, so it could land among another
person's items until the listing was loaded again. A row put back in
place now goes by its sharer first there, as the listing does.

* fix(dashboard): keep every item, and empty folders' places, in a downloaded zip

Downloading several items as one zip keyed each by its name, so two items
with the same name (in Files → Shared, two people can each share a
"Notes.txt") came out as one file, the other silently gone. Such names
now get " (1)" and so on in the zip, as a folder does for a duplicate.

Empty folders were put at the top of the zip by their own name, not where
they sat: zipping a folder holding x/y gave "y/" next to "x/". They now
keep their place.

dedupedName moves to helpers/, now that the zip uses it as well.

* fix(dashboard): give Files grid tiles both lines of their name

A tile has room for two lines of name, but names were cut to fit one:
"Quarterly financial report final version.pdf" showed as
"Quarterl…l version.pdf" over an empty second line. A name now shows whole
when it wraps into the two lines, and is cut in the middle, keeping its
extension, only as far as it has to be to fit them.

Where the browser wraps a name can't be told from text widths alone, so
the tiles are written and read back together, one layout a round rather
than one a tile.

* fix(dashboard): keep the Files listing when a refresh can't reach the server

A refresh of the folder on screen that failed for any reason replaced the
listing with "This folder couldn't be opened." and took New Folder and
Upload away. Returning to the tab triggers one, so opening a laptop before
its Wi-Fi reconnected, or a moment's lost connection during a re-sort, left
a folder that was fine looking broken until it was opened again. A refresh
that doesn't reach the server now leaves the rows as they are; one the
server answers with an error (the folder is gone) still says so.

* fix(dashboard): don't drop files added to a Files folder while a new folder is made

While New Folder (or New › file) waited for the server, every item added
to the folder on screen was dropped, not only the one being made, so a
file uploaded from another tab or device in that moment didn't show up
until the folder was listed again. Such items are now held until the
create is done and added then; the create's own comes through as an
update of the row already drawn for it.

* perf(dashboard): refresh Files on return to the browser tab only when Files is showing

Coming back to the browser tab listed the Files folder again even with
another dashboard tab in front, and in Files → Shared that meant every
share all over again. The refresh now waits until Files is next shown.

* fix(dashboard): move the keyboard focus off a Files arrow that disables itself

Pressing Enter on Back to reach the first folder disables Back while it has
the focus, and a disabled arrow is drawn at 10% opacity, focus ring
included, so the keyboard user lost track of where they were. The focus
now moves on to an arrow that still works, or to the listing.

* fix(dashboard): show "can't drop here" when dragging files from the computer where Files takes none

Files dragged in from the computer showed the copy pointer everywhere in
Files, then dropping on the Trash, in Shared, on a folder shared to view
only, or while a folder was loading did nothing. The pointer now shows the
drop isn't possible there.

* fix(dashboard): let Files' type-to-select find names with "i" in a Turkish browser

Typed letters were lower-cased for the browser's language and the names
they're matched against without one. In Turkish, "I" lower-cases to "ı",
so typing it never reached "Invoices". Both are now lower-cased alike.

* fix(dashboard): bring Files' Back/Forward along when a folder rename is undone

Renaming a folder points Back and Forward at its new name, but undoing the
rename left them there: Forward then led to "This folder couldn't be
opened" for a name the folder no longer had. Undo now takes them back to
the old name too.

* fix(dashboard): don't shrink Files → Shared's name column when it's dragged wider

In a narrow Shared listing, "Shared by" is laid out narrower beside a name
column with no set width. Dragging the name's divider gave it a width, at
which "Shared by" sprang back to its own and pushed the name narrower than
it started: dragging right made it 30px smaller, and there it stuck.
Sizing the name now keeps "Shared by" at the width it had on screen.

* docs(fs): note that move and copy never overwrite a folder holding the source

* fix(dashboard): keep the Files drop zone after files dragged in cross a folder row

Leaving a folder row while dragging files in from the computer also took
the highlight off the listing around it, as the row's notice bubbled up to
the listing's own. With the "can't drop here" pointer going by that
highlight, a drop on the listing after crossing a folder row was refused.
The row's notices stay with the row, and the listing goes by whether the
folder takes uploads.

* fix(dashboard): don't let a Files folder wait for ever on a rename to open

Opening a row waits for a rename on its way, which settles in the last
step of the rename's handling. A failure after the rename itself (such as
refreshing the folder's websites) never got there, and the folder then
wouldn't open until it was listed again. Opening waits five seconds at
most.

* docs(fs): drop the note on overwriting a folder that holds the source

The refusal is an edge case, and its error message already explains it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-25 18:59:24 -07:00
sabrimete 18f5c9f2c7 fix(ai-chat): admit only chat-capable models from the Alibaba catalog
The feed admission gate now applies uniformly: declared capabilities
must include a chat capability (TG, Reasoning, VU), request modality
must include Text, and response modality must include Text. Signal-less
entries are admitted only when they match a curated model.
2026-10-25 21:07:31 +00:00
sabrimete f0f97a08cf fix(ai-chat): give uncurated Alibaba catalog models a puterId
Uncurated entries admitted from the DashScope catalog feed previously
surfaced with their raw feed id. They now get an
alibaba:<vendor>/<model> puterId, vendor-prefixed ids keep their org,
and others infer the org from the curated family sibling. Entries whose
synthesized puterId collides with a curated entry are skipped so the
model is not listed twice.
2026-10-25 21:04:28 +00:00
Daniel Salazar 24aac2f6d2 fix: app token (#4338) 2026-10-25 11:50:25 -07:00
Daniel Salazar 1eb4e36cf4 fix: add beta flag to kv share docs to match kvhandle beta status (#4337) 2026-10-25 09:17:12 -07:00
Daniel Salazar 3160b99f11 chore: bump ver (#4335) 2026-10-25 08:24:40 -07:00
Daniel Salazar 589a98b12c feat(kv): share KV prefixes with other users at watch/read/write, and bound what apps can be granted (#4331)
* feat(kv): share key-value prefixes with other users, and bound what apps can be granted

- KV share handles move out of Events into KvShareService, served at /kv/shares
  (the /events/kv-handles routes stay as an alias).
- Shares have watch, read and write levels, encoded in the permission root
  (kv-share, kv-share-read, kv-share-write) with write implying read and watch.
- Every puter.kv method takes optConfig.handle; keys are relative to the shared
  prefix, cursors are sealed, and private entries stay hidden.
- New puter.kv.share, unshare, listShared and listSharedByMe; optional label;
  kv.share.received notice in the app's mailbox only; recipient blocks apply.
- puter.kv.set gains onlyIfAbsent.
- fs and kv events gain viaWorker, and kv events report self from the writer.
- /auth/grant-user-app only accepts canonical permissions from a fixed set of
  families; the consent dialog, puter.perms and docs follow.
- Migrations: kv_share_handles gains revoked_by, label, and grantee and app
  indexes (mysql_mig_49, postgres_mig_38, sqlite 0095).

* fix(kv): never name a private key in a cross-app or handle cursor, and bound flat-permission scans

- Listings of another namespace (cross-app or through a handle) resume after
  the last entry they returned, and their cursors are sealed, bound to the
  namespace or handle, and padded to one length. Old plain cross-app cursors
  are still read when they name the target namespace; other namespaces get 400.
- PermissionStore scans flat permissions a few roots at a time and skips
  roots nested under another root in the same call.
2026-10-25 08:16:19 -07:00
Toshit ChawdaandReynaldi Chernando ec3ad1cb2f docs: easy/medium/hard networking recipes (#4040)
* add net category + http request recipes

* add tcp/tls socket recipes

* add complex recipes

* minor refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-25 19:22:45 +07:00
Reynaldi ChernandoandDaniel Salazar c7cc64e45c docs: medium events recipes (PUT-1971) (#4333)
* docs: easy events recipe (PUT-1961)

* docs: catch-up-on-notifications events recipe (PUT-1961)

* docs: events recipe corrections from a run against a live server (PUT-1961)

* docs: medium events recipes (PUT-1971)

* docs: simpler wording in medium events recipes (PUT-1971)

* refine

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-25 19:18:55 +07:00
Nariman Jelveh 0c67499dc2 fix(dashboard): serve stale profile pictures to every caller, skip detached roots
While a picture refreshed after its TTL, only the first caller got the
stale answer; every later one waited on the network, so in Files → Shared
one row showed the picture at once and the rest fell back to initials.
paintAvatars now also folds a frame's roots into one pass and skips roots
re-rendered away before the frame, so they fire no lookups.
2026-10-24 19:05:54 -07:00
Nariman Jelveh 34d74d36bd feat(dashboard): show profile pictures on people avatars (#4328)
* feat(dashboard): show profile pictures on people avatars

Avatars that name a person now show that person's profile picture once
it loads, and keep the initial-and-color avatar when there is none:

- Files → Shared: the "Shared by" cell and the people filter chips.
- Share dialog: the owner row and each person with access. Teams,
  invited emails and link shares keep their initial.
- Properties: a new Owner row with avatar for items someone else owns,
  and avatars beside each version's author.

Lookups go through one cache: each person is looked up once and shared
by every avatar of theirs, answers (including "no picture") are kept
for five minutes, at most 100 are kept, and at most four run at once.
Calls made while a listing renders row by row fold into one pass per
frame. The signed-in user's own picture is read from their loaded
profile, so it takes no request and follows changes in the Account tab.
The picture is set as an image's src property, never through markup.

* fix(dashboard): profile picture review — stale refresh, picker avatars, limiter

- Serve a stale picture while a fresh one loads, so a listing re-rendered
  after the cache expires no longer drops everyone back to initials.
- Show pictures on people in the share recipient suggestions, matching the
  rows of the dialog they open from.
- Free a lookup slot when a lookup throws before returning a promise.
- Move profilePictures.js to helpers/, now that a shared helper uses it.
2026-10-24 19:01:05 -07:00
Nariman Jelveh 0bdc2197b0 feat(dashboard): show who shared each item in Files → Shared (#4327)
* feat(dashboard): show who shared each item in the Files tab's Shared view

The Shared view now names the person behind every item:

- List view gets a sortable, resizable "Shared by" column with the
  share modal's initial-and-color avatar. The tooltip names the owner
  too when someone else re-shared the item.
- Grid tiles show the sharer under the item name.
- Narrow listings lead each row's second line with the sharer.
- A people filter above the listing (one chip per sharer, with counts)
  narrows the view to one person. It shows once two or more people have
  shared something, and scrolls sideways on small screens.

Sorting by "Shared by" applies only in the Shared view; other folders
keep sorting by name while it is the saved sort.

* fix(dashboard): keep the name column readable beside "Shared by"

In a listing between the phone layout and about 700px wide, the fixed
160px "Shared by" column squeezed an auto-width name down to a few
characters. Beside an auto-width name, "Shared by" now takes at most 40%
of the room the two split, and resizing it stops at the same width.

* fix(dashboard): keep the "Shared by" header on one line

Narrowed, its two-word label wrapped onto a second line that spilled out
of the 32px header row. It now stays on one line and ends in an
ellipsis, as longer translations need too.

* fix(dashboard): fit "Shared by" to its longest name on double-click

Auto-fitting the column measured names in a font the cell doesn't use
and left out part of its padding, so the longest name still ended in an
ellipsis. It now reads the font, padding, avatar and gap off a rendered
cell.

* fix(dashboard): keep the people filter on coming back to Shared

Opening a folder from a filtered Shared view and going Back reset the
filter to Everyone, so going through one person's folders meant picking
them again each time. The filter now lasts while the user browses what
was shared with them, and clears once they go back to their own files.

* fix(dashboard): update Shared after Remove from Shared

Remove from Shared only took the row away, leaving the people filter
counting the item. Removing the last item of the person filtered to left
a blank pane under a chip that still claimed one. Shared is now listed
again afterwards, which updates the filter, the empty notice and the
footer.

* fix(dashboard): show a notified share past the people filter

Opening a "shared with you" notification, or a share link, while Shared
was filtered to someone else picked out nothing: the item was filtered
away. The filter now gives way to everyone's when it hides an item the
notification or link names.

* fix(dashboard): truncate a long sharer name on phone-width rows

In the two-line row layout, a long username grew the details line past
the row, pushing the date off screen and under the ⋯ button. The name
now ends in an ellipsis, and the line keeps clear of the button as the
name line does.

* perf(dashboard): sort the Shared view by sharer with one collator

Each sharer comparison called localeCompare with options, which builds
a collator per call: sorting 2,000 rows took about 25ms in Chrome. One
shared Intl.Collator orders them the same in under 1ms.

* fix(dashboard): make share avatar initials readable for every name

The initial-and-color avatar drew its letter at too little contrast for
nearly half the hues a username can map to: yellows and greens came out
near 3:1 in light mode, and lower still on a selected row. The letter is
now darker in light mode and lighter in dark mode, reaching 4.5:1 for
every hue on the page, selected rows and the share suggestions' hover
row. The share dialog and the Shared listing change together, so a
person keeps one look.

* fix(dashboard): show every folder when a drag springs into Shared

Dragging an item out of a shared folder and holding it over Up, Back or
the Shared crumb opens Shared to drop into a folder there. The people
filter, which now lasts while browsing shared items, could hide those
folders, and its chips can't be pressed mid-drag. A drag arriving in
Shared now sees everyone's folders.

* fix(dashboard): keep filtering Shared when it can't be listed again

A chip click lists Shared again so its rows are current. When that
request failed, the listing was replaced with "This folder couldn't be
opened" and the chips disappeared, though only a filter had changed. A
filter change now applies to the rows already loaded when the listing
can't be fetched.

* fix(dashboard): return a drag let go over Shared to where it began

A drag that springs into Shared and is let go over the listing rather
than a folder tried to move the items into Shared itself, which is a
query and not a folder: nothing moved, nothing said so, and the user was
left in Shared. Let go over a listing that takes no items, the drag now
goes back to the folder it started in, as the cancel zone does.

* fix(dashboard): don't offer read-only shared folders as drop targets

Folders shared read-only lit up under a drag and took the drop, which
the server then refused with "Forbidden". They no longer light up, and
a drop on one sends the drag back where it began; they still open on a
dwell.

* fix(dashboard): slide a drag back when a read-only folder refuses it

A drop a read-only shared folder refused left the drag chip to vanish
where it was let go, as if something had happened. It now slides back
to where the drag began, as any drop that doesn't land does.
2026-10-24 18:10:27 -07:00
Nariman Jelveh 7759f7afe9 fix(dashboard): seventh adversarial review — bugs, UX, a11y, perf (#4326)
* fix(dashboard): show notification times in Brazilian Portuguese, Ukrainian and Traditional Chinese

The notification panel and toasts handed Puter's language code straight to
Intl, and a few of those codes aren't BCP 47 tags. Brazilian Portuguese
(`br`) is Breton to Intl, so times read "5 munut zo"; Ukrainian (`ua`)
fell back to the browser's default language; and `zhtw` threw, so Taiwan
users got English. Those codes are now mapped to pt-BR, uk and zh-TW
before formatting.

* fix(dashboard): count "yesterday" and "N days ago" in calendar days

Notification times rounded the elapsed hours to whole days, so at 23:30
something from 00:45 the same day read "yesterday", and on Wednesday
morning something from Monday night (34 hours earlier) read "yesterday"
too. Days are now counted from midnight to midnight in the reader's time
zone; anything from today stays in hours.

* fix(dashboard): don't let deploy results crowd notifications out of the panel

The panel listed the 30 newest notification rows and then dropped the
ones it never shows, a worker's deploy results among them. A developer
with 30 deploys since their last share saw "No notifications" and no
badge while the share sat behind them. The listing now asks for up to
200 rows and keeps the newest 30 the panel shows.

* fix(dashboard): keep every unread notification in the panel

The socket replays up to 200 unread notifications on connect, but the
panel's listing held only the 30 newest, and the first refresh after that
dropped the rest: 45 unread read "45" until the panel was opened, then
"30", and "Mark all as read" left the other 15 unread on the server for
the next visit. The panel now keeps unread notifications past the newest
30, up to the 200 a listing returns.

* fix(dashboard): let Undo take back a new Files folder or file

The Files tab creates folders and New › files itself rather than through
the desktop's helpers, and never recorded them for Undo. Undo then
reversed whatever came before: upload notes.txt, create a folder, press
Ctrl+Z, and notes.txt was deleted for good (undoing an upload deletes it
outright, not to Trash) while the new folder stayed. Both now record
themselves, so Undo removes what was just created, as on the desktop.

* fix(dashboard): let Undo take back a Files cut and paste

Pasting cut items in the Files tab moves them itself rather than through
the desktop's move_items, and recorded nothing for Undo. Undo after a
paste reversed the action before it instead, which deletes an upload for
good. The paste now records the move as move_items does, so Undo puts
the items back where they were cut from.

* fix(dashboard): don't drop a Files selection onto one of its own folders

With folders A and B selected, dragging A onto B moved A into B and then
stopped on "Cannot move a directory into itself" for B. Hovering B also
sprang it open mid-drag. A folder row that is part of the drag is no
longer a drop target, so the drag reverts as it does over any other
non-target. A drop elsewhere (a sidebar folder, say) also leaves out the
target folder itself and any folder that holds it.

* fix(dashboard): don't drop a Files refresh asked for during another

A refresh of the folder on screen was ignored while another refresh of
it was loading. Drop two files: when the second finished during the
first's refresh, its own refresh was dropped, the earlier listing (read
before the file existed) was drawn, and the file's item.added was
ignored meanwhile too, so it stayed missing until a manual refresh.
Paste, Refresh and returning to the browser tab could lose items the
same way. A refresh now takes over from one in flight, as opening
another folder already did; a navigation in progress still holds.

* fix(dashboard): keep the phone selection bar after cancelling a delete

In select mode on a phone, tapping Delete permanently and then Cancel
left every item selected but hid the floating action bar, the only way
to act on the selection, until the selection next changed. The same
happened when nothing in the selection could be moved to Trash. The bar
now goes away only once something is actually deleted or trashed.

* fix(dashboard): show the phone selection bar when Select starts with a selection

Uploaded or pasted items arrive selected. Tapping Select on a phone then
turned select mode on with them selected but no floating action bar,
until a row was toggled. Entering select mode now brings the bar up for
the selection it starts with.

* fix(dashboard): give Files footer sizes in the rows' format

The footer used the desktop's byte formatter while the rows use their
own, so a 5-byte file read "5 B" in its row and "5.00 Bytes" below it,
1.5 KB read "1.50 KB", and a folder of folders read "0 Byte". The footer
now formats sizes as the rows do.

* fix(dashboard): title a trashed folder's Properties with its name

Properties from the breadcrumb of a folder opened inside Trash was
titled with the folder's uid, the last segment of its trashed path,
while the crumb itself showed the original name. It now uses that name.

* fix(dashboard): show delete and log-out confirmations without entities

The Files tab's "delete these items permanently?" confirmation and the
dashboard's "close your apps and log out?" one passed HTML-encoded i18n
strings to UIAlert, which encodes its message and button labels again.
In Igbo both questions read "Ị ji n&#039;aka …". These alerts, and the
Files tab's name-conflict and folder-depth ones, now pass plain strings.

* fix(gui): read the Trash's state fresh when updating its icon

refresh_trash_state stats the Trash to pick the empty or full icon, and
its comment calls for a strong read, but it asked for an eventual one.
Once a listing of Home had cached the Trash entry (without `is_empty`),
the stat came back without it: permanently deleting the last items from
Trash in Files left the full-Trash icon up, and other tabs were told the
same. The stat is now strong, as intended.

* fix(dashboard): don't add a Files row to a folder opened meanwhile

A file added from elsewhere (an upload, another tab, an app) is checked
against the folder on screen and then drawn once its icon resolves. A web
link's or app's icon is read from the file, and opening another folder
during that read put the row into the new folder's listing, where it
stayed and opened the file from the old one. The checks now run again
once the icon is in.

* fix(dashboard): let a focused folder or crumb take the Menu key

The Files tab answered the Menu key and Shift+F10 with the selection's
menu wherever focus was. With a file selected and a sidebar folder, a
breadcrumb or Home's Shared row focused, the key opened the file's menu;
with nothing selected it cancelled the browser's contextmenu, so the
focused element opened none. A focused control now gets the browser's
contextmenu event, and its own menu, as with a right-click.

* fix(dashboard): don't drop New Folder during a Files refresh

New Folder did nothing while the folder on screen was refreshing (after
an upload or paste, or on returning to the browser tab). A refresh keeps
the rows up with no spinner, so the click just looked ignored. It now
takes over the refresh and makes the folder once the listing is in;
during a navigation to another folder it is still dropped, as before.

* fix(dashboard): step through Files names on a repeated letter

Type-to-select meant to move past the selected row to the next name with
the same start, but returned early whenever the selected row matched, so
the stepping never ran: with readme, report-a and report-b, pressing "r"
again stayed on readme, and "rr" matched nothing. A repeated letter now
steps through the names it starts, wrapping around, as file managers do;
a longer prefix still stays on a selected row it matches.

* fix(dashboard): open one touch menu per long-press

A long-press on Android fires both contextmenu and taphold. The Files
rows already ignore the second, but a breadcrumb, a sidebar folder,
Home's Shared and Trash rows and the listing's empty space each opened a
second sheet over the first; picking an item or tapping the backdrop
then left the other one up. The touch menu now doesn't open while
another is on screen.

* fix(dashboard): treat Ctrl+click on a Mac as a right-click in Files

On a Mac, Ctrl+click opens the context menu, but the Files rows read it
as a Ctrl-toggle first: Ctrl+clicking one of three selected files
deselected it and the menu then acted on that file alone, and on an
unselected file it was added to the selection. On a Mac it now leaves
the selection alone, as a right-click does; Cmd+click still toggles.

* fix(dashboard): keep arrow keys in the open phone drawer on Apps

With the drawer open over the Apps tab, an arrow key moved focus out of
it to the first app tile behind the scrim, and the next Tab then walked
the page behind it. The open drawer now keeps the arrow keys, as the
notifications panel and dialogs already do.

* fix(dashboard): act once on a held Enter or Space in Apps

Holding Enter on an Apps folder opened it, moved focus to its first app,
and the key's repeats then launched that app; held on the "+" tile, the
repeats picked "Browse the App Center" in the dialog it opened, and held
in the search box they launched the first match again and again. Key
repeats now do nothing on a tile, in the search box or on the add-app
dialog's options.

* fix(dashboard): don't blank the Apps grid when it re-flows on resize

When a resize changed how many columns or rows fit (resizing the window,
collapsing the sidebar, zooming, an Android keyboard), the grid was
rebuilt behind the first load's fade, which waits on page one's icons.
With an icon host that was slow or unreachable the grid went blank for
up to 1.5s each time. A grid already on screen now re-flows at once;
one still loading keeps its fade.

* fix(dashboard): open an app from the add-app dialog at once during a search

"Browse the App Center" and "Build with AI" play an intro from the
app's tile, and wait up to 3s for that tile to appear. With a search
that leaves the app out (the case the "+" tile stays for: a search that
found nothing), the tile never appears, so the app opened only after the
full 3s. The intro now gives up as soon as the search is what hides the
tile, and the app opens straight away.

* fix(dashboard): make the Apps search's clear control a button

The "×" that clears the Apps search was a bare SVG with a click handler:
Tab couldn't reach it, screen readers had nothing to announce, and on a
phone it was a 16px target inside the field. It is now a labelled button
after the field in the Tab order, with a focus ring, and on touch it
hit-tests at 44x44 while keeping its look.

* fix(dashboard): tell screen readers when an Apps search finds nothing

Typing in the Apps search re-renders the grid silently, and the "No apps
match your search" line is drawn into it fresh each time, which screen
readers don't announce. A search that matched nothing gave no feedback
at all. A persistent status region now says so.

* fix(dashboard): keep keyboard focus on an Apps tile across a re-render

The grid is rebuilt wholesale whenever its data or size changes: the
refresh that follows opening the tab, a resize or zoom that changes how
many columns fit, an uninstall that fails. A keyboard user who had
arrowed onto a tile lost focus to <body>, so Enter did nothing and the
next arrow started over from the first tile. Focus now moves to the
same app's or folder's new tile.

* fix(dashboard): don't offer Change username to a temporary account

A temporary account's Account tab offered Change username, whose dialog
asks for an account password the account doesn't have: left empty it
said "All fields are required", and anything typed got a 403 from the
server, which refuses temporary accounts. The button is now left out
until the account is saved, as the Password card already is; saving it
re-renders the tab with the button.

* fix(dashboard): don't offer Change email to a team seat

A team seat's Account tab offered Change email, and after the user
filled in the new address and their password the server refused it:
"Your team set this address. Ask a team admin to change it." The Email
card now shows that note in place of the button, as the Username card
already does for seats.

* fix(dashboard): don't let a slow picture removal undo a newer picture

Saving a profile picture is ordered so an older save can't put its
picture back over a newer one, but Remove picture stood outside that
order. Clicking Remove on a slow connection and then saving a new photo
before the removal answered left the default avatar on screen, and
Remove hidden, while the account had the new photo. A removal now
yields to a picture saved after it.

* fix(dashboard): show placeholders in the Usage headers until they load

The Storage and Resources headers started out as " ·  used of " with the
numbers left blank, and stayed that way when reading the storage space
or the month's usage failed. They now start at "--% · -- used of --",
as Home's cards start at "--", and keep that if the read fails.

* fix(dashboard): write Home's trial dates in the dashboard's language

The free-trial note on Home formats its dates with the browser's
language, not the one chosen in Puter, so with Puter in French and the
browser in English it read "…le Mar 5", and the other way round "First
charge on 5 mars". The dates now follow the dashboard's language. The
notification panel's mapping of Puter's language codes to Intl tags
(Brazilian Portuguese, Ukrainian, Traditional Chinese) moves to
i18n/intlLocale.js so both use it.

* fix(dashboard): write credit and dollar amounts in the dashboard's language

Credits were always formatted with English separators and dollar amounts
with "$" and English separators, so in German Home read "5,000
monatliche Credits", which a German reader takes as five, and Usage
showed "2.5" and "$1,234.50". Both now use the dashboard's language
("5.000", "2,5", "1.234,50 $"); English is unchanged.

* fix(dashboard): say why Create token is off where everyone can read it

When the account's email isn't confirmed, Create token is disabled and
the reason ("Verify your email address to create an API token.") lived
only in a hover tooltip: a disabled button isn't reachable by Tab, a
phone has no hover, and screen readers don't read a plain span's title,
so those users met a dead button with no explanation. The reason now
replaces the card's description while the button is off. The tooltip,
which encoded the already-encoded text again, now uses it as is.

* fix(dashboard): announce that a new API token is shown only once

After a token was created, focus moved to Done and a screen reader read
"Done, button" and nothing else: the done state arrives already filled
in a status region, which isn't announced, and Done is described by the
message only for plain-text results. The token dialog's warning ("Copy
this token now — it won't be shown again…") now describes Done, so it
is read out when focus lands there.

* fix(dashboard): show an email change confirmed in another tab

Confirming a change of email from its link sets the new address and
marks it confirmed, but sends no socket event, unlike confirming a code.
Back in the dashboard the user was re-read on returning to the browser
tab, yet nothing redrew: Account kept the old address under the
username, Create token stayed disabled and Security had no 2FA card
until a reload. Account and Security now redraw after that re-read when
the address or its confirmation changed.

* fix(dashboard): close the share dialog after leaving a share

Someone the owner let share could remove their own row in the share
dialog to leave. The removal went through, but the dialog then reloaded
its access list with the access just given up: the server refused it,
the success line gave way to "Subject does not exist" over an empty
list, and the item stayed in the Shared view until a reload. Leaving
now closes the dialog and refreshes the folder on screen, which drops
the item, as "Remove from Shared" does.

* fix(dashboard): say the access list couldn't load, not that sharing failed

When the share dialog couldn't read who has access (a network error,
which carries no message of its own), it said "Could not share this
item." though nothing had been shared or attempted. It now says "Could
not load who has access."

* fix(dashboard): don't pull focus back after a share change saves

When a change in the share dialog finished saving (a level change, Add
to all, a removal, a share), focus was put back on that row's control or
the add field, wherever the user had gone meanwhile. Someone who changed
a person's level and then started typing a name had the next keystrokes
land on that person's level select, where a matching letter picked
another level and granted it. Focus now goes back only when the save
took it away; focus the user moved elsewhere stays there.

* refactor(gui): move the localized time formatting to i18n/formatTime.js

formatRelativeTime and formatAbsoluteTime lived in the notification
center's module; Files and Properties are about to use them too.

* fix(dashboard): write Files and Properties dates in the dashboard's language

The Files tab's Modified column and Properties' Modified, Created and
version times came from timeago, which has no language but English
registered, so a German dashboard read "3 hours ago" beside "Geändert".
They now use the same localized formatter as the notification panel
("vor 3 Stunden"), which also gives a date past a week rather than
"3 weeks ago". Its formatters are now made once per language, since a
listing formats one date per row.

* fix(dashboard): style the Usage table's Show less button in dark mode

The button's dark variant was written for an `html.dark-mode` class that
nothing sets, so in dark mode it kept its light look: grey text at 3.4:1
on the dark card, a light border, and a light chip on hover. Its rules
move under prefers-color-scheme, as the table's fade overlay already
did.

* fix(dashboard): show the Files breadcrumb separators in dark mode

The chevrons between breadcrumbs are a black stroke at 20% opacity,
which vanishes on the dark header: a nested folder read "Home Documents
Work" with nothing between the names. In dark mode they are now
inverted, like the toolbar's icons.

* fix(dashboard): show the website-link badge on dark Files tiles

In grid view a file inside a published folder carries a link badge: a
black glyph on a disc of the page's background color. In dark mode that
is black on near-black and can't be seen. The badge now gets the white
disc the shortcut and worker badges already have.

* fix(dashboard): skip disabled Back, Forward and Up when tabbing in Files

The Files header's arrows stayed in the Tab order while disabled, drawn
at 10% opacity, which takes their focus ring with them. Tabbing into
Files, the first two stops (Back and Forward, disabled on arrival)
looked like nothing had focus. A disabled arrow now leaves the Tab
order, as a disabled button does, and returns once it can be used.

* fix(dashboard): give the Delete account button an AA-contrast red

The Account tab's Delete account button used the desktop's danger red
(#f00808), which puts its white label at 4.4:1, under the 4.5:1 that
WCAG AA asks of text. It now uses the dashboard dialogs' danger red
(#d90707, 5.3:1) and their darker hover, so the two match.

* fix(dashboard): label collapsed sidebar items for keyboard users too

With the sidebar collapsed, each item shows only its icon, and its name
appeared in a tooltip on mouse hover alone. Tabbing through the sidebar
showed a row of icons with no names. The tooltip now also shows while
an item has keyboard focus.

* fix(dashboard): keep Share and Properties inside the visible screen

Both dialogs capped their height at 85vh (88vh as phone sheets), and vh
is the viewport with the browser's bars hidden. With the bars showing,
on a phone in landscape say, the card plus the overlay's padding was
taller than the screen, and the overlay centers without scrolling: the
title and the buttons were cut off with no way to reach them. Each card
is now also held within its overlay, as the dashboard's dialogs are,
and its body scrolls instead.

* fix(dashboard): outline the off state of the 2FA switch

Off, the switch was a light grey track (#e0e0e0) with a white thumb,
1.3:1 against a white card and 1.2:1 against the amber card the 2FA
toggle sits on while 2FA is off, so the control could barely be seen
where it matters most. WCAG asks 3:1 for a control's boundary. The off
track now carries a 1px slate outline (4.3:1 on the amber card); the
on state is unchanged.

* fix(dashboard): make the keyboard focus ring visible on sidebar, tiles and lists

The sidebar items, the Files folders, crumbs and arrows, the app tiles,
Home's cards and recent apps, the notification rows and the Usage sort
buttons show keyboard focus with a 2px outline and nothing else, drawn
in --select-ring: the selection color at 40% (50% dark), about 1.6:1
against the surfaces it sits on. Keyboard users could hardly tell where
focus was; a focus indicator needs 3:1. Those outlines now use the solid
selection color (3.7:1 on white), as the switch and pager dots already
do. Field halos, which come with a colored border, are unchanged.

* fix(dashboard): bring text in the selection blue up to AA contrast

The selection blue is 3.7:1 against white, and a few places set text in
it or white text on it, under the 4.5:1 that text needs: the active
folder in the Files sidebar, the share dialog's "Add to all" link, the
add-app dialog's Send button, the notification count badge, the drag
count badge, and a breadcrumb whose menu is open. They now use the same
hue darkened to 45% (5.3:1; a lighter one for text in dark mode), from
two new tokens, --select-text and --select-fill.

* fix(dashboard): give the Apps page dots a 24px touch target

The Apps pager's dots, and a folder's, are 17px buttons side by side,
under the 24px a target needs when it has neighbors, so on a phone or
tablet a tap meant for one page often opened the next. On a touch
screen each dot is now 24px around the same 7px dot; mouse layouts are
unchanged.

* fix(dashboard): return from a drag-opened folder while it is still loading

Hovering a dragged file over a folder springs it open. Cancelling the
drag (Escape, the Cancel zone, or a drop outside the listing) is meant
to go back where the drag started, but while the sprung folder was still
loading the return was dropped like any navigation during another, and
the user was left in the folder they had backed out of. The return now
takes over from that load.

* fix(dashboard): drop into a drag-opened folder while it is still loading

When a dragged file springs a folder open, the drop anywhere in the
listing moves it into the folder now shown. Until that folder's listing
arrived, though, the Files tab still counted the folder the drag left as
the one on screen, so a drop in that time was taken as a move to where
the file already was and did nothing. It now goes to the folder being
opened.

* fix(dashboard): say "Incorrect password" when the account password is wrong

The account dialogs showed the server's message as is. A wrong current
password in Change password read "Password mismatch", in English in
every language, beside a dialog that says "New Password and Confirm New
Password do not match" when those two differ, so users corrected the
wrong fields. A wrong code read "Incorrect code." and too many attempts
"Too many requests.", also always in English. These now show the
translated messages the GUI already has, and a wrong password puts the
status on the password field, focused for retyping.

* fix(dashboard): don't send an account change to the value it already has

Change username with the account's own name got "This username is
already taken." and used up one of the hour's password attempts. Change
email with the current address was accepted: it mailed a confirmation
and a change notice to the same inbox, and following the link unlinked
any sign-in provider. Change password to the current password signed
out every other session and mailed a "password changed" notice. Each
dialog now says the value is already the account's and sends nothing.

* fix(dashboard): ask for the new password after a provider-only account sets one

An account that signs in only through a linked provider can set a
password with Change password, after which the server treats it as a
password account. This session's copy still said provider-only, so
Change username, Change email and Delete account went on hiding the
password field and sending the user through the provider's popup until
a reload. The copy now changes with the password.

* fix(dashboard): let Escape cancel a session rename without closing Manage sessions

In Manage sessions (Security), Escape in a session's rename field
cancelled the rename and then closed the whole modal, and Escape in the
search box closed it too, search and all. Escape now cancels the rename
or empties the search, and only an Escape with neither closes the
modal.

* fix(dashboard): show the server's message when a session can't be revoked

Manage sessions put a refused request's raw response body in its alert,
so revoking a session already signed out from another tab read
{"message":"Session not found","code":"not_found"}, and the stale row
stayed listed; a failed rename and "Revoke all other sessions" did the
same. The alerts now show the message itself, and a session found gone
reloads the list.

* fix(dashboard): show 2FA as it is when its dialog closes without success

The Security tab's 2FA toggle assumed nothing had changed whenever its
dialog closed without success. With 2FA turned on in another tab, the
setup here refused ("2FA is already enabled."), the toggle went back to
off, and every retry got the same refusal, with Disable never offered;
a code accepted just as the user closed the setup dialog left the same
state. The tab now re-reads the account and shows its real 2FA state.

* fix(gui): leave Ctrl/Cmd+Z to the text field being typed in

Ctrl/Cmd+Z ran the file Undo wherever focus was, and cancelled the
field's own text undo. Typing a new name (F2) or anything in another
field and pressing it undid the last file action instead; when that was
an upload, the uploaded file was deleted for good. Undo now leaves text
fields alone.

* fix(gui): let Escape on a context menu close only the menu

Escape removed any open context menu and then went on to the Files tab's
own Escape handling, which, finding no menu left, cleared the selection
and closed the image preview. Select three files, open their menu,
press Escape: the menu closed and the selection was gone. When Escape
closes a menu, nothing else now acts on it; a second Escape clears the
selection as before.

* fix(dashboard): close the Files image preview when an app opens over it

The image preview sits above every window, and only a double-click
closed it on the way to an app. Opening the image with Enter, or with
Open or Open With from its menu, started the app underneath the
preview, which went on covering it while keyboard focus moved into the
app, so Space and Escape no longer reached the preview. It now closes
whenever an app window comes on screen.

* fix(dashboard): close the Files image preview when its file goes away

Deleting the previewed image (the Delete key, the menu, the phone
selection bar), moving it, or having it removed from another tab took
its row out of the listing but left the preview up, still showing the
image. The preview now closes with its row; a refresh of the folder,
which redraws the row, keeps it.

* fix(dashboard): download one selected file as itself from the phone selection bar

In select mode, the floating bar's Download zipped whatever was
selected, so a single photo arrived as "photo.jpg.zip" where its own
menu's Download gives "photo.jpg". One selected file now downloads as
itself; folders and several items are still zipped.

* fix(dashboard): don't offer New Folder, Upload or Paste in a view-only share

Inside a folder shared with the user to view only, the Files tab still
showed New Folder and Upload, offered New and Paste in its menu, and
took files dropped on the listing or on a folder in it; each then
failed: an upload window ending in an error, or a new folder row with
its name field open, then taken back with a Forbidden alert. Those are
now left out there, as in Trash. A share whose level isn't known yet
stays as before, the server deciding.

* fix(dashboard): move through Files with Home, End, Page Up and Page Down

The Files listing took the arrow keys but not Home, End, Page Up or Page
Down, and as the listing itself never has focus the browser didn't
scroll it on them either: in a folder of hundreds of items the keyboard
could only step through them one row at a time. Home and End now select
the first and last item, Page Up and Page Down move a screenful, and
Shift extends the selection as with the arrows. With a menu open they
are left to it, as the arrows are.

* fix(dashboard): apply the hidden-files setting to a renamed Files row

With Show hidden off, renaming notes.txt to .notes.txt left the row in
the listing until the folder was next loaded, and with it on the row
wasn't dimmed as other dot-files are; renaming .env to env left it
dimmed. A rename made in another tab did the same. A renamed row now
follows the setting as listing the folder would.

* fix(dashboard): open a recent app once on a double-click

Double-clicking an app under Home's "Recently used" opened it in two new
browser tabs, one per click. The second click of a double-click is now
ignored, as on the Apps tab's tiles.

* fix(dashboard): don't say "No recent apps yet" before Home's list has loaded

Home's "Recently used" card was drawn from the list as it stood when the
page opened, which is empty until the request answers: every visit
first read "No recent apps yet / Apps you use will appear here" and
then swapped in the apps, and if the request failed the card kept
saying there were none. It now stays blank until the list arrives, and
a first load that fails says the apps couldn't be loaded; a list
already shown stays when a later refresh fails.

* fix(dashboard): drop "Save session" when the account was saved in another tab

A temporary account saved in one browser tab stayed temporary-looking
in its other dashboard tabs: Home kept its "Save session" button, which
opened Save Account for an account already saved, and Account and
Security kept their temporary-account layout. Coming back to such a tab
re-reads the account; that now removes the button and redraws the two
tabs when the account is no longer temporary.

* fix(dashboard): let go of the phone drawer when the window widens

With the sidebar open as a drawer below 768px, widening the window past
it (an iPad turned to landscape) drew the sidebar as the desktop one but
left it marked open: Tab stayed trapped in it, so the content couldn't
be reached by keyboard, and the Apps grid ignored the arrow keys, until
an Escape or a click in the content. The drawer now closes when the
window widens, leaving focus where it is.

* fix(dashboard): don't let a #files/<path> link pull Files back to its folder

Files doesn't keep the folder on screen in the URL, but a link like
/dashboard#files/alice/Documents stayed in the address bar once opened.
After the user moved on to Pictures, went to Home and pressed Back, the
browser returned to that link and Files jumped back to Documents; a
reload did the same, and Files' own Back/Forward lost track, as the jump
wasn't recorded in it. The link's path is now consumed once opened, as
a share link's is, and a folder opened from such a link is a step in
Files' history.

* fix(dashboard): close the drawer and Files' dialogs on browser Back and Forward

Browser Back or Forward switched tabs but left whatever was open on
screen: on a phone, Android's Back changed the tab under the open menu
drawer and left it open; in Files, the Share or Properties dialog
stayed up over Home, still acting on Files rows; an open context menu
stayed too. These now close with the tab, as with a sidebar click.

* fix(dashboard): fill Home's storage and usage meters without waiting on the plan

Home read the plan first and only then asked for storage and the month's
usage, so every visit paid for one more round-trip before either meter
filled in. The plan request had no timeout either: one that never
answered kept both meters at "--" and, as later loads wait for the one
in flight, held every later refresh of the cards for the rest of the
session. The three now load side by side, and the plan read gives up
after 15 seconds.

* fix(dashboard): link Home's card headers to their tabs

The headers of Home's cards (Recently used, Usage, Your Plan, Your
Storage, Resources) linked to "#", so opening one in a new browser tab
(middle-click, the browser's menu) showed Apps rather than the tab it
names, and Cmd/Ctrl+click switched tabs in place. They now link to the
tab they open, and a modified click is left to the browser, as on the
sidebar.

* fix(dashboard): prefill the username when saving a session from Home

Home's "Save session" button opened Save Account with an empty username,
so a temporary user had to make one up, while the same action in the
user menu fills in the account's current name and starts at the email
field. Both now open it the same way.

* fix(dashboard): measure Home's storage meter by the amount it shows

On a self-hosted install the storage card names the disk's use (Puter's
files and everything else on it) but filled its meter, and gave its
percentage, from Puter's files alone, so it could read "93 GB used ·
0% of 466 GB". The meter and percentage now come from the same amount
the card shows.

* fix(dashboard): refresh Home's recently used apps on returning to the tab

Recent apps open in a new browser tab, and coming back to the dashboard
refreshed the usage cards but not "Recently used", whose comment said
it followed the same triggers: the app just opened only moved up the
list after switching away from Home and back. Returning to the tab now
re-reads the list too.

* fix(dashboard): leave IME keystrokes to the IME in Apps

Typing Japanese or Chinese in the Apps search or a folder's name, the
keys that drive the input method reached the tab's own handlers:
ArrowDown to pick a candidate moved focus to the first tile and cut the
conversion short, Enter to confirm one named the folder half-written or
launched the first match, and Escape put the old name back. The search
box's Enter meant to skip composition, but read isComposing off jQuery's
event, which never carries it. Keys during composition are now left to
the IME.

* fix(dashboard): leave IME keystrokes to the IME in the share recipient field

With a Japanese or Chinese input method on, even a Latin username is
composed, and the share dialog's recipient field took the keys that
drive it: the arrows moved through its suggestions instead of the IME's
candidates, and Enter to confirm the text picked a suggestion instead.
Keys during composition are now left to the IME.

* fix(dashboard): re-read the Apps layout on returning to its browser tab

The Apps tab read its order and folders only when it was opened, and
saves write its whole copy of them. With the dashboard open in two
browser tabs, a folder made in one was wiped as soon as the other,
showing Apps since before, changed anything (a reorder, a new folder, a
rename), as was the order arranged in the first. The tab now reads the
layout again when its browser tab comes back into view.

* fix(dashboard): undo a drag out of an Apps folder when the drag is cancelled

Dragging an app past the edge of an open folder takes it out, closes the
folder and carries the drag on over the grid, and the removal was saved
right then. Pressing Escape, or the window losing focus, cancels the
drag but left the app out; out of a two-app folder that dissolved the
folder for good, its name with it (dragging the app back made a new
"Folder"). A cancelled drag now puts the folder and the order back.

* fix(gui): show the 2FA setup's confirmations without entities

The 2FA setup window, which the dashboard's Security tab opens, encoded
already-encoded translations for its labels, so in French the two
checkboxes before enabling read "J&apos;ai enregistré mes codes…" and
"Je suis prêt à activer l&apos;A2F". The labels, and the window's other
translated text set the same way, are now encoded once.

* fix(gui): don't leave the 2FA dialogs stuck when a request can't be sent

Turning 2FA on or off from the dashboard's Security tab while offline,
or on a dropped connection, left the dialog busy for good: enabling kept
its spinner up with the code boxes disabled, and disabling kept its
button and password field disabled, with nothing said either way. A
request that gets no answer is now reported like a refused one, and the
dialog can be tried again.

* fix(gui): stop disabling 2FA when the provider sign-in doesn't happen

For an account that signs in through a linked provider, turning 2FA off
opens the provider's sign-in popup. When the popup was closed or
blocked, the dialog showed why for a moment and then sent the request
anyway, whose refusal ("OIDC revalidation required") replaced the
reason. It now stops at the popup's error.
2026-10-24 15:05:56 -07:00
404oopsandClaude Fable 5.1 68249028ef test(ai): assert replayed Kimi tool arguments reach the wire decoded once
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:59:04 +02:00
404oops cbcbdc2599 Merge branch 'fix/replayed-tool-call-arguments' into codex/kimi-k3-1-pro 2026-10-24 22:57:54 +02:00
404oopsandClaude Fable 5.1 3296c1b0d6 fix(ai): bill Kimi K3.1 Pro from reported usage at the published rates
Cache writes now meter under `cache_write_tokens` at the model's write
rate, split out of `prompt_tokens` by the shared usage splitter, so K3 gets
the same breakdown. The credit hold reserves the prompt and any replayed
reasoning at the input rate, and a stream that ends without a usage chunk
is charged the estimate like every other provider. The per-TTL response
headers, the 502 accounting error and the one-hour write prices are gone:
the cache is scoped to the account and nothing here requests a one-hour
TTL, so those writes never occur and Moonshot publishes no rate for them.

The catalog entries carry the release date so responses normalize by
default, the provider suite no longer imports the SDK, and the docs cover
the reasoning efforts, replay and cache usage keys in their existing
sections.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:55:10 +02:00
404oopsandClaude Fable 5.1 a10a9d9b93 fix(ai): keep replayed tool-call arguments serialized once
An OpenAI-shaped `tool_calls` entry normalizes with its `arguments` still a
JSON string, and hoisting it back into `tool_calls` stringified that string
again, so every OpenAI-compatible provider received a JSON string literal
instead of the call's arguments. Hoisting now sends a string `input` as is
and only serializes object inputs.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-24 22:54:39 +02:00
Filip KujundžićandClaude Fable 5.1 bbb780eb31 feat(ai): MiniMax video generation (H3 and Hailuo) (#4309)
* feat(ai): MiniMax video generation (H3 and Hailuo)

Adds a minimax-video-generation provider to the video driver. H3 models go
through MiniMax's v2 task API and are billed per second of output; Hailuo
models go through the v1 API, exchange a file id for a download URL, and
are billed per clip. The shared `minimax` chat key is reused, with the
chat block's `/v1` base URL reduced to its host.

WIDTHxHEIGHT sizes now map onto tiers for catalogs that include a 2k or
4k tier.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai): validate MiniMax video inputs and resolution tiers

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-10-24 13:49:21 -07:00
Juan Castro ad5f9b6196 fix(push): canonical app uid in the push consent check 2026-10-24 16:03:05 -04:00
Juan Castro 65b966c264 feat(push): notifications:push app permission
Apps need a per-app notifications:push grant for their notifications to
go out as pushes. hasPushConsent checks it as the app acting for the
user, so only that user's grant to that app answers yes, and a revoke
takes effect on the next check. The permission dialog describes it and
the perms docs list it.
2026-10-24 16:03:05 -04:00
Juan Castro c701b8d7c0 fix(notifications): show the sending app on app messages
App messages carry the sending app's uid and name (and its icon when it
is an https URL), set by the server; the desktop toast and the dashboard
panel show the name above the title. Title and text drop invisible
formatting marks and control characters, with newlines kept in text and
whitespace-only text left out. The icon is stored as its normalized URL,
and a target may not hold dot segments or non-ASCII characters. Both
rate-limit windows are read before either is spent, so a refused call
counts against neither. The docs say what ttl means for push delivery.
2026-10-24 16:02:54 -04:00
Juan Castro 4b41b9cd54 feat(notifications): apps can notify the user they act for
Adds the app.message type (audience app-user). An app, its workers and
the tokens it issues can create it through puter-notifications create,
only for their own user and under their own app. The value is validated
against a fixed format, the write is not silent, and each app has a
per-user budget of 30 per minute and 300 per day, configurable with
notificationAppMessageLimits. Holders keep creating the other types
silently and cannot create app.message.
2026-10-24 16:02:54 -04:00
Nariman Jelveh 8a55debdae fix(dashboard): sixth adversarial review — bugs, UX, a11y, perf (#4314)
* fix(dashboard): render in the user's saved language

The dashboard never read the account's preferences: only the desktop did,
on boot. Everyone saw it in English whatever language they had chosen,
until the Files tab's Show hidden toggle switched a running session over,
to the browser's language when none was saved, since that was the
fallback the toggle wrote back.

The desktop's preference loading moves into load_user_preferences, which
the dashboard now runs in the background. The first paint uses the
language this browser last loaded, so nothing waits on the read; a saved
language that differs reloads into it, and a changed Show hidden setting
refreshes the open folder. The tab names are read when the sidebar is
drawn rather than when the modules load. A failed read keeps the cached
preferences instead of resetting them to English.

* fix(dashboard): label credit amounts with a key meant for them

Home's allowance and the Usage tab's total and Credits column used the
`credits` key, which the translations render in its acknowledgments sense:
"5,000 Mitwirkende" (contributors) in German, "Авторы" in Russian,
"特别鸣谢" in Chinese. They now use `usage_credits`, which falls back to
English until it is translated, as `monthly_credits` already does.

* fix(dashboard): enable the token and 2FA cards once the email is confirmed

The dashboard had no handler for the `user.email_confirmed` socket event,
and Account and Security pick the token button's state and whether to
offer 2FA when they render. A user who confirmed their email in another
tab, or in this one through the confirmation prompt, kept a disabled
"Create token" and no 2FA card until a reload.

The dashboard now refreshes the user on that event and renders both tabs
again, as it already does after a temporary account is saved.

* fix(dashboard): save only the profile fields that were edited

Edit profile sent both the display name and the bio on every save, taken
from this session's copy of the profile. That copy comes from the cached
user and can be stale (the profile changed on another device) or empty
(the profile read failed), so changing only the bio wrote the old display
name back, and a save from an empty prefill erased the bio.

The dialog now sends just the fields whose value changed from what it
opened with, and closes without a request when nothing did.

* fix(dashboard): don't show storage as 100% while there is room left

Home and Usage rounded the storage share to a whole percent, so an
account at 99.5% or more read "100%" while uploads still fit. The usage
meter already holds at 99% until the allowance is spent; that rule moves
into meterPercent and the storage meters use it too.

* fix(dashboard): keep keyboard focus inside the open phone drawer

Below 768px the sidebar is a modal drawer over a scrim, but opening it
left focus on the menu button, and Tab past its last entry walked into
the dimmed content behind the scrim. Opening it now focuses the current
page's entry, and Tab and Shift+Tab cycle within it while it is open.

* fix(dashboard): tapping into a Files name field edits the name

On a phone or tablet, tapping the name field of a new folder or a rename
(which iOS needs before it shows the keyboard) reached the row's tap
handler, which opened the folder or launched the file and threw the edit
away. A long-press in the field opened the touch sheet instead of
selecting text, and on desktop a right-click in it showed the file menu
instead of the browser's Cut/Copy/Paste.

The row's tap and menu handlers now leave the name field alone.

* fix(dashboard): don't offer Cut on items the user can't move

A file's menu offered Cut on anything that wasn't immutable, and Shared
rows aren't marked immutable. Cutting a file someone shared with you, or
anything inside a read-only shared folder, then pasting it, ended in a
"Forbidden" alert and an emptied clipboard. Cut now needs the same
permission as Delete in that menu, as Ctrl+X and the selection's Cut
already do.

* fix(dashboard): don't paste or upload into the folder being left

Opening a folder clears the listing at once but only moves currentPath
when the new listing arrives. Ctrl+V, a file dropped from the computer
onto the pane, or Paste from the background menu during that wait went
into the folder the user had just left, and since the refresh then
showed the new one, the items seemed to vanish.

Those three now refuse while another folder is loading, as New Folder
already did.

* fix(dashboard): clear the drop highlight when Escape cancels a drag

Escape ends a Files drag with a synthetic mouseup, which jQuery UI
treats as a release outside every target: it fires no `out`, so the
folder row under the pointer kept its dashed drop outline and tint (and
a sidebar folder or breadcrumb its highlight) after the drag was gone.
The Escape branch now takes those off.

* fix(gui): show the "can't rename in Trash" alert without entities

Pressing F2 on a trashed item (on the desktop or in the dashboard's
Files tab) showed "This item can&apos;t be renamed because it&apos;s in
the trash": the message was encoded by i18n and then again by UIAlert.

* fix(gui): show the failed-upload title without entities

The upload progress window writes its error title with innerText, but
was handed the HTML-encoded i18n string, so French showed "Échec de
l&apos;importation des fichiers". The Files tab's two upload paths and
the desktop's now pass the plain string.

* feat(dashboard): open a Files row's menu with the Menu key or Shift+F10

Share, Download, Open With and Properties were reachable only by
pointer: the ⋯ button isn't focusable, and the keyboard's menu keys
fired a contextmenu on <body>, which the listing ignores. Both keys now
open the selection's menu at its row's ⋯ button, where arrow keys and
Enter already work.

* fix(dashboard): let Shift+Arrow shrink a Files selection

Shift+Arrow only ever added the next row, so after Shift+Down twice,
Shift+Up left all three rows selected instead of going back to two. A
run of Shift+Arrows now selects the range from the row it started on to
the cursor, on top of whatever was selected before the run, so moving
back shrinks it. A click or a plain arrow starts a new run.

* fix(dashboard): name folders in the Back/Forward history menus as the crumbs do

Holding Back or Forward listed each folder by its path's basename: `/`
was a blank entry, the Shared view read "shared", Trash stayed in
English, and a trashed folder showed its uid. The menus now use the same
names the breadcrumbs show.

* fix(dashboard): show a shortened Files name in full on hover

Grid tiles and a resized Name column shorten long names in the middle
("Quarterl…draft v2.xlsx"), and nothing showed the rest short of opening
Rename or Properties. A shortened name now carries the full one as its
tooltip.

* fix(dashboard): keep keyboard focus on the Files breadcrumbs

Every listing rebuilds the breadcrumbs, which took a keyboard-focused
crumb with it: pressing Enter on one, or any refresh of the folder while
a crumb had focus, left focus on <body>, and the next Tab started over
from the top of the page. Focus now moves to the same folder's crumb,
or to the one for the folder just opened.

* fix(dashboard): reach Home's Shared and Trash rows from the keyboard

At 480px wide or less (a phone, or a desktop browser zoomed in) the
folder sidebar is hidden and Home lists Shared and Trash as rows instead.
Those rows only had a click handler, so a keyboard user couldn't get to
either. They are now focusable links that open on Enter or Space, with
the sidebar's focus ring.

* fix(dashboard): keep the Files selection across a sort or view change

Sorting by a column, switching between list and grid, or toggling Show
hidden lists the folder again, and the rebuild dropped the selection and
the shift anchor: select three files, sort by Size, and nothing was
selected. Only the refresh on returning to the browser tab carried them
across. That logic is now refreshKeepingSelection, and those three use
it too.

* fix(dashboard): don't drop a folder open during a Files refresh

A refresh of the folder on screen (after an upload or paste, Refresh, a
sort, or returning to the browser tab) set the same "already rendering"
flag that a navigation sets, so double-clicking a folder, a sidebar
folder, a crumb or Back during that round-trip did nothing, with no
spinner to explain it.

Opening another folder now takes over from a refresh in flight; the
refresh notices it was superseded and drops its result. A navigation
during another navigation is still ignored, as before.

* fix(dashboard): keep the Files marquee when an item arrives mid-drag

Adding a row clears the empty-folder notice by removing every non-row
child of the listing, and the selection marquee is one. A file added
from elsewhere (an upload, another tab, an app) while the user was
drawing a selection made the rectangle vanish under the pointer.

* perf(dashboard): don't rewrite unchanged Files names while resizing

Every width change (a window resize, collapsing the sidebar) re-fits the
item names, at most once a frame, and wrote each name's text back even
when it came out the same. Each write invalidated layout for the whole
list. With 300 rows that took ~6.4ms a frame in list view and ~4.9ms in
grid; skipping the unchanged writes brings it to ~0.7ms and ~3.2ms.

* fix(dashboard): tell assistive tech the Files Select toggle's state

The Select button only toggled a class, so a screen reader announced a
plain button with no state; it now carries aria-pressed. Sort and View
open menus and now say so with aria-haspopup.

* fix(dashboard): show a person's strongest grant in the share list

When someone held two grants on one item (the owner gave them view
access, and someone the owner let share gave them edit), the share
dialog showed whichever grant the listing returned first, so their row
could read "Can view" while they could edit. A second grant on the same
item now counts toward the row with the stronger of the two levels.

* fix(dashboard): only offer share controls on grants the user can change

The backend lets an item's owner change or withdraw any share of it,
and anyone else only the grants they issued (or their own access). The
share dialog offered the level select and Remove on every row. Someone
the owner let share who removed a person the owner had added got
"Removed bob" while nothing changed, and lowering their level added a
second grant that changed nothing either.

The access list now records who issued each row's grants. A user who
doesn't own the selection gets the controls only on rows whose grants
they issued all of, plus Remove on their own row to leave; other rows
show their level as a tag.

* fix(dashboard): update Files' shared markers after the share dialog closes

The share dialog marks the Files rows it covers as shared or not from
the listings it reloads after each change, and skipped that whole reload
once it was closed. Closing it while a share or removal was still saving
left the row without its shared marker (or with a stale one) until the
folder was reloaded. The markers are now updated either way; only the
dialog's own list is skipped once it's gone.

* fix(dashboard): mark rows shared after sharing more than 50 at once

The share dialog updates the Files rows' shared markers from the share
listings it reloads after a change, and doesn't reload them for more
than 50 items. Sharing such a selection, with a person or by link,
reported "Shared with bob on 55 items" while no row showed the marker
until the folder was reloaded. Those grants now mark their rows
directly.

* fix(gui): unbind each context menu's mousemove handler with the menu

The submenu-aim code bound a mousemove handler on the document for every
menu and submenu and never removed it. Each one kept its closed menu and
items alive and ran on every mouse move for the rest of the session; ten
right-clicks in Files left five such handlers behind. The handler now
comes off when its menu is removed. Moving diagonally into a submenu
still keeps it open.

* fix(gui): ignore clicks on a context menu's dividers

A divider is an <li> like the items, so a click on one (they're 11px
tall, easy to hit between two items) ran the item handler with no
action behind it and threw "Cannot read properties of undefined". The
item handler now skips dividers; the menu stays open, as a native
menu's separator does.

* fix(dashboard): warn that removing a team removes everyone in it

Removing a team from a single item's share dialog asked "Remove Design's
access to this item?", the wording used for one person. The desktop's
share window adds "Everyone in the team loses it."; the dashboard now
uses that confirmation for teams too.

* fix(dashboard): count folders, not items, in "via N folders"

A person who reaches a multi-item selection through more than one shared
ancestor gets a "via N folders" note in the share dialog, and N was the
number of items reached that way: five files under two shared folders
read "via 5 folders". It is now the number of distinct folders.

* fix(dashboard): don't treat a long tail after a dot as a file extension

Files middle-truncates long names and keeps everything after the last
dot whole as the extension. For "Report v1.2 final draft for the board
meeting" that kept ".2 final draft for the board meeting", so the tile
read "….2 final draft for the board meeting" and lost the start of the
name. Only a short tail without spaces now counts as an extension; this
one shortens to "Report v1.…d meeting".

* fix(dashboard): keep Tab inside the touch context menu

The touch menu sheet is modal, but Tab and Shift+Tab left it for the
page behind the backdrop while it stayed open; closing it then no
longer returned focus, since focus was elsewhere. Tab and Shift+Tab now
move between its items like the arrow keys, wrapping at the ends.

* fix(dashboard): say "the other item" when one item is missing a grant

The share dialog's "Add to all" link told screen readers "Give bob
access to the other 1 items" when the person was missing from a single
item of the selection. That case now has its own singular string.

* fix(dashboard): announce a failed Properties load, on both tabs

When Properties couldn't read the item, the General list showed the
error but stayed aria-busy, so screen readers kept it unannounced, and
the Versions tab stayed blank. The list is no longer busy once the
error shows, and Versions shows the same error.

* perf(dashboard): launch the add-app modal's apps without reloading the grid

"Browse the App Center" and "Build with AI" go through the same intro
as a deep link, which first awaited loadApps(). With the grid already
loaded that started a fresh fetch of every app list and the saved
layout, and the launch waited on it: with an 800ms connection, ~1.35s
before anything opened instead of ~0.1s. The intro now uses the grid
already loaded, and only joins or starts a load when there is none.

* fix(dashboard): don't hide the Apps grid on every search keystroke

Search re-rendered the grid behind the first load's fade-in, which waits
for page one's icons. Each keystroke made fresh <img> elements, so an
icon that was slow or failing (an unreachable icon host) blanked the
whole grid until it settled, about a second per keystroke with a 1s
icon and up to 1.5s if it never answers. Search, its clear button and
entering reorder mode now render the results at once.

* fix(dashboard): stop the Apps pager's dots jumping back mid-flip

A page arrow, a dot, or the wheel starts a smooth scroll and marks the
target page at once, but the scroll handler then read the pages passed
over on the way as the current one. The active dot went forward, back
to the old page, and forward again, and a second click on the arrow
added to the stale page: two quick clicks showed 0 → 1 → 0 → 2 → 1 → 2.

While such a scroll is on its way, the pager (and the open folder's)
now ignores the positions in between, until it arrives or a second has
passed. The user's own scrolling still moves the dots.

* fix(dashboard): keep paging while a dragged app is held at the edge

Holding a dragged tile still at the Apps grid's edge (or an open
folder's) turned one page and stopped: the next dwell only started on
another pointer move, so a mouse held steady stayed on the second page
of three. Once a flip settles, the dwell starts again if the pointer is
still in the edge zone.

* fix(dashboard): don't turn Apps pages on a trackpad pinch

A pinch on a trackpad reaches the page as wheel events with ctrlKey
set, and the Apps grid (and an open folder) read them as a wheel flick
and turned the page while the user zoomed. Those events are now left
alone.

The handlers also called preventDefault() from a listener the app makes
passive, which does nothing and logged a console warning on every wheel
tick; that call is gone, as the scroller has no vertical scroll to stop.

* feat(dashboard): open the first Apps search result on Enter

Typing in the Apps search and pressing Enter did nothing; reaching a
result took ArrowDown and then Enter. Enter in a non-empty search now
opens the first match (or a folder that matches), as launchers do. It
does nothing while an IME composition is open, or when nothing matches.

* fix(dashboard): include an open folder's page dots in its Tab cycle

The open folder keeps Tab inside it, but its list of stops left out the
page dots, which are buttons. Tab never reached them, and once one had
focus (clicking it focuses it) the next Tab walked out of the modal
folder to the page behind. The dots are now part of the cycle.

* feat(dashboard): open an app tile's menu with the Menu key or Shift+F10

An app or folder tile's menu (Open in New Tab, Copy Link, Uninstall,
Remove from Folder; a folder's Rename and Ungroup) opened only on a
right-click or long-press, and browsers don't all turn the keyboard's
menu keys into a contextmenu event (macOS has none). The Menu key and
Shift+F10 on a focused tile now open its menu at the tile, where arrow
keys and Enter work.

* fix(dashboard): keep keyboard focus on the grid after Ungroup

Ungrouping a folder from its menu re-rendered the grid without the
folder's tile, so a keyboard user who had it focused was left on
<body> and the next Tab started from the top of the page. Focus now
moves to the folder's first app, which takes the folder's place.

* fix(dashboard): give links AA contrast in the light theme

Link-colored text (Home's "View more" and "Upgrade", the notification
panel's "Mark all read" and "Retry", Properties' website links, the
token dialog's sessions link) used #5271ff, 4.07:1 on white and 3.97:1
on the content background, under the 4.5:1 AA minimum for text that
size. The light theme's link is now #3d5bd9 (5.7:1), with a darker
hover. Dark mode already passed.

* fix(dashboard): give success text AA contrast in the light theme

Success text (the 2FA "enabled" line, Home's active-plan badge, the
share dialog's confirmations) used #03933a: 4.01:1 on white and 3.80:1
on its green tint, under the 4.5:1 minimum. It is now #15803d (5.02:1
and 4.75:1); the success check icons that share the token darken with
it. Dark mode already passed.

* fix(dashboard): give danger buttons and error text AA contrast

White text on the red fills fell short of 4.5:1: the Apps uninstall
confirm (#e53935, 4.23:1), the delete-account button (#f00808, 4.43:1)
and the Files drag "Cancel" zone (#ef4444, 3.76:1). So did light-theme
danger text on its pink tint (#dc2626 on #fef2f2, 4.41:1), used for
form, share, crop and notification errors and the past-due badge.

The fills are now #d32f2f, #d90707 and #dc2626 (4.8:1 to 5.3:1), each
with a darker hover, and danger text is #b91c1c (5.9:1 on the tint).

* fix(dashboard): give the app feedback dialog's Send button AA contrast

The feedback dialog's primary button (Send, and Close once sent) put
white text on a light-blue gradient, 2.66:1 at the top and 3.42:1 at the
bottom. It now uses the dashboard's primary button colors, which meet
4.5:1 in both themes, like the dialogs around it.

* fix(dashboard): show the drop outline and dwell fill on grid folders

Dragging files over a folder in grid view only tinted it like a
selection: the grid's own border, hover and selected rules out-ranked
the shared drop-target and spring-load rules, and the hover rule's
background shorthand wiped the dwell fill. The folder then opened after
700ms with no sign it was about to. Grid and compact-grid folders now
get the list's dashed outline and filling dwell, with the padding
trimmed to match so the tile doesn't shift.

* fix(dashboard): keep the Files preview and drag ghost above the window

The Space-bar image preview (z-index 9999) and the drag ghost (10000)
are appended to <body>, beside the dashboard window. On a touch-only
tablet with a keyboard every window is pinned to 9999999, so the preview
opened invisibly behind it from the first press; on desktop the
window's z-index climbs with each click and overtakes both in a long
session. Both now sit in the band the sidebar tooltip and the toasts
already use.

* fix(dashboard): give small dashboard controls 44px touch targets

Several controls a phone or tablet user taps were under the 44px
minimum touch size, with nothing enlarging them on a coarse pointer:
the Files toolbar buttons (32px), the touch menu's rows (36px tall),
the drawer's close button (32px), the notifications close (30px), the
Properties close (32px), the feedback dialog's ✕ (30px), the add-app
Back arrow (26px), the Apps modal buttons (34px tall) and the share
dialog's Copy link (38px tall).

Following the earlier touch fixes, the icon buttons keep their look
and get a 44x44 hit area, and the text buttons and menu rows grow to
44px. Mouse and trackpad layouts are unchanged.

* fix(dashboard): keep a landscape phone's notch clear of the dashboard

A phone held sideways and wider than 768px (an iPhone Pro Max, a large
Android) gets the desktop layout, and the page asks for the full screen
with viewport-fit=cover. Nothing in that layout read the side safe-area
insets, so with the notch on the left the sidebar's icons sat under it
(x=24 against a 47px inset), and with it on the right the content's
edge and the toasts did.

The sidebar (open or collapsed) now adds the left inset, and the content
area, including Apps and Files, and the toasts add the right one. The
insets are 0 everywhere else, so other layouts don't change.

* fix(dashboard): size the touch menu and bottom sheets to the visible height

The touch context menu, the app feedback dialog and the profile-photo
crop sheet capped their height in vh. On phones 100vh is the viewport
with the browser bars hidden, so while they show these could be taller
than the screen. The touch menu, which ContextMenu.js keeps inside the
visible height, then ran its last rows off the bottom on a landscape
phone, and the bottom sheets could push their header off the top. They
now cap in dvh, as style.css already does.

* fix(gui): move keyboard focus into the Manage sessions dialog

The sessions dialog is modal but never took focus: opening it from the
keyboard left focus on the button behind the backdrop, and Tab walked
the page underneath instead of the session list. It now focuses its
close button on open, keeps Tab inside, and hands focus back to the
control that opened it when it closes.

* fix(dashboard): return focus after the Security tab's dialogs close

Closing Blocked people, or the 2FA setup or disable dialog opened from
the switch, left keyboard focus on <body>, so the next Tab started at
the top of the page. The 2FA switch also disables itself while its
dialog is up, which drops focus even before then. Focus now goes back
to the button or switch that opened the dialog once it has gone.
2026-10-24 08:30:57 -07:00
Filip Kujundžić ce5f28e844 Validate Kimi completion accounting and preserve reasoning request semantics 2026-10-24 16:02:17 +02:00
Daniel SalazarandReynaldi Chernando f3e8539eaf docs: easy events recipe (PUT-1961) (#3998)
* docs: easy events recipe (PUT-1961)

* docs: catch-up-on-notifications events recipe (PUT-1961)

* docs: notification examples use notif:app-user, which apps can read

* docs: events recipe corrections from a run against a live server (PUT-1961)

* refine and update agents md

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-24 15:26:07 +07:00
Nariman Jelveh b3b02b1ebb Fix/dashboard fifth review (#4313)
* fix(dashboard): close the Apps modals and folders only on a press that starts on the backdrop

The uninstall and add-app modals and the open folder closed on any click
whose target was their backdrop. Drag-selecting text in the app request
box and releasing outside the card lands that click on the backdrop, so
the modal and the typed request were thrown away.

The backdrop logic the dashboard dialogs already use moves into
bindBackdropDismiss, and the three Apps overlays use it too.

* fix(dashboard): double-clicking "Add an app" no longer opens App Center

The add-app modal opens under the pointer, so the second click of a
double-click on the "+" tile landed on its first option and launched App
Center (or the AI builder, depending on where the tile sat) without the
user choosing anything. An option now ignores the second click of a
double-click.

* fix(dashboard): act on an app tile once per double-click

A tile acted on every click, so a double-click opened an external app's
site in two browser tabs. Double-clicking an option in the add-app modal
also launched whichever tile sat under the pointer once the modal had
closed (App Center and Recorder together, say). A tile now ignores the
second click of a double-click; the first already launched, focused or
opened what was asked for.

* fix(gui): show app launch refusals in a Puter alert, translated

launchApp reported "You don't have access to …" and "Couldn't open …"
through window.UIAlert, which nothing defines, so both fell through to
the browser's native alert(): a blocking, unstyled, English-only dialog
over the dashboard or desktop. They now use UIAlert directly, with the
text in en.js, and pass the app title unencoded since UIAlert encodes
its message itself.

* fix(dashboard): clicking a toast with nothing to open keeps apps on screen

A toast click brings the dashboard forward before acting on the
notification, hiding every app window. Only share notifications lead
anywhere, so clicking an "events subscription ended" toast (or any other
type) to dismiss it minimized the app the user was working in and showed
a dashboard with nothing to look at. Those toasts now just mark the
notification read.

* fix(dashboard): show an emptied Trash as empty

Emptying the Trash from the Files tab took its rows away but left the
footer on "N items · X MB" and the pane blank, without the empty-folder
notice; the selection bar kept offering actions on rows that were gone.
Emptied from another tab, the listing was cleared with .empty(), which
also dropped the loading overlay and likewise never updated the footer.

Both paths now finish with the empty notice and a fresh footer.

* fix(dashboard): say a folder is empty when its last item goes elsewhere

When another tab, device or app deleted or moved away the last item in
the folder Files was showing, its row faded out and left a blank pane:
the item.removed and item.moved handlers updated the footer but never put
the "folder is empty" notice up. They now do, for rows that were in the
Files listing.

* fix(dashboard): list failed drag-and-drop uploads one per line

The upload-button path was fixed to put each failed item on its own line,
but files dropped onto the listing still joined them with "\n", which
UIAlert's HTML collapses into spaces: every failure ran together on one
line. Both paths now build the message in failedUploadsMessage.

* fix(dashboard): dropping items back into their own folder does nothing

Dropping a selection on the sidebar entry for the folder it came from, or
spring-loading into a folder and coming back to release in the listing,
asked move_items to move every item where it already was, and each one
stopped on a "Cannot move item to its current location" alert in turn.
The breadcrumbs already ignore such a drop; moveDraggedRows now leaves
those rows out too, and a drop with nothing left to move does nothing.

* fix(dashboard): a Ctrl- or Alt-drop on the Trash trashes the items

Ctrl-dragging a file onto the sidebar Trash (or a Trash breadcrumb)
copied it into the Trash: the original stayed put, and the copy carried
no original path, so it was offered Restore and Restore failed. Alt-drops
made shortcuts there the same way.

The four drop handlers' copy/shortcut/move choice moves into
dropRowsInto, which only ever trashes what lands on the Trash.

* fix(dashboard): the first row click after a marquee released outside Files counts

A marquee sets a flag so the click that ends it doesn't clear the new
selection; the listing's click handlers consume it. Released over the
sidebar or outside the window, that click lands elsewhere, the flag
stayed set, and it swallowed the user's next click on a row. The flag now
lapses once the release's own click has had its turn.

* fix(dashboard): Back and Forward follow a folder the user renamed

Open a folder, go back, rename it, then press Forward: history still
held the old path, so Files showed "This folder couldn't be opened", and
the Back/Forward long-press menus listed the old name. A folder renamed
in Files now rebases the history entries in and under it, using the
same path rebase the username-change handler does.

* fix(dashboard): a folder that can't be opened becomes the current folder

When a folder failed to list (deleted elsewhere, share revoked, offline),
Files said "This folder couldn't be opened" but stayed on the previous
folder underneath: history had moved on while the breadcrumbs, sidebar
highlight and New Folder/Upload/Paste still pointed at the old folder,
so a new folder or upload landed somewhere other than where the user
seemed to be.

The tab now moves to the folder it was asked for before showing the
message, and creates nothing in it until it lists again.

* fix(dashboard): double-clicking Save in the photo crop uploads it once

Save closes the crop dialog, but the dialog stays in the page for its
200ms fade-out with its handlers live, so the second click of a
double-click ran Save again and sent the profile picture to the server
twice. Save does nothing once the dialog has closed.
2026-10-23 19:04:36 -07:00
Toshit ChawdaandClaude Opus 5 bdc9aa9e93 Revamp networking api with epoxy try 2 (#3476)
* revamp networking api with epoxy

* update to new epoxy build

* force wisp auth

* emit Error objects from socket 'error' events

main tightened the socket error contract to pass an Error (see
types/modules/networking.d.ts, which documents the reason as
`error.message`); the epoxy rewrite still emitted a bare string,
matching the older `(reason: string)` declaration.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* test: cover the epoxy-based networking client

Replaces the coverage lost when the wisp implementation was removed:
PSocket.test.js and PWispHandler.test.js were bound to PWispHandler,
parsers.js, and wispInfo, none of which survive the epoxy rewrite.

  PSocket.test.js  - connect/retry, inbound data, write normalisation,
                     close, and the TLS event remapping. Driven through
                     real ReadableStream/WritableStream pairs so
                     reader locking, cancel, and abort behave as they
                     do in the browser.
  index.test.js    - relay-token exchange (auth header, 401 re-auth and
                     retry, malformed responses) and the epoxy client
                     cache (keying, refresh, in-flight sharing).
  requests.test.js - pFetch delegation, cache invalidation, and the
                     api call logger's request description.
  epoxy.test.js    - the hand-packed wisp password extension payload.

The wasm runtime loader is left to integration coverage; it needs a
network fetch and a browser. createPuterPasswordBuilder is exported so
the byte layout can be tested against an injected fake runtime.

One skipped test records a pre-existing defect: on a failed write,
#readLoop's normal-termination path races #closeStreams and emits
close(false) after an error.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix: report socket close flags and epoxy init failures accurately

PSocket #readLoop raced the teardown it was being torn down by. A failed
write sets #closing and hands the close event to #closeStreams, which has
to await reader.cancel() and writer.close() before emitting close(true).
That cancel resolves the loop's pending read with {done: true}, so the
loop broke, reached its own #emitClose(false), and set #closed first --
callers were told the socket shut down cleanly right after an error.
The loop now defers the close event whenever a teardown is under way, in
both its normal and its throwing path, and #closeStreams always emits.

getEpoxyClient swallowed every init failure and resolved undefined, so a
dead relay reached callers as "cannot read properties of undefined
(reading 'connect')" instead of its cause. It now re-throws, still
without caching the failed attempt. A socket reports "wasm unavailable"
or "Failed to create relay token (HTTP 503 ...)" and closes with the
error flag set.

Two related cache faults fell out of that rewrite: the in-flight
early-return ignored `refresh`, handing a caller the very attempt it
asked to replace, and it ignored the cache key, so an attempt started
before a token change satisfied a request made after it. Reuse is now
conditional on both.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* restore the public API surface dropped in the epoxy revamp

The epoxy rewrite kept the shape of the old networking client but lost
three things callers depend on:

- JSDoc on the exposed methods. Re-added on PSocket (constructor, on,
  addListener, write, close), PTLSSocket and pFetch, along with the
  SocketEvent typedef and a Networking type on netAPI. `on` carries the
  TLS event aliasing docs that used to live on PTLSSocket, since that
  behaviour moved here.

- socket.write's error message, which lost its trailing '!!'. Apps match
  on the message, so it is restored verbatim.

- pFetch's client-side validation. Epoxy reports a bad request as an
  opaque wasm value from deep inside the transport, so the scheme check,
  the Content-Length/body check and the malformed-request TypeError are
  made before a relay is dialled, rejecting with exactly what they
  rejected with before. Validation builds its Request from a clone, or it
  would consume the body epoxy is about to send.

The latter two are what the net API suite asserts on node, browser and
workerd, and it has been failing on all three since the revamp.

addListener takes named parameters so it can be typed, and three unit
tests that pinned pFetch's habit of forwarding unvalidatable input were
updated to the documented behaviour.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* update epoxy client to 04e4930

connectionPrefs changed shape: the handshake and its required extension
list are one object now rather than a tuple, and handing the new build
the old form throws part way through the handshake.

The bump is not optional. The previous build cannot reach the current
relay at all: it enforces the required puter password extension even when
a relay answers with a v1 CONTINUE, and fails with "Protocol extensions
not supported" -- which is what puter.cafe answers with today.

Verified against a local relay: fetch, POST bodies, https, raw sockets and
TLS sockets, the last exercising connectTls, whose third parameter became
an options object in this build.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* bump epoxy

* dont require ext

* chore(puter.js): update Epoxy runtime to 68c6904

* chore(puter.js): update Epoxy runtime to affd9f5

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-10-23 16:57:25 -07:00
Nariman Jelveh 21c1e4142e fix(dashboard): fourth adversarial review — bugs, UX, a11y, perf (#4312)
* fix(dashboard): keep Escape working when two share or properties dialogs are open

Each share and properties dialog bound its document keydown under one fixed
namespace, so closing either instance unbound every instance's Escape.
Double-clicking Share… or Properties in the context menu opens two; after
the top one closed, Escape no longer closed the one left behind. Each dialog
now binds under its own namespace.

* fix(dashboard): don't close Properties when a text selection ends on the backdrop

Drag-selecting the path or UID in the properties dialog and releasing over
the dimmed backdrop closed the dialog: the click lands on the overlay when
press and release are on different elements. The backdrop now closes it
only when the press started there, as the share dialog already does.

* fix(dashboard): keep focus in Properties after disassociating one of several websites

With more than one website on an item, disassociating one removed the
focused button and returned early before refocusing the dialog, so focus
fell to the page and the next Tab landed in the dashboard behind the modal.
The dialog takes focus back as soon as the entry is removed.

* fix(dashboard): handle the Properties stat failure instead of leaving it unhandled

puter.fs.stat both calls the error callback and rejects. The properties
dialog rendered the error from the callback but dropped the promise, so
opening it on a deleted item or offline logged an unhandled rejection.

* fix(dashboard): don't double-encode the "anyone" and withheld-invite share rows

The share dialog encodes row names when it renders them, but these two
labels came from i18n() already HTML-encoded, so a translation containing
an apostrophe or ampersand would show its entity text. They're now taken
unencoded like every other row name.

* fix(dashboard): make the touch context menu reachable by assistive tech and keyboard

The touch action sheet had no roles and never took focus: it was appended
at the end of the page, so a screen reader announced nothing on open and
focus stayed on the page behind it. It's now a menu of menuitems (with
separators and submenu popups marked), takes focus on open, gives it back
on close, moves between enabled items with the arrow keys and Home/End,
and shows an inset focus ring for keyboard use. Taps draw no ring.

* fix(dashboard): keep Apps folders and order when a settings read fails

The Apps tab read the saved order, folders and uninstalled-apps list with
`.catch(() => null)`, so a failed read looked like a user who had never
saved any. One flaky request emptied the folders and reset the grid to
the default order, and the next drag or folder edit wrote that loss back
to kv for good.

A failed read now keeps what the session already has (the last
uninstalled-apps list included). Until a read of the order or folders has
succeeded, saves stay in memory instead of overwriting the stored value.

* fix(dashboard): drop the Apps tile a failed deep-link launch drew

Landing on /app/<name> (or choosing an option in the add-app modal) adds
the app's tile before the launch runs, because the launch is what installs
it. When the launch failed — no access, no app token, a thrown error — the
tile stayed for the rest of the session, survived refreshes, and every
click on it failed again.

The launch outcome is now passed to settleDeepLinkLaunch, which takes a
tile it drew back out (and restores a previous uninstall record) when
nothing was launched.

* fix(dashboard): put a re-installed app back in its saved slot

Uninstall keeps the app's name in the saved order so a re-install returns
it to its place, but a deep-link landing appended the tile at the end of
the grid. The install animation played on the last page and the tile then
jumped back to its saved slot on the next refresh. The landing now places
the tile through reconcileAppOrder like every other load.

* fix(dashboard): don't double-encode the add-app launch failure alert

UIAlert encodes its message, so passing it the already-encoded i18n string
showed entities on screen in languages with an apostrophe (French read
"s&apos;est mal passé").

* fix(dashboard): say so when an app tile fails to launch

A tile click whose launch rejected (for example the app lookup failing)
only logged to the console: the icon flourish played, settled, and nothing
opened. It now shows the same alert the add-app modal shows for that
failure.

* fix(dashboard): keep focus inside the Apps modals and return it on close

The uninstall and add-app modals are aria-modal but had no Tab trap, so
Tab walked into the search box and tiles behind the scrim, where typing
filtered the grid under the modal. Closing the add-app modal, and
confirming an uninstall, dropped focus to the page body.

Tab now cycles inside both cards; closing the add-app modal returns focus
to the add tile, and an uninstall leaves focus on the tile that takes the
removed app's slot.

* fix(dashboard): upload from a folder's menu into that folder

Upload in a sidebar or breadcrumb folder's menu opened the shared file
picker, whose handler always uploaded into the folder on screen: files
picked for Pictures while viewing Documents landed in Documents. The menu
now hands its folder to the picker.

* fix(dashboard): no New, Upload or Paste where nothing can be created

New Folder, Upload, the New/Paste/Upload menu items and file drops were
only refused in the Trash itself and the Shared view. Inside a trashed
folder they created items without the metadata Restore needs, and at /
every create failed (an upload there threw before its progress window
existed and showed nothing). One check now covers the Trash, folders in
it, the Shared view and /.

* fix(dashboard): make Paste Into Folder work after a cut

The row menu's Paste Into Folder handed a cut to the desktop's
move_clipboard_items, which finds items by their on-screen elements and
plain-string paths. The Files tab's cut stores {path, uid} objects, so
nothing moved and the clipboard was emptied. The menu now takes a paste
handler, and Paste, Ctrl+V and Paste Into Folder share one.

* fix(dashboard): don't cut or copy trashed items out of the Trash

The single-item menu still offered Cut on trashed items, and Ctrl+C
copied them; pasted elsewhere, they kept their raw name in the Trash, a
uid. Trashed items leave through Restore, which puts the original name
back.

* fix(dashboard): Delete Permanently works in every language

The confirmation compared the clicked button's label with 'Delete', so in
any other locale confirming did nothing. The buttons carry values now.

* fix(dashboard): refresh Files on return after a cancelled rename

Escape, or keeping a name unchanged, hid the rename editor but left its
-active class, and the refresh on returning to the browser tab takes that
class for an open editor: the listing stopped refreshing until the next
navigation.

* fix(dashboard): clear a drop target's highlight after the drop

jQuery UI fires no `out` on drop, so the folder row a selection was
dropped on stayed selected (and went to the Trash with the next Delete),
a sidebar folder stayed highlighted as the current one, and a breadcrumb
kept its drop outline.

* fix(dashboard): listing drops keep working after a drop on a folder row

Dropping files from the computer onto a folder row stopped the native
event before the listing's dragster saw it, leaving its enter state stuck:
later drags didn't highlight the listing, and drops on its background
uploaded nothing.

* fix(dashboard): offer Restore only for items Restore can handle

Selecting files inside a trashed folder offered Restore in the
multi-select menu and the selection bar, which then failed once per item:
only direct children of Trash carry restore metadata. The trashed
multi-select menu also drew two separators in a row.

* fix(dashboard): show names as typed in conflict and move-failure alerts

These alerts html-encoded the file name before UIAlert encoded the
message again, so "R&D.txt" read "R&amp;D.txt". "already exists." is
translatable now too.

* fix(dashboard): list failed uploads one per line

The failed-upload list was joined with newlines, which the alert collapses
onto a single line.

* fix(dashboard): translate the shortcut badge's tooltip

* fix(dashboard): select mode toggles rows with any pointer

In select mode, a stylus or mouse tap on a phone or tablet toggled the row
in pointerdown and again in click, leaving it as it was; in a narrow
desktop window a click replaced the selection and the selection bar never
showed. Select mode now toggles in click for every pointer and shows its
bar whenever it is on.

* fix(dashboard): show items added elsewhere while New › Web Link prompts

The New menu's create wrapper suppressed item.added for as long as it ran,
which for Web Link includes its URL prompt, so files added to the folder
meanwhile never appeared. It now counts only from the actual create. A
create that fails now says so instead of failing silently.

* fix(dashboard): keep the Files spinner under the mobile drawer

The loading overlay sat at the maximum z-index, so on phones it drew over
the open drawer and its scrim and took their taps while a folder loaded.
It only needs to sit above the rows.

* fix(dashboard): don't show a failed usage request as zero usage

getMonthlyUsage() resolves with the error body when the request fails, and
Usage and Home read it as a month with nothing used: $0.00 of $0.00, 0%,
and on Usage an empty table replacing the one already on screen. A body
without a usage report is treated as the failure it is, so the last good
numbers stay and a first load shows the "unavailable" note.

* fix(dashboard): draw Files select-mode accents in the selection color

The checked checkbox, the active Select button and the Done button were
filled with --primary-color, the desktop's text color rather than an
accent: charcoal by default, and white (on white text and ticks) for
anyone whose desktop theme uses light text. They use --select-color, the
accent the rest of the selection already uses.

* fix(dashboard): stop the mobile drawer animating under reduced motion

The reduced-motion override for the drawer, its scrim and the menu button
came before the drawer rules it was meant to cancel, and the open
drawer's rule outranked it too, so the drawer still slid and the scrim
still faded. The override now follows those rules and covers the open
state.

* fix(dashboard): make the toast close button readable in dark mode

Dark mode gave the close button the card color and then inverted the
whole element, which turned the disc light grey and the cross white:
about 1.4:1. Without the background the invert gives the light-mode
button's mirror, a dark disc with a white cross.

* fix(dashboard): leave the temporary-account state once the account is saved

Saving a temporary account from the user menu promoted the user in place
without a reload, but Home kept its Save session warning (reopening the
save dialog for a saved account) and Account and Security stayed without
their password and email cards until the page was reloaded. The dashboard
now reacts to account-saved, which fires after the user is updated.

* fix(dashboard): give the API token field 16px text on touch screens

The coarse-pointer rule that sets the token field to 16px, as the other
dialog inputs are, came before the field's own 13px rule of the same
specificity, so the 13px won and iOS zoomed the page when the field was
focused. The touch rule now follows it.

* fix(dashboard): lighten the General access dropdown arrows in dark mode

The dark-mode chevron override covered the share modal's recipient
selects but not the two General access ones, which kept the darker
light-theme stroke on the dark card.

* fix(dashboard): don't leave the sidebar collapse button shaded after a tap

Its hover style was the sidebar's only one not limited to devices that
hover, so on a wide touch screen (an iPad in landscape) the button kept
its hover shade after being tapped.

* fix(dashboard): refresh the Usage tab when the subscription changes

An upgrade from the Usage tab's own button, or a return from the billing
portal, refreshed Home behind it while the Usage meter and plan button
kept the old allowance until the user left the tab and came back. Usage
now handles puter:subscription:changed while it is showing.

* perf(dashboard): load usage and storage once when the dashboard opens

Home and Usage both load the monthly usage and the storage space when the
dashboard opens, whichever tab it opens on, so every open sent
/metering/usage and /df twice. Overlapping loads now share one request.
Nothing is cached: a load after the shared one settles fetches again.

* fix(dashboard): keep keyboard focus on the Usage table's Show more/less

The table redraws without the button that was pressed, so focus fell back
to the page and the next Tab started from the top. Focus now moves to the
button that replaces it, as the sort headers already do.

* fix(dashboard): give primary buttons readable contrast

The dialogs' primary buttons, the share modal's Share and the photo
crop's Save were filled with the link color under white text: 4.07:1 in
light mode, 3.08:1 in dark, where hover lightened it further to 2.38:1.
They take their own fill now, #4a67f5 (4.62:1) darkening to #3d5bd9
(5.68:1) on hover in both themes; links keep their color.

* fix(dashboard): keep collapsed-sidebar labels above the dashboard

The hover label is attached to body at z-index 1000, while the dashboard
window's z-index climbs with every click (focusWindow raises it). Over a
long session the window passed 1000 and the labels rendered behind it,
so hovering the collapsed sidebar showed nothing.

* fix(dashboard): bring muted and tertiary text up to AA contrast

The muted and tertiary greys carry real information (form hints, file
dates and sizes, share "via" lines, the sidebar's team name,
notification text and times) but fell short of 4.5:1: 2.56:1 and 3.54:1
on white, 3.18:1 and 3.13:1 on dark cards. They now clear it on the
cards and the sidebar in both themes (light #5f6b7d / #707070, dark
#8b95a5 / #94949c). Disabled buttons keep the fainter grey through their
own token, so they still read as disabled.

* fix(dashboard): name the plan on self-hosted installs instead of "--"

A self-hosted install has no /marketplace/subscriptions/current route, and
Home treated its 404 like any failed read, leaving "Your Plan" at "--" for
good. A 404 now shows the plan every account starts on. The upgrade badge
and link, which have nothing to reach there, go too, unless the hosted
upgrade dialog is loaded.

* fix(dashboard): make hover and press feedback visible in dark mode

Dark mode's hover color, #2a2a2a, was 1.05:1 against the #262626 cards
and #252525 sidebar, so hovering a notification or sidebar item and
pressing a touch-menu or drawer item showed next to nothing. #333 sits
as far off them as light mode's hover sits off white, and stays below
the selected item's #3d3d3d.

* fix(dashboard): follow a username change in the Files tab

Changing the username, or saving a temporary account under a chosen
name, moves the home folder. The GUI rewrote data-path attributes and the
path globals, but the Files tab kept its current folder, back/forward
history, clipboard and drawn rows on the old path, so sorting, New
Folder, Upload, Up, Back and the breadcrumbs all failed with "This folder
couldn't be opened" until a reload. update_username_in_gui now announces
the change, and the Files tab moves its paths over and redraws.

* perf(dashboard): lay out only the new row when an item is added to Files

Every item added to the open folder from elsewhere (another tab, an
upload, a move into it) restyled and re-truncated every row in the
listing, so a burst of adds was quadratic in the folder's size: 100 adds
into a 300-item folder took ~800ms of main-thread work. The added row now
takes the listing's column layout on its own (~130ms for the same burst).

* fix(dashboard): open the Sort and View menus at their buttons from the keyboard

Enter or Space on either button fires a click with no coordinates, so the
menu opened at the top-left corner of the screen, far from the button.
A keyboard press now opens it below the button; a pointer click still
opens it at the pointer.

* fix(dashboard): open a breadcrumb's long-press menu as the touch sheet

A breadcrumb's long-press opened the desktop context menu at the event's
pointer position, which a long-press doesn't have, so on tablets the menu
landed wherever an unset position put it. Breadcrumbs now open the same
touch sheet, anchored to the crumb, as the sidebar folders and rows do.

* fix(dashboard): don't drag-move items the user can't move

Delete, Cut and their buttons already leave out shared roots and items in
a read-only share, but dropping them on a folder, a sidebar entry or a
breadcrumb still sent the moves, and each came back as a "Forbidden"
alert. Drag moves now leave those rows out too; copying them still works.

* fix(dashboard): clear "1 item selected" when a double-click opens a file

Opening a file by double-click deselects its row, but the footer kept
saying "1 item selected" with nothing selected.

* fix(dashboard): only the Trash gets the Trash label in breadcrumbs

Any folder named "Trash" was shown with the translated Trash label in the
breadcrumbs, so in German ~/Documents/Trash read as "Papierkorb". The
label now goes on the user's Trash folder only.

* fix(dashboard): name a trashed folder by its old name in the breadcrumbs

A trashed item's name on disk is its uid, so opening a trashed folder
showed a crumb like "ffaff8cb-dcaf-44f7-…" while its row in the Trash
read "Old project". The crumb now uses the name the row shows.

* fix(dashboard): show why Delete Permanently failed and keep the item

delete_item fades the item out before the request; when the request
failed it left the item hidden, as if deleted, and passed the alert the
undefined `responseText` of a puter.js error, which threw. The item now
comes back and the alert shows the error's message.

* fix(dashboard): translate the New menu's Web Link and templates strings

"Web Link", its URL prompt, validation and failure messages, the "User
templates" submenu, and the account prompt before publishing a worker
were hardcoded English in the Files New and item menus.

* fix(gui): stop leaking a document click handler per context menu

Every context menu bound its item-click handler on document, delegated
to its own id, and never unbound it. Each menu opened therefore left a
handler that ran on every later click in the page and kept the menu's
items alive, rows and fs entries included. The handler is now bound on
the menu itself and goes with it. It still returns false, so item clicks
keep not reaching document-level click handlers, as before.

* fix(gui): run a context menu item once on a double-click

The menu stays clickable during its 200ms fade-out, so the second click
of a double-click ran the item again: double-clicking Share… or
Properties in the dashboard opened two stacked dialogs. A menu that has
started closing now ignores item clicks.

* fix(dashboard): drag a select-mode selection from its own rows

Select mode returned from pointerdown before recording whether the
pressed row was already selected, so a drag started from that stale
value: dragging selected rows didn't move them, and a stale "selected"
could let a drag from an unselected row clear the selection. The value is
recorded first now.

* fix(dashboard): leave Files select mode when the window widens past 480px

Select mode's Select and Done buttons only show up to 480px, so turning
it on in a narrow window and widening it left every click toggling rows,
with no way out short of narrowing the window again.
2026-10-23 15:24:02 -07:00
Neal Shah 03e483e372 PUT-2252 (#4311) 2026-10-23 17:33:59 -04:00
Nariman Jelveh 9c90c7afde fix(dashboard): third adversarial review — bugs, UX, a11y, perf (#4310)
* chore(dashboard): add the standard file header where it was missing

* fix(dashboard): rename editor keeps entity-like names; rows follow remote renames

The name editor decoded its already-decoded value a second time, so opening
it on a file such as "R&amp;D.txt" and clicking away renamed it to "R&D.txt".

A rename or update from another tab only patched a row's attributes, while
its open/copy/share handlers kept using the entry they were rendered with;
the row now keeps that entry current and refits its name.

* fix(dashboard): Files selection, preview and Trash/Shared edge cases

- A marquee drag without modifiers now replaces a selection made by click,
  Ctrl+A or the arrow keys instead of adding to it.
- With the image preview open, clicking another image switches to it, and
  moving onto a non-image closes it instead of keeping the old image up.
- Cut (Ctrl+X, multi-select menu) skips trashed items, which leave the Trash
  through Restore; Ctrl+V refuses the Trash and the Shared view like the
  Paste menu item does.
- Drops on the Shared sidebar entry or breadcrumb are refused (the drag
  snaps back) instead of sending writes to a path that isn't a folder.
- The Trash selection bar shows Delete permanently again after a selection
  elsewhere had hidden it.

* fix(dashboard): keep an open rename across refreshes; re-sort renamed rows

A same-folder refresh (an upload or paste finishing, a sort change) rebuilt
every row and silently dropped a rename in progress; the editor now comes
back on the same item with the text and caret the user left.

A renamed row whose new name sorts last stayed where its old name was.

* fix(dashboard): Files columns follow the room the listing has

- Resizing a column past the listing's width clamps it instead of resetting
  every column to the defaults; a window too narrow for the saved widths
  lays out at the defaults but keeps the saved widths for when it widens.
- Columns and grid names re-fit when the window or sidebar resizes the
  listing, not only on the next navigation, and grid names hidden behind
  another tab are fitted when the Files tab is shown.
- Names are measured in the font they render in (the compact grid's is
  smaller), and a column drag relayouts at most once a frame.

* fix(dashboard): Files keyboard and touch edge cases

- The marquee selects file rows only, not the Shared/Trash place rows Home
  shows on narrow screens.
- Row shortcuts stand aside while the touch action sheet is open.
- Breadcrumb long-press opens its menu on iPadOS too.
- A folder rendered before the saved view preferences load uses the
  defaults instead of throwing and blocking navigation.

* fix(dashboard): share and properties modals — focus, Escape, link state

Share modal:
- Escape pressed in a window opened over the modal (a verification or
  upgrade gate) no longer closes the modal underneath.
- Turning the link on or off for a selection too large to list now shows
  the result instead of leaving the control on its old state.
- Opening the link up starts at view access; it no longer reuses a hidden
  mode left over from earlier, which could hand out edit access.
- Focus returns to the select the user changed.

Properties modal:
- Takes focus when it opens, keeps Tab inside, gives focus back on close,
  and leaves Escape to a window opened over it.
- Shows a loading state, and opening Versions on a folder before its
  details load no longer leaves an empty body.
- Disassociate is a real button, ignores repeat clicks while pending,
  reports a failure, and keeps the website badge while other sites remain.
- Tabs carry tab/tabpanel roles; transitions respect reduced motion.

* fix(dashboard): account dialogs, avatar saves and usage figures

- A form dialog ignores Escape and backdrop clicks while its request runs,
  so the outcome isn't lost behind a closed dialog; the close button still
  works.
- A new API token, shown only once, survives a stray click outside the card.
- Copy token works on origins without navigator.clipboard (plain-http
  self-hosts) instead of throwing, falling back to a selected field.
- Saving one profile picture and then another before the first finished
  can no longer leave the first one's picture or failure on screen.
- The storage host bar is clamped to the room left beside Puter's share and
  hides its label when there is no host figure.
- Sub-cent dollar costs read "<$0.01" rather than "$0.00", and the usage
  meter reads 99% until the allowance is actually spent.

* fix(dashboard): Apps folder drag, search, keys and loading

- A drag inside an open folder that is cancelled (Escape, blur) or dropped
  back in place puts the tiles back in their saved order; the cached panel
  markup used to skip the rebuild, and the next drag then saved the
  cancelled move too.
- Search no longer matches substrings of generated app ids, so "app" or a
  single hex letter stops returning every website shortcut.
- Arrow keys and Enter stay with the notifications panel, an account dialog
  or the touch action sheet while one is open over the grid.
- The grid shows after 1.5s even if an icon host never answers.
- Refreshing a grid already on screen keeps the page and skips the fade.
- Rename from a folder's menu focuses the name on touch devices too, and
  the folder dialog's label follows renames and counts the apps shown.
- i18n inserts %% values literally, so a name containing $& or $' no longer
  comes out mangled.

* fix(dashboard): shell navigation, a11y and notification sync

- Missed notifications replayed on load toast as one burst (a few, then a
  summary) instead of one toast each.
- A notification read in another tab while this tab's first listing is in
  flight is no longer shown as unread when that listing lands.
- Focus stays in the notifications panel when Mark all read or Retry hides
  the button that had it.
- The active sidebar tab carries aria-current, and the account button is
  named by the username even while the collapsed sidebar hides it.
- Landing with ?ref= replaces the history entry, so one Back leaves.
- Ctrl/Cmd/Shift-click on a sidebar tab opens it in a new browser tab.
- Recently used keeps the newest list when refreshes overlap, and app names
  are URL-encoded in the links it opens.

* fix(dashboard): focus rings, dark-mode contrast and reduced motion

- App tiles show a focus ring, not only the hover enlargement; Home card
  headers get an inset ring the card's rounded clip can't cut off; the
  Delete account button gets the settings cards' ring.
- Pager arrows are mouse-only (the dots page by keyboard), so invisible
  arrows are no longer Tab stops, and an arrow with no page leaves the
  accessibility tree.
- Dark mode: the Files toolbar's action icons match the nav arrows and stay
  readable on the hover chip, hovered breadcrumbs keep their contrast, and
  the shared-item dot's ring follows the row background.
- The row '⋯' button is visible at rest on touch screens.
- A long username or email wraps inside the profile card.
- The touch action sheet, the Files selection bar and the app feedback
  modal stop scaling/sliding under reduced motion.

* fix(gui): desktop rows store remote renames raw; column resize stays in bounds

- The desktop's item.renamed/item.updated handlers wrote HTML-escaped names
  into data-name, the title and the name editor (and titled windows by an
  undefined uid). They store the raw name now, as rename_file does, so the
  editor shows the real name now that it no longer decodes a second time.
- Resizing a Files column while saved widths are too wide for the listing
  starts from the widths on screen, and a width is held at its column's
  minimum, so a drag or auto-fit can't save a negative width.
- Double-clicking a row closes the image preview, which would otherwise
  float above the app window it opens.
2026-10-23 12:32:33 -07:00
Filip Kujundžić 97beffb572 Make completed Kimi accounting failures terminal without estimated billing 2026-10-23 21:05:39 +02:00
Filip Kujundžić 03eab894a0 fix(ai): reserve Kimi cache writes and preserve tool arguments 2026-10-23 20:56:42 +02:00
Filip Kujundžić 53c77c8333 feat(ai): support Kimi K3.1 Pro models 2026-10-23 20:43:35 +02:00
Rasad RegmiandDaniel Salazar acd3df8c1b fix(puter-js): replay the request after a successful token_auth_failed reauth (#3954)
* fix(puter-js): replay the request after a token_auth_failed reauth

A 401 carrying the legacy `token_auth_failed` code opened the sign-in
prompt and then dropped the re-authenticated token on the floor:
`resolveReauth()` fell through to `return null` instead of returning
`{ action: 'replay' }`, so the original call was never re-issued and
rejected with `Unauthorized` even though the new session was valid.

Return the replay verdict the `reauth_required` branch already returns.
Both replay paths stay one-shot (`ctx.done.add('reauth')`,
`spec._replayed`), and the auth gate rejects before the route handler
runs, so a replay has no side effects.

Closes #3953

* fix(puter-js): sign in once when a replayed XHR is refused again

handle_resp ran the reauth flow on the replay's own 401, so a
token_auth_failed that survived the replay reset the fresh token and
opened a second sign-in before rejecting. Treat a replay's 401 as final,
as the 403 verification-gate branch already does.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-23 00:07:56 -07:00
Daniel Salazar 4729d733bb feat(aws): DynamoDB, Polly and Textract can use the default credential chain (#4307)
`useCredentialChain: true` in `dynamo.aws`, `providers['aws-polly']` or
`providers['aws-textract']` takes credentials from the default AWS chain
instead of static keys, as `s3.s3Config` already does. One provider is shared
and refreshes temporary credentials itself.
2026-10-22 21:31:23 -07:00
Nariman Jelveh 26d843778e feat(dashboard): account dialogs as dashboard modals instead of windows (#4305)
* feat(dashboard): account dialogs as dashboard modals instead of windows

Edit profile, change username, change password, change email, create API
token, and delete account in the Account tab (and change password in the
Security tab) now open as dashboard overlay cards, like the share,
properties, and confirm dialogs, instead of desktop UIWindows. They are
centered cards on desktop and tablet and bottom sheets on phones.

- dashboardOverlay.js: the overlay shell UIDashboardDialog already had
  (reveal, Esc/backdrop/close dismissal, focus trap, focus return), pulled
  out so the new form dialog shares it. It also pins the overlay to the
  visual viewport so iOS's keyboard can't cover a sheet's fields, and
  leaves Esc to a modal opened on top of it.
- UIDashboardFormDialog.js: labelled fields, password show/hide, an
  aria-live status line that marks and focuses the offending field, a busy
  submit button, and a done state.
- postUserProtected.js: the cookie-retry / provider re-sign-in / reauth
  flow the four account windows each carried, in one place.
- Deleting an account is one dialog now (warning plus confirmation) rather
  than two stacked windows.
- Touch: 44px controls, 16px inputs (no iOS focus zoom), no autofocus
  that pops the keyboard mid-animation, form buttons side by side on
  phones, safe-area padding dropped while the keyboard is up.
- The replaced windows had no other callers and are removed with their
  tests, which are ported to the new modules.

* fix(dashboard): a double-click that opens a dialog no longer dismisses it

The second press of the double-click lands on the freshly mounted backdrop
and closed the dialog straight away. Backdrop presses with a click count
above one no longer count toward dismissal.

* fix(dashboard): a failed form submit keeps focus in the dialog

Locking the fields disabled the focused control, which drops focus to the
page, so after an error Enter no longer retried and screen readers lost
their place. The control that had focus gets it back once the fields
unlock, unless the handler moved focus elsewhere in the dialog.

* fix(dashboard): a form error keeps the field it names in view

The status line scrolled the dialog body to the bottom, which on a short
screen (a phone with its keyboard up) left the field being focused and
marked invalid scrolled off the top. The body now scrolls back up just
far enough to show that field.

* fix(gui): a blocked re-sign-in popup fails instead of waiting forever

openRevalidatePopup polled a popup that was never opened, so when the
browser blocked it the account dialog stayed busy indefinitely. It now
rejects straight away and the dialog says the sign-in didn't finish.

* fix(dashboard): password show/hide reads as one action, not a pressed toggle

The button flipped its label between Show and Hide password and also set
aria-pressed, so assistive tech announced "Hide password, pressed" while
the password was showing. It keeps the action label and drops aria-pressed.

* fix(dashboard): screen readers hear a form dialog's success message

The done state arrives as a new status region that already holds its text,
which screen readers don't announce, so after a change succeeded they only
heard "Done, button". The Done button that takes focus is now described by
the message.

* fix(dashboard): requesting an email change no longer sets the cached email

The server only stages the new address until the emailed link is
followed, but the dialog wrote it into window.user.email straight away.
That cached a wrong address for the session and kept refresh_user_data from
updating the Account tab once the change was confirmed, since the two
already matched.

* fix(dashboard): form dialog labels drop the global label shadow

style.css's bare `label` rule gave the field labels a white text-shadow,
which ghosted them on the dark theme, and an extra 3px under each label.

* fix(dashboard): dialog inputs keep a 1px border when focused

style.css's global input :focus rule still set a 2px border width, so a
dialog field's text shifted by a pixel each time it took focus. The focus
ring is the box-shadow; the border stays 1px.

* fix(dashboard): phone dialog inputs stay 16px instead of jumping on focus

On phones style.css's .device-phone input rule outranked the dialog's
16px size at rest, so fields drew at 17px and shrank to 16px the moment
they took focus.

* fix(dashboard): temporary accounts can delete themselves again

The server reports oidc_only for every passwordless account, temporary
ones included, so the shared user-protected helper sent a temporary
account into the linked-provider sign-in, which has no URL for it, and the
deletion never went out. Temporary accounts post directly, as the route
admits them without credentials.
2026-10-22 17:13:50 -07:00
Daniel Salazar 80a9cd85ee fix(ai): chat requests run to their upstream usage report after the caller disconnects (#4302)
- Providers send upstream calls with a driver-owned `upstreamSignal`
  instead of the caller's `abortSignal`. A disconnect detaches the chat
  stream: output stops, the provider keeps reading until the upstream
  reports usage, and that usage is metered.
- A detached stream gets SDK_TIMEOUT_MS to finish. After that the driver
  aborts the upstream, and the stream is metered like any other stream
  that ends without usage.
- The disconnect check also runs right before dispatch.
- The OpenAI stream handlers report usage that arrived before an abort.
- Docs: stopping a stream stops the output, and a request that reached
  the model is billed in full. Detached streams get 10 more minutes.
2026-10-22 16:08:05 -07:00
Nariman Jelveh 63370fcfa5 fix(dashboard): mobile and tablet review — layout, touch, safe areas (#4303)
* fix(dashboard): give narrow Files listings the two-row layout

The two-row rows (name, then size and date) were keyed to a phone-width
viewport, so between 481px and ~930px, where the sidebar and folder column
take their share, the fixed size and modified columns squeezed names to a
few characters. The layout now switches on the listing's own width with a
container query, which phones still meet and wide listings never do.

* fix(dashboard): wrap the Files toolbar onto two rows in narrow listings

On tablets the nav buttons, crumbs and actions sat on one row wider than
the listing, so the sort, view, new-folder and upload buttons ran off the
right edge. Narrow listings now use the phone's two-row toolbar (crumbs
above, buttons below); only the menu-button clearance stays phone-only.

* fix(dashboard): keep a deep folder path on one row with the current folder in view

A long path wrapped the crumbs over several lines on top of the column
headers and pushed the toolbar actions off-screen; where the crumbs already
scrolled (phones), the row opened at the root, hiding the current folder.
The crumbs now scroll within their row and are scrolled to their end on
render and when the tab is shown.

* fix(dashboard): end clipped file names in the list with an ellipsis

With the name column auto-sized, which is always the case on phones and
tablets, a long name was cut off mid-letter with no sign it continued.

* fix(dashboard): keep the Files selection bar under the drawer and sheets

The bar's visible state set z-index 99999999999999999 (clamped to the
maximum), so in phone select mode it drew over the open navigation drawer,
covering the account button, and over the share sheet it opens. Its base
z-index already clears the rows and the footer.

* fix(dashboard): let the last Files rows scroll clear of the phone selection bar

In phone select mode the list's bottom padding only cleared the footer, so
the last row stayed behind the floating action bar even fully scrolled.

* fix(dashboard): keep the phone Files footer and selection bar above the home indicator

The installed app runs edge to edge, so the footer's item count sat under
the iPhone home indicator. The footer now grows by the bottom safe-area
inset, and the list padding and floating action bar move up with it, as
the Apps tab and the dashboard's bottom sheets already do.

* fix(dashboard): clear the home indicator in the item properties sheet

On phones the sheet sits flush with the bottom edge, and unlike the share
sheet it mirrors, its body didn't pad for the bottom safe-area inset, so
the last details scrolled under the iPhone home indicator.

* fix(dashboard): keep the app feedback sheet's buttons above the home indicator

As a phone bottom sheet it sits flush with the bottom edge, so Cancel and
Send ended under the iPhone home indicator.

* fix(dashboard): size the touch context menu as a card on tablets

The touch menu took 90% of the viewport, which on an iPad meant a ~920px
strip with its labels far from the row they belong to. It now stops at
400px, which every phone stays under. Its height is also measured after
the width is set, so the clamp to the viewport uses the real height.

* fix(dashboard): let the Apps modals scroll in a short viewport

The Add-an-app and uninstall cards had no height cap and their overlay
centers without scrolling, so on a landscape phone with the keyboard up
(the viewport resizes for it) the request form's title and buttons sat
off-screen with no way to reach them.

* fix(dashboard): align Account and Security with the other tabs on phones

Below 768px their wrapper added a 16px gutter inside the content area's
own, so both tabs sat 32px in while Home, Apps and Usage (and the menu
button above them) sit at 16px. The extra padding stays for Apps, whose
content area has none.

* fix(dashboard): keep Home's recent apps inside their card on tablets

Each recent app was a fixed 200px inside 1fr columns, whose minimum is
that width, so in the half-width card on a landscape iPad the second
column ran past the card's edge and its name was cut off with the
ellipsis out of view. The rows now shrink with their column.

* fix(dashboard): keep every Files folder reachable in a short viewport

Trash was fixed to the bottom of the viewport, so in a landscape phone or
a short window it sat on top of Public and Videos, while Shared ran below
the column with no way to scroll to it. The folder column now scrolls,
and Trash sits at its foot in the flow: in the same spot when there is
room, after Shared when there isn't.

* fix(dashboard): don't leave Account and Security buttons highlighted after a tap

Their hover background wasn't limited to hover-capable pointers like the
rest of the dashboard's hover styles, so on touch devices a tapped button
(Change Password, Manage Sessions, ...) stayed shaded after its dialog
closed.

* fix(dashboard): give the Files row menu button a full-size touch target

On phones the row's '⋯' is 40x23, right beside a row whose tap opens the
item, so a slightly-off tap opened the file or folder instead of its
menu. In the two-row layout it now has a 44px square hit area on touch
screens.

* fix(dashboard): enlarge the Files back, forward and up touch targets

They were 28px squares on touch screens. On coarse pointers they now take
36px with a matching negative margin, so the toolbar layout is unchanged.

* fix(dashboard): size Home's recent apps card to its apps on phones

Its 310px minimum height matches the square welcome card beside it on
wider screens. In the single-column layout the welcome card drops its
square shape but the minimum stayed, leaving a block of empty card under
a short list of recent apps.

* fix(dashboard): stop clipping app names' descenders on phones and small tablets

Below 768px the tile height budgeted 52px icon + 10px margin + 12px for a
label line that is 15.6px tall, so the bottoms of g, p and y were cut off
in every app name. The tile now has the full line (78px) and the row gap
gives back the 4px, so tiles sit exactly where they did.

* fix(dashboard): show Usage amounts in full on phones

Every Usage table cell ellipsized, numbers included, so on a phone a
token count like 12,345,678 read as "12,345...". Below 480px the units
and cost columns now keep their full width and the resource name is the
one that truncates. Wider layouts are unchanged.

* fix(dashboard): stack Home's recent apps when their card is narrow

In the two-column Home on a portrait tablet (iPad Air and Pro 11 at
820-834px, or 769px windows) the Apps card is about 200px wide, and two
columns of recent apps cut every name to a letter or two. The list now
switches to one column on the card's own width; 1024px and wider keep two.

* fix(dashboard): fit two grid tiles a row in narrow Files listings

Grid view's 170px column minimum left one 330px tile per row on 360px
phones (and in a tablet's narrow listing), so barely two files fit on
screen. Narrow listings now use two equal columns, and the tile's icon
box shrinks with the tile instead of keeping its fixed 150px.

* fix(dashboard): align the Shared and Trash rows' menu button on phones

Item rows get position: relative from jQuery UI draggable, but the Shared
and Trash rows above Home's folders aren't draggable, so their '⋯' was
positioned against the whole list and sat 3px off from every other row's.
2026-10-22 15:39:47 -07:00
Daniel Salazar 631b264c1c test(ai): every AI driver holds and meters at the AI cost factor (#4304)
Image, video, TTS, speech-to-text and voice-changer driver tests run one
request at factor 1 and again with every `ai.cost.factor.*` hook set to
2, and check that both the credit hold and the recorded cost double.
Chat and OCR already had this coverage. A driver that hands a provider
the raw metering service instead of its factored view now fails its
test.

`withAiCostFactorHook` in integrationTestUtil keeps the hook in place
for a callback, since providers record their usage unawaited.
2026-10-22 15:31:15 -07:00
Filip KujundžićandClaude Opus 5.5 809c303117 fix(ai): treat a gateway 4xx with no upstream error body as a route failure (#4301)
Infron (a new-api gateway) answers "bad response status code N" with the
upstream's status when the upstream replied without a JSON error, e.g. a
model the upstream no longer serves. That 4xx was classified as the
caller's bad request, so the request skipped fallback, the dead route
stayed in rotation, and the caller got upstream_bad_request. It now counts
as an upstream failure: fallback runs, the route is marked unhealthy, and
an exhausted chain reports upstream_provider_unavailable. A gateway error
that carries the vendor's own message still stays request-level.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-22 15:14:28 -07:00
VonypetandCursor afad53d3f4 feat(gui): add Filipino and restore the language menu (#4263)
The desktop built the language list and never placed it on the user menu, so a saved locale could not be chosen. Register Filipino and show that list.

Co-authored-by: Cursor <[email protected]>
2026-10-22 15:13:43 -07:00
Daniel Salazar 8b8dd19eae fix(teams): refuse to boot when teams_enabled has nothing registering Teams (#4300)
* fix(teams): refuse to boot when teams_enabled has nothing registering Teams

Core's seat checks (team 2FA, seat password reset, email and username
changes, self-delete, the signup placeholder claim) read the team store and
skip when it is absent. With `teams_enabled` on and no extension registering
the store and service, the server now fails at boot instead of serving
without those checks.

- whoami no longer string-matches the extension proxy's "missing property"
  error: with the flag on, the store is always there.
- Tests that turned the flag on without Teams drop it; the puter.js harness
  sets it only under the `teams` capability.
- Drop a stale services/index.ts comment.

* refactor(gui): drop the Teams console styles, which ship with the console now
2026-10-22 15:11:36 -07:00
Juan Fernando Castro 1fdde02f6a feat(metering): free and no-allowance plan traits, provisional resolver answers (#4284)
* feat(metering): let registered plans declare free and no-allowance traits

A policy passed to registerPolicy can now set `free: true`, so
isFreeSubscription and everything that reads it count the plan as free.
It can also set `noAllowance: true`: the plan grants no monthly
allowance, so only purchased credit can be spent, and the cached
credit check refuses an actor that has none. A non-positive allowance
without the trait still means the plan is not metered.

getActorSubscriptionStatus returns the plan along with whether it was
paid for and whether the answer is provisional (a lookup behind it
failed, or the actor lacks a field resolvers key on), so callers can
tell a fallback from a resolved plan. getActorSubscription is unchanged.

* feat(metering): provisional resolver answers, and tighter no-allowance accounting

A subscription resolver can answer { id, provisional: true }. The answer
wins like any id but is cached only for the fallback window.

On a no-allowance plan, the month record now records allowanceUsed = 0,
so spend that purchased credit paid for is not counted again against an
allowance the actor gets later in the month. An actor that never had
budget raises no credit-state event. setMonthAllowance refuses plans
whose allowance an override never applies to.

* fix(metering): keep resolved plans clean when a default fails, and announce moves onto no budget

A failed or provisional default resolver now marks the answer provisional
only when the default is what applied; an explicit plan that won is
cached for the full window. A provisional answer with no id still marks
the result provisional.

A plan or credit change now re-arms an actor's credit alert instead of
forgetting it, so an actor moving onto a plan with no budget hears
'exhausted' once. An actor never seen with budget still hears nothing.

* chore(metering): let a provisional answer carry no id, and fix two resolver docs

* chore: one-line comments
2026-10-22 15:47:01 -04:00
Juan Fernando Castro 1627650abc refactor: make teams an optional feature (#4248)
* refactor: let core run without a teams registration

Teams becomes optional: `services.team` and `stores.team` may be absent, and
core stops assuming either is there.

Seventeen call sites across share, signup, account deletion and the homepage
now degrade instead of throwing: a team recipient that cannot be resolved is
"no such recipient" rather than a fault, a group share folds in nothing, and
account deletion stops asking what a seat was charged for. This generalises
the pattern `gates.ts` and `SocketService` already used.

Teams' own files take a private handle to their store and service, so the
feature's code stays free of optionality that only core needs.

No behaviour changes where Teams is registered.

* refactor: put a contract between core and teams

`core/teams.ts` holds the shapes core reads and the calls it makes, and the
registries declare the pair as provided rather than built in. The service,
store and controller are registered by whatever runs the feature.

Typing them properly turned up eight AuthController call sites the previous
commit could not see: the controller reads its stores from the built-in
registry, so an absent `team` fell to an index signature and typed as
`unknown` instead of erroring. They now go through the same optional path as
the rest.

The feature's own tests move with it. The cases in core suites that build a
team to exercise a core rule still need relocating, so those suites are red
here.

* test: stub core's seat rules, rehome the rest with the feature

A core suite can no longer build a team, so the rules that refuse a seat get
`testFixtures/seatStub.ts` instead: it answers the contract, which is all
those rules read, and puts the registry back when the test ends.

Seventeen cases move that way. Four could not: group shares and a seat's 2FA
enrolment over HTTP need the real feature and the `/teams` route, so they go
with it, as does the whole of `GroupGrants.test.ts`.

Two whoami cases asserted a seat's team through `extension.import`, which
resolves the registry rather than the stores object. They now assert the
absent case, and the behaviour they covered is asserted against
`buildUserDetails`, which takes its stores as an argument.

Suite: 12252 passed, 38 skipped.

* refactor: take the teams console out of the built-in tabs

`TabTeams`, `teamsConsole` and `teamActionIcons` are no longer built in. The
dashboard already invites a tab through its `dashboard-will-open` event, so
whatever provides Teams contributes the console the same way.

Its 113 strings go with it and register into the shipped English dictionary
at load, so `i18n('teams_*')` resolves as it did. Only `en.js` ever carried
them, so no translation is lost.

Three modules stay, because things other than the console use them:
`teamBadge` (the dashboard shell), `orgSeat` (Home, Account, Security) and
`shareTeams` (the share picker). They already render nothing when the
account has no team. `teams_account_of` stays with `teamBadge` for the same
reason, and the `share_*` and `plan_team_*` strings stay with their callers.

The GUI builds clean with no console at all, and builds it in when the tab
is contributed.

* docs: seats past the owner's current limit get no allowance
2026-10-22 15:32:39 -04:00
Daniel Salazar 4968a69d45 fix(s3): keep credential-chain credentials an hour from expiry when signing (#4299)
Presigned URLs die with the credentials that signed them, and the SDK only refreshes temporary credentials five minutes before expiry, so a URL signed late in a credential's life could expire long before its requested lifetime (up to an hour). The chain provider now reports expiry an hour early and force-refreshes credentials that are already inside that window.
2026-10-22 00:47:48 -07:00
Juan Fernando Castro aa47dcd3e5 feat(workers): internal teardown that takes the script down before the row, and destroy reports Cloudflare refusals (PUT-2219) (#4297) 2026-10-21 18:50:54 -07:00
Daniel Salazar c8424f6775 fix(fetch): raise HTTP/2 receive windows for plain fetch (#4296)
Node 26's bundled undici negotiates HTTP/2 by default with a 256 KB
per-stream receive window, which capped large downloads at about
window / RTT (8.6 MB/s vs 46 MB/s over HTTP/1.1 at 23 ms). The guard swaps
in an Agent of the same class with 4 MB stream / 16 MB connection windows.
2026-10-21 13:54:31 -07:00
Daniel Salazar b3c85c86e9 feat(puter.js): open the upgrade page with what was refused (#4294)
The web upgrade dialog's button opens the dashboard with the refusal
reason, the app the call ran as, and the refused method as query params,
so the dashboard can open its upgrade flow with that context.
2026-10-21 13:09:02 -07:00
Daniel Salazar d22d419c4d chore: move to Node 26 (#4295)
* chore: move to Node 26

Docker image and CI run on Node 26 (bookworm-slim, same Debian base as before).

Node 26's bundled undici reads the global dispatcher from the
`undici.globalDispatcher.2` slot, so the guard's 10-minute timeout wrap on
`.1` no longer reached plain fetch() and provider calls fell back to the
5-minute headers timeout. The guard now wraps every present slot and tracks
wrapped dispatchers in a set, since compose() can return a Proxy that writes
a marker property through to the original dispatcher.

* test(puter-js): disable Node's web storage in test workers

Node 26 defines its own localStorage global, undefined without a storage
file, and vitest's jsdom environment doesn't replace globals that already
exist, so jsdom tests saw localStorage as undefined.
2026-10-21 13:04:23 -07:00
Daniel Salazar 0760530d0c feat(gui): edit display name and bio from the Account tab (#4293) 2026-10-21 12:57:15 -07:00
Juan Fernando CastroandDaniel Salazar e18a5c2b54 fix(apps): refuse claimed subdomains as app index_url hosts (PUT-2110) (#4267)
index_url may not point at a system host: the origin, the api host, or a
subdomain an extension has claimed. App create and update now apply the
same refusal to `<claimed>.<domain>` (any case, any port) that they apply
to the origin and api hosts.

The host-to-claim match is a new exported `subdomainLabelOf` in
hostRedirects, also used by the claim dispatch middleware, so both
decide the same way.

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-21 13:19:31 -04:00
Kailasnath TandDaniel Salazar 75385501a7 fix(fs): support offset and byte_count in read operations (#4266)
* fix(fs): support offset and byte_count in read operations

Allow partial content reads via offset and byte_count parameters in puter.fs.read and backend FS controllers.

- In puter.js read operation, check options.offset and options.byte_count against null/undefined so offset: 0 is preserved in query params.
- In handle_resp, accept any 2xx status code so 206 Partial Content responses resolve successfully.
- In backend FS controllers (LegacyFSController and FSController), derive the HTTP byte range from query offset/byte_count parameters when an explicit Range header is not present.
- Add unit and integration tests across controllers and SDK test suites.

* fix(fs): strict range params, 416 past EOF, drop format churn

- resolveReadRange accepts only decimal digit strings and safe integers
- readContent maps S3 InvalidRange to 416 instead of a 500
- revert unrelated union-type reformatting, restore truncated test comment

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-20 21:20:51 -07:00
Daniel Salazar 5b0205f2bf fix(hosting): apps publish with write again (#4291)
Reverts the hosting part of #4290. manage stays a user-to-user grant;
an app with write on its user's directory can publish and keep serving it.
2026-10-20 17:36:36 -07:00
Daniel Salazar 3bce3b056e misc: hardening (#4290) 2026-10-20 17:12:13 -07:00
dependabot[bot] f70240784f chore(deps): bump follow-redirects from 1.16.0 to 1.16.1 (#4287)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 16:53:12 -07:00
dependabot[bot] 356e7b2b94 chore(deps): bump follow-redirects in /src/puter-js (#4288)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 16:29:27 -07:00
jelveh f02b4a6b73 feat: tell apps when a launched item is a folder; Open with AI above Open With
A folder opened with an app (Open With, Open with AI) reached puter.js as
a file: the GUI never forwarded the signature's fsentry_is_dir, so the
launched FSItem always had isDirectory false. The GUI now appends
puter.item.is_dir, and puter.js reads it in onLaunchedWithItems and the
onItemsOpened launch replay. Both now share one URL-to-FSItem builder.

"Open with AI" moves directly under Open, above Open With, in the
desktop and dashboard file menus.
2026-10-20 16:16:20 -07:00
dependabot[bot] 0768978111 chore(deps-dev): bump vitest from 4.1.11 to 4.1.12 (#4275)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 4.1.11 to 4.1.12.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.12/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-version: 4.1.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 15:47:43 -07:00
Nariman Jelveh 66093e78f3 feat(gui): "Open with AI" in the dashboard Files menu, opening like Open With (#4289)
"Open with AI" now launches the `ai` app with the item the same way
Open With launches any app: launch_app with file_path/file_uid, so the
app gets a signed launch item via puter.ui.onLaunchedWithItems. It
replaces the custom `ai:openFsEntries` postMessage, which went to an
existing AI window.

- The helper moved from UIItem into helpers/openWithAI.js, shared by the
  desktop item menu and the dashboard Files menu (generate_file_context_menu).
- A shortcut opens its target.
- Dropped from multi-select menus, since a launch opens a single item.
- Renamed open_in_ai -> open_with_ai ("Open with AI") and removed the
  unused ai_app_unavailable string.
2026-10-20 15:33:12 -07:00
dependabot[bot] ed531e3992 chore(deps): bump undici from 7.30.0 to 7.30.1 (#4280)
Bumps [undici](https://github.com/nodejs/undici) from 7.30.0 to 7.30.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.30.0...v7.30.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.30.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 15:27:17 -07:00
dependabot[bot] 53e27f3d41 chore(deps): bump mysql2 from 3.24.5 to 3.24.6 (#4279)
Bumps [mysql2](https://github.com/sidorares/node-mysql2) from 3.24.5 to 3.24.6.
- [Release notes](https://github.com/sidorares/node-mysql2/releases)
- [Changelog](https://github.com/sidorares/node-mysql2/blob/master/Changelog.md)
- [Commits](https://github.com/sidorares/node-mysql2/compare/v3.24.5...v3.24.6)

---
updated-dependencies:
- dependency-name: mysql2
  dependency-version: 3.24.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 15:25:23 -07:00
dependabot[bot] deac121675 chore(deps-dev): bump eslint-plugin-prettier from 5.5.6 to 5.5.7 (#4278)
Bumps [eslint-plugin-prettier](https://github.com/prettier/eslint-plugin-prettier) from 5.5.6 to 5.5.7.
- [Release notes](https://github.com/prettier/eslint-plugin-prettier/releases)
- [Changelog](https://github.com/prettier/eslint-plugin-prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/eslint-plugin-prettier/compare/v5.5.6...v5.5.7)

---
updated-dependencies:
- dependency-name: eslint-plugin-prettier
  dependency-version: 5.5.7
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 15:22:09 -07:00
Nariman Jelveh 53e6e3179b refactor(gui): remove the AI side panel and its desktop button (#4286)
The desktop no longer launches the `ai` app at boot. This removes the
`is_panel` window mode, which existed only for that launch, and the
toolbar/taskbar/window/popover offsets tied to it. It also removes the
desktop sparkles button that showed the panel, and the
`silent_on_failure` launch option, whose only caller was the boot launch.
"Open in AI" in the item context menu still launches the app as a
normal window.
2026-10-20 15:21:21 -07:00
dependabot[bot] 73624bf1d0 chore(deps): bump smtp-server from 3.19.21 to 3.19.23 (#4276)
Bumps [smtp-server](https://github.com/nodemailer/smtp-server) from 3.19.21 to 3.19.23.
- [Release notes](https://github.com/nodemailer/smtp-server/releases)
- [Changelog](https://github.com/nodemailer/smtp-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/smtp-server/compare/v3.19.21...v3.19.23)

---
updated-dependencies:
- dependency-name: smtp-server
  dependency-version: 3.19.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-20 15:19:00 -07:00
Nariman Jelveh 1a9e43f5e5 fix(dashboard): adversarial review — bugs, a11y, perf, i18n (#4285)
* fix(dashboard): size the Notifications sidebar row like its neighbors

The bell button's font: inherit overrode the sidebar item's font-size
(16px instead of 14px, 15px in the drawer), and its icon wrapper stayed
18px while the drawer grows nav icons to 20px.

* test(dashboard): scope the directory-note assertion to the directory row

The 2FA row reuses .teams-directory-note, so the unscoped selector read
both notes and the test failed.

* fix(dashboard): only treat a picked file as a new avatar while the avatar picker is open

The Account tab listened for every file_opened on the dashboard window,
so choosing an image for a web link's Change Icon in the Files tab opened
the profile-picture crop modal, and saving it replaced the avatar.

* fix(dashboard): let dialogs over the Files tab keep their keyboard

The Files keydown handler ran under an open confirm, so pressing Enter on
the focused Delete button of the permanent-delete prompt opened the
trashed item instead (stacking a second alert), and arrows or Delete
acted on the rows behind the dialog.

* fix(dashboard): keep Apps grid keys away from an open context menu

Choosing a tile's menu item with Enter (e.g. Open in new tab) also
launched the focused app in place, and arrows moved tile focus while
they walked the menu.

* fix(dashboard): Escape on a menu or modal inside an app folder keeps the folder open

The menu's and the uninstall modal's Escape handlers removed them before
the folder's listener ran, so it saw nothing above it and closed the
folder with the same keystroke.

* fix(dashboard): a tap on the drawer or sheet scrim only dismisses it

Outside-press handlers closed the mobile drawer and the notifications
sheet on touchstart/pointerdown, which took away the scrim's hit area
before the click, so the tap also landed on the app tile or file row
underneath.

* fix(dashboard): let the sidebar nav scroll on short viewports

The nav clipped its overflow, so in a landscape phone's drawer (or a
short desktop window) Usage, Account, and Security fell below the fold
with no way to reach them.

* fix(dashboard): keep the mobile drawer full width when the desktop sidebar is collapsed

The remembered collapse outranked the drawer's width rule, so below the
drawer breakpoint it opened as an unlabeled 50px icon strip, and the
toggle that would undo it is hidden there.

* perf(dashboard): stop refetching account data each time focus leaves an app

Home's window focus listener fired whenever focus came back from an app's
iframe, so every click from an app to the dashboard re-pulled whoami,
subscription, storage, and usage (5 requests), on any tab. It now only
refreshes after the page itself lost focus to another window.

* fix(dashboard): show a seat's team name unescaped in the plan card

i18n escapes its arguments for HTML, but the note is set with .text(), so
a team called "Bob's & Co" read "Managed by Bob&apos;s &amp; Co".

* fix(dashboard): stop double-escaping app names in uninstall alerts

UIAlert escapes its message, so an app titled "Tom & Jerry" read
"Tom &amp; Jerry" in the uninstall-failed and sessions-not-ended alerts.

* fix(dashboard): roll back an uninstall whose revoke failed

puter.perms.revokeApp resolves with error set instead of rejecting, so a
failed revoke still removed the tile, marked the app removed for later
sessions, and ended its sign-in, while the app kept its permissions and
the rollback and error alert never ran.

* fix(dashboard): read a team account's suspension when its name matches an Object member

The per-account state map was a plain object, so a username such as
constructor or toString was already "in" it, its audit entries were
skipped, and a suspended account showed as active with no Delete.

* fix(dashboard): update the profile card and sidebar name after a username change

Neither carried the .username class that the rename refreshes, so both
kept showing the old name until reload.

* fix(dashboard): ask how to restore a trashed item whose name is taken

Restore moved without overwrite or dedupe, so when the original location
already had an item of that name the move 409'd and nothing happened:
an uncaught error from the item menu, a console line from the others.
All three restore entry points now share one path that offers the
Replace / Keep Both prompt the paste flow uses, alerts on other errors,
and updates the footer and empty notice once the rows are gone.

* fix(dashboard): refresh a renamed row's name, icon, and position in the Files tab

The Files tab passed rename_file a callback it never accepted, so a
renamed row kept the desktop's 20-character truncation even in a wide
column, kept its old type icon, and stayed out of sort order. rename_file
now takes an optional onDone (carried through undo), which the Files tab
uses to fit the name to its column, refresh the icon and type, and move
the row into place. A cancelled rename no longer truncates the name
either.

* fix(dashboard): drop Publish and Properties from the Shared view's menu

The Shared view is a query, not a folder, so both acted on the virtual
shared path: Properties showed an unknown error and Publish opened a
dialog for a path that doesn't exist.

* fix(dashboard): close the Files image preview on Escape and when leaving the folder

Escape cleared the selection but left the Space preview open with
nothing selected, the preview stayed over a different folder after
navigating, and a preview whose URL arrived after it was dismissed or
replaced still opened.

* fix(dashboard): don't move Files history for a navigation that's dropped

renderDirectory ignores a navigation while another folder is loading,
but every caller had already pushed or stepped history, so after a click
during a load the listing and history disagreed and the next Back or
Forward seemed to do nothing. History now only moves when the render
will run; back/forward steps share one helper.

* fix(dashboard): reopening the current folder adds no Files history entry

Clicking the folder already on screen (e.g. Home in the sidebar) pushed
a duplicate entry, so the next Back reloaded the same folder.

* fix(dashboard): show installed apps when the recommended list fails to load

get-launch-apps sat in the same Promise.all as the installed apps and
its body was parsed without a status check, so a network error or a
proxy's HTML error page replaced the whole grid with "Failed to load
apps". A failed list now falls back to the last one that loaded.

* fix(dashboard): fit the app uninstall dialog on narrow phones

The card is mounted outside the dashboard's border-box rule, so its
320px min-width plus padding came to 370px and was clipped on 360px
screens; a long unbroken app title also overflowed it. Desktop sizing is
unchanged.

* fix(dashboard): keep a new team account's one-time password on screen until dismissed

The credential box lived only in the DOM inside the panel that every
refresh repaints, so revisiting the Teams tab or any later action wiped
a password that can't be retrieved again. It is now kept in state, for
the team it belongs to, and drawn with the panel.

* fix(dashboard): reload the Teams tab after a change made while it was loading

refresh() handed back any load already in flight, so an action or team
switch made during a revisit's background load awaited data read before
the change: a toggled setting snapped back, and a team switch could
paint one team's roster next to another's record. A change during a load
now queues one more load, the stale one skips its paint, and revisiting
the tab doesn't start a second load while init's is still running.

* fix(dashboard): send one re-enable request for a double click in Teams

Re-enable has no confirm dialog to absorb a second click, and nothing
held the action while it ran, so a double click enabled the account
twice (two audit entries). A suspend or re-enable already in flight for
an account now ignores further clicks.

* fix(dashboard): revert the avatar and say so when saving a new photo fails

The crop's Save showed the new picture everywhere and revealed Remove
without waiting for the upload, and update_profile swallows failures,
so a failed save looked saved until reload. The save is now awaited: a
failure puts the previous picture back with an error hint, and Remove
waits for the upload so it can't race it.

* fix(dashboard): hide the Usage tab's plan button for team seats

A seat's plan belongs to its team, so Home already hides its upgrade
link; the Usage tab still offered Upgrade, which only reaches a refusal.

* fix(dashboard): don't drop Home's forced usage refresh during a running load

loadUsageData handed any caller the load already in flight before looking
at force, so a subscription change (or a return from the billing portal)
during a routine load was answered with data read before the change, and
the plan card stayed stale until the next refresh. A forced call now runs
once more after the running load.

* fix(dashboard): activate Apps when the URL names an unknown tab

An unknown hash rendered Apps as the active tab but skipped its
activation, so the grid kept the generic content padding and Apps'
onActivate never ran.

* fix(dashboard): make the sidebar user menu reachable from the keyboard

The user button was a div with no tabindex, so Log Out, account
switching, and Contact Us were out of reach without a pointer. It is
now a button (rendering unchanged), with aria-haspopup.

* fix(dashboard): name the sidebar's collapse and drawer buttons for assistive tech

The collapse toggle held only two unlabeled images, so screen readers
announced an unnamed button; it now carries a translated label and
aria-expanded that follow its state. The drawer's open/close labels were
hardcoded English and now go through i18n.

* fix(dashboard): make Home's recent apps reachable from the keyboard

Each recent app was a click-only div, so keyboard users couldn't open
one. They are now buttons with a focus ring; the icon is decorative
since the title names the button.

* fix(dashboard): name the Security tab's 2FA switch for assistive tech

The checkbox sat in a text-less label, so screen readers announced an
unnamed checkbox. It now carries the translated Two Factor
Authentication label, as the Teams switches do.

* fix(dashboard): sort the Usage table from the keyboard

The sortable headers were click-only th cells. Each now holds a button,
carries aria-sort, and keeps focus across the re-render a sort causes.
The Resource/Units/Cost headers go through i18n.

* fix(dashboard): don't rebuild the Files listing under an open menu

Returning to the browser tab re-rendered the folder even with a context
menu or action sheet open, so its actions then targeted detached rows:
Rename did nothing, Restore faded a row that was already gone. The
refresh now waits while a menu is up, as it does for a rename.

* fix(dashboard): point the Files sidebar's Documents at the current home

The entry read window.documents_path, which globals.js sets once from
the cached user and nothing updates; login and username changes keep
window.docs_path current instead. After a username change, Documents
pointed at the old home and showed "This folder couldn't be opened",
even after reloading.

* fix(dashboard): let only the newest share listing render in the share dialog

Each action ends in a refresh, and refreshes from overlapping actions (or
the opening load) could finish out of order, so an older listing landed
last and undid what the user just did, e.g. the link control snapped
back to restricted after turning the link on. Older refreshes now drop
their result.

* perf(dashboard): load a team's record, roster, and plan in parallel

loadSelected awaited the audit log (every page of it), then the members,
then the plan, on every visit to the Teams tab. They don't depend on
each other, so they are now fetched together.

* fix(dashboard): keep keyboard focus on the Teams control that was used

Every Teams action repaints the whole panel, which dropped focus to
<body>: a keyboard user toggling a switch or suspending an account was
sent back to the top of the page. The repaint now refocuses the same
control, or the same account's new action when the old one is gone.

* fix(dashboard): give the app uninstall prompt dialog semantics and focus

The prompt had no dialog role or label and never took focus, so keyboard
and screen-reader users stayed on the inert grid behind it. It is now a
labelled modal dialog that focuses Cancel on open and returns focus to
the app's tile when dismissed.

* fix(dashboard): keep the add-an-app tile when there are no apps

With no apps at all, the empty state replaced the whole grid, add tile
included, so a new user on an instance with no recommended apps had
nothing to act on. The empty line now sits above a grid that keeps the
tile, as a search with no matches already does.

* fix(dashboard): don't rebuild the Apps grid under an active drag

An uninstall finishing (or a failed one rolling back) re-rendered the
grid even while another tile was being dragged, replacing the dragged
tile with a fresh copy: the copy showed under the ghost and the drop
was lost. Renders asked for mid-drag now wait for the one the drop does;
the folder eject, which rebuilds the grid as part of the drag, still can.

* fix(dashboard): return focus to the drawer toggle when the mobile drawer closes

Closing the drawer (Escape, the close button, a tab pick) left focus on
controls that had just become hidden, so it fell to <body>; closing the
notifications sheet opened from the drawer tried to refocus the bell
inside the closed drawer. Both now land on the drawer toggle.

* fix(dashboard): truncate Files names between graphemes, with a binary search

The middle truncation sliced UTF-16 code units, so at many widths a long
name with emoji showed a half surrogate (rendered as a replacement
character) or a broken joined emoji. It also measured one more character
per step, a measureText call per character of every long name on each
render and column resize. The cut now falls between graphemes and is
found by binary search; output for other names is unchanged.

* fix(dashboard): translate the Home tab's hardcoded English

The tab label, the recent-apps empty state, card subtitles, the plan
badge and trial wording, the upgrade/manage links, and the usage cards'
"Used" / "of" lines were English literals. They now go through
i18n, with placeholders where values are interpolated.

* fix(dashboard): translate the sidebar's remaining English

The Apps, Files, and Usage tab labels, the user button's fallback name,
and the user menu's Developers item were English literals; they now use
i18n keys like the other tabs.

* fix(dashboard): translate the Usage tab's hardcoded English

The subtitle, the "used of" meter lines (now the existing used_of
key), Usage Details, the plan button, Show more/less, and the details
error were English literals.

* fix(dashboard): translate the Account and Security tabs' hardcoded English

Both subtitles, the delete-account description, the sessions hint, the
username fallback, and the avatar picker's window title were English
literals.

* fix(dashboard): translate the Apps tab's hardcoded English

The uninstall prompt and its failure alert, empty and no-match notices,
pager and folder-dot labels, the search placeholder, the edit toggle,
and the tile menu's Open in New Tab / Quit / Uninstall were English
literals.

* fix(dashboard): translate the Files tab's hardcoded English

The folder sidebar, footer counts (now the existing item-count keys),
sort and view menus, upload status and errors, the empty and
can't-open notices, the reserved-name alerts, the move error, and the
loading text were English literals.

* fix(dashboard): translate the action sheet's Back row and match delete in any language

The touch action sheet styled an item red when its label contained the
English word "delete", so Delete lost its danger styling in every other
language; it now compares against the translated Delete labels. Its
submenu Back row and the properties panel's Directory/disassociate
labels were English literals.

* fix(dashboard): make the Files sidebar, crumbs, and nav arrows keyboard-operable

The folder sidebar, the breadcrumbs, and the back/forward/up arrows were
click-only li/span/img elements, out of the tab order. They now take
focus, carry button/link roles (the arrows with labels and
aria-disabled), answer Enter and Space, and show a focus ring. The rows'
Enter/Space shortcuts also no longer swallow those keys from a focused
control: with rows selected, Enter on Upload opened the selection.

* perf(dashboard): read grid name widths before writing any in the Files tab

fitItemNames read each tile's name width right after writing the
previous name, forcing a layout per row. Reading them all first, then
writing, cuts switching 400 files into grid view from about 300ms to
170ms; the rendered names are identical.
2026-10-20 13:44:42 -07:00
242a654166 feat(ai): add Kling for video, image and speech generation (#4274)
* feat(ai): add Kling for video, image and speech generation

One `kling` API key now registers a provider on three drivers:

- txt2vid: Kling 3.0 Turbo, 3.0, 3.0 Omni, O1, 2.6 and 2.5 Turbo,
  with first/last frames, reference images on the omni models, native
  audio, and editing a source video on 3.0 Omni and O1. Billed per
  second at the resolution, audio and video-input rate; an edit is held
  at the longest source and billed for the clip Kling returns.
- txt2img: Kling Image 3.0, 3.0 Omni, O1 and 2.1, billed per image.
- txt2speech: Kling TTS with its English and Chinese voices, billed per
  request.

Kling's business error codes map onto the driver error contract, so an
exhausted resource pack reports as credit exhaustion and a submit-time
content block as `moderation_flagged`. "Risk control" failures now
count as content-filter refusals.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ai): treat Kling's unknown-model codes as a Puter-side failure

Kling answers 1202/1203 when the method or model isn't one it serves.
Puter picks both, so these meant catalog drift but surfaced as a 400 that
blamed the caller and raised no alarm. They now return 502 upstream_failed
with a warning alarm.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ai): tighten Kling input checks and error mapping

- 3.0 Turbo takes at most 2,500 prompt characters with a first frame,
  the image-to-video endpoint's limit.
- O1 refuses reference images together with a first and last frame.
- 3.0 Omni accepts a source of up to 15.5 seconds, and an edit is held
  at 16 seconds to cover it.
- A source video Kling can't decode fails as 400 upstream_bad_request
  instead of a Puter-side 502.
- A 1201 about the model counts as catalog drift, like 1202/1203.
- Auth failures no longer pass Kling's console wording to the caller.

Adds integration tests for the image, video and TTS providers, gated
on PUTER_TEST_AI_KLING_API_KEY.

* feat(ai): wait up to 14 minutes for a video job

Video jobs, Kling edits in particular, can take over 10 minutes. The
poll window and the Gemini Omni deadline move to 14 minutes, under the
SDK's 15-minute idle timeout, with holds and signed source links
lengthened to match. The keep-alive default moves to 920 seconds so it
stays above a 15-minute proxy idle timeout.

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-20 12:32:19 -07:00
Daniel Salazar 41e4b582d3 fix(subdomains): emit subdomain.delete for sites deleted with an app (#4273)
Sites and workers tied to an app go with it by FK cascade, on app delete and
on deleting the app owner's account. Only caches were dropped for them, so
the subdomain.delete listeners (CDN purge, worker script teardown, worker
sessions, site cleanup) never ran.
2026-10-20 12:31:40 -07:00
Reynaldi Chernando 2a49689fad fix contradicting jsdocs and docs (#4282) 2026-10-20 18:39:49 +07:00
Juan Fernando CastroandDaniel Salazar b54b76715a feat(fs): finish a home rename that stopped partway (#4255)
* feat(fs): finish a home rename that stopped partway

FSEntryStore.rebaseUserSubtree runs renameUserHome's owner-scoped
cascade on its own (same subtree matching, same rebased-cache
invalidation) and also drops the home row's cached copies at either
path. countUserSubtree counts the owner's rows under a prefix with that
same matching, from the primary. renameUserHome now uses the shared
cascade.

* fix(telemetry): resolve the tracer per span so a test's provider is the one used

withSpan held a ProxyTracer built at module load, which caches the first
provider it delegates to. Test files sharing a worker then sent spans to a
provider an earlier file had already shut down.

* fix(fs): drop cached copies of only the rows a home rebase moves

The owner-scoped rewrite read its invalidation set from everything under the new prefix, which for a finishing rebase is the whole live home; it now reads the rows it is about to move.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-20 00:59:40 -07:00
Juan Fernando Castro d978538c9d feat: let a subdomain claim keep the main pipeline (#4268)
`extension.claimSubdomain(name, { mainPipeline: true })` serves
`<name>.<domain>` the way core's own subdomains are served: through CORS,
body parsing, auth and every route, `subdomain: '*'` ones included. Routes
declared for the name are ordinary routes, with no option restrictions.
Gated claims (`{ authorize, checkRoute? }`) behave as before.

onlyoffice's subdomain now comes from its extension's claim, so core no
longer lists it among the subdomains it serves itself. Without that claim
`onlyoffice.<domain>` gets the unclaimed-subdomain 404 like any other name.
2026-10-20 00:43:54 -07:00
e83e27f306 feat(ai): edit a source video with txt2vid (Gemini Omni, FLUX 3) (#4272)
* feat(ai): edit a source video with txt2vid on Gemini Omni

txt2vid takes an `inputVideo` option (sent as `input_video`): a URL, data
URI or Puter file path to an MP4, QuickTime or WebM video of up to 32 MiB.
Gemini Omni receives it inline next to the prompt, which describes the
edit or extension. Google enforces the 10-second limit.

Models declare `videoInput` in the catalog, and the driver rejects a
source video for any other model instead of letting the provider drop it.
The credit hold adds 16,000 input tokens for a source video, since
countTokens reports far fewer tokens than the reply bills.

loadFileInput takes an optional `timeoutMs` for web URLs, and cancels
the body of a failed response.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(ai): edit a source video with FLUX 3 on Together AI

FLUX 3 takes `input_video` as Together's `media.source_video`. A web URL
goes through for Together to fetch; a data URI or Puter file is read with
the caller's access, capped at 32 MiB, sniffed as a video and sent inline.
Editing is estimated and billed at the 1080p rate of $0.54/s, since FLUX 3
jobs report no cost. Image inputs can't be combined with a source video.

Seedance 2.0 ignores `source_video`, treats `reference_videos` as a loose
reference rather than an edit, and Seedance 2.5 fails on reference videos,
so neither declares `videoInput`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(ai): name FLUX 3 as a video editor in the AI overview

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ai): match Hailuo 02 to Together's price and document estimate billing

Together quotes $0.56 for a Hailuo 02 clip; the catalog had $0.49.

Together's video jobs no longer report what they cost, so every Together
clip is billed at the catalog's highest published rate. The docs said the
charge was the amount Together reports; they now describe the estimate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: enforce video edit durations and propagate source fetch cancellation

* fix(ai): pass media inputs as Puter files and stream source videos

Source videos were read whole into the API process, base64-encoded and
sent twice to Gemini (countTokens and the interaction), which held about
128 MiB per in-flight edit for up to ten minutes.

- Video inputs are a URL, a Puter file path or a data URI of at most
  1 MiB. Together gets a Puter file as a 15-minute presigned S3 URL and
  fetches it itself; Gemini streams it into the request body with an
  exact Content-Length, so it's never held as one string.
- Gemini counts only the prompt and image for a source video, and both
  providers read the source only after the credit hold.
- txt2img and txt2vid image inputs accept Puter file paths, read within
  the caller's access right before the provider call. Existing URL,
  data URI and base64 inputs are unchanged.
- puter.js writes a Blob, File or data URI over 1 MiB to a temporary
  file under .tmp, sends its path, and deletes it once the call settles.
- FLUX 3 edits bill at the 720p editing rate, matching their 1280x704
  output.

* fix(ai): stage inputs after website sign-in, keep images working without a write

- puter.js chose the temporary file path before signing a website visitor
  in, so it fell outside the app's data directory and the upload was
  refused. It now signs in first, then picks the path.
- An image that can't be written (no access, storage full) is sent inline
  as before instead of failing the call. Videos still fail, since a
  large inline video isn't accepted.
- Telling raw JPEG base64 from a path decodes only the first 24
  characters instead of the whole input.

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: 404oops <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-19 23:33:03 -07:00
Udit AgarwalandDaniel Salazar 54f542ace4 fix(gui): refresh Start menu apps without blocking cached rendering (#4244)
* fix(gui): refresh Start menu apps without blocking cached rendering

* docs: remove desktop Start menu limits from SDK reference

* fix(gui): drop the 1 MiB cap on the launch-apps refresh

Recent entries carry each app's raw icon column, which can still hold a data URL of up to 5 MB, so a large icon tripped the cap on every request, including the one at startup, and left the Start menu with no apps. Replace the cap's test with one for the unlaunchable-app filter.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-19 16:13:55 -07:00
Daniel Salazar 18b96d529c fix: misc scan fixes (#4270) 2026-10-19 12:32:13 -07:00
Kunal Jaiswal f146da16d1 fix(gui): drop deleted apps from the start menu (#4243)
The start menu read a session-cached launch-apps list and only refetched when it was empty, so an app deleted after page load (for example in Dev Center) stayed in Recent and Recommended until a reload. Refetch on every open, and filter out entries the backend marks unlaunchable, so a reclaimed or inaccessible app leaves the list too. Apply the same filter to the dashboard's Recent apps. Adds helpers/launchAppsList.js with filterLaunchableApps and its unit test.
2026-10-19 10:50:36 -07:00
Daniel Salazar dd56d54753 docs(bug-bounty): add eligibility rules (#4269)
Bounties cover user security/privacy and platform security/health only.
Reports must show a direct attack without pre-assumed access, and
exploits that rely on sharer/sharee trust are hardening or lower
severity. Also generalizes the metering non-issue, adds existence-only
oracles, and tightens duplicate, proof and first-contact rules.
2026-10-19 10:49:12 -07:00
Daniel Salazar e5fd30ccf0 fix(ai): validate large base64 by decoded length, not regexps (#4265)
* fix(ai): validate large base64 without anchored regexp loops

Gemini TTS replies over ~4 MB of base64 (~65 s of audio) failed with
"Maximum call stack size exceeded". Once a process holds enough compiled
code, V8 compiles new regexps unoptimized, and `^[...]+={0,2}$` then
pushes a backtrack entry per character until its stack overflows.

TTS now searches for a character outside the alphabet instead. The raw
base64 icon/reference-image check drops its own anchored regexp;
decodeStrictBase64 already rejects the same inputs.

* perf(ai): validate base64 by decoded length instead of regexp passes

Unoptimized regexp scans dominated large payloads: 186 ms for max-length
TTS audio, 880 ms in decodeStrictBase64 for 30 MB of video. Buffer.from
skips characters outside the alphabet, so comparing the decoded length
with the one the input length predicts catches them for free.

decodeStrictBase64 now does that plus rejecting the URL-safe alphabet
(~10 ms for 30 MB), and TTS reuses it instead of its own check.
Non-canonical trailing bits now decode instead of being rejected.

* fix(ai): reject non-ASCII base64 that Node decodes by its low byte

Node reads code units above 0xFF by their low byte, so U+012B decodes
as '+' and keeps the decoded length the input predicts. Require ASCII
(~1-2 ms for 43M chars) before the length check.
2026-10-18 21:56:29 -07:00
Neal Shah 517efda38a automerge emergency fix (#4264) 2026-10-19 00:18:58 -04:00
Neal Shah ea3801fb74 PUT-2058 and PUT-2096 (#4256) 2026-10-18 23:30:01 -04:00
Daniel Salazar 0786cc7db2 fix(webdav): PROPPATCH checks write access and answers with the masked path (#4260)
PROPPATCH now requires write access on the target, like PUT and DELETE,
and its response href goes through the request's share-path masker the
way PROPFIND's do.
2026-10-18 00:33:54 -07:00
Daniel Salazar 592020c899 docs(bug-bounty): list re-claimed abandoned site and worker names as a known non-issue (#4261) 2026-10-18 00:33:49 -07:00
Daniel Salazar 9ff224e22f perf(backend): cut sequential round trips on auth, fs, driver, event and site paths (#4259)
Pipeline
- CORS preflights send Access-Control-Max-Age: browsers stop re-preflighting authed calls every 5 s.
- `req.ua` is parsed on first read instead of on every request.
- JSON `rawBody` reuses the parsed buffer instead of copying it.
- `emitAndWait` returns early when nothing listens: no span or closure per unlistened driver call.
- SQLBatcher sends a lone query on the next tick while fewer than 8 flushes are in flight, and
  waits `maxTimeInQueue` as before otherwise. Lone reads skip the 10/30 ms wait; batching under
  load is unchanged.

Auth and permissions
- Token-to-actor auth starts the user, session and app lookups together and checks them in the
  same order: fewer sequential Redis round trips per authed request.
- get-user-app-token and check-app reuse the resolved app row. getOrCreateApp skips the re-select
  after its own insert and doesn't wait on the cache write for replica rows.
- getOrCreateApp reads the race winner from the primary when its insert loses, so it no longer
  hands out a session that was never written.
- whoami runs the email check and the OIDC/taskbar/team lookups in parallel; key order unchanged.
- OIDC providers are memoized once every discovery succeeds, with one discovery fetch in flight
  per URL, and the Apple key is parsed once.
- gui-token and sync-cookie use `req.actor.user` instead of re-reading the user.
- listSessions batches app lookups. App icon probes run together, and the original-icon probe
  skips the lineage fallback.
- Default-granted permissions are answered from config before the scan cache, and exploded
  options are deduped: one less Redis read per /drivers/call.
- ACL resolves ancestors lazily (owners and full-access tokens skip them) and scans the node
  first, then the ancestors concurrently on a miss.

Filesystem
- Missing-path lineage lookups run 10 at a time: batch-upload path resolution drops from about
  1.3 s to about 40 ms per 100 new files.
- MySQL single-file inserts return the row built from the written values, falling back to the
  primary read: one less primary round trip per new file.
- Batch thumbnails are stored with one UPDATE per chunk (200 rows or 4M chars), with a per-row
  fallback.
- storage.quota.bonus listeners run concurrently next to the usage SUM, and the payload carries
  `userUuid`: quota checks and /df no longer serialize the listener reads.
- readdir cache backfill is one pipeline instead of 3xN SETEX.
- Profile reads skip the lineage fallback; the own profile is read alongside the user lookup.
- Legacy /stat runs is_empty (new `hasChildrenByParentUid`), owner and thumbnail concurrently.
- Downloads and the video proxy pipe through `pipeToResponse`: aborts release the storage stream
  at once with no extra 'close' listeners, and the video proxy no longer crashes on a mid-stream
  upstream error.
- mkdir looks up target and parent together.
- Upload completion reuses the session from the access check (status is still re-read before
  completing), and multipart signing reads it once.
- Moving a shared file to Trash skips two lookups that always returned the entry itself.

Drivers and events
- Subscription resolvers run concurrently and are folded in registration order.
- SubdomainDriver and WorkerDriver overlap independent lookups.
- The events "anyone subscribed?" check is one Redis pipeline when the region is warm.
- Handler publish overlaps the count with the size check.
- Broadcast flushes peers concurrently over a keep-alive agent, so a slow peer no longer delays
  cross-region invalidations to the others.

Sites, share and AI
- Hosted sites resolve the owner first, then the app and root together. Site config loads only
  on 404s. The hosted-app query selects only the columns it uses and moves into AppStore.
- shareFlags asks about the plan only when an item is shared by link only.
- Share listings run the count, teams and plan reads alongside their page query.
- Claude attachment cleanup no longer blocks the response.
- New util `awaitLater` for promises started early and awaited later.
2026-10-17 12:03:35 -07:00
Daniel Salazar 9ad7b99607 fix(puter-js): handle file picker cancel message (#4257) 2026-10-16 23:51:55 -07:00
Daniel Salazar 28f162ee92 fix(metering): fence settles, idempotent monthly claim, split addon consumption (PUT-2001) (#4236)
* fix(metering): pin metering writes to one replica, fence settles, idempotent monthly claim (PUT-2001)

store-kv-v1 is a last-writer-wins global table, and every region settled
its metering buffer into its local replica, so concurrent increments to the
same per-actor item from two regions could drop one side. Writes and
consistent reads of metering:* and policy:actor:* items now go to the
dynamo.home replica.

Buffered settles record each write on the claim with a fence token before
sending it, and the KV store refuses a token the item already holds, so a
re-driven claim applies its write once. Claims are serialized per counter
within a cache so one writer's tokens land in order.

The monthly charge claim is a set-if-absent of a random token, and a failed
claim write keeps its token for the retry, so a write that landed anyway
still charges exactly once.

* fix(metering): drop home-replica pinning; keep addon consumption off the purchase record (PUT-2001)

Pinning every metering write and exact read to one replica put a
cross-region round trip on credit checks near the allowance and on every
settle, and made one region's outage stall metering everywhere. The
per-item clash it guarded against is rare and undercharges at most a
flush window.

The one clash that loses money is a credit purchase and an overage settle
landing on the addons item in different regions at once. Consumption now
goes to its own item and getActorAddons sums it with what the purchase
record already holds.

* perf(metering): read addon consumption only for accounts with purchased credit (PUT-2001)

* perf(metering): settle counters through a sliding pool instead of waves (PUT-2001)

* feat(metering): consistent-read option on getActorAddons (PUT-2001)

Admin credit checks read the purchase record directly to get a consistent
read, which misses consumption now that it lives on its own item. They can
call getActorAddons with consistentRead instead. Also formats the lines this
PR added.
2026-10-16 23:49:23 -07:00
Daniel Salazar fa62b3fb76 move template menu entries into a tested helper (#4258)
Builds the "New > User templates" entries in helpers/templateMenuItem.js
and covers the label with tests, including HTML encoding of both the
extension and the name.
2026-10-16 23:45:04 -07:00
Filip Kujundžićand404oops e950a87f29 Add Gemini Omni video generation with token metering (#4235)
* Add Gemini Omni video generation with token metering

* Keep explicit Gemini video model selection within provider

* Settle reported Gemini video usage before validating output

---------

Co-authored-by: 404oops <[email protected]>
2026-10-16 23:21:48 -07:00
1cb9dd756a Add Gemini transcription with consistent usage settlement (#4234)
* feat(ai): add Gemini unary transcription

* Clarify transcription language results and omit pricing

* style(ai): keep Gemini transcription sources prettier-clean

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ai-speech2txt): charge Gemini transcriptions that fail after Google answers

Google bills an interaction it processed whether or not it completed, so an
incomplete, failed or malformed reply was free to the caller and paid by
Puter; audio whose transcript overran the output cap could be resubmitted at
no cost. Usage is now recorded before the reply is checked, falling back to
the counted audio tokens when the reported usage can't be read.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Preserve transcript text with optional malformed annotations

* Keep speech provider spacing consistent

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-16 23:21:47 -07:00
be16bca1d9 Keep Gemini speech instructions separate and validate speech requests (#4233)
* fix(ai): keep Gemini speech instructions separate from transcript

* Clarify Gemini speech instructions

* style(ai-tts): keep Gemini cost comment prettier-clean

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Validate speech requests and settle reported Gemini usage

* Clarify speech settlement for unreadable responses

* Reserve the published Gemini speech token budget

* Clarify speech accounting for bounded responses

* Limit only Gemini TTS text to 3000 characters; charge overrun replies

The 3000-character text cap moves from the TTS driver into the Gemini
provider, so other providers keep their own limits.

The Gemini reply cap rises to 64 MiB, which fits the full 16,384-token
audio budget. A reply that still overruns it or times out reports no
usage, so it is charged the held budget and the error tells the caller
to split the text.

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-16 23:21:47 -07:00
2305d50e36 Restore supported Gemini chat models and align scheduled pricing (#4232)
* feat(ai): restore available Gemini models and align pricing

* Exclude deprecated Flash Image additions and share scheduled pricing

* Bill Gemini 2.5 Pro at its current list rates

Cache reads are $0.125/M, and prompts over 200k tokens bill every rate at
the long-context tier ($2.50 in, $15 out, $0.25 cached).

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-16 23:21:47 -07:00
352427687e Add Gemini Argon with mode-specific limits and native usage metering (#4231)
* feat(ai): add Gemini 4 Argon with native usage metering

* test(ai): isolate Argon HTTP smoke test from optional extensions

* fix(ai): preserve Gemini native completion and refusal reasons

* Refer model specifications and pricing to the directory

* Bound unary Argon requests and reject unsupported content

* Keep Gemini tool metadata out of Claude requests

* Describe provider-specific chat timeout boundaries

* Keep Gemini extra_content off other providers; report Gemini timeouts as 504

Replayed tool calls only carry extra_content for providers that opt in
(Gemini), so OpenAI-compatible and Responses vendors don't see an unknown
field. The native Gemini client now owns its request deadline: the SDK's
own timeout aborted without a reason and read as an internal error. A
deadline timeout returns 504 upstream_timeout and leaves the route in
rotation.

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-16 23:21:46 -07:00
145d54cf69 Handle unavailable chat model routes and exclude batch variants (#4230)
* Handle model access denials and filter batch chat variants

* Raise a low alarm when a provider denies model access

A 403 model_not_found still returns 400 upstream_bad_request, but now
alarms at info so a lost provider entitlement gets noticed. HttpError
gains an alarmSeverity option the terminal alarm gate honors.

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-10-16 23:21:46 -07:00
Reynaldi ChernandoandClaude Opus 5.5 471b900462 remove private access routing from docs (#4252)
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-16 20:14:21 +07:00
Daniel Salazar 2b254b8892 fix: misc bugs (#4251) 2026-10-15 23:49:17 -07:00
Neal Shah 2a15afc570 Resolve PUT-2059 (#4250) 2026-10-16 01:45:56 -04:00
dependabot[bot] f9860b036b chore(deps-dev): bump prettier from 3.9.9 to 3.9.10 (#4240)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.9 to 3.9.10.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.9...3.9.10)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.10
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-15 14:46:31 -07:00
dependabot[bot] 76e1f79b5e chore(deps): bump smtp-server from 3.19.17 to 3.19.21 (#4242)
Bumps [smtp-server](https://github.com/nodemailer/smtp-server) from 3.19.17 to 3.19.21.
- [Release notes](https://github.com/nodemailer/smtp-server/releases)
- [Changelog](https://github.com/nodemailer/smtp-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/smtp-server/compare/v3.19.17...v3.19.21)

---
updated-dependencies:
- dependency-name: smtp-server
  dependency-version: 3.19.21
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-15 14:43:54 -07:00
dependabot[bot] 943a3427c6 chore(deps-dev): bump @types/busboy from 1.5.4 to 1.5.5 (#4241)
Bumps [@types/busboy](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/busboy) from 1.5.4 to 1.5.5.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/busboy)

---
updated-dependencies:
- dependency-name: "@types/busboy"
  dependency-version: 1.5.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-15 14:38:42 -07:00
dependabot[bot] 38469a3615 chore(deps): bump libphonenumber-js from 1.13.14 to 1.13.15 (#4239)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.14 to 1.13.15.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.13.14...v1.13.15)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.15
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-15 14:35:15 -07:00
dependabot[bot] a4d4a5595e chore(deps-dev): bump vite from 8.1.5 to 8.1.6 (#4238)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.1.5 to 8.1.6.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v8.1.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.1.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.1.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-15 14:35:07 -07:00
dependabot[bot] de69269b10 chore(deps): bump uuid from 14.0.2 to 14.0.3 (#4237);'''"????????????????????
Bumps [uuid](https://github.com/uuidjs/uuid) from 14.0.2 to 14.0.3.
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/uuidjs/uuid/compare/v14.0.2...v14.0.3)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 14.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-15 13:25:49 -07:00
Neal Shah 8faff74793 PUT 2057 and PUT 2068 (#4245) 2026-10-15 02:02:00 -04:00
Neal Shah e5ca303c50 PUT-1903 (#4226) 2026-10-14 20:29:55 -04:00
Juan Fernando Castro c6c0dd5e29 fix: bound how many accounts a team owner can provision (#4223)
* feat: bound provisioning by what an owner may mint, not only what is live

Seat and team caps express how much may exist at one time. This adds the
other half of the rule: how much an owner may create over a window, and the
checks a provisioned account answers to.

- Provisioning puts each seat through `SignupService.runSignupGate`, the same
  extension-owned gate a signup answers, and refuses what it blocks. It fails
  closed, since a check that could not run is not a check that passed.
- A per-owner budget over a rolling window, sized to every seat the owner may
  hold live across every team they may own, times the churn allowed on top.
  It is spent on creation and is not returned.

Keyed on the owner's uuid so a recycled id cannot inherit someone else's
spend, and charged after validation so a taken username costs nothing.

* refactor: tighten the provisioning gate and budget after review

- The gate runs before the seat-cap lock. A listener has no deadline and the
  lease is ten seconds with nothing to renew it, so the lock must not be held
  across one.
- `failOpen: false` rethrows rather than returning false, so a backend outage
  is answered 503 instead of escaping the handler.
- An address goes through `validateEmail` first, as it does on every other
  path that creates an account.
- The gate always runs; a request only enriches what it is told, so one being
  absent is no reason to skip it.
- The gate is told it is looking at a seat, whose address is optional.
- A refusal carries the gate's own code and request code, as a blocked signup
  does, so support can correlate it.
- Restored `provisionAccount`'s docblock, which an insert had displaced.

Tests: the spy restores in `finally`, and the cap overrides are taken after
the fixtures, so one failure cannot cascade through the file.

* fix: show a provisioning refusal as a rule, not a failure

The console special-cases the seat cap and sends everything else to
"Something went wrong", which reads as a fault rather than an answer. The
provisioning budget refuses the same way the cap does, so it is presented
the same way.
2026-10-14 11:54:35 -04:00
Juan Fernando Castro e32bfa839d fix: count one app open per account per app per hour (#4152)
* fix: count one app open per account per app per day

`app_opens` is read two ways at once: `open_count` is a row count and
`user_count` is distinct accounts. A repeated open moves the first
without the first moving the second, and the incentive report prices
opens.

An open is now marked as it is recorded and skipped for the rest of the
day, before both the row and the event ClickHouse ingests, so the two
stores agree. The marker is set only when absent, so the first open of
the day wins a race; an unreachable marker counts the open, since
counting twice beats not counting.

* fix: count one app open per account per app per day, at the count

Dropped the row-suppression approach. `app_opens` is not only a counter:
`getRecentAppOpens` orders the desktop's recent list by `MAX(_id)`, and
`app.opened` is the socket event other tabs reorder on. Suppressing the
repeat row froze both at the first open of the day.

Every open is recorded as before. `open_count` counts each account once
per day instead, which is the figure `user_count` is read against.

`countDistinctUserDays` folds the pair into one number because
`COUNT(DISTINCT a, b)` is MySQL-only, and `dayBucket` covers MySQL's
decimal division. ClickHouse has uniqExact over the tuple.

Note the figure's meaning changes at deploy: a period spanning it mixes
rows with account-days, so a comparison across that point is not one.

* fix: count one app open per account per app per hour

Review feedback: a day is too long a window for the repeat to collapse
into. The period is now an argument rather than a baked-in 86400, held in
one constant on AppStore beside the other store policy numbers, so the SQL
and ClickHouse sites cannot drift and a future change is one line.

The fold `COUNT(DISTINCT user * N + bucket)` is only injective while the
bucket index stays under N. Hourly buckets are 24x the day index and pass
the old 1000000 in 2084, so N moves to 10000000.

Buckets are clock hours, not a rolling window, so two opens a minute apart
either side of the hour still count twice — tested, and the same property
the day version had at UTC midnight.

The detailed-stats fixture gave its repeat open a 60s offset, which under
hourly buckets would have counted twice whenever the suite ran in the first
minute of an hour. It now shares a timestamp with the open it repeats.
2026-10-14 11:54:17 -04:00
Daniel Salazar ff815f730d fix: harden token delivery, session revokes, app identity and fs scoping (#4229)
- relay sign-in popups take the opener origin from the referrer only
- session revokes read the primary, batch per user, and tombstone caches in every region
- wisp relay tokens bound to the minting session; verify rechecks the account
- hosted origins keep app identity with the site owner; deleted app uids recorded in deleted_apps
- subdomain cache tombstoned on delete, update, reclaim and app/account cascades
- email lookups require an exact address match after the SQL lookup
- workers deploy only from files the deployer owns
- share_app_data:false enforced at ACL time, plus copy/move/rename/share/recursive listing
- kv remove returns no value without read; kv:expire op grant usable
- fs errors name the entry instead of the owner's path
- shared-by-me drops rows a delegate no longer has authority over
- private apps can't move index_url onto another user's origin
- GUI: default apps saved and launched by uid; save picker accepts only copy/move; permission dialog waits for focus; Publish as Worker hidden on others' files
2026-10-14 06:56:44 -07:00
Juan Fernando Castro 84aaf005ec fix: give every route behind the user-protected gate a password budget (#4150)
* fix: give every route behind the user-protected gate a password budget

Five of the six routes behind the gate declare a per-user hourly bucket
for the password check. The sixth, delete-own-user, declared none.

The bucket moves into the gate itself, so it covers every route behind
it and a new one cannot be added without one.

It is charged by the outcome rather than by the request: a wrong answer
spends one and a right answer spends nothing, so a caller who knows the
password never meets the limit.

* fix: the review's findings on the password budget

Reading the budget and spending it on the outcome bounded rounds rather
than attempts: everything already in flight read the same budget and was
answered before any of it was spent. The charge lands before the
comparison now, so a burst is bounded by the same number as a sequence.

The key gained a route component. One bucket across all six meant ten
mistyped passwords on one route closed the other five for the hour,
including the one an account owing its team's 2FA has to reach.
2026-10-14 09:52:23 -04:00
Daniel Salazar f6a1e54642 fix(fs): keep S3 read stream errors from crashing the process (#4227)
getObjectStream now attaches an 'error' listener to every body it returns.
A body that fails before its consumer attaches (socket timeout, reset, or a
checksum mismatch on a truncated read) used to emit 'error' with no listener
and exit the process, failing every in-flight request on the task. It now
logs a warning; consumers that attach later still receive the error.
2026-10-13 17:42:28 -07:00
Filip Kujundžić d211a25c6d Merge pull request #4218 from HeyPuter/codex/put-1851-deepseek-peak
Protect direct DeepSeek peak-only pricing
2026-10-14 00:54:11 +02:00
404oops 2e1bf86d2b test(ai): protect direct DeepSeek peak-only pricing 2026-10-13 20:44:00 +02:00
404oops 8c31670453 Align ZAI modality metadata with source conventions 2026-10-13 20:43:58 +02:00
404oops 16d0ce8b3e Advertise supported image input for ZAI Flash models 2026-10-13 20:43:58 +02:00
404oops 8c64f54a87 Remove ignored-option paragraph from speech reference 2026-10-13 20:43:54 +02:00
404oops 7abd7b4417 Remove unsupported speaker reference option 2026-10-13 20:43:54 +02:00
404oops 29e0e1e2aa Clarify speech catalog units and cover ignored transcription options 2026-10-13 20:43:54 +02:00
404oops 4857898af4 Assert finite ElevenLabs catalog prices 2026-10-13 20:43:54 +02:00
404oops 526a250c95 Correct speech catalog pricing and legacy option declarations 2026-10-13 20:43:54 +02:00
404oops 8c18c45e63 feat(ai): refresh verified provider pricing catalogs 2026-10-13 20:43:43 +02:00
404oops 124f95a5f9 Label provider catalog fixtures as test data 2026-10-13 20:43:43 +02:00
Neal ShahandClaude Opus 5.5 4401b4c4dc Fix worker test (#4156)
* PUT-2012 fix broadcast service drops

* Revert pre-commit lint churn on files merged from main

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix test bug

* Remove other test

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-13 14:10:10 -04:00
Neal Shah 62270ff9aa Delete worker sessions when workers are deleted, don't treat sessions from deleted workers as still valid. (#4163) 2026-10-13 12:43:45 -04:00
404oops ddafc2d04c fix(ai-chat): normalize remote catalogs and conditional pricing 2026-10-13 18:25:36 +02:00
Daniel Salazar b52fc01712 perf: cut godmode app launch latency (cached token renewal, launch reuse) (#4221)
* perf(auth): renew godmode app tokens from cache instead of the primary

Every godmode launch read its token row from the primary and rewrote its
expiry, a few cross-region round trips per launch far from the primary.

- A cache pointer names the row per (user, web session, app). The row is
  read through the uuid cache token auth uses and re-checked (owner,
  session, app, revoked, expired, max age); anything else falls back to the
  primary read. Revoking the row drops the pointer.
- The expiry write is skipped when it would move by under an hour. A
  renewal that hits a revoked or lapsed row mints a new one instead.
- The response carries the row's actual expires_at.

* perf(gui): reuse a just-minted godmode token for repeat launches

A launch of a godmode app reuses the token this page minted for it within
the last ten minutes under the same desktop session, instead of asking
again. The cache is dropped when an app reports its token dead, after a
session revoke from the session manager or an uninstall, and across tabs
via a BroadcastChannel that carries token row ids, never tokens. A mint in
flight during a drop is not remembered.
2026-10-13 08:53:55 -07:00
Himanchal Kaushale f37cb22d8a fix(gui): resize app content above mobile keyboard (#4165)
Add interactive-widget=resizes-content to the mobile viewport meta tag
so the layout viewport shrinks when the soft keyboard opens. The app
window's height chain resolves against the layout viewport, so the
iframe folds up above the keyboard instead of clipping behind it.
2026-10-13 08:53:44 -07:00
Reynaldi Chernando 948772165d Various docs fixes and AGENTS.md update (#4211) 2026-10-13 17:29:46 +07:00
Neal Shah b8bb6292d6 Fix webdav locks issue and tighten GUI ipc rules for setWindow* (#4207) 2026-10-12 17:39:09 -04:00
Filip Kujundžić 542ca9ee1c Merge pull request #4202 from HeyPuter/filipkujundzic/speech-contract-fixes
fix(ai-speech): honor ElevenLabs output formats, voice IDs and URL audio
2026-10-12 16:21:33 +02:00
404oopsandClaude Opus 5.5 8e92bf6803 fix(ai-speech): honor ElevenLabs output formats, voice IDs and URL audio
- TTS sends output_format as the query parameter ElevenLabs reads; the body
  field was ignored, so every request came back as MP3. Unknown formats are
  rejected before credits are reserved, and the OpenAI-style names mp3, opus,
  pcm and wav map to native formats.
- Voice IDs go straight to synthesis. The library listing omits voices that
  still synthesize, the default 21m00Tcm4TlvDq8ikWAM among them, and runs to
  9 MB per call. Names resolve through /v2/voices search, and premade voices
  ("Bella - Professional, Bright, Warm") also match their short name.
- speech2speech accepts HTTPS audio URLs through secureFetch, as documented.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 12:40:43 +02:00
404oopsandClaude Opus 5.5 b61448c8b7 feat(ai-image): add Gemini Nano Banana 2.1
Catalog gemini-nano-banana-2.1 (aliases google/gemini-nano-banana-2.1 and
nano-banana-2.1) at Google's standard rates: $1.50/1M input, $7.50/1M text
and thinking output, $30/1M image output, with 1K/2K/4K tiers at
1120/1680/3780 image tokens. 3.1 Flash Image stays the provider default.

Pre-flight credit checks assumed 50 text/thinking output tokens per image
for every Gemini model; live samples run 270-1122 depending on the model, so
each model now carries its own estimate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 12:39:49 +02:00
404oops f9cde23937 Merge branch 'filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so' into filipkujundzic/put-1907-ai-streaming-buffering-and-error-shape-cleanup
# Conflicts:
#	src/backend/drivers/ai-chat/utils/inlineImages.test.ts
#	src/backend/drivers/ai-chat/utils/inlineImages.ts
#	src/backend/drivers/util/fileInput.ts
2026-10-12 12:39:35 +02:00
404oops 94e84a2819 Merge branch 'main' into filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so 2026-10-12 12:36:26 +02:00
404oopsandClaude Opus 5.5 f0b55e7605 docs(ai): trim listModels return value and AI overview per review
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 12:33:05 +02:00
Daniel Salazar d8f9aff3dd fix: follow-ups to #4161–#4203 (rate limits, broadcast, AI routing and billing, fs subtree walks) (#4205)
* fix(rate-limit): count the newest KV rate rows first

The KV backend's rate check, peek and concurrent acquire list the key's
rows with a limit + 1 cap that the store applies before dropping expired
rows, so unswept rows from past windows (oldest keys) could fill the fetch
and the live count read as zero. They now list newest first.

* fix(broadcast): split webhook flushes under a byte budget, per-peer backlogs

Flushes go out in bodies of at most `broadcast.outboundMaxBodyBytes` (default
4 MiB) instead of one POST, and each peer keeps its own backlog of what it
missed, capped by count and four bodies' worth of bytes, oldest dropped first,
so one failing peer can no longer grow every peer's body past their 50 MB limit.

* fix(ai-chat): classify dropped upstream connections as provider failures

A connection reset or refusal (SDK APIConnectionError, Mistral's
ConnectionError, an undici reset on the cause) carries no status, so a chain
of them fell through to 500 internal_error. Attempts now record
connectionFailed; a chain of only resets and 5xx answers 400
upstream_provider_unavailable, and a reset counts as an upstream signal in a
mixed chain.

* fix(ai-chat): keep SDK retries on Anthropic count_tokens

The Claude client is built with SDK retries off so the driver can fall back,
but count_tokens has no fallback route. It now passes sdkRetryOptions, like
the OpenAI moderation call, so a transient 429 or reset is retried.

* fix(ai-chat): mark a route unhealthy for 60 s after a transient failure

With SDK retries off, one 5xx, 429, timeout or dropped connection kept a
route marked for 10 minutes, sending that model's traffic to resellers. Those
failures now mark the route for 60 s; bad credentials, an exhausted account
and other route-level failures keep 10 minutes.

* fix(metering): don't memoise spent addons inside the force-exact window

The spent-addons memo could cache a pre-grant record read from a lagging
replica right after a credits-changed announcement, so overage skipped the
new credit and requireCredits kept answering 402 for up to 15 s. A spent
record is now only cached once the actor's force-exact window has ended.

* fix(share): send the digest entries a flush claimed when another take fails

Digest entries are taken concurrently, and a single rejected take threw
away every entry the other takes had already removed from KV. The flush
now logs the failed takes and sends what it claimed; an entry whose take
failed stays queued for a later flush.

* fix(fs): announce an overwriting move's replaced entry only when it took its name

/move and v2 /fs/move looked up the entry at the destination name before
moving and always announced its removal. A move into someone else's Trash
renames instead of replacing, so the owner's desktop hid a live entry and
legacy /move reported it as `overwritten`. Both routes now pass the target
on only when the moved entry kept the name it was looked up by.

* fix(ai): name the vendor in upstream error fields from media providers

upstreamFetch now takes the provider id plus a display label, and
UpstreamResponseError carries the id. translateProviderError copies a
string `provider` off the error into `fields`, and the BytePlus video and
Cloudflare image errors set one, so upstream_auth_failed and
upstream_credits_exhausted alarms say which vendor failed again.

* fix(teams): delete a provisioned seat whose home folders could not be created

provisionUser logs a failed default-folder creation and still returns the
account, so provisionAccount went on to add the membership, issue a password,
mail the seat and emit team.account.created for an account with no home. It
now cascade-deletes the new user and returns 500 internal_error before any of
that.

* fix(ai-chat): load model catalogs without holding up server start

onServerStart awaited every gateway catalog (up to its 10s fetch timeout)
while HTTP was already listening, so drivers started after it had no
providers yet and chat requests found an empty model map. The first build
now runs in the background; complete(), models() and token counting wait
on it instead of failing.

* fix(ai-chat): serve direct models while gateway catalogs load at boot

The first model-map build now publishes a fresh map as each provider's
catalog lands, so static catalogs are routable at once. complete() and
token counting wait for the full first build only when the model, or the
provider asked for, isn't in the map yet. Refreshes still swap once.

* fix(ai-chat): split gateway-billed cost across input and output

A completion billed at the cost OpenRouter, Infron or Neuralwatt reports
was quoted to `ai.prompt.cost-calculated` as all input and no output. The
ledger still records one `billedUsage` line; the reported per-key costs now
spread that bill over the usage keys at the catalog's relative rates, in
whole µ¢ that sum to it. Neuralwatt energy units take no share.

* fix(ai-image): hold Replicate credits for the whole prediction

The output and input-surcharge holds took the default 10-minute TTL, the
same as the prediction window alone, so a slow prediction could outlive
them once creation and input measurement were added. They now last
15 minutes, like the video job holds.

* feat(ai-chat): report what a completion was charged as usage.cost

Chat usage (the non-stream result and the stream's final usage line) now
carries `cost`: the µ¢ the ledger records for the request after the cost
factor, from the request's recorded charges and the gate's factor lookup.
`usd_cents` is unchanged. A hooked recording now copies the usage object
first, so keys the driver adds later never land in the ledger.

* fix(ai): charge streamed speech once its first byte arrives

Per-character TTS and the voice changer recorded their charge when the
upstream's response headers arrived, so a body that failed before any audio
was billed in full. A shared relay now charges on the first byte, or when
the caller hangs up, and charges nothing for an upstream that fails or ends
before sending audio.

* fix(ai): release credit holds only after the request's usage lands

A hold was released as soon as its operation returned, while the usage it
stood for was still being written (behind the cost-factor lookup), so a
concurrent request briefly saw neither. AI metering now tracks the
request's usage writes and its holds (chat included) release once those
settle, without delaying the response; a streamed charge chains its own.
A failed write still releases the hold, and the hold's TTL backstops it.

* test(ai-chat): wait for a stream's credit hold to come back after its usage lands

* perf(fs): page subtree delete and copy by parent link, not by depth over the path range

Each page of a recursive delete or copy sorted the whole remaining subtree by a
computed depth, so a walk cost grew with the square of its size. It now pages
the children of up to 100 sibling directories at a time by parent_uid, depth
first, keyset on (parent_uid, id), so a page reads only its own rows; rows with
no stored path are reached too. SQLite gets the parent_uid index the others have.

* fix(fs): find children by parent_uid in the empty check and supplementary-name lookup

Rows the old backend wrote carry parent_uid and no parent_id, so a non-recursive
delete of a directory holding only such rows passed the empty check and orphaned
them, and on MySQL the supplementary-character child lookup missed them, leaving
their subtrees out of listings, counts and renames. Both now key on parent_uid;
the empty check still probes parent_id too, which the delete cascade follows.

* docs(fs): drop the single-POST note from the entry broadcast cap
2026-10-12 01:10:54 -07:00
Daniel Salazar b88aa55b24 fix: drop moved descendants from the entry cache; read cursor=null as the first page (#4204)
Renaming or moving a directory rewrote its descendants' paths but left their
cached copies (by id, uuid and old path) live for up to 60 s. A lookup of an
old path kept resolving, and a write beneath one landed at a path its parent
no longer had. updatePathPrefixForUser and renameUserHome now drop those keys,
here and, within the broadcast budget, in peer regions.

puter.js sent `cursor=null` for `listShared`/`listSharedByMe({ cursor: null })`,
which decoded to null and failed every keyset listing with a 500. decodeCursor
reads it as no cursor, and the SDK no longer sends it.
2026-10-12 00:19:11 -07:00
Daniel Salazar ff13edc19a Merge branch 'main' into filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so 2026-10-12 02:21:00 -04:00
Daniel Salazar f07a786b41 fix: reserve storage before creating directories on signed write starts (#4161)
startBatchWrite and startWrite created directory items and missing
parents before (or alongside) the storage reservation, so an over-quota
start paid for every directory write, left the directories behind, and
waited on them before returning 413. File targets are now resolved and
storage reserved first; directories are created only after, and the
reservation is released if that fails.
2026-10-11 22:14:57 -07:00
Daniel Salazar 21131ad3d6 fix(kv): spread upload sessions across system KV partitions (#4162)
Upload sessions shared the system namespace's single partition key. They
now go to one of 16 system partitions hashed from the session id, via a
new system-actor-only `systemPartition` KV option; partitions are never
cached and emit no mutation events. Reads fall back to the unpartitioned
key for sessions written before this (max session lifetime is one hour).
2026-10-11 22:14:11 -07:00
Daniel Salazar 38c986927c fix(fs): page subtree walks over supplementary-character names on MySQL (#4203)
listDescendantsInOrder, which drives paged recursive delete and copy, still
used a bare `path LIKE` range, so on MySQL a child whose name starts with a
supplementary character (and its subtree) was skipped: delete left it to the
parent_id cascade and copy dropped it. It now ORs the subtree patterns like the
other subtree queries, paging each pattern group with the same seek and
merging the pages in order.
2026-10-11 22:11:39 -07:00
Daniel Salazar 590c87c00a refactor(stores): one row cache and keyset page helper, batched cache round trips, dead store methods removed (#4191)
* refactor(stores): one read-through RowCache for UserStore and AppStore

UserStore and AppStore each carried the same multi-key row cache: key
builder, cache read, tombstone-guarded backfill, broadcast refresh,
invalidate and tombstone-on-delete. They now share RowCache on PuterStore,
which also single-flights concurrent misses for one key and batches
getByIds/getByUids: one pipelined tombstone check for the loaded rows and
one pipelined backfill instead of per-row awaits. Cache keys and TTLs are
unchanged.

* perf(apps): invalidate cached app lists by generation, not by tracker scan

Every app write SMEMBERS'd a global set of every cached list key, parsed
each key back into its filters and deleted the matches. List pages are now
keyed by their owner's generation token (or a shared one for unscoped
lists), and a write replaces the tokens it affects: O(1) per write, and a
page loaded before a write can no longer be cached after it.

* refactor(stores): one keyset page helper for store listings

Thirteen store listings each cut a LIMIT n+1 read to a page and sealed the
last row's position, and each opened an id cursor its own way. They now use
keysetPage() and openIdCursor() from util/pagination. Cursor payloads and
sealing are unchanged; a cursor without a usable id still fails with 400 in
ShareStore and still restarts the listing elsewhere (strict option).

* refactor(stores): delete store methods nothing calls

AppStore.existsByIndexUrl, OIDCStore.unlinkByUserId,
TeamStore.listByOwner/listActiveSeatUuids, SubdomainStore.getByDomain/
listByDomain, FSEntryStore.ensureDirectoriesForUser,
ShareStore.listByIssuer/listByRecipientEmail/deleteByRecipientEmail/
getDailyShareCount/listByHolderAndFsentries and
PermissionStore.invalidateScanCache/deleteUserUserPermsByPermissionPrefix
had no production callers. ShareStore.listByFsentries was a copy of
listReaching; its callers use the latter.

SubdomainStore SMEMBERS'd a per-user prefix-list tracker on every write and
broadcast its deletion, but nothing ever wrote that set or cached a prefix
listing. The reads are gone.

* refactor(stores): shared escapeLike and bumpGeneration helpers

escapeLike replaces the LIKE escaping copied into PermissionStore (twice),
ShareStore and FSEntryStore, and now also guards SubdomainStore's prefix
filters, which passed caller prefixes into LIKE unescaped.

bumpGeneration sends INCR and EXPIRE in one pipeline instead of two awaited
commands in PermissionStore, EventSubscriptionStore and PresenceStore.

* refactor(stores): read affected rows off WriteResult

Every database client returns affectedRows and anyRowsAffected, so the
`affectedRows ?? changes ?? 0` fallbacks in ShareStore, NotificationStore,
UserStore and UserBlockStore read those directly.

* perf(stores): batch flat-permission KV deletes and durable cache drops

PermissionStore deleted each flat KV entry with its own kv.del; one
kv.batchDel now covers the set. DurableSubscriptionStore's suspend and reap
dropped each row from the cache separately; EventSubscriptionStore gains
dropDurables, which drops them per owner in one pass, and dropDurable calls
it. suspend keeps one conditional UPDATE per row, which is how concurrent
settles learn which rows each of them suspended.

* refactor(fs): fold FSEntryStore's duplicate getters, sort maps and cache keys

- getEntryByUuid and getEntryById share one by-column getter.
- listChildren and listChildrenPage read one sort table; the offset listing
  now coalesces NULL modified/size like the cursor listing, so both order
  NULLs the same way on every engine.
- Entry cache keys come from one builder instead of hand-built strings.
- getEntriesByIds, getEntriesByPaths and the per-user path reader fetch
  their cache hits in one pipelined round trip instead of a GET per key,
  through readCacheValues, which RowCache.readMany now uses too.

* fix(apps): cache the row a lost createFromOrigin race reads from the primary

Keeps AppStore's private #writeCache as a RowCache write, so code that
still calls it (#4166's read-back) loads in either merge order.
2026-10-11 22:07:53 -07:00
Daniel Salazar ed22e2b285 docs: tighten contributor and architecture guides (#4194) 2026-10-11 22:07:02 -07:00
Daniel Salazar 9030b45459 refactor(ai-chat): one OpenAI-compatible provider, one pricing path, cached catalogs (#4192)
* perf(ai-chat): cache gateway catalogs with one remote-catalog helper

cachedRemoteCatalog serves a fetched catalog for 15 minutes, shares one
in-flight fetch between callers, abandons a fetch after 10 s, remembers a
failure for 30 s and keeps the last good value through a failed refresh.

Together, OpenRouter, Infron, Neuralwatt (models and quota) and Ollama use it
in place of their kv-singleton caches. The cached value is the converted
catalog, so a completion no longer re-coerces the whole gateway catalog
(OpenRouter did it twice per request). Ollama now refreshes instead of caching
its first non-empty list forever, and Together's catalog failure no longer
rejects models().

* refactor(ai-chat): build every chat SDK client with retries off

sdkClientOptions() gives each OpenAI, Groq, Together and Anthropic client
maxRetries: 0 and an explicit timeout. The SDK default of two retries, each up
to the full timeout, stacked under the driver's three-route fallback chain; a
failed attempt now goes straight back to the driver, which marks the route
and moves to the next provider for the model.

Timeouts keep what each provider already had: ten minutes, Groq's SDK default
of one minute, and Claude's 10 m 0.6 s. Non-streaming Mistral calls, which had
no timeout once a cancellation signal was passed, are bounded at ten minutes.
The Azure providers are left for after the open Azure fixes land.

* refactor(ai-chat): meter every OpenAI-family completion through one pricing path

meterChatUsage records a completion's usage at buildCostsOverride's rates, or
as one billedUsage line when the gateway reports what it billed, and returns
the per-key costs it recorded. Providers hand those to the driver through the
usageCosts channel, now carried by the Chat Completions handlers too, so the
usd_cents and cost-calculated figures equal the ledger. The driver's own
#computeCost is replaced by pricing.ts usageCost, which shares the rate
resolution with buildCostsOverride.

Billing changes, all toward what the upstream charges:
- DeepSeek, Moonshot, Alibaba, Groq, MiniMax, Z.AI, Hoonify, BytePlus and the
  Neuralwatt catalog fallback took cached reads out of prompt_tokens nowhere,
  so a cached token was billed at input + cache rate. splitCachedPrompt bills
  it once, at the cache rate; usage.prompt_tokens on those routes now excludes
  cached reads, as it already did for OpenAI, Gemini and Meta.
- Catalog entries that listed cached_tokens: 0 (38 Alibaba, 5 Groq) relied on
  that double count to bill cached reads at the input rate. The zero is
  dropped, so they keep exactly that price through the input-rate fallback.
  Neuralwatt models with no published cache price likewise.
- An unpriced key falls back to the input/output rate instead of NaN (0), e.g.
  OpenRouter cached reads on its no-cost fallback path.
- A gateway-reported cost of 0 records 0 µ¢ rather than 1 µ¢.

Claude, Azure and xAI keep their metering; xAI's cached-token handling waits
on the open Grok decision.

* refactor(ai-chat): one OpenAICompatProvider for the OpenAI-compatible vendors

DeepSeek, Moonshot, Alibaba, Groq, MiniMax, Z.AI, Hoonify, BytePlus, Together,
Gemini, Meta, Ollama and the complete() paths of OpenRouter, Infron and
Neuralwatt were copies of one Chat Completions call. Each is now an
OpenAICompatOptions entry (client, catalog, metering prefix, wire-id prefix,
max-token param and default, passthrough fields, openAICompatParams scope,
message cleanup, image inlining, stream usage, context-length retry) plus the
hooks its API needs: prepareMessages (DeepSeek, Groq, Gemini), vendorParams
(MiniMax, Hoonify, Z.AI, BytePlus, Meta, OpenRouter, Infron, Neuralwatt),
meteredUsage (Gemini and the gateways) and reportedUsage (Neuralwatt). xAI is
left until the open xAI fix lands.

Deliberate changes:
- Tools go through make_openai_tools for every vendor (six did, ten sent them
  raw): Anthropic-only tool extras and server tools no longer reach these
  upstreams, and an empty tool list is omitted.
- Model lookup is one indexed, case-insensitive pass per catalog for every
  vendor (Together and Neuralwatt already ignored case).
- Per-provider console logging of upstream errors is gone; the driver logs
  every failed attempt.
- Every vendor honours a configured apiBaseUrl.

tool_choice forwarding stays as each vendor had it (MiniMax, Hoonify, Z.AI
auto-only, BytePlus, Meta auto-only, OpenRouter); widening it needs each
vendor's support confirmed.

* perf(ai-chat): resolve the model once in the driver and keep its map fresh

The driver hands each provider its own catalog entry for the route it chose,
so providers stop re-resolving the id per request; the OpenAI-compatible
providers only look an id up (indexed, case-insensitively) for a direct
caller, and an unknown id is a 400 instead of silently becoming the default
model. Gateways with mixed-case ids (Infron, Neuralwatt) used to miss the
driver's lowercased id and fall through to that default.

The model map is now built from every provider's catalog in parallel, with a
failing catalog left out rather than aborting the build, swapped in whole,
awaited in onServerStart, and rebuilt every 15 minutes so gateway catalog
changes reach routing without a restart.

* refactor(ai-chat): drop provider surface and OpenAIUtil code nothing uses

- list() leaves IChatProvider and every provider that isn't Azure or xAI
  (both wait on the open fix for those files); only tests called it.
- OpenAI Chat, OpenAI Responses and Meta no longer inline puter_path parts:
  the driver does it before every attempt, so theirs found nothing to do. The
  stores and FS service they took for it go too.
- providers/ChatProvider.ts had no subclasses.
- OpenAIUtil: one hoistToolUseBlocks for the Chat and Responses message
  passes (Responses keeps canonical_id), and the dead create_usage_calculator,
  unused Responses deviations and constant mode/toolblock are gone.

checkModeration stays on IChatProvider: the open Azure moderation fix calls it
through that interface.

* refactor(ai-chat): register keyed providers from one table

Sixteen near-identical blocks in #registerProviders, with three constructor
argument orders, become a { name, configKey, Class } table read by one loop;
every entry takes (config, metering) and gets apiBaseUrl when configured.
Claude, Azure, OpenAI (sibling wiring), Ollama (enabled unless turned off) and
the fake provider stay explicit. Gemini's constructor takes config first like
the rest.

* fix(ai-chat): send Ollama the model name it listed

Ollama catalog ids are ollama:ollama/<name>, built from /api/tags. Only the
ollama: part was stripped, so the local server was asked for ollama/<name>, a
model it doesn't have. Strip the whole ollama:ollama/ namespace.

* fix(ai-chat): keep SDK retries on the OpenAI moderation call

The completion clients run with SDK retries off so the driver can fall back.
A moderation check has no fallback, so it passes the SDK's default two
retries per request.
2026-10-11 22:06:18 -07:00
Daniel Salazar f3134b2530 refactor(ai-media): shared upstream fetch, provider registry and bounded inputs for AI media drivers (#4188)
* refactor(ai): route TTS, STT and voice-changer REST calls through upstreamFetch

upstreamFetch bounds a provider call with AbortSignal.timeout and throws an
UpstreamResponseError carrying the upstream status and code, which the
/drivers/call boundary maps like any SDK error. Replaces five hand-written
status translators (ElevenLabs TTS, xAI TTS, Speechify, xAI STT, voice
changer); caller-facing messages are now sanitized by the shared mapper.

* perf(ai): bound input image fetches and share one data-URI parser

fetchImageBytes now streams under a byte cap (30 MB by default) with a
30-second timeout, via a readBoundedBody helper shared with loadFileInput's web
branch and Replicate's input measuring. assertInputImagesShape caps
input_images at 16. Chat image inlining uses fetchImageBytes instead of
buffering with no timeout. fileInput, imageInput and mediaParts parse data URIs
through one parseDataUri.

* refactor(ai-tts): one per-character metering path for TTS providers

meterPerCharacter holds text length x price, runs the synthesis and meters the
same amount once it succeeds; characterCostReport builds the per-character
cost lines. Both sit on a new withAiCreditHold (reserve, run, release) that
Gemini TTS uses directly. SAMPLE_AUDIO_URL is defined once. The TTSProvider
base class is gone: its methods were stubs every provider overrode, and only
Polly read its untyped providerConfig.

* refactor(ai): one ProviderRegistry and key reader for the AI media drivers

ProviderRegistry holds a driver's provider catalog (ids, default, aliases) and
its configured instances: alias resolution, the not-found 400, default
fallback and parallel 'all' listing. TTS, speech-to-text, OCR and the voice
changer route through it. readProviderKey reads apiKey, secret_key, api_key and
key in that order for every AI media driver, so providers.xai.secret_key now
enables xAI TTS as it already did xAI image and speech-to-text. ElevenLabs
caches its voice list for 10 minutes, as Polly does.

* perf(ai): stream TTS and voice-changer audio instead of buffering it

OpenAI, xAI and ElevenLabs TTS and the voice changer hand the upstream body
straight to the caller through upstreamBodyStream; their costs are known
before the call, so nothing waits on the download. A body that fails
mid-stream ends the response early instead of raising an unhandled stream
error. xAI speech-to-text keeps one copy of the audio (the upload Blob)
instead of up to three. The voice changer and xAI STT hold credits through
withAiCreditHold.

* perf(ai-ocr): read Puter files only when the provider needs the bytes

resolveFileInput resolves and access-checks a Puter file and defers the read
to load(); loadFileInput is that plus the read. OCR resolves first: Textract
reads the object straight from S3 and downloads it only if Textract can't
reach the bucket, and only Mistral loads the bytes. A file over the size cap
is refused from its recorded size before any read. countPdfPages inflates
object streams on the libuv pool instead of the request thread.

* perf(ai): build image and video model catalogs once and share output saving

ModelCatalog holds what the public model routes serve (models, sorted names,
cost lines), computed once after boot instead of on every listing. The video
driver now awaits its model map before serving. resolveOutputPath and
saveGeneratedMediaToFS replace the copies of puter_output_path handling in
both drivers, using expandTildePath and normalizeAbsolutePath. The video
driver normalizes a copy of the caller's args instead of mutating them, and
its never-taken stream branch is gone.

* fix(drivers): abort the response when a streamed driver result fails mid-body

upstreamBodyStream now errors when the provider body fails instead of ending
cleanly, and DriverController pipes stream results through pipeStreamResult
(stream.pipeline): a failed source destroys the response so the client sees
an aborted transfer, the failure is logged once, and nothing is left to raise
an unhandled stream error. A client hanging up stops the source.

* fix(drivers): drop a streamed driver result when the caller already hung up

pipeline() throws synchronously on a closed destination, which surfaced as
a 500 alarm on every early disconnect and left the upstream body open.

* fix(ai): bound streamed upstream calls by time to first byte, not download time

upstreamFetch takes a `stream` option: the timeout covers the request until
the response (or a non-2xx error body) is read, then is cleared so a body
piped to the client drains at the client's pace. ElevenLabs TTS, xAI TTS and
the voice changer pass it; non-streamed calls keep the full-response bound.
A stalled upstream body is still cut by undici's body idle timeout.

* fix(drivers): type pipeStreamResult's response by the members it uses
2026-10-11 22:05:37 -07:00
Daniel Salazar 45f6e9d8ee refactor(drivers-wiring): AppService views and launch gates, batched app lookups, shared driver policy and guards (#4186)
* perf(workers): skip hot-reload lookups for files no worker is bound to

The fs.write.file / fs.remove.node / fs.move.node listeners listed every
workers.puter.* row of the file's owner on every event, for every user.
The event's FSEntry already carries the worker rows bound to it, so a file
with none returns before any I/O; a bound file reads just its own rows,
batched, from the primary.

* perf(apps): read the uid alias only after a direct lookup misses

Every AppDriver uid lookup fired an uncached system-KV GetItem for the
merge alias in parallel with the direct read, even when the direct read
hit. The alias is now read only on a miss.

* refactor(apps): one app view with the launch gates, built by AppService

AppDriver's serializer and its private-app / hosted-backing gates move to
AppService.views, built on a pure toAppView that is the one place
index_url is withheld. AppDriver, the subdomains driver's embedded apps and
the GUI shell view all use it. Visibility of protected apps
(filterReadable) moves with it.

Subdomain embeds (associated_app, app_owner) used to return index_url with
no gate; they now withhold it from a viewer who may not open a private app
or whose hosted backing is gone, and carry privateAccess, godmode and
feedback_enabled like a direct app read.

* refactor(apps): one toAppSummary; batch the recommended and taskbar lookups

SuggestedApps, RecommendedApps, /get-launch-apps and the taskbar each built
their own launch summary; they now share toAppSummary (the extra
feedback_enabled / external flags appear on the lists that lacked them).

getRecommendedApps awaited getByName once per recommended app (29) and
getTaskbarItems did the same per taskbar entry on every whoami; both now
make one batched call per key through the new AppStore.getByNames, which
keeps getByName's case-insensitive and old-name matching. The taskbar's
raw UPDATE of the user row goes through UserStore.update, and
/get-launch-apps checks hosted backings in one batch and reaches the
stores without casts.

* refactor(apps): app routes read through AppService; /delete-site calls the subdomains driver

AppController used the global driversContainers through a getter whose
boot-order comment no longer held; it uses the injected drivers and
AppService. GET /apps/:name looked each name up, read it again through the
driver and resolved private access a second time, emitting
app.privateAccess.resolveLaunch twice per private app; it now batches the
names and resolves once. POST /query/app did a lookup, a driver read and a
raw app_filetype_association SELECT per selector (up to 200, serially); it
now makes one batched lookup per key, one grant check and one filetype read.

The driver's read shape is a pure toAppReadView both use.
HostingController's /delete-site re-implemented the subdomains driver's
delete (protected check, delete, subdomain.delete event); it keeps its
owner-scoped 404 and calls the driver. Hosting and Desktop controllers drop
constructors that discarded the drivers argument, and trivial getters.

* refactor(drivers): one per-method policy validator and resolver; drop @Driver

Four policy blocks (rateLimit, concurrent, requireSubscription,
requireReputation) share the { default, methods } shape but each had its
own validator and resolver. They become validatePerMethod(value, label,
policy) and resolvePerMethod(cfg, method); error messages are unchanged.

The @Driver class decorator and the prototype keys it wrote had no users
outside its own tests; drivers declare their fields, and resolveDriverMeta
reads only those. DriverController takes the driver name from the meta it
already resolved instead of probing the prototype.
resolveDriverMethodRateLimit/Concurrent stay as deprecated aliases for the
email extension.

* refactor(drivers): shared actor guards and crud-q concurrency policy

The apps, subdomains, workers and notifications drivers each carried a
private #requireActor, two carried the same #requireVerified, and apps and
subdomains the same owner/app_owner write predicate. These become
requireContextActor, requireContextUserActor, assertActorEmailVerified and
actorOwnsRow in core/actor.ts; each driver keeps its own error for the
cases that differed (system actors still read apps; notifications still
answer 403 for an actor without a user). The identical crud-q concurrency
block is one CRUD_CONCURRENT constant.

The metering, installedApps and appTelemetry extensions re-checked an actor
their routes already require; those checks and the tests for them go.

* refactor(extensions): store methods instead of raw SQL in OSS extensions

installedApps and appTelemetry queried user_to_app_permissions directly
and metering read apps by name with its own SELECT. PermissionStore gains
listAppsGrantedByUser, listAppAuthenticatedUsers (with the email-share
check folded in) and countAppUsers; metering uses AppStore.getByName.

* refactor(server): one layer builder and one lifecycle hook runner

PuterServer built each of its five layers with its own copy of the same
loop (built-ins, then extension additions, into the layer and its export
container), and ran each lifecycle hook with five near-identical loops per
hook. Both are now one helper each: #instantiateLayer and
#runHook(order, hook). Construction order, the args each layer receives,
and hook order (start and prepare bottom-up, shutdown top-down) are
unchanged.

* fix(workers): tear down a deleted source's worker after its FK nulls root_dir_id
2026-10-11 22:04:23 -07:00
Daniel Salazar d162549c1d refactor(controllers): one signup, login, confirm and window.user path; userProtected/internalAuth route options (#4189)
* refactor(auth): one signup path for form, temp, OIDC and team accounts

Signup steps (validate hook, phone/card forcing, bonus verdict, provisioning,
signup events) move from AuthController and OIDCService into SignupService.
Username rules, availability and generation move to util/username.ts, and
user-row + group + default-folder creation to provisionUser, which
TeamService and DefaultUserService now share.

Behaviour fixes: a provisioned team seat joins default_user_group like every
other account, and generated temp usernames skip names whose home path is
occupied.

* refactor(auth): one web-session start for password and OIDC sign-in

startWebSession creates the session with the request's IP, user agent, origin
and host and sets the session cookie. OIDC sign-ins used to create their
session without that metadata, so they showed up blank in the session list.

* refactor(auth): one confirmEmail for the code and link confirmation routes

Both routes now re-run the email checks, refuse an address another account
confirmed, strip it from other rows, promote the group and push
user.email_confirmed to the account's sockets (with the originating socket id
on the code route). The link route's full-table UPDATE clearing other
accounts' pending change-email requests is dropped: /change_email/confirm
already refuses an address a confirmed account owns.

* refactor(auth): build window.user once for whoami and sign-in

buildUserDetails and scrubSensitive move from the whoami extension into
util/userDetails.ts. /whoami calls them; sign-in responses now return the same
user-actor payload, scrubbed, so the login copy no longer ships the phone
number and a no-reload login gets oidc_only/otp/directories up front. Taskbar
resolution is best-effort on both.

* refactor(http): userProtected route option replaces AuthController's route walker copy

The user-protected gate is materialized from a userProtected RouteOption
(implies requireUserActor), so AuthController no longer re-implements the
decorator walker to wire it per instance; those routes and the session
routes are plain decorators now. AuthRequired also counts requireVerified
and userProtected.

* refactor(http): internalAuth route option and a shared bodyRecord

EventsController and PeerController each checked x-puter-internal-auth by
hand with different compares; an internalAuth RouteOption now picks the
secret from config and one constant-time gate checks it. ShareController and
EventsController share bodyRecord instead of private copies.

* fix(fs): cap legacy /delete at 500 paths per request

The bulk form resolved, ACL-checked and removed an unbounded paths[] list one
by one. It now refuses more than FS_BATCH_WRITE_MAX_ITEMS (500), the cap the
batch write routes already use, before touching any entry.

* refactor(controllers): user and subdomain writes and reads through their stores

AuthController's 2FA setup/enable/disable and recovery-code writes go through
UserStore.update instead of raw UPDATEs plus a manual invalidate, and
/readdir-subdomains pages SubdomainStore.listByUserId instead of an unbounded
SELECT; the response shape is unchanged.

* refactor(auth): note SignupService's call-time peers

* fix(auth): type userDetails' taskbar deps independently of TaskbarDeps.clients
2026-10-11 22:00:53 -07:00
Daniel Salazar 576e893a5b refactor(services-core): read-only socket reauth, one token-grant read per check, shared grant/revoke and hosted-asset paths (#4187)
* perf(socket): re-check live sockets without touching their sessions

The 5-minute sweep reused the request-path authenticate(), which slid
every connected session's expiry and wrote user activity each tick, and
could fill SessionStore's touch-throttle map. authenticate() now takes
touch: false for re-checks; the sweep uses it, checks credentials
through runWithConcurrencyLimitSettled, and a call while a sweep runs
joins it instead of starting another.

* perf(acl): read a scoped token's grants once per check

ACLService.check did a Redis GET + JSON.parse of the token's grant list
for every ancestor x mode, and the permission scan did the same per
option. Both now read the list once (PermissionStore.listAccessTokenPerms,
which replaces the per-permission hasAccessTokenPerm) and test it in
memory.

* refactor(metering): incrementUsage goes through batchIncrementUsages

incrementUsage was a copy of the batch path that let an infinite amount
through and skipped NaN, where the batch path counts either as one unit.
It now calls batchIncrementUsages with a one-entry batch, so both apply
the batch clamping (documented on batchIncrementUsages). The overuse
alarm loses the single-path fields nothing sets any more.

* perf(metering): memoize an actor's spent addons on the increment path

Every increment and credit refresh read the actor's addons record from
the KV store. A record with no purchased credit left can only change
through a grant, and every grant announces credits-changed, so it is now
kept per actor beside the credit memo and dropped with it. Records with
credit left are still read fresh, since increments on other nodes draw
them down without announcing it. A read that straddles a drop isn't
kept.

* refactor(services): route raw table writes through their stores

- access_token_permissions: AuthService inserted grants one row at a
  time through a cast to the store's db client, read the authorizer, and
  deleted the manifest with raw SQL, then asked PermissionStore to drop
  its cache. PermissionStore now owns insertAccessTokenPerms (chunked
  multi-row insert), deleteAccessTokenPerms and
  getAccessTokenAuthorizerId, and invalidates inside the writes.
- TeamService's 2FA reset clears the OTP columns through UserStore.update.
- AppIconService's data-URL icon swap is AppStore.replaceDataUrlIcon.

* fix(user): correct cascadeDelete's FK note; seat cleanup is best effort

The header said most user_id foreign keys are ON DELETE SET NULL; the
migrations cascade nearly all of them and keep SET NULL for audit and
analytics history. forgetSeat ran unguarded after the row was already
gone, so a cache failure there failed the whole delete; it is now
logged and stepped over like the other teardown steps.

* refactor(acl): one grant/revoke path per holder kind

ACLService's user and group stat/set pairs were copies differing only in
which PermissionService methods they called. They now share #stat and
#set over a holder adapter ({ lockKey, query, grant, revoke }); lock keys
and errors are unchanged.

PermissionService's eleven grant/revoke methods repeated the same guard
blocks. Each keeps its own order of checks, now through shared helpers
(#requireUserId, #refuseAppActor, #assertCanManage, #requireUser,
#requireApp, #assertFitsColumn, #audit), so every refusal keeps its
status, legacyCode and wording. Audit writes stay fire-and-forget.

* refactor(auth): one hosted-asset token API keyed by kind

The private-asset and public hosted-actor cookies had separate create/
verify methods, four cookie-name getters and two identical options
getters, differing only in the kind claim and what the host field was
called. They are now createHostedAssetToken(kind, claims),
verifyHostedAssetToken(kind, token, expected),
getHostedAssetCookieOptions(), and the exported HOSTED_ASSET_COOKIES
names. Token claims, cookie names and every error message are unchanged;
puterSite and privateAppGate call the new methods.

* refactor(auth): one revoked/expired check for every actor builder

The session, app-under-user and access-token builders (and the access
token's parent-session check) each spelled out the same revoked/expired
tests. They now share #endedSession, which returns the same reauth
results; only a session token passes its auth_id. Adds coverage for an
access token whose parent session expires, is revoked, or is gone.
2026-10-11 21:55:20 -07:00
Daniel Salazar ecef154630 refactor(services): page fs subtree walks, gate share fan-out, one fs ACL helper (#4190)
* refactor(acl): read ancestor chains from FSEntryStore and share one fs ACL helper

getAncestorChain (split a path, batch-read the entries) moves to
FSEntryStore, with getAncestors for callers that want the rows. Services
read it from the store instead of calling FSService sideways;
FSService.getAncestorChain stays for drivers and extensions.

ACLService gains fsDescriptor (memoized chain), assertFsAccess,
assertFsCreate and refusal, plus the pure aclRefusal mapping. The eight
hand-rolled descriptor and safe-error copies in FSService, ShareService,
events/authorization, FSController, legacyFsHelpers, LegacyFSController,
AuthController and WebDAVController use them. Each route keeps its
status, legacyCode and message: fs routes answer access_denied on a 403,
the legacy routes still hide refusals from apps as 404, share and events
keep forbidden and their own 404 text.

* perf(fs): page recursive deletes and batch their event dispatch

A recursive delete read every descendant into memory, and an empty-check
for a non-recursive one did the same. It now walks the subtree a page at a
time in descending path order (children before the directory holding
them, so the parent_id cascade never takes a row before it is handled),
deleting objects and rows and publishing each page as it goes. A
non-recursive delete probes with one LIMIT 1 query.

Removals publish once per entry, after the row is gone (the same change
as the open fix for the double emit), with one surviving-ancestor chain
read at most once for the whole subtree. Events-API dispatch for a batch
runs as one background task at bounded concurrency instead of a
fire-and-forget promise per entry.

FSEntryStore gains hasDescendants and listDescendantsInOrder (keyset
paged by path, then id).

* perf(fs): copy directories a level at a time

A directory copy walked every descendant in memory and copied them one
by one: an insert per directory and, per file, an S3 CopyObject, a
batchCreateEntries of one and its read-back. It now pages the source in
ascending path order (parents before children) and takes each page a
depth at a time: rows for directories, links and empty files at bounded
concurrency, then the depth's S3 copies at bounded concurrency and one
batchCreateEntries for all of them. A single-file copy goes through the
same path.

A failed S3 copy still fails the call (a ghost file still answers 404),
but only after the objects that did copy have their rows, so nothing is
left in the bucket unreferenced. Copied entries publish per depth in one
dispatch.

* refactor(share): build masked share paths in one place

sharePathMask exports maskedRoot(owner, uuid, name) and
maskUnder(root, realPath). ShareService's maskedPathVia goes, its
maskedSelfPath and shareDeepLink's maskedSharePath build on them, and the
masker, maskUnderAnchor and learnShareRoots use maskedRoot. Output is
byte-for-byte the same; tests pin it.

* fix(share): send old_path on a recipient's item.moved

The desktop's item.moved handler reads old_path; ShareService sent the
former masked path only as from_path. Both are sent now.

* perf(share): skip the audience lookup for owners with nothing shared

Every fs write, create, rename, move and delete looked for share
recipients: an ancestor walk plus two share queries, for an owner who in
nearly every case has shared nothing. #sharesReaching now reads the
chain through FSEntryStore.getAncestors (no third copy of the walk) and
asks a per-owner "has anything shared" flag before the share queries.

The flag is the per-user cache events already keeps for subscriptions
(SubscriptionCache): one ShareStore.ownerHasShares read per owner per
TTL, coalesced across a burst. Creating a share (user, team, claimed
invite) or moving a subtree to a new owner bumps it locally and over
outer.pubsub.share.ownerShared; the TTL covers a process that misses
the bump. A failed lookup reads as shared, so the full lookup decides.

* perf(share): batch the per-row reads behind share listings

Share listings looked things up one row at a time:

- teams by id: TeamStore.getByIdsIncludingDeleted (the single-id read
  calls it) for listSharedByMe, listSharesOf and #revokeGroupSharesBy
- team shares per node and ancestor: ShareStore.listGroupOnFsentries,
  one query for the chain, in the same node-then-ancestor order
- #liveGroupGrants: PermissionStore.queryIssuerGroupPermsByPrefixes
  reads every issuer/group pair under every prefix in one query
- #revokeGroupSharesBy: the five canManagePermission checks per row run
  together instead of one after another
- ShareNotificationService #unblocked: users via getByIds and blocks via
  UserBlockStore.blockersAmong, two queries instead of two per member
- digest flush: entries are taken at bounded concurrency rather than one
  at a time

* perf(events): ask background consent once per holder and app on a revoke settle

Withdrawing events:background re-checked consent row by row, resolving the holder's actor and asking the permission service for every one of their background rows. Rows of one holder and app stand on one consent, so it is asked once and shared.

* refactor(events): drop the registry's unused push fields

notify, defaultDelivery, PushMessage, NotifyProjection and pushProjection were read only by registry.test. Every subject was unpushable and broadcast by default; the subscription's delivery class stays in DeliveryClass, which the stores use.

* refactor(share): retire trashed shares through onEntryOwnerChanged

onEntryTrashed was a copy of onEntryOwnerChanged: the same subtree share sweep, grant retire and row delete.

* fix(fs): page subtree walks by depth, not path

MySQL sorts a PAD SPACE string on its first max_sort_length (1024) bytes,
so under utf8mb4_unicode_ci paths past about 512 characters tie and fall
back to id. A file moved under a newer directory then sorted after it: the
delete walk reached the directory first and the parent_id cascade took the
file without its object delete or event, and the keyset skipped or
repeated rows. Pages now order by segment count, then id, which no
collation can reorder.

* fix(share): cache an owner as shared when the bump arrives

A bump only dropped the cached answer, so the next lookup could read a
replica that hadn't applied the new share yet and cache "nothing shared"
for another TTL. The bump now records "shared" outright.

* fix(fs): probe a directory for children by parent_id, from the primary

* fix(fs): keep the copied-size check in the batched directory copy

A copied file records the size CopyObject actually duplicated; growth past
the source row is charged against the allowance, and a copy that no longer
fits is deleted and refused, as a single-file copy already does.
2026-10-11 21:54:13 -07:00
Daniel Salazar c815dc82a1 refactor(core-util): shared bounded map, cheaper route lifecycle, DDB/DB/rate-limit/gate dedup (#4185)
* refactor(util): add BoundedTtlMap for core caches and log throttles

One size-capped, LRU-evicting map with an optional TTL and a once-per-window
shouldEmit helper. Replaces the hand-rolled maps in the DynamoDB throttle and
repair loggers (the repair one was unbounded), the Slack repeat throttle, the
auth probe's reauth log throttle, and the card-verification status memo.

* perf(http): skip route lifecycle work when nothing listens

The per-route lifecycle middleware built a payload, opened an
event.emitAndWait span and hooked finish/close on every request, listeners
or not. Each phase now checks hasListeners first. EventClient's three copies
of the key-prefix walk become one memoized #matchKeys.

* perf(ddb): write batch chunks concurrently and retry the TTL sweep

batchPut/batchDel sent their 25-item chunks one after another; they now run
up to four at a time through runWithConcurrencyLimit, and a failed chunk
stops the ones not yet started. batchGet and the batch write share one
#sendWithUnprocessedRetry instead of two copies of the backoff loop, and the
TTL sweep goes through #batchWrite, so the unprocessed deletes it used to
drop are retried. The AWS and local clients are built by one #bindClient.

* refactor(db): share migrations and tryHardRead in the base client

MySQL and Postgres each carried a copy of the migration-path runner and of
the replica-racing tryHardRead; both now live on AbstractDatabaseClient
(applyMigrationPaths, and tryHardRead gated on hasReadReplica), and SQLite
drops its own tryHardRead override. Removes MySQL reinitPrimary,
reinitReplica, isRetriableError and readWithRetry and the base requireRead,
which had no callers outside tests. SQLite prepared statements and the
Postgres SQL rewrite are memoized per query text in a bounded map.

* refactor(rate-limit): route every entry point through one charge()

The nine rate/peek/acquire entry points each repeated key and limit
resolution, the backend call and the fail-open catch. They now share
charge(), routeKey() and guardSlot(); log lines are unchanged. Driver
concurrency slots release at most once like the other slot handles, and
checkRateLimit takes { failOpen: false } for limits that must hold when the
backend is down.

* refactor(util): add secretsEqual for constant-time secret comparison

One HMAC-then-timingSafeEqual helper, safe for inputs of any length, so
callers stop hand-rolling the length guard. File signature verification
uses it.

* refactor(fs): infer signed-file MIME types with mime-types

mimeFromName looks extensions up in mime-types instead of a 35-entry inline
table, keeping the two types the table reported differently (ico, wav). The
purpose-labelled signing key is derived once per secret instead of on every
signature, where signFile derived it twice.

* refactor(http): build route gates from one gate() helper

Each gate repeated a try/catch -> next(err) wrapper or a .then(next, next)
chain around an assertion. gate(check) does that once: a throw or rejection
goes to next(err), and a synchronous check still calls next synchronously.
The credits, subscription and reputation gate files held nothing but such a
wrapper; their factories move into gates.ts and the files go.

* perf(hosting): build the puter-site hosting config once

createPuterSiteMiddleware rebuilt the normalized hosting-domain config on
every hosted-site request; it is derived from static config, like the domain
lists the middleware already builds at construction.

* fix(db): tryHardRead's replica leg doesn't fail over to the primary

MySQL's read() retries on the primary after a load-shed or transient
replica error, so routing tryHardRead through it read the primary twice
on that path. The replica leg now reads the replica alone.

* fix(ddb): settle in-flight batch chunks before rethrowing

A failed chunk stopped unstarted chunks but rejected while others were
still retrying, so writes could land after the caller had handled the
error and invalidated its cache.
2026-10-11 21:53:33 -07:00
Daniel Salazar b5eefaa9cc fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135) (#4198)
* fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135)

MySQL ends the idx_fsentries_path range for `path LIKE 'p/%'` at 'p/' +
U+FFFF, and utf8mb4_unicode_ci sorts supplementary characters above it, so
a child whose name starts with one (most emoji) was skipped with its whole
subtree by listing, delete, copy, rename/move rewrites, counts and sizes.
On MySQL, FSEntryStore now looks up those children by parent_id and ORs a
LIKE pattern for each onto the subtree match, recursing into directories.
Every extra pattern starts with the prefix, so only the scanned ranges
widen; rewrites past 500 patterns run in several statements.

The prefix rewrite also passed a UTF-16 length to SUBSTR, which counts
characters, so a rename under a path holding an emoji dropped characters
from every descendant path on all engines. It now counts code points.

* test(fs): seed the supplementary-name subtree with createNonFileEntry
2026-10-11 21:21:18 -07:00
Daniel Salazar af9ebb0ef1 fix(ai): map BytePlus video and Cloudflare image upstream errors at the driver boundary (#4184) 2026-10-11 21:20:27 -07:00
Daniel Salazar 2131b980ce fix(redis): release locks with an atomic compare-and-delete (#4183)
* fix(redis): release locks with an atomic compare-and-delete

ACLService, TeamService and the share digest flush each hand-rolled a SET NX
lock whose release was a GET-then-DEL (or a blind DEL), so a holder whose TTL
lapsed could delete the next holder's lock. They now share withRedisLock,
which claims with a random token and releases via a Lua compare-and-delete.

* chore(team): place the redis lock import with the other runtime imports
2026-10-11 21:19:37 -07:00
Daniel Salazar 96909b0800 fix(fs): send the same outer.gui.item.* payload from every fs controller (#4179)
* fix(fs): send the same outer.gui.item.* payload from every fs controller

The v2 /fs/move and /fs/delete routes spread the raw camelCase FSEntry into
item.moved / item.removed and named the old path `from_path`, while the
desktop reads the legacy snake_case shape and `old_path`. One helper,
emitGuiItemEvent, now builds the payload for LegacyFSController,
FSController and WebDAVController.

v2 and WebDAV copy/move with overwrite now also emit item.removed for the
entry they replace, as the legacy routes do, and v2 /fs/move forwards
new_metadata to the service like the legacy /move.

* fix(fs): announce a replaced entry before the one replacing it

The desktop hides rows by path on item.removed, so a removal sent after the
overwriting copy's added (or move's moved) hid the new row too. The shared
emitter now takes the replaced entry and sends its removal first, awaited,
on the v2, legacy and WebDAV copy/move paths.
2026-10-11 21:18:47 -07:00
Daniel Salazar 906d6b7886 fix(email): one canonicalization and blocklist for every address path (#4176)
Share invites, share notifications and app feedback checked addresses through
EmailClient.clean/validate, a second copy of the rules in util/email.ts that
stripped dots on iCloud, stripped `+tags` on every domain, and matched the
blocklist case-sensitively. They now use util/email.ts: isBlockedEmail for the
blocklist, abuseKey for the per-address notification budget, and cleanEmail
for the invite digest key. EmailClient keeps transport and templates only.

isBlockedEmail also lowercases the configured suffixes, so a mixed-case
blocklist entry still matches.
2026-10-11 18:24:28 -07:00
Daniel Salazar 6a938bd8e8 fix(db): detect duplicate keys with isUniqueViolation everywhere (#4175)
FSService, OIDCStore, UserStore and TeamService each carried their own
duplicate-key check. FSService only matched mysql and the bare
SQLITE_CONSTRAINT code, so the mkdir/move race recovery never ran for a
SQLite or Postgres unique violation. OIDCStore matched any message
containing "UNIQUE" or the bare SQLITE_CONSTRAINT code, UserStore any
SQLITE_CONSTRAINT_* code, and TeamService any message mentioning unique or
duplicate. All four now use util/dbError's isUniqueViolation; UserStore
still checks the index name on top.
2026-10-11 18:24:13 -07:00
Daniel Salazar 847a4fc031 fix(fs): route thumbnail writes through FSEntryStore (#4174)
* fix(fs): route thumbnail writes through FSEntryStore

Add FSEntryStore.updateEntryThumbnailByUuid, which sets a thumbnail with
no owner check (callers authorize) and refreshes the entry cache, with an
optional expected-value guard.

The thumbnails extension's inline data: URL migration, its copy
duplication, and the legacy /update-fsentry-thumbnail endpoint now write
through it instead of raw UPDATEs, so cached entries stop serving the old
thumbnail. The migration validates the image like the upload path, awaits
the write, and only repoints a row that still holds the data URL; a read
that loses that race deletes its upload.

* fix(fs): report a thumbnail update by whether the row changed

updateEntryThumbnailByUuid returned the refreshed entry, which is null
when the cache misses and the replica hasn't seen the row yet. The
read-time migration took that null for a lost race and deleted the object
the row had just been pointed at. It now returns whether the UPDATE
changed a row; the cache refresh still runs after it.
2026-10-11 18:23:58 -07:00
Daniel Salazar 3296526e8e fix(ai-chat): derive Azure model pricing from the OpenAI and xAI catalogs (#4173)
Azure entries that front an OpenAI or xAI model now come from a `mirror`
helper that copies every non-identity field from the source catalog entry.
The hand-copied entries had drifted: none carried `long_context_pricing`, so
long prompts on Azure-served gpt-5.4 and Grok 4.3/4.20 were billed at 1x;
grok-4.3 had lost `pdf` input and the Grok 4.20 deployments still listed the
old 2M context. Models no longer in the direct catalogs stay inline.
2026-10-11 17:57:14 -07:00
Daniel Salazar c553a380ae fix(ai-chat): forward max_tokens to xAI; moderate Azure through OpenAI (#4172)
xAI sent a fixed max_tokens of 1000, ignoring both the caller's value and
the credit gate's cap. It now forwards max_tokens like the other
OpenAI-compatible providers and omits it when unset.

Azure chat's checkModeration threw "Method not implemented", so
`moderation: true` on an Azure route metered the completion, then failed it
as a route error: Azure was marked unhealthy and the fallback billed a
second completion. Azure's v1 API has no /moderations endpoint, so both
Azure providers now delegate moderation to the OpenAI chat provider
(Azure chat via its Responses sibling, as it does for web_search).

A moderation call that throws is now a 400 `moderation_unavailable` that
the driver treats like `moderation_flagged`: final, no fallback, and the
route stays healthy.
2026-10-11 17:55:38 -07:00
Daniel Salazar cf9df8719a fix(alarm): forget idle alarms and cap how many the client holds (#4171)
AlarmClient kept every alarm id it had seen for the life of the process,
and ids that embed a user, key or invoice grew it without bound. The
registry is now ordered by last occurrence: alarms idle for 24 hours are
dropped on the next create, and past 1000 the least recently raised goes
first. Clearing or evicting an alarm only removes its short-id alias when
the alias still points at it.
2026-10-11 17:54:26 -07:00
Daniel Salazar 7ad963bacf fix(cache): broadcast fs entry and team invalidations to peer regions (#4169)
* fix(cache): broadcast fs entry and team invalidations to peer regions

FSEntryStore's invalidations called publishCacheKeys without broadcast,
so a delete, move, rename or overwrite only cleared the writer's
cluster; other regions served the old row until the 60 s TTL. They now
broadcast. Batch paths (deleteEntries per chunk, the batch overwrite)
send one event for the whole batch. Read-path backfills stay local.

TeamStore's busts were a local redis.del, so a removed member kept
passing getMembership in other regions until the TTL. They now go
through publishCacheKeys with broadcast, and softDelete busts every key
in one call.

Remove FSEntryCacheInvalidationEventHandler. It re-derived the same
invalidations from outer.gui.item.* with a replica SELECT per holder,
and its fs.remove.node branch never ran (FSEntry has no get()). The
store's own broadcast covers what it did.

* fix(fs): cap how many entry invalidations a node broadcasts per window

FSEntryStore now broadcasts at most 10,000 entries' cache invalidations
per broadcast flush window (outbound_flush_ms, 2 s by default). Past
that, keys are dropped locally only and peers fall back to the 60 s
entry-cache TTL; the store logs once per window when this happens.

Each flush goes to a peer as one POST that is re-sent whole on failure,
so a large recursive delete could otherwise build a batch the peer
rejects on every retry. The budget is per store instance, not per call,
so a delete issued page by page is bounded too, and single-entry writes
count against the same budget.
2026-10-11 17:53:26 -07:00
Daniel Salazar 0066d49f74 fix(fs): emit fs.remove.node once per entry on recursive delete (#4167)
#removeDescendantsStorage emitted fs.remove.node for every file
descendant while grouping S3 keys, and remove() emitted it again for
every descendant after deleting the rows, so file listeners ran twice.
Drop the storage-pass emit; remove() now emits once per entry after its
row is deleted.
2026-10-11 17:52:27 -07:00
Daniel Salazar b67acdbf6f fix(stores): return written rows without reading them from a lagging replica (#4166)
* fix(stores): read rows back from the primary after writing them

ShareStore (create, upsertPending, upsertActive, upsertActiveGroup,
applyPending, upsertAnyone), AppStore (create, createFromOrigin, update,
claimOwnership), NotificationStore.create, SubdomainStore.update and
NotificationDriver.create returned the row they had just written by
reading it back through `db.read`, which goes to the replica. Under
replica lag that came back null or stale. Those read-backs now go to the
primary, through a `primary` option on the getters. AppStore caches the
primary copy on create, and claimOwnership no longer reads through
getById, which cached the replica's pre-claim row on a miss.

* fix(stores): skip the notify read-back; read OIDC link conflicts from the primary

NotificationStore gains `insert`, which writes and returns the uid;
`create` is now `insert` plus the primary read-back. NotificationService
.notify ignores the row, so it uses `insert` and no longer pays a primary
round trip per recipient. NotificationDriver.create still reads the row
back from the primary.

OIDCStore.link reads the conflicting row from the primary after a unique
violation, so a replica that has not seen another account's link can no
longer make the call report success.

* perf(stores): return written rows instead of reading them back

ShareStore.create, NotificationStore.create, AppStore.create and
AppStore.createFromOrigin build the row they return from the values they
inserted and the insert id, instead of reading it back from the primary.
The DB-defaulted creation timestamp is now written explicitly from unix
seconds (`timestampFromUnixSql`), so the returned value matches what a
read returns (`timestampValue`); postgres INSERTs add RETURNING id. The
AppStore rows fill every column the INSERT leaves to the schema and are
cached as before.

ShareStore.upsertPending's re-invite branch selects the full pending row
in its dedupe read and returns it with the new mode and data; it falls
back to the primary only when the UPDATE matched nothing.

The remaining primary read-backs stay and say why: the share upserts and
the new-invite insert (an upsert's surviving uid/created_at and the
entry-owner subquery are only known to the primary), applyPending and
claimOwnership (only an id or uid in hand), AppStore.update and
SubdomainStore.update (the base row is a replica copy that would be
cached), NotificationDriver.create (notify returns only the uid) and the
OIDC link conflict.
2026-10-11 17:51:38 -07:00
404oops 64db150a90 fix(ai): bound streaming buffers and normalize input errors 2026-10-11 21:14:16 +02:00
Daniel Salazar f88cadb4b9 fix: a copy records the size of the bytes it duplicated (#4197)
A copied file's row took its size from the source row the request had
resolved, which can predate an overwrite whose bytes CopyObject then
copied. The copy now reads the new object's size and records that. When
it is larger than the source row said, the allowance is checked again
against the real size; a copy that no longer fits is refused with 413
and its object deleted.

If the size check fails, the copy falls back to the source row's size,
as completing a signed upload already does.
2026-10-11 10:48:52 -07:00
Daniel Salazar aef0a32d72 fix(ai): hold credits for image and video generation (#4164)
Image and video providers checked the balance with hasEnoughCredits /
getRemainingUsage and took no hold, so concurrent requests each saw the
full balance until the first one recorded usage. They now reserve the
estimate through the cost-factored reserveAiCredits and release it in a
finally block once usage is recorded, like chat, speech and OCR.

The video duration clamp now prices seconds at the model's cost factor
and holds the clamped clip's cost for the job (poll window plus create
time). Replicate holds the output-only estimate before fetching inputs
and the input surcharge on top once it is measured.
2026-10-11 09:18:13 -07:00
404oops 800e300a8c fix(ai-tts): resolve ElevenLabs voice names before synthesis 2026-10-11 16:33:05 +02:00
Daniel Salazar 491729e060 fix: backend test flakes and a socket shutdown hang (#4170)
- SocketService's prepare-shutdown hook awaited io.close(), which also
  closes the http server and waits for every HTTP connection to end. A
  connection mid-request when close() runs stays open for the keep-alive
  timeout, so shutdown stalled before PuterServer severs connections. The
  hook no longer waits on the http close.
- Test servers disable Ollama unless a test configures it. Its boot probe
  logged after files had ended, which failed shared-worker shards with
  EnvironmentTeardownError.
- vitest ignores that teardown error for a console call still in flight.
- setupPuterTestEnv boots again on a fresh port after EADDRINUSE, and a
  failed setupTestServer start shuts the server down.
- UploadReservationStore expiry tests pin the clock instead of racing a
  50 ms TTL.
2026-10-10 19:34:32 -04:00
Daniel Salazar d4589caefe fix: session store revokes (#4168) 2026-10-10 07:55:43 -07:00
Daniel Salazar 4fca4809b9 fix(health): discount total event-loop stall time from db liveness latency (#4160)
The stall watcher kept only the longest gap, so a saturated loop with many sub-second stalls still failed database-liveness. It now sums blocked time across the probe (ignoring timer jitter) and reports total and longest.
2026-10-09 18:14:48 -07:00
Daniel Salazar cc8b9858a2 feat: let extensions claim a subdomain (PUT-2100) (#4155)
`extension.claimSubdomain(name, { authorize, checkRoute? })` lets an
extension serve `<name>.<domain>` without a hard-coded reserved name.

- A claimed host is dispatched where the unknown-subdomain 404 used to be,
  before CORS, the OPTIONS responder, body parsing, auth and every route
  (including `subdomain: '*'` ones).
- `authorize` returning anything but true, or throwing a non-HttpError,
  yields exactly the unknown-subdomain 404. A thrown HttpError goes to the
  error handler.
- Routes whose `subdomain` is the claimed name (decorated controllers and
  extension routes alike) are materialized onto the claim's own router, so
  they only run after `authorize` and never on the main app. Their options
  are limited to ones that don't rely on the main pipeline's actor; an
  optional `checkRoute` lets the claim refuse routes at boot. Case-variant
  labels and claimed names mixed into subdomain arrays fail the boot.
- The unknown-subdomain check now compares the port-stripped hostname and
  requires a dot boundary, so a host with an explicit port behaves like one
  without it. Hosts served by the local worker proxy still pass through.
2026-10-09 18:14:04 -07:00
Daniel Salazar 6060de8fbd fix(alerts): overspend alarm in credits at warning; no-choices completions stop paging (#4154)
- The per-account overspend alarm now reads in credits and raises at
  warning so it reaches the pager instead of only chat.
- A non-stream completion with no choices (an upstream answering 200 with
  an error body) threw a TypeError that classified as internal_error and
  paged critical. It now throws bad_response, which falls back like any
  other empty response and surfaces as a 400.
2026-10-09 18:13:44 -07:00
Daniel Salazar 0e0cedb02b fix(auth): no email password recovery for built-in admin accounts (#4153)
- /send-pass-recovery-email for `admin`/`system` (by username or email)
  returns the usual generic message, writes no token, sends no mail, and
  raises a critical alarm with the requester's IP, user agent and, when
  signed in, their user.
- /verify-pass-recovery-token and /set-pass-using-token reject recovery
  tokens for those accounts as no longer valid.
2026-10-09 18:13:22 -07:00
Neal Shah 43094a84a0 PUT-1901 (#4159) 2026-10-09 20:12:45 -04:00
Neal Shah 5fb025c109 Misc subdomain and worker fixes (#4157)
* PUT 1902

* LocalWorker fetch policy fixes for 127.0.0.1
2026-10-09 19:59:52 -04:00
Juan Fernando Castro 50bcdc9c6c fix: let a revocation be seen by the reads and the nodes that outlive it (#4151)
* fix: let a revocation be seen by the reads and the nodes that outlive it

Four places where a revoke lands on the primary and the next reader can
still be told the old answer.

The u2a cache key was dropped but not re-filled, so the next miss read a
replica that may still carry the row and cached it for the TTL -- and a
revoke's own settling pass is usually that miss. It re-fills from the
primary, as the u2u side already does.

A revoked session dropped its keys the same way. It now leaves the
revoked row in the cache, read from the primary, so there is nothing for
a lagging replica to re-introduce.

A link share is its own authority rather than an index over a
permission, so a replica still carrying a withdrawn row answers with it.
For a short window after one is withdrawn, those reads go to the
primary.

Socket eviction rode a local event, so it only reached the node that did
the revoking. It is announced on the fan-out channel too, which is where
the node actually holding the connection hears it.

* refactor: carry the revocation on outer.* instead of outer.pubsub.*

Eviction runs through `io.in(room).disconnectSockets()`, which goes via the
socket.io Redis adapter a cluster shares, so one node handling the event
disconnects that user on every node in it. Fanning out to siblings had each
of them repeat the same cluster-wide disconnect.

The adapter's Redis is per-cluster, so peer clusters still need the hop --
`outer.*` reaches them and skips the sibling fan-out. The local
`auth.sessions.revoked` goes with it: `outer.*` fires on the emitting node
too, and nothing else subscribed.
2026-10-09 16:36:26 -04:00
Juan Fernando Castro ff35fbd66e Share attribution and cascade: finish PUT-1856 and PUT-1857 (#4119)
Replayed onto main after #4118 squash-merged. Same tree as the reviewed
head (8d8bd871f); only the redundant merges from the old base are gone.
2026-10-09 16:23:18 -04:00
Juan Fernando Castro 1ff5d93244 fix: a signed overwrite lands on the live key only once its completion is accepted (#4136)
* fix: a signed overwrite lands on the live key only once its completion is accepted

A presigned overwrite URL was signed against the live object key, so the
browser's PUT replaced the owner's bytes directly. That PUT never passes
through Puter, which leaves the access check on `completeWrite` with
nothing left to protect: by the time it runs, what /read serves has
already changed.

The upload now goes to a key of its own. `completeWrite` is already the
authorized step, so that is where the staged object is copied onto the
live key, server-side, and the staged one dropped. The entry keeps its
uuid, so everything written against it still holds. Aborting deletes the
staged object, which the old shared key made unsafe to do.

`copyObjectOfAnySize` carries the copy past the single-request ceiling
by copying part by part, so a large file is no different.

Two behaviours follow from the key no longer being shared: completing an
overwrite that uploaded nothing is refused rather than reported as an
overwrite of unchanged bytes, and abort always drops the session's
object. Only the two presigned paths changed; `prepareBatchWrites`
uploads server-side within one authorized request and still writes in
place.

* fix: the review's findings on the staged overwrite

The batch signed path still signed against the live key, so the route
the SDK's signed upload actually takes was unchanged. It stages now,
which also makes the abort simplification safe: the guard that had
protected a shared key was removed while that path still shared one, so
aborting a batch overwrite would have deleted the owner's object.

The completion is allowed against a path but lands on the uuid captured
at start, and a move between the two puts the bytes somewhere the caller
was never authorized for. The promote refuses unless the entry is still
where the check looked.

The batch promote failing left every session pending with its lease
held; it now fails them as the rest of the function does.

The pending GUI tile and the thumbnail key were keyed on the staged
uuid, which no entry ever carries, so neither correlated with what the
completion emitted. Responses carry `entryUid` for that.

* fix: do not treat a pre-staging session's key as this upload's own

A session signed by the previous release points straight at the live key,
since `objectKey` was `existingEntry?.uuid ?? uuidv4()`. Completing one is
already handled -- the promote returns early when the two keys match -- but
abort deleted `objectKey` unconditionally, which during a rolling deploy
would take the owner's file rather than the upload's litter.

Abort now deletes only a key the upload owns. The multipart branch was
always safe: discarding parts never touches an object already in place.
2026-10-09 16:22:36 -04:00
Juan Fernando Castro 86cc3490f7 fix: settle which plan an org seat resolves to, and confirm a seat's address (#4139)
* fix: settle which default plan an org seat resolves to, and what counts as paid

Default subscription resolvers ran in registration order, which is boot
order: whichever layer registered first answered first. A seat is
provisioned by a team and its address is never confirmed, so the seat
resolver now states a priority and wins over anything a deployment
infers from an address.

The seat cap asked whether the owner's plan was outside the free set,
which a default plan can be without anyone having bought it. Metering
now reports whether an explicit subscription answered, and the cap rests
on that.

* fix: the review's findings on plan resolution

`fromDefault` was read off which resolver answered, not off which policy
applied. An explicit id naming a policy nobody registered falls through
to the default one, and that read as bought. It follows the applied
policy now.

A deployment with unlimited metering has no explicit resolver, so every
owner on it became unpaid and its teams dropped from the paid seat cap
to the free one. Unlimited answers for what a paid plan would, so it
counts as one.

Also: the new constants had been placed between `FREE_SUBSCRIPTION_IDS`
and its docblock, the register docstring still described first-wins, and
prettier had reflowed an untouched licence header.

* feat: a seat confirms the address its team typed in

An address on a seat is the team's claim about someone else's inbox, and
nothing asked the inbox. It is now demanded and sent a code, so the
address counts for nothing until the seat hands it back. Resolvers that
read a domain are the reason this matters.

No address is still fine and demands nothing: the team is the trust
anchor for the account, not for a mailbox it does not hold.

`/confirm-email` allows an unconfirmed actor, so a seat can sign in with
its temporary credential and clear the demand; nothing locks it out.

* fix: clear the confirmation demand with the address it names

`clearUnconfirmedEmail` releases a seat's address so the inbox's owner can
sign up with it. The demand to confirm that address has to go at the same
write: the gate reads `requires_email_confirmation && !email_confirmed`, and
once the address is gone there is nothing left to send a code to.

* fix: keep a seat's address correctable, and only ask what can be answered

Review of the confirmation demand turned up two ways a seat could be left
holding one it had no way to meet:

- With no mail transport configured, the code is never sent and a resend
  cannot reach the seat either, so the demand stood with nothing able to
  lift it. The seat is now gated only when a code can actually travel.
- An address the owner mistyped gated the seat on an inbox nobody could
  read, and the gate itself closes `/user-protected/change-email`, the
  route the seat would have fixed it by. `updateSeatEmail` lets the owner
  who typed the address retarget it, and refuses once it is confirmed,
  when it belongs to whoever proved it.

Also resolves the owner's plan sequentially, so the second read takes the
cache the first wrote and the pair cannot come from two resolutions, and
restores the licence header a jsdoc reflow had rewrapped.

* fix: let the confirm-email window open before the profile has loaded

Booted from a stored token, the window renders before `window.user` is
populated, and reading `.email` off null threw inside the window itself --
so the one screen that could have asked for the code never drew, leaving a
blank page. Reproducible without a team: sign up, then load the GUI again
before confirming.

The address is now encoded rather than interpolated raw, and both forms of
the line are i18n keys; it was hardcoded English before.
2026-10-09 16:17:51 -04:00
Juan Fernando Castro 7de8360a6e Sharing cleanup: bounded outbound listings, and what see does not reach (#4118)
* fix: keep a file name to one line in a subject, and the inbound total to the account

**A name reaches the digest subject as the user typed it.** Subjects compile
with `noEscape` on purpose -- a header is not HTML, and escaping would show
entities to the reader -- and the comment there says free-form values collapse
whitespace upstream. A file name does not: `#assertUsableName` refuses only an
empty name, a slash, `.` and `..`, so a name may carry newlines. It is
collapsed to one line and bounded before it goes in.

**`includeTotal` answered an app with the account's figure.** The page is
filtered by what the caller can reach -- an app sees only what its user handed
it -- while `countByHolder` counts every inbound share. The docs already call
`total` an approximation for the filtering; they did not cover an app reading
the size of everything else. It is answered only for a caller acting as the
account now, and the page documents that.

The team-share fixture built its actor as `{ user }` with no `effectiveApp`,
which reads as an app rather than a session. A session acts as the account, so
it says so.

* fix: stop `see` reaching inside a folder, and bound mode changes

**A `see`-only folder recipient was pushed every child's name, size and type.**
The fan-out picked holders by reach -- any live grant on a reaching root -- and
`see` is a grant. But `see` says a node is there; `list` is what says what is
inside it, and the payload carries exactly what `list` would: name, path,
`is_dir`, type, size, modified. Descendants now go only to a holder whose mode
reaches inside. A change to the shared node itself still goes to everyone who
holds it, which is what `see` covers.

**Mode changes skipped the daily quota entirely.** Not spending share budget is
right -- changing a mode is not new reach, and the existing test says so -- but
unbounded it lets one account write a pair as fast as it can ask. They have
their own budget now, ten times the share limit by default and configurable,
so the exemption stays an exemption from *that* budget rather than from any.

* fix: let a half-finished downgrade be finished, and stop Trash answering

**A mode downgrade that failed between its two halves could not be repaired.**
`setUserUser` grants the new mode and then revokes the superseded ones. If the
revokes did not all land, the higher grant stood -- and the retry returned
early, because the mode it was asked for was now present. "Nothing to do" now
means this mode is the only one standing, so asking again is what finishes it.
The same two halves, and the same early return, were in the group variant.

**Trashing someone else's item answered for what is in their Trash.** Moving
into the owner's Trash is deliberately authorized without write on it -- that
is what lets a recipient delete from a shared folder -- but a name collision
there came back as `An entry already exists at /owner/Trash/<name>`. The mover
picks the name, so that is a question they can ask about a folder they cannot
list. It dedupes instead, which is also what the GUI wants: trashing should
not fail over a name.

* perf: record the entry's owner on the share row

The outbound listings answer "shares on things I own that someone else issued"
by joining `share` to `fsentries` and filtering on the owner there. Neither side
of that join bounds the work: driven from `fsentries` it walks every file the
user owns, driven from `share` it walks every share until it has a page. For a
user who owns a lot and delegates nothing, that is the whole table for an empty
result. `#outboundAppsSql` is worse than the ticket says -- it has no LIMIT at
all, so every call aggregates the join across both tables.

The owner now rides on the row, indexed with `id`, so all three read it
directly. An entry cannot change owner -- a move out of the owner's tree is
refused -- so it is written once, with the row.

Set in SQL from `fsentry_id` rather than threaded through callers: four insert
sites, each of which already has the id, and none of which can now forget.
The risk in denormalising is a write path that does: a row with no owner
recorded is a share that silently stops appearing in its owner's listing. The
new test covers exactly that -- it fails when one insert is made to write NULL.

Verified against a real boot: a pre-existing share row, on a database that
genuinely predates the column, comes out with the owner its entry has.

* fix: the review's findings on this branch

**The Trash collision still had both halves open.** `overwrite` was checked
before the trash case, so a caller setting it -- it is wire-supplied -- could
still delete whatever it collided with in the owner's Trash, and the 409
survived for anyone who did not. The trash case is decided first now.

Dedupe alone did not close it either: a name that changes only when there is a
collision answers the same question the refusal did, and the mover sees that
name even though the path is masked. Trashing someone else's item renames it
every time, so the result carries no signal. `original_name` already records
the real name, which is what a restore reads. Two tests pinned the old name;
they say what it is now.

**The owner column had no fallback.** Reads went only through it, so a row
written by anything that predates it -- a pod mid-roll -- would never appear in
its owner's listing again. The three reads now also cover `IS NULL` rows
through the old join, indexed on the null and empty once none remain.

**The grant became conditional on a replica read.** `if (!already)` skipped it
on a stale row while the revokes below stayed unconditional, which could leave
the holder with nothing where the old early return left their access alone. It
grants unconditionally again; only the early return changed.

**The re-issue budget covered one path of four.** Team, link and invite
re-issues reserved nothing, so three ways round it were left open. They share
one `#reserveRemodeQuota` now.

Two docblocks had been pushed off the functions they describe, and two comments
still described the join that is gone.

* chore: one-line the comments added here

* test: measure the prefix walk by what it keys on, not by the clock

`checks many long paths without re-reading every prefix of each` gave
the walk 1500ms of wall clock. It measured nothing useful: the shape it
guards against — keying each step on the accumulated prefix text rather
than on the parent id — costs 29ms against the current 8ms on that
input, so both ends sit far under the bound. Under coverage the same
test ran 1511ms and failed, which is the only thing the number could
ever report on.

`PathPrefixIds` now counts the characters it keys on, and the walk is
asserted on that: proportional to the input, and twice the depth costs
twice as much rather than four times. Keying on the accumulated text
fails both. The original case keeps its real assertion, that a request
this size is refused.

* fix: keep the recorded entry owner current, and the trash guard on the retry

A move into someone else's tree re-owns the whole subtree, which the
migration comment claimed could not happen. The shares on it are
re-pointed at the new owner, or they drop out of the listing the column
keys and stay in the old owner's.

The unique-violation retry re-entered the collision resolver without the
trash flag, which is the one call site the guard was not threaded
through.
2026-10-09 16:14:23 -04:00
Daniel Salazar be78281d9d ci: shard backend tests and share workers across server-booting files (#4149)
The suite was CPU-bound on one runner, and about half of that CPU went to
re-importing the backend for every test file.

- Run the suite as 4 shards and merge their blob reports for coverage.
  `test (pr)` stays as the required check and fails if any shard fails.
- Drop the per-PR base leg; pushes to main publish the coverage PRs
  compare against.
- Files that boot a server without module mocks or extensions run in a
  `shared-worker` vitest project (`isolate: false`), so the backend is
  imported once per worker. testSharedWorkerSetup.ts resets the redis
  mock, configContainer and extensionStore between files.
2026-10-09 01:50:33 -07:00
Daniel Salazar 8a6929847d Auth limits, reborn-app tokens, seat addresses, drop-area encode (#4148)
* auth: per-account 2FA code limit, single-use otp-login tokens, recovery mail limits

- /login/otp and /login/recovery-code share a per-account bucket charged on
  wrong codes; an otp-login token completes one sign-in.
- /send-pass-recovery-email adds an IP bucket, a per-account cap and captcha,
  and reuses the outstanding recovery token instead of rotating it.
- Reauth tokens carry the rejection reason; /signup re-attaches a temp
  account only from an expired session.

* auth: app-issued access tokens older than their app stop authenticating

An access token whose row or parent session predates the app now holding its
uid is refused, the same check app and worker sessions already get.

* teams: signup can take an address a seat never confirmed

Password signup, save-account and a provider-verified OIDC sign-up take an
address a team seat holds unconfirmed; the seat's copy is cleared at the
write. A seat address that was confirmed still blocks.

* dev-center: encode dropped directory child names
2026-10-09 00:12:59 -07:00
Daniel Salazar 360f6b1079 fix(events): bill reached deliveries, bound dispatch reads, seal cursors (PUT-1900) (#4146)
* fix(events): bill reached deliveries only, bound dispatch reads, seal cursors

- A broadcast socket copy is billed only when a connection for its holder
  exists on this node, anywhere in this region (connection count) or, for a
  durable row, in a region presence names. Without peers it bills as before.
- Compiled match filters are held in a bounded LRU with a TTL, and dropped
  when another process reports a subscription ended.
- GET /events/fetch reads the continuation from `cursor`, falling back to
  `after`.
- Cursors from /events/fetch, /events/subscriptions and /events/kv-handles
  are encrypted with sealCursor/openCursor; plain cursors still read.
- A durable subscribe counts again after its insert and removes its own row
  when over the cap; the generation bump for that removal is published.
- Subscribe checks access before reading the anchor's owner, and an anchor
  gone in between answers with the subject the caller sent.
- Dispatch reads at most the rows per token routing can use, scanning large
  token hashes; a removal still settles every row on the removed anchor.
- puter.events.unsubscribe() stops routing after the server ends the
  subscription or reports it gone; fetch() accepts `cursor`.

* fix(events): forward kv values only to regions with rows that ask

- A region counts its session rows asking for kv values per token and
  announces the token's value variant (`v#<token>`) over the existing
  `watch` item on the first such row, withdrawing it with the last. Add,
  remove, reap, reanchor and refresh all keep it, reading `includeValue` off
  the row as it is dropped.
- A kv write reads the value variants alongside the watch index and attaches
  the value only to forwards bound for a region that announced one.

* fix(pagination): seal id-keyed cursors on every list endpoint

Apps, subdomains, workers, team (members, directory, teams, audit, own
audit), shares (inbound, outbound), the user-to-user permission audit and
readdir/descendant listings now hand out cursors sealed with sealCursor and
read them with openCursor, so a cursor no longer carries a row id. Plain
cursors issued before still read.

* test(events): pass socket ids to noteConnect in the billing tests
2026-10-09 00:08:13 -07:00
Daniel Salazar 8ecfeec8c3 fix(puter-js): request idle timeout, callback/listener leaks, guarded message handlers (PUT-1908) (#4143)
* fix(puter-js): free one-shot callbacks and listeners, guard message handlers

- setAuthToken starts the GUI cache refresh loop once instead of per call
- IPC reply callbacks are dropped after their reply; dehydrated callbacks stay
- AppConnection removes its window listener when the target app closes
- reply branches for app data and instance count drop their callback
- UI, Debug, tool and GUI-token message listeners no longer reject; an
  unknown or failing tool answers toolResponse with an error
- parseResponse handles a blob response with no content-type
- retry waits remove their abort listener when the wait ends
- EmailConfirmationDialog escapes its message
- EventListener uses a real Map and snapshots listeners per emit
- Context get/set/runWithContext route every known key the same way

* fix(puter-js): time out requests that make no progress

- buildXhr starts an idle clock on send, reset by state changes and body
  progress (upload progress only on requests already carrying a bearer)
- read-safe requests get 60 s while progress is observable, everything
  else 15 min; timeout: 0 on fetchUrl/driverCall/sendWithRetry turns it off
- a timed-out read replays once; writes and AI calls never replay
- timeouts reject with code request_timeout on fetchUrl (TypeError),
  driverCall, streams and the legacy XHR path
- document the limits in rate-limits-and-quotas and the chat stream note
2026-10-09 00:07:20 -07:00
Daniel Salazar ae234ffcc2 fix: only sign-in and picker popups hand the opener a token (#4147)
* fix: only sign-in and picker popups hand the opener a token

Popup actions are an allowlist now: sign-in popups (no action, sign-in,
login, signup) deliver at boot, pickers deliver with the pick, and
request-permission keeps its exchange. Any other action runs no exchange
and posts no token. The consent check in postAuthActions applies to every
action that delivers at boot instead of only no-action and sign-in.

* fix: a dismissed account picker hands the opener no token
2026-10-09 00:05:44 -07:00
Daniel Salazar 2ae2e093e7 keep archive entries inside the extract folder, encode template labels (#4145) 2026-10-08 23:59:33 -07:00
Daniel Salazar dcfb1263de fix(events): single-writer presence items, live-socket counts and region-wide leave grace (PUT-1759) (#4142)
* fix(events): keep each region's presence item written only by that region

- A region that receives a forward for a pair it holds no socket for
  retires its own presence item through the pin-ordered leave path,
  gated by a region-shared claim. The sender no longer retires another
  region's item, and the forward reply no longer carries noSocket.
- Presence reads skip items for regions outside the peer list instead
  of retiring them.
- removeConnection decrements and deletes the count in one script.
- A failed join releases the join pin, and any connect that finds the
  pin free writes the join.
- A single delivery treats a socket on any node of the region as local,
  using the region-wide connection count, and picks the next remote
  candidate by the regions already tried instead of by index.
- Correct the connection-count TTL comment.

* fix(events): count presence by live socket and honor the leave window region-wide

- A region's presence for a pair is a sorted set of socket ids scored by
  last renewal under a new key, replacing the integer count. A socket
  counts while renewed within the concurrency-slot window; add, renew and
  remove each prune lapsed members and report the live count in one
  script, so a node that dies stops holding the pair.
- The last socket going marks the pair as leaving region-wide for the
  grace window, in the same script; a connect clears it. A forward that
  lands on another node inside the window does not retire the item.
- Add the events.presence.write counter, labelled by op (join, leave,
  refresh, retire).
2026-10-08 23:55:55 -07:00
velzie 50e2af34bf feat(puter-js): Add custom streams and perfect negotiation to Peer (#4058)
* add perfect renegotiation

* more advanced peer logic

* add video tracks to peer api

* several fixes

* small fixes, revert e006b1977

* proper reconnect

* fixes

* fixes 2

* fixes 3

* fixes 4

* remove uneeded tests
2026-10-08 23:34:55 -07:00
Daniel Salazar 7adc03cb75 fix: team seats caps (#4141) 2026-10-08 17:50:45 -07:00
dependabot[bot] 807368f7b8 chore(deps): bump handlebars from 4.7.9 to 4.7.10 (#4137)
Bumps [handlebars](https://github.com/handlebars-lang/handlebars.js) from 4.7.9 to 4.7.10.
- [Release notes](https://github.com/handlebars-lang/handlebars.js/releases)
- [Changelog](https://github.com/handlebars-lang/handlebars.js/blob/v4.7.10/release-notes.md)
- [Commits](https://github.com/handlebars-lang/handlebars.js/compare/v4.7.9...v4.7.10)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-version: 4.7.10
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-08 17:20:25 -07:00
Daniel Salazar 4f3a524ace fix: credit usage gui (#4140) 2026-10-08 17:20:00 -07:00
Krish KumarandDaniel Salazar 95b6f86fcc fix(events): resolve relative fs: paths like fs operations do (#4085)
* fix(events): resolve relative fs: paths like fs operations do

* docs(events): use bare relative paths in examples, test both relative forms

* docs(events): clarify that relative fs: subjects come back resolved

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-08 15:23:53 -07:00
Juan Fernando Castro bb829885ef fix: a picker popup hands its opener a token once the user has picked (#4133)
* fix: a picker popup hands its opener a token once the user has picked

A picker opens because the site asked, not because the user agreed to
anything, so the hand-off now waits for the answer the popup was opened
to get. The exchange still runs at boot — the app row and host_app_uid
are what the dialog is opened against — but the token travels with
fileOpenPicked, directoryPicked or fileSaved. Cancelling sends nothing.

deliversTokenAtBoot is the one gate every boot-time hand-off reads, so
the plain exchange, the cross-origin-isolated /login/set path, temp-user
creation and manual signup all follow the same rule. A failed exchange
still reports and closes, having nothing to defer.

* fix: the review's findings on the picker hand-off

Double-clicking a file answers the opener from openItem.js, not from the
UIWindow button, so that pick returned the item and no token. The
deliverer is a global now and every path that answers an opener calls
it: both UIWindow handlers, openItem, and the save flow.

A failed exchange posted `{success: false, token: null}`, and the SDK's
handler sets that as readily as a real token, so a picker that failed
wiped whatever the site already held. The failure now goes only where a
boot-time token would have.
2026-10-08 15:11:47 -04:00
Juan Fernando Castro c55437245b fix: settle the url file-access gate from a grant held on a folder above (#4132)
* fix: settle the url file-access gate from a grant held on a folder above

The gate asks `/auth/check-permissions` with `app_uid`, which resolved
through the permission scan. `fs:` access inherits down a tree and only
ACL walks that, so a grant on a folder never answered for a file inside
it and the gate asked again, writing another per-file row on each Allow.

The `app_uid` branch now routes `fs:<uid>:<mode>` through
`ACLService.check` as the app-under-user actor, which is the same
authority the operation itself will face. Everything else keeps the
scan, including `puter.perms.holds`, which sends no `app_uid`.

* fix: the review's findings on the consent gate

A permission that is not a string reached `startsWith` and faulted the
whole request, where it used to answer false inside the catch; both
parsers now refuse a non-string, which also covers the other callers
that read a request's list.

The path spelling of a permission took the scan while the uid spelling
took ACL, so one file could answer two ways. Both spellings resolve to
their entry and take the same route.

The checks run together rather than one after another: the caller gives
up after five seconds and prompts, and a deep path is several reads.
2026-10-08 15:05:01 -04:00
Juan Fernando Castro d1d55a6bc3 SDK auth plumbing: cache scope, dedupe keys and the token listener (#4112)
* fix: pin the token sender, scope the grant flag, and stop two stragglers

Five of the SDK auth-plumbing items.

**The global `puter.token` handler pinned origin but not source.** Its three
siblings all pin both, and each says why: the GUI origin is shared by every
frame on that domain, so origin alone identifies a domain, not a sender.
`signIn` pins its popup and binds `msg_id`, the consent dialog pins its popup,
and the reauth wait pins the embedder. This one pinned the embedder only when
`env === 'app'`. Since `authenticateWithPuter` now hands off to `signIn`, which
adopts the token itself, the embedder is the only sender this handler is left
serving, so it pins in every environment.

**`is_grant_user_app_permission` was a request-wide flag held across an await.**
It tells the `app-root-dir` rewriter that a row is being written, and while it
is set that rewriter resolves to a real `fs:` permission instead of
`PERMISSION_FOR_NOTHING_IN_PARTICULAR`. Anything evaluating an `app-root-dir:`
permission beside the grant -- the grant route runs one under `Promise.all`
with fs work -- read the same flag. It now lives in a scope derived for that
call, so what the write needs to say about itself is not also said to its
neighbours. `runInDerivedContext` is new: `runWithContext` starts an empty
store, which would drop the `actor` the rewriter reads.

**`fs.stat` and `fs.readdir` deduped on parameters alone**, so two identities
in one realm could share a result. They now key on origin and token as
`os/user.js` does, which documents the reason.

**`signIn`'s relay poll never stopped.** Closing the popup settles the promise;
the loop kept asking `/login/wait` every second for the life of the page.

**`triggerReauth`'s five-minute timeout was never cleared**, so a successful
reauth left a timer to fire into a settled promise.

The concurrency test parks the rewrite and reads the flag while it is held --
my first attempt read before the flag was even set and passed either way.

* fix: scope the fs cache per identity, and retarget two paths that never ran

The remaining SDK auth-plumbing items.

**`puter._cache` outlived the identity that filled it.** `_clearAuthToken`
dropped the token and the stored session but left the cache and `whoami` as
they were, and its keys were raw paths -- `item:/alice/Documents` and a
`~`-relative key alike, identical whoever asked. A GUI user switch without a
reload could be served the previous user's stat or readdir. Keys now carry the
origin and token they were read as, built in one place so the writers, the
readers, the invalidations and the warm-up probes cannot drift apart, and
signing out drops the cache and `whoami` with the token.

**The driver permission prompt could not fire.** Both paths waited for a 200
carrying `{success:false, error:{code:'permission_denied'}}`. The driver route
answers `success: true` or throws, so a refusal arrives as a 403 -- the shape
the retry now reads. Its two tests synthesised the old envelope, so they passed
against a response the API does not send; they use the real one now, and the
prompt goes from never called to called once.

This is a behaviour change: a driver call refused for permissions will now
prompt, which is what the path was written to do. Happy to delete it instead if
the prompt is not wanted.

The half-written copy of the same idea in `utils.js` is gone -- it prompted for
one hard-coded interface and its retry was a `// todo`.

**The verification-gate single-flight was keyed on nothing**, so a phone-gated
request in flight beside an email-gated one took the email answer, replayed,
failed again and surfaced raw without ever asking. Keyed per gate: same gate
still shares one dialog, a different gate gets its own.

* test: ask the cache the same way the code does

The API suites built `item:<path>` keys by hand, so they stopped finding
anything once the real keys carried the identity they were read as. They go
through `fsCacheKey` now, which is the point: a test that spells the key itself
passes until the moment the spelling changes, and then fails for a reason that
has nothing to do with the behaviour it is guarding.

These run against a live server, so a plain `vitest run` does not reach them --
they were green locally and red in CI.

* fix: the review's findings on this branch

Eight, and three of them were regressions this branch introduced.

**Pinning the token sender to `globalThis.parent` in every environment broke
the case it was meant to protect.** On a top-level page `parent` is the page
itself, and the token arrives from a popup, so the handler stopped accepting
anything. The GUI posts `puter.token` to `window.opener` for the file pickers
too, not only for sign-in, and this handler is their only consumer -- a site
calling `showOpenFilePicker()` would have signed the user in, dropped the token
and run the following read with none. It now pins the embedder when framed and
a window this SDK opened when not; `UI.js` already tracked its picker popups
for the same reason, so that set is shared rather than duplicated.

**The GUI still spelled a cache key by hand.** `updateSubdomainsForItems` built
`item:<path>`, which no longer matches what the SDK writes, so its `get` could
not hit and its `set` wrote an orphan nothing reads and nothing evicts -- a
published website's badge quietly stopped appearing on a cached render. It goes
through `fsCacheKey` now, and that is the last hand-spelled key in the tree.

**Retargeting the driver permission prompt to 403 was wrong and is reverted to
the deletion the ticket asked for.** The route refuses with `forbidden`, not
`permission_denied`, so the retarget still missed it; meanwhile a driver that
raises `permission_denied` for its own reasons would have opened a prompt that
had nothing to do with the refusal. And the prompt asks for
`driver:<iface>:<method>` while the route checks `service:<driver>:ii:<iface>`,
so granting it changes nothing. The path cannot work as written.

The rest were pre-existing or latent:

`cacheWhoami_` wrote its answer without checking the token it was issued under
was still current, so a reauth landing mid-flight could repopulate `whoami`
with the previous identity. `_dropIdentityCaches` also left `whoamiCache_`,
which other call sites await.

`setAuthToken` now drops the identity caches when the token actually changes:
keys carry the token and the store is persisted with no TTL, so a rotation that
did not go through `_clearAuthToken` orphaned a whole key space with nothing to
reclaim it.

The verification-gate entry was registered after the body could already have
deleted it -- a synchronous throw stranded a resolved "not verified" in the map
for the life of the page. It registers first and clears on a microtask.

`signIn`'s poll is bounded as well as settle-checked: on a cross-origin
isolated page opened from a gesture, nothing watches the popup, so closing it
never settled the promise and the loop ran on exactly as before.

`runInDerivedContext` says in its docblock that callee writes do not survive;
only the one flag needs the scope, and a future caller should not expect to
hand a value back through it.

* fix: the review's findings on the SDK auth plumbing

**The relay bound stopped asking without saying so.** The deadline left the
`signIn` promise pending and its listener attached. On an isolated page opened
from a gesture the relay is the only thing that can settle it -- nothing watches
the popup there -- so a user who took longer than the deadline was left with a
promise that never resolved, and `authenticateWithPuter` queues every later
implicit-auth call behind it. It rejects now.

**Pinning the token source left `signIn`'s own popup untracked.** Only the
picker popups registered, so on a top-level page the global handler bailed on a
sign-in it had opened itself: `puter.onAuth` never fired, and
`puterAuthState.authGranted` was never set. `openAuthPopup` is the one place
auth popups are opened, so they register there.

**A closed popup is ours too.** Rejecting one that had already closed loses a
message from a popup that posts and then closes itself, which is why `UI.js`
tolerates the same case. Kept, with the set bounded instead.

**The gate key carried the factors**, so one route naming them and another not
opened two dialogs for the same gate. It keys on the gate, as `ctx.done` does.

**`fsCacheKey` left `~` unresolved**, so a writer and an invalidation spelling
the same place differently missed each other -- the drift the one key builder
was meant to end.

The dead `opts.permission` went with the path that used to read it.

Four workerd-runner failures are left: `ai > chat stream adapter supports
cancellation`, `kv > a GUI boot key written by another client reads fresh` and
two cache-warmer ones. All four fail the same way on `origin/main`, and the node
and browser runners pass.

* chore: one-line the comments added here

* chore: reflow the comments the removed retry cause left ragged
2026-10-08 15:04:49 -04:00
Michael Lip 4ea93e6a73 fix(i18n): add GUI keys that rendered as raw key names (#4074)
The 2FA spinner showed "verifying", the files selection bar showed a
"done" button, and two login and signup errors showed their key names,
because i18n() returns the key itself when en.js has no entry for it.
The login error now uses the existing login_email_username_required key,
which already has translations. A test checks that every literal i18n()
key in the GUI exists in en.js.
2026-10-08 11:42:07 -07:00
Daniel Salazar 80736de0f1 test(events): wait for the recorded failure, not any future lease score (#4127)
The in-flight claim's lease (2x the invoke timeout) is also a future score, so
polling heldForMs > 0 could pass before the failed attempt's hold was written:
the backoff test read 600 instead of 2000, and the 404/hang/429/suspended-owner
tests ended while their attempt was still in flight, letting it land in the
next test. Wait on the subscription's failure counter instead, which only moves
after the hold or gap marker is written.
2026-10-08 11:39:06 -07:00
dependabot[bot] 085ab1774f chore(deps): bump shell-quote from 1.10.0 to 1.12.0 in /src/puter-js (#4120)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.10.0 to 1.12.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.10.0...v1.12.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.12.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-08 11:32:57 -07:00
dependabot[bot] 505fd13d97 chore(deps): bump shell-quote from 1.9.0 to 1.12.0 (#4121)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.9.0 to 1.12.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.9.0...v1.12.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.12.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-08 11:30:52 -07:00
Reynaldi Chernando 47a1be7f70 Rename "Add Existing Account" menu item to "Add Another Account" (#4124) 2026-10-08 11:25:52 -07:00
Filip Kujundžić 38c7191009 Merge pull request #4135 from HeyPuter/feat/claude-haiku-5-5 2026-10-08 19:56:01 +02:00
404oops b6f31e3b02 feat(ai): add Claude Haiku 5.5 2026-10-08 19:44:17 +02:00
Filip Kujundžić 3c852cf9cf Merge pull request #4134 from HeyPuter/feat/mistral-large-4-sonnet-pricing
feat(ai): add Mistral Large 4 and update model pricing
2026-10-08 19:37:53 +02:00
404oops 0afe8ccfdd docs(ai): keep model update documentation focused 2026-10-08 19:29:36 +02:00
404oops 266390d23d feat(ai): add Mistral Large 4 and update model pricing 2026-10-08 19:26:06 +02:00
Filip Kujundžić b6150d63f3 Merge pull request #4090 from HeyPuter/fix/ai-stream-cancellation
fix(ai): propagate cancellation and stream termination failures
2026-10-08 13:33:50 +02:00
404oops 4e8277ea4b docs(ai): clarify streaming errors and stopping streams 2026-10-08 13:21:38 +02:00
Daniel Salazar b6dfc6f903 fix: harden card verification (#4126) 2026-10-07 23:11:49 -07:00
dependabot[bot] 3d7deaa403 chore(deps): bump smtp-server from 3.19.13 to 3.19.17 (#4083)
Bumps [smtp-server](https://github.com/nodemailer/smtp-server) from 3.19.13 to 3.19.17.
- [Release notes](https://github.com/nodemailer/smtp-server/releases)
- [Changelog](https://github.com/nodemailer/smtp-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/smtp-server/compare/v3.19.13...v3.19.17)

---
updated-dependencies:
- dependency-name: smtp-server
  dependency-version: 3.19.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 22:36:57 -07:00
Daniel Salazar 199d72cbdf fix(events): refuse scoped access tokens on durable events routes (PUT-2048) (#4105)
Durable subscribe, list, fetch and unsubscribe now apply the same scoped
token check as the handler, worker and kv-handle routes:
`isAccessTokenActor(actor) && !isAccountContext(actor)`. An access token
an app issued (effectiveApp = the app) is treated the same as one the
account issued: subscribe answers 403 `events_durable_requires_account`,
list and fetch return an empty page, unsubscribe answers 404.

`rowInActorScope` returns false for these tokens, which covers durable
unsubscribe and ack. The session verbs that also use it already refuse
access tokens at the socket handshake.

Full-access tokens, app tokens and sessions are unchanged.
2026-10-07 22:35:09 -07:00
Daniel Salazar f234bd477c fix(auth): answer a username lost to a concurrent write with the taken-name 400 (#4115)
The duplicate-username check in the signup placeholder claim, change-username
and save-account runs before the user-row write, so a concurrent write can
take the name in between and the unique index rejects ours. Each path now
re-reads the holder from the primary on a unique violation and returns the
400 its pre-check returns for a taken name, instead of a 500. In all three,
the home-directory rename, rename budget charge and username-changed event
run after the user-row write, so nothing has happened yet when it fails.
2026-10-07 22:34:31 -07:00
Daniel Salazar 2638e9ce93 fix(kv): batch-read GUI boot keys only when the SDK runs as the GUI (PUT-1899) (#4117)
The boot-key batch now starts only when puter.env is 'gui'. In every other
environment a get() of one of those names reads that key on its own, like
any other key, instead of fetching all 14 and serving them for 4 s.
2026-10-07 22:34:22 -07:00
dependabot[bot] accf33a50e chore(deps-dev): bump source-map-js from 1.2.1 to 1.2.2 (#4123)
Bumps [source-map-js](https://github.com/7rulnik/source-map-js) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/7rulnik/source-map-js/releases)
- [Changelog](https://github.com/7rulnik/source-map-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/7rulnik/source-map-js/compare/v1.2.1...v1.2.2)

---
updated-dependencies:
- dependency-name: source-map-js
  dependency-version: 1.2.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 17:14:30 -07:00
dependabot[bot] ab175b0fb4 chore(deps): bump proxy-addr from 2.0.7 to 2.0.8 (#4122)
Bumps [proxy-addr](https://github.com/jshttp/proxy-addr) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/jshttp/proxy-addr/releases)
- [Changelog](https://github.com/jshttp/proxy-addr/blob/master/HISTORY.md)
- [Commits](https://github.com/jshttp/proxy-addr/compare/v2.0.7...v2.0.8)

---
updated-dependencies:
- dependency-name: proxy-addr
  dependency-version: 2.0.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 17:13:23 -07:00
dependabot[bot] 3fcbd895a7 chore(deps): bump sharp from 0.35.4 to 0.35.5 (#4081)
Bumps [sharp](https://github.com/lovell/sharp) from 0.35.4 to 0.35.5.
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.35.4...v0.35.5)

---
updated-dependencies:
- dependency-name: sharp
  dependency-version: 0.35.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 16:12:01 -07:00
dependabot[bot] 4c9e4d8e79 chore(deps): bump mysql2 from 3.24.4 to 3.24.5 (#4082)
Bumps [mysql2](https://github.com/sidorares/node-mysql2) from 3.24.4 to 3.24.5.
- [Release notes](https://github.com/sidorares/node-mysql2/releases)
- [Changelog](https://github.com/sidorares/node-mysql2/blob/master/Changelog.md)
- [Commits](https://github.com/sidorares/node-mysql2/compare/v3.24.4...v3.24.5)

---
updated-dependencies:
- dependency-name: mysql2
  dependency-version: 3.24.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-07 16:11:46 -07:00
Juan Fernando Castro cf574e4b34 Merge pull request #4110 from HeyPuter/juancastro/icloud-dot-significance
Treat dots as significant on Apple's mail domains
2026-10-07 17:11:59 -04:00
Daniel Salazar 2e275a8f6b fix: godmode installation (#4116)
* fix: godmode installation

* fix: godmode classifier

* fix: vitest
2026-10-07 13:16:10 -07:00
Nariman Jelveh b5bd917119 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-10-07 12:39:47 -07:00
Nariman Jelveh 3999dd062f Update RecommendedAppsService.ts 2026-10-07 12:39:34 -07:00
Daniel Salazar 3e234d7329 fix: 402 error messaging (#4114) 2026-10-07 12:34:45 -07:00
Daniel Salazar f674e51b4d fix: answer lost write races with 4xx, and time health-check timeouts against loop stalls (#4107)
- signup: a username taken by a concurrent signup between the duplicate
  check and the insert now returns the pre-check's 400 instead of
  surfacing the unique-key error as a 500. The holder is read from the
  primary.
- fs move: when the UPDATE trips the (parent_id, name) unique key after
  the collision probe missed the occupant, re-read it from the primary and
  resolve it the same way the probe would have (overwrite, dedupe, or the
  409 item_with_same_name_exists), then retry the move once.
- server health: a check that hits the per-check timeout reports how long
  the event loop was blocked during it, like the database-liveness latency
  failure already does.
2026-10-07 12:02:05 -07:00
Daniel Salazar d395a3aa94 fix(kv): page through flush, cap per-call batch sizes, validate decr and del (#4106)
- flush follows LastEvaluatedKey until the namespace is empty and rejects
  when a batch delete fails; cached reads for every key a page returned are
  still invalidated and the flushed event still fires.
- decr validates the amount map before negating it, so it accepts exactly
  what incr accepts.
- del checks the key like every other store method (oversized key is a 400).
- list offset emulation treats a missing continuation key as end of data
  instead of restarting from the first key.
- Array get maps results through a Map instead of a linear find per key.
- The driver gates includeTotal on credit like value listings.
- New driver caps in drivers/kv/limits.ts: 1,000 keys per array get, 1,000
  items per batchPut (both rejected past the cap), list limit lowered to
  1,000. Published in rate-limits-and-quotas.md and the set/list pages.
2026-10-07 12:00:50 -07:00
Daniel Salazar fde323842e fix(kv): SDK list rejections, boot-cache read-after-write, byte-accurate size checks (PUT-1899) (#4108)
- puter.kv.list() no longer throws synchronously for invalid options; it
  returns a rejected promise, or for stream: true an iterator whose first
  next() rejects. Error codes unchanged.
- Writes through puter.kv (set, del, incr, decr, add, remove, update,
  expire, expireAt, flush) stop the boot-key batch from serving the keys
  they touch, so the next get() reads the store. Untouched boot keys are
  still served from the batch.
- Client-side key/value size checks measure UTF-8 bytes, and values are
  measured as their JSON encoding, matching the store. Same codes and
  messages.
2026-10-07 11:48:45 -07:00
Daniel Salazar 4a54beac6b fix(fs): SDK uid stat cache, readdir page types, move lookup errors, UID-shaped paths (#4109)
- stat: only path-addressed results are cached; uid-addressed stats no
  longer share one cache key.
- readdir: overloads declare the page result for recursive + limit/offset
  and for the positional page/stream forms.
- move: only a not-found destination lookup means "new path"; other
  lookup failures reject. A uid destination skips the lookup.
- getAbsolutePathForApp always resolves paths. read/copy/move/delete use
  getAbsolutePathOrUidForApp, which keeps sending UID-shaped strings as uids.
- Docs for read, copy, move, delete and readdir.
2026-10-07 11:46:22 -07:00
Juan Castro 0555ea6479 Merge branch 'main' into juancastro/icloud-dot-significance 2026-10-07 11:01:12 -04:00
Juan Fernando Castro 27eebc7300 Merge pull request #4086 from HeyPuter/juancastro/permissions-access-control-cleanup
Permissions and access-control cleanup
2026-10-07 10:57:41 -04:00
Juan Castro 4b5f509066 fix: only rewrite rows the old rule actually wrote, and fix the invites too
Two findings from review, both of which contradict what the last commit
claimed.

**The backfill could merge keys, not only split them.** I asserted it "can only
split rows apart, so it cannot collide". Not true, and reproducible: a row
predating the `clean_email` column has a NULL there, so its index key is
`lower(email)` -- filling it in folds `+tag`, and `[email protected]` lands on a
key `[email protected]` already owns. A legacy row holding an un-lowercased
`clean_email` collides the same way once lowercased. Either throws, and the two
engines that re-run every file on every boot would then refuse to start until
someone operated on the database by hand.

Both updates now fire only where the stored value is exactly what the old rule
produced and differs from the correct one. That makes "splits only" true: the
sole change is dots coming back. NULLs and legacy values are left alone.

The same predicate makes them write-idempotent. Postgres has no no-op-update
suppression, so without it every boot rewrote a tuple per Apple row forever.

**The pending-invite residue misdelivers rather than just going missing.**
`claimPendingShares` looks the row up by the canonical address, so whoever
confirms `[email protected]` claims the invites addressed to
`[email protected]` -- someone else's file, handed over on confirmation. I had
written this off in the PR as unfindable invites that "expire on their own";
`share` has no expiry column and the lookup has no time predicate, so they are
claimable indefinitely.

Fixed here rather than later: the typed address is kept as `invitedAddress`
whenever it differed from the canonical, which is exactly the collapsed rows,
so the invites recompute from it under the same guard.

Verified by seeding a database at the old version with the two rows that used
to abort the migration, and booting: it reaches version 87, repairs the
collapsed row, and leaves the NULL and legacy rows untouched. Replaying the
statements changes zero rows.
2026-10-07 10:23:30 -04:00
Juan Castro ece8ad5f1d fix: register the Apple-domain backfill with the sqlite runner
The sqlite migration list is hand-maintained, so the file alone did nothing —
a fresh boot still reported 87 files and stopped at version 86. MySQL and
Postgres read their directories, so only this one needed the entry.

Caught by booting a local instance rather than by the suite: the client tests
pin `CURRENT_SCHEMA_VERSION`, which fails once the entry exists but says
nothing while the file is simply unregistered.

Verified against a seeded database at the old version. `[email protected]`
went from `[email protected]` to `[email protected]`, `[email protected]`
from `[email protected]` to `[email protected]` — subaddressing still folded — and a gmail row
was left alone.
2026-10-07 09:45:26 -04:00
Juan Castro cf817b0f0a fix: treat dots as significant on Apple's mail domains
`cleanEmail()` applied gmail's `dots_dont_matter` rule to icloud.com, me.com
and mac.com. Apple allocates the exact string: `[email protected]` and
`[email protected]` are two mailboxes with two owners. Collapsing them made one
canonical address stand for both.

What that canonical address feeds:

- `findEmailOwner` falls back to it, so an OIDC sign-in whose verified claim is
  the dotted address resolves to the account holding the undotted one, and
  `linkProviderToUser` has nothing left to object to -- the provider verified a
  real mailbox and the account's address is confirmed.
- The unique index on COALESCE(clean_email, lower(email)) reads it, so the
  rightful owner of a dotted Apple address is refused signup because the
  undotted account already "owns" it.
- Pending share invites are stored and claimed on it.

Subaddressing is untouched: Apple does fold `+tag`, so that rule stays. Gmail
keeps both rules -- there the fold is correct.

A migration recomputes `clean_email` for those domains on all three engines,
since rows written before this still hold the collapsed value and the index
reads the stored one. Verified the SQL against the function rather than by
inspection: same five inputs through `cleanEmail()` and through sqlite give
byte-identical output. It can only split rows apart, never merge them, so it
cannot collide with the unique index.

Not changed: `findUserByEmail` still matches on the canonical address. The
cross-match it documents is correct where the provider really does fold --
`[email protected]` is the same inbox as `[email protected]`. Requiring an
exact match would break that for no benefit here; the fix is for the data to
say what the provider does.
2026-10-07 09:17:03 -04:00
Juan Castro 6efb96d560 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-07 09:00:40 -04:00
Juan Castro 5596e70f25 Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-07 09:00:18 -04:00
Daniel Salazar e477bca3eb test(events): wait for the failure hold, not any future lease score
The in-flight claim's lease (2x the invoke timeout) is also a future score,
so polling for "held" could read it before the failed attempt's backoff
hold was written. Wait for deferAfterFailure to settle instead.
2026-10-06 22:32:02 -07:00
Daniel Salazar 11e48c68a9 fix: verification for sms (#4101) 2026-10-06 22:03:07 -07:00
Daniel Salazar 464f758a97 fix: verification for sms (#4100) 2026-10-06 22:00:14 -07:00
Daniel Salazar fadee6fac5 fix: verification for sms (#4099) 2026-10-06 21:59:58 -07:00
Daniel Salazar 6bce256589 refactor(ai): trim redundant cancellation checks
- Chat driver runs each attempt through one helper and checks the abort
  signal only on entry and in the attempt catch blocks, where it keeps a
  cancelled request from marking the route unhealthy.
- A stream ended after cancellation still emits ai.prompt.cost-calculated;
  only ai.prompt.complete is skipped.
- pipeNdjsonStream owns its line buffer and onEnd is optional; drop the
  errored guards that can no longer fire.
- Drop the Claude post-loop abort check (finalMessage already throws).
- Skip provider error logs when the caller cancelled.
- Simplify the usage_limited cap condition.
2026-10-06 19:49:25 -07:00
Daniel Salazar d2cdde5147 fix: verification for sms (#4098) 2026-10-06 19:36:18 -07:00
Daniel Salazar 50ee0818ea feat: trial cleanup (#4097) 2026-10-06 18:20:18 -07:00
Juan Castro ad1be70d06 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-06 19:44:37 -04:00
Juan Castro 6277f106c4 Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-06 19:44:19 -04:00
Juan Castro 738a8058ef fix: close the path oracle where it actually was
Local testing against a running instance found the service-side fix was not
enough. Both answers were 403, but they did not read the same:

  exists  -> permission_denied: fs:/other/rd:read
  missing -> Cannot create outside your home directory   (forbidden)

The second comes from `#planFsPathCreate` in the controller, which probes the
path itself and only calls the create guard once it knows the path is absent.
Its comment said so outright -- checked "only once the path is confirmed
missing, so an existing shared path outside the caller's home is not a new
rejection" -- which is the oracle stated as an intention.

A foreign path now refuses word for word whichever it is. Responses are
byte-identical apart from the caller's own input.

`rewritePermissionForActor` collapses the create guard's `forbidden` the same
way it already collapsed the 404, for the paths that reach it through the
service rather than the controller.

The existing test pinned `forbidden` here; its subject is that nothing is
created in someone else's tree, which still holds. Alongside it is the
invariant itself: refuse a present and an absent foreign path, mask the path,
compare the whole error. Without the controller check that fails.
2026-10-06 19:41:55 -04:00
Juan Fernando Castro 0e2f9663b2 Merge pull request #4091 from HeyPuter/juancastro/teams-cleanup
Teams cleanup: pagination, resumable delete, and credentials that stop leaving by mail
2026-10-06 19:27:09 -04:00
Juan Castro b8fcbffe3f fix: the five remaining review findings
**The path oracle was not closed.** The rewrite only remapped the *error*, so
success-vs-403 still answered whether a foreign path exists. Narrower than
reported: every other grant path already gates on `canManagePermission`, and
only `grantUserAppPermission` has no authority check -- deliberately, since
granting your own app something you don't hold is inert. It now refuses a
foreign path the actor cannot see, with the same error a missing one gives.

My first test for this passed without the fix: `makeUserActor` creates no
fsentries, so both paths 404'd and nothing distinguished them. It inserts a
real entry now, and the control fails with "expected null to match 403" --
null being the grant that succeeded, which is the oracle.

**`#validateEmail` still failed open** on the main signup / change-email /
confirm path while its OIDC twin was hardened. Same shape as OIDC now: a gate
that did not run is not a pass.

**The 16-permission cap broke `puter.perms.check()`** for longer lists. The
SDK's own docblock promised "one round trip however many are asked about",
which the cap made untrue, so it batches instead. The reported second half --
the UI grant poller burning its timeout -- is not reachable:
`MAX_REQUESTED_PERMISSIONS` already caps that path at the same 16.

**`listener_failed` was set by wildcard listeners.** An unrelated observer on
`email.*` throwing would refuse every signup. Only an exact-key listener
implements the event, so only that one speaks for the gate.

**`whoami()` labelled everything `Unauthorized`.** A 502 with an HTML body is
not a 401; the server's `code` now rides along and the fallback names the
status.
2026-10-06 19:23:40 -04:00
Juan Castro 755b58fd49 fix: don't hand the owner's subtree size to a see-only caller
Switching `getSubtreeSize` to `entry.userId` fixed a recipient reading 0 for a
shared folder, but `/stat` is gated on `see` and `return_size` is caller-
supplied. `see` is the weakest level -- the folder is visible, what is under it
is not -- so the sum now covered entries the caller had no way to enumerate.

It is answered only for a caller who passes a `list` check, which is where the
contents become enumerable anyway. Everyone below that gets the directory row's
own size, which is what they got before this branch.

`#assertAccess` grew a non-throwing sibling for this; the descriptor both need
moves into `#aclDescriptor`.

Test grants `see`, asserts 0, grants `list`, asserts the real size. Without the
gate the `see` case reads the file's bytes.
2026-10-06 19:08:46 -04:00
Juan Castro bd5aa901ab perf: batch the per-page reads in a team delete
Measured first, against a real server, counting DB round trips: a 40-seat
delete was 284 queries, 7.1 per seat, scaling linearly. Four of those seven
answered for the whole page and had no reason to be asked per account.

  120x SELECT * FROM `user` WHERE `id` = ?
   41x INSERT INTO `audit_team_membership`
   40x SELECT COALESCE(SUM(size), 0) FROM fsentries
   40x UPDATE `user` SET `suspended` = ?
   40x SELECT `uuid` FROM `sessions` WHERE `user_id` = ?

The page now reads who is already suspended, what each holds, and the rows the
notices need in one query each, and writes the audit as one statement. What is
genuinely per account -- the suspension, its billing event, the mail -- stays
per account under the same concurrency bound.

Same 40-seat delete is now 170 queries, 4.3 per seat.

Three new store methods rather than raw SQL at the call site:

`getSuspendedByIds` reads the primary, deliberately not `getByIds`, which is
cache-first. A stale `suspended` is exactly what re-charges a seat on a retry,
which is the thing this path was just fixed to stop doing.

`getHeldBytesForUsers` is `getHeldBytes` with a GROUP BY.

`appendAuditMany` is `appendAudit` as one multi-row insert, still written
before any suspension, so a failed append cannot leave an unlogged one.

What is left is concentrated in `#suspend`: the UPDATE, the forced re-read the
user store does afterwards to refresh its cache, and the session drop. Those
are shared store paths and batching them means owning their cache invalidation,
so they are not in scope here.
2026-10-06 18:55:53 -04:00
Juan Castro b84fc6c158 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-06 18:26:31 -04:00
Juan Castro edb0c2a712 Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-06 18:13:27 -04:00
Juan Castro 55e29666d0 Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup 2026-10-06 18:13:02 -04:00
Juan Castro 66774ff90c Merge branch 'main' into juancastro/permissions-access-control-cleanup 2026-10-06 18:12:39 -04:00
Juan Castro 0fbb6def5a Merge branch 'main' into juancastro/flat-permission-view-scope 2026-10-06 18:12:25 -04:00
Juan Fernando Castro d9ee207ed0 Merge pull request #4092 from HeyPuter/juancastro/storable-email-gate
Gate an email address once, before it is stored
2026-10-06 18:10:19 -04:00
Juan Castro 0495ea0f9d fix: close the holes the review found in the storage gate
Four, all real.

The gate threw on a non-string. `validator.isEmail` asserts its argument, and
two call sites hand it an unvalidated body value -- `send-pass-recovery-email`
is public, so `{"email": 1}` was a 500 rather than a 400. The check belongs in
the gate, not in each caller, which is the whole premise of having one.

OIDC signup never shape-checked the claim at all. `claims.email` went from a
truthiness check straight into `user.create`, so a provider we do not run could
store exactly the string this branch exists to keep out. `email_verified`
attests the mailbox, not the characters.

The change-email confirm handler promoted the staged column without re-reading
it through anything. A row staged before this deploys still lands on `email`;
one check at promotion closes that window.

The invite path kept its own regex, and it was never the same rule. It admitted
`[email protected]`, `.[email protected]`, `[email protected]`, `a;[email protected]` and `a,[email protected]` -- the last of
which nodemailer reads as two recipients -- each one a permanent pending row
that spent daily quota, which is the failure the comment above the constant
said it existed to prevent. It uses the gate now, and the second regex is gone.

A regression test each, all four confirmed to fail without their fix.
2026-10-06 17:20:35 -04:00
Juan Castro 73698be964 fix: one gate for an address we are about to store
`validator.isEmail` accepts a quoted local part, so an address whose local part
is an arbitrary string passes it. Five call sites relied on that check alone
before writing the address to `user.email`, and the invite path's own shape
regex was equally permissive.

`isStorableEmail` is now the single gate: the library check, plus a refusal of
`<`, `>` and `"`, which are legal in an address only inside the quoted form.
Nothing deliverable uses that form, so the rule costs no real address --
apostrophes and ampersands in a local part still pass, and the suite covers
them.

One function rather than a check each caller has to remember, because the next
place that stores an address will not remember.
2026-10-06 16:47:15 -04:00
Juan Castro 25f132b28d fix: address code review on the teams cleanup
Six findings, all real.

`total` described rows the page withheld. The count ran over every live team the
user belongs to, while the app-actor filter ran afterwards on the items only, so
`GET /teams?includeTotal=true` from an app returned `items: []` with a non-zero
`total` -- the exact thing the filter exists to prevent, and newly reachable
because `includeTotal` was inert before this branch. The directory-open
restriction moves into the query, so items, cursor and total all describe the
same set.

The GUI told owners the opposite of what the server now does: "we email the
username and temporary password to it." An owner who believed it would dismiss
the one copy of the password and strand the seat. Rewritten.

`runWithConcurrencyLimit` rejects on the first error but cannot cancel the
workers beside it, so a failed seat let up to seven others finish suspending and
emailing "your team has been closed" for a team that was still live. The page
now settles before the rejection is rethrown, which leaves a state the retry can
resume from instead of an arbitrary one. My earlier claim that failure
semantics were unchanged was wrong.

`#disableSeatForDelete` returned `true` on both paths, so the boolean read as if
it distinguished them. It returns nothing; the counter uses the settled results.

`listTeamsForUser` had picked up a second stacked docblock -- the same defect
this branch removes from `#cached` one file over.

`reissueCredential` sent the provisioning mail, which after the credential
change was a byte-identical duplicate saying nothing about the password it had
just invalidated, while the GUI promises the member hears about it. It has its
own template now.

Regression tests for the count leak and the reissue notice; both confirmed to
fail without their fix.
2026-10-06 16:22:48 -04:00
Juan Castro 36a22b05a2 docs: stop claiming the directory discloses what /members does
Both docblocks on the directory route said it carries nothing a colleague
cannot already read through `/members`. It carries `uuid`, and `/members`
withholds `uuid` from everyone but the owner, with its own comment pointing
apps at the directory to get one.

The disclosure itself is the opt-in working as designed: `directory_enabled` is
the owner's consent to expose a roster, every member reaches the route the same
way whether they call it from a session or through an app, and the SDK and the
teams recipes already document the directory as where a stable id comes from.
So the comments are what was wrong, not the route.

Removing `uuid` from the directory would be the other way to make them agree,
but it breaks `listDirectory()` for every caller and contradicts the published
recipes; that wants a decision, not a cleanup commit.
2026-10-06 16:02:13 -04:00
Juan Castro 9d8e37dc6d fix: stop an unconfirmed claim on an address from blocking invites to it
`#resolveRecipient` already refuses to treat an unconfirmed address as an
identity, and says why: "resolving it would hand the share to whoever registered
the address first." `#invite` then looked the same address up again and honoured
whatever that row said about blocking -- `blocksAllShares`, or a block on this
particular sender.

So registering an address you do not own, and never confirming it, was enough to
refuse every invite anyone sends to that mailbox. Nothing about that row is
evidence the mailbox is yours, which is the premise resolution already works
from. A team seat reaches this cheaply: those accounts are provisioned with
`requires_email_confirmation: false` and an address the owner typed.

The block is now honoured only for a row that confirmed the address. The alias
reasoning behind the lookup is unchanged -- a confirmed holder still cannot be
reached through a variant they did not type.

Not addressed here: the same unconfirmed seat still holds the address against
the real owner's signup, because `idx_user_owned_email` keys on having a
password rather than on confirmation. Narrowing that is a schema change and
wants its own ticket.
2026-10-06 16:00:32 -04:00
Juan Castro 6826071156 fix: stop mailing a plaintext team credential to an unconfirmed address
Provisioning a seat lets the owner supply any `email` that passes a format
check. Puter then mailed a working temporary password to it. Nothing confirms
the owner controls that address -- provisioned accounts are created with
`requires_email_confirmation: false`, because the team is the trust anchor for
the account, not for the mailbox. `resendActivation` reissues the credential and
mails it again, up to the 20/day `team:reset-password` budget, so the same
address can be hit repeatedly.

The credential now stays out of the mail on both paths, which is what
`resetMemberPassword` in the same file already does. Nothing is lost: all three
routes return `temporary_password` to the owner, and the SDK has always
documented that as the delivery -- "The returned password is shown once and is
not retrievable afterwards -- deliver it out of band." The mail still goes out,
naming the account and saying the team will supply the password.

The two tests that pinned the old shape are inverted rather than deleted; one of
them carried the rationale "without it this is the only copy", which the return
value contradicts.
2026-10-06 15:56:33 -04:00
Juan Castro d896250ae2 fix: make team deletion resumable, and stop it serializing its mail
`deleteTeam` tore seats down one at a time, each one awaiting a mail send, so a
full 40-seat team was 40 round trips end to end inside a single request. The
per-seat work is independent, so it now runs under the same bounded-concurrency
helper the rest of the backend uses for fan-out sends.

The loop also had no equivalent of the guard `disableMember` carries. That one
checks `suspended` before emitting, with the reason written down: a second
`team.account.disabled` opens a second byte charge that only one `enabled` will
ever close. Deleting a team emitted unconditionally, so any partial failure
followed by a retry double-charged every seat the first attempt had already
disabled. The seat teardown moves into one method both paths can state the same
way, and a retry now resumes instead of re-charging.

Failure semantics are unchanged: a seat that throws still aborts the delete and
leaves the team live, which is what makes the retry meaningful.

Touching this file reformats a few untouched spans -- it carried ten
pre-existing `prettier/prettier` errors that the repo's own pre-commit `eslint
--fix` resolves on any commit that stages it.
2026-10-06 15:56:10 -04:00
Juan Castro 665314535e docs: say what TeamStore actually caches
The cache docblock claimed `isMember` and `getByUid` were left uncached "for
that reason". Both go through it -- `getByUid` directly, `isMember` through
`getMembership` -- so the comment told a reader the opposite of the code, on the
two reads that decide authorization.

What makes it safe is the busting, not an exemption: every mutation here drops
the keys it touches, and the TTL is the backstop for the one path that does not
come through this store, the ON DELETE CASCADE on the membership row. No stale
window is constructible today; the comment now says why.

Also drops a one-line docblock stranded above it, describing a method that is no
longer in the file.
2026-10-06 15:55:40 -04:00
Juan Castro f3b05ca822 fix: page GET /teams like every other team listing
`listTeamsForUser` returned every row and the controller never read `req.query`,
so `limit`, `cursor` and `includeTotal` were accepted and ignored on the one
team route that did not paginate. The docs promise a capped page there, and
every sibling listing already honours it.

Keyset-paginated on `id` per doc/pagination.md, with the envelope the other
routes return. The app-actor directory filter still runs over the page, so an
app sees the same teams it saw before, just a page at a time.
2026-10-06 15:55:22 -04:00
404oops 96f0c51d1e fix(ai): propagate cancellation and stream termination failures 2026-10-06 20:24:19 +02:00
Filip Kujundžić a1eca46a80 Merge pull request #4087 from HeyPuter/fix/responses-sampling-capabilities
fix(ai): omit unsupported sampling options in Responses requests
2026-10-06 20:22:16 +02:00
Daniel Salazar 484c498859 fix: error cleanup (#4089)
* fix: message

* fix: error cleanup
2026-10-06 11:16:04 -07:00
Reynaldi Chernando 71399c5939 sync limits ai, fs, events, networking (#4088) 2026-10-06 23:26:53 +07:00
404oops b1da48b738 fix(ai): omit unsupported Responses sampling options 2026-10-06 18:14:13 +02:00
Filip Kujundžić 9a9972b026 Merge pull request #4079 from HeyPuter/DS/danielsalazar/anthropic-route-fidelity
feat(ai): carry the full Anthropic and OpenAI wire shapes through the chat driver
2026-10-06 18:12:04 +02:00
Juan Castro 4629b544e6 docs: say why AppDriver.select lists apps it does not own
Reads as the same enumeration surface `/query/app` refuses to be, so write down
that it is not: `/query/app` answers for apps the caller names, which is why it
must not confirm one it may not see. `select` is the listing a launcher reads,
and a listing that hid every app but your own would not be one. Protected apps
are still filtered by the permission check, and `user-can-edit` narrows it to
the caller's own.

A test already pins the behaviour by name; this is the reason behind it.
2026-10-06 11:35:20 -04:00
Juan Castro cbb8d708b4 fix: scope the rewrite collapse to fs paths
The collapse caught every `subject_does_not_exist` the rewriters raise, not just
the path resolver's. `app-root-dir:` 404s when the caller's own app has no
resolvable root directory, and that answer is theirs to have — four of its tests
said so.

Only a permission that parses as an `fs:` path can oracle someone else's tree,
so only that one collapses now.
2026-10-06 11:33:27 -04:00
Juan Castro 95809c7dd5 fix: bound the permission check, and let whoami fail
`/auth/check-permissions` took an array of any length. `checkMany` batches the
scan so it is not the fan-out the report describes, but nothing stopped a caller
sending thousands of strings; it now takes the same 16 the grant and revoke
routes take.

`whoami()` returned `resp.json()` without looking at the status, so an expired
token resolved to an error body typed as a `User`. Callers read `username` off
it as undefined — which is how a consent prompt came to ask for
`/undefined/Documents`. It throws the documented `{ status, message }` shape
now, the same one the no-token path already threw.
2026-10-06 11:17:43 -04:00
Juan Castro ac5fa20388 fix: keep the other account out of the OIDC link conflict
Linking a provider identity already bound elsewhere answered 409 with the raw
provider subject and the other account's internal id in the message. The caller
needs to know there is a conflict, not whose it is; both stay in the log for
whoever is debugging it.

Status and legacy code are unchanged — OIDCService branches on the 409.
2026-10-06 11:14:19 -04:00
Juan Castro 3389d3f8a1 fix: treat an abuse hook that could not run as a refusal
The signup paths read `allow` off a payload they initialise to `true`, so a
harness listener that throws leaves the verdict untouched and the signup goes
through. The outer try/catch the report points at is not where it happens:
`emitAndWait` already swallows each listener's error and carries on, so that
catch almost never fires and the throw never reaches it.

`emitAndWait` now records the failure on the metadata it was handed, and both
signup paths — OIDC and the ordinary one — refuse when it is set. Other emitters
are unaffected: the flag is only read where somebody is asking for a verdict.

The report named OIDC alone; the ordinary signup had the same hole, and it is
the path that matters more.

Test boots a server, registers a listener that throws, and posts a signup: 200
before, 403 now.
2026-10-06 11:13:18 -04:00
Juan Castro 39099e2056 fix: size a shared folder from its owner, and keep sub-scopes when widening
Two unrelated bugs in the same area.

`stat({ returnSize: true })` summed `fsentries` filtered on the *acting* user,
so a recipient asking for the size of a folder shared with them matched no rows
and read 0. The sum is over the owner's rows; the ACL check that let them stat
it is what decides whether they may ask at all.

`fs-access-levels` destructured the mode out of the permission but kept the
`.slice(1)` that used to skip it, so widening `fs:<id>:see:thumbnail` produced
`fs:<id>:read` and dropped the sub-scope. Anything granted below the mode stopped
being implied by a wider one. Latent — nothing ships a sub-scope today — and the
test fails without the fix.
2026-10-06 11:05:52 -04:00
Juan Castro acd6ea3c75 fix: do not spend a consent prompt on an app root dir that is not the caller's
`perms.request('appRootDir', { app: <someone else's> })` could never succeed —
only the app itself may claim its root dir — but the SDK read the refusal as
"not granted yet", prompted the user, took the grant, then polled until it gave
up and resolved undefined. A prompt the user cannot act on, and a live grant for
a directory the app will never reach.

The route now says which kind of refusal it is. `forbidden` alone could not
carry it: the SDK already treats that as the refusal a grant is about to lift,
and a unit test pins exactly that sequence, so a new `reason: 'app_mismatch'`
field rides alongside instead of changing the code clients read.

An entry refused that way is held back from the pooled grant and resolves
undefined without a prompt, which is what it resolved to before.
2026-10-06 10:55:19 -04:00
Juan Castro d6c606fb21 fix: keep the owner's path out of the ghost-file 404
A share recipient addresses an entry as /<owner>/<uuid>/<name>, which is the
point of the mask: it says where the item is without naming the folders above
it. The 404 raised when a file's backing object has gone put `entry.path` in the
error fields, so reading or copying a shared file whose object had vanished
handed back the real path.

The fields carry the uuid alone now, on both the read and the copy site. The
caller already has it — it is what they named the entry by.

The existing ghost test asserted the path was present, so it is the regression
pin; the copy one was not checking fields at all and now does.
2026-10-06 10:33:59 -04:00
Juan Castro 8c7372adc6 fix: stop the path rewrite confirming entries the caller cannot see
`fs:/path:mode` resolves through `getEntryByPath` with no ACL check, and a
missing entry answers 404 while an existing one carries on to the authorization
check and answers 403. The pair is a membership test for any path on the
deployment: grant routes take the permission as given, so anyone could ask
whether /someone-else/taxes.pdf is there.

The resolver keeps its 404 for a path inside the caller's own home, where
knowing it is missing is the useful answer and reveals nothing. Everywhere else
it answers with the same `permission_denied` the authorization check gives, so
the two cases are one case. ACLService already collapses them; this is the same
rule reaching the grant and revoke paths, which take an actor and so can tell
whose tree is being named.

Internal resolution still goes through `rewritePermission`: only the paths that
take a caller-supplied permission moved over.

The test grants on an existing foreign path and a missing one and asserts the
outcomes are identical; without the fix they are 403 and 404.
2026-10-06 10:30:58 -04:00
Juan Fernando CastroandReynaldi Chernando 3711d1e837 docs: medium teams recipe (PUT-1976) (#3976)
* docs: add the medium teams recipe

Built around what an app actually does with a team: an assignee picker
keyed on uuid, paging a large directory, scoping stored data per team,
sharing with everyone at once, and telling the three empty cases apart.
One playground example per section.

* docs: split the teams recipe into the two use cases it covered

The medium teams recipe carried six sections, which is the wrong shape for a
recipe: an agent reading llms.txt to answer "add team sharing" had to find it
inside a page called "Work With a Team", next to paging and team switching.

Two recipes, one use case each:

  share-a-file-with-a-team   the whole team, or one teammate via the directory
  show-a-team-owner-panel    gating an admin view on isOwner

Dropped rather than moved: paging and the empty/closed-directory cases, both
already in Add Team Support, and per-team data namespacing, which assumes
multiple teams we do not support yet.

Two corrections while rewriting. A team share names the team in `holder_team`;
`holder` is a username, so the old example printed the wrong field and told the
reader it meant "no one person". And `share()` takes a username for a person,
never a uuid — the picker now shares by username and keeps the uuid for what it
stores, which is what the rename advice was always about.

The old page also linked to /recipes/teams-read-membership/, which does not
exist; the slug is add-team-support.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-06 20:31:27 +07:00
Daniel Salazar 957784bdb8 fix(ai): keep existing chat callers on their previous behavior
Narrows the wire-shape changes to the callers that need them.

- request-level upstream 4xx stays strict only on the Anthropic route;
  /drivers/call and the OpenAI routes fall back on any error again
- outage chains return 400 with the old codes; the Anthropic error
  renderer maps them to 502/529 from the carried upstream status
- native Claude results report finish_reason 'stop' again (real reason
  in stopReason); OpenAI routes still map the real value, and a stream
  that carried tool calls never ends as a plain stop
- Claude usage carries thinking_tokens again, informational only and
  added after pricing and the funds cap
- reasoning_effort is clamped to values each OpenAI model accepts;
  Hoonify, MiniMax, ZAI and Meta only receive params they support
- OpenAI-form tool_choice from /drivers/call is normalized again
- /responses accepts background:false; mcp_servers is ignored; legacy
  media block shapes pass the Anthropic route's validation
- OpenAI web search holds one call; only search actions are billed;
  Azure bills Bing grounding requests; usd_cents includes search fees
2026-10-06 15:22:53 +02:00
Neal ShahandReynaldi Chernando dc029b43a8 Worker and Subdomain Recipes (#4075)
* Worker recipes medium

* subdomain recipes medium

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-06 20:09:38 +07:00
Daniel Salazar 985c76c9c3 docs: medium kv recipes (PUT-1972) (#4041)
* docs: medium kv recipes (PUT-1972)

Adds three recipes:
- Store a Set of Unique Values: an object as a set via update/remove,
  quoted paths for values with dots, like button, recent searches
- Run Code Only Once: incr as an atomic claim, once per day, quotas,
  sequential numbers, webhook dedupe in a worker
- Keep a Version History: incr version numbers, batch set with expireAt,
  reverse list paging, restore, keep last N, delete by prefix

* docs: more medium kv recipes (PUT-1972)

Adds six recipes:
- Keep an Index in Sync: batch set for record + index keys, moving keys
  on change, delete order, per-group counts vs includeTotal, search as
  you type, rebuild and stale-entry cleanup
- Share Data Between Apps: appData scopes, appUuid on every call,
  disableSharing private entries and the set-without-flag gotcha
- Save User Settings: defaults merge, nested paths, reset with remove,
  form binding, sets inside settings, renaming a setting
- Split a Large Value Across Keys: size-based read/write cost, shards
  numbered by incr, latest/older/all reads, expiring old shards
- Export or Delete Data: stream export, filtered batch import, delete by
  prefix, delete before a date, flush
- Build a Shopping Cart: quoted SKUs, incr quantities, TTL via update,
  hide-at-zero, server-side prices at checkout

Also adds a high score section to Add Counters, links Query a Collection
to the index recipe, and replaces its stale newest-first advice with
list({ reverse: true }).

* docs: simplify wording in medium kv recipes

Plainer prose throughout: shorter sentences, no bold or step-by-step
scaffolding, fewer defined terms. Code is unchanged apart from comments.
2026-10-06 19:10:27 +07:00
Daniel Salazar f21cd51edf fix(ai): retry a fast-mode 429 at standard speed on Claude
Fast mode has its own quota. Its 429 used to count as a route failure,
so the request was replayed on a reseller and the Claude route for that
model was marked unhealthy. The provider now retries once without speed.
2026-10-05 20:54:01 -07:00
Daniel Salazar c2fc45a8d1 fix(ai): keep the driver's token count off the /drivers/call surface
count_tokens is only for the Anthropic route, so the driver exposes it
under a symbol key that the RPC resolver never lists.
2026-10-05 16:56:48 -07:00
Juan Castro 614c4c9c2f test: pin the fs delegate path the escalation was reported on
The generic test covers the defect's shape; this one is the case the report
describes. `fs-access-levels` puts `manage:fs:<id>` in the higher set for
`fs:<id>:read`, so a delegate holding the manage terminates the issuer scan with
it as the leaf — which is how the old warm came to write a manage permission
under a holder granted nothing but read.

Without the fix the holder ends up with `manage:fs:<id>` in the flat view and
`canManagePermission` answers true for a read-only share, so they can re-share
the entry at any mode.
2026-10-05 18:19:25 -04:00
Juan Castro ac740984a3 fix: keep the flat permission view to the row it stands for
Two ways the flat KV view could say more than the SQL behind it.

On a miss the warm flattened the whole traversal and wrote every leaf under the
holder. Those leaves belong to the issuer: a holder granted one mode of a
resource by someone who holds the resource itself had the issuer's wider
permission written under their own key, and it answers later checks. The warm
now writes the holder's own row and nothing else, which is what the grant path
has always written.

Deleting an account cascaded the grants it issued out of SQL, but the flat
entries are keyed on the holder, so nothing removed them and they are trusted
without an issuer check. The access outlived the account, and the owner could no
longer see it to withdraw it. The refs are read before the row goes and cleared
after it, through the same batched delete a revoke uses, so peer regions hear
about it once.

Both fail without the fix: the warm test sees the issuer's wider permission
written under the holder, and the delete test reads the grant back from an
account that no longer exists.
2026-10-05 17:51:31 -04:00
Juan Fernando CastroandReynaldi Chernando 63eabf8c7c docs: medium fs recipe (PUT-1968) (#3975)
* docs: add the medium fs recipe

readdir paging, streaming, sorting and recursion, plus the write and
mkdir options that decide what happens to an existing name. One
playground example per section.

* docs: ground the medium fs recipe and examples in the UI that lists

A file browser with infinite scroll and a count footer, a gallery sort
header, an upload endpoint deciding what to do with a repeated filename,
and a report generator writing into a folder tree that does not exist.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-05 21:38:10 +07:00
Juan Fernando CastroandReynaldi Chernando 445588f857 docs: medium perms recipe (PUT-1975) (#3974)
* docs: add the medium perms recipe

Batch requests behind one prompt, appData scopes, raw permission
strings with create, and the entries rejected before any dialog.
One playground example per section.

* docs: ground the medium perms recipe and examples in the screens that ask

A first-run setup, a settings page that renders its real state, a
companion widget reading another app's data, and an importer that wants
folders created on approval.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-05 20:22:09 +07:00
Daniel Salazar 53ca2b61e5 fix(ai): review fixes for the wire-shape PR 2026-10-05 04:09:08 -07:00
Daniel Salazar a242e31343 feat(ai): carry the full Anthropic and OpenAI wire shapes through the chat driver
The Anthropic-compatible route flattened requests into the OpenAI-style
driver shape and rebuilt them for Claude, which dropped system blocks,
thinking, tool_choice, cache_control, betas and stop reasons. Claude
Code requests 400ed upstream and were silently served by a reseller.

- normalized driver args, stream chunks and results gain tool choice,
  thinking, output format, stop sequences, caching, stop reasons, usage
  details, reasoning signatures, server-tool blocks and safeguard results
- ClaudeProvider builds requests from those fields with a beta, tool and
  body-field allowlist; meters fast mode, web search and advisor calls,
  and no longer bills thinking tokens twice
- fallback only on route-level failures; outages map to 502/529
- vendor error envelopes on the Anthropic and OpenAI routes
- Anthropic route: native SSE with pings, count_tokens, /v1/models
- OpenAI-family providers and routes map the same fields; Responses web
  search is metered; reasoning_effort gating fixed; /responses rejects
  org-scoped stateful params and forces store:false
- puter.js: typedefs, new chat options, silent toString for new chunks
2026-10-05 03:35:54 -07:00
Daniel Salazar ba081e2332 feat(ddb): log the table and key of throttled DynamoDB requests (#4078)
A client middleware inside the SDK retry loop logs every throttled attempt
(including ones that succeed on retry) with its operation, table, key,
client region and DynamoDB's throttling reason, plus batch items returned
unprocessed. Each target logs at most once a minute with a count of the
throttles folded into it. Puts log only the key attributes, using key
schemas registered by the KV store at startup.
2026-10-05 02:01:38 -07:00
Juan Fernando CastroandDaniel Salazar a46eb15d5e Let the account's own API token reach the routes a privileged app needs (#4032)
* feat: let the account's own API token reach the routes it should

A privileged app is launched with the GUI session token today. Running one on a
full-access personal access token instead costs much less when it leaks, but
several routes those apps depend on refuse every access token, so the swap would
break them.

Admitted, each a read or a write the token's own HTTP reach already covers:

  - the realtime socket handshake, which refused access tokens outright; FS
    live updates and notifications stop without it
  - GET /get-dev-profile and POST /profile, both puter.js surface
  - GET /auth/list-permissions — a read of what was granted, not a grant
  - GET /share/shared-by-me/apps and GET /share/audit, over grants
    /share/shared-by-me already lists for the same credential
  - POST /rao, which puter.js calls on every setAuthToken

Gated on `isAccountContext`, not the `full_access` claim alone, so a token with
an app anywhere in its chain is still refused and a socket for one never joins
the user room.

Left session-only: mint, session, 2FA, grant, team and billing, plus two the
list named. /share/blocks is a personal safety control whose only caller is the
desktop's Blocked Senders window. /app-feedback is reachable only from our own
GUI pages, and being unsubmittable programmatically on a user's behalf is the
property it was built with.

The new HTTP suite drives a real server with a minted token and fails on all six
admissions without the change; the block routes' existing metadata check already
pins their refusal.

* test: sort the two token shapes at the events handshake

* fix: carry the account-context invariant into the HTTP gate

`requireUserActorGate` admitted on the raw `full_access` claim while the socket
and /rao use `isAccountContext`, so one shape — a token carrying the claim with
an app in its chain, or an actor built without `makeActor` — would have been
refused in two places and admitted in the third. Unreachable today, since the
mint and the auth-time read both drop the claim when `app_uid` is present, but
it is the invariant this change states and it should hold at every enforcement
point.

The payout address goes back behind a plain session: `/get-dev-profile` is open
to the account's own token now, and the rest of what it answers is a developer's
own name and incentive-program status.

Also drops a cross-reference in EventsService to the socket posture this change
relaxed, and rewords the block-routes comment, which read as if access tokens
were admitted there rather than refused.

* fix: drop a revoked token's sockets, and read the dev-profile columns

Two gaps the review found, both reachable only because this branch lets the
account's own token hold a socket and call /get-dev-profile.

`#revokeAccessTokenTail` soft-revoked the session row and told no one, so a
revoked token kept its connection — and the account's `outer.gui.*` fan with it
— until the five-minute reauth sweep. Sockets now join a per-token room and a
new `auth.access-token.revoked` event drops exactly that room, so the account's
other tabs stay up. Session revoke keeps its account-wide eviction.

`/get-dev-profile` read `first_name`, `last_name`, `paypal` and the two
incentive flags off the user row, which carries none of them: the columns have
always been `dev_`-prefixed, so the endpoint answered nulls to everyone and
`puter.apps.getDeveloperProfile()` has never returned anything. Reads the real
columns, keeping the unprefixed name as a fallback for a deployment carrying
both. The payout address stays behind a plain session, so opening the route to
the account's token hands over a name and incentive status and nothing else.

Both fail without the fix: the socket test waits five seconds on a connection
that should already be gone, and the profile test reads back a seeded row.

* fix: close /open_item to tokens, and stop the node runner leaking sockets

`/open_item` carried `allowFullAccessToken`, but it grants an app a permission
on the user's behalf and mints the app token to go with it — a grant and a mint,
which stay session-only. Pre-existing, and missed by the first sweep here, which
only asked whether routes lacking the flag should gain it and never re-read the
ones that already had it. Refused at the top of the handler too, so a credential
that gets past the gate cannot leave an ACL row behind.

The node runner loads each SDK into a vm context and never closed it, so every
test leaked a socket, and every socket holds a per-(user, origin) connection
slot. Admitting the account's own token to the handshake pushed that past the
cap, and the whole events block failed with `events_connection_failed` while the
browser and workerd runners — which do not leak — passed. Closing the FS socket
and the events channel after each test gives the slots back.

* fix: announce only the revokes that can have a socket to drop

Every access-token revoke broadcast an eviction cluster-wide, including the
read-URL tokens `revokeReadUrl` retires constantly. A scoped token is refused at
the handshake, so none of those broadcasts could ever reach a connection.

The announcement now goes out only for a token the handshake would have
admitted: full access, no app in the chain. `revokeOwnAccessToken` refuses
full-access tokens outright, so that whole path is silent; a revoke by raw uuid
says nothing about the token and still announces, which costs a no-op broadcast
rather than leaving a revoked socket up.

Fails without it: the test revokes a scoped token and a full-access one through
the same entry point and counts the announcements.

* test: pin the mint route against the account's own token

`/auth/create-access-token` carries only `requireAuth`, so it never showed up
in the sweep over `requireUserActor` routes — the refusal lives in
`AuthService.createAccessToken`, which turns away any access-token actor
outright. That holds today, and a leaked full-access token minting itself a
sibling that survives revoking the original is exactly what the token swap is
meant to rule out, so it is worth a test rather than a reading.

* fix: let the recommended-apps test follow the list it tests

Two commits on main rewrote `RECOMMENDED_APP_NAMES` without touching the test,
which seeds `editor` and `camera` and asserts the result contains them. Neither
is on the list any more, so the resolved set came back empty and `test (base)`
has failed on every PR since.

The test now seeds the first two names off the list itself and asserts on those,
so the next edit to it cannot strand the test: prepending two unknown apps keeps
all four cases green.

* feat: let an extension waive a plan gate for one route or driver method

A plan gate about to refuse a caller now emits
`subscription.gate.<surface>` through `emitAndWait` first:
`route.<method>.<path>` for a route, `driver.<iface>.<method>` for a driver
method, `share.anyone` for link shares. The payload carries the actor, the
request and the requirement, and a listener that sets `allow` waives the plan
for that call. Callers the plan already covers never reach the hook, and a
listener that throws leaves the refusal in place.

* feat: mint godmode apps their own full-access token

`/auth/get-user-app-token` answers a desktop launch of a godmode app (the
`godmode` column) with a full-access token instead of an app token. Origin
lookups, the sign-in popups for pages outside the desktop, keep getting
ordinary app tokens, and only a web session may mint one.

The token carries the app in `godmode_app_uid`, read into
`accessToken.godmodeApp` for attribution only: `effectiveApp` stays null, so
it keeps the account's reach and passes the same gates as a personal access
token, including the vendor-compatible AI routes. It is refused once the app
is gone or no longer godmode.

Its row is an `access_token` session parented to the web session that asked,
with `app_uid` set, so signing out, revoking that session or revoking all
sessions takes it along, and the sessions list shows it under the app. Each
mint re-signs onto the same row with a 12h `exp` and moves the row's expiry
out, so the desktop keeps an open app alive while a copy taken earlier still
expires on schedule.

A full-access token may now mint scoped tokens (read URLs), parented to its
own row; it still may not mint full access. Revoking a full-access token
cascades to the tokens it minted.

An access token whose session row has a parent now authenticates only while
that parent is live, so a read URL a godmode app minted lapses with the app's
token, and through it with the desktop session.

* feat: launch godmode apps on their own token and renew it

The desktop asks for a godmode app's token at launch and blocks the launch if
it can't get one, as it does for other apps; a godmode app never runs on the
desktop's session token. While the window is open the desktop re-mints the
token once less than 6h of its 12h remain, checking on a timer and when the
tab becomes visible, and answers the app's `reauth_required` with a fresh one,
posted to the app's origin.

puter.js keeps a godmode token in sessionStorage like a session token, never
in localStorage. In app mode a 401 on a godmode token, background requests
included, asks the desktop for a new token and replays, and a `puter.token`
message is only taken from the embedding frame.

The sessions list shows a godmode token row with its app's title and icon.

* fix: keep a godmode token the same across renewals

Apps hand their launch token to code that never sees a replacement (the
terminal's Drive mount, a CLI's API key), so a token re-signed every renewal
broke those copies once the first one expired. The JWT now carries no `exp`
and a renewal moves only its row's expiry, so every copy keeps working while
the desktop keeps the app alive.

What still bounds it:
- only the desktop's web session can extend it; using the token never does,
  and signing out or revoking that session revokes it;
- it lapses 12h after the desktop last asked, and a lapsed row is never
  revived: the next launch gets a new token;
- however often it is renewed, a token is replaced after 7 days, and the old
  one is revoked on the spot along with the tokens it minted and its sockets.

The desktop no longer re-posts a renewal that kept the same token, so the
app's sockets aren't rebuilt every few hours; an app that reports its token
stopped working is always answered.

* fix: reserve the feedback and no-reply mailbox names again

Signup checks names against `util/reservedUsernames`, but `fbl`, `noreply` and
`no-reply` were added to a second copy of the list in AuthController that
nothing reads any more, so they became claimable. Move them into the live list
and drop the copy.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-10-05 01:10:29 -07:00
Neal Shah 970092f0a0 PUT-1995 (#4020) 2026-10-05 03:26:54 -04:00
Neal Shah 6dc61b2d67 PUT-2011 (#4077) 2026-10-05 03:23:14 -04:00
Neal Shah 7976dc9a29 attach egress actor to /down (#4076) 2026-10-05 03:09:16 -04:00
Daniel Salazar a76800d693 fix(thumbnails): duplicate a copied thumbnail with GetObject + PutObject instead of CopyObject (#4072)
* fix(thumbnails): duplicate a copied thumbnail with GetObject + PutObject instead of CopyObject

CopyObject names its source as `<bucket>/<key>` in a header, which skips
any path in the client's endpoint, so it can miss an object that
Bucket/Key requests reach. The copy handler now reads the source object
(still behind the HeadObject size bound) and writes it under the copy's
key with its content type.

* fix(thumbnails): check the GetObject length before buffering a copied thumbnail

The size check runs on a HEAD and the read on a separate GET, so an object
replaced in between was read whole into memory. Reject a GET response whose
length exceeds the cap and destroy its body before reading.
2026-10-04 21:11:00 -07:00
Daniel Salazar 1f1e5674d8 chore: bump puter js ver (#4073) 2026-10-04 17:17:38 -07:00
Daniel Salazar 8db4fb4f5b fix(ai): chat credit gate ceiling, in-flight holds and disconnects (#4071)
* docs(agents): no credits or bug-report mentions in commits and PRs

* fix(ai): chat credit gate ceiling, in-flight holds and disconnects

- Cap output at min(max_tokens, context - prompt). The gate used
  max_tokens - prompt, so any prompt longer than a model's output limit
  was refused with 402 regardless of balance. Gemini 2.5 Pro's output
  limit corrected to 65,536.
- Abort the upstream stream when the caller disconnects (driver calls and
  the OpenAI/Anthropic-compatible routes), which ends the credit hold.
- A request refused only because the account's own in-flight requests
  hold its balance gets 429 `credits_reserved` instead of 402; puter.js
  retries it with exponential backoff (1s to 30s, about 2 minutes).
- When the balance caps a completion and it uses the whole cap, the
  response carries `usage_limited` (non-stream `metadata`, stream `usage`
  line), which puter.js already turns into the upgrade prompt.

* perf(ai): read the balance and its holds once per credit gate

The gate re-read holds, and the whole balance when any were held, to tell
spent from committed. getUsageHeadroom returns both halves from the same
reads getRemainingUsage already did.
2026-10-04 17:10:51 -07:00
Daniel Salazar e7c89b0639 fix(apps): keep ids longer than an app uid out of the app-stats query (#4070)
ClickHouse's app_opens.app_uid is FixedString(40) and fails the whole
query on a longer value. Marketplace callers pass item ids into
getAppsStats, so one long listing-request id turned the seller's
listing-request list into a 500. Such ids can't have opens; they now
resolve to zero without being queried.
2026-10-04 17:03:19 -07:00
Daniel Salazar 83d64ac685 fix: follow-ups from the reviews of #4047, #4049, #4051, #4053, #4055 and #4057 (#4068)
* fix(auth): issue reauth tokens only for browser sessions

A rejected token's 401 carried a signed reauth_token, which /signup turns
into a session on a temp account. Any revoked app or access token, a stale
token from a deleted app whose uid was reused, or a worker credential could
get one. Only a GUI/browser session's rejection now carries a reauth_token;
everything else still gets reauth_required without it. The stale-app check
also runs before the session is touched.

* fix(events): refuse worker creation from an events handler

workers.create minted a worker token with no handler depth, so a handler
could deploy a worker and chain writes through it at depth 0. A handler
actor can no longer create a worker; delivery, deploys and hot reload mint
from plain owner actors and are unaffected.

* fix(ai-ocr): bound the page count's parsing and close its under-counts

- Object-stream headers are capped (/N, /First) and read incrementally, and
  the inflate budget scales with the file, so a crafted PDF can't spend
  seconds and gigabytes before the credit check.
- An escaped /ObjStm type, an indirect /Count, untyped page leaves and an
  object redefined to a smaller tree no longer lower the count.
- An account with no balance is refused before the file is parsed.

* fix(fs): add a switch to stop accepting pre-binding signed URLs

Signed URLs issued before owner binding are still accepted, and owner-signed
ones never expire, so the move-to-another-tree attack stays open for them
indefinitely. legacy_file_signatures (default on) can now turn them off, a
rate-limited warning counts their use so it's visible when they've died
out, and a non-UUID uid is refused before the entry lookup.

* fix(fs): hide the issuer's home uid from a scoped token's entries

A token scoped below the home could still learn the home's uid as the
parent_uid of a direct child (e.g. stat on ~/Documents). For access-token
actors that can't list the home, a direct child's parent uid is now null in
the v2 and legacy entry shapes, using the same rule as the root listing.

* fix(gui): ask for the 2FA code on the last step, right before enabling

The setup flow verified a code before the recovery-codes screen and sent
that same code to enable afterwards; by then it had usually expired, and
the user was sent back to the start. The code is now entered last and goes
straight to enable; a wrong code keeps the user on that step, and other
failures show the server's message. Enter submits the password step.

* test(fs): expect the legacy-signature flag in the signing config

* fix: apply the review's follow-ups to the merged-PR fixes

- Reauth tokens also require the session row to be a browser ('web')
  session, not just the absence of the worker claim.
- The OCR page count only treats an untyped object as a page when it sits in
  a page tree's /Kids, so outline items, widgets and name-tree leaves can't
  inflate an honest PDF's count.
- The signed-URL uid guard checks the uuid's shape only, so a legacy row
  whose uuid has nonstandard version/variant bits still resolves.
2026-10-04 14:46:21 -07:00
Daniel Salazar 5e34258da5 fix: drop database_id from the subdomain reclaim; MySQL has no such column (#4067)
reclaimByUuid (#4064) cleared `database_id`, which exists only in the SQLite
and Postgres schemas. On MySQL the UPDATE fails with "Unknown column", so
ensureSystemSite throws: the puter-profiles row stays registered to the
admin, the site keeps 404ing, and every profile save returns 500 as it
retries the bootstrap. puter-app-icons takes the same path if its row isn't
already protected. System rows never carry a database binding, so there is
nothing to clear.
2026-10-04 02:45:03 -07:00
Daniel Salazar 51eba174b3 fix: alert errors from timeouts (#4066) 2026-10-04 02:44:40 -07:00
Daniel Salazar f533eb84ae fix(ai-ocr): check credits for the whole document before calling the provider (#4055)
* fix(ai-ocr): check credits for the whole document before calling the provider

The pre-flight checked one page's cost because the page count was only
known from the provider's answer, so a 130-page PDF passed on a balance
worth a fraction of it and drove the account negative.

The driver now estimates pages before the call: a PDF's own count (read
from its page tree, including compressed object streams), capped by a
`pages` selection; one page for images and Textract, whose synchronous
API reads single-page documents. Other documents and PDFs it can't read
count 20 pages per MB, Mistral's 1,000-page limit at its 50 MB size cap.
The estimated cost is checked and held while the provider runs, so
concurrent calls see it. Usage is still metered from the pages the
provider reports.

* fix(ai-ocr): price the credit hold at the AI cost factor

Billing scales OCR usage by the ai.cost.factor hook, but the hold was sized
at the raw cost, so a balance covering N pages passed the check and was
then charged more. Hold through reserveAiCredits like the other AI drivers.
2026-10-04 00:53:19 -07:00
Nikhil Jangid 3dfc70cab7 fix: clarify team directory app access (#4031) 2026-10-04 00:51:02 -07:00
Daniel Salazar ebf5086e1c fix(fs): keep restore metadata out of reach of anyone but the tree's owner (PUT-2050) (#4063)
Follow-up to #4048. original_path/original_name decide where a GUI restore
lands, and the owner's restore is unchecked, so whoever sets them can aim a
file at another app's AppData root and block that app's launch. #4048 only
pinned them on move for non-account actors; a write could still set them,
and a share recipient's session counts as account context.

Writes now drop the trash keys for every caller. On move, only the account
that owns both source and destination may set them; anyone else moving into
the owner's Trash gets them recomputed from the entry, and other moves drop
them, including keys already on the row. Copies drop them on every branch.
The GUI only offers Restore for direct children of Trash, since nested rows
never carry legitimate trash metadata. The AppData/Trash checks for
cross-app deletes and create grants now ignore case, like the root guard.
2026-10-04 00:48:48 -07:00
Daniel Salazar 59e28bed23 fix(metering): keep a failed lookup's fallback out of the credit cache (PUT-2002) (#4062)
Follow-up to #4052. A failed subscription lookup was cached for 5 s, but the
credit cache built from its fallback plan lived 15 s, so a paid user who
had spent past the free allowance got 402s for ~15 s after one failed read.
Credit entries built from a provisional answer (a failed lookup, or an
actor missing its email or numeric id) are now capped at that answer's
expiry and don't drive credit-state alerts.

A failed lookup that finishes after a concurrent successful one no longer
replaces the good cached answer, actors without a numeric id aren't cached
(resolvers key on it), and a failed credit refresh fails open briefly
instead of replaying a stale answer or blocking on the failing store.
2026-10-04 00:48:35 -07:00
Daniel Salazar 73ebe3f47a fix(metering): decide app-usage access in MeteringService only (PUT-2015) (#4061)
The /metering/usage/:app handler repeated the "an app reads only its own
usage" check while the service still let any app read the global bucket.
The service now refuses an app actor anything but its own app, and the
handler only resolves names. Status codes for app actors are unchanged; a
user reading os-global gets their global usage instead of a name-lookup 404.

whoami no longer sets taskbar_items to undefined for app actors, and only
keys with a value count as built, so a listener can't fill one in for an
app. getMonthlyUsage's JSDoc notes allowanceInfo is account-wide.
2026-10-04 00:47:37 -07:00
Daniel Salazar c981a51f21 fix(fs): bind signed file URLs to the owner they were issued under (#4053)
New /sign signatures are an HMAC over uid, action, expiry and the entry's
owner at signing time. /file and /writeFile resolve the entry first and
verify against its current owner, so a move that hands the entry to another
account ends its URLs and stops /file minting child URLs from them. The URL
shape is unchanged; signatures in the previous format still verify, without
the owner check.
2026-10-04 00:47:16 -07:00
Daniel Salazar 4ef5e6c110 fix(socket): hold unlisted free plans to the free connection cap (PUT-1830) (#4060)
The per-account socket cap looked a plan up directly in its map, so
org_seat_free (team seats on a free team) got the paid cap. Route and
events limits already gave an unlisted free plan the default free entry;
that rule now lives in one helper, subscriptionOverride, used by all three.

Also publishes the socket connection caps, corrects the seat-plan line in
rate-limits-and-quotas.md, and makes the isFreeSubscription test assert
literal ids.
2026-10-04 00:46:59 -07:00
Daniel Salazar 794469845d fix(thumbnails): bind thumbnail keys to their entry and size-bound uploads (PUT-1999) (#4054) 2026-10-04 00:46:35 -07:00
Daniel Salazar 9ca77be5a9 fix(auth): require a step-up to set up 2FA and a valid code to enable it (#4051)
Setup returned a fresh secret and recovery codes to any session, and enable
flipped otp_enabled without proof of the secret, so a stolen session could
enroll its own authenticator and lock the owner out. Setup now lives at
/user-protected/setup-2fa behind the same password / OIDC revalidation gate as
disable-2fa, and enable requires a live code for the stored secret. The GUI
setup window asks for the password (or opens the revalidation popup) first,
and sends the code it already collected to enable.
2026-10-04 00:26:28 -07:00
Daniel Salazar 007d5391a1 fix(events): carry handler depth onto tokens a handler mints (#4049)
An events handler could mint a fresh token for its own app through /sign or
/auth/create-access-token, and writes made with it started a new chain. An app
token minted from a handler token now carries its handler_depth and expires
with it; an access token carries the depth at the lifetime asked for.
2026-10-04 00:25:54 -07:00
Daniel Salazar c0f4fe93b1 fix(auth): stop a deleted app's sessions authenticating as the next app with its uid (PUT-2049) (#4057) 2026-10-04 00:24:55 -07:00
Daniel Salazar d57f638eda fix: keep system sites writable and their names out of reach (#4064)
Follow-up to #4056. Icons and profiles are now written as the system user,
so on a storage-limited install they counted against its allowance (100 MB
by default) and every write past that returned 413, including a user's
profile save. Both writes now pass the unmetered allowance, as writes the
system makes on users' behalf already may.

ensureSystemSite reclaims a mismatched subdomain row in place instead of
delete+create, so the name is never free for a user to register, the row
keeps its uuid, and `protected` is never cleared; the previous holder's
bindings are dropped. If the row vanished since the read it falls through
to create. puter-app-icons is now protected like puter-profiles, both names
are reserved from user registration, and they live in util/systemSite so
the subdomain driver doesn't import the services. The icon service's
shutdown wait is capped at 5 s so a stalled run can't hold up the metering
flush.
2026-10-04 00:23:01 -07:00
Daniel Salazar 26abd4e4a5 fix(fs): hide the issuer's home from scoped-token root listings (PUT-2053) (#4047) 2026-10-04 00:22:43 -07:00
Daniel Salazar 3defae3cfc fix: own system sites by the system user (#4056)
* fix: register system sites to the owner of their directory

The hosting middleware refuses a site whose user doesn't own its root
directory (absent a manage grant). The app-icons and profiles bootstraps
always registered their subdomain to the admin, but the directory can belong
to another user: a carried-over `/system` owned by the `system` user, which a
new child directory inherits. Those sites then 404 on every request.

Both bootstraps now go through ensureSystemSite, which registers the
subdomain to the directory's owner and replaces a row registered to anyone
else or pointing at another directory.

Fixes #4045

* fix: own system sites by the system user

`/system`, `/system/app_icons` and `/system/profiles` and their subdomains
belong to the system user, and icon and profile files are written as it.
ensureSystemSite re-owns directories another account created and replaces a
subdomain row registered to anyone else, so installs bootstrapped under the
admin move over on boot.

The bootstrap no longer waits for an admin account, so DefaultUserService
stops re-running it, and deployments without an admin get icons and profiles.
AppIconService lets in-flight icon runs finish on shutdown.
2026-10-03 17:06:13 -07:00
Daniel Salazar 205f19a167 fix(metering): stop caching subscription answers from email-less actors and failed lookups (PUT-2002) (#4052) 2026-10-03 16:39:08 -07:00
Daniel Salazar 2cfc1461e4 fix: scope whoami billing details and monthly usage to the calling app (PUT-2015) (#4050)
* fix: scope whoami billing details and monthly usage to the calling app (PUT-2015)

* fix(metering): refuse an app another app's detailed usage (PUT-2015)
2026-10-03 16:38:31 -07:00
Daniel Salazar 3fa23bb809 fix(fs): refuse renaming or moving AppData roots (PUT-2050) (#4048)
* fix(fs): refuse renaming or moving AppData roots (PUT-2050)

* fix(fs): refuse a forged trash original_path from apps (PUT-2050)
2026-10-03 16:37:36 -07:00
Daniel Salazar e59f6285ba refactor(metering): route free-plan checks through isFreeSubscription (PUT-1830) (#4046) 2026-10-03 16:36:05 -07:00
Filip Kujundžić c3abec8663 Merge pull request #4035 from HeyPuter/filipkujundzic/put-1895-chat-credit-gate-cost-factor
fix(ai): price the chat credit gate at the factored cost (PUT-1895)
2026-10-03 19:44:51 +02:00
Neal Shah 04406373f4 event based FBL implementation (#3969) 2026-10-03 05:54:04 -04:00
Daniel Salazar 91aff909e1 fix: keep launch tokens out of URLs and storage (PUT-2040) (#4038)
* fix: keep launch tokens out of URLs and storage (PUT-2040)

- Strip the consumed launch token param from the app URL after load
- Never persist user session (godmode) tokens to localStorage; purge any stored one at boot
- Worker log socket sends the token as its first message instead of in the URL path

* fix: strip the private-app sign-in token from top-level URLs, keep godmode tokens across reloads (PUT-2040)

- Private-app gate: a top-level GET carrying ?puter.auth.token= that passes the gate gets the sticky cookie and a 302 to the same URL without the token. Framed launches are untouched.
- Godmode session tokens go to sessionStorage (never localStorage), so a frame reload stays signed in; cleared on sign-out or when an app token replaces it.
2026-10-03 00:26:31 -07:00
Daniel Salazar a242105540 fix(events): keep app-less scoped tokens off the account's durable rows and mailbox (#4042)
A personal access token scoped below the account (e.g. the one in an
account-minted getReadURL() URL) has effectiveApp === null, which the durable
list/unsubscribe/ack scope check and notif fetch read as account context. Gate
those on isAccountContext, as workers/handlers/kv-handles already do, and refuse
such tokens on durable subscribe.
2026-10-02 22:37:07 -07:00
Reynaldi Chernando 47510897ed add ai builder link to docs (#4044) 2026-10-03 10:27:23 +07:00
dependabot[bot] d0da8c5b1e chore(deps): bump engine.io from 6.6.9 to 6.6.11 (#4017)
Bumps [engine.io](https://github.com/socketio/socket.io) from 6.6.9 to 6.6.11.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@6.6.11)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-version: 6.6.11
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 15:03:34 -07:00
dependabot[bot] 303ee6c2c7 chore(deps-dev): bump brace-expansion from 1.1.18 to 1.1.21 (#4015)
Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 1.1.18 to 1.1.21.
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.18...v1.1.21)

---
updated-dependencies:
- dependency-name: brace-expansion
  dependency-version: 1.1.21
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 15:02:56 -07:00
Daniel Salazar 37c1678897 fix(fs): keep a vacated username off-limits to other accounts for an hour (#4039)
* fix(fs): keep a vacated username off-limits to other accounts for an hour

Renaming a home rewrites descendant paths in the database, but their
cached entries keep the old path until they expire. ACL grants a home by
path prefix, so whoever takes the old name next must not be able to
before those entries are gone. renameUserHome now records the vacated
name, and the claim check every username-claim site already runs treats
it as taken for anyone but the account that left it.

* test: seed recommended apps that are still in the default list

The default list no longer includes editor or camera, so the ordering
test found neither.
2026-10-02 14:53:34 -07:00
Filip KujundžićandClaude Opus 5.5 7b8b3752e4 fix(ai): price the chat credit gate at the factored cost
The chat gate read `model.costs` directly to decide affordability, cap
`max_tokens` and size the credit hold, while usage is recorded at cost
times the AI cost factor, so all three ran low by the factor on every
request. The gate now resolves the factor and applies it to both rates.

The factor is looked up by the key the provider records usage under.
Providers expose that key through `meteringModelKey` and use it when
recording, so the gate and the charge can't drift apart; the
unreported-stream backstop records under the same key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:21:18 +02:00
404oopsandClaude Opus 5.5 521812be02 refactor(ai): move the AI cost factor out of MeteringService
MeteringService knew about AI: it built the cost-factor facade, resolved
the `ai.cost.factor.*` hook and, since the previous commit, offered
`reserveAiCredits`. Metering stays agnostic to what it meters, so all of
that now lives with the AI drivers in `drivers/util/aiCostFactor.ts`.

`withAiCostFactor(metering, events, driver)` returns an
`AiMeteringService`: the service with recording scaled by the factor,
plus `costFactor` and `reserveAiCredits` for gates. The speech providers
take that type. MeteringService itself has no AI methods left.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 21:20:24 +02:00
Nariman Jelveh bb38e8c86f Update RecommendedAppsService.ts 2026-10-02 11:29:42 -07:00
Nariman Jelveh 74d89b64fd Update RecommendedAppsService.ts 2026-10-02 11:19:37 -07:00
Daniel Salazar 31e5b64a0f fix: make app-data delete grants imply write and read (PUT-1998) (#4022) 2026-10-02 10:47:58 -07:00
Daniel Salazar e96cf77982 feat(metering): per-month allowance override for partly billed plan changes (PUT-1996) (#4027) 2026-10-02 10:40:52 -07:00
Filip KujundžićandClaude Opus 5.5 22d52cf544 fix(ai): name data-URL audio with its real extension
A data URL's filename came from its MIME subtype, so `audio/mpeg`
became `input.mpeg` and `audio/x-wav` became `input.x-wav`. OpenAI's
transcription API picks the decoder from the extension and rejected
both. Map the MIME types whose subtype isn't the extension.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 19:28:13 +02:00
Daniel Salazar 86bb94d3d9 fix: cap FS batch array lengths (PUT-1894) (#4024) 2026-10-02 10:06:21 -07:00
Daniel Salazar a4f1e665df fix: return 400 when a streamed upload is shorter than its declared size (#4023)
A streamed write sends the caller's declared size to the object store as
the body length. A body that ends short of it was rejected as
IncompleteBody and surfaced as a 500.
2026-10-02 10:05:13 -07:00
Filip KujundžićandClaude Opus 5.5 abd0d809d5 fix(ai): price speech credit gates at the charged cost and hold them in flight
Text-to-speech, speech-to-text and voice-changer providers checked
affordability against the unscaled provider cost and never reserved it,
so the gate ran low by the AI cost factor and concurrent requests could
each spend the whole balance. They now go through
`reserveAiCredits`, which scales the amount by the model's cost factor,
checks it, and holds it until the request finishes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:56:53 +02:00
Filip KujundžićandClaude Opus 5.5 82bd7a7360 chore(ai): sync model catalogs with provider listings and deprecations
Remove models providers have deprecated or retired, add newly released
ones, and move provider defaults off retired models. Gemini video (Veo
via the Gemini API) is removed; Infron and OpenRouter skip models their
listings mark deprecated. xAI bills cached prompt tokens only at the
cached rate. Claude Sonnet 4.6 max output raised to 128K.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:52:35 +02:00
Reynaldi Chernando b32ae7dcfc Add rate limit detail for contact form recipe (#4030)
* Add rate limit detail for contact form recipe

* handle ipv6
2026-10-02 23:19:41 +07:00
Juan Fernando Castro d5e6d5ea07 Merge pull request #4007 from HeyPuter/juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant
fix: refuse opaque origins cleanly instead of a logged 400 (PUT-1896)
2026-10-02 09:58:13 -04:00
Reynaldi ChernandoandCopilot Autofix powered by AI f46d187d51 contact form recipe (#4029)
* contact form recipe

* Potential fix for pull request finding 'Network failures cause unhandled fetch rejections'

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* minor

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-10-02 20:24:05 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 0fcee62884 Update recipes for storing a small list and store per id (#4028)
* Update recipes for storing a small list and store per id

* Potential fix for pull request finding 'Clarify that IDs must be path-safe or properly escaped'

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-10-02 19:23:39 +07:00
Reynaldi Chernando 384524a027 improve teams docs (#4025) 2026-10-02 14:45:54 +07:00
Daniel Salazar 42b204f46e fix: don't count failed username change towards rate limit (#4021) 2026-10-02 00:26:37 -07:00
Daniel Salazar 91c8b2385b fix: run a broadcast persistent handler in the worker or the clients, not both (PUT-1889) (#4016) 2026-10-02 00:08:02 -07:00
jelveh 05e2e7428a feat(gui): let users remove their profile picture
The account tab only offered changing the avatar. Add a "Remove photo"
action, shown while a picture is set, that clears it via
update_profile({ picture: null }) and resets every avatar to the default.
On failure the hint line says so and the picture stays.

The tab can render before the profile loads, so the profile loader also
reveals the button once a picture arrives. Also move the avatar hint
into i18n.
2026-10-01 23:21:57 -07:00
dependabot[bot] f387a7aa68 chore(deps): bump fast-uri from 3.1.7 to 3.1.8 (#4014)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.7 to 3.1.8.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.8)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 18:39:45 -07:00
Daniel Salazar e6bda950f0 perf: batch per-app origin, canonical and hosted-backing lookups in app listings (PUT-1892) (#4013) 2026-10-01 18:30:06 -07:00
Daniel Salazar 4f31388dad fix: cap app lookups and fields, dedupe icon processing, rate-limit app landing pages (PUT-1891) (#4012) 2026-10-01 17:54:25 -07:00
dependabot[bot] 376a047ecc chore(deps): bump @grpc/grpc-js from 1.14.4 to 1.14.5 (#4010)
Bumps [@grpc/grpc-js](https://github.com/grpc/grpc-node) from 1.14.4 to 1.14.5.
- [Release notes](https://github.com/grpc/grpc-node/releases)
- [Commits](https://github.com/grpc/grpc-node/compare/@grpc/[email protected]...@grpc/[email protected])

---
updated-dependencies:
- dependency-name: "@grpc/grpc-js"
  dependency-version: 1.14.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:50:14 -07:00
dependabot[bot] fe7b0ce535 chore(deps): bump axios from 1.18.1 to 1.20.0 (#4008)
Bumps [axios](https://github.com/axios/axios) from 1.18.1 to 1.20.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.18.1...v1.20.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.20.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:49:54 -07:00
dependabot[bot] e85c6917f7 chore(deps): bump fastify from 5.12.3 to 5.12.5 (#4009)
Bumps [fastify](https://github.com/fastify/fastify) from 5.12.3 to 5.12.5.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/commits/v5.12.5)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:49:42 -07:00
Juan Castro 989399a857 Merge branch 'main' into juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant 2026-10-01 20:30:41 -04:00
Juan Castro 143948ed2c Merge branch 'main' into juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant 2026-10-01 20:30:00 -04:00
Juan Fernando Castro e48dc0cbf3 Merge pull request #4011 from HeyPuter/juancastro/put-1997-bound-grant-and-token-route-inputs
fix: bound the inputs on the grant and token routes (PUT-1997)
2026-10-01 20:29:42 -04:00
Juan Castro a67cdac380 fix: bound the inputs on the grant and token routes
Three permission routes sized their work by the request body rather than by
the route, so one call could cost far more than its rate-limit slot implies.

- `extra` and `meta` were only type-checked. They ride every row a grant
  writes (up to 16) and are re-serialised into the per-(user, app) cache on
  each miss, so they are now capped at 4 KiB.
- `/auth/create-access-token` took a `permissions` array of any length: one
  permission check and one sequential INSERT each. It now uses the same
  16-per-request cap the grant routes already had, and runs every entry
  through the validator those routes use.
- Withdrawing an app's cross-app data grants looks them up by permission
  text, and no index on `user_to_app_permissions` led with `permission`.
  Indexed per engine, mirroring what mysql_mig_22 / postgres_mig_11 /
  sqlite 0067 did for `user_to_user_permissions`.

`grant-dev-app` validated none of its input — not even the type of `extra` —
so it now runs the same validator as its user-app sibling.

Two paths could also carry a permission wider than the `varchar(255)` column
it lands in. `grantUserAppPermission` already rejected that after rewriting;
`grantDevAppPermission` now does the same, and `createAccessToken` checks it
before the session row a failing INSERT would otherwise orphan.

Caps are published in rate-limits-and-quotas.md. Every in-tree caller sends
one permission and a small `extra`, so none of them change behaviour.
2026-10-01 16:51:39 -04:00
dependabot[bot] c6da95beb0 chore(deps-dev): bump undici from 7.29.0 to 7.30.0 (#3991)
Bumps [undici](https://github.com/nodejs/undici) from 7.29.0 to 7.30.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.29.0...v7.30.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.30.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 09:52:01 -07:00
Daniel Salazar ed7b8f828a fix: kv path errors and ttl hardening, events subscription end and race fixes (#4004) 2026-10-01 09:50:34 -07:00
Juan Castro 29c6f78b7c fix: refuse opaque origins cleanly instead of a logged 400
A page the browser gives no origin to — one opened straight from disk, or an
iframe sandboxed without `allow-same-origin` — serialises its origin as
`"null"`. `appUidFromOrigin` logged that at error level on every attempt and
the GUI sent it on three endpoints, so each refusal cost a 400 and an
error-level line while the user got nothing useful.

An opaque origin cannot name an app to mint a token for, and is not a valid
`postMessage` target to deliver one to, so these clients are refused rather
than supported. Make the refusal quiet, early and legible instead:

- AuthService: drop the `console.error` and say what the caller should do.
  The accept/reject set is unchanged; a 400 is already counted per route.
- puter.js: `signIn()` rejects with `unsupported_origin` before opening
  anything, which covers implicit auth too since it routes through `signIn`.
  The load-time warning now covers sandboxed iframes, console-only — a modal
  belongs in nobody else's embed.
- GUI: a popup whose opener has no attested origin says so and closes,
  instead of rendering a blank window and wedging on a failed exchange.

Also fixes two faults this made reachable: `PuterDialog.open()` prefers the
popup branch, which reaches the `puter` global before the constructor that
assigns it has returned (`puter is not defined`, no SDK at all); and
`showModal()` throws where modals are blocked. `openNotice()` does neither.

Ordinary http(s) sign-in is unaffected.
2026-10-01 11:09:04 -04:00
Juan Fernando Castro c542e4675b Merge pull request #4000 from HeyPuter/juancastro/put-2036-shared-file-issue
fix: let a share recipient create entries in a folder shared with them
2026-10-01 09:56:09 -04:00
Daniel SalazarandReynaldi Chernando d3b46a6b86 docs: easy kv recipes (PUT-1962) (#3999)
* docs: easy kv recipes (PUT-1962)

* docs: publish the append-to-list kv recipe (PUT-1962)

* docs: bare-object add rejects with invalid_path (PUT-1962)

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 18:03:07 +07:00
Juan Fernando Castro fd8b8288d3 Merge pull request #3988 from HeyPuter/juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-30 20:28:02 -04:00
Daniel Salazar fc62ea2ee1 Revert "variable color for text in input fields (#3996)" (#4001)
This reverts commit 8862420594.
2026-09-30 16:12:19 -07:00
Juan Castro d1b11c793c Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a
One conflict, in workerInvoker.integration.test.ts: #3997 fixed the same
gap-marker flake this branch did, with a different barrier. Took theirs
— waiting on the `ev:qf` counter, which `#handlerFailed` writes strictly
after `discard`, so the marker is in and the lease cleared by the time it
reads 1. Their ordering also asserts the counter before flipping `answer`
back to 200, which closes the redelivery-clears-failures hole this
branch's version had removed the line to avoid.

Converted the two `vi.waitFor` calls #3997 added to `waitUntil`, the
helper this branch introduced for the rest of the file. Neither is wrong
today, since the new block fakes no timers, but a `jump()` added near
them later would hit exactly the drained-backoff trap the helper exists
for.
2026-09-30 17:49:36 -04:00
Juan Castro d6b7c8ff52 docs: drop the replica-lag testing runbook from the open-source tree
It describes our local two-server MySQL replication setup, which is
internal tooling rather than anything a self-hoster or contributor
needs. It now lives in the heyputer repo alongside the other internal
docs.

The suite it documented stays: the header comment now states the
env-var gate directly instead of pointing at a file this repo no
longer carries.
2026-09-30 17:42:04 -04:00
Daniel Salazar 4448103ef0 fix: misc events and kv issues found during recipes (#3997)
* fix: misc events and kv issues found during recipes

* test: cover handler depth on narrowed actors and created notifications

* test: fix stale legacy batch directory upload test and seat user-agent flake

* test: wait for the refusal to be recorded before claiming its gap marker
2026-09-30 14:40:47 -07:00
Juan Castro 79e496e79d test: wait on the gap marker rather than a depth that never moves
The refused-delivery test waited for `depth(subId)` to reach 1 before
claiming the marker, but `discard` removes the entry and appends the
marker in its place, so the depth is 1 before, during and after. The
wait returned at once, and the claim came back `inflight` — the refused
delivery still held the lease it was invoked on — leaving the assertion
reading `undefined`. It only passed on the slack in `invoked`'s poll
interval, which a loaded runner takes away.

Waiting on the claim itself is the barrier the depth was standing in
for. Dropping `answer = 200` with it: `beforeEach` already resets it,
and a redelivery settling before the last assertion would clear the
failure count it reads.
2026-09-30 16:36:57 -04:00
Juan Castro 2e0bf26cac Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-30 16:31:42 -04:00
Juan Castro b126cf9c5a fix: carry the owner's path into WebDAV events and expand tilde before touch
WebDAV emits the same GUI events the FS controllers do, and had the same
confusion: the payload was masked for the actor and addressed to the owner. A
recipient mounting a shared folder in Finder reproduced the disappearing-item
symptom exactly.

/touch prefixed `/` before expanding, so `~/AppData/<app>/f.txt` became a literal
`/~/...` that no longer looked like a tilde path — the shape puter.js sends for
every app-relative write answered 404.
2026-09-30 16:21:39 -04:00
Juan Castro ec4d3b31cf fix: let a share recipient create entries in a folder shared with them
A recipient addresses the owner's entries by a masked `/<owner>/<uuid>/<name>`
path. Routes that resolve an existing row unmask it on the way in, but the ones
that create a new entry had nothing to resolve and ran the ACL check against the
mask itself, which names no row — so mkdir, touch and the batch write/mkdir/
shortcut ops answered 404 `subject_does_not_exist` inside any shared folder.
They now expand client paths the same way every other legacy route does.

The GUI events carried the same confusion the other way: the payload was masked
for whoever made the request but addressed to the entry's owner, who cannot
resolve another user's mask. The owner's client dropped the row by uid and then
failed to re-add it under a path it had never heard of, so anything a recipient
touched vanished from the owner's open window until a refresh. Those events now
publish the owner's real path; responses to the actor stay masked.

Also fixes the desktop's `item.moved` handler reading `metadata` as an object
when the wire carries a JSON string: the name of a trashed item came out empty
(trashing renames the entry to its uid), and re-encoding the string left the
restore path parsing a string instead of an object. The parse the dashboard and
the explorer already open-coded is now one helper.
2026-09-30 15:28:02 -04:00
hairyEagle 8862420594 variable color for text in input fields (#3996) 2026-09-30 11:41:32 -07:00
Neal ShahandReynaldi Chernando 3b4f3f6d06 PUT-1959 PUT-1960 subdomains & worker recipes (#3993)
* PUT-1960 worker recipes

* subdomain recipes

* fixes

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 01:33:29 +07:00
Juan Fernando CastroandReynaldi Chernando 3d1357455f docs: teams recipe and playground examples (PUT-1966) (#3967)
* docs: add the easy teams recipe and playground examples

Covers the read-only surface an app actually has: list(), listMembers()
and listDirectory(). Adds the teams recipe tag the build validates
against, and a Teams section to the playground.

Each example handles the two states that are easy to confuse: teams
turned off on the deployment (not_found) and a team that has not opened
its directory to apps (team_not_found).

* docs: open the teams recipe with what it lets an app build

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 00:11:42 +07:00
Juan Castro f8d8003fea test: stop the delivery-backoff test reading holds against a moving clock
`holds one it could not answer, for longer each time` freezes `Date` so a
hold written as `now + backoff` reads back as exactly `backoff`. But every
wait in the file went through `vi.waitFor`, which advances the faked clock
once per poll — measured at 50ms a poll. Each poll between the hold being
written and read therefore ate into the value: CI read a 2s hold as 525ms
and failed, on main one run and on this branch the next.

Polling now goes through `waitUntil`, which retries on `performance.now`
and real `setTimeout` — neither faked — so the clock only moves where
`jump` moves it. The holds then read exactly 2s/4s/8s/16s, so the
assertion is an equality rather than a one-second window that a slow
enough runner could always slide out of.
2026-09-30 12:25:40 -04:00
Juan Castro 23c8f73bcb test: verify the replica-lag fix against real MySQL replication
The unit tests inject the driver error because sqlite has neither a read
replica nor foreign-key enforcement, so neither half of this bug can
occur locally — they prove the handling, not that the situation arises.

This suite runs two MySQL servers in real replication and freezes the
follower with STOP REPLICA, so the row really is gone on the primary and
really is still served by the replica, and MySQL raises the foreign-key
errors itself. Against the pre-fix code all four cases fail, including
the unhandled "Cannot add or update a child row" the reported 500s were.

Opt-in behind PUTER_TEST_REPLICA_LAG, matching the env-gated provider
integration tests and their skipUnlessEnv helper, so CI and anyone
without the containers skip it. It builds and drops its own database
rather than touching local dev data.
2026-09-30 10:02:36 -04:00
Daniel Salazar b9efac2363 fix: misc fixes from alert logs (#3995) 2026-09-30 00:51:27 -07:00
Daniel Salazar ad12da2942 feat: add gpt 6.1 sol to our direct apis (#3994) 2026-09-29 17:38:40 -07:00
Neal Shah 9aa3989e9d fix PUT-1888 (#3992) 2026-09-29 17:06:28 -04:00
dependabot[bot] cd37cf147f chore(deps): bump socket.io-client from 4.8.3 to 4.8.4 (#3982)
Bumps [socket.io-client](https://github.com/socketio/socket.io) from 4.8.3 to 4.8.4.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@4.8.4)

---
updated-dependencies:
- dependency-name: socket.io-client
  dependency-version: 4.8.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 12:56:15 -07:00
Filip Kujundžić f60d94ef3f Merge pull request #3987 from HeyPuter/fix/put-1874-chat-test-mode
Fix chat test mode routing to fake provider
2026-09-29 19:19:44 +02:00
Juan Castro 8bce62e880 Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-29 13:13:01 -04:00
Juan Castro abd27c0192 fix: replicate tombstones, and attribute foreign-key failures correctly
Review of the first pass turned up two holes that made it ineffective.

CacheReplicationService always deleted the keys a peer broadcast and
never adopted a payload, so tombstones stopped at the region boundary:
peers dropped the marker, kept caching deleted rows off their own
replicas, and the multi-region case the fix was for stayed broken.
Tombstones are now the one payload a peer adopts — they carry no row,
only the fact of a delete.

user_to_app_permissions keys both the app and the user, and the drivers
don't all name the constraint, so a deleted *account* was being read as a
deleted app: a live app got tombstoned, the request 404'd for the wrong
reason, and the dead account was never retired, so the client retried and
re-tombstoned indefinitely. The insert site now asks the primary which
parent actually went away and answers 404 or 401 to match.

Also: re-check the tombstone after a cache write lands, since a row
cached under a live tombstone is never looked at again; extend the guard
to #refreshCache, which update() could otherwise use to undo a delete;
stop markDeleted tombstoning an address the deleted row never owned,
which was blocking the account that did own it; let the primary rather
than the tombstone decide in batched lookups; tombstone the id key when
no cached row names the others; and correct the leaf-store comment in
stores/index.ts now that SessionStore depends on UserStore.
2026-09-29 13:12:31 -04:00
dependabot[bot] 6a31b2af9e chore(deps): bump libphonenumber-js from 1.13.13 to 1.13.14 (#3984)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.13 to 1.13.14.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.13.13...v1.13.14)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 10:11:03 -07:00
dependabot[bot] 3ebb753eb1 chore(deps): bump socket.io from 4.8.3 to 4.8.4 (#3985)
Bumps [socket.io](https://github.com/socketio/socket.io) from 4.8.3 to 4.8.4.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@4.8.4)

---
updated-dependencies:
- dependency-name: socket.io
  dependency-version: 4.8.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 10:08:32 -07:00
Juan Fernando CastroandReynaldi Chernando ee4d55027d docs: easy perms recipe (PUT-1965) (#3972)
* docs: add the easy perms recipe

Covers the supported surface: request() and check(). Adds the perms
recipe tag the build validates against.

* docs: add a playground example per operation in the perms recipe

One runnable example for each section: request a folder, request the
email address, request apps or subdomains, and check before prompting.

* docs: open the perms recipe with what it lets an app build

* docs: ground the perms examples in the UI that would ask

A photo editor exporting to Pictures, a checkout emailing a receipt, a
launcher listing apps, and a settings toggle that reads its own state
without raising a prompt.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-09-30 00:00:34 +07:00
dependabot[bot] 7eb9ace733 chore(deps-dev): bump prettier from 3.9.8 to 3.9.9 (#3986)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.8 to 3.9.9.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.8...3.9.9)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 09:51:06 -07:00
dependabot[bot] 14ae6a69ee chore(deps): bump postal-mime from 3.0.0 to 3.0.1 (#3983)
Bumps [postal-mime](https://github.com/postalsys/postal-mime) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/postalsys/postal-mime/releases)
- [Changelog](https://github.com/postalsys/postal-mime/blob/master/CHANGELOG.md)
- [Commits](https://github.com/postalsys/postal-mime/compare/v3.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: postal-mime
  dependency-version: 3.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 09:50:45 -07:00
Juan Fernando CastroandReynaldi Chernando 3b00cd2d14 docs: easy fs sharing recipe (PUT-1958) (#3973)
* docs: add the easy fs sharing recipe

share(), listShared(), getShares() and unshare() — the fs ground the
existing recipes do not already cover.

* docs: add a playground example per operation in the fs sharing recipe

One runnable example for each section of the recipe: share, mode,
listShared, getShares and unshare.

* docs: open the sharing recipe with what it lets an app build

* docs: ground the sharing examples in a document-editor flow

Each example now sits in the UI that would call it — a Share button, a
role picker, a shared-with-me inbox, an access list, ending a review.

* remove redundant examples and refine recipe

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-09-29 23:47:23 +07:00
Juan Castro 785304e807 fix: stop deleted apps and users from being resurrected by a lagging replica
A delete lands on the primary and invalidates the cache, but a concurrent
request that misses the cache reads a replica that hasn't applied it yet,
finds the row, and writes it back under a fresh TTL — 24h for apps, 15m
for users. Every write referencing that resurrected row then fails its
foreign key and escapes as a raw driver error, so the route 500s until
the TTL expires. Origin-bootstrapped app uids are deterministic, so the
same origin keeps hitting the stale row and never re-bootstraps.

Deletes now tombstone their cache keys before dropping them. While a
tombstone is live the read path goes to the primary and the write path
refuses to cache; re-creating a row clears it. As a backstop, a write
rejected for a missing parent is answered as one: 404 for a vanished app,
401 for a vanished account, with the stale entry retired either way.

Also fixes cascadeDelete's identity lookup, which selected a column some
schemas lack and so threw into its catch, skipping the invalidation
entirely; and drops SessionStore's private copy of isUniqueViolation in
favour of the shared helper.
2026-09-29 10:16:48 -04:00
Juan Fernando Castro 8e31d9149a Merge pull request #3966 from HeyPuter/juancastro/put-1994-group-scan-cycle
fix: stop group permission scans re-entering the sharer's own grant (PUT-1994)
2026-09-29 10:12:54 -04:00
Juan Castro 9539c7b553 test: cover team-share access through the real authorization path
The existing team-share suite resolves shares through listings and store
reads, neither of which enters the permission scan, so a scan that never
terminates leaves it green. These ask the question the product asks —
can this member open the file — via ACLService.check.
2026-09-29 09:41:05 -04:00
Juan Fernando Castro 27c4873042 Merge pull request #3971 from HeyPuter/juancastro/put-2033-team-create-read-after-write
fix: read a team back from the primary after writing it (PUT-2033)
2026-09-29 09:06:43 -04:00
404oops 0b1a4283f5 fix(ai): honor chat test mode 2026-09-29 12:17:33 +02:00
Daniel Salazar 1cc091206c fix: PUT-1885 PUT 1950 (#3981)
* fix: app uuid fixes

* fix: bound kv update/add createPaths writes (PUT-1885)

* fix: upload progress socket events on the v2 tracker (PUT-1950)
2026-09-28 23:38:34 -07:00
Daniel Salazar 09887747e0 fix: app uuid fixes (#3980) 2026-09-28 19:43:23 -07:00
Daniel Salazar f80d4ae136 fix: events docs (#3968)
* fix: events docs

* fix: docs
2026-09-28 17:24:37 -07:00
Filip KujundžićandClaude Opus 5.5 42e0c28d0a feat(ai): add Claude Sonnet 5.5 (#3970)
Add claude-sonnet-5-5 ($2/$10 per MTok, 1M context, 128K output) and move
the bare claude-sonnet / claude-sonnet-latest aliases to it. Sonnet 5.5
rejects non-default sampling params and disabled thinking, so it joins the
temperature-omit, effort, and adaptive-thinking gates.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-09-28 15:29:21 -07:00
Juan Castro d888377138 Merge remote-tracking branch 'origin/main' into juancastro/put-2033-team-create-read-after-write 2026-09-28 17:20:43 -04:00
Juan Castro 8a53d14540 fix: read a team back from the primary after writing it (PUT-2033)
create() inserted on the primary then read the row back through the
replica, so a replica that had not caught up answered the same as a team
that was not there and the call threw. Nothing rolled the insert back,
and the owner membership was never written, leaving a team its owner
could not see but which still counted against their cap.

Read back through pread in create() and update(), seed the row cache
with the result, and delete the row if the read-back still finds
nothing. #cached no longer stores a null, which would otherwise hold a
lagging replica's answer for the full TTL.
2026-09-28 16:55:19 -04:00
Juan Fernando Castro 12e6bc3d71 Merge pull request #3950 from HeyPuter/juancastro/put-1988-team-force-2fa
feat: let a team require two-factor authentication of its accounts (PUT-1988)
2026-09-28 16:52:34 -04:00
Filip Kujundžić 359d68e51f Merge pull request #3949 from HeyPuter/fix/ai-ocr-model-sync-and-normalization 2026-09-28 21:23:28 +02:00
Filip Kujundžić 020e61065f Merge pull request #3947 from AnujShrivastava01/anuj/fix/remove-deprecated-ai-chat-models 2026-09-28 21:22:33 +02:00
Juan Castro 480d9d032b Merge remote-tracking branch 'origin/main' into juancastro/put-1988-team-force-2fa
# Conflicts:
#	src/backend/clients/database/SqliteDatabaseClient.test.ts
#	src/backend/clients/database/SqliteDatabaseClient.ts
#	src/backend/clients/database/migrations/mysql/mysql_mig_42.sql
#	src/backend/clients/database/migrations/postgres/postgres_mig_31.sql
2026-09-28 10:24:39 -04:00
Juan Castro 528eb192b5 fix: stop group permission scans re-entering the sharer's own grant
The group path recursed without the anti-cycle state, so a grant issued
into a group the issuer belongs to resolved back into itself and the
scan did not terminate. Thread the scan state through the group scanner
and skip an issuer already on the path, matching the user-to-user path.

Readings produced under a non-empty path are pruned for that path only,
so they are no longer written to or served from the shared scan cache.
2026-09-28 10:05:54 -04:00
Rupanjana ChoudhuryandDaniel Salazar ca52b5236c Fix/3638 shared link account (#3845)
* fix: open shared links with the intended account

* refactor: validate shared account hints in the GUI

* fix: scope the shared-link account hint to plain top-level visits

The hint no longer switches accounts in popups, embeds or action flows, and
only counts next to a share path the GUI would open. It is dropped from the
URL before any switch or prompt, so a reload never asks again. An unsaved
recipient gets a dismissible choice (sign in, or continue as the current
account) instead of a picker with no way out.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-27 18:25:28 -07:00
Neal Shah 153f581011 fix PUT-1872 (#3952) 2026-09-25 22:34:53 -04:00
Neal Shah 1fc2eae1ae self hosted mail (#3932)
* self hosted mail

* update lock

* address daniel review
2026-09-25 21:48:15 -04:00
Daniel Salazar dcb61bc4fd fix: PUT-1824 + PUT-1884 + PUT-1952 + PUT-1827 (#3951) 2026-09-25 17:47:03 -07:00
Juan Fernando Castro b445680f10 Merge pull request #3937 from HeyPuter/juancastro/put-1879-oidc-popup-return-proof-binding
fix(auth): bind the OIDC popup-return proof to its purpose, browser, and popup
2026-09-25 18:04:59 -04:00
Juan Castro 95ee796157 Merge branch 'main' into juancastro/put-1879-oidc-popup-return-proof-binding 2026-09-25 17:22:12 -04:00
Nariman JelvehandDaniel Salazar 3d3678a555 feat: let the Contact Us form carry screenshots and recordings (#3563)
* feat: let the Contact Us form carry screenshots and recordings

The form doubles as our bug-report channel, and the bugs worth reporting
are often the ones that need a picture: a visual glitch, or something that
takes five steps to reach. Up to 5 images or videos may now ride along
with the message, delivered as attachments on the support email.

Nothing the client says about a file is believed. The endpoint takes bare
base64 and re-derives all three of the things that matter from the decoded
bytes:

- Type, sniffed from magic numbers and matched against an allow-list of
  PNG/JPEG/GIF/WebP and MP4/QuickTime/WebM. A declared MIME is never read,
  so it cannot smuggle anything past the list. SVG is excluded on purpose
  (it carries script, and support tooling renders what it is sent), as are
  the non-video brands that share MP4's `ftyp` box -- HEIC, M4A, JPEG 2000.
- The file name, reduced to a display label with the extension taken from
  the sniffed type. PNG bytes named `payload.html` arrive as
  `payload.png`; a name can never carry the CR/LF that would break out of
  a Content-Disposition header, nor the bidi overrides that make
  `report<RLO>gnp.exe` render as `report.exe.mp4`.
- Size: 10 MB per file, 15 MB per submission, counted on decoded bytes.
  Encoded length is capped before decoding, so an oversized payload costs
  a length check rather than a 10 MB allocation. The total stays well
  under the 25 MB most providers enforce, since the outgoing mail base64s
  these again.

Base64 is decoded strictly, reusing the round-tripping decoder that
already guards app icons -- `Buffer.from(s, 'base64')` silently drops
characters it does not recognise, and the round-trip is what rejects
bytes smuggled after the payload. That decoder and the image sniffer move
from appIcon.ts to a new mediaSniff.ts, which gains the video counterpart.

One request is now worth megabytes of parsing and outbound mail, so the
existing per-user rate limit gains a per-IP backstop (which the per-user
counter cannot see through freshly minted accounts), a concurrency cap,
and a Content-Length gate that refuses an impossible body before anything
decodes it.

Payloads are not stored. They ride the email; the new
`feedback.attachments` column records names, types and sizes only, so an
abusive submission stays attributable once the mail has been dealt with.

Also fixes the form posting an empty message when Send was pressed with
nothing typed, and surfaces submit failures instead of leaving the button
disabled with no explanation.

* fix: stream Contact Us attachments as multipart instead of base64 JSON

Base64 in a JSON body went through the global JSON parser before the route
ran: raw buffer, rawBody copy, decoded string, parsed strings, then decoded
Buffers plus a re-encode for the strict check. A max-size submission peaked
around 110-135 MB of heap for 15 MB of files, and the route's Content-Length
check ran after all of it.

Attachments now arrive as multipart/form-data, which the global parser skips.
Auth, rate limit, concurrency and the Content-Length 413 all run before the
body is read, and busboy enforces the count, per-file and total caps while
streaming, so only the decoded file bytes are held (~16 MB peak). On a broken
limit the reader stops and the response closes the connection.

JSON stays supported for message-only posts; a JSON `attachments` field is
rejected.

* fix: deliver Contact Us 413s instead of resetting the upload

Closing the socket on a mid-stream limit sent the 413 and then reset the
connection with the client still uploading, so the client could see a
reset instead of the error. After a limit trips, the rest of the body is
now read and discarded, bounded by the body budget; past the budget the
request is destroyed.

Adds an HTTP-level test through the full middleware stack (FormData
upload, unauthenticated refusal, 413 on declared length before the body
is sent, 413 on a chunked upload over the per-file cap), trims comments
to the AGENTS.md length rule, and drops a box-drawing test divider.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-25 14:19:35 -07:00
hairyEagle a9ee3d1ea6 fix(selfhost): configure bucket CORS in s3-init so browser uploads work (#3934)
Browser uploads (Dev Center deploy, puter.fs.upload) PUT file bytes
directly to presigned URLs on the public S3 endpoint. That cross-origin
PUT is preflighted, and the bucket had no CORS rules, so RustFS answered
without Access-Control-Allow-Origin and the browser blocked the upload.

s3-init now runs put-bucket-cors on every boot after ensuring the bucket
exists (idempotent). Origins are open because the presigned URL is the
credential and the SDK sends no cookies. Document how to re-run and
verify in doc/self-hosting.md.
2026-09-25 12:12:02 -07:00
Juan Castro ee3f928b0e fix: annotate the seat-lookup catch so the baselined typecheck passes 2026-09-25 14:39:23 -04:00
Juan Castro 176c8848f5 fix: let a seat enrol in 2FA from the console, and hold it once required
The Security tab only rendered the 2FA card for accounts with a confirmed
email, so a seat — provisioned without one by design — had no way to
enrol from the UI at all. It was the GUI twin of the enable-route check
fixed earlier: the rule could be required of accounts that could not
satisfy it. The card now shows for a seat too, giving it the ordinary
authenticator flow: QR, manual secret, six-digit code, recovery codes.

Disabling is refused while the seat's team requires 2FA. Without that a
member could switch off the factor their team mandates and lock
themselves out of everything until they enrolled again.
2026-09-25 14:31:58 -04:00
Juan Castro beaf4a9a7c docs: record the team 2FA rule where the other member-facing gates are
No method pages: requiring 2FA and resetting a member's factor are team
administration, which the docs deliberately do not publish — only
`list`, `listMembers` and `listDirectory` are app-callable.

But an app acting for a gated seat does meet `two_factor_required`, and
an undocumented 403 leaves it nothing to go on. It goes beside
`password_change_required` in rate-limits-and-quotas, which already
explains what a team's administration does to a member's requests.
2026-09-25 14:18:30 -04:00
404oopsandClaude Opus 5.5 d36da7522b fix(ai-ocr): address review: playground examples, Textract options, inline size
Adds the Mistral and document-annotation img2txt examples to the
playground, states that Textract takes no options, and drops the
normalize mentions from the img2txt docs and types.

The SDK now checks Mistral inline sources against the size a base64
upload can reach inside the 50 MB JSON body (36 MB) instead of the
10 MB Textract limit; 50 MB let 38-50 MB inputs through to a body-parser
413. Retired-model messages no longer cite vendor dates.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 18:56:29 +02:00
Daniel Salazar e6cfe8c492 fix: codeowners (#3948) 2026-09-25 09:23:26 -07:00
404oopsandClaude Opus 5.5 eef8258539 fix(ai-ocr): sync OCR models and prices, route by model, cap input size
Model sync against Mistral's listing, docs and a live call per id:
mistral-ocr-latest now serves OCR 4.1, so the catalog pins it (with
mistral-ocr-4 and the retired-but-redirected 2503) to mistral-ocr-4-1,
adds mistral-ocr-4-0 and OCR 3 (mistral-ocr-2512 / -3 / -3-0), and
rejects the retired mistral-ocr-2505 before any upstream call. Mistral
usage is metered per model at vendor rates: $4/1000 pages for OCR 4.x
with $5 annotations, $2 and $3 for OCR 3, replacing the flat $1 and $3.

Textract now calls DetectDocumentText instead of AnalyzeDocument with
LAYOUT. The layout feature cost $4/1000 pages while we billed the
$1.50 detect rate, and it only added text-less LAYOUT_* blocks that put
blank lines into img2txt output. Multi-page PDFs and unsupported
formats become a 400 that names Mistral as the alternative.

Request shape: a model name alone now selects its provider, so calls
differ only by model; an explicit provider still picks its default and
must agree with the model. Mistral options gain tableFormat,
extractHeader, extractFooter and documentAnnotationPrompt; annotation
formats accept the REST (json_schema.schema) or SDK
(jsonSchema.schemaDefinition) spelling and are validated before
billing. Non-image documents such as DOCX go as document_url.

Response shape: both providers return model, text and blocks with a
0-based page; Mistral also returns document_annotation, which was
billed but dropped. img2txt resolves to that annotation when
documentAnnotationFormat is set and to the text otherwise; the return
is a string for every model, so normalize has no effect on it.

Input size: OCR input is capped at the provider limit (10 MB Textract,
50 MB Mistral), and web inputs in loadFileInput are streamed under the
cap instead of buffered whole, which also bounds speech2txt URLs.

Docs: img2txt models, options, input limits, return value and errors;
OCR limits in the quotas page.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 18:19:54 +02:00
Juan Castro 3105378fdc fix: let a team seat enrol in 2FA, and fit the rule's note on its card
Found by driving the flow against a live instance: a provisioned seat
could reach `configure-2fa/setup` but never `enable`, which demands a
confirmed email. Seats are provisioned without one on purpose (PUT-1792,
the team being the trust anchor), so requiring 2FA of a team would have
locked out precisely the accounts the rule is for — permanently, since a
seat cannot enrol and the rule never lifts.

The check now passes for an account a team provisioned; every other
account still needs its address confirmed, with tests either way.

The card's note was also a line too long and rendered clipped, so it says
the same thing in the space the layout gives it.
2026-09-25 11:58:32 -04:00
Juan Castro 695a26b272 feat: surface the team 2FA rule in the console and SDK (PUT-1988)
An owner toggle beside the directory one, confirmed on the way on since
it locks out every account without 2FA until they enrol, and unconfirmed
on the way off since that only lifts a requirement.

A per-member Reset 2FA action, shown only while the rule is on — with it
off a member can clear their own factor and needs nobody's help. The
confirmation says what it does and what it leaves alone: signed out
everywhere, told about it, password unchanged.

`require2fa` rides the Team shape and `update()`, and `resetTwoFactor()`
joins the module beside `resetPassword()`, whose docblock already says
2FA is left alone — the two now read as the pair they are.
2026-09-25 11:28:59 -04:00
Juan Castro cab70254b1 feat: let a team owner reset a seat's second factor (PUT-1988)
Forcing 2FA makes lockout materially likelier, and nothing could clear a
factor except the account holding it — `resetMemberPassword` leaves 2FA
alone on purpose, and admin only displays it. A seat that lost both its
authenticator and its recovery codes had no way back.

Its own route and its own audit action rather than folded into the
password reset, so that method keeps the property its comment claims: a
password reset alone is not takeover. The two together are, and for an
account the team created and pays for that is the team's to do — but it
has to be chosen and recorded, not arrived at sideways.

Bounded to `org_owned = 1`: a joined member's own login is not a team's
to reach into. The member is told, and signed out everywhere.
2026-09-25 11:15:23 -04:00
Juan Castro db1a95260a feat: let a team require 2FA of the accounts it provisioned (PUT-1988)
`group.require_2fa`, toggled by the owner through `PUT /teams/:uid` and
recorded in the audit log like `directory_enabled` — it changes what
every seat must hold to sign in, so it is not something a team should be
able to alter silently.

Enforced by deriving, not by stamping a flag on the user: a stamp left
behind by a member leaving, or by the rule being turned off, would lock
someone out of their own account with no team left to clear it. The seat
row already joins `group` and is already read per request, so it carries
the flag and the gate costs nothing extra. Toggling busts those rows, or
turning the rule off would leave members locked out until the TTL lapsed.

The gate rides the same `allowUnconfirmed` opt-out as the other pending-
requirement gates, so the routes a gated member must still reach stay
reachable — `configure-2fa` among them, which would otherwise be walled
off behind the very requirement it clears.

Scoped to `org_owned = 1`: the owner is not a seat of their own team, and
a joined member is an ordinary self-paying account whose personal login
is not a team's to dictate. The owner is held to the rule at the point it
matters — they cannot turn it on without 2FA of their own.
2026-09-25 10:35:24 -04:00
Anuj Shrivastava e041fb6b9b Remove deprecated AI chat models flagged by live model check
OpenAI's o3-pro, Mistral's mistral-large-2512 and zai-glm-5-2, and
DeepSeek's deepseek-v4-flash / deepseek-v4-flash-vision-exp are no
longer served by their upstream APIs, so drop them from the hardcoded
provider model lists (closes HeyPuter/puter#3945).

DeepSeek's legacy chat/reasoner aliases and default model previously
pointed at deepseek-v4-flash; they now resolve to deepseek-v4-pro, the
remaining model in that catalog, so existing callers keep working.

Signed-off-by: Anuj Shrivastava <[email protected]>
Signed-off-by: Anuj Shrivastava <[email protected]>
2026-09-25 19:40:02 +05:30
Nariman Jelveh 5721bdeb1d Add link to AI Builder in README (#3946) 2026-09-24 21:00:04 -07:00
Daniel Salazar 132cae9f6a Bug bash fixes (PUT-1876, 1877, 1880, 1881, 1948, 1949, 1951) and KV event fan-out by plan (#3943)
* fix(kv): keep disableSharing entries out of other apps' events (PUT-1876)

Private kv keys and values are no longer delivered to cross-app subscriptions, including forwarded and durable deliveries.

* fix(fs): refuse usernames whose home path still holds another account's rows (PUT-1877)

Username claims (signup, save_account, change-username, OIDC, team provisioning) now also check for leftover rows under /<name>/, and the root listing only returns the actor's own home.

* fix(puter-js): reconnect the events channel after a server-side disconnect (PUT-1881)

A server hang-up is retried with a bounded backoff; persistent handlers survive it and get an optional onError when the connection can't be restored.

* fix(backend): let the graceful shutdown finish before telemetry exits (PUT-1948)

The telemetry preload no longer exits on signals; the entry point drains, tears down top-down, flushes telemetry last, then exits. DB pools stay open until teardown.

* fix(fs): reserve quota for signed uploads in progress (PUT-1880)

startWrite/startBatchWrite hold each upload's declared size in a per-owner cache reservation until it completes, aborts or expires, so a burst of starts can't all read the same committed usage. Caps in-flight uploads at 10,000 per account.

* fix(fs): refuse to complete a signed upload whose object never arrived (PUT-1951)

Completion now 400s when the object store reports the key missing (transient errors stay lenient), abort no longer deletes an object a live entry still uses, and a ghost entry is only removed after a second miss at its recorded location.

* fix(metering): move usage to v2 keys with a small totals item, per-model detail shards and a 3-month TTL (PUT-1949)

The billed item holds only totals; per-model detail lives in 100 hash-sharded items read in one batch and cached for 60s, and more than 5,000 distinct usage types in a month fold into other. Global and per-app aggregates store totals only, appTotals comes from a prefix listing, a refused path only skips the key that refused it, exact reads near the allowance are throttled per key, and every metering record expires after 3 months. September usage restarts at deploy; the monthly-charge claim stays on the v1 key through September so recurring charges don't fire twice.

* test(events): let the worker backoff test tolerate a loaded run

* feat(events): scale KV event fan-out with the key owner's plan

A KV change delivers to up to 512 matching subscriptions (2,048 filter evaluations) for a paid key owner and 128 (512) otherwise, inline values are dropped once more than 128 match, and paid accounts may hold up to 512 share handles per app. The owner's plan is only looked up when a change has more candidates than any cap.

* test(metering): seed the 150-app listing test directly so it fits under coverage
2026-09-24 16:00:52 -07:00
Neal Shah 4a9e5a2440 Ns/worker no socket (#3944)
* remove socket creation from puter worker startup

* Remove really extra comment
2026-09-24 18:55:10 -04:00
Juan Fernando Castro 600064dcee Merge pull request #3942 from HeyPuter/juancastro/put-1944-teams-hardening
fix: Teams lifecycle robustness — owner deletion, cap serialization, provisioned-account scope (PUT-1944)

fixes requested by Daniel pushed
2026-09-24 16:51:43 -04:00
Juan Castro 66ee50317c style: restore prettier formatting displaced by the main merge 2026-09-24 16:35:09 -04:00
Juan Castro 8f0a317a97 Merge remote-tracking branch 'origin/juancastro/put-1944-teams-hardening' into juancastro/put-1944-teams-hardening 2026-09-24 16:33:40 -04:00
Juan Castro 8c816060a3 fix: answer a cap-lock failure as 409, like contention, per review 2026-09-24 16:33:12 -04:00
Juan Castro 0131ebfe9b Merge branch 'main' into juancastro/put-1944-teams-hardening 2026-09-24 12:20:12 -04:00
Juan Castro ca48477db9 Merge branch 'main' into juancastro/put-1879-oidc-popup-return-proof-binding 2026-09-24 12:19:59 -04:00
Juan Fernando Castro 2cb7516141 Merge pull request #3936 from HeyPuter/juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and
fix(hosting): keep alternate-host bootstrap stubs from shadowing private apps
2026-09-24 12:19:31 -04:00
Juan Castro 0c5c1be777 fix: Teams lifecycle robustness ahead of wider enablement
Closing an account that owns a team left `owner_user_id` NULL while
`deleted_at` stayed NULL, so the team remained live while every team
route resolves by membership — which the deletion cascades away. The team
could then not be administered by anyone, and the accounts it provisioned
could not be closed. Declined now in both directions: live teams via
`countOwned`, and soft-deleted ones still holding accounts via a new
store read, since a deleted team's accounts are suspended rather than
removed.

The cap lock gave up after 200ms and proceeded unserialized, so
overlapping provisioning could each read the seat count before any insert
committed. It declines instead; a single create never contends. An
unreachable lock backend is a 503 rather than a silent pass.

An account a team provisioned can no longer create a team of its own
(PUT-1890) — that is not the intended lifecycle, since such accounts do
not go through the normal sign-up path. Keyed on `org_owned = 1`, so the
owner and joined members are unaffected, and not via `teamsAvailableTo`,
which is permissive by design.

The directory declines scoped access tokens, as `listMembers` already
bounds non-account callers.
2026-09-24 11:06:00 -04:00
Juan Castro 8ecaa77a0e Merge remote-tracking branch 'origin/main' into juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and
# Conflicts:
#	src/backend/controllers/auth/AuthController.test.ts
2026-09-24 10:04:00 -04:00
Neal Shah 07a11240ea PUT-1868 (#3940) 2026-09-24 00:30:47 -04:00
Daniel Salazar 7758aea78c fix: acl check (#3939) 2026-09-23 19:03:16 -07:00
Rasad RegmiandDaniel Salazar 83de67cca9 fix(tools): spawn npm.cmd through a shell on Windows (#3923)
* fix(tools): spawn npm.cmd through a shell on Windows

* fix(tools): join the npm command line on Windows to avoid DEP0190

Passing args alongside shell: true warns on Node 24. Test the spawn
tuple as data and run a real npm spawn instead of mocking child_process.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-23 18:54:43 -07:00
Echa ApriliyantoandDaniel Salazar 9f69483593 feat: allow browser extension origins in auth requests (#3907)
* feat: allow browser extension origins in auth requests

Add chrome-extension://, moz-extension://, safari-extension://, safari-web-extension://, and extension:// to the protocol allow-list so browser extensions can obtain app tokens via /auth/get-user-app-token.

Extract WEB_AND_EXTENSION_PROTOCOLS constant in validation.js so the allow-list is defined once and shared by AuthService, AppStore, and AppDriver.

* fix: harden the extension-origin allow-list

Review follow-ups on the extension-origin change:

- Require a host in `validateUrl`. Only "special" schemes need an
  authority, so `chrome-extension:` parsed with an empty hostname and
  slipped past the reserved-system-host guard in AppDriver.
- Lowercase the host in `AuthService#normalizedOrigin`. `new URL()`
  lowercases http(s) hosts but leaves opaque ones alone, so one
  extension in two spellings resolved to two app uids — two AppData
  trees, two permission sets — and missed the origin blocklist.
- Drop `extension:`. No browser emits it, and it accepted
  `extension://evil.com` as an app origin.
- Freeze the allow-list and derive `validateUrl`'s http(s) default from
  `WEB_PROTOCOLS` so the two spellings can't drift apart.
- Pin the tests to the real uid derivation, use unique extension ids so
  a row left by another test can't mask the bootstrap path, and cover
  the host-less and near-miss schemes.
- Fix the prettier/eslint failure in AppStore.js.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-23 18:24:31 -07:00
Daniel Salazar 606f5a27a3 chore: CODEOWNERS (#3938) 2026-09-23 18:24:09 -07:00
Daniel Salazar 58ed2f485f feat(auth): signup bonus codes
Signup links can carry a bonusCode. Core checks its shape and previews it through `puter.signup-bonus.check`, refuses a dead code before the abuse hook records the attempt, then hands a live one to `puter.signup-bonus.validate` after `puter.signup.validate`, which may raise the phone/card gates; the accepted code rides on `puter.signup.success`, and OIDC carries it in the signed state. `user.card-verified` is now awaited like `user.phone-verified`. The signup form shows the offer and sends the code.
2026-09-23 14:47:34 -07:00
Juan Castro d73933b8a5 fix(auth): bind the OIDC popup-return proof to its purpose, browser, and popup
Tokens under the `oidc-state` scope share one signing key, so the payload is
what says which job a token belongs to. The popup-return proof now carries a
`purpose` claim, and each verifier accepts exactly one kind of token.

The return leg also sets a single-use companion cookie, mirroring the nonce
`/start` already uses, and stamps the action its redirect lands on. Redeeming
a proof requires the matching cookie — consumed on success only, so a rejected
call can't invalidate an in-flight return — and the popup honors only a proof
minted for its own action.

The cookie is host-only, so `/auth/oidc/verify-popup-return` is also served on
the GUI origin and the popup redeems it same-origin. The return leg always
lands the popup on `config.origin`, where both the cookie and the route live.
2026-09-23 17:21:35 -04:00
Daniel Salazar 4c3c684cb4 fix: refuse app-issued scoped tokens on events handler and worker routes (#3935)
* fix: new cache costs for claude

* fix: refuse app-issued scoped tokens on events handler and worker routes

An access token an app mints (e.g. a getReadURL() token) resolves
effectiveApp to the issuing app, so #handlerApp and listEventsWorkers
treated it as the app: it could publish, remove and list handlers, and
list or destroy the app's worker, regardless of its permission manifest.
Refuse any scoped token there, as the kv-handle routes already do.

* test: freeze the clock before the first backoff read in workerInvoker

The first hold was read on real time, so a slow run measured the 2s
backoff as 1s.
2026-09-23 13:58:49 -07:00
Juan Castro 438a4584de fix(hosting): keep alternate-host bootstrap stubs from shadowing private apps
A public origin-bootstrap app row planted on an alternate hosting domain
could resolve ahead of the owner's private app and skip the private
access gate (PUT-1883).

- get-user-app-token canonicalizes hosted-subdomain origins before uid
  derivation and bootstrap, so every hosting variant shares one app row
- the app create/update conflict check crosses hosting-domain variants,
  so an existing alt-host stub is absorbed by the owner's create
- resolveOwnedAppForHostedSite and the canonical-uid lookup order
  matches private-first with a deterministic id tiebreak
2026-09-23 16:39:22 -04:00
Daniel Salazar b3fda186e7 fix: let apps read opted-in teams, scope worker listing, fix revokeReadURL (#3933)
- GET /teams and /teams/:uid/members admit apps for teams whose owner opened
  the directory; apps get active members' usernames only
- events.workers.list() from an app returns only its own worker
- hide account-session-only SDK methods from the public types
- unpublish the updateProfile docs page (redirects to getProfile)
- revokeReadURL() works through a new /auth/revoke-own-access-token route
2026-09-23 12:06:56 -07:00
Filip KujundžićandClaude Opus 5.5 bc0f4dc279 feat(ai): add GPT-6 Sol and Luna and sync Claude pricing (#3931)
* feat(ai): sync GPT-6 Sol, GPT-6 Luna, and Claude Opus 5.5

* chore: remove documentation changes from model sync

* fix(ai): bill OpenAI cache writes and long-context pricing

GPT-5.6 and later bill prompt-cache writes at 1.25x input and report them
in `cache_write_tokens`, inside the input total. Both OpenAI calculators now
split them out of the prompt count and meter them under their own key, and
the six GPT-5.6+ models carry the rate.

GPT-6, GPT-5.6, GPT-5.5 and GPT-5.4 (incl. Pro) bill a request with more
than 272K input tokens at 2x input (cached reads and cache writes included)
and 1.5x output for the whole request. Models declare this as
`long_context_pricing`, and the ledger overrides, the reported `usd_cents`
and the credit gate all apply the multipliers.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ai): bill GPT-5.6 Sol at OpenAI's promotional pricing

The catalog carried GPT-5.5's $5/$0.50/$30, but OpenAI bills GPT-5.6 Sol
at $4/$0.40/$20 per million input/cached/output tokens, promotional
through at least 2026-11-21. PUT-1943 tracks re-checking before then.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-23 01:59:10 -07:00
Juan Fernando Castro 17f5454265 Merge pull request #3915 from HeyPuter/juancastro/put-1871-authcheck-app-reports-every-app-as-already-authorized
fix: check-app reported every app as already authorized
2026-09-22 18:24:20 -04:00
Filip Kujundžić 5ddff2b3d6 Merge pull request #3927 from HeyPuter/404oops/put-1870-ai-upstream-credit-exhaustion-is-the-largest-error-class-in
fix(ai): classify upstream credit exhaustion, contain the route, redact vendor text (PUT-1870)
2026-09-22 22:29:58 +02:00
404oops 52f0176854 fix(ai): classify and sanitize upstream credit exhaustion 2026-09-22 21:51:44 +02:00
Daniel Salazar 300fb115a1 feat: opus 5.5 (#3930)
* feat: opus 5.5

* fix: bad typing
2026-09-22 12:31:44 -07:00
dependabot[bot] 6debd7df76 chore(deps-dev): bump prettier from 3.9.6 to 3.9.8 (#3925)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.6 to 3.9.8.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.6...3.9.8)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-22 11:33:49 -07:00
dependabot[bot] 70404579f2 chore(deps-dev): bump @types/node from 24.13.4 to 24.13.6 (#3926)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.4 to 24.13.6.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-22 11:33:40 -07:00
Juan Fernando Castro 7462a8eda9 Merge pull request #3914 from HeyPuter/juancastro/put-1869-unhandled-rejection-on-the-hottest-permission-path-can-take
fix: stop leaking an unawaited permission traversal on every check
2026-09-22 14:28:44 -04:00
Daniel Salazar 718de8a9c6 fix: bump server timeouts for better error handling (#3929) 2026-09-22 10:13:46 -07:00
Juan Fernando Castro 8abdcf4f7f Merge pull request #3928 from HeyPuter/juancastro/put-1873-restrict-godmode-app-launches-to-godmode-callers
fix: restrict godmode app launches to godmode callers
2026-09-22 12:32:32 -04:00
Juan Fernando Castro 476f7e075d Merge pull request #3913 from HeyPuter/juancastro/put-1865-apps-repeatedly-ask-permission-when-it-wants-to-access-files
fix: settle a URL file-access prompt from a grant the app already holds
2026-09-22 12:32:06 -04:00
Juan Castro 1bc8cb964e fix: restrict godmode app launches to godmode callers
`ExecService.launchApp`, reachable from `puter.ui.launchApp`, launched any
named app with no check on the target. Gate it: a godmode target may only
be launched by a godmode caller. Desktop launches (tile, double-click,
URL) call `launch_app` directly and are unaffected; `connectToInstance`
already had its own allowlist.
2026-09-22 09:12:25 -04:00
Daniel Salazar 2690f4934c fix: tests (#3922) 2026-09-21 23:33:23 -07:00
Daniel Salazar d07ddeaaf5 fix(fs): keep a home directory on its account's username (#3920)
A root was free to be renamed or moved, so it could be parked on a
username no account held yet. The next account to take that name
provisioned a second root at the same path, and from then on either row
could answer a lookup of it — new entries inherit their owner from
whichever row resolved as the parent, so one account's files were
created owned by the other, and renaming the parked root dragged the
other account's subtree along with it.

Rename and move now refuse a root, every username claim checks the home
path before the user row is written, and renameUserHome refuses to heal
onto a path another account holds.
2026-09-21 23:29:09 -07:00
Daniel Salazar 1261976088 feat: user profiles in a system-owned store, public only for paid accounts (#3919)
- ProfileService keeps each profile as /system/profiles/<uuid>.profile,
  admin-owned and served by the protected puter-profiles subdomain
- GET/POST /profile; puter.auth.getProfile/updateProfile, with
  getProfilePicture reading through them
- another user's profile is served only while its owner is on a paid
  plan (profileGate.enabled kill switch); the hosted file is gated
  through the new site.access.check hook the hosting middleware asks
  before streaming any file
- SubdomainStore.create takes isProtected
- the GUI reads and writes the profile through the SDK
- docs: getProfile, updateProfile, UserProfile, limits

PUT-1859 PUT-1860 PUT-1861 PUT-1862
2026-09-21 23:28:56 -07:00
Daniel Salazar 2abd5533ac fix(puter-js): restore file pickers on third-party sites (#3917)
The clickjacking guard in #3912 pinned `event.source` to `messageTarget`,
which is only assigned when `env === 'app'`. On a third-party site it is
undefined, so the UI listener dropped every message and
showOpenFilePicker, showSaveFilePicker and showDirectoryPicker never
settled.

Split the two environments. `app` still pins the host frame, whose origin
is whatever the deployment is served from. `web` has no host frame — the
picker popups post back directly — so pin the GUI origin we opened them
on, plus the set of popups we opened. Origin pinning is safe there,
unlike the parent-frame case, because we chose the URL, so self-hosted
and local deployments keep working.

Falls back to the origin alone when `event.source` is null: the picker
calls window.close() right after posting and a discarded browsing
context can drop the source.

PUT-1867
2026-09-21 21:43:58 -07:00
922d203e18 fix(ai): send stable, non-sequential user identifiers to AI providers (#3856)
* fix(ai): send stable, non-sequential user identifiers to AI providers

A precedence bug in the AI providers' identifier expression made every
request send `user: ":undefined"` (the ternary bound the app-uid suffix to
the whole `actor.user.id + actor.app?.uid` sum instead of just the suffix),
or read `actor.user.id` on a missing user. The same expression also shipped
the sequential internal user id, letting AI vendors correlate a single
account across apps and sessions.

All eight OpenAI-, Azure-, xAI-, Meta- and ZAI-style providers now build
the identifier through one shared helper, `aiUserIdentifier()`:

- `puter-<user-uuid>[-<app-token>]`: the random user UUID is always
  preserved in full; `maxLength` constrains only the app-bearing form
- app attribution reads `effectiveApp`, so access-token requests name the
  issuing app instead of looking like direct user traffic
- the app token is truncated to fit the budget, and omitted entirely when
  the remaining budget is below 8 chars, where a truncation could collide
  with another app's uid
- nothing is sent for the system actor
- Meta and ZAI keep a caller-supplied `safety_identifier` / `user_id`
  override, applied before the helper result

`user` is deprecated by OpenAI; the SDK types direct callers to
`safety_identifier` (abuse detection) and `prompt_cache_key` (cache-hit
bucketing). The four OpenAI/Azure chat providers and MetaProvider now send
`prompt_cache_key` as well, defaulting it to the same per-user identifier
unless the caller supplies one; Azure's Grok branch drops both fields,
matching its rejection of unknown args. The cap comment cites only verified
limits: OpenAI's 64 for `safety_identifier` (from the SDK types) and Z.AI's
6-128 for `user_id` (from Z.AI's docs); Meta and xAI document none, so none
is claimed.

The xAI image `#edit` path now carries the identifier like generation, and
takes a named-options param so `user` cannot be transposed with the
adjacent same-typed `aspectRatio`.

Tests share a four-actor matrix (`user` / `user+app` / `access token` /
`system`) with `assertActorMatrixIdentifiers()` across the six
OpenAI-style suites; the helper has exact-string and boundary coverage
(size caps, zero-budget and sub-base cases, no dangling separator, UUID
never truncated, collision guard); the Azure Grok assertions run under a
real user actor so they cannot pass vacuously. 212 provider-suite tests
pass; typecheck and ESLint are clean.

* fix(ai): lock the vendor identifier down and keep vitest out of the test util

Meta and Z.AI no longer let `custom` override the abuse identifier; it is
Puter's attribution, not the caller's. The shared test util exposes pure
field pickers instead of importing vitest into a file the production
tsconfig compiles. The helper's length-cap comment now matches vendor docs
(Meta does cap `safety_identifier` at 64), the redundant budget branch and
the unused export are gone, and the per-user `prompt_cache_key` trade-off is
stated once in the helper instead of five times in providers.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 16:21:47 -07:00
404oopsandClaude Fable 5.1 027e9a71f3 fix(ai-image): refresh model catalogs, unify txt2img requests, expand docs (#3889)
* fix(ai-image): sync image catalogs, unify request shape, document models

Model sync against every vendor listing and a live generation sweep:
Gemini stable ids replace the retired preview spellings (2.5 Flash Image
delisted ahead of its 2026-10-02 shutdown), OpenAI gpt-image-1/-1-mini/-1.5
are delisted but routable until their shutdown dates with gpt-image-2 as
the default, xAI gains grok-imagine-image-2.0 with its quality tiers,
BytePlus gains the 3K/4K tiers and per-model pixel bounds, Cloudflare
gains SDXL Lightning/Base and SD 1.5 Inpainting, Replicate gains a
schema-driven catalog of 88 additional models with version-pinned
community predictions, and every Together image route is excluded for
the third-party data-sharing requirement.

Request normalization: the driver collapses ratio/width/height/aspect_ratio
into one imageSize with aspect-versus-pixel intent, validates prompt,
quality and resolution once, resolves provider hints (short names or
full driver ids) and aliases with exact ids winning over resellers, and
hands each provider an immutable copy of the caller's args. Providers
share prompt validation, aspect snapping, and content-sniffed data URIs
so responses carry the right MIME type. Replicate predictions are created
once, cancelled on abort or deadline, and bounded in input fan-out.

SDK: txt2img copies caller options, rejects blank prompts with
prompt_required in every call form, and documents that normalize has no
effect because the image return shape is already uniform.

Docs: txt2img options per provider, provider defaults and discovery,
availability notes, a new image model catalog and pricing page, and the
image-generation limits in the quotas page.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai-image): sniff SVG outputs correctly, reject hex/exponent dimensions

imageDataUri only decoded the first 24 base64 chars (18 bytes) before
sniffImageMime, which cannot see an <svg> root behind an XML prolog -
SVG outputs from recraft-v*-svg models were being labeled image/png.
Decode enough of the payload to cover the 8 KB SVG sniff window.

dimension() accepted string number literals ('0x10', '1e3') as if they
were decimal dimensions; restrict to plain decimal notation.

* docs(ai): link to the model directory instead of a static catalog

The image model catalog and pricing page duplicated the always up to date
model directory at developer.puter.com/ai/models. Link to the directory
from txt2img-related docs and keep the Together data-sharing exclusion
note self-contained.

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 15:00:02 -07:00
Juan Castro 259cb69fde fix: check-app reported every app as already authorized
The token grant writes `flag:app-is-authenticated` on the user->app row,
but the check asked for `service:<app_uid>:ii:flag:app-is-authenticated`.
Those never match, and the parent walk reduces the question to a bare
`service` — a root every user holds by default — so the check answered
true for any app_uid and handed back an app-under-user token for an app
the user had never opened.

Read the row the grant writes, through one shared constant so the two
cannot drift again, and mint against the resolved uid.

A cancelled account picker no longer leaves `popup_signin_consent` set:
with no relationship there is no token to flip it.
2026-09-21 16:13:53 -04:00
404oopsandClaude Fable 5.1 081941a6f3 fix: null-prototype model maps in chat/video drivers; txt2vid prompt and option guards (#3887)
Chat and video drivers keyed their provider and model maps on plain objects,
so `model` or `provider` values such as `__proto__` reached Object.prototype
and surfaced as 500s. Both maps are now null-prototype objects and reject
those names as ordinary unknown models.

txt2vid now rejects blank or non-string prompts with `prompt_required` before
any request, tolerates `null` in either argument slot, and copies the caller's
options before resolving the `duration` alias and output path so frozen
option objects work and caller objects are never mutated.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 13:12:35 -07:00
404oops e2a5ab8cdc docs: document local Docker builds with Compose overrides (#3867) 2026-09-21 13:12:16 -07:00
Juan Castro 07e889f135 fix: stop leaking an unawaited permission traversal on every check
`validateUserPerms` started the linked SQL traversal beside the flat read
and returned on a flat hit without awaiting it. The hit is the common
case, so nearly every user-permission check left a floating promise: with
`keep_alive_on_uncaught` unset no `unhandledRejection` listener is
registered, so a DB wobble inside the traversal exits the process.

Run it only when the flat read misses. That also stops a SQL walk the
cache had already answered.
2026-09-21 16:07:26 -04:00
Juan Castro ab7a5f325a fix: refuse an empty app_uid and survive a failed grant check
A present-but-empty `app_uid` fell through to checking the user, where
every `fs:` scope on their own file answers `true` — a prompt settled by
a question nobody answered. It is a 400 now.

The launch-path check also sat outside any catch, so a rejecting read
would have taken the whole launch down instead of falling through to the
prompt. Renamed the seam it is injected through to match the function it
defaults to, since wiring the token-based sibling in its place would send
an app uid as a bearer token and silently always prompt.
2026-09-21 15:56:39 -04:00
Juan Castro b4b464554b fix: settle a URL file-access prompt from a grant the app already holds
Opening `/app/<name>?file=<path>` asked for consent on every launch: the
gate prompted unconditionally and never read the `fs:<uid>:write` grant
its own Allow had written.

`/auth/check-permissions` takes an optional `app_uid` so the account can
ask what one of its apps holds, and the launch gate skips the dialog when
the answer is yes. Sessions only — an app or a scoped token asking would
be a window onto its neighbours' grants.
2026-09-21 15:00:02 -04:00
Nariman Jelveh 4de1d1fb39 Redesign the Teams tab in the dashboard (#3901)
* Redesign the Teams tab in the dashboard

Hero card with a lettered tile, role pill, handle and headcount; the
directory setting becomes a settings row with the shared toggle switch;
the add-account form gets labels, Enter to submit, and a copy button on
the one-time password. The accounts and record tables drop the grid for
hairlines, avatar tiles and status pills, and stack into labelled rows
on phones so actions stay reachable. Loading, empty and error states
get a skeleton, a create call to action, and a retry.

Also fixes audit dates showing 1970 (the audit routes send unix
seconds), the actions cell breaking the row border by being a flex
table cell, and the "0 suspended" clause on a healthy team.

* Use the dashboard's overlay dialog in the Teams tab, not UIAlert

Adds UIDashboardDialog, a confirm / alert / prompt card built like the
share and properties modals: mounted in the dashboard window, revealed
after a frame, dismissed by Escape, the close button or a backdrop
press-and-release, with a focus trap and focus restored on close. On
phones it docks to the bottom as a sheet. The Teams tab's confirms,
error alerts, seat-limit warning and rename/create prompts all go
through it; UIAlert and UIPrompt are no longer imported there.
2026-09-21 09:30:03 -07:00
Daniel Salazar 4e9b03bf5c fix: harden clickjacking (#3912) 2026-09-21 09:19:37 -07:00
Juan Fernando Castro a63172e9ae Merge pull request #3892 from HeyPuter/juancastro/put-1806-invite-email-addresses-are-disclosed-to-apps-manage
fix: stop disclosing invite addresses to apps, tokens and delegates

approvals already in place
2026-09-21 12:05:08 -04:00
Reynaldi Chernando 29d805d793 show app icon when login from external app (#3905)
* show app icon when login from external app

* fix
2026-09-21 22:33:38 +07:00
Juan Fernando Castro 059fae3d15 Merge pull request #3872 from HeyPuter/juancastro/put-1813-team-share-blocklist-and-unshare-paging
fix: make the block list bite on team shares, and unshare every re-share (PUT-1813)

Approvals in place to move with the merge
2026-09-21 11:31:24 -04:00
Juan Castro 18ada4764a chore: drop a stray lockfile marker
A local `npm install` (needed to pick up main's new `postal-mime`) wrote
`"extraneous": true` onto a bundled `@clack/prompts` dependency. It was the
branch's only lockfile change and it does not belong to this PR.
2026-09-21 11:26:27 -04:00
Juan Castro 72203152d9 fix: bound a scoped token to what it issued, which is nothing
CI caught three HTTP-level tests the earlier merge left behind, and they
were right to fail: dropping this branch's `manage` gate in favour of
main's row filter lost a case main's filter does not cover.

Main bounds an app to the rows it issued. A *scoped* access token is not
an app, so it was falling through unbounded — `/fs/stat` with
`return_shares` handed a `fs:<uid>:list` token the owner's whole share
list. Addresses stayed withheld, but who else can reach a file is no
more a narrow token's business than the addresses are.

So the filter is generalised rather than the gate restored: a scoped
token is bounded to nothing, since it issues nothing under its own name.
Filtering it by a null app would have been worse than not filtering —
that matches the owner's own rows. Apps and sessions behave exactly as
they do on main, and a full-access token still holds the account's reach.

The three tests now assert the answer instead of a refusal, including
the one whose name had always promised a refusal its body never checked.
2026-09-21 11:12:37 -04:00
Juan Castro cec8382d4d Merge branch 'main' into juancastro/put-1806-invite-email-addresses-are-disclosed-to-apps-manage
Two textual conflicts, both additive on each side: `isAccountContext`
(here) and `isPlainUserActor` (main) are both imported and both used,
and the `stat()` note keeps both sentences — this branch's on who may
read an invite address, main's on the share-read limit it spends.

The rest is adapting to main, which grew its own answer to half of what
this branch was for. `listSharesOf` there bounds an app to the rows it
issued itself; this branch instead required the credential to hold
`manage` and refused it otherwise. Main's is the better mechanism — it
answers the app rather than turning it away, and it hides other
issuers' rows outright rather than redacting a field on them — so the
`manage` gate goes, and with it the two tests that asserted the
refusal. They are replaced by tests that hold main's line: an app sees
none of the invites it did not send, with or without `manage`.

What this branch still carries is the gap main does not close. Its row
filter only applies to apps, so a plain manage delegate still reads the
owner's invite addresses; `#maySeeInviteAddress` is what withholds
those, and its delegate tests pass unchanged. The `stat()` note is
corrected to describe main's behaviour rather than the removed gate.
2026-09-21 10:46:07 -04:00
Juan Castro 3f335939b3 fix: team shares are not subject to a recipient's per-sender block
Review call from the ticket's author: a team share is the team's, not
one colleague's to withhold from another, so a personal block should not
hide it. This drops the enforcement added earlier on the branch — the
listing, its total and the fs-event fan-out no longer filter team rows
by the recipient's block list, and the SQL fragment that did it goes
with them.

What a block still does for a team share is suppress the notification,
which was already true before this branch: it stops the interruption
without pretending to stop the access. Leaving the team is what ends
that. Said so in the block API docs, the settings copy and the code,
since the mismatch between the two was the original complaint.

The unshare sweep is untouched — that half of the ticket stands.
2026-09-21 10:28:50 -04:00
Juan Castro 4e821c128c Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging
# Conflicts:
#	src/backend/services/share/ShareService.ts
#	src/backend/stores/share/ShareStore.js
2026-09-21 10:28:33 -04:00
Juan Fernando Castro c313a381b6 Merge pull request #3896 from HeyPuter/juancastro/put-1843-plan-card-unknown-not-free
fix: a team seat's plan card names its team, and an unreadable plan is not Free (PUT-1843)
2026-09-21 09:58:41 -04:00
Juan Fernando Castro 76e0d2ec93 Merge pull request #3871 from HeyPuter/juancastro/put-1798-sharing-email-notification-changes
feat: name the issuing app in share emails, and cut the digest window to 5s
2026-09-21 09:58:09 -04:00
Juan Fernando Castro 24fb679dd3 fix: charge stat's return_shares against the share-listing budget (#3870)
* fix: charge stat's return_shares against the share-listing budget

`return_shares` on /fs/stat and the legacy /stat runs the same work as
GET /share/shares, but was only metered under fs:stat's far more
generous limit — and the two scopes stacked instead of sharing one
counter.

Adds consumeRouteRateLimit(req, spec), an imperative charge that
resolves the key and per-subscription limit exactly as rateLimitGate
does, so a handler can conditionally spend a second scope when a
request flag makes the route expensive. Both stat handlers now charge
share:list before doing the listing work; SHARE_LIST_LIMIT moves to a
shared share/limits.ts so all callers pin the same spec.

Closes PUT-1597.

* feat: consumeRouteRateLimit takes the array spec form too

Review follow-up on #3870: a multi-window spec passed whole would have
read an undefined window and silently never pruned. Charge each window
in order instead, refusing on the first refusal, matching the gate.
2026-09-19 14:11:29 -07:00
Daniel Salazar eb9f03e984 fix: misc hardening + other fixes (#3906) 2026-09-19 12:57:57 -07:00
Daniel Salazar 9292771554 fix: hardening (#3904) 2026-09-18 11:22:39 -07:00
Reynaldi Chernando f4974581d0 Add auth message for external apps using puterjs (#3893) 2026-09-18 08:56:02 +07:00
Daniel Salazar b39bb771d9 feat: sortby for recursive readdir (#3900) 2026-09-17 17:29:55 -07:00
Nariman Jelveh 766379ae9f feat(puterjs): add defensive profile picture lookup (#3899) 2026-09-17 17:16:54 -07:00
Daniel Salazar 0be3bc55c2 feat(metering): AI cost multiplier hook for AI drivers (#3898)
Emits ai.cost.multiplier.<driver>.<provider>:<model> before recording AI
usage, so what a model costs to charge is policy an extension owns rather
than a number in core. Nothing listening records the provider cost.

MeteringService.withAiCostMultiplier(driver) returns a view of the service
whose recording paths scale costOverride by the hook's answer; every AI
driver hands that view to its providers, so all of them are covered without
touching provider code.
2026-09-17 15:18:38 -07:00
Nariman Jelveh d1484af754 One recipient field in the sharing dialogs, with suggestions (#3897)
* feat: one recipient field in the sharing dialogs, with suggestions

Both sharing dialogs asked twice: a text field for a person, and a
separate select, label, note and button for a team. Which control to use
was a fact about the API — a bare string is read as an email or a
username, so a team could not be typed — not something a user should
have to know.

Now there is one field. Clicking it offers who this account shared with
before, the teams it belongs to, and the people in them; typing narrows
the list. Choosing an offer locks the field to that recipient, so a team
picked by name still goes out by uid, and the note about a team grant
reaching everyone in it appears at the moment it applies. Anyone already
on the access list is left out of the offers, and a typed address still
works untouched.

Past recipients are kept in the account's key-value store, so they
follow the user between browsers; a typed address that turned out to
belong to an account is filed under the username the backend resolved.
Colleagues are read one page per team, cached for five minutes, and only
looked up once the field is actually used.

The picker is one helper shared by both dialogs so the two can't drift,
styled through `share-suggest-*` tokens each host restates in its own
palette. It opens in the flow rather than floating, which a scrolling
modal body and a mobile bottom sheet would otherwise clip.

* fix: float the recipient suggestions instead of resizing the dialog

The list opened in the flow, which pushed the Share button and everything
under it down and back up as it opened and closed — a dialog that resizes
under the cursor. It now hangs off the recipient row, out of the flow, and
floats over what follows it.

It goes inside the row rather than after it, so the row is its containing
block; as a sibling it resolved against whatever was positioned further up
and landed at the bottom of the scrolling body. Being out of flow it can
be cut off by that scroll container rather than scrolling itself, so on
open it measures the room between the row and the nearest clipping
ancestor, caps the list to it, and opens upwards where below is too tight.

* fix: keep the recipient field focused when the picker is wired onto it

Both sharing dialogs focus the recipient field as they open, and the
picker then moves that field into its wrapper. Moving an element takes
it out of the document for an instant, which drops its focus, so every
dialog opened with the caret in the field opened with the caret on
<body> instead.

* fix: keep the recipient suggestions inside the screen

The list sized itself against the nearest clipping ancestor alone. That
ancestor is the dialog's own scrolling body, which reaches past the
bottom of a short screen — or of any screen once the dialog has been
dragged low — so the list was placed below the fold and the user saw
nothing at all. The viewport bounds it too.

* fix: stop the suggestions flashing a loading box over the dialog

Clicking the recipient field put up a panel saying it was loading before
it knew there was anything to load. For the many accounts with no teams
and nobody shared with before, that panel covered the Share button for
as long as the round trip took and then vanished again. It now waits a
beat before saying anything, and leaves whatever is already listed in
place while a reload is in flight. A list still on its way also no
longer swallows the Escape that closes the dialog.

* fix: do not double-encode the recipient field's placeholder

`i18n()` encodes what it returns, which is right for a string dropped
into markup and wrong for one handed to `.attr()` — the entities show up
as themselves. Nothing is lost today because only English carries the
key, but the next translation with an apostrophe in it would read
`l&apos;équipe`.

* chore: drop the styles for the team picker the field replaced

The desktop dialog's separate team control went with the one recipient
field; its rules stayed behind. The dashboard's equivalents were already
removed with its own markup.

* fix: let Enter share from the desktop dialog's recipient field

Typing a name and pressing Enter did nothing there: the field is in no
form, so the only way to send was to reach the button. The Dashboard's
dialog has always submitted on Enter, and the field now takes Enter to
choose a suggestion, which makes a second press that does nothing read
as a dead key. The picker still gets the press first while it is
choosing a row.
2026-09-17 13:54:03 -07:00
Juan Castro dfd7925872 fix: say which team a seat's plan belongs to, paid or free
A provisioned account read as a plain "Free" plan with an "Upgrade for
more features" badge: nothing said the account belongs to a team, and
the prompt asked for an upgrade only its owner can buy. Its free tier is
not the free plan either — org_seat_free is half the allowance.

The card now names the team on both paths: a paid seat keeps its tier
and status and gains "Managed by <team>", and a seat with no tier reads
"Team account" instead of an upgrade pitch.
2026-09-17 14:15:01 -04:00
Juan Castro e50d210423 fix: an unreadable plan is not a free plan (PUT-1843)
The Home tab read /marketplace/subscriptions/current and folded every
failure into the free state: a non-OK response left `subscription` null,
and the catch said so explicitly. So a refusal rendered as a confident
"Free" with an Upgrade badge.

That is reachable: verification gates are default-on for authenticated
routes, and a team seat that still owes its password change is refused
there — it then reads Free while its team pays for a tier. A rate limit
or a blip does the same to anyone.

The plan card moves into its own method so a failed read can return
without touching it, leaving the card as it was rather than naming a
plan the account does not have. A 200 carrying no subscription still
reads Free, which is the one case that actually means it.
2026-09-17 13:52:03 -04:00
Daniel Salazar 51ce868d1e fix: shared kv handles also allow value opt in (#3895) 2026-09-17 10:34:35 -07:00
Juan Castro 461cb7d0a9 fix: close two gaps the review found in the invite-address fix
- A full-access token was denied the address while `shared-by-me` still
  handed it the same rows, so the clause bought no privacy and cost an
  API client the address `unshare()` takes. `isAccountContext` is the
  boundary the rest of the codebase already uses for 'acting as the
  account': plain session or full-access token, never a scoped one.
- The Dashboard share modal dropped a withheld invite entirely, since
  its aggregate keys a pending row on the address — so a delegate saw no
  sign of an outstanding invite and accessCount under-reported who could
  reach the item. It is now kept, keyed on the share uid, labelled, and
  without the controls that would need a recipient to address.
2026-09-17 13:21:13 -04:00
Daniel Salazar bb97660ad7 feat: kv events value opt in (#3894) 2026-09-17 00:17:12 -07:00
Juan Castro 3e3d02bafe fix: stop disclosing invite addresses to apps, tokens and delegates
getShares (and stat's return_shares, which runs the same listing) gated
on #assertCanManage's default 'see' mode, so any credential that could
see the node got every unclaimed invite's raw email — including an app
handed one file by the picker, a list-scoped token on the stat surface,
and a manage delegate reading the owner's invitees.

Two bounds, matching the invariant clientShare.ts already claimed:

- An invite's address goes only to the item's owner and to whoever sent
  it, and never to an app or token. A delegate can revoke only what they
  issued, so withholding costs them nothing they could act on.
- An app or token must hold manage reach of its own to read the listing
  at all; it answers for the ancestors too, which is not what being
  handed one file grants. tryListSharesOf turns that into an empty
  shares array, so stat itself keeps working.

The share dialog names an unattributable invite rather than rendering a
blank row with a dead revoke button.

Closes PUT-1806.
2026-09-16 18:59:10 -04:00
Neal Shah b2fef45fd6 email read api (#3869)
* email read api

* address docs changes

* remove playground examples
2026-09-16 18:38:58 -04:00
Juan Castro 728966bfb1 Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 18:05:58 -04:00
Juan Castro 3f7ee34706 Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 18:05:39 -04:00
Juan Fernando Castro ff333d05e8 Merge pull request #3890 from HeyPuter/juancastro/put-1819-keep-plan-console
feat: tell the plan picker when a seat's plan is on the way out
2026-09-16 18:05:12 -04:00
Daniel Salazar 194ca7c789 fix: list a link share only while the owner's plan covers it (#3891)
* fix: list a link share only while the owner's plan covers it

* chore: types + jsdoc cleanup
2026-09-16 14:42:52 -07:00
Juan Castro 0b85203edb feat: tell the plan picker when a seat's plan is on the way out
The billing view now names each seat's status; the console passes it
through to the picker so a cancel-pending plan can be kept, plus the
two strings that flow renders.
2026-09-16 15:55:52 -04:00
Juan Castro 23c33b2e06 Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging
# Conflicts:
#	src/backend/stores/share/ShareStore.js
2026-09-16 15:27:09 -04:00
Daniel Salazar 1b6334c02c feat: a paid plan counts as a verified card for the sharing gate (#3886)
* fix: list a link share only while the owner's plan covers it

* fix: fs limits for signed urls

* feat: a paid plan counts as a verified card for the sharing gate
2026-09-16 10:18:47 -07:00
Juan Castro 0593844acf Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 13:05:03 -04:00
Juan Castro 67d048148c Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 13:04:43 -04:00
Juan Fernando Castro 05cbd33211 Merge pull request #3888 from HeyPuter/juancastro/bump-sqlite-schema-version-80
test: schema version is 80 since 0084
2026-09-16 13:04:15 -04:00
Juan Castro 5c8f02384c test: schema version is 80 since 0084
#3874 added 0084_share-anyone-with-link.sql, taking the sqlite chain's
target user_version to 80, but CURRENT_SCHEMA_VERSION stayed at 79 —
and the test workflow only runs on PRs, so main broke silently and every
open PR's 'test (base)' job now fails on it.
2026-09-16 12:48:07 -04:00
Juan Castro 5d20e054ac Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 10:33:03 -04:00
Juan Castro 3bd3955e0d Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 10:32:05 -04:00
Reynaldi Chernando 79c856386f document model variants (#3875) 2026-09-16 19:31:28 +07:00
Daniel Salazar 59f73528cb feat: share with anyone with the link (PUT-1580) (#3874)
* feat(email): inline cid attachments and Puter mailbox delivery for sendTransactional

EmailAttachment gains cid/contentDisposition so the transactional driver can send inline images. The SDK's EmailAttachment typedef now comes from types.js, which already carried cid. Docs describe delivery to <username>@puter.email recipients and the not_found code.

* feat: share with anyone with the link (PUT-1580); gate sharing on a verified phone or card
2026-09-15 21:06:12 -07:00
Juan Castro 27e858d760 fix: gate the group-share index delete on authority, qualify the live filter
Review follow-ups on #3872: deleteActiveGroup ran even when every
canManagePermission check was denied, deleting the index row while the
team's grant survived — unlisted live access, the class this PR exists
to remove. And memberIdsAmong's join left deleted_at unqualified, which
only works while jct_user_group has no such column; #live now takes an
alias.
2026-09-15 18:57:50 -04:00
Juan Castro 0bd6458441 test: pin the mixed-digest button link carrying no app
Review follow-up on #3871: the flush-level rule — shared_app on the
button only when every entry came through one app — had no direct
coverage. An app share and a plain share folded into one digest now
prove the line and item link keep their attribution while the button
stays clean.
2026-09-15 18:56:38 -04:00
Daniel Salazar 45aff36f0c fix: auto create folders for fs perms (#3873) 2026-09-15 15:11:13 -07:00
Nariman Jelveh a1f7577acb fix: warn before closing the tab while an upload is in flight
Uploads stream from the page, so closing the tab loses a nearly-finished
one with no warning. The beforeunload handler only existed behind
prompt_user_when_navigation_away_from_puter, which is off by default.

Install it unconditionally and have it consult window.active_uploads,
the registry the progress code already maintains. The feature flag keeps
its open-window behavior.

Uploads now register in active_uploads before the first await in the
init callback: an upload failing while the progress window was still
opening used to run its delete ahead of the insert, leaving a phantom
entry that would have made the tab unclosable.
2026-09-15 13:07:43 -07:00
Juan Castro af6547724d Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-15 15:23:22 -04:00
Juan Castro e9922aea26 Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-15 15:23:11 -04:00
Juan Fernando Castro b136c56cb5 Merge pull request #3846 from HeyPuter/juancastro/put-1792-optional-seat-email
feat: the team seat experience — no email required, forced password change, team label, and plan-based limits (PUT-1792)

Note: Bypassing the code owners rule, since there are couple approvals in place for this.
2026-09-15 14:53:06 -04:00
Juan Castro aadcda073d fix: keep blocks out of the authority graph, and sweep what unshare missed
Code review on the previous commit confirmed three ways the block filter
inside readUserGroupPerms turned a reversible block into permanent loss:
canManagePermission reads the same rows, so a block-suspended grant
looked revoked to #revokeDownstream's cascade, to unshare's authority
gate, and to invite claiming — deleting re-shares and invites that were
supposed to come back on unblock. The scan is now deliberately blind to
blocks; a block suspends delivery only (listing, count, fan-out,
telling), which the share store filters itself through one shared
notBlockedSql fragment owned by UserBlockStore.

The team-unshare sweep also now covers what it claimed to: a member's
re-share *to another team* is revoked with them (group rows swept in
 #revokeDownstream, user path included), and the sweep is skipped
entirely while another issuer's grant still backs the team — members'
re-shares rest on that authority and revoking them would orphan live
access. The blanket block-all note in settings now says team shares
still arrive, matching what the code deliberately does.
2026-09-15 14:49:54 -04:00
Juan Castro 9da03555ae fix: make the block list bite on team shares, and unshare every re-share (PUT-1813)
A sender a member blocked could still land items in that member's
shared-with-me — and grant them real access — by sharing with a common
team. The group grant is one row and cannot exclude a member, so the
block is now enforced where delivery is derived per member: the
permission scan, the inbound listing and its count, and the fs-event
fan-out all skip team rows whose issuer the member blocked. Nothing is
revoked, so unblocking restores everything. Blocking now also bumps the
blocker's permission cache so it bites immediately. Direct shares keep
their contract: existing ones stand.

#unshareTeam swept re-shares by listing members with a 200 cap and
never following the cursor, so members past the cap kept their orphaned
rows. It now walks the share rows in the subtree — the set that can
actually need sweeping — and filters those issuers to members, which
has no cap by construction.
2026-09-15 14:49:54 -04:00
Juan Castro f428797a79 feat: render the teams UI for allowlisted users without the global switch
`gui_params.teams_ui` stays the deployment-wide switch; with it off, the
tab now also renders for a signed-in user who passes the email-domain
allowlist (membership included, so seats see their roster). Anonymous
renders hide it, and the API keeps deciding real access either way.
2026-09-15 14:48:11 -04:00
Juan Castro ad3d15e7e2 feat: stage the teams rollout behind an email-domain allowlist
`teams_allowed_email_domains` limits who may enter the teams surface;
unset keeps today's behavior. Gated on the two routes that constitute
entry — creating a team, and the listing that shows the tab — with the
same 404 a teams-off deployment answers, so the GUI needs no change and
a staged rollout is indistinguishable from the feature being off.
Members of an existing team always pass, whatever their domain: an
allowed owner brought them in, and the surface follows the team.
2026-09-15 14:23:49 -04:00
dependabot[bot] 07bf0be9c7 chore(deps): bump peter-evans/create-pull-request from 6 to 8 (#3865)
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 8.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6...v8)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:39:50 -07:00
dependabot[bot] d116f49669 chore(deps): bump dorny/paths-filter from 3 to 4 (#3866)
Bumps [dorny/paths-filter](https://github.com/dorny/paths-filter) from 3 to 4.
- [Release notes](https://github.com/dorny/paths-filter/releases)
- [Changelog](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/dorny/paths-filter/compare/v3...v4)

---
updated-dependencies:
- dependency-name: dorny/paths-filter
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:35:19 -07:00
dependabot[bot] a5f267c2d4 chore(deps): bump docker/setup-qemu-action from 3 to 4 (#3864)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:35:07 -07:00
Daniel Salazar 2fdd67c72e fix: tighten up fs perm strings (#3860) 2026-09-15 10:33:47 -07:00
dependabot[bot] 92dfe4019e chore(deps): bump compression from 1.8.1 to 1.8.2 (#3861)
Bumps [compression](https://github.com/expressjs/compression) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/expressjs/compression/releases)
- [Changelog](https://github.com/expressjs/compression/blob/master/HISTORY.md)
- [Commits](https://github.com/expressjs/compression/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: compression
  dependency-version: 1.8.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:30:41 -07:00
dependabot[bot] 61a6820767 chore(deps-dev): bump @types/node from 24.13.3 to 24.13.4 (#3862)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.3 to 24.13.4.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:29:37 -07:00
dependabot[bot] 38b131e5e8 chore(deps): bump libphonenumber-js from 1.13.12 to 1.13.13 (#3863)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.12 to 1.13.13.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.13.12...v1.13.13)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-15 10:29:09 -07:00
Juan Castro 230241d6d2 fix: let an owner retire the seats of a deleted team, and only those
Deleting a team suspends every provisioned seat, but every member route
resolved live teams only — so the suspended seats could never be
deleted afterwards, stranding the accounts and, on the billing side,
their paused subscriptions. deleteMember now resolves the team
soft-deleted or not, matching the audit reader.

It also gains the guard enableMember always had: only the team's own
suspension qualifies. A platform-suspended seat could previously be
cascade-deleted by the owner, destroying an account Puter had frozen.
2026-09-15 13:22:13 -04:00
Juan Castro 577693bf84 feat: name the issuing app in share emails, and cut the digest window to 5s
When an app shares on a user's behalf, the mail now says so — the
digest line reads 'alice shared report.txt via Mail App', preferring
the app's title, then its name, then its index_url host. Both the
per-item links and the 'Open Puter' button carry a new shared_app
query param (the app's name, or its uid) so the GUI can match the
share back to the app; the button only carries it when the whole
digest came from one app. Plain shares are byte-for-byte unchanged.

Also drops SHARE_EMAIL_BATCH_SECONDS from 30 to 5: the window only
has to fold one gesture's worth of calls, and every second there is
a second the common single-share email arrives late.

Closes PUT-1798.
2026-09-15 12:15:30 -04:00
Juan Castro 27dd0f8ce6 docs: publish only the team surface an app can actually call
Every team route except the directory refuses app and API-token callers —
administration belongs to the account console. Publishing provisioning,
credentials, suspension and audit in the app-developer docs invited
integrations that would 403 on their first request, so those thirteen
pages are gone and the overview is rewritten around what an app can do:
detect team context with list(), look up colleagues with listDirectory(),
and share with a whole team.

listDirectory — the one app-callable method — was also the only one with
no page; it has one now, leading with its consent gate. The team
recipient joins the share() docs, where the integration actually happens.
2026-09-15 09:34:17 -04:00
Daniel Salazar 50d3794283 feat: handle sub required endpoints better (#3859) 2026-09-14 21:39:01 -07:00
Daniel Salazar 7fdbc0247f fix: bug bounty dupes (#3858) 2026-09-14 16:04:00 -07:00
Juan Castro 3bf5ba3525 feat: i18n keys for the seat cancel flow 2026-09-14 18:40:37 -04:00
Juan Castro 018196f8b4 feat: i18n keys for the team billing surfaces
The private billing extensions render user-facing strings; their keys
live here with every other translation.
2026-09-14 17:22:04 -04:00
Juan Castro c3e2717e8e fix: close the review findings on the seat experience
From the adversarial review of this PR.

The forced password-change gate could deadlock: it POSTed to the
cookie-only route with a bare fetch, and both initgui call sites open it
before update_auth_data mints the session cookie — a fresh browser with a
token URL 401'd every submit inside a non-dismissible loop. It uses the
session-cookie retry wrapper now, taking the caller's token because
window.auth_token does not exist yet on that path. It also gains the
logout footer its sibling gates have; a lost temporary password was a
hard lock with devtools as the only exit.

Password recovery refused for seats: the address is admin-supplied and
never verified, so whoever holds that inbox could take the seat over at
any later time. A seat's recovery channel is its admin's reset.

change-email gets the same seat guard as change-username and deletion —
the address is where admin-issued credentials go.

The login response now carries `team` alongside requires_password_change:
no-reload logins store that payload as window.user verbatim, and every
seat restriction keys on it.

Smaller: the team-badge tooltip no longer double-encodes; the create-token
hint for an emailless account stops pointing at a verification it can
never perform; the quotas doc records the halved org_seat_free allowance;
the config template tells upgrading operators how to keep the old flat
cap; the SDK suite covers emailless provisioning and the owner-only uuid.
2026-09-14 17:00:18 -04:00
Juan Castro 15527eac4c fix: hold a team seat to the free caps it was meant to have
Review catch by @Salazareo: `org_seat_free` reached `FREE_SUBSCRIPTION_IDS`
and so the `requireSubscription` gate, but two other surfaces decide on
plan and neither consults that set.

`bySubscription` maps name `user_free` and `temp_free`. A plan that
matches no key fell through to the top-level `limit` -- the paid cap --
so a seat outranked an ordinary free account: 240 event listings a minute
against their 120, and the same shape across the kv, notification,
subdomain and worker drivers. Both resolvers now fall back to the
`user_free` entry for anything in the free set, which covers every driver
at once and any free plan added later.

`subscriberOnly` compared against the two named ids, so a seat could
reach a paid-only model. It asks the set now.

A paid plan that names no cap of its own still takes the base, and an
unresolved plan still takes the base; there are tests for both so the
fallback cannot widen into "free by default".
2026-09-14 09:44:32 -04:00
Nariman Jelveh 4a23c296f0 Fix/dashboard apps files audit (#3857)
* Dashboard Files: keep rows in sorted order when icons resolve late

renderDirectory appended each row as soon as its icon resolved. Icons for
weblinks and .app files are read from the file itself, so those rows always
landed at the end of the listing regardless of the sort column or direction.
Resolve every icon first, then append the rows in sorted order; renderItem
takes the pre-resolved icon and still looks it up itself for single-row
callers (socket adds, instant folder creation).

* Rename: store the new item name raw in data attributes and the editor

rename_file wrote html_encode(new_name) into data-name, title and the
name editor's value. jQuery's attr()/val() store strings literally, so a
file renamed to "a&b.txt" carried data-name "a&amp;b.txt": the dashboard
Files tab, which re-reads data-name for column truncation, then showed the
entity on screen, type-to-select and sorting compared the encoded string,
and the editor reopened on the encoded name. The initial render already
stores these raw; make the rename path match.

* Dashboard Files: Enter with several folders selected enters only the first

The Enter handler called pushNavHistory + renderDirectory for every selected
folder. Only the first render runs (renderDirectory drops calls while one is
in flight), but every folder was pushed onto the navigation history, so after
selecting two folders and pressing Enter, Back led to the folder already on
screen and Forward to one that was never opened. Enter now navigates into the
first selected folder only; selected files still open in their apps.

* Dashboard Files: don't offer to move items the user can't move

Trashing or cutting sends an item to its owner's trash or a new folder, which
a shared root or a read-only share can't do — the server answers Forbidden and
the user gets an error alert for an action that should not have been offered.
The single-item context menu already leaves Delete out for such rows; the
Delete key, Ctrl/Cmd+X, the multi-selection menu and the mobile selection bar
did not check. Route all of them through can_restructure, skipping rows that
can't move and hiding Cut/Delete when the whole selection can't.

* Dashboard Files: hide New Folder and Upload in the Shared view

Shared is a query over other people's items, not a directory. Both buttons
stayed visible there: New Folder did nothing, and Upload opened the OS file
picker and then silently dropped whatever the user picked. Hide them the same
way Trash already does.

* Dashboard Files: let the same file be picked again after a failed upload

The upload input was only cleared in the success callback, so after an upload
that failed, was cancelled, or was blocked (Shared view), choosing the same
file again fired no change event and nothing happened. Snapshot the picked
files and clear the input as soon as the change fires. This also drops the
document.querySelector('form').reset() that reset whichever form came first
in the document rather than this one.

* Dashboard Files: read saved preferences in parallel and re-sort immediately

init awaited four independent kv reads one after another before the first
listing could start, and handleSort awaited two kv writes before re-rendering,
so a sort click cost two round-trips before anything moved. Issue the reads
together and let the writes settle in the background. While here, set
$el_window before the first await (renderDirectory reached through a route
change or socket event during init threw and left renderingDirectory stuck),
and tolerate a corrupt saved column_widths value instead of failing init.

* Dashboard Files: show the loading spinner on the first directory load

showSpinner keyed off a loading flag, but clearing the listing at the start of
a navigation removed the overlay without resetting it: init's spinner was
wiped by the first render's clear, and the render's own showSpinner call then
no-oped, so the initial load ran with no indicator. Guard on the overlay's
presence instead.
2026-09-13 11:31:16 -07:00
Ayoub Ait ChikhandDaniel Salazar 790db87e13 Fix/node signed batch upload (#3816)
* puter_signed_upload_error_correction

* docs: update signed upload behavior

* fix(puter-js): complete a bodiless response in the XHR shim

A response with no body at all (`fetch` gives `null`) threw inside the
shim's own `then`, the same way a missing content-type did: no 'load',
no 'error', the request hangs forever. Signed storage PUTs answer in
exactly that shape.

Adds the regression tests the content-type fix was missing — all three
hang against the shim as it was.

* refactor(puter-js): drop the signed batch-write env allowlist

With `nodejs`, `service-worker` and `web-worker` added, the list held
every value `puter.env` can take, so the gate decided nothing — and a
new env would have been silently routed to the legacy path.

The backend capability check (`signedBatchWriteSupported`, set from a
404/405/501 on `startBatchWrite`) is the one that still does work.

* test: run the directory-upload suite on node and workers

Both were pinned to browser because the legacy `/batch` fallback cannot
create a directory tree. Now that every platform takes the signed path
they are the regression test for it, and the stale comment goes with
them.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-12 16:32:36 -07:00
dependabot[bot] b901040c14 chore(deps): bump nodemailer from 9.0.6 to 9.1.1 (#3832)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.0.6 to 9.1.1.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.0.6...v9.1.1)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 9.1.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-12 00:32:34 -07:00
Daniel Salazar ed7e0b9cc4 fix: sharing, events and worker-credential authorization hardening (#3855)
- PUT-1800: gate `createWorkerSessionToken` on actor type, so an app or an
  access token can no longer mint an app-less, root-shaped worker session;
  `WorkerDriver` binds on `effectiveApp` instead of `app`.
- PUT-1799: add `isAccountContext` and read it where "no app" was being read
  as "the account" — handler publish, events-worker listing, kv handle
  mint/revoke/list. A scoped API token is no longer an account session.
- PUT-1802: re-authorize a durable row before its backlog drains, settling it
  permanently when the grant is gone. Covers an ancestor-level unshare, which
  the revoke settle deliberately leaves to the delivery re-check.
- PUT-1803: mask the owner's absolute path out of deliveries and subscription
  anchors on a foreign node, the way every FS surface already does.
- PUT-1804: let a revoke reach rows already suspended for a resumable reason,
  re-stamping them so a resume cannot hand over the held backlog.
- PUT-1805: apply the subscribe path's audience gate to `/events/fetch` before
  the query, so a cursor can no longer count and name invisible notifications.
- PUT-1807: refuse `mode: 'manage'` from any actor holding an app — inside its
  own AppData the ACL short-circuit would otherwise supply the reach.
- PUT-1808: take the sending peer from the verified signature header rather
  than the request body.
- PUT-1810: re-base a kv share-handle row's stored match filter on the handle,
  so the owner's absolute key prefix stays hidden.
- PUT-1814: escape LIKE wildcards and anchor the issuer-prefix queries on a
  segment; anchor `manage:` stripping; reject a backslash in a share prefix;
  assert a resolved actor in `subscribeDurable`.
- PUT-1815: bound the char/varchar columns behind `event_subscriptions` and
  `kv_share_handles` at the store layer.
2026-09-11 21:18:49 -07:00
Neal Shah 883072d144 add email composer api (puter.email.send) (#3854)
* add email composer

* fix imports

* add import for fetchUrl

* add import for compose

* add API origin to fetchUrl

* CI related fixes

* types fix

* address leaf function nitpick

* tests and validation checks

* Fix lockfile
2026-09-12 01:20:41 +00:00
Juan Castro 8d90418c9e fix: don't offer a seat a plan link it cannot use
The Usage card's Upgrade link relabels to "Manage →" for anyone on a
plan, including a seat whose plan is its team's. Following it only
reaches a dialog saying so. Hidden for a seat, unchanged for everyone
else.
2026-09-11 18:00:53 -04:00
Juan Castro de74b0c488 fix: name the plan a team bought, not its policy id
`i18n()` echoes a key it has no translation for, and a team tier has
none — so a seat's dashboard read "team-basic" where a personal plan
reads "Basic". The offering already travels with the subscription, so
its display name is right there; `i18n` stays the fallback for the
personal tiers that do have keys.
2026-09-11 17:44:30 -04:00
404oops 967b974fe9 Merge pull request #3852 from HeyPuter/404oops/infron-fix-tiers
fix: quote and pin Infron service tiers
2026-09-11 23:14:52 +02:00
Daniel Salazar 93885dfc54 fix: email driver rename (#3853) 2026-09-11 13:58:53 -07:00
Filip KujundžićandClaude Opus 5 ab077f047b Revert "docs: document Infron service tiers"
This reverts commit 5fbdc7cac5. Docs will land
in a separate PR.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-11 21:32:42 +02:00
Juan Castro 63f857ce3c fix: make the seat cap the owner's plan decides actually apply
`config.default.json` shipped `max_seats_per_team: 50`, and that flat
override wins over the plan, so the free/paid caps were dead code on
every deployment that did not delete the key. A free owner got 50 seats.
The defaults now carry the two per-plan numbers and leave the flat
override unset; a deployment that wants one number for everyone still
sets it and still wins.

`#ownerPays` asked metering about an `{id, uuid}` stub, so a resolver
keyed on any other field missed and the answer came down to whether
something else had cached that user's plan first — the same team, at the
same seat count, was refused or allowed depending on nothing.

Running out of seats now says what raises the limit and offers the plan
picker, rather than reporting the number and stopping there. The record
table pages at ten rows instead of running off the bottom, and the team
card no longer labels the absence of a handle nothing here can set.
2026-09-11 14:51:54 -04:00
Juan Castro 480d6bc3b5 feat: give a team seat the account surface that is actually its own
A provisioned account could rename itself, delete itself, and see a
Billing tab for a subscription it does not hold — all of it the team's,
not the account's. Each is now refused server-side and dropped from the
UI, keyed on one predicate: whoami reports a team only for an org-owned
seat, and the owner joined their own team.

The Teams tab showed a seat nothing but its own audit rows, so a member
could not see who else was on the team they were told they shared it
with. It now lists them; listMembers was already membership-gated and
already withholds from a member what is not theirs.

The dashboard share modal had no way to reach a team, though the desktop
dialog has had one since team sharing shipped and the SDK has always
taken `{ team }`. Same control, same copy, same helper. The access list
needed a team bucket to go with it: a team share names no holder, so the
aggregate dropped it and a team you had just shared with vanished.
2026-09-11 14:09:07 -04:00
Juan Castro 1efe086565 fix: give the owner the seat uuid the plan action is keyed on
The per-account subscription button did nothing. `listMembers` never
returned a member's uuid, the SDK's `toMember` dropped it, and the Plan
column read `seatTiers[undefined]`, so every seat rendered as Free and
the action dispatched with no seat.

The uuid is owner-only: it is what billing keys a seat's plan on, and one
member has no business identifying another.
2026-09-11 13:14:09 -04:00
Juan Castro 06bdc26efd refactor: let the billing extension own the plan picker
The Teams tab was asking which tier with a UIAlert, which looked nothing like
the personal plan modal. It now dispatches the seat and lets the extension
render the grid, so the two match and OSS stays free of prices.

Carries `username` and `currentTier` so the picker can title itself and mark
the plan the account is already on.
2026-09-11 10:51:57 -04:00
Juan Castro c2a975b3f7 refactor: drop the team plan card
It described a team-wide tier, which stopped existing when plans moved to the
account. What it still showed -- what each tier costs -- is in the Change plan
picker, next to the account it applies to, so nothing is lost by removing it.

`teamPlan.js` and its tests go with it, along with twelve i18n keys nothing
reads any more. `state.plan` stays: the Plan column and the picker both need
the catalogue and the seat assignments.
2026-09-11 10:20:09 -04:00
Juan Castro b8b0650990 feat: list the uuids of the seats a team is billed for
Per-tier quantities need to know *which* seats are active, not just how many:
the count has to be split by the tier each one is on. `countActiveSeats` answers
the old question and stays for the cap and the console copy.

Keyed by `uid` rather than the numeric id, because the billing side holds uids
and would otherwise have to look the team up twice. Same suspension filter, and
a test asserts the two agree so they cannot drift apart.

Falsified: dropping the suspension clause fails "leaves out a suspended seat,
matching countActiveSeats".
2026-09-11 10:11:34 -04:00
Juan Castro bafe8af71b feat: icon buttons for the per-account actions
A row now carries up to five actions -- change plan, reissue, suspend or enable,
delete -- and five labelled buttons do not fit the cell. Each becomes a 30px
icon button.

The label is not dropped, only hidden: it stays as `title`, as `aria-label`, and
as visually-hidden text, so a screen reader and a hover both still get it. The
glyphs are `aria-hidden`, since the button already carries the name.

Inline SVG rather than files under `icons/`: these are one-place 24px line
glyphs, and the dashboard already inlines its sidebar chevron the same way. An
`edit` glyph is defined but unused, ready for the account-edit action.

Falsified: removing the visually-hidden label fails "keeps the label reachable
without showing it".
2026-09-11 10:09:39 -04:00
Juan Castro 187e4c8a2f feat: set a plan per account, not one tier for the whole team
Reverses PUT-1788 D1 at the UI. A team keeps one subscription; each account sits
on its own tier within it.

The plan card stops being the chooser and becomes a summary: what each tier
costs per account, and how many accounts are on it. Choosing happens on the
account, with a Change plan action per row, because that is the thing the tier
now belongs to.

`memberPlanLabel` reads the seat's own assignment rather than the team's single
tier, so two accounts can honestly show different plans. The other three states
are unchanged and still matter: the owner is the payer, a suspended seat is not
billed whatever it was on, and a seat nobody bought a tier for is free.

The action only appears where a billing extension is present and a catalogue
came back, so a deployment that sells nothing shows prices and no dead buttons.
2026-09-11 10:04:40 -04:00
Juan Castro f2c9737bd8 feat: show each account's plan in the team's accounts table
Per-seat billing context without per-seat subscriptions. PUT-1788 D1 stands: one
tier per team, one Stripe subscription, quantity = seat count. The tier is still
changed once, in the plan card.

Four states, because "on Team Basic" is not true of every row. The owner is the
payer and keeps their own personal plan, so they show as such rather than
inheriting the team's. A suspended seat reads "not billed" -- it stops costing a
per-account charge, which is the same rule the billing summary and the plan
card's seat count already use. A seat of a team that bought nothing reads free,
which is the reduced org-seat allowance. Everyone else shows the tier.

The rule is a helper in teamsConsole so it is testable, like the rest of that
file.

Falsified: dropping the payer branch fails "says the owner is the payer, not a
seat"; dropping the suspended branch fails "says a suspended seat is not
billed".
2026-09-11 09:37:34 -04:00
Filip KujundžićandClaude Opus 5 5fbdc7cac5 docs: document Infron service tiers
Tiers beyond a model's default are reachable as `<model>:<tier>` ids, so
the chat docs gain a Service tiers section covering what the suffixes
mean, that the tier named is the tier billed, and that a tier can carry
a smaller context window than its siblings.

The listModels return value described a `cost` object with `input` and
`output` keys. No such field is returned — pricing comes back as `costs`
alongside `costs_currency`, keyed per vendor and named by the model's
own `input_cost_key` / `output_cost_key`. Correct the prose and replace
the example with a real entry.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-11 14:52:28 +02:00
Filip KujundžićandClaude Opus 5 3d2bdfceef fix: quote and pin Infron service tiers
Infron sells the same model at several service tiers. `min_prompt_price`
and `min_completion_price` are the floor across all of them, so every
model with a flex tier was advertised at a batch-job price nobody gets
by default: 25 of 286 chat models, among them gpt-6-astra at $5/$25
against the $7.5/$37.5 a default request actually bills.

Price each tier from its own row in `providers[]` and pin that tier on
the request, so the price quoted is the price charged. Tiers beyond the
default are listed under their own `<model>:<tier>` ids, letting callers
opt into flex or priority by model name:

    puter.ai.chat(prompt, { model: 'infron:openai/gpt-6-astra:flex' })

Suffix parsing matches the catalog exactly before reading a trailing
segment as a tier, since catalog ids can carry a colon of their own
(`deepseek/deepseek-v4-flash:free`). Tier variants take the context
window of their own offering, which differs from the model-level figure
for 11 of them.

Billing was never wrong — it bills Infron's reported `cost` — but the
understated prices fed the credit gate's output cap, which let a request
run roughly 50% past the balance it was gated against, and the fallback
path that prices per token when a response carries no cost.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-11 14:47:41 +02:00
Juan Castro 2083ef47c0 fix: the plan card counted no seats
It filtered on `org_owned`, but the GUI annotates members to `orgOwned` --
so the count was always zero and the card read "billed for 0 accounts" beside
an accounts table saying two.

Uses `membersBillingSummary` now, the same count the table shows, which also
excludes suspended seats: they stop costing a per-account charge, which the
naive filter would have billed for.
2026-09-10 18:32:31 -04:00
Daniel Salazar 9940beb4bb fix: card verification missing email (#3850) 2026-09-10 15:32:01 -07:00
Juan Castro fa36f2e3cf fix: run the forced password change on the login path, not only token-in-URL
`initgui` runs the verification gates in two places: the token-in-URL branch and
the session-restore/login branch. The password gate went into the first only, so
it never fired for the case it exists for -- a seat signing in through the login
form with the credential its administrator issued. The account reached the
desktop and then failed at every gated call with no prompt, which is the state
the gate was written to prevent.

Caught by manual testing, not by any test: both chains looked right in
isolation. Added an invariant test over initgui's source asserting each gate
appears on both paths and each loops until cleared, which is the shape of the
mistake rather than the instance of it.

Falsified: removing the login-path gate fails "runs the forced password change
on both paths" with `expected 1 to be 2`.
2026-09-10 18:27:01 -04:00
Juan Castro efe6ef09bd feat: show a team's plan in the Teams tab, and let the owner change it
PUT-1796's UI half. The owner already manages accounts here, so the plan they
are billed for belongs on the same page rather than a tab away.

Pricing stays out of OSS. The card is drawn from whatever catalogue the server
returns, so this code knows no tier, no price and no payment provider -- a
deployment that sells nothing serves no catalogue and the card does not render.
`TabHome` already reads `/marketplace/subscriptions/current` the same way, so
OSS reading a prod-served endpoint is not a new idea here.

Two things are deliberately not offered rather than offered and broken. A tier
the server marks unavailable gets no button, because no Stripe price is
configured for it and buying would 422. And no button appears at all unless a
billing extension has set `window.team_billing_ui` -- the prices still render,
which is useful on its own, but nothing invites a click nobody can handle.
Checkout is Stripe's, so the button only dispatches `team-plan-purchase` with
the team and item id and lets the extension take it from there.

The markup is a helper rather than another branch in TabTeams, matching
teamBadge/credits/usageBudget, so it can be tested without the window stack.

Falsified: offering an unavailable tier fails "offers no button for a tier with
no configured price"; rendering buttons regardless of the extension fails
"offers no button without a billing extension to act on it".

342 GUI/SDK tests, 136 team backend tests, typecheck clean.
2026-09-10 17:42:51 -04:00
Daniel Salazar 367d20a55c test: kv tests (#3844)
* test: kv tests

* chore: bump deps
2026-09-10 14:35:46 -07:00
Juan Castro e886e86cb5 feat: size a team by whether its owner pays, and halve a free seat's allowance
Three related limits.

Seats per team now depend on the owner's plan: 4 free, 40 paid, decided by
`subscriptionSatisfies(id, true)` -- which is `!FREE_SUBSCRIPTION_IDS.has(id)`,
so a plan an extension adds counts as paid without core knowing its name. The
existing `max_seats_per_team` still overrides both, so a deployment that already
set it keeps what it asked for, and `max_seats_per_team_free` / `_paid` tune
each. An unreadable plan takes the smaller cap: over-provisioning a free team is
the worse failure.

A seat of a team that pays for nothing resolves `org_seat_free`, half the
registered free plan. Without it a team is a way to mint free tiers -- provision
four seats and each arrives with a full free allowance nobody paid for. The
figures are derived from `REGISTERED_USER_FREE` rather than restated, so the two
cannot drift, and the id joins `FREE_SUBSCRIPTION_IDS` because nobody paid for
it either and it must not satisfy a plan gate.

It is a *default* resolver, so a paid team tier -- which only prod knows about,
through `registerSubscriptionResolver` -- still outranks it. The lookup is
`getOrgSeat`, already cached with its negatives, because almost nothing is a
seat.

Found while testing: the suite was reading `max_seats_per_team` out of the
developer's own config.json, so the cap under test was whatever that file said.
It would have passed here and failed in CI, which has no such file. The suite
now pins the value and the cap tests set their own.

Falsified three ways, each breaking only its own test: equal caps fails "lets a
paid owner past four"; a resolver returning null, and an unhalved allowance,
both fail "resolves half the free plan".

186 team/whoami tests, 159 metering tests, typecheck clean.
2026-09-10 17:04:43 -04:00
Juan Castro 106978e714 feat: tell a seat which team its account belongs to
A provisioned account had no way to know it was one. That matters: the team can
reset its password and close it, which is exactly what the account-created email
already warns about, and nothing in the product repeated it afterwards.

`whoami` now carries `team: { uid, name }`. Two gates on it. Only user actors --
a seat's employer is no more an app's business than its phone number, which the
same handler already withholds. And only where `teams_enabled` is on, so a
deployment without teams is byte-identical.

It rides whoami rather than a route of its own because the sidebar needs it at
first paint. A `/teams/whoami` would add a request to every page load for every
user, and almost none of them are seats. The lookup costs nothing either way:
`getOrgSeat` is already cached, negative results included, precisely because
almost nothing is a seat. `team_name` comes off a join the query already made.

In the sidebar it sits under the Puter wordmark -- the conventional slot for
workspace context -- as a muted second line, hidden when the sidebar collapses.
Owners see nothing: they already know, and one may own several teams, so there
would be no single name to show.

The markup is a helper rather than another branch inside UIDashboard, matching
how appGroups/credits/usageBudget were pulled out, so it can be tested without
mocking the window stack.

Falsified: dropping the `isUser` gate fails "withholds it from an app actor" and
nothing else.

181 backend tests, 331 GUI/SDK tests, typecheck clean.
2026-09-10 16:52:10 -04:00
Juan Castro fb74789de1 feat: email a seat its username and temporary password, when an address is given
Follows the previous commit. Dropping the email field entirely went one step too
far: without an address the temporary password shown once in the panel is the
only copy, and an admin who closes that panel has to issue a new one. The field
is back, marked optional, and now it buys something concrete.

`team_account_created` carried no credential -- it said the team "will send you
a temporary password separately". It now carries the username and the temporary
password, so an admin who supplies an address hands nothing over by side channel.

`#notifyUser` takes extra template variables, and both credential-issuing paths
pass the one they just minted: provisioning and re-issue. Re-issue passing the
fresh credential rather than the stale one is the case worth checking, and there
is a test that asserts the old password is absent from that mail.

With no address nothing is sent, which was already true -- `#notifyUser` returns
early without one -- and is now covered.

The docs said the notice carries no credential in two places. Both corrected.

Falsified: dropping the credential from the re-issue call fails "emails the fresh
credential on re-issue, not the old one" and nothing else.

178 backend tests, 326 GUI/SDK tests, typecheck clean.
2026-09-10 16:41:03 -04:00
Juan Castro a983969f32 feat: prompt a team seat to choose its own password on first sign-in
The backend already refused every route with `password_change_required` until a
provisioned account replaced the password its administrator chose, and
`/user-protected/change-password` was already exempt so the account could act.
The client half was missing entirely: nothing in the GUI referenced that code,
so a seat signed in and then failed at everything with no prompt and no way out.

Two gaps, both closed here.

The flag never reached the client. Neither the login response nor `/whoami`
carried `requires_password_change`, so the GUI could not have known even if it
wanted to. It ships from both now, alongside the other three verification flags
the whoami extension already describes as "the flags the GUI acts on".

There was no window to show. `UIWindowChangePassword` is a settings dialog --
closable, and it never resolves on success -- so it cannot act as a gate.
`UIWindowPasswordChangeRequired` mirrors the existing `*Required` windows: it
resolves true only once the change lands, and `initgui` loops on it. It runs
last in the boot chain, matching the server order in assertVerifiedAccount.

It refuses a new password equal to the current one. Without that the account
stays on the credential its administrator still holds, which is the entire thing
the gate exists to end.

Falsified twice: dropping the same-password guard fails "refuses to reuse the
password the admin handed over", and resolving on a rejected response fails
"stays open on a rejected change, so the gate cannot be escaped" -- each
breaking only its own test.

175 backend tests, 326 GUI/SDK tests, typecheck clean.
2026-09-10 16:35:05 -04:00
Juan Castro b19b312e30 feat: a team seat needs no email address, and is never asked to confirm one
PUT-1792. Two separate problems, both from treating a provisioned account like
a self-registered one.

`email` was required, so an admin creating ten seats had to invent ten addresses
and then keep track of ten uniqueness constraints -- for accounts that sign in
by username and never use the address. It is now optional at every layer, and
the add-account form does not ask for it at all: username is the only thing a
seat needs.

`requires_email_confirmation` was set to true, with the reasoning that an
admin-supplied address is unverified. True, but `requireVerifiedAccount` turns
away on exactly `requires_email_confirmation && !email_confirmed`, so a
freshly created seat was asked to confirm an address it may not hold and could
not use the product until it did. The team creating the account is the trust
anchor, not the mailbox, so this is now false either way.

An address is still accepted and still stored when given, because the notices
are worth delivering. `#notifyUser` already returned early on a missing
address, so `team_account_created`, `team_account_disabled`,
`team_password_reset` and `team_closed` degrade quietly with no new branching --
the temporary password is in the API response, which is the documented delivery.

`idx_user_owned_email` is partial and skips password-null rows, so omitting the
address sidesteps it rather than creating a collision surface. Two seats with no
address do not conflict, and there is a test for it.

Docs now say an emailless seat is recoverable only through its team's owner.
That falls out of the design rather than being a limitation of this change, but
it should be written down rather than discovered.

Falsified: putting `requires_email_confirmation: true` back fails
"never demands confirmation, with or without an address" with
`expected true to be false`, and nothing else.

164 team tests, 40 SDK tests, typecheck clean.
2026-09-10 15:55:50 -04:00
Daniel Salazar a8a78736bb feat: kv list ordering toggle + addressable path improvements (#3843)
* fix: put-1787

* feat: kv list reverse
2026-09-09 17:24:03 -07:00
Juan Fernando Castro 9b465aba9f Merge pull request #3837 from HeyPuter/juancastro/team-count-active-seats
feat: add TeamStore.countActiveSeats for the billable seat count
2026-09-09 19:33:55 -04:00
Daniel Salazar 0edee23dd9 Update CODEOWNERS (#3842) 2026-09-09 15:55:19 -07:00
Daniel Salazar 99f3037110 Update CODEOWNERS (#3841) 2026-09-09 15:53:36 -07:00
Juan Castro 7ab48c590e feat: add TeamStore.countActiveSeats for the billable seat count
Per-account billing needs "how many accounts is this team getting the
benefit of", and neither existing count answers it. `countSeats` includes
suspended seats, because it bounds the seat cap and a suspended seat still
holds its username. `countPayers` counts the owner. Billing off `countSeats`
overcharges for suspended accounts, contradicting what the admin console
tells the owner: "0 suspended accounts cost nothing per account."

So this is a third, separate count rather than a change to either — the cap
must keep counting suspended seats.

An unactivated seat is deliberately still counted: it holds a temporary
password, not a suspension, and it is paid for. That is why the filter is
narrower than `listDirectory`'s, which also excludes
`requires_password_change`.

Falsified: dropping the suspended clause fails exactly one test
("drops a suspended seat from the active count but not from the cap",
expected 3 to be 2) and leaves the other 34 green.
2026-09-09 18:13:04 -04:00
404oopsandClaude Fable 5.1 6146ab9dd3 feat(ai): refresh txt2vid catalogs, retire Sora, default to Veo 3.1 Lite (#3828)
OpenAI shuts the Sora Videos API down on 2026-09-24 and sora-2 was the
default txt2vid model, so the default moves to Veo 3.1 Lite on Gemini
and the OpenAI video provider goes. While there, the video catalogs are
brought in line with what each vendor serves today, the request options
are unified across providers, and the txt2vid docs are rewritten.

- driver: default provider gemini-video-generation with
  veo-3.1-lite-generate-preview; a request under the generic `ai-video`
  driver name lands on the default instead of the first-registered
  provider; `WIDTHxHEIGHT` sizes map onto tier catalogs by the shorter
  side and fill width/height
- openai video provider, the `openai-video-generation` alias, its
  config template and migration entries, and Together's openai/sora-2*
  rows removed
- gemini: Veo 3.1 Fast rates 10/12/30 cents per second for
  720p/1080p/4K, Veo 3.1 Lite accepts reference images, URL image
  inputs are fetched server-side through the SSRF-guarded fetch
- together: drop nine models retired upstream, add eighteen from the
  live listing; per-second models are estimated from the catalog rate,
  clamped to remaining credit and billed at the cost Together reports
  on the job; tier-sized models take resolution/ratio;
  input_reference/last_frame map onto keyframes; generate_audio is
  forwarded
- byteplus: Seedance 2.5 (dreamina-seedance-2-5-260628) with per-model
  reference-image caps
- util/imageInput: string-level image helpers shared by the image and
  video drivers; ai-image/inputImage re-exports them unchanged
- puter.js types: provider and generate_audio options; docs: txt2vid
  page rewritten with per-provider model tables, unified options and
  four new playground examples

Known follow-up: Veo returns a key-protected Google file URL, so the
default clip cannot be played directly by a browser until the provider
fetches it server-side.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-09 14:56:17 -07:00
404oopsandClaude Fable 5.1 c8f4906af4 fix(ai): keep per-provider failures when every chat route fails (#3836)
When the chat fallback chain is exhausted, the driver already records
each attempt (model, provider, status, code, message, timeout) in the
error's `fields.attempts`, but the alarm keyed on the classified message
alone and the alarm client printed the error at inspect depth 2, so the
log and Slack line read `internal_error:All providers failed` with
nothing about which providers failed or why. Deduped repeats printed
only a count.

- The HTTP alarm gate now attaches an HttpError's `fields` to the alarm
  under a single `details` key. One key can't shadow the gate's own
  request fields, and a repeat from another thrower on a shared id
  replaces it instead of merging into it.
- The chat driver logs one warn line per exhausted chain with the
  completion id, the resolved route, the classified code and the
  attempts as JSON, so every occurrence is greppable by trace even when
  the alarm dedupes it. Chains marked `noAlarm` don't log.
- Docs: `fields` reaches both the client and the alarm, so it has to be
  safe to show the caller.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-09 14:56:11 -07:00
Reynaldi Chernando 435629afc5 Add gpt 2.5 image (#3839) 2026-09-09 14:29:41 -07:00
Juan Fernando Castro e132f765d6 Merge pull request #3745 from HeyPuter/juancastro/put-1736-team-directory
🏗️ PUT-1736: a team directory apps can read, once the team opens it
2026-09-09 16:09:03 -04:00
Juan Castro 4e45fdc172 feat: a team directory apps can read, once the team opens it
Members can already enumerate each other: `/teams/:uid/members` needs a user
actor and nothing more. The only thing this adds is admitting an app actor to
the same names, so an app can offer colleagues without the member driving it.

That is the whole risk, so it is off until the team owner turns it on.
`group.directory_enabled` defaults to 0, and a team that has not opted in
answers 404 rather than 403 -- whether a team has this on is not something an
app should be able to probe for either.

Three things bound what an app sees. The membership tested is always the
person's, never the app's, so an app installed by a member of one team can
never read another's. The page carries username and uuid and nothing else --
no email, activation state, usage or role. And suspended accounts and ones
that never took up their credential are left out, since offering someone who
cannot sign in is noise and their existence is not this list's to disclose.

Activation is the forced-change flag clearing, not the password existing: a
provisioned seat holds its temporary password from birth, so testing
`password IS NOT NULL` would have leaked exactly the accounts meant to be
excluded. A test covers that distinction.

Turning the directory on or off writes an audit row, because it changes who
can read the member list and that is not something a team should be able to
alter silently. Setting it to the value it already has records nothing.

The toggle lives in TabTeams, and turning it on asks for confirmation while
turning it off does not -- one grants access, the other only takes it away.

Closes PUT-1736.
2026-09-09 14:51:05 -04:00
Juan Fernando Castro f6b519874e Merge pull request #3742 from HeyPuter/juancastro/put-1740-1762-workspace-gui
🏗️ PUT-1740 + PUT-1762: the workspace surfaces in the GUI
2026-09-09 14:50:56 -04:00
Reynaldi ChernandoandCopilot Autofix powered by AI 59be4ad237 more recipes (#3819)
* more recipes

* addition

* final

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* styling fix

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-10 01:07:26 +07:00
Juan Castro 9121e66c3f feat: the team surfaces in the GUI
Two places a team becomes visible to a person rather than an API: the
Dashboard tab that administers it, and the share dialog that shares with it.

Squashed because they are one change to a reviewer. Both are the first GUI
consumers of `puter.teams`, both add strings to the same `en.js`, and both had
to answer the same question -- what a team looks like to someone who has
never seen one. Splitting them would mean reviewing that answer twice and
resolving the same translation conflict twice.

TabTeams (PUT-1740)

  Registered in `builtinTabs` after `TabFiles`, as a plain object with
  `html()` and `init($el_window)` like every other tab. The member table
  carries the operations the API allows and nothing it does not: provision,
  resend an activation, disable, restore, delete.

  Deleting asks for confirmation naming the account, because the API deletes
  on a single call -- PUT-1732 places the reversible step at disable, not in
  front of the request, so the client is where a confirmation belongs.

  The wording says plainly that disabling stops the per-account charge and
  keeps the storage one, and that deleting is the only thing that ends it.
  This is the one place a person decides between the two, so leaving them to
  infer the difference costs them money.

  Rendering and the billing arithmetic are pure functions in
  `teamsConsole.js`, with tests beside them; the tab file is the DOM.

Share dialog (PUT-1762)

  `UIWindowShare` gains the teams the caller owns as recipients. A
  team is one entry in the list, not its members expanded -- sharing
  reaches every member, and the per-person wording would understate what the
  grant does.

  `shareWorkspaces.js` holds the resolution and the deduplication, so the
  dialog does not gain a second source of truth about who a recipient is.

Closes PUT-1740 and PUT-1762.
2026-09-09 11:51:44 -04:00
Juan Castro afb5b0d82a feat: add the puter.teams SDK module
Team administration reaches the backend through the `/teams` routes
rather than a driver interface, because the gates it needs are route-level:
a user actor, a verified account, and a dual-window rate limit.

The module follows `apps/` and `perms/` in layout -- one file per method,
a thin `index.js`, a JSDoc-only `types.js`, and the `METHODS` rebinding so a
destructured method keeps its `this`. It does not follow `perms/lib/req.js`:
those endpoints resolve `{ error: true }` for backward compatibility, and
nothing here has callers to keep compatible, so this throws `PuterJSError`
with the backend's own code.

Every method takes a team `uid`. A handle is a mutable label that
deleting the team releases, so a stored handle can later resolve to a
different team.

The list methods offer the three forms `puter.apps.list()` does -- an array
by default, the page envelope under `cursor`/`includeTotal`, an async
iterator under `stream`. They refuse `offset`: these routes are keyset-only
and would otherwise return page one however far you asked to skip.

The surface covers the routes that exist today. Usage totals and member-email
correction have no backend route yet and are deliberately absent rather than
shipped as methods that 404. `deleteMember()` is here because the route it
needs lands in the commit below this one.
2026-09-09 11:51:44 -04:00
Juan Castro a60a4a98b4 feat: delete a team seat for good, once it is disabled
A disabled account persists indefinitely. Removing it is an explicit request,
never a timer, and it is refused on a live account with
`account_must_be_disabled_first` — which puts a reversible step in front of the
only irreversible operation in the feature.

The audit row is written before `cascadeDelete` runs. The FKs are ON DELETE SET
NULL and the `_keep` columns carry the identifiers, so the record of what was
done survives the account it names.

No second billing emit here: `cascadeDelete` already captures the seat and
fires `team.account.deleted` through UserAccountService, and emitting again
would close the storage charge twice. Disabling closed the per-account charge;
this closes the storage one, and it is the only thing that does.

There is no restore window, and none was wanted: the reversible step already
exists earlier at disable, a disabled account costs only the bytes it holds so
nothing pressures a hasty delete, and a restore promise means retaining data
the team explicitly asked to be rid of.

Published in rate-limits-and-quotas.md alongside the team-deletion note,
since the two are easy to confuse and only one of them frees a seat.

Closes PUT-1732.
2026-09-09 11:51:44 -04:00
Juan Castro 81d15f412b feat: tell a team member what was done to their account
Two halves of the same question -- what did the team do to me, and how do
I find out. One is pull, the other push.

The activity view (PUT-1746) was already user-scoped and already 404'd a
non-member; what was missing is the load-bearing half. A member is told a reset
happened, but a reset only matters alongside who signed in afterwards, so
`SessionStore.listSignIns` merges their own sign-ins into the same stream,
newest first, with a per-stream cursor. Without that row the audit says a
credential was issued and never says whether it was used.

The notices (PUT-1733) cover the two things a member cannot discover for
themselves: their account being disabled, and their team being closed.
Deliberately one each -- closing a team disables every account in it, so
sending both would tell one person twice about one event. Both say plainly that
nothing was deleted; the accounts persist, suspended, holding their files.

Squashed because they are one change to a reviewer: same audience, same
purpose, seven files, overlapping only in TeamService. Neither touches shared
platform code.
2026-09-09 11:51:44 -04:00
Juan Castro f5b2365e83 feat: enforce the forced password change on team seats
`user.requires_password_change` shipped with the team columns but nothing
enforced it and nothing ever cleared it, so a provisioned seat kept its
administrator-issued password indefinitely and `reissueCredential`'s
"already activated" 409 was unreachable.

Adds the fourth clause to `assertVerifiedAccount`, the only place a
verification gate may live -- WebDAV builds its own actor and calls that
function directly, so a second implementation would bypass it the way the
phone and card gates once were bypassed.

A gate that refuses everything also refuses the endpoint that clears it,
so `/user-protected/change-password` opts out with `allowUnconfirmed`.
That widens the route: an account pending email, phone or card
verification can now change its password, which it could not before. The
caller is authenticated and proves the current password, so this is
benign, but it is a behaviour change to a shared route.

Also here, because the gate is worthless without them:

- change-password and the recovery-token path clear the flag, and record
  an `activate` entry when the account is a seat.
- Reset takes a live account back with a fresh credential, capped at 20
  per day and audited as `reset_member_password` with no credential in
  the row. Re-issue is audited the same way; it stays closed once a seat
  has chosen its own password.
- An issued credential expires after 24h (new `temp_password_expires_at`
  column, three dialects) and login refuses it after that, so an unused
  reset dies instead of becoming a standing credential.
- 2FA is untouched by a reset, so a reset alone is not takeover.
2026-09-09 11:51:44 -04:00
Juan Fernando Castro f3943f6a55 Merge pull request #3729 from HeyPuter/juancastro/put-1726-put-1727-put-1729-team-share-recipient
🏗️ PUT-1726 + PUT-1727 + PUT-1729: share with a workspace
2026-09-09 10:41:21 -04:00
Juan Castro 7a02fa7741 feat: tell every team member when something is shared with them
`notifyShared` keyed every map off `ResolvedShare.holderId`. A team share
has no individual holder -- the grant is one row against the group and
`holder_user_id` is NULL -- so it was skipped everywhere and the call returned
having told nobody. Nothing errored: the share was created, resolved and listed
correctly. The failure mode was silence.

`#recipientsOf` now expands a team share to its live members at
announcement time, so the grant stays one row and only the telling fans out.
`#announce` and `#emailHolder` are both inside the per-recipient loop, so
members get the in-app notification and the email digest.

Bounded by `NOTIFY_FANOUT_CAP`, which logs when it bites rather than truncating
silently -- a shortened fan-out otherwise reads as "everyone knows". The
existing per-recipient budgets absorb the volume from there.

Wording is unchanged: "shared N items with you", not the team name.

A member who joins later resolves the grant through the scan but is not
retroactively told; announcements describe a moment, not a state.
2026-09-09 10:22:56 -04:00
Juan Castro da68074401 feat: share with a team
Covers PUT-1726, PUT-1727 and PUT-1729. Together because a recipient without
the listing work produces a share that is created correctly, resolves
correctly, and never appears in "Shared with me" — a state nobody would ship.

The recipient. `ShareRecipient` gains `team` (uid) and `teamHandle`, resolved
before the email and username branches and never falling through to them.
Passing both is an error rather than a precedence rule, so a call site always
shows which was chosen: handles are released on soft delete and can be
reclaimed by an unrelated team, and a scripted share to a handle would
silently retarget. There is no bare-string spelling, since that would change
how existing strings are interpreted.

ACLService.setUserGroup mirrors setUserUser: same read-modify-write, same
one-mode-per-node rule, under a node lock keyed on the group.

One grant against the team, not one per member, so membership changes
apply without touching the grant and a share spends one unit of the daily
quota however many members there are. A member who joins afterwards gets
access, which is asserted.

The index row carries `holder_group_id` and leaves `holder_user_id` NULL, so
`0077`'s group index constrains it rather than the user-holder one.

Listing. `listByHolder` and `countByHolder` union the caller's teams into
the same keyset page — `ORDER BY id` still holds — and `#grantEvidence` gains
group grants as a third source. Without that third source the share is
filtered out of every listing as dead: nothing errors, the share simply is not
there, which is the one place a user would look for it.

Unsharing revokes the grant as well as deleting the index row. Deleting the
row alone would hide the share while leaving every member holding real access.

Closes PUT-1726, PUT-1727 and PUT-1729.
2026-09-09 10:22:56 -04:00
Juan Castro 9f615cc632 feat: add the two-team fixture and group permission grants
Two tickets, together because the fixture has no behaviour of its own — the
grant tests are its first real use, and "a grant to team A must not
resolve for B's members" is exactly a two-team assertion.

PUT-1719. Every query resolving team-scoped data has to take the
team as a parameter rather than infer it. One that forgets returns the
right rows against a database holding a single team, so a one-team
fixture does not give weaker coverage — it gives false confidence.

The fixture provides two teams, each with its own master and two
activated seats, plus a user in neither. Each team having its own master
is also what the shipped `max_teams_per_user: 1` requires, so it runs
against the real cap rather than lifting it the way the existing team suites
have to. Seats are activated before use: provisioning leaves
`requires_email_confirmation` set and `requireVerified` rejects them outright,
so an unactivated seat cannot call anything and every authorization assertion
built on one would be vacuous.

PUT-1725. The write half of group grants; the read half already worked, since
`#scanUserGroup` joins `jct_user_group` itself and resolves for whoever is a
member at scan time.

  PermissionStore   resolveGroupId, listGroupMemberUuids,
                    upsertUserGroupPerm, deleteUserGroupPerm,
                    auditUserGroupPerm
  PermissionService grantUserGroupPermission, revokeUserGroupPermission

Both rewrite the permission first, so `fs:/path:mode` collapses to
`fs:<uuid>:mode` and a revoke names the same string the grant wrote; both gate
on `canManagePermission`; both audit into the existing
`audit_user_to_group_permissions`.

The delete is scoped to the issuer, so one issuer's revoke cannot drop
another's identical grant — the PK is (user_id, group_id, permission) and
user_id is the issuer.

Cache invalidation goes through `bumpCacheGenerations` with every member uuid
at once. That already announces to peer regions in a single event, so the
fan-out costs one broadcast rather than one per member.

There is no flat-KV equivalent for group grants, so unlike the user-to-user
path there is no second write to keep consistent and no second invalidation
surface.

Closes PUT-1719 and PUT-1725.
2026-09-09 10:22:56 -04:00
Daniel Salazar 8a70fa2e5c fix: CODEOWNERS + perm dialogs outside puter dashboard/desktop (#3833) 2026-09-08 22:56:03 -07:00
Felix-Ayush 9bb5bffd01 fix(puter-js): account for base64 padding in estimateDataUrlSize (#3805)
Math.ceil(len * 3 / 4) over-counts by 1-2 bytes on padded payloads, so thumbnails at the size limit were wrongly rejected. Subtract padding like ai dataUriByteLength.
2026-09-08 22:33:33 -07:00
Felix-Ayush d7877359d4 fix(puter-js): return key_undefined for missing kv keys (#3803)
get/del/incr/decr/add/expire/expireAt only checked size, so undefined keys threw TypeError. Add assertKeyPresent first, matching set/update/remove.
2026-09-08 22:32:53 -07:00
Felix-Ayush cdc524f504 fix(puter-js): preserve zero amount in kv incr/decr (#3802)
parseCounterArgs used a falsy check, so incr(key, 0) sent amount 1. Only default when amount is undefined/null.
2026-09-08 22:30:02 -07:00
dependabot[bot] 8b872545dc chore(deps): bump docker/setup-buildx-action from 3 to 4 (#3823)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:27:06 -07:00
dependabot[bot] 52f9fa57a6 chore(deps): bump docker/metadata-action from 5 to 6 (#3822)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:26:49 -07:00
dependabot[bot] eb28012897 chore(deps): bump peter-evans/dockerhub-description from 4 to 5 (#3826)
Bumps [peter-evans/dockerhub-description](https://github.com/peter-evans/dockerhub-description) from 4 to 5.
- [Release notes](https://github.com/peter-evans/dockerhub-description/releases)
- [Commits](https://github.com/peter-evans/dockerhub-description/compare/v4...v5)

---
updated-dependencies:
- dependency-name: peter-evans/dockerhub-description
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 22:26:27 -07:00
Daniel Salazar e5f322f68c fix: misc event hardening + perm fixes (#3831) 2026-09-08 19:16:21 -07:00
Neal Shah ab3bba5525 add file query param (#3829) 2026-09-08 21:39:19 -04:00
Juan Fernando Castro 9f3c658975 Merge pull request #3830 from HeyPuter/juancastro/phase3-land-on-main
Teams phase 3: land the remaining three PRs on main (#3733, #3724, #3725)
2026-09-08 18:27:11 -04:00
Juan Castro bd19fd18ec feat: cap teams per user and seats per team
A seat is a real Puter account: it takes a name from the global username pool
and gets a home directory. Nothing charges for one — that is prod's job — so
until it does, the only bound on creation is the request rate limit, which
bounds the rate and not the total.

  max_teams_per_user   default 1
  max_seats_per_team   default 50

Ordering is the substance of both checks. The team cap is tested before
the handle, so a capped user is told they are capped rather than that the name
they picked was unusable. The seat cap is tested before any account state
exists, so a refused provision does not burn a global username.

Soft-deleted teams do not count toward the owner's cap, so deleting frees
the slot — which does mean create, provision, delete, repeat still consumes
usernames over time, bounded by the daily rate limit. The caps raise the cost
and make the cycle audited; they do not close it.

Lowering the seat limit blocks new provisioning and disables nobody.

Both limits are published in rate-limits-and-quotas.md, and both keys are
documented in config.template.jsonc and config.default.json.

Closes PUT-1758.
2026-09-08 16:59:29 -04:00
Juan Castro 41a05b68fd feat: announce credit-state transitions so billing can act on them
A seat that exhausts its allowance is refused and cannot fix it itself — only
the team owner can add capacity. Something has to tell the owner, and that
something lives in prod, so OSS's job is to say when it happened.

`MeteringService` emits `metering.credit-state` from `rememberRemainingCredits`,
the single point where the verdict is computed, at 90% of the allowance and
again on exhaustion.

Emitted on a transition, never per request. A blocked account keeps trying and
every retry recomputes the verdict, so without that a retry loop would announce
hundreds of times. The state is tracked per uuid in process, under the same FIFO
bound as the credit cache, and dropped in `#dropCachedCredits` — so added
capacity re-arms the alert for the rest of the month.

The dedup that turns this into exactly one notice per member per month belongs
with whoever sends the mail, and needs a cross-region marker rather than a
per-cluster one.

Part of PUT-1750; the notification half is prod's.
2026-09-08 16:59:29 -04:00
Juan Castro 0c6045ed90 feat: emit team billing events for the payment integration
The charge lives outside this repo, the way the marketplace extension cancels
Stripe subscriptions off `user.delete`. This is the trigger, and it is the
whole of what OSS owes billing.

  team.account.created    a seat exists and can be used
  team.account.disabled   it stopped, and still holds its bytes
  team.account.enabled    it resumed
  team.account.deleted    it is gone
  team.deleted            the team is gone; its accounts are not

Each carries the team uid, the affected account, and the owner's
`stripe_customer_id`. That column ships in the mysql and postgres schemas but
not sqlite, so the read is guarded and degrades to null, as `cascadeDelete`
already does.

Deleting a team emits one `team.account.disabled` per seat plus the
team event, rather than one bulk event: the accounts persist, suspended,
holding their files and their usernames. Deleting a team is not a way to
stop paying for the accounts in it.

`team.account.deleted` is captured before the row is deleted.
`jct_user_group.user_id` is ON DELETE CASCADE, so by the time a listener on
`user.delete` runs, nothing can say which team paid for the account.

`getOrgSeat` deliberately admits soft-deleted teams: their accounts still
exist, so the charge is still running.

Closes PUT-1712.
2026-09-08 16:59:28 -04:00
Juan Fernando Castro 00afea1285 feat: a seat is created, never adopted (#3720)
`addMember` refuses to turn an account that already has a password into a
team seat. No service path did this — `provisionAccount` always creates —
but the store permitted it, and the design rules out existing accounts joining
a team.

Provisioning passes the guard because it admits the account before setting its
temporary password.

There is no bypass parameter. Both HTTP suites now provision a real seat and
authenticate as it, using the same token-minting the harness uses for
`POST /login`. That surfaced something worth knowing: an unactivated seat
cannot call the API at all. Provisioning leaves `requires_email_confirmation`
set and `requireVerified` rejects it, so the suites activate the seat first —
which is the state a member is actually in when making requests.

`listMembers` and `getMembership` also return `u.uuid`, which the billing
events need in order to name the account without a second lookup.
2026-09-08 16:45:16 -04:00
dependabot[bot] 84368f8d02 chore(deps): bump otpauth from 9.5.1 to 9.5.2 (#3820)
Bumps [otpauth](https://github.com/hectorm/otpauth) from 9.5.1 to 9.5.2.
- [Release notes](https://github.com/hectorm/otpauth/releases)
- [Commits](https://github.com/hectorm/otpauth/compare/v9.5.1...v9.5.2)

---
updated-dependencies:
- dependency-name: otpauth
  dependency-version: 9.5.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 13:26:41 -04:00
Daniel Salazar 86050f131b fix: hardening events for shared kv (#3827) 2026-09-08 09:04:50 -07:00
dependabot[bot] e50f2911b6 chore(deps): bump actions/upload-artifact from 4 to 7 (#3824)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:58:37 -07:00
dependabot[bot] 414c121ae4 chore(deps): bump actions/setup-node from 4 to 7 (#3825)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 7.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:58:24 -07:00
Toshit ChawdaandClaude Opus 5 1d6d0da950 feat(puter.js): transferables in AppConnection.postMessage (#3776)
`postMessage(message, transfer)` (or `{ transfer }`) moves ArrayBuffers,
MessagePorts, streams and the rest to the target app instead of copying
them. The second argument is optional, so existing callers are unchanged.

The transfer list rides in the message body as well as the real transfer
list: structured clone's memory map keeps those objects identical to the
ones inside `contents`, which is how the desktop picks them out mid-relay
and keeps transferring them onward rather than leaving copies behind. Both
`messageToApp` paths carry it — the direct iframe relay and the connection
path that `launchApp()` between apps actually uses.

The desktop forwards the list as-is and lets postMessage judge it. A list
that arrived through the SDK was already validated by the browser on the
first hop, and validating again here would mean an allowlist of
transferable types that silently downgrades an unrecognised one to a copy.
A bad list only reaches us from an app that hand-wrote the envelope; that
throws without detaching anything, and is caught so it cannot escape
`ipc_listener` as an unhandled rejection.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-09-08 08:57:52 -07:00
dependabot[bot] 72889d72c8 chore(deps): bump undici from 7.29.0 to 7.29.1 (#3821)
Bumps [undici](https://github.com/nodejs/undici) from 7.29.0 to 7.29.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.29.0...v7.29.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.29.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 08:52:50 -07:00
Nariman Jelveh 8a6daee9ab feat: let extensions customize recommended apps 2026-09-07 16:44:41 -07:00
404oops e980b131a0 Merge pull request #3815 from HeyPuter/filipkujundzic/ai-vision-canonical-media-parts 2026-09-08 00:54:41 +02:00
404oopsandClaude Fable 5.1 1cb5472c4c fix(ai): canonicalize image parts once and translate them per provider
The SDK shorthand sends `{ image_url: { url } }` with no type. Claude
forwarded it untouched (Anthropic: "content.1.type: Field required"),
the OpenAI/Azure Responses providers sent a Chat Completions part the
Responses API rejects, and Mistral emitted snake_case `image_url` where
its SDK validates `imageUrl`. All three only appeared to work because
the fallback loop re-served them through OpenRouter.

- utils/mediaParts: canonicalise every inbound media part (untyped
  shorthand, bare string URLs, Responses input_image, Anthropic image
  blocks, Gemini inline_data) to the Chat Completions form in the
  driver, so providers translate from one shape
- Claude → Anthropic image blocks (url / base64 sources), copy-on-write
- Responses processor → input_image with a string URL and detail, and
  copies messages instead of mutating the caller's objects
- Mistral → camelCase imageUrl; the test that asserted the old shape
  is corrected
- driver: 400 up front when the catalog says the model has no image or
  video input; resolve puter_path parts per attempt for providers
  without their own upload path (Claude keeps the Files API)
- move the Moonshot http→data-URL inliner to utils/inlineImages and
  apply it to Grok-via-Azure and Gemini, whose upstreams cannot fetch
  http image URLs

Fixes #3409

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-07 15:23:40 +02:00
Daniel Salazar 927317bc4e fix: events hardening (#3814) 2026-09-06 22:44:07 -07:00
Daniel Salazar a441e7f748 fix: harden events (#3813)
* fix(events): rename carries from, one retry on deploy-timeout, forward-path counters

- FSService.rename passes the pre-rename path so a folder subscription
  sees op move with from, like a real move does
- deploy-timeout from the dispatcher gets one retry with the deployed
  header and no second upload (ALREADY_DEPLOYED_MISS_REASONS)
- OTel counters events.forward.sent/received and events.single.attempt
- cross-app KV subscribe error hints at the three-segment parse
- docs: lease is 60 s, kv prefix example is fully qualified, move covers
  rename

* feat(events): forward session subscriptions across regions, fast bumps, worker session cleanup

- session (onLocal) subscriptions now receive writes committed in other
  regions: a transition-maintained remote-watch index (ev:sc / ev:rw),
  watch/event forward items, replay through dispatchForwarded against
  session rows only; events.forwardSession=false is the kill switch
- subscription and presence generation bumps also ride the addressed
  forward channel (kind bump) so a peer sees a new durable row within
  a queue window; the webhook fan stays as backstop
- workers.destroy revokes every holder's events:handlers session; app
  deletion reaps the app's rows, backlog and handlers and revokes the
  sessions; an hourly sweep revokes sessions whose app is gone
- docs: cross-region latency, per-region caps footnote, session
  lifecycle
2026-09-06 16:44:07 -07:00
dependabot[bot] 2784feb47d chore(deps): bump fast-uri from 3.1.5 to 3.1.7 (#3810)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.5 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.5...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-06 16:43:02 -07:00
Daniel Salazar 2f70cd1a14 feat(email): sendTransactional alias and streaming file inputs (#3812)
- puter.email.sendTransactional is the new name; send stays as a
  deprecated alias with the same arguments and result
- fileInput: openFileInputStream / resolveFileInputEntry expose the
  ACL-checked FS read as a stream; loadFileInput wraps them
- EmailAttachment accepts a `path` the transport streams on its own
2026-09-06 16:41:02 -07:00
Felix-Ayush b1d3a7c191 fix(puter-js): require prompt in ai.txt2img (#3804)
txt2vid rejects missing prompts client-side with prompt_required, but txt2img sent the request to the backend. Add the same guard before the driver call.
2026-09-06 16:39:35 -07:00
Nariman Jelveh 092a9245ff Fix PDF thumbnail worker loading from CDN bundles 2026-09-06 14:31:42 -07:00
Nariman Jelveh e6cedde732 Add resilient PDF thumbnails to GUI uploads (#3809)
* Add resilient PDF thumbnails to GUI uploads

* Keep GUI image thumbnails working with SDKs lacking the callback context

The desktop loads puter.js from js.puter.com by default, and the SDK
there predates the thumbnail callback context, so passing the PDF
generator made every image upload lose its thumbnail until the SDK
deploys. Fall back to the SDK's bundled image generator whenever the
running SDK passes no usable context, and cover both paths in the unit
and browser tests.

* Ignore preparation failures that land after an upload is cancelled

Cancelling during preparation already rejects the upload and fires the
abort callback. If the step that was in flight then fails, such as a
dropped directory that cannot be read, the error callback also fired and
the GUI showed an upload error for an upload the user had just
cancelled. Skip error reporting once preparation has been aborted.

* Give each PDF thumbnail worker four seconds

The per-PDF budget covers downloading PDF.js as well as rendering, and
the first PDF of a session on a slower connection ran out of time
before its assets had even loaded. Four seconds fits that first load on
ordinary connections while staying under the five-second batch cap, so
one stuck PDF still leaves the rest of the batch a chance.
2026-09-06 13:51:02 -07:00
404oops 9b4d8f11d4 Merge pull request #3801 from HeyPuter/filipkujundzic/remove-retired-imagen-moonshot-models
fix(ai): remove retired Imagen 4 and legacy Moonshot models
2026-09-06 13:26:14 +02:00
dependabot[bot] bf4e810f9c chore(deps-dev): bump browserstack-local (#3808)
Bumps [browserstack-local](https://github.com/browserstack/browserstack-local-nodejs) from 1.5.5 to 1.5.9.
- [Release notes](https://github.com/browserstack/browserstack-local-nodejs/releases)
- [Commits](https://github.com/browserstack/browserstack-local-nodejs/compare/v1.5.5...v1.5.9)

---
updated-dependencies:
- dependency-name: browserstack-local
  dependency-version: 1.5.9
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:43:42 -07:00
dependabot[bot] 89a4947f68 chore(deps): bump webpack from 5.101.3 to 5.110.3 in /src/puter-js (#3800)
Bumps [webpack](https://github.com/webpack/webpack) from 5.101.3 to 5.110.3.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Changelog](https://github.com/webpack/webpack/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack/compare/v5.101.3...v5.110.3)

---
updated-dependencies:
- dependency-name: webpack
  dependency-version: 5.110.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:47 -07:00
dependabot[bot] 9ebe1092f1 chore(deps): bump dependabot/fetch-metadata from 2 to 3 (#3763)
Bumps [dependabot/fetch-metadata](https://github.com/dependabot/fetch-metadata) from 2 to 3.
- [Release notes](https://github.com/dependabot/fetch-metadata/releases)
- [Commits](https://github.com/dependabot/fetch-metadata/compare/v2...v3)

---
updated-dependencies:
- dependency-name: dependabot/fetch-metadata
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:22 -07:00
dependabot[bot] 1caf3e121f chore(deps): bump actions/checkout from 3 to 7 (#3764)
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:40:13 -07:00
dependabot[bot] 3620f7bd85 chore(deps): bump docker/build-push-action from 5 to 7 (#3765)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:39:57 -07:00
dependabot[bot] e48802252e chore(deps): bump actions/download-artifact from 4 to 8 (#3767)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 15:39:21 -07:00
Daniel Salazar da65b7f569 feat: the invoking backend deploys an events worker the dispatcher cannot find (#3807)
The dispatcher's rehydrate callback reaches whichever backend answers the
API's public hostname. A backend with the runtime flag on that is not behind
that hostname, or the only one in the fleet with it on, could never get its
scripts deployed that way — the callback answered "disabled". The invoking
backend already knows the app and script, so on a dispatcher miss it deploys
the set itself and retries once, telling the dispatcher to skip its callback
and negative cache. The callback stays the path for evicted scripts.
2026-09-05 14:40:44 -07:00
404oopsandClaude Fable 5.1 ded3d3695b fix(ai): remove retired Imagen 4 and legacy Moonshot models
Google shut down imagen-4.0-fast/standard/ultra in the Gemini API on
2026-08-17; live calls now return 404 "not found ... or is not supported
for predict". Drop the three entries, the generateImages code path that
only they used, and their tests; the integration test moves to
gemini-2.5-flash-image.

Moonshot's live /models listing and the Kimi pricing docs now carry only
kimi-k3, kimi-k2.7-code(-highspeed) and kimi-k2.6. Drop kimi-k2.5 and the
whole moonshot-v1-* family and repoint the unit and integration tests at
current models. kimi-k2.5 still resolves through OpenRouter.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-05 22:22:16 +02:00
dependabot[bot] e451d507ad chore(deps): bump lodash from 4.17.21 to 4.18.1 in /src/puter-js (#3797)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.21 to 4.18.1.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.21...4.18.1)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.18.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 12:21:36 -07:00
dependabot[bot] 7099cadc6f chore(deps): bump shell-quote from 1.8.1 to 1.10.0 in /src/puter-js (#3774)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.8.1 to 1.10.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/shell-quote/compare/v1.8.1...v1.10.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.10.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 12:19:35 -07:00
dependabot[bot] 73cb60f5ae chore(deps): bump docker/login-action from 3 to 4 (#3762)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:58:57 -07:00
Daniel Salazar e478087260 fix: keep matrix legs of test/coverage jobs always instantiated (#3799)
Skipping the whole matrix job via a job-level `if` collapses it to a
single check with the unevaluated name template ("test (\${{
matrix.artifact }})") instead of expanding to "test (base)"/"test (pr)",
since there's no matrix context to fill in when the job never runs. That
check never matches a required-status-check pinned to "test (pr)", so
PRs that skip backend/puterjs testing block forever waiting on a check
that will never be reported. Gate the steps instead so the matrix always
expands with stable names, doing no real work when irrelevant.
2026-09-04 23:58:12 -07:00
dependabot[bot] b6fb4f3cae chore(deps-dev): bump @eslint/js from 9.39.4 to 9.39.5 (#3791)
Bumps [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js) from 9.39.4 to 9.39.5.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/commits/v9.39.5/packages/js)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.39.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:56:03 -07:00
dependabot[bot] 61566431cc chore(deps): bump fs-extra from 11.3.5 to 11.3.6 (#3785)
Bumps [fs-extra](https://github.com/jprichardson/node-fs-extra) from 11.3.5 to 11.3.6.
- [Changelog](https://github.com/jprichardson/node-fs-extra/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jprichardson/node-fs-extra/compare/11.3.5...11.3.6)

---
updated-dependencies:
- dependency-name: fs-extra
  dependency-version: 11.3.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:55:31 -07:00
dependabot[bot] d3baf0ef7e chore(deps): bump js-yaml from 4.3.1 to 4.3.2 (#3792)
Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.3.1 to 4.3.2.
- [Changelog](https://github.com/nodeca/js-yaml/blob/4.3.2/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/4.3.1...4.3.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:54:54 -07:00
dependabot[bot] 4c3fe8ef7b chore(deps-dev): bump @vitest/coverage-v8 from 4.1.10 to 4.1.11 (#3793)
Bumps [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) from 4.1.10 to 4.1.11.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.11/packages/coverage-v8)

---
updated-dependencies:
- dependency-name: "@vitest/coverage-v8"
  dependency-version: 4.1.11
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:53:49 -07:00
dependabot[bot] c164b5724f chore(deps): bump nodemailer from 9.0.3 to 9.0.6 (#3795)
Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 9.0.3 to 9.0.6.
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v9.0.3...v9.0.6)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 9.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:52:48 -07:00
Daniel Salazar bc9cb2d7e7 fix: withdrawing background consent revokes the app's events session (#3777)
* fix: withdrawing background consent revokes the app's events session

A background handler runs as a worker session for the subscriber and app.
Revoking `events:background` or uninstalling the app suspended the
subscriptions but left that session valid, so a token a handler had copied
out kept working until the user found the row in the sessions list. The
revocation settle now revokes the session too; the next consented delivery
mints a fresh one.

* fix: cleanup docs
2026-09-04 21:10:27 -07:00
dependabot[bot] 5669a2d960 chore(deps): bump fastify from 5.10.0 to 5.12.3 (#3749)
Bumps [fastify](https://github.com/fastify/fastify) from 5.10.0 to 5.12.3.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/commits)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 21:01:11 -07:00
dependabot[bot] 50aea0cc64 chore(deps-dev): bump vite from 8.1.3 to 8.1.5 (#3790)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.1.3 to 8.1.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.1.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.1.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:36 +00:00
dependabot[bot] c67f1c3653 chore(deps): bump uuid from 14.0.1 to 14.0.2 (#3789)
Bumps [uuid](https://github.com/uuidjs/uuid) from 14.0.1 to 14.0.2.
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/uuidjs/uuid/compare/v14.0.1...v14.0.2)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 14.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:13 +00:00
dependabot[bot] 78f44c8f8c chore(deps-dev): bump vitest from 4.1.10 to 4.1.11 (#3788)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 4.1.10 to 4.1.11.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.11/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-version: 4.1.11
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:53:03 +00:00
dependabot[bot] 45ee0ef102 chore(deps-dev): bump @types/node from 24.13.2 to 24.13.3 (#3787)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.2 to 24.13.3.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:52:45 +00:00
dependabot[bot] c4da663533 chore(deps): bump sharp from 0.35.3 to 0.35.4 (#3786)
Bumps [sharp](https://github.com/lovell/sharp) from 0.35.3 to 0.35.4.
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.35.3...v0.35.4)

---
updated-dependencies:
- dependency-name: sharp
  dependency-version: 0.35.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:52:31 +00:00
dependabot[bot] 69b5901278 chore(deps): bump mysql2 from 3.22.5 to 3.22.6 (#3784)
Bumps [mysql2](https://github.com/sidorares/node-mysql2) from 3.22.5 to 3.22.6.
- [Release notes](https://github.com/sidorares/node-mysql2/releases)
- [Changelog](https://github.com/sidorares/node-mysql2/blob/master/Changelog.md)
- [Commits](https://github.com/sidorares/node-mysql2/compare/v3.22.5...v3.22.6)

---
updated-dependencies:
- dependency-name: mysql2
  dependency-version: 3.22.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:54 +00:00
dependabot[bot] 99fb4c2dad chore(deps-dev): bump prettier from 3.9.4 to 3.9.6 (#3783)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.4 to 3.9.6.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.4...3.9.6)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:41 +00:00
dependabot[bot] 25b6a80cb5 chore(deps): bump libphonenumber-js from 1.13.6 to 1.13.12 (#3782)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.6 to 1.13.12.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/commits/v1.13.12)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:34 +00:00
dependabot[bot] b334961f75 chore(deps): bump @esbuild/linux-x64 from 0.25.11 to 0.25.12 (#3781)
Bumps [@esbuild/linux-x64](https://github.com/evanw/esbuild) from 0.25.11 to 0.25.12.
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG-2025.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.11...v0.25.12)

---
updated-dependencies:
- dependency-name: "@esbuild/linux-x64"
  dependency-version: 0.25.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:51:10 +00:00
dependabot[bot] aa95a76253 chore(deps): bump @prelude.so/js-sdk from 0.12.0 to 0.12.1 (#3780)
Bumps [@prelude.so/js-sdk](https://github.com/prelude-so/web/tree/HEAD/packages/js-sdk) from 0.12.0 to 0.12.1.
- [Changelog](https://github.com/prelude-so/web/blob/main/packages/js-sdk/CHANGELOG.md)
- [Commits](https://github.com/prelude-so/web/commits/HEAD/packages/js-sdk)

---
updated-dependencies:
- dependency-name: "@prelude.so/js-sdk"
  dependency-version: 0.12.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:50:51 +00:00
dependabot[bot] 85bf595d2d chore(deps): bump highlight.js from 11.11.1 to 11.11.2 (#3779)
Bumps [highlight.js](https://github.com/highlightjs/highlight.js) from 11.11.1 to 11.11.2.
- [Release notes](https://github.com/highlightjs/highlight.js/releases)
- [Changelog](https://github.com/highlightjs/highlight.js/blob/main/CHANGES.md)
- [Commits](https://github.com/highlightjs/highlight.js/compare/11.11.1...11.11.2)

---
updated-dependencies:
- dependency-name: highlight.js
  dependency-version: 11.11.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 03:50:38 +00:00
Daniel Salazar 2f15e40a77 ci: add CODEOWNERS, restrict Dependabot to patch bumps, make test workflows always-run (#3778)
- CODEOWNERS: default owners @Salazareo @ProgrammerIn-wonderland @jelveh
  @jfcastro92, plus @reynaldichernando for src/docs.
- dependabot.yml: ignore semver-minor/major npm updates, patch only.
- backend-tests.yaml / puterjs-tests.yaml: trigger unconditionally and gate
  the real work behind a dorny/paths-filter job instead of a workflow-level
  path filter, so the checks always report (skipped when irrelevant) and
  can be marked required. Also pins matrix job names to the artifact label
  instead of the branch ref, which previously made the check name change
  per PR and unusable as a required check.
2026-09-04 20:49:14 -07:00
Daniel Salazar b784b51cf3 fix: harden the events stack for flag-on (#3752)
* fix: harden events dispatch, single delivery and KV share handles

Dispatch: a filtered subscription used the anchor path stored at subscribe
time, so renaming or moving the anchor folder silently ended its deliveries;
dispatch now resolves the anchor's live path from the event's own ancestor
chain. A move out of a watched folder now reaches that folder's subscribers,
with `from` only for rows that watched the source side. Gap markers are
authorized like deliveries and coalesced per subscription and subject instead
of fanning per lost event. Session subscriptions: the per-socket cap decides
on the write, not before it; an orphaned watched-set token heals on refresh;
durable rows keep their watch window when a session subscribe touches the
same keys. `self` is false when the acting user is unknown.

Single delivery: a subscription in backoff or suspended with a backlog pinned
the sweeper's head and starved everyone behind it — the sweep now defers it.
Only a settled handler run bills a delivery. A socket-only account row no
longer wedges after two attempts nobody received. The lease is twice the
handler timeout; remote candidates have their own attempt counter; the region
depth reconcile runs once a minute region-wide with a bounded scan.

KV share handles: a grantee no longer sees the owner's namespace and absolute
prefix on the subscribe answer or listing, nor in the delivery token; revoking
a wider handle retires the handles it covers; minting the same handle twice
returns the existing one, after the delegation check; a row whose event
cannot be re-based onto its handle is dropped rather than delivered raw.

* fix: presence survives replication, long sessions and region churn

One presence item per (user, app) with per-region map fields lost a region
whenever two regions joined inside the replication window, and nothing ever
put it back. Presence is now one item per (user, app, region): each region
writes only its own, a leave or repair retires it conditionally on its own
write stamp, and a read is a prefix query. Items carry a 48 h ttl refreshed by
a claim-gated write off the existing socket renew path, at most once per
12 h, so a tab that stays connected keeps its region in the row. A region
that answered "no socket" or completed a leave releases a shared pin, so a
reconnect on another node rejoins and a flapping client cannot force a
replicated write per cycle. Cached rows expire after a minute; unaddressable
region names are filtered and pruned; relayed acks settle under a bounded
concurrency; the forward queue is bounded in bytes as well as items.

* feat: indexes for the event_subscriptions hot queries

Handler publish, remove and listing, and the hourly expiry and suspension
sweeps, all scanned `event_subscriptions`. Adds (app_uid, handler_name),
(expires_at) and (suspended_at, id), guarded on every engine. Existing
migrations: the postgres widens are now guarded so a boot does not take an
exclusive lock for a no-op, the kv_share_handles grantee FK gets an index,
the sqlite notification rebuild is transactional and idempotent.

* fix: notification writes go through the registry

The driver's `create` bypassed the type registry, producing uncatalogued
rows with no size bound; it now requires a registered type, caps the payload,
and answers 400 rather than 500 for a bad one. `mark_acknowledged` emits the
ack other tabs listen for, and only when a row was actually changed.

* fix: the handler scanner, unsubscribe, and the in-tab handler environment

The free-variable scanner skipped arrows inside a declaration's initializer,
so `const ids = event.items.map(x => x.id)` was refused, and treated a name
after a comma in a nested initializer as bound, so a real free variable slipped
through to fail on first delivery. `unsubscribe()` now drops the durable
routing entry so the events socket can close. A broadcast handler running in
the tab gets `user` and `fetch` like the worker gives it. `single` without
a handler name is refused before the round trip.

* docs: events limits, error codes and the background-workers section

Retention is deployment-configured rather than a fixed 14 days, and the
template no longer ships it armed. Documents `events_terminal`, the two
per-event gap reasons, the subject length and listing caps, the `from` field
on moves, and the handle-relative anchor. The sessions manager hides the
background-workers section when the server has none to show.

* feat: a background handler acts as the app does for its user

A handler's `user` was a five-minute access token scoped to the subscription's
`list` grant, which could stat the changed file but not read it, and could
not reach the app's KV or AppData — so an app told that a file was written
could do nothing with it. It now runs with the same authority the app has for
that user in a tab: an app-under-user worker session, one row per (user, app)
named `events:handlers`, visible and revocable in the sessions list. The
`events:background` consent is what authorizes running it unattended, and is
re-checked before every mint.

The wider token exposed two things: puter.js opens a filesystem socket the
moment it has a token, which would have parked the isolate in the app's own
delivery room and steered deliveries at it; the events client now opts out of
sockets (and the per-open bookkeeping) before construction, and is memoized
per token in the isolate. And four filesystem operations assumed a socket
exists; they no longer do.
2026-09-04 17:32:57 -07:00
Daniel Salazar f30baa2a1c feat: the per-app events worker runtime (#3697)
* feat: bake published handlers into a generated events worker

* feat: deploy and address the per-app events worker behind a flag

* test: single delivery end to end through a real local worker

* feat: events workers run their own runtime, in their own namespace

An events worker was being deployed as an ordinary worker: default dispatch
namespace, a `subdomains` row, the router preamble, and an app-scoped worker
token baked in. The public dispatcher resolves any script in that namespace
straight off the hostname, so the worker answered at `<name>.puter.work`, and
the only thing in front of it was an unguessable name plus a check that a
`puter-auth` header was present — which the router never validates. Anyone who
learned the hostname could run an app's handlers with a body of their choosing,
in an isolate holding the owner's token as `me`.

Instead:

- Handlers run on their own runtime (`src/worker/src/events-runtime.js`), which
  provides no `router` and no `me`, owns the single invoke route, and hands a
  handler only `{ event, ctx, user, fetch, ack }`. `user` is built from the
  invocation's delivery token, so a handler acts as the subscriber whose
  delivery it is and nothing wider. The preamble build emits one bundle per
  runtime; the shared half of the template is now included by both.
- The deploy target carries the runtime to prepend, the source to deploy, and
  whether to mint a worker token at all, so an events worker deploys into the
  `events` dispatch namespace from generated source with no token binding, no
  `subdomains` row, and no claim on the owner's worker quota or worker list.
- An invocation carries a key derived from the deployment secret and the script
  name, bound as a secret and checked in constant time inside the isolate,
  which reads it once and drops it before handler code runs.
- Scripts are named after the handler set they contain, so publishing writes
  rows and deploys nothing: a set is deployed the first time a delivery needs
  it, and a changed set is a new script rather than an overwrite of a running
  one. Publish responses keep the shape they had before the runtime existed.
- Invocations reach a worker only through the events dispatcher, which has no
  zone route and requires the internal secret; the backend's own deploy path is
  the rehydrate route the dispatcher calls on a namespace miss. Locally there is
  no dispatcher, so the controller hands the service an in-process transport
  that deploys on miss itself.

The SDK stops allowlisting `puter` as a handler global — a handler that reaches
for an ambient SDK is now refused at publish time, naming `user` instead, rather
than passing the scan and failing on its first delivery.

Requires `events.workerNamespace`, `events.dispatcherUrl` and
`events.internalSecret`; without them nothing is addressable and background
deliveries stay retriable, as they did with the runtime off.

* fix: a handler's delivery token gets through the read routes

An events handler acts as the subscriber through the access token its
invocation carried, but every FS read route refused scoped access tokens
outright, so `user.fs.stat(event.path)` — the design's own example — answered
403 inside the worker. The read-side routes now admit them; the ACL each
handler already runs intersects the token's grant with its issuer's, which is
the check that keeps a token to what it was minted for. The end-to-end suite
asserts the stat from inside the isolate.

* fix: shorthand-method handlers publish as functions

`{ ingest({ event }) { … } }` stringifies without the `function` keyword, so
its source is not an expression and the events worker baked it as a broken
stub — every delivery a retriable 500 until the subscription suspended, with
nothing at publish time to say why. The SDK now gives a shorthand method the
keyword before hashing and sending; getters, setters and computed names are
left for the server-side check to refuse.

* feat: an app's events worker is listable and destroyable

An app with published handlers has an events worker, and hosted deployments
bill it monthly per app, so its owner needs to see it and be able to take it
down. The core announces the lifecycle on the bus — `events.worker.create`
when an app's first handler is published, `events.worker.destroy` when its last
one goes — with the owner as the actor, so pricing can plug in from outside.
`GET /events/workers` lists the caller's workers (paginated, with the script
each set deploys as) and `POST /events/workers/destroy` removes every handler
of an app under the same owner scoping as the handler routes, suspending the
subscriptions bound to them. `puter.events.workers.list/destroy` in the SDK,
a docs page, and a 5 MB cap on an app's combined handler source
(`events_worker_too_large`) so a set that publishes can always deploy.

* fix: harden the events worker runtime for production

- A 4xx is terminal only when it carries the handled marker the runtime (and
  the dispatcher) stamp on every answer that came from a script; an unmarked
  4xx — an edge 404 for a wrong dispatcher hostname, a WAF page — stays
  retriable and is logged, once per script per minute, with the runtime's
  reason header.
- Script names are scoped to this backend's exposed API origin, so two
  backends sharing a namespace never resolve one script with the wrong
  endpoint binding or key. Shape unchanged.
- Each handler is validated in the exact context it is emitted into and the
  whole generated file is compiled once; a source that would break the script
  marks every handler broken instead of deploying a SyntaxError.
- Locally, events scripts live under their own registry key: the public local
  worker host cannot reach them and an ordinary worker cannot take their name.
- A suspended or deleted app owner stops invocations; deploys are throttled
  per app per hour; in-flight deploys are keyed by app and script; the
  upstream deploy call times out; the generated source is size-capped with a
  margin over the publish cap; boot fails when the runtime is on but its
  preamble is not built. Byte-length secret compare, appUid shape check,
  dispatcher URL prefix preserved, wider connection pool.

* feat: background workers are listed in the sessions manager

A user paying for an app's events worker needs somewhere to see it and take it
down. The sessions manager gets a section listing the apps that run event
handlers in the background, with a Destroy action that removes their published
handlers.
2026-09-04 17:32:57 -07:00
dependabot[bot] dfab3d50cd chore(deps-dev): bump browserslist from 4.28.5 to 4.28.9 (#3759)
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.28.5 to 4.28.9.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.28.5...4.28.9)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-version: 4.28.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 23:20:29 +00:00
dependabot[bot] 9daaec4ec4 chore(deps): bump qs from 6.15.3 to 6.16.0 (#3757)
Bumps [qs](https://github.com/ljharb/qs) from 6.15.3 to 6.16.0.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.15.3...v6.16.0)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.16.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 16:17:36 -07:00
dependabot[bot] b7effe1e01 chore(deps): bump fast-uri from 3.1.0 to 3.1.7 in /src/puter-js (#3756)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.0 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.0...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 16:17:08 -07:00
Daniel Salazar 2a68e7aafb ci: add Dependabot config and patch auto-merge workflow (#3760)
Weekly npm/docker/github-actions updates plus vulnerability PRs;
patch-level Dependabot PRs auto-merge, minor/major stay manual.
2026-09-04 16:16:04 -07:00
Daniel Salazar 3355190a17 feat: add astra to models (#3755) 2026-09-04 15:56:53 -07:00
Nariman Jelveh 6ebbc6ae0c feat(gui): let users reposition and zoom a new profile picture (#3754)
* feat(gui): let users reposition and zoom a new profile picture

Picking a photo in the dashboard's Account tab used to stretch the whole
image into a 150x150 square, so anything that was not already square came
out distorted and off-center. The pick now opens an adjust step: a
dashboard-style modal (centered card on desktop, bottom sheet on phones)
where the user drags to reposition and zooms with the slider, pinch, or
wheel before saving. The saved result is the same 150x150 PNG as before.

Geometry lives in profilePictureCrop.js with unit tests; the modal owns the
DOM and pointer handling.

* fix(gui): stop double-encoding the crop modal's hint and frame label

i18n() already HTML-encodes its output, so wrapping it in html_encode()
again turned any apostrophe or ampersand in a translation into a literal
"&apos;" / "&amp;" on screen. Also puts the new profile_picture_* keys
in alphabetical order.

* fix(gui): let the crop frame take focus on click so arrow keys work after a drag

pointerdown's preventDefault() also cancels the click-to-focus that a
mousedown would have done, so after dragging the photo the arrow keys and
+/- went to the dialog container and did nothing. The frame now focuses
itself on pointerdown. Focus that arrives by pointer draws no ring; the
first key press lifts that so keyboard users still see where they are.

Also ignores secondary mouse buttons and treats a lost pointer capture as
a release so a pointer can't stay stuck in the gesture map.

* fix(gui): return focus to the avatar when the crop modal closes

The modal remembered document.activeElement to restore focus later, but
at that moment focus sits inside the file picker, which closes right
after -- so on Save, Cancel or Escape focus fell to <body>. The Account
tab now names its avatar as the place focus returns to, and the avatar
becomes a real button (role, tabindex, label, Enter/Space) so it can
hold that focus and be reached from the keyboard at all.

* fix(gui): announce the crop zoom as a magnification, not a 0-100 slider value

Screen readers read the range input's raw value, which maps to nothing a
user can picture. aria-valuetext now carries the zoom factor (1.0x-4.0x)
and follows every zoom source: slider, buttons, keys, wheel, pinch.
2026-09-04 14:38:55 -07:00
404oopsandClaude Opus 5 7248951410 fix(ai): make chat fallback reach streamed Claude calls and rank Azure explicitly (#3743)
* fix(ai): make chat fallback reach streamed Claude calls and rank Azure explicitly

- ClaudeProvider opens the upstream stream and awaits its connection
  before returning the populator, so an overloaded or rate-limited route
  throws from complete() and reaches the driver's fallback loop instead
  of surfacing as an error frame on a 200
- the OpenAI-compatible chat and completions routes only pin a provider
  when the caller sent one, so they get the same preferred healthy route
  puter.js callers do and unhealthy-route skipping applies to their
  first attempt
- Azure is ranked ahead of the vendors it fronts by an explicit tier in
  modelRouting rather than a price tie plus registration order
- drop Together's synthetic always-failing model-fallback-test-1 entry
- test that a 4xx leaves a route in rotation while a 503 marks it

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai): surface swallowed Claude stream errors, keep compat-route defaults

Review follow-ups on the fallback work.

- The pre-created event iterator only receives an error if a reader is
  already waiting on it, so a failure landing between the connect and the
  populator's first pull ended the stream cleanly — truncated content
  billed and reported as a success. Rethrow when the stream is errored.
- A refused stream deleted its Anthropic uploads but left the caller's
  message parts pointing at those file ids, so the fallback route was
  handed handles it cannot resolve. processPuterPathUploads now returns a
  restore() that both failure paths call.
- The OpenAI-compat routes keep pinning OpenAI when the caller sends no
  model at all, so the default model stays put instead of moving to
  Azure's.
- Say why /openai/v1/responses and /anthropic/v1/messages stay pinned:
  each translates one provider's native shape by hand.
- PREFERRED_PROVIDERS is unexported and its doc now states the rank is
  unconditional; the duplicated hidden-model list is one constant.

Co-Authored-By: Claude Opus 5 <[email protected]>

* fix(ai): undo the puter_path rewrite by field instead of snapshotting the part

Copying the content part kept whatever the caller sent on it — a large
inline `source` or `text` alongside `puter_path` — reachable until the
request ended, where overwriting the field used to make it garbage right
away. The only fields this function writes are `type`/`source` on success
and `type`/`text` on failure, and the fallback uploader keys off
`puter_path` alone, so restore undoes those three by name.

Co-Authored-By: Claude Opus 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-04 11:10:05 -07:00
Reynaldi Chernando 2966cee6b0 explicit mention of kv and fs being user scope and workers for backend (#3751) 2026-09-05 01:02:22 +07:00
Reynaldi Chernando d8735d950e fix kv set value type (#3750) 2026-09-05 00:30:17 +07:00
Reynaldi Chernando 48abf73d1a kv add array example (#3748) 2026-09-05 00:22:46 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 11c5a7ce52 dynamic workers docs (#3747)
* dynamic workers docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-04 22:57:11 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI b6876c50bf PUT-1477 Add fs commands to cli (#3746)
* Add fs commands to cli

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-09-04 21:31:33 +07:00
Daniel Salazar 6d7be8bd6a docs: kv key layout is the ACL boundary (PUT-1689) (#3693) 2026-09-03 19:43:28 -07:00
Daniel Salazar b451d05d10 feat: app-minted kv share handles (PUT-1688) (#3692) 2026-09-03 19:43:27 -07:00
Daniel Salazar f139588f89 feat: revocation settles kv handles (PUT-1687) (#3691) 2026-09-03 19:43:27 -07:00
Daniel Salazar ac5446f877 feat: cross-user KV share grants and handles (PUT-1686) (#3690) 2026-09-03 19:43:27 -07:00
Daniel Salazar adde6c3a29 feat: notification GUI rides the events surface (PUT-1685) (#3689)
Widen the notif: match filter and fetch scope for a session's own
generic developer/app-user subscribe: today it pins ref to the
session's own uuid, so a row naming an app (handler-suspension
notices, app-bound worker deploys) never matches live and never
replays on reconnect, even though the audience predicate already
grants the holder every such row it owns. The predicate is the
authority and already reruns per row/page after the match, so
widening the filter to it (account is unaffected — it never names an
app) adds no exposure.
2026-09-03 15:39:22 -07:00
Daniel Salazar fe5f81e52e feat: scoped app access to notifications (PUT-1684) (#3688)
An actor holding an app reads the `app-user` rows naming that app, plus its
`developer` rows when the holder owns it. `account` rows reach no app, and a
slice an actor may not see comes back empty rather than refused. The audience
predicate becomes the enforced read path in the same change that lifts the
blanket app-actor 403, layered behind an audience/app_uid SQL scope; two-segment
`notif:` subjects expand server-side from the actor's own app, so an app can
never name another app's uid.

No feature flag: `audience` defaults to 'account', so every pre-registry row is
default-denied to app actors and the backfill can only narrow.
2026-09-03 15:39:22 -07:00
Daniel Salazar 796683a133 feat: missed-event fetch and notification fold-in (PUT-1681) (#3687) 2026-09-03 15:39:22 -07:00
Daniel Salazar eb8f497e9f feat: presence and cross-region event forwarding (PUT-1679) (#3686)
* feat: presence and cross-region event forwarding (PUT-1679)

* fix: fan cache bumps to sibling nodes and stop the forward shed cascading (PUT-1679)

`outer.events.generationBumped` and `outer.events.presenceBumped` rode
`outer.*`, which the broadcast service only webhooks to peer regions;
only `outer.pubsub.*` also fans over Redis to a region's other nodes.
Both caches are per-process maps with no expiry, so a bump landing on
one node left its siblings stale until that user's next transition.
Renamed onto `outer.pubsub.events.*`; the listeners already accept the
`from_outside` copy the Redis re-emit carries.

`PeerForwardQueue.push` called `onOverflow` synchronously and the
handler pushed markers straight back, each of which re-tripped the
bound and shed the next item: one item over a 5000 bound recursed ~2200
deep, threw a RangeError, and turned ~2200 queued deliveries into gap
markers. It also re-summed `bytes` over the whole queue per drop. The
handler now returns its markers and the queue appends them past the
bound check, sheds deliveries before markers, keeps one pending marker
per (peer, subscription), and subtracts bytes per dropped item.
2026-09-03 15:39:21 -07:00
Daniel Salazar 32a838d2e2 fix: one ** per match pattern and one * per segment (#3736)
`compileMatch` turned every `*` into an unbounded `[^/]*`, so a subject
like `fs:~/x/*a*a*a*a*a*a*a*a*a*a*z` tested against a 24-char filename
with no `z` — both named by the same subscriber — cost the regex engine
C(34,10) splits before failing: 1.0 s per event at ten stars, 3.7 s at
eleven, on the fs-write dispatch path, inside the 256-character cap.

With one `*` per segment the delimiters pin each star and a wrong split
dies in one step; the single `**` is the only choice point left, so the
worst allowed shape is O(depth × length): 0.4 ms at depth 60 of 200-char
segments. Every documented pattern (`*.png`, `**/build.log`, `**/*.png`,
`dir/**`, `report-?.csv`) stays valid; `*a*`, `a**b` and `**/x/**` are
refused with `invalid_subject_pattern`. The rule is stated where the
syntax is introduced, on the limits page, and in onLocal's error table.
2026-09-03 15:08:49 -07:00
404oops 1b777cd760 Merge pull request #3730 from HeyPuter/filipkujundzic/put-1620-video-generation-timeouts-return-500-and-page-on-call
fix: stop video generation timeouts and failures from paging (PUT-1620)
2026-09-03 22:54:58 +02:00
Daniel Salazar 799fc4ac3c feat: send app icons as a subdomain URL plus an API fallback (#3734)
App payloads carried only the /app-icon endpoint URL, which 302s to the
icons hosting subdomain. Networks that mangle that redirect render no icon
at all, and every icon load pays a round trip for the hop.

Ship the direct subdomain URL as `iconCdnUrl` alongside it (taskbar items,
installedApps, recent/recommended launch apps, suggested apps), and have the
GUI load that first with the endpoint URL as a one-shot retry - desktop
taskbar, start menu, dashboard app grid and recents. Only rows whose `icon`
column is already an http(s) URL get one: a data: column means the resize
pipeline has not written anything to the subdomain yet.

Also folds the four copies of the generated-size list into one exported
APP_ICON_SIZES.
2026-09-03 13:32:55 -07:00
Juan Fernando Castro d855315db8 Merge pull request #3714 from HeyPuter/juancastro/put-1708-26-teamcontroller-and-audit-writes
🏗️ PUT-1708 + PUT-1709 + PUT-1743: TeamController, teams_enabled, and the isolation suite
2026-09-03 16:21:02 -04:00
Juan Castro 9bc2cd741c feat: add TeamController, the teams_enabled flag, and the isolation suite
Covers PUT-1708, PUT-1709 and PUT-1743.

Twelve routes, every one setting requireUserActor -- that option is what
installs requireAuthGate, requireVerifiedAccount and requireNonAccessTokenGate,
because server.ts derives `needsAuth` from the route options. Reads need it as
much as writes: without an auth option a route gets no suspension check and
admits access tokens, so a just-disabled member could still read the roster and
a scoped third-party token could read the audit log.

Authority is checked before anything observable. Validating the body first made
POST /members answer 400 before 403, and resolving :username first turned the
member routes into a global username-existence oracle.

Provisioning applies the same username and email rules as signup rather than
its own -- USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and
validator.isEmail, now exported from AuthController. Without them a workspace
could mint accounts signup would refuse, claim unregistered reserved names, and
mail arbitrary unvalidated addresses.

Handle problems are 400 or 409 rather than a bare Error, which the server turns
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable drops sessions through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key; without that a
disabled member kept authenticating from cache for the session TTL, which is
exactly the "takes effect on the next request, not after a cache TTL" property
disable is supposed to have. Revoking also preserves last_ip/last_user_agent,
which the member-facing audit view reads.

Audit writes live in TeamService at the point of each action rather than in the
route, so a caller reaching the service directly cannot skip them, and the SQL
lives in TeamStore. Audit reads map internal user ids to usernames, and remain
readable by the owner after the workspace is soft-deleted -- otherwise the
delete_team entry was written and immediately unreachable.

teams_enabled gates route registration through an optional isEnabled() the
server honours, so with it off the paths do not exist rather than existing and
refusing. It does not gate DDL.

TeamIsolation.http.test.ts asserts the negative the feature rests on: the
workspace manages accounts and cannot read them, including through a
full-access token and after the member is disabled. It asserts outcomes rather
than the absence of an implicator.
2026-09-03 15:57:23 -04:00
Juan Castro 673f7fe75f feat: provision org accounts from a workspace
Covers PUT-1705. The master account supplies { username, email }; the account
is created with no password, gets the default filesystem tree, joins with
org_owned = 1, and receives a one-shot activation link.

Activation reuses password recovery rather than new token machinery: the same
pass_recovery_token, the same one-hour purpose-scoped JWT, the same
/action/set-new-password link. No team_activation table, no new token type,
and no unauthenticated endpoint on the team surface. Activation state needs no
column either -- an unactivated account is one with no password.

Applies the same username and email rules as signup rather than its own:
USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and validator.isEmail,
now exported from AuthController. Without them a workspace could mint accounts
signup would refuse -- the username becomes the /username home-directory
segment -- claim unregistered reserved names, and send activation mail to
arbitrary unvalidated addresses at the route's daily limit.

Usernames come from Puter's global pool, so a taken one is refused with free
alternatives rather than silently modified: a suffixed name would appear in
every share dialog that person ever sees, and they never agreed to it. The
check runs before any write, so a rejected provision leaves no orphaned user
row -- asserted by a test on the workspace's member count.

The new account carries requires_email_confirmation, since the address came
from the administrator rather than its holder.

Adds a team_account_activation email template stating what the workspace can
and cannot do -- including that it can reset the password, which the design
requires be said rather than only claiming files are private.

free_storage stamping and the billing event are phase 3.
2026-09-03 15:57:23 -04:00
Juan Castro da2b42d7fe feat: add TeamService workspace lifecycle and disable/re-enable
Covers PUT-1704 and PUT-1707: creating a workspace, admitting the master
account, and the whole of offboarding.

`createWorkspace` admits the creator with org_owned = 0, which is what makes
the master pay for itself and stay an invalid target of every member route.
`checkOwnerInvariant` asserts the rule no dialect can express -- the owner is
a member with org_owned = 0 and the only such member -- and a test breaks it
deliberately, since the schema cannot refuse a second one.

Three authority checks: 404 to a stranger so the endpoint is not an existence
oracle, 403 to a member who is not the master, and the master refused as a
target of member routes.

Handle problems surface as 400 (unusable) or 409 (taken), including the
unique-index race. `TeamStore` throws a bare Error, which the server would turn
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable writes `user.suspended` as well as suspended_at and suspended_reason.
PUT-1707 named only the latter two, but those are siblings added by 0061 and
0063 -- `userProtected` rejects on `if (user.suspended)` and reads neither.
Setting only the timestamp and reason would have recorded a disable that never
took effect, and disable is the whole of offboarding here.

Sessions are dropped through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key with its double-delete
pattern; without that a disabled member keeps authenticating from cache for the
session TTL, which is exactly the "next request, not after a cache TTL"
property disable is supposed to have. Revoking also preserves last_ip and
last_user_agent, which the member-facing audit view reads.

Files are untouched and re-enable restores the account.

Adds team_not_found, not_the_master_account and not_an_org_account to the
HttpError legacy codes, which the controller also needs.

Billing events, invalidateActorSubscription, audit rows and the GUI push are
deliberately not here -- they belong to phase 3 and PUT-1708.
2026-09-03 15:57:23 -04:00
Juan Castro 8b4e259421 feat: add team membership reads and writes to TeamStore
Membership management for workspaces: addMember, removeMember,
getMembership, isMember, listMembers and listTeamsForUser. The permission
scan is untouched -- readUserGroupPerms already joins jct_user_group and
resolves group grants; this is the management side.

Resolves the ticket's "do not leave two writers" by splitting domains and
enforcing the split in SQL rather than by convention. Every existing caller
of GroupStore targets a seeded system group -- ADMIN_GROUP_UID,
default_user_group, default_temp_group -- never a team, so the two stores
were already disjoint in practice. GroupStore.addUsers/removeUsers now carry
`AND kind IS NULL`, making a team uid a no-op there, which costs no extra
query because it folds into the existing subquery and matches how addUsers
already treats an unknown username. TeamStore's writes select group_id from
a kind-filtered subquery, so neither store can reach the other's rows.

org_owned is written here but never accepted from a request; TeamService
sets it at provisioning and workspace creation only.

listMembers is keyset-paginated on id per doc/pagination.md, using the
shared cursor and limit helpers and fetching one row past the limit to
decide whether a cursor is warranted.

Passes 1/0 for org_owned rather than db.booleanValue, which yields a real
boolean on postgres and is rejected by the smallint column there -- sqlite
accepted it silently.
2026-09-03 15:57:16 -04:00
Juan Castro 37cb11df8c feat: add TeamStore for workspace CRUD
`GroupStore` has only addUsers/removeUsers; nothing creates, reads back or
lists a group at runtime. `TeamStore` is that missing half, scoped to rows
with `kind = 'team'`.

A workspace is addressed by `uid`, which `group` has carried as NOT NULL
UNIQUE since 0015. `handle` is a mutable display label with no addressing
role, so a rename invalidates nothing and a stale reference can never
resolve to a different workspace.

Soft delete releases the handle and keeps `name`. Nothing points at a
handle, so the name returns to the pool instead of being reserved forever
by a global unique index that cannot exclude dead rows -- mysql has no
partial indexes, so that exclusion was never available.

Handles validate to ^[a-z0-9]+(-[a-z0-9]+)*$, 3-64 chars, against a
reserved list. The charset is deliberately narrower than the column so the
engines' collations cannot disagree: mysql's utf8mb4_unicode_ci also folds
accents and eszett, which sqlite's NOCASE and postgres's lower() do not.

Every read filters `kind = 'team' AND deleted_at IS NULL`, which is what
makes the seeded admin/system groups unreachable rather than merely absent.
Handle lookups compare lower(handle) on postgres, where the index is on
that expression rather than the column.
2026-09-03 15:20:07 -04:00
Juan Fernando Castro ad70e8763b Merge pull request #3708 from HeyPuter/juancastro/put-1701-13-add-audit_team_membership-and-shareholder_group_id
🏗️ PUT-1701: Add audit_team_membership and share.holder_group_id
2026-09-03 15:17:34 -04:00
Juan Castro 64db3fa050 feat: add audit_team_membership and share.holder_group_id
Insert-only record of what a workspace administrator did to an account,
shaped like \`audit_user_to_group_permissions\` after 0019: nullable FK
beside a NOT NULL \`_keep\` column. The FKs are ON DELETE SET NULL, never
CASCADE, so hard-deleting an account cannot erase the record of the
resets performed on it.

Two indexes rather than one. The member's own view is the only place a
reset becomes visible to the account it was performed on, so
(user_id_keep, id) is a read path, not an optimisation.

\`share.holder_group_id\` mirrors \`holder_user_id\` from 0067. The existing
unique index does not constrain team shares at all -- it leads with
\`holder_user_id\`, which is NULL on every team share, and NULLs are
distinct -- so the group-scoped unique index is what prevents duplicates.

Drops the \`role\` column from the specified DDL: it contradicted the
settled single-administrator model.
2026-09-03 15:07:53 -04:00
Juan Fernando Castro 43f62cf7d0 Merge pull request #3705 from HeyPuter/juancastro/put-1700-12-deduplicate-jct_user_group-before-adding-the-unique-pair
🏗️  PUT-1700: Deduplicate jct_user_group and make membership writes unique
2026-09-03 15:07:49 -04:00
Juan Castro dc7b456afa fix: run the membership dedup once instead of on every boot
Review follow-ups on the dedup migration.

mysql and postgres track no per-file applied state and re-execute every
migration on each boot, so the unguarded DELETE self-joined the whole table at
every process start, forever. Both now sit behind the same index-existence
check that guards the ALTER, which also stops a rolling deploy deleting on one
instance while another adds the index.

The dedup test was a false green. `targetVersion: 67` never applies 0071 -- the
loop breaks on `threshold + 1 >= targetVersion` but stamps the target anyway --
so the fixture asserted the current schema version on a database missing a
migration. It now replays the real 0072 file against a fully migrated database,
and a second test pins the off-by-one so nobody builds a fixture on it again.
2026-09-03 14:33:46 -04:00
Juan Castro c1c1588939 fix: deduplicate jct_user_group and make membership writes unique
\`jct_user_group\` had no unique constraint on (user_id, group_id) and
\`GroupStore.addUsers\` had no conflict clause, so re-adding a member
inserted a second row. \`readUserGroupPerms\` joins the junction table on
group_id alone, so each duplicate returned another copy of every group
permission the user holds.

Deduplicate keeping the lowest id, add the unique pair index, and make
\`addUsers\` ignore conflicts via the existing \`insertIgnoreInto\` helpers --
without that last part the index turns a re-add into a raised error, which
five call sites would log as a failed signup step.

mysql cannot delete from a table it reads in a subquery (error 1093), so
it uses a self-join with the same lowest-id-wins semantics.
2026-09-03 14:32:30 -04:00
Juan Fernando Castro abb426a717 Merge pull request #3704 from HeyPuter/juancastro/put-1699-11-add-team-columns-to-group-and-jct_user_group-three
🏗️ PUT-1699: Add team columns to group and jct_user_group
2026-09-03 14:26:58 -04:00
Juan Castro eae25459f4 fix: make handle lookups case-insensitive and pin the reset flag
Review follow-ups on the team columns.

NOCASE moves onto the `handle` column itself, not just the index. Index-only
NOCASE makes uniqueness case-insensitive while leaving `WHERE handle = ?`
case-sensitive, so the same lookup would match on mysql (utf8mb4_unicode_ci)
and miss on sqlite. Postgres still needs lower(handle) at the call site.

`requires_password_change` becomes NOT NULL DEFAULT 0 on all three dialects,
matching the three sibling `requires_*` flags. Left nullable, any query written
as `= 0` would silently exclude every pre-existing user.
2026-09-03 12:10:48 -04:00
Juan Castro 2a5c089dbe feat: add team columns to group and jct_user_group
Schema foundations for teams. Ships dark — nothing reads these columns yet.

group:          kind, name, handle, plan_id, deleted_at
jct_user_group: org_owned
user:           requires_password_change

`kind = 'team'` marks a workspace; the seeded system groups keep it NULL so a
team query can never surface them. `org_owned` distinguishes accounts the
workspace created from the master account, deciding who pays rather than who
may read. `requires_password_change` is a fourth `requires_*` flag for
assertVerifiedAccount, needed by the phase 5 reset flow but shipped here to
avoid a second three-dialect migration.

`handle` uniqueness is case-insensitive, and the dialects disagree by default:
mysql gets it from utf8mb4_unicode_ci, sqlite needs COLLATE NOCASE (as
0055 does for usernames), and postgres indexes lower(handle). Without this the
same migration would accept `Design-Team` next to `design-team` on sqlite and
postgres while mysql rejected it. Postgres handle lookups must therefore
compare lower(handle) to use the index.

idx_group_owner is sqlite-only: mysql and postgres already index that column.
idx_jct_user_group_group is composite, unlike the existing single-column keys.
2026-09-03 12:10:40 -04:00
404oops 55fa5cadcb Merge pull request #3732 from HeyPuter/gemini-38-flash
gemini 3.8 flash direct integration
2026-09-03 17:35:22 +02:00
Reynaldi Chernando 644f59f9dd gemini 3.8 flash direct integration 2026-09-03 22:18:02 +07:00
404oopsandClaude Fable 5.1 8b01077d44 docs: describe the real txt2vid error body and per-provider wait windows
The Errors section claimed a `{ message, code }` body and a single
ten-minute wait. The body also carries `error`, `errorCode`, `provider`,
`upstreamCode` and, for pre-job rejections, `upstreamStatus`, and the
Sora provider still waits five minutes and folds refusals into
`upstream_failed`. Document the fields and the exceptions.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-03 16:57:36 +02:00
404oopsandClaude Fable 5.1 627a5b2d5e fix: stop video generation timeouts and failures from paging (PUT-1620)
A video job that outlived its poll window, an SDK request that timed out,
or a Veo operation that finished with an error all reached the HTTP error
handler as plain Errors. Each became an unhandled 500 with critical
severity and paged on-call for what is the provider's pace or the
provider's fault.

Video providers now share one poll loop that gives up with a 504
`upstream_timeout`, treats a transient poll failure (timeout, dropped
connection, 408/429/5xx) as a missed poll rather than a failed job, and
stops polling with a 400 `client_aborted` when the caller disconnects, so
nothing is metered for a clip nobody will receive. The driver controller
exposes the disconnect as an `abortSignal` on the request context. The
window is ten minutes for every provider; Together and BytePlus move up
from five.

Failed jobs are classified: content-filter refusals become a 400
`bad_request` with `errorCode: moderation_flagged`, rejected parameters a
400 `upstream_bad_request`, and anything else a 502 `upstream_failed`,
each carrying the provider's own code. Veo's filtered output keeps
`disallowed_value` and gains the same `errorCode`. The sanitizer and
content-filter pattern move from the Replicate provider into a shared
util so image and video agree.

Status-less SDK connection timeouts are translated to a 504
`upstream_timeout` at the driver boundary, and the chat driver records
them per attempt so an all-timeout chain is a 504 and a mixed chain is
`upstream_failed` instead of an `internal_error` 500. The Together chat
client gets the same ten-minute request timeout as the other providers.

The OpenAI video provider is left alone beyond an import path: its API is
scheduled to shut down on 2026-09-24.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-03 16:47:16 +02:00
Daniel Salazar 72011fa6ed feat: events metering and quotas (PUT-1683) (#3685)
* feat: events metering and quotas (PUT-1683)

* fix: drop the standing subscription charge and price single deliveries at 100 µ¢

An idle durable row costs nothing worth billing; the plan quotas bound how
many an account holds. Removing the daily line also removes the global
day-claim, the whole-table scan and the timer it rode on.

* test: stop asserting on documentation pages

The limits test read rate-limits-and-quotas.md and grepped it for numbers,
so every rewording of the page failed the backend suite. Docs are kept in
step by the PR and checked in review; AGENTS.md now says so.
2026-09-03 01:39:07 -07:00
Daniel Salazar 41b837b398 feat: background delivery seam, retries, and consent (PUT-1682) (#3684) 2026-09-03 01:39:06 -07:00
Daniel Salazar 04c00385b9 feat: event handlers, context, and suspension machine (PUT-1680) (#3683) 2026-09-03 01:39:06 -07:00
Daniel Salazar 9626ab9c71 feat: KV change events and KV subjects (PUT-1678) (#3682) 2026-09-03 01:39:06 -07:00
Reynaldi Chernando 3de6eeb474 init docs recipes (#3728) 2026-09-03 10:04:01 +07:00
Neal Shah af27115f68 Attempt 1: Fix PUT-1625 (#3727) 2026-09-02 21:36:04 -04:00
Daniel Salazar d8ebfc8023 feat: delivery re-check cache, revocation and anchor settle (PUT-1677) (#3681)
* feat: delivery re-check cache, revocation and anchor settle (PUT-1677)

* fix: authorize re-anchors, settle each row once, purge revoked backlog (PUT-1677)

- A path-form row whose anchor is deleted only climbs to an ancestor its
  holder may still watch under the mode it subscribed with; otherwise it ends
  with `anchor_deleted`. It used to land on any surviving ancestor (a guest's
  row on the owner's home), where the re-check denied every delivery but the
  row still held an anchor slot and a filter evaluation there.
- After a climb the new anchor is re-verified and the climb repeated if a
  recursive delete took that level too, instead of leaving the row on a dead uid.
- suspend() is one conditional write per row and reports which rows it was the
  one to suspend; concurrent settles of the same grant (an unshare revokes
  several strings) no longer each purge, forget and notify the same rows.
- One "subscriptions ended" notification per holder and app, carrying the count
  and subjects, instead of one per row.
- A revoke that removed nothing no longer announces; the sweeper purges (not
  defers) the backlog of a permission_revoked row; the reap purges pending
  entries with the row.
- The delivery auth cache indexes entries by subscription so forget() is not a
  scan of the whole cache.
2026-09-02 16:00:14 -07:00
Daniel Salazar 41d4850ab4 feat: pending event deliveries and delivery-class invariants (PUT-1676) (#3680)
* feat: pending event deliveries and delivery-class invariants (PUT-1676)

* fix: make pending delivery claims and drains atomic, keep the region under its ceiling (PUT-1676)

- claim() and the drain-time reindex run as Lua over the subscription's own
  {subId}-tagged keys. Two claimers can no longer both lease the head, and a
  drain that finds the queue empty deletes it in the same step it checks, so a
  concurrent enqueue is never wiped between the two.
- An append writes the entry and its queue position in one MULTI (same slot),
  with the index seeded before it and corrected after, so an entry is never
  visible without its position and never left out of the sweeper's index.
- Pipelines no longer mix slots (index/counter vs. per-subscription keys), so
  the store works on a multi-shard cluster, not only a single-shard one.
- Region shedding counts the marker it leaves behind; it used to stop one over
  the ceiling and convert a real event into a marker on every enqueue after.
- A claimed or suspended subscription moves to the back of the sweeper's index,
  so a delivery nobody settles cannot hold the head against every other backlog.
- `single` rows must carry a `worker` target: with sockets exhausted and no
  handler, an unacknowledged delivery would sit at the head forever.
- A gap marker for a row with no socket target is dropped rather than counted
  as a delivery of nothing.
- Backlog keys carry a 7-day TTL, refreshed by every claim, as a backstop for
  keys a purge/enqueue race left unindexed.
2026-09-02 16:00:14 -07:00
Daniel Salazar 0dfbceb047 feat: durable event subscriptions store, cache, and routes (PUT-1673) (#3679)
* feat: durable event subscriptions store, cache, and routes (PUT-1673)

* fix: durable subscription hardening (PUT-1673)

- Expired rows stop delivering at dispatch time and no longer count toward
  the per-account cap, instead of waiting for the sweep.
- The expiry sweep runs hourly with a jittered first pass shortly after boot;
  a 24 h interval never fired on a fleet that redeploys more often than that.
- Only a durable generation bump marks peer regions cold. A session
  subscribe/unsubscribe in one region used to force a primary read in every
  other region on its next dispatch.
- `subject`/`anchor_path` widen to varchar(4096) to match `fsentries.path`,
  and subjects longer than that are refused with `invalid_subject` rather than
  failing the insert on MySQL/Postgres.
- The dispatch and durable integration suites wait for the specific delivery
  they expect and assert only within their own folder; the old any-delivery
  `settle()` let a late event from a previous test satisfy or pollute the
  next one under CI load.
2026-09-02 16:00:13 -07:00
Daniel Salazar cab5253923 feat: puter.events SDK module (PUT-1675) (#3678)
* feat: puter.events SDK module (PUT-1675)

* fix: end event subscriptions on a server-side disconnect and retry budget refusals (PUT-1675)

- A disconnect socket.io will not retry (`io server disconnect`) now fails
  every subscription with `events_connection_failed` and drops the socket, so
  the next onLocal() starts a fresh connection instead of reusing a dead one.
- The connection closes when the last subscription lapses on re-subscribe.
- A re-subscribe refused with `too_many_requests` is retried after 10 s
  rather than ended for good.
- The API suite no longer passes when it cannot connect; every runtime it
  runs on (node, browser, workerd) carries the socket.
2026-09-02 16:00:12 -07:00
404oopsandClaude Fable 5.1 d89dc347a2 fix: keep the credit-bounded max_tokens on the context-length retry (PUT-1628) (#3718)
OpenRouter and Together reject a request whose prompt plus max_tokens
overflows the model's context window. Both providers retried by deleting
max_tokens, which threw away the output cap the credit gate had sized to
the caller's remaining balance and let the retry run to the model's full
output limit with no second gate and no new hold.

The retry now goes through a shared helper that sizes a new cap from the
window and input count the rejection reports, falling back to the model's
declared context and a doubled prompt estimate, and never exceeds the cap
the gate set. When no window can be determined or no output fits, the
original rejection is rethrown instead of retrying uncapped. The rejected
params are copied rather than mutated, so the first attempt's record is
not rewritten after the fact.

Also corrects the estimator's own comment, which described the mean of
two approximations as a deliberate halving.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-02 13:28:36 -07:00
Daniel Salazar 66dc0a6396 feat: subscribe ACL and app scoping for event subscriptions (PUT-1672) (#3677) 2026-09-02 13:08:23 -07:00
Daniel Salazar fb8e2976dc fix: don't redirect puter.com (#3723) 2026-09-02 11:57:30 -07:00
404oopsandClaude Fable 5.1 fd7b517448 fix: classify Replicate prediction failures instead of 500ing (PUT-1608) (#3719)
A Replicate prediction that ran and ended `failed` reaches the provider as a
plain Error with no HTTP status, so the driver-boundary translator could not
classify it and it surfaced as an unhandled 500, a critical alarm, and an
on-call page. Most of these are the model's content filter refusing the
user's prompt.

Wrap the run call and classify the failure: content-filter refusals become
a 400 with `errorCode: moderation_flagged` (the code chat refusals already
use); anything else becomes a 502 `upstream_failed`, which the alarm gate
skips. Status-bearing SDK errors pass through untouched so the boundary
translator keeps handling them. Upstream messages are stripped of markup and
bounded so an HTML error page can no longer ride into a response body or an
alarm signature.

Documents the codes callers can now act on in the txt2img reference.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-02 11:51:08 -07:00
Neal Shah 262f1dc5c5 allow concept of "home regions" (#3699)
* allow concept of "home regions"

* remove extraneous config value not applicable to repo
2026-09-02 14:11:55 -04:00
Daniel Salazar 7ec674d33b feat: app sockets join their own events room (PUT-1671) (#3676) 2026-09-02 09:01:22 -07:00
Daniel Salazar b07d2e109f feat: session event subscriptions and dispatch hot path (PUT-1666) (#3675) 2026-09-02 09:01:22 -07:00
Daniel Salazar 92e1b40361 feat: events subject registry, parser, anchors, filters (PUT-1660) (#3674) 2026-09-02 09:01:21 -07:00
Daniel Salazar d0ee19a2c2 chore: remove dead notification surfaces (PUT-1669) (#3673) 2026-09-01 22:04:33 -07:00
Daniel Salazar 82076ea767 feat: notification retention sweep (PUT-1668) (#3672) 2026-09-01 22:04:32 -07:00
Daniel Salazar 5f37817a5c feat: notification type registry and audience rules (PUT-1665) (#3671) 2026-09-01 22:04:32 -07:00
Daniel Salazar 148b930182 feat: notification scope columns and backfill (PUT-1659) (#3670) 2026-09-01 22:04:31 -07:00
Daniel Salazar 0385072e06 feat: better verification step handling (#3717) 2026-09-01 21:17:19 -07:00
Daniel Salazar 1736d23ee7 Ds/put 1674 (#3715)
* feat: app-scoped share listing and revoke surface (PUT-1670)

* fix: address review on app-scoped share surface

- Scope the uid-addressed revoke to the named row: only that row's
  issuer's grant is withdrawn, and only that one invite cancelled —
  an app or owner addressing one row no longer takes another issuer's
  grant on the same (item, recipient) pair with it.
- Delete a pending row directly on uid-addressed revoke, so an invite
  whose address registered but never claimed can still be withdrawn.
- Read the legacy `issuerAppUid` data key in the SQL app filter and
  grouping, alongside the unified `issuedByApp`.
- Refuse malformed `appUid` input (duplicated param, empty string)
  instead of silently listing everything, and refuse app-listing
  cursors that decode but name no appUid.
- Derive the acting app from `effectiveApp` alone, per the actor
  contract; drop the second derivation site.
- Pin the attribution semantics with tests: one row records one
  issuance, so re-sharing the same pair re-attributes it to whoever
  issued last, in both directions.
- Soften the uniform-404 docblocks to what the gates actually answer.

* feat: readable grant audit trail (PUT-1674)

* fix: cover the apps summary's no-app-group first page (PUT-1670)

listOutboundApps sorts the no-app group first via an empty-string
sentinel. Add a regression test pinning that a first page (no cursor)
actually returns it, and that the cursor it hands back resumes past it
into the app-keyed groups rather than skipping or repeating.
2026-09-01 14:07:15 -07:00
Daniel Salazar 66a975f659 feat: global outbound share listing (PUT-1664) (#3694)
* feat: global outbound share listing (PUT-1664)

* fix: address review on outbound share listing

- Check share-row liveness per (holder, entry, issuer) so a grant
  withdrawn outside unshare doesn't stay listed while another issuer
  still reaches the same holder; batch the permission reads across the
  whole page instead of per holder.
- Retire a revoked issuer's unclaimed invites in the revoke cascade,
  and hide invites whose issuer lost their authority at read time.
- Unify the pending/active app-attribution key on `issuedByApp` and
  dual-read the legacy `issuerAppUid` spelling.
- Add the missing share issuer index (sqlite, postgres) and correct
  the listOutbound plan comment.
- Refuse cursors that decode but name no id instead of silently
  restarting from page one.
- Consolidate the five hand-built ResolvedShare literals and the two
  listing endpoints' parse/shape code.
- Ship the SDK surface: puter.fs.listSharedByMe() with docs, types,
  suite coverage, and the rate-limit page entry.
2026-09-01 13:52:37 -07:00
404oopsandClaude Fable 5.1 6478a8f47d Add Claude Fable 5.1 (#3711)
Register claude-fable-5-1 in the Claude catalog and gate it in the
provider the same way as Fable 5: no sampling params, effort via
output_config, adaptive thinking with summarized display. The bare
claude-fable / claude-fable-latest aliases move to 5.1, matching how
the Opus aliases moved when Opus 5 landed.

Fable 5.1 keeps Fable 5's $10/$50 per MTok, 1M context and 128K output,
but bills cache reads at 0.025x input instead of the 0.1x every other
Claude model uses, so the catalog row carries its own rate and a test
pins it.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-01 15:34:49 -04:00
Nariman Jelveh fdd86ceb03 Stop server-rendered landings flashing for signed-in users (#3707)
The shell renders its anonymous markup — the marketing homepage, an
`/app/<name>` landing — off the session cookie alone, and that cookie is
set with no maxAge, so a browser drops it on quit while the GUI's
localStorage token lives on. A returning user is served the anonymous
page and the GUI only tears it down once `whoami` answers, a network
round-trip after first paint. That teardown is the flash.

Gate it before the paint instead. The shell now emits, as the first thing
in <head>, a rule hiding `.hide-if-logged-in` under an <html> class that
an inline script adds iff `auth_token_v2` is in localStorage. The rule is
already in the cascade when the markup is parsed, so a browser holding a
token never paints it at all.

`initgui` settles the guess the token represents: `whoami` confirming the
session removes the nodes outright (replacing the old `#appLanding`
removal), and no session — none stored, or one `whoami` rejected — drops
the class so the markup comes back. The gate carries its own 12s failsafe
so a bundle that never boots can't strand a blank page.

SEO is unaffected: the HTML is byte-identical for every client, nothing
branches on user-agent, and a crawler has no stored token so it never
adds the class. Unreadable storage fails open the same way.

Anonymous markup opts in with `class="hide-if-logged-in"`, which
`home.html` already carried.
2026-09-01 09:19:00 -07:00
Daniel Salazar 7584dffc4b fix: metering for ai (#3703) 2026-09-01 09:11:04 -07:00
jelveh b400e06473 Remove login landing fade-out overlay
Drop the delayed fade-out when a logged-in user reaches the GUI and remove the landing overlay immediately instead. This avoids the overlay sitting above the email/phone/card verification gates because of its z-index.
2026-08-31 20:13:58 -07:00
jelveh 22c3316109 Hide app landing after session restore
Remove the /app landing overlay once `whoami` confirms the user is authenticated, even if the server rendered the page as anonymous because the session cookie was missing. This prevents the overlay from covering the email, phone, and card verification gates during localStorage-based session restores.
2026-08-31 20:07:47 -07:00
404oops a27852de0a Merge pull request #3655 from HeyPuter/FK/normalize-openai-format
Comprehensive Normalization.
2026-08-31 22:24:34 +02:00
jelveh f80a0d72b3 fix: dashboard 'Add Existing Account' dialog buried on phones/tablets
On device-phone/device-tablet, style.css pins every .window to
z-index 9999999 !important — including the fullpage dashboard window.
The Add Existing Account login dialog opens with backdrop: true, and
the .window-backdrop wrapper is not a .window, so it kept its small
inline z-index and rendered entirely behind the dashboard.

Pass stay_on_top: true like every other backdropped dashboard dialog,
and make the Forgot-password child dialog inherit the flag so it does
not open buried under the now stay-on-top login window's backdrop.

Verified with Playwright on iPhone 13 emulation and a desktop
viewport: login and recover-password dialogs stack above the
dashboard in both.
2026-08-30 21:45:43 -07:00
Nariman JelvehandClaude Fable 5 3e8b2735bb Cap /app/<name> meta descriptions at 150 characters (#3698)
The app shell fed the row's full description into <meta name=
"description">, og:description and twitter:description. Long
descriptions now collapse whitespace and cut at a word boundary with an
ellipsis, never exceeding 150 characters.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-30 20:17:04 -07:00
Daniel Salazar 5065bcbf0a fix: document allowedAppIdsGate's real non-app-gate behavior (PUT-1667) (#3669)
The JSDoc claimed non-app actors are rejected; they are passed through, and
the check reads `actor.app` rather than `effectiveApp`, so an app in the
token chain is invisible to it. Callers depend on the pass-through — routes
combining `adminOnly` + `allowedAppIds` are reached with an admin's browser
session, and the dev-account surface is called from the desktop's own
session — so the docs are corrected instead of the behavior, on the gate and
on the `allowedAppIds` route option. Tests cover the gate per actor shape
(user session, worker, full-access token, app-issued access token,
app-under-user allowed/disallowed) and the admin composition.
2026-08-30 12:55:39 -07:00
404oopsandClaude Fable 5 7e1cdefd98 feat(ai): backfill provider catalogs from live /models listings
A live audit of every keyed provider's /models endpoint against the hardcoded
catalogs found no stale entries but large gaps. This backfills them under four
rules: nothing vendor-deprecated, nothing without a price confirmed on the
vendor's official pricing page (each entry's source was recorded during
review), nothing absent from the live /models listing, and nothing that fails
a live routing probe.

Added: 46 Alibaba entries (qwen3/3.5/3.7/3.8 families, VL/omni/MT lines, and
Model Studio's hosted GLM/DeepSeek/Kimi third-party models) plus 9 dated
aliases; OpenAI chat-latest and gpt-4o-2024-11-20 plus 16 snapshot aliases;
Gemini gemma-4-31b-it and gemma-4-26b-a4b-it (vendor-documented free tier)
plus rolling -latest aliases; Mistral-hosted zai-glm-5-2 and a
mistral-medium-3.5 alias; deepseek-v4-flash-vision-exp; glm-5.3-flash.

Culled by the rules: 15 vendor-deprecated OpenAI entries (the 3.5/4/4-turbo
legacy line, gpt-4o-2024-05-13, o1-pro, four chat-latest predecessors, the
5.x codex line — deprecations page, most shut down 2026-10-23) and dated
aliases onto the deprecated o1/o3-mini/o4-mini; qwen3-vl-flash-2025-10-15
(live routing probe returned upstream 400 twice). Tiered Alibaba prices are
encoded at the base tier and busy-hour rates where time-of-day priced, noted
in comments.

Every surviving addition was verified end-to-end through a local deployment:
58/58 answered a live prompt, including all 46 Alibaba entries and every
spot-checked alias.

Not changed, flagged for maintainers: pre-existing o1, o3-mini and o4-mini
entries are now vendor-deprecated (shutdown 2026-10-23); the pre-existing
deepseek-v4-flash/-pro prices no longer match DeepSeek's current pricing
page; gemma-4 emits its own <thought> markup inline in content.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-30 20:50:46 +02:00
404oopsandClaude Fable 5 4199a2506d feat(ai): forward Mistral prompt_mode so magistral can emit separable thinking
Live-probing magistral-small-latest showed the model inlines its reasoning as
answer prose in a flat string — no ThinkChunk content, no markers, nothing a
client can separate. Mistral's chunked thinking shape is requested via
`prompt_mode: 'reasoning'`, which the provider previously dropped on the
floor: there was no way to even ask for it.

`custom.prompt_mode` now forwards to the SDK's `promptMode`, following the
BytePlus custom-params precedent. Opt-in rather than a default because the
API rejects the mode where the account/model lacks it ('Reasoning prompt
mode is not enabled for this model', code 3051) — verified end-to-end: the
3051 travels back through the stack, which also proves the parameter is
delivered. The moment Mistral enables the mode, the ThinkChunk content flows
into the existing splitter and comes out as `message.reasoning` and
`reasoning` stream chunks with no further changes.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-30 20:50:46 +02:00
404oopsandClaude Fable 5 4aa66c7c9b feat(ai): make puter.ai.normalize tri-state — unset means the release-date policy
The SDK-wide flag now mirrors the per-call option's tri-state. Left unset
(the default), the release-date policy applies: models released on or after
2026-09-01 are normalized to the OpenAI shape, older models keep their
vendor-native shape, and nothing rides the wire — the server's policy
resolution decides. Setting `true` force-normalizes every chat() call
regardless of release date; `false` disables normalization for every call;
either explicit value is sent on each call that doesn't set its own.
A per-call `normalize` overrides the flag in both directions, and assigning
`undefined` restores the policy.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-30 20:50:15 +02:00
Daniel Salazar afe46a17a0 fix: alaem cascade (#3668) 2026-08-29 03:41:23 -07:00
Juan Fernando Castro 81d700d146 fix: publishing a directory takes ownership, not write (PUT-1619) (#3654)
Creating a hosted subdomain gated `root_dir` on `write`, and hosting serves
everything under that directory with the ACL deliberately bypassed. So a
recipient of a `write` share could point a `*.puter.site` subdomain at the
owner's folder and make the subtree world-readable — continuously, covering
files the owner added later, with the row under the recipient's account where
nothing the owner can list would show it. `update` had the same gate for a
changed `root_dir`.

`#checkPublishAccess` now decides both: the actor's own tree still takes
`write`, anyone else's takes `manage` — "Can edit & share", the level that
delegates the decision.

Keyed on who owns the entry rather than asking for `manage` outright, which is
what the ticket proposed. `manage`'s is-owner implicator declines to answer for
app actors, so a flat `manage` would refuse every app publishing a directory
its user handed it, with no way for the app to obtain the grant. The write
check still runs first — it is what masks a directory the caller cannot see as
a 404 — and `manage` satisfies every lower mode, so the order costs a
manage-holder nothing.

The GUI's Publish As Website item reuses the own-it-or-`manage` answer it
already computes for sharing, so it is not offered where this would refuse.
Docs state the rule on `hosting.create()` and in `share()`'s level list.

Regression tests fail without the driver change: a write-share recipient is
refused on create and on repointing an existing subdomain, while `manage` and
the actor's own directory are accepted.
2026-08-29 01:46:14 -07:00
LEANDER ROSHAN L d5e9c14b0d fix: update sharing indicator #3649 (#3665) 2026-08-29 01:43:40 -07:00
Juan Fernando Castro c17613669c fix: open read-only shared files instead of forwarding write_url=undefined (#3667)
Opening a file you hold read-but-not-write access on (a read-only share)
failed silently. For such a file the backend correctly omits write_url from
the /open_item signature, but launchApp appended it to the app iframe URL
unconditionally. URLSearchParams coerces undefined to the string \"undefined\",
so the app received puter.item.write_url=\"undefined\" — truthy, so the editor
believed the file was writable, and an invalid URL, so it broke on open. Only
read-only shares hit this; files you can write carry a real write_url.

Extract the puter.item.* param building into append_signed_item_params and
guard the write_url append so it is only added when present. launchApp.js is
too coupled to UIWindow/jQuery/window globals to unit-test directly, so the
pure helper carries the logic and its own test, matching the helpers/ pattern.

Regression test pins both directions: a read-only signature omits write_url
entirely (no \"undefined\"), and a writable signature still forwards it.
2026-08-29 01:42:46 -07:00
Juan Fernando Castro cdfd38bd4b fix: stop an embedded fs: in a path from escalating to full FS access (#3666)
The fs-path-to-uid permission rewriter split permission strings on the raw
\`fs:\` substring instead of parsing on component boundaries. A path can
itself contain \`fs:\` — a home dir named \`…fs\`, or \`fs\` in the mode
position — and the raw split mistook that for the mode delimiter.

Crafting \`fs:/<victim>fs:junk:read\` made the split do two things at once:
it dropped the \`junk:read\` mode, and it consumed the trailing \`fs\`, turning
the harmless-looking nonexistent path shown in the consent dialog into the
victim's real home path. The rewriter then stored a bare \`fs:<home_uuid>\`,
which subsumes every mode via parent-permission matching — full
read+write+delete from a request the user saw as \`junk:read\`.

Parse with PermissionUtil.split() in both matches() and rewrite(), matching
every other permission parser, and locate the \`fs\` component by position
(after an optional \`manage\` prefix). The path component is now exactly what
sits between \`fs\` and the next unescaped colon, and all trailing components
are preserved as the mode — so an embedded \`fs:\` is data, never a delimiter.
The crafted input now addresses the nonexistent \`/<victim>fs\` and 404s.

Regression tests pin both halves: the exact PoC must 404, and an \`fs:\` in
the mode position must survive the rewrite instead of collapsing to a bare
permission. Both fail on the old rewriter and pass on the fix.
2026-08-29 01:41:40 -07:00
Juan Fernando Castro 693e86d763 Merge pull request #3657 from HeyPuter/juancastro/put-1616-shareblocks-is-missing-requireuseractor-any-authorized-app
🔧 PUT-1616: Close /share/blocks to app tokens
2026-08-28 16:46:04 -04:00
Daniel Salazar 6d10ef2821 fix: stop a URL parameter putting puter.js into app mode (#3660)
`env = 'app'` was decided by the presence of a `puter.app_instance_id` query
parameter and nothing else, so a crafted link put any page that loads the SDK
into app mode — and app mode is what makes the URL's `puter.api_origin`
authoritative for every credentialed call.

App mode now also requires the document to be framed. The GUI only ever
launches an app into an iframe, so this costs a real app nothing while a
top-level document carrying the parameters is treated as the third-party site
it is. It is not an attestation that the framing document is the GUI — a
cross-origin ancestor's identity is not readable — so the token paths carry the
rest:

- The `web` boot branch adopted a stored token without consulting the origin it
  was bound to, which is what completed the fixation: one link plants a token
  bound to an attacker's origin, and every later visit adopted it. It now
  applies the same binding rule the app branch does, and drops a token that
  fails it rather than leaving it to be re-read.
- `signIn()` had no env guard, and in app mode delivered a real token to
  whatever `puter.api_origin` the launching URL named. Apps get their token
  from the session that launched them, so it now rejects there with
  `not_available_in_app`. Nothing internal reaches it in app mode:
  `authenticateWithPuter` and both implicit-auth call sites already gate on
  `env === 'web'`.
- The cross-origin-isolated branch polled `${this.APIOrigin}/login/wait` and
  adopted whatever came back. Pinned to `defaultAPIOrigin`, the same way the
  popup and its message handler already pin `defaultGUIOrigin`.

Backward compatibility: no signature, response field or existing error code
changes. The only behaviour a caller can observe is the new `signIn()`
rejection, which replaces a call that could not have worked correctly.

Covers the SDK side of the parameter PUT-1395 and PUT-1427 closed on the GUI.
2026-08-28 11:16:10 -07:00
Daniel Salazar 9fab4742c9 fix: complete the malformed-input guards, including the two paths still live (#3663)
Review of the previous change found three of its claims unmet.

The image-generation crash it reported fixed is still reachable. The assert was
scattered across three helpers, and Gemini and OpenAI call `isHttpUrl` directly
on `input_images` without going through any of them — so two of seven providers
still 500 on a non-string. `isHttpUrl` now refuses a non-string itself, and the
shape is settled once in `ImageGenerationDriver.generate`, where the driver call
arrives, rather than per helper. That also covers `input_images` that isn't an
array, which produced a different crash per provider.

The sixth case in the ticket, previously unlocated, is
`Messages.js` reading `tool_call.function.name` with no guard — reachable with
`{"messages":[{"role":"assistant","tool_calls":[{"id":"x"}]}]}`. Guarded, along
with the same shape in `make_claude_tools`: a TypeError there carries no status,
so the retry loop reads it as a provider failure and marks the route unhealthy
for every caller.

`#hardExpiryFromExpiresIn` returning null for a bad type moved the failure past
the session INSERT, leaving an orphaned non-expiring row and still answering
500. Reverted; the controller guard is the fix, now covering fractions,
negatives and unparseable durations rather than only wrong types.

Also: the batch write handlers check that the body is an array but not what is
in it, so a null element 500s the same way; `#requireObjectBody` accepted an
array despite its name; `handleCreateAccessToken` destructured a body that may
be absent; and two AGPL notices had been rewrapped with a Markdown link.
2026-08-28 11:14:30 -07:00
Daniel Salazar f3a46a9be4 fix: stop sockets outliving the session that authenticated them (#3658)
A socket was checked once at handshake and never again. Nothing in the backend
disconnected one, so logout-everywhere, password reset, session revoke and
suspension all left every connection streaming legacy FS entries, upload paths
and notification bodies — up to 400 per account — on a credential that had
already been revoked.

Three gaps, three fixes:

- The handshake skipped the suspension and pending-verification checks every
  authenticated HTTP route gets. `decideSocketAuth` now applies both.
- `revokeCascade` reports which rows it revoked, AuthService announces that as
  `auth.sessions.revoked`, and SocketService drops the account's room. The
  whole room goes, not just the revoked session: narrowing it would need
  `fetchSockets`, which the adapter builds on `serverCount()` — and that calls
  node-redis's `send_command`, which ioredis does not implement. A connection
  whose session survived reconnects on its own and re-authenticates.
- A bulk suspension writes `user.suspended` without touching `sessions`, so no
  revoke fires. A five-minute sweep re-verifies each live socket's token and
  drops the ones that no longer authenticate. De-duplicated by token, since a
  browser's tabs share one.
2026-08-28 11:12:27 -07:00
Daniel Salazar e460e9034c fix: answer 400 instead of 500 on four malformed-input paths (#3662)
Each of these read a field off caller input that wasn't the shape the code
assumed, threw a TypeError, and was served as a 500 with a critical page.

- POST /fs/write and /startWrite: a request whose body never parsed left
  `req.body` undefined, and the first read of `fileMetadata` threw.
- POST /drivers/call, image generation: `input_image` / `input_images` entries
  are documented as strings but nothing checked, so a number or an object
  reached `.startsWith`. Type confusion on caller input, so reachable on
  demand rather than by accident.
- POST /auth/create-access-token: `expiresIn` went to the expiry parser
  unvalidated, where anything but a string or a number has no `.trim`.
- POST /login/wait: destructuring `session` out of an absent body threw.
  Same class as the POST /login body ticket; that one covers /login itself.
2026-08-28 10:22:51 -07:00
Daniel Salazar 5c8defb940 fix: bound failed WebDAV Basic-auth attempts per account and per address (#3659)
The only thing in front of the bcrypt compare on the DAV host was the 600/min
request ceiling, keyed on a fingerprint that rotates with client-controlled
headers. /login guards the same credential with a captcha and two much tighter
buckets; DAV had neither, which left password and TOTP guessing viable from a
host that answers any origin.

The request ceiling can't double as a credential ceiling — a working DAV client
resends its credentials on every request — so the new buckets count only
verifications that failed: 10 per account and 50 per address per 15 minutes,
sized like /login's. They're read before the compare, so an exhausted bucket
costs no bcrypt round, and successful requests never draw them down. `-token`
attempts are held by the address bucket alone; bucketing them per account would
let bad tokens lock out good ones.

Reading a bucket without spending from it is new, hence `peekRateLimit` and the
matching `peek` on all three backends.

Also documents the DAV limits, which were undisclosed.
2026-08-28 10:21:34 -07:00
Daniel Salazar 88b851a423 fix: derive the signup client IP from req.ip instead of x-forwarded-for (#3661)
Both signup paths read the x-forwarded-for header directly for the
puter.signup.validate event, the puter.signup.success event and the
signup_ip_forwarded column. Behind an appending proxy a client can prefix that
header with anything it likes, which gives every per-IP abuse signal a fresh
bucket per request.

req.ip is the value `trust proxy` resolves, so it is the one the server can
stand behind; server.ts already uses it for the ip.validate gate for exactly
this reason. All three now share one derivation, so a per-IP counter is written
and read under the same key. The raw chain is still recorded, but only as
audit_metadata.ip_fwd, where nothing keys on it.

Rows written before this hold whatever the proxy appended, so per-IP velocity
over the trailing window is undercounted until they age out.
2026-08-28 10:19:24 -07:00
Juan Castro 94d798ccce fix: close /share/blocks to app tokens
All three block routes set only `subdomain`, `requireVerified` and a rate
limit, and the controller's actor check accepts anything carrying a user, so an
app-under-user token passed. Unlike `share` and `revoke`, nothing else bounded
them: an app the user authorized once could read exactly whom they had blocked,
then clear the list — blanket switch and per-username — and reopen the channel.

A block list is a personal safety control, not something an app can otherwise
reach, so it falls outside the "an app can share what it can already reach"
carve-out the other share routes rely on. `requireUserActor` on all three, and
deliberately without `allowFullAccessToken`: this is security management, which
stays closed to every access token, personal ones included.

The only caller is the desktop's Blocked Senders window, which sends the user's
own session token.

Tests fail without the gate: an app token minted through
/auth/get-user-app-token is refused on all three routes while the user's own
session still succeeds, and a metadata check pins the gate on every block route
so a fourth cannot quietly ship without it.
2026-08-28 12:32:17 -04:00
404oopsandClaude Fable 5 07998c32ed test(ai): point the provider-level replay test at the real driver fallback test
The comment claimed the fallback loop cannot be driven from a test — false
since ChatCompletionDriver.test.ts gained a two-attempt fallback test that
drives the actual loop. Say where that test lives instead.

Co-Authored-By: Claude Fable 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 04d12d9524 fix(ai): route Mistral streamed thinking to the reasoning channel, unconditionally
Third triple-check round. Ten findings were put through independent skeptics
first; six did not survive — pre-existing on main, inert, or resting on a false
premise — and are not acted on here.

Mistral streamed thinking now goes to `reasoning` on every path. The previous
commit gated the split, which made this the only place in the repo where
chain-of-thought reached the visible text channel, and made Mistral the only
provider whose streamed chunk *types* depend on a response-format flag. Every
other reasoning path routes thinking to `reasoning` unconditionally —
ClaudeProvider's thinking_delta, the DeepSeek/OpenRouter rename, and this
branch's own Responses summary-delta handler. Removing the gate restores that
uniformity and makes the documented promise that streaming is unaffected by
normalization true again; the two opposing tests collapse into one that runs the
same fixture with and without `normalize` and asserts identical event streams.

Docs stop claiming older models are unchanged. Four reasoning fields were made
consistent across all models, ungated, and one of them removes a field: on
non-streaming responses `message.reasoning_content` is now `message.reasoning`.
chat.md gains a table naming all four so a caller reading `reasoning_content`
learns why it disappeared, instead of reading that nothing changed for them.

Adds the driver-level fallback test. Writing it surfaced that the invariant it
was meant to assert is false and always was: the driver rewrites string
`content` into text blocks in place on the caller's own messages
(`normalize_single_message`, pre-existing) before any provider runs. The test
now asserts what is true and load-bearing — both attempts receive the same array
reference, and the reasoning artifacts survive attempt 1 so attempt 2 can still
replay them.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oops 875aac5db0 Revert "docs: record AI-normalization review rules in CLAUDE.md"
This reverts commit f220276724fdc1a16ae56f9c95e41773fe4231ed.
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 216e8d68ed fix(ai): scope the normalized flag, gate the Mistral stream split, correct four doc claims
Acts on a second triple-check audit.

`normalized` means one thing. The driver stamped it on both the OpenAI-shape
path and the legacy `response.normalize` path, which converts toward Anthropic
blocks — so a caller could get `normalized: true` alongside array content, and
the flag told them nothing they could branch on. The legacy branch no longer
sets it. That branch is reachable only by a direct driver call with
`response.normalize` and no `normalize`; the four wire routes pin
`normalize: false`, which skips it.

Mistral streaming, split by concern. Flattening a reasoning model's chunked
`delta.content` to a string is a correctness floor and stays ungated — the
shared handler passes the value straight to `addText`, so an array reaches the
caller as stringified objects. Splitting the thinking text out into a
`reasoning` delta is the dialect change and now sits behind the policy gate
like the non-streaming remap. On the native path the thinking text is kept
inline rather than dropped.

Mistral `finishReason` is deleted only once its value carried over. The delete
ran unconditionally, so a non-string `finishReason` with no `finish_reason`
left the choice with no finish reason at all.

Four doc claims corrected against the code paths they cover: `content` is
string-or-null on normalized responses (tool-only turns carry no text, and the
`// always a string` example comment was wrong); `reasoning_details` is not
scoped to normalized responses, since Responses models emit it either way; and
the release-date rule depends on the serving provider's own dates — OpenRouter
derives them from its live API, so models newly listed there from 2026-09-01
normalize by default.

Adds the test the copy-on-write fix was actually for: one messages array sent
through two sequential calls, asserting the caller's array is untouched and
both attempts carried the thinking signature. That is the fallback hazard; the
harness wires one provider per model, so the fallback loop itself cannot be
driven from a provider test.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 c1b420e480 fix(ai): gate the Mistral remap, unify the reasoning join, stop mutating caller messages
Acts on a triple-check audit of this branch.

Gate the Mistral dialect remap. The camelCase→snake_case rewrite and the
chunked-content flattening were firing for every Mistral call regardless of
`normalize` or the cutoff, deleting `finishReason` and `message.toolCalls` out
from under any caller reading them. Both now sit behind the policy resolution
the driver already used, extracted as `shouldPresentAsOpenAI` so the provider
and the driver cannot drift. The streaming chunk-array split stays ungated:
handing an array to `addText` is a plain bug, and streamed chunks are
provider-uniform by design. The conformance matrix now passes `normalize: true`,
which is the contract it was always testing.

Unify the reasoning join. Three code paths produced two separators while one
doc sentence described them all: the coercer joined thinking segments with '',
the Responses handler and Mistral with '\n\n'. The coercer now matches, and
chatresponse.md's claim is true for every path it covers. Text blocks still
join with '' — Anthropic splits prose mid-sentence across them.

finish_reason is an open set. chat.md's normalize bullet and the SDK
ChatMessage typedef still declared a closed four-value set, contradicting the
documented pass-through of unmapped vendor reasons and the coercer that
implements it.

Stop mutating caller messages. Both reasoning-replay input paths deleted
output-only fields from the caller's own message objects, which the driver
reuses across fallback attempts. Both strip a copy now; tests pass a frozen
message through each.

Drop three dead things the type cleanup left: the no-op ChatProvider
checkModeration stub (no subclasses, no callers — its removal restores a
pre-existing baselined TS2420), the redundant second normalizeReasoningContent
call in BytePlus and ZAI, and the coercer's bare-string branch that no provider
reaches. A bare string now passes through by reference instead of being
coerced.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:09:14 +02:00
404oopsandClaude Opus 5 fbb9038651 revert: restore the 31 typecheck-baseline entries this branch removed
Commit 6b4f5b01f dropped 31 entries (49 error instances) from
tools/typecheck-baseline.json as a side effect of fixing the AI-code type
errors they suppressed. The file is a ledger the maintainer owns, and shrinking
it inside a feature PR ships a CI-gate change nobody asked for.

Restoring is free: tools/typecheck.mjs fails only on *regressions*
(count > baseline), so stale entries are non-fatal — the gate prints "49
baselined error(s) fixed. Run npm run typecheck:update to lock that in." and
exits 0. Nothing else in the repo reads the file, vitest included, and
tsconfig.build.json's noCheck:true means emit is unaffected. The errors stay
fixed in the source either way; only the ledger's own bookkeeping is deferred
to whenever the maintainer chooses to regenerate it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 74640d4e83 fix(ai): flatten Mistral reasoning models' chunked content
`magistral-*` returns `message.content` as a ContentChunk[] rather than a
string, with the thinking text nested one level deeper inside `thinking`
chunks. The camelCase remap did not touch it, so a non-streamed magistral
response reached the caller as an array with no `reasoning` — the one case
left where a provider did not produce the equalized shape this branch
promises. Streaming had the matching bug: the chunk array was handed to
addText, which would have stringified it into the text stream.

Both paths now split chunked content into a string `content` plus a
`reasoning` string, joining multiple thinking chunks with a blank line as the
Responses handler and the Anthropic coercer do. The streaming fix rides the
existing Mistral-only `chunk_but_like_actually` hook, so no new deviation is
introduced.

The conformance matrix had no Mistral reasoning fixture, which is why it
missed this; it now has one carrying chunked content, verified to fail
without the flattening.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 3b8de6a90b docs: record AI-normalization review rules in CLAUDE.md
Rules from the post-normalization review: don't silently change what existing
models return, verify doc claims against every code path they cover, record
self-disclosed defects in the PR draft, finish with a fresh build + suites, and
check git stash before concluding an edit was lost. Plus the standing
typecheck-baseline no-go.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Opus 5 a6830f341a fix(ai): round-trip reasoning artifacts and pass unmapped stop reasons through
Closes the reasoning gaps left open by the normalization work.

Reasoning replay. The coercer dropped Anthropic thinking-block signatures and
the Responses handler dropped reasoning item ids/encrypted_content, so a
normalized reasoning turn could not be replayed — Anthropic rejects an
extended-thinking tool-use continuation whose thinking blocks lost their
signature. Both now ride `message.reasoning_details` verbatim, and both input
paths accept them back: ClaudeProvider splices the blocks ahead of the content
(Anthropic requires them to lead), and the Responses input processor expands
them into standalone `reasoning` items. Output-only fields a replayed message
carries (`reasoning`, `refusal`, `normalized`) are stripped on both paths,
since neither upstream accepts them. The docs caveat recommending
`normalize: false` for agentic Claude loops is gone; it is no longer true.

Unmapped stop reasons. chatresponse.md promised a vendor `finish_reason` with
no OpenAI analog "passes through unchanged" — true for the Mistral remap, false
for the Anthropic coercer, which discarded it. Anthropic's `pause_turn` means
"continue this turn", so flattening it to `stop` destroyed the signal. The
coercer now passes unmapped values through verbatim, matching both the doc and
the Mistral path, and the docs gain the full Anthropic stop-reason table.

Reasoning summaries. Multi-part summaries joined with '' instead of a blank
line, and the streaming Responses path emitted no reasoning at all;
`response.reasoning_summary_text.delta` now feeds the same `reasoning` stream
channel the chat-completions handler uses.

Types. `text?: string & { verbosity?: ... }` was an uninhabitable intersection
(providers read `text?.verbosity` as an object), and the verbosity enum was
`'concise' | 'detailed'` where OpenAI accepts `'low' | 'medium' | 'high'`.
Adds `reasoning`, `reasoning_details`, and `refusal` to the SDK ChatMessage
typedef.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-28 01:08:48 +02:00
404oopsandClaude Fable 5 548d26f884 test(ai): cover Hoonify in the provider-consistency matrix
Adds the upstream Hoonify provider (feat #3499, merged after this
branch was cut) to the cross-provider output conformance suite. It
speaks the OpenAI chat-completions dialect and conforms out of the box.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-28 01:08:48 +02:00
404oops 9b67a2b1dd Add AI response normalization controls
Introduces a new `normalize` option for chat completions, plus release-date based default normalization (post-2026-09-01) to coerce provider-native outputs into a consistent OpenAI-style shape. Adds shared normalization utilities, extensive driver/provider consistency tests, and controller safeguards that pin provider-native output where route-specific translators are used. Also wires the option through puter.js (`chat` options and `puter.ai.normalize` default), updates AI/chat response docs and examples, and resolves related TypeScript typing issues reflected in the typecheck baseline.
2026-08-28 01:08:47 +02:00
Juan Fernando Castro f952b5006b Merge pull request #3652 from HeyPuter/juancastro/put-1594-permission-requests-should-auto-return-true-and-resolve
🔧 PUT-1594: Settle a permission request from what is already granted
2026-08-27 19:02:15 -04:00
Nariman Jelveh 56b4ef7f2b Leave worker notifications out of the dashboard entirely 2026-08-27 15:59:25 -07:00
Nariman Jelveh 1d3963539f Keep worker notifications out of the dashboard toasts 2026-08-27 15:47:57 -07:00
Juan Castro c4be7fabac fix: settle a permission request from what is already granted
`puter.perms.request()` already pooled a permission read and prompted only
for what was missing. The raw `puter.ui.requestPermission()` did not, so
every caller still on it re-asked the user on each launch — including
`perms.requestAppData()`, whose own docs promise the opposite, and the
driver-denial retry.

- puter.js: `ui.requestPermission()` reads what is held before prompting and
  resolves true when the whole request is covered. Only in env=app and
  env=web, the environments that raise a prompt; elsewhere the method still
  answers false without asking anyone. A check that cannot be made — no
  token, an unreadable request shape, a failed read, or one that outlasts its
  timeout — falls through to the prompt rather than standing in for an
  answer. Public signature unchanged.

- GUI: the request-permission popup asks the same question as the app, using
  the user-app token its own exchange already mints, and skips the dialog
  when the access is held. This is the one case the SDK cannot settle for
  itself: a signed-out site holds no token to check with. An origin the
  browser does not vouch for never reaches the check, since the exchange
  fails first.

Both checks are time-boxed, because each one stands in front of something
that is waiting: the popup's gates the dialog, so a stalled read would leave
the prompt unshown and the opener pending, and the SDK's spends the browser's
transient activation, which a slow read would cost the popup.

Note that driver, service and feature scopes are implicitly granted to every
app (backend/data/hardcoded-permissions.js), so requests for those now
settle silently — the dialog was asking about access the app already had.
Consent scopes (email, fs, apps, subdomains, app-data, app-root-dir) are
unaffected and still prompt until granted.

Fixes a bug this method already had on the way past: `pollDecision` read an
undeclared `permission`, so every attempt threw a ReferenceError into its
network-failure catch and the COOP-severed-opener recovery burned its full
five-minute timeout before answering false. It polls `requested` now, and
requires the whole list.

Tests: the e2e suite drove its dialogs with an implicitly-held driver
permission, so the fixture now asks for a driver nothing implies, fresh per
page load, which also removes the cross-test grant carry-over the old
revokes worked around. The reconciliation tests ask for the held scope plus
an unheld one, since a fully-held request no longer reaches a dialog. Adds a
backend contract test for check-permissions under an app-under-user actor,
which is what the two new client paths rest on.
2026-08-27 16:35:06 -04:00
Nariman Jelveh 167e281cc8 Keep read notifications in the dashboard panel
The panel listed only what the user hadn't dismissed, so acting on a
notification made it vanish. It now shows the 30 most recent regardless
of state, with read/unread as a visual flag: unread rows carry a dot and
a check to mark them read; reading one (check, click-through, toast
close, mark-all, or an ack from another tab) leaves it listed but
quieter. The badge, tab title, and header count follow the unread count.
2026-08-27 13:31:39 -07:00
Nariman Jelveh c6ecc0c786 Soften dashboard notifications empty-state title 2026-08-27 13:14:05 -07:00
Nariman Jelveh f38dec07ea Remove empty-state hint from dashboard notifications 2026-08-27 12:59:01 -07:00
Nariman Jelveh 7332993ff3 Show share notifications in the dashboard (#3653)
* Show share notifications in the dashboard

The dashboard opens its own socket but never listened for notif.* events,
so a file shared with a user on the dashboard was announced to nobody.

Add a notification center to the dashboard: a bell in the sidebar with the
unread count (a dot when collapsed, and on the mobile hamburger), a panel
anchored to it — a bottom sheet on phones — listing what the server still
holds as unacknowledged, and toasts for what arrives live. Clicking a share
lands on Files › Shared with the item selected; dismissing, and "Mark all as
read", acknowledge on the server so other tabs clear too. A toast timing out
is not a dismissal: the entry stays unread in the panel.

Arrivals fold in by uid (a regrouped share rewrites its row and toast in
place), the panel refreshes on open, reconnect and tab focus so shares the
backend folds in silently past the sender's budget still surface, and a
burst on connect is capped at three toasts plus a summary.

Backend: notif.unreads carries created_at so delivered-on-connect items can
be dated, and listings break same-second ties by id so order is stable.

The desktop's mark-ack calls move to a shared helper; UINotification gains an
optional auto-hide timeout that pauses on hover/focus.

* fix: look up notification glyphs and icons by own key only

A notification whose `source` or `icon` was a prototype key such as
`constructor` made the glyph lookup return a function, and the toast
icon builder then threw inside the socket handler — which dropped the
whole burst of toasts it was part of. The list rendered the same entry
as "[object Object]".

* fix: show dashboard toasts above open app windows

App windows opened from the dashboard are stay-on-top and stack in the
99999999+ band, so the toast container at 10000000 sat underneath them:
any notification arriving while an app was open — the usual state of a
dashboard session — was drawn behind the app and never seen.

* fix: reveal the dashboard when a toast is clicked over an open app

Clicking a share toast switches the dashboard to Shared, and the "N new
notifications" toast opens the panel — both inside the dashboard window.
With an app maximized over it, all of that happened out of sight and the
click appeared to do nothing. Minimize the covering app windows first,
the way the minimize controls do, landing the URL on the dashboard's own
route, and act once the history pop has settled so the tab switch is not
traversed over.

* fix: keep keyboard focus in the notifications panel across re-renders

Every change to the list rebuilds its rows, which dropped focus to the
document body: dismissing an entry with Enter threw the keyboard out of
the dialog (the row it had just been moved to was replaced), and so did
any arrival while an entry was focused. Put focus back on the same entry
after a render, or on the one that took a dismissed entry's place, and
let Shift+Tab from the panel itself wrap to the last control instead of
leaving the dialog.

* fix: keep the notifications panel inside short viewports

The anchored panel is bottom-aligned with the bell and grows upward, but
its height was capped only by the viewport (560px or 100vh - 24px), not
by the room above the bell — on a short window the header ran off the
top of the page. Cap it to that room, so the list scrolls instead.

* fix: make notifications with nowhere to go inert in the panel

An entry with no target (a worker deploy result) showed a default cursor
yet, when clicked, was dismissed and took the panel with it — nothing
opened, the entry was gone, and the list had closed. Render such entries
as text with the ✕ as their only action; only share entries remain
buttons. The ✕ now names its notification for screen readers, since on
an inert entry it is the only focusable control.

* fix: stop a listing in flight from resurrecting dismissed notifications

The list is refreshed on open, on reconnect, and when the tab comes back
into view; a dismissal landing while that listing is in flight was undone
when it resolved, since the server's snapshot predates the ack. When the
listing also resolved after the server's own ack event, nothing removed
the entry again and it stayed until the next refresh. Leave out of the
reconciliation whatever was acknowledged — here or in another tab —
since the listing was requested.
2026-08-27 12:49:20 -07:00
Daniel Salazar 909949c68d fix: failed email alarms (#3651) 2026-08-26 23:22:14 -07:00
Juan Fernando Castro 5a6a1020a1 Merge pull request #3644 from HeyPuter/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat
🔧 PUT-1585: Share flag in readdir shared users in stat
2026-08-26 18:51:30 -04:00
Nariman Jelveh 7472002cf3 feat: add Hoonify AI chat provider (#3499) 2026-08-26 15:39:29 -07:00
Juan Fernando Castro 7bd6a2d89b Merge pull request #3646 from HeyPuter/juancastro/put-1590-sharing-shared-files-are-not-different-than-regular-files
🔧 PUT-1590: Mark shared items in the file listings
2026-08-26 17:55:19 -04:00
Juan Castro 3f7dac63b9 Merge remote-tracking branch 'origin/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat' into juancastro/put-1590-sharing-shared-files-are-not-different-than-regular-files
# Conflicts:
#	src/gui/src/UI/Dashboard/UIShareModal.js
#	src/gui/src/UI/UIWindowShare.js
2026-08-26 17:06:25 -04:00
Juan Fernando Castro 3d28b2bb99 Merge pull request #3647 from HeyPuter/juancastro/put-1586-1599-share-dialog-correctness
🐛 PUT-1586 + PUT-1599: Share dialog says what it can grant, and what it did
2026-08-26 16:58:36 -04:00
Juan Castro 63688f81b4 Merge branch 'main' into juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat 2026-08-26 16:55:51 -04:00
Juan Fernando Castro 14ac6542e3 Merge pull request #3645 from HeyPuter/juancastro/put-1589-shared-file-fs-events-seem-to-be-lost
🐛 PUT-1589: Shared file fs events seem to be lost
2026-08-26 16:54:09 -04:00
Juan Castro 5739f4e59f fix: stop pushing file metadata to revoked recipients (PUT-1611)
The realtime fan-out resolved its audience straight from the `share`
index, which has no live-grant check. `/auth/revoke-user-user` deletes
the permission and leaves the index row, so a revoked recipient's socket
kept receiving name, size, masked path and mtime for every write and
move under the folder, with no expiry.

The service already solves this elsewhere — `#reachingHolders` returns
exactly the holder/entry pairs whose grant still stands, and
`listSharedWithMe` was moved onto it for the same reason. The realtime
path never got the same treatment; it does now.

Free on the unshared path: with no share rows reaching the entry there
are no holders to check, so the write path every user takes is unchanged.
Pinned by a test that counts permission reads.
2026-08-26 16:35:38 -04:00
Juan Castro cae51ce67a fix: trashing a top-level share is a removal, not a move
A grant on the entry itself is keyed on uuid, so it follows the entry
into the owner's Trash. Both ends of the move then resolved, and the
recipient was told the shared item had moved — to the GUID name Trash
gave it. Their own copy got renamed to a GUID and stayed on screen.

`shared-with-me` has always omitted trashed entries, so the listing and
the event disagreed; only the event was wrong. Trashing now reports
item.removed at the path the recipient knew, which is also what the
desktop's data-path selector needs to find the row. Restoring out of
Trash reports item.added.

A move that leaves the recipient's masked address unchanged now stays
quiet — a share masks its own root, so the owner shuffling it around
their tree is invisible to the recipient and the event carried nothing.
2026-08-26 16:25:02 -04:00
Juan Castro 8e622189ca fix: tell every recipient when a shared file disappears
Both ways a file can vanish were silent for anyone holding the folder
above it, so a third party's window kept showing a file that was gone
and 404'd on click.

A delete built its audience from the permission rows it removed, and a
file inside a shared folder has no grant of its own — only the folder
does — so the audience was empty. A move resolved its audience from the
entry's new path, and the GUI's Delete is a move to the owner's Trash,
where no recipient has a share.

Resolve the audience from where the entry was rather than from the
grants that went with it:

- Deletes also fan out to holders reaching the entry through an
  ancestor, coalesced by parent so a subtree stays a couple of queries.
  A holder covered by both passes is told once.
- Moves resolve both ends. Reaching both is item.moved, only the
  destination item.added, only the origin item.removed.

Recipients are named by the path they knew, masked through their own
share rather than the owner's tree.
2026-08-26 14:54:06 -04:00
Daniel Salazar c1c830fc32 docs: add puter site config docs (#3648) 2026-08-26 11:11:25 -07:00
Juan Castro cf00ebb9d4 Merge branch 'main' into juancastro/put-1589-shared-file-fs-events-seem-to-be-lost 2026-08-26 13:33:24 -04:00
Juan Fernando Castro b05ee63cb2 Merge pull request #3641 from HeyPuter/juancastro/put-1568-rename-srcguisrchelpers-files-to-camelcase
🧹 PUT-1568: Rename GUI helper files to camelCase
2026-08-26 13:06:58 -04:00
jelveh 40667bc811 Open Document Picture-in-Picture windows on behalf of apps
Browsers only allow documentPictureInPicture.requestWindow() from a
top-level document, and an app lives in an iframe, so an app calling it
gets NotAllowedError ("only allowed from a top-level browsing context").
The `document-picture-in-picture` token in the iframe's `allow` list does
nothing — it is not a policy feature the browser knows. Video PiP
(video.requestPictureInPicture) already works inside apps.

The GUI is the top-level document, so a new PictureInPictureService opens
the window for the app and fills it with an iframe of a page the app names,
which must come from the app's own origin (checked against the message's
origin, now carried on the IPC caller context). One window per app
instance; it closes with the app's window, and the app hears about a close
it didn't ask for. The window's opener is the GUI, so the page inside it
can reach its app's frame through parent.opener.frames and share objects
directly — a MediaStream included, which postMessage cannot carry (tracks
are not transferable between windows in Chromium).

puter.js gains puter.ui.requestPictureInPicture({ url, width, height,
onClose }) and puter.ui.exitPictureInPicture(), with docs.
2026-08-26 09:10:02 -07:00
Juan Castro 4b3d1112d1 Say when a share changed nothing
Sharing a file with someone who already had it answered "Shared with
X", the same as a first share, so the dialog claimed to have done
something it had not.

The service already knew — it computes isNew to decide whether to
notify the recipient — but the flag stopped at the controller. It now
travels on share results only; a listing describes standing access and
says nothing about it.

Which of the three things happened is settled client-side, since the
mode each recipient holds is already on screen: created access, raised
or lowered it, or changed nothing. That keeps the previous mode off the
wire, and an older backend that omits isNew still reads as a share,
which is what these dialogs said before.
2026-08-26 11:57:05 -04:00
Juan Castro 70edef561f Stop offering an access level a delegate cannot grant
A recipient given "can edit & share" could not pass that level on: the
dialog offered it, the server refused it, and the refusal was a bare
403 Forbidden that reads as a bug. Handing out manage needs authority
over manage, which only the owner has — the refusal is right, the dead
end and the silence were not.

The dropdown now withholds it from anyone who does not own the item; a
row already set to it keeps it, so opening the dialog cannot downgrade
the owner's own grant, and a mixed selection follows its strictest item.

The server says why, and only to someone who can already share the item
— a stranger still gets the ACL's own safe error, which does not admit
the node exists.

Verified against a running server: a delegate grants read and write as
before, and manage now answers cannot_delegate_manage with a sentence
naming the owner as the one who can.
2026-08-26 11:48:54 -04:00
jelveh e430ade7bd Reach Shared and Trash from the dashboard Files tab on phones
Below 480px the Files tab hides its directories sidebar, which was the only
entry point to Shared (a query, not a directory) and Trash (filtered out of
the Home listing). Home now carries a row for each; CSS shows them only at
the widths where the sidebar is hidden, using the complementary breakpoint so
the two never both show or both disappear.

The rows borrow the item markup for layout but aren't `.item`, so sorted
insert, selection restore, share-link selection and socket updates ignore
them. The footer count and keyboard select-all exclude them explicitly, and
the placeholder-removal sweeps leave them in place. They navigate on tap and
offer the same menu as their sidebar entry via the ⋯ button or long-press;
the Trash icon tracks empty/full alongside the sidebar's.

Also guard the `window.user.directories` lookup in renderDirectory: it is
undefined for some sessions, and since `puter://shared` doesn't look like a
path it always reached that branch, threw outside the try, and left the
spinner up with navigation stuck — from the desktop sidebar as well.
2026-08-26 08:18:13 -07:00
Juan Castro 519b132355 Inset the shared marker onto the icon glyph
It sat on the icon box's corner, which is 5px outside the artwork on
every side, so the dot read as clipped — half of it hanging over empty
padding with the icon's own drop-shadow falling across it. Nudged in on
both axes: 7px on the desktop, 2px on the dashboard rows.
2026-08-26 11:16:49 -04:00
Juan Castro b08dc70654 Move the shared marker onto the icon's corner and shrink it
Review feedback: too big, and in the wrong place. It sat in the badge
cluster, which the dashboard pins to the row's top-left corner rather
than to the icon — and at 12px it dominated a 24px row icon.

Now a dot on the icon itself, lower-right: 9px on the desktop's 45px
icons, 7px on the dashboard's 24px rows. Anchoring to .item-icon rather
than to the badge cluster is what keeps it on the corner at both sizes.

Dropping the people glyph with it — unreadable at either size, and
colour was the signal the ticket asked for.
2026-08-26 11:07:48 -04:00
Juan Castro a1d027e4a1 Mark shared items in the file listings
A shared file looked exactly like a regular one. The data to tell them
apart arrived with the readdir/stat share flag; nothing rendered it.

Adds the badge to both listings — UIItem (desktop, explorer windows,
file dialogs) and the dashboard's Files rows — fed from is_shared, and
keeps it in step with the share dialogs: both funnel every grant, mode
change and revoke through one render, so the badge follows without
waiting for a re-listing.

Inherited access is deliberately not badged. It is a state of the folder
that was shared, so marking every file inside would repeat one fact on
hundreds of items; the backend flag is direct-only for the same reason.

The icon (owner-shared.svg) and the strings (item_shared_by_you, in 40
locales) were already in the tree, unused — only the wiring was missing.
The blue ring is doing the work: list view shrinks badges to 8px, where
a glyph is illegible and the white circle the sibling badges use
disappears into the row.
2026-08-26 10:56:10 -04:00
jelveh defa37c106 Fix dashboard files tab mobile scrolling
Replace the Files tab's viewport-height sizing with flex-based layout sizing through the dashboard content chain. This keeps the internal file list within the visible content area on mobile browsers, preventing the bottom rows from being clipped when browser toolbars are shown.
2026-08-26 07:42:44 -07:00
Juan Castro 1514ab3cb5 Ignore fs events replayed by replication
Raised in review: could an event from another node re-trigger the
fan-out? Not today — broadcast carries outer.* and pubsub.* only, so
fs.* never crosses a node boundary, and the emitted outer.gui.* is
consumed on the peer by SocketService while ShareService listens to
fs.* alone, so nothing re-enters.

That safety is a property of what broadcast happens to replicate, which
is not this service's to rely on. The handlers now skip anything tagged
from_outside: the node that did the write has already told the audience,
and a second fan-out would only duplicate it.
2026-08-26 09:22:49 -04:00
roar_zx ca25915a38 docs: link Dutch and Odia README translations (#3635) 2026-08-26 00:59:16 -07:00
Juan Castro ee2b5adcf9 Tell share recipients about creates and renames
A recipient's client keeps its cache fresh from fs events pushed over
their socket, and ShareService fans those out to holders — but only for
write, move and delete. A new entry emits fs.create.<flavor>, not
fs.write.file, and an in-place rename emits fs.rename; neither had a
listener, so a recipient watching a shared folder never learned that a
file appeared in it or was renamed. Part of why: those keys and
outer.gui.item.renamed were missing from the typed event map, so a
listener for them did not compile.

Delivering the event is only half of it. Paths were masked against the
entry itself, so item.added named a parent no cached listing was keyed
on, and the payload carried no dirpath, which is how the desktop finds
the container to render into — the event would have arrived and changed
nothing. Paths are now masked at the share the holder reached the entry
through, which is the address their own reads returned, and from_path on
a move and old_path on a rename travel the same way (dropped when the
move started outside the share, self-masked when the share is on the
entry itself, where the root already carries the new path).

Creates fire per entry, so an upload would have cost one share lookup
per file; they are coalesced by parent folder the way subtree deletes
already are. Measured on a 25-file burst into one folder: 25 lookups
before, 1 after. A holder with a share on both a folder and something
inside it is told once, by the nearer of the two.
2026-08-25 19:38:38 -04:00
Juan Castro 2ef5d00d9b Type is_shared and say who the flag counts
stat() and readdir() return FSItemRead, so the is_shared the docs lead
with typechecks for TypeScript consumers rather than erroring on FSItem.

The docs said "you have shared", but the query has no issuer predicate:
a manage delegate's re-share sets the owner's flag too, which is the
useful answer and matches getShares().
2026-08-25 18:15:15 -04:00
Juan Castro bac3dde945 Forget cached entries when sharing changes
Nothing invalidated the SDK entry cache on share or unshare — the socket
handlers only fire on item mutations — so is_shared, which now rides in
the cached entry, stayed stale for every consistency: 'eventual' read.
That includes the GUI's own listing refresh, which defaults to it, so a
badge would not have appeared until an unrelated write flushed the cache.
2026-08-25 18:15:05 -04:00
Juan Castro 1600dc38a5 Retire the share index row with the grant it records
revoke-user-user withdraws a grant without touching the share index, so
the row outlived the access — invisible until now, because listSharesOf
filters against live grants, but the new flag reads the index and would
report a file as shared to nobody, permanently.

Drop the row where the grant goes. The alternative, filtering liveness on
the read side, is the per-entry work the flag exists to avoid.
2026-08-25 18:14:52 -04:00
Juan Castro 0c8dd8b6e4 Document the share flag and returnShares 2026-08-25 17:45:15 -04:00
Juan Castro 3217533594 Give the puter.js API suites rate-limit headroom
Around 500 tests run through one seeded free-tier account, so the
per-tier windows were throttling the suite rather than anything it
tests — adding a single readdir was enough to trip fs:readdir-burst.
Resolve the seeded users to the unlimited policy, alongside the
subscription gate the harness already turns off for the same reason.
Test config only; no published limit changes.
2026-08-25 17:44:56 -04:00
Juan Castro 0591f13cc2 Expose is_shared and returnShares in puter.js
Carry the flag into the v1 entry shape and add returnShares to stat(),
whose shares are mapped into the same Share objects getShares() returns.
returnShares joins the dedupe key and bypasses the entry cache in both
directions — a result carrying share data is never written to it, so a
later plain stat cannot serve other people's names from cache.
2026-08-25 17:44:50 -04:00
Juan Castro ee2f14576b Report sharing in stat and readdir
isShared on /fs/stat and /fs/readdir, is_shared on their legacy
counterparts, and return_shares now fills the shares array the legacy
stat has been stubbing with []. Null for entries the caller does not own,
so a share recipient is never told who else can reach the owner's files.
The share-to-wire mapper moves out of ShareController so both controllers
publish one shape.
2026-08-25 17:44:41 -04:00
Juan Castro 5115fdd48c Answer which entries are shared in one indexed query
Directory listings need a shared/not-shared flag per entry. Ask the share
index once per batch, chunked, rather than a query per entry, and answer
only for entries the caller owns so listing a foreign directory never
queries at all. Direct shares only — inherited access stays with
listSharesOf, which already reports it.
2026-08-25 17:44:25 -04:00
jelvehandClaude Fable 5 e006b19773 puter.peer: room names, relayed guest grants, servers that come back
A peer server could only be reached by the invite code the signaller
minted for it, which is unknowable ahead of time and dies with the
server's socket. Any app that wanted a link to share in advance or reuse
had to run a registry of its own that maps a stable code to whatever
invite is live — and keep it live, since the signaller socket can drop
without the library saying a word (`onclose` was an empty function),
leaving a host serving an invite nobody can dial.

`serve({ name })` serves under a room name of the caller's choosing;
`connect(name)` dials it. A name is held by whoever serves it and free
once they stop; serving a held name from another identity rejects with
`name_in_use`, from the same identity it takes over. A connection to a
room nobody serves fails with `no_host`, so a lobby has a definite answer
to poll on instead of guessing between "not yet" and "gone".

`serve({ guestGrant })` and `server.setGuestGrant()` leave a guest grant
with the signaller, which hands it to every anonymous guest in the connect
reply; the connection redeems it before making its offer, so a guest with
no session gets relays without the app publishing the grant anywhere.

A server whose signaller socket drops now re-registers on its own with
backoff — under the same name, or a fresh code announced by the new
`reconnect` event — and pings the socket to keep idle proxies from cutting
it. It fires `close` (`replaced`, `name_in_use`) when it has to stand
down for good. Existing connections are peer-to-peer and never affected.

Signaller refusals reach the connection's `error` event as an Error with
a `code` next to the message. Keepalive replies and stray frames no longer
throw in the message handlers.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-25 13:24:45 -07:00
Juan Castro 07e73efa40 Merge remote-tracking branch 'origin/main' into juancastro/put-1568-rename-srcguisrchelpers-files-to-camelcase 2026-08-25 15:10:29 -04:00
404oops 2a45a10854 Merge pull request #3630 from HeyPuter/FK/dedupe
refactor: dedupe model catalog aliases and stop mutating shared catalogs
2026-08-25 18:07:27 +02:00
Nariman Jelveh e052aec7fb Carry a share link through the OIDC round trip (#3642)
A share email lands on `/?shared=…`, and its recipient is signed out or
in a temporary session more often than not. Password login reloads the
same URL, so the link came back with it — but an OIDC login leaves the
origin and returns to a URL the backend builds from a whitelist of bare
paths. `return_to` was a pathname only, and `/` wasn't on the list, so
the recipient came back signed in on the Dashboard's Apps tab with
nothing to say what had just been shared with them.

`shared` now travels with the path. The GUI sends it along with the
return path; the backend accepts it as the one parameter a `return_to`
may carry, and only values shaped like the masked share path the mail was
built from — everything else rejects the whole value, the path is still
matched as a raw string so a protocol-relative one can't smuggle an
origin through, and the query is rebuilt rather than reflected. The items
ride onto the error page too, so a failed attempt retries on the link
instead of losing it.

The root joins the whitelist for this, but only when it names something:
on its own it is where the flow already lands.
2026-08-25 08:42:47 -07:00
Daniel Salazar cf4d743854 fix: testing for peer (#3643) 2026-08-25 08:31:45 -07:00
Daniel Salazar 3c866c645b feat: guest turn for peer (#3640) 2026-08-25 07:51:02 -07:00
Juan Castro 4a5d689af7 refactor: rename GUI helper files to camelCase
src/gui/src/helpers was mostly snake_case, disagreeing with itself and
with the camelCase file convention in AGENTS.md. Rename the 45 offenders
with git mv and repoint every import specifier at the new paths.

Files only — exported symbols are unchanged. applyItemAddedToContainers.js
still exports apply_item_added_to_containers and no call site is edited
beyond its import line; those identifiers cross module boundaries and some
hang off window, so renaming them is a separate, riskier change.
2026-08-25 10:49:59 -04:00
jelveh 6f1548e26a Move Share above Download in file context menus 2026-08-25 07:28:53 -07:00
Daniel Salazar 764ce2f8e2 feat: rep score middleware gateway (#3639) 2026-08-25 07:11:41 -07:00
Daniel Salazar 717c098503 fix: metering buffer improvements for manual editing (#3633) 2026-08-24 18:36:43 -07:00
Nariman Jelveh 2c9a5c4f7e Land share-email links in the Dashboard's Shared view (#3634)
* Land share-email links in the Dashboard's Shared view

A share email's links opened the item on the desktop, and "Open Puter"
went to the bare origin — the recipient arrived on Home with nothing to
say what had just been shared. Every link now lands in the Dashboard's
Files tab, on Shared, with the shared items selected.

Each named file still links to itself. "Open Puter" carries every item
in the mail (?shared=a&shared=b), so the whole batch lands highlighted;
the digest records now keep each item's masked path for that. The
button's href is rendered raw like the item links, so `=` and `&` read
the same in the html and text parts, and the unsubscribe link is built
from a separate origin value.

On the GUI side `/?shared=` boots the dashboard rather than the desktop;
`/desktop?shared=` keeps opening the item in place. The Files tab
selects the rows by uid, which survives a rename. A share only reaches a
real account, so a share link never mints a temp user and an existing
temp session is asked to sign in first, as the desktop already did. The
address-bar cleanup both shells need moves into a helper.

* Keep the share email's button link within what mail clients tolerate

The "Open Puter" link names every item in the mail, capped at twenty. But
a single item's parameter is ~150 characters once the owner, uuid and an
encoded name are in it, so twenty of them run to several kilobytes — past
the ~2000 characters where older mail clients cut a URL off or stop making
it clickable — and the count alone couldn't do what its comment promised
for the mail's primary button.

Add a length budget beside the count: items go in, in digest order, only
while the whole link stays under it, so the button always works and the
first items are the ones highlighted. A single item always fits.

* Keep a share link's own item however long its name runs

`shareDeepLink` now builds through `sharedViewLink`, whose length budget
applied to the first item too: a parameter that alone overran it was
dropped, and the link came out as a bare `?shared=`. A name of a few
hundred characters — the GUI allows five hundred, and encoding triples
every non-ASCII one — was enough, so that item's own link in the mail
landed on Shared with nothing picked out, where it used to open the item.

The first item now always goes in; the budget only decides how many more
join it. One long link is still the item the mail is about, and it is no
worse than a bare origin for the clients that truncate it.
2026-08-24 17:01:55 -07:00
jelveh 34af8d18d0 Share a whole selection from the Dashboard's Files tab
Sharing was one item at a time: the only way to give someone four files
was to open the dialog four times. A multi-selection now offers Share in
its context menu and in the mobile selection bar, and UIShareModal takes
a list of items.

With several items the access list folds into one row per person rather
than repeating per file, so a grant, a mode change or a revoke is one
decision about a person. A row says what it can't otherwise show: how
much of the selection the person reaches ("On 2 of 4 items", with an
"Add to all" that extends them), a mode select that rests on a
placeholder when their grants disagree rather than presenting one item's
mode as the batch's, and inherited grants left uncontrollable where they
belong. Changing a mode touches only the items the person already holds
— nothing here widens access without saying so.

Requests are chunked to the documented 50-items-per-request cap, and a
selection past it skips the per-item listing instead of firing one
request per file on every refresh.

The eligibility rule the Share entry already used moves into a shared
can_share() helper, since the multi-select menu needs the same answer
for every row.
2026-08-24 14:08:37 -07:00
Juan Fernando Castro 0527bcd98e Merge pull request #3632 from HeyPuter/juancastro/put-1560-file-sharing-deeplink-in-emailnotifications
🔧 PUT-1560: Deep-link the shared item from email and notifications
2026-08-24 12:30:07 -04:00
Juan Castro ea530e604c Merge branch 'main' into juancastro/put-1560-file-sharing-deeplink-in-emailnotifications 2026-08-24 12:21:21 -04:00
Juan Fernando Castro 97d137b94b Merge pull request #3623 from HeyPuter/juancastro/put-1564-perms-docs-drop-the-app-root-dir-raw-permission-string-wire
🛠️ PUT-1564: Perms docs — drop the app-root-dir raw string, wire the playground examples
2026-08-24 10:59:56 -04:00
jelveh 1ceffbe291 Add sudoku to recommended apps
This change adds Sudoku to the curated recommended apps list so it appears alongside the other bundled games in the app recommendations flow.
2026-08-23 21:59:52 -07:00
Daniel Salazar 684d6752f9 fix: provide fallback for signup verification (#3631) 2026-08-23 21:49:35 -07:00
jelveh 76cea02513 Treat .md like .txt; remove markus suggestion
Merge the .md extension case into the plain-text branch so it suggests ['editor', 'code'] instead of ['markus', 'editor', 'code']. Update the related test to use 'viewer'/'png' as the built-in guard example since 'markus'/'md' no longer applies.
2026-08-23 21:17:28 -07:00
404oopsandClaude Fable 5 308b78e96c refactor: dedupe model catalog aliases and stop mutating shared catalogs
- Add modelLookupNames() and route every provider's list() through it so
  repeated or self-referential aliases can't double-list a model.
- Stop the chat/image/video drivers' #buildModelMap from writing through
  to the module-level catalog constants (ids/aliases accumulated across
  instantiations); bucket copies preserve the models() wire shape.
- Remove self-aliases and exact duplicates from the claude, deepseek,
  groq, mistral, and openai catalogs; fold the duplicated
  gemini-3.7-flash entry — this corrects the live cached-token price
  from 8 to 7.5 and the knowledge date to 2026-03 (the previously
  effective first entry carried stale values).
- ClaudeProvider: derive default max_tokens from the resolved entry
  instead of a second name-based lookup that capped dated ids at 4096.
- Add cross-provider catalog invariants (modelCatalogs.test.ts) with a
  disk-scan registration guard, plus regression tests for catalog
  non-mutation, alias-dedup routing, wire shape, and max_tokens ceilings.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-08-24 03:42:47 +02:00
nevrkla 2d787c3fdf feat(i18n): add Czech GUI localization (#3617)
* feat(i18n): add Czech GUI localization

* feat(i18n): add Czech GUI localization

* fix(i18n): polish Czech GUI translation
2026-08-23 17:22:44 -07:00
Daniel Salazar 5918e3f4e2 fix: kv max number value (#3629) 2026-08-23 06:31:08 -07:00
Nariman Jelveh 2c9e818e5f Docs/remove discord reddit links (#3628)
* docs: remove Discord and Reddit links

Drops both links from the docs README (header row and Support list) and
from the footer generated on every built docs page.

* docs: remove Mastodon link
2026-08-22 22:14:56 -07:00
Nariman Jelveh 8826a9fc51 Remove Discord and Reddit links from README (#3627)
Removed links to Discord and Reddit from the README.
2026-08-22 13:07:33 -07:00
Daniel Salazar 8307e8c9f0 chore: bump up deps (#3626) 2026-08-21 21:03:54 -07:00
Daniel Salazar 732786a029 fix: clean up email validation (#3622) 2026-08-21 20:27:05 -07:00
Juan Castro 202c43f46e feat(share): deep-link the shared item from email and notifications
A share email named the file but had nowhere to go: the only link was "Open
Puter", and finding what someone shared meant hunting for it under Shared. Each
named file in the digest now links to itself, and a notification covering a
single item points at that item.

The link carries one parameter, the masked path a recipient is already given:

    https://puter.com/?shared=%2Falice%2F<uuid>%2Freport.txt

Its second segment is the uuid, so there is no second copy of it to disagree
with the path, and the GUI can still find the entry when a rename has left the
name segment stale - it stats the path, then falls back to the uuid.

Built from the owner, uuid and name rather than from `ResolvedShare.path`.
That path is masked for whoever made the request, and the issuer owns the
entry, so it comes back as the owner's *real* path - mailing it would tell the
recipient which folders the owner keeps things in, which is the one thing
masking exists to prevent. A test asserts the real path never reaches the mail.

`digestLines` now returns `lead`/`items`/`trail` beside `what`, so the template
can put an anchor around each name while Handlebars keeps escaping the names
themselves; the URL is machine-built from the configured origin and one encoded
path, so it stays literal. Concatenating the parts reproduces `what` exactly,
which a test pins - the linked and sentence forms must not describe different
shares.

Notifications carry the masked path rather than a URL: the recipient is already
in the GUI, which opens the item in place instead of reloading. Only a
single-item notification gets a target; folding into a group drops it rather
than picking one of five.

In the GUI, `?shared=` joins `?download=` and `?app=` as a param that keeps the
desktop booting at `/`, and the handler reuses the `/@user` public-folder flow -
extracted to `open_path_target`, which carried a TODO asking for exactly this -
so a file opens in its associated app and a folder in an explorer window. The
param is stripped from the address bar first, so a reload lands on the desktop
rather than opening the item twice.

Invites are deliberately not linked: there is no account to route to yet, and
the invite's own call to action is to create one.

Rolling-deploy safe: a digest entry queued before this has `names` and no
`items`, and still flushes - without links. New entries write both, so a node
on the previous build can flush them too.
2026-08-21 16:08:39 -04:00
Juan Castro e7be457a56 docs: redirect the retired perms pages, add the missing playground examples
Both were left as follow-ups on the previous commit.

Thirteen `puter.perms` method pages went when the surface collapsed onto
`request(resource, details)`, and `requestAppData` moved to `/Perms/appData`.
All fourteen had shipped, so an external link or a bookmark landed on nothing.
The build already generates meta-refresh redirects from a map; they are entries
in it now. Pages that only ever existed on the branch that removed them are
deliberately absent — nobody can hold a link to a URL that was never published.

Six documented APIs annotated a code block with a playground id but had no
example file behind it, so the build warned on every one and the Try-it link
resolved to nothing: `fs.share`, `fs.unshare`, `fs.listShared`, `fs.getShares`,
`ai.txt2speech` with Speechify, and `ui.showFeedbackDialog`. The files are the
documented blocks themselves, extracted rather than rewritten, so the example
and the page it appears on cannot drift. Each is indexed in `examples.js` beside
its siblings.

The build now runs clean: 12 warnings to none.
2026-08-21 11:26:32 -04:00
Juan Castro 9bf49b7574 docs(perms): drop the app-root-dir raw string, wire the perms playground
`app-root-dir:` was listed among the raw permission strings, but it reads as
nothing during a permission check - the rewriter yields the sentinel outside a
grant, so `check()` reports it ungranted even once it is and `request()` prompts
every call. Both contradict what the same page promises. The `'appRootDir'`
resource is the supported path and asks the server instead; the list now says so
rather than implying the string behaves like its neighbours.

The playground listed no perms examples at all: six files existed but nothing
referenced them in examples.js, so a freshly-reshaped API shipped with nothing
runnable. Adds a Perms group after UI, matching the sidebar's order, and two
examples for the features that had none - one prompt covering several resources,
and check() deciding whether to prompt.
2026-08-21 10:20:05 -04:00
f0cd251626 🛠️ PUT-1521: Cleanup puter js permissions api + backend routes (#3607)
* refactor(puter-js): collapse puter.perms request* to resource + access

Fifteen request methods differed only by a folder name or an access level, so
every new resource meant another method. Replace them with requestFolder,
requestApps, requestSubdomains and requestAppRootDir, each taking the access
level as an argument.

The old names stay as @deprecated aliases: puter.js ships unpinned from
js.puter.com/v2, so removing them would break live apps. They remain in the
generated declarations because stripInternal has no effect on declarations
emitted from JavaScript, and hand-omitting them would break TypeScript callers
the runtime still serves.

Also drops the user-to-user and user-to-group grant wrappers (groups.js and the
grantUser/grantGroup half of grants.js) plus the req_ shim, none of which were
documented or called. The app, origin and dev-app grants stay: the dashboard
uses puter.perms.revokeApp() to clear grants on app uninstall.

* docs(perms): document the collapsed puter.perms surface

Replace the twelve one-method-per-task pages with requestFolder, requestApps
and requestSubdomains, and rewrite the Perms overview around the seven public
methods. The deprecated aliases keep working but are no longer documented.

Boy Scout: drops the long-dead commented-out grantUser/revokeOrigin sidebar
block for pages that were never published.

* refactor(perms): drop unused user-to-user and user-to-group permission routes

Filesystem access is shared through /share, which records the grant so the
owner can see and revoke it. The older direct-grant paths were left behind with
no caller anywhere - not the GUI, not a doc, not an app: grant-user-user
(already a 501 stub), revoke-user-user, grant/revoke-user-group, and the five
/group/* CRUD routes.

Removing them orphans PermissionService.grant/revokeUserGroupPermission and its
group-members cache bump, the three PermissionStore group writers, and six
GroupStore methods, so those go too.

What stays, and why:
- grant/revokeUserUserPermission - ACLService and ShareService power fs.share
  through them.
- The group permission read path (#scanUserGroup, readUserGroupPerms) - a
  migration seeds the admin group unrestricted driver access, so it is
  load-bearing.
- GroupStore getByUid/addUsers/removeUsers - signup, save_account, OIDC and the
  self-hosted default user assign group membership.

No schema change: user_to_user_permissions, user_to_group_permissions and their
audit tables are untouched. Group rows now come only from migrations, so tests
that need one seed it with SQL the way a migration does.

* refactor(perms): reduce GroupStore to membership writes

With the /group/* routes gone, `getByUid` had no production caller left — the
routes were the only thing that read a group back. Removing it takes the row
decoder and the GroupRow type with it, since they exist only to shape its
result.

What remains is `addUsers`/`removeUsers`: signup, save_account, OIDC and the
self-hosted admin bootstrap all assign group membership. Permissions attached to
a group are read through PermissionStore, which joins the junction table itself
and never needed the store.

Tests that wanted a group id now select it, which is all `getByUid` was doing
for them.

* chore(perms): drop the three by-hand groups no code reads

freeai, experimental and dangerous exist in prod but in no migration — they
were added by hand when hardcoded permissions were keyed by group name. That
map is now a flat per-user floor (`default_user_permissions`), so a group
nothing looks up grants nothing.

Guarded rather than unconditional, because both tables the delete can reach
cascade: dropping a group that still carries permissions or members would
silently revoke them from every member. Only a group with neither goes. One that
survives has dependents and needs a deliberate decision — query
user_to_group_permissions by group_id to see what it holds.

system, admin, user and temp are untouched: config names two of them and code
names the others.

Matches on `extra.name`, not `metadata.name` — `metadata` carries the display
title and colour, and `critical: true` is set on all of these including freeai,
so it does not discriminate.

* feat(puter-js): collapse puter.perms onto request(resource, details) + check()

One method per task meant a new method, doc page and sidebar entry for every
resource. `request` now takes the resource and a payload whose accepted fields
depend on it, and `check` answers the same question without prompting.

    request('folder', { name: 'Documents', access: 'write' })  -> path
    request('apps', { access: 'read' })                        -> boolean
    request('email')                                           -> address
    check('folder', { name: 'Documents', access: 'write' })     -> boolean

Returns stay per-resource: a folder gives its path, email the address, the rest
a boolean, and anything denied is falsy so one `if` covers both.

An array asks for several at once. Everything already held is settled first, so
the prompt covers only what is missing and does not appear when the whole set is
held - the user answers once for the lot. `check` answers per entry, in order,
so a caller can tell which parts are missing rather than only that some are.

Each resource declares four things in one registry entry: how to ask for it
alone, whether it is held, the strings a batch pools into a prompt, and the
value once held. The strings themselves are defined once in
lib/permissionStrings.js, so a request and its check cannot name them
differently. `check` is built on /auth/check-permissions, already live and
already used by UI.js, and it throws rather than answering false when the check
cannot run: a caller that cannot tell "denied" from "never ran" would prompt
someone who had already granted it.

Backward compatibility: all 22 older methods stay callable and typed, marked
@deprecated with the call that replaces them. A lone string still routes to the
raw-permission path - no resource name contains a `:` and every permission
string does, so the two forms cannot collide. The grant/revoke app methods are
untouched; the consent dialog and the dashboard's uninstall path use them.

Also drops three copies of the access-level assertion onto one shared
validator, and gives `appRootDir` a non-prompting server probe, since
`app-root-dir:` only resolves while a grant is being written and a permission
check on it always answers false.

* docs(perms): document request() and check() as the perms surface

Five per-method pages became one `request()` page carrying the resource table,
the batch form and the raw-string escape hatch, plus a `check()` page. The
overview is rewritten around the two methods.

requestAppData's page is re-homed as /Perms/appData rather than deleted - its
scope table, private-entry guidance and lifetime notes are not signature
documentation and have nowhere else to live. Inbound links from KV/set.md and
Objects/app.md follow it.

Playground examples move to the new call form. They are not wired into
examples.js, but an example demonstrating a deprecated method is worse than one
nobody loads.

* fix(perms): keep /auth/revoke-user-user as a deprecated route

Dropping this route with the rest of the unused user-to-user plumbing went too
far. The grant side is retired and stays retired - puter.fs.share() is the only
way in - but access those grants left behind has to remain withdrawable, and a
caller reaching the endpoint over HTTP directly had no replacement. Revoking can
only ever narrow what someone can reach, so keeping it carries no risk.

revokeUserUserPermission never left the permission service; it is load-bearing
for puter.fs.share(). This only re-wires the handler to it, with the gates it
always had.

Nothing in this repo calls the route, which makes it exactly what a later
cleanup reads as dead, so a test pins the registration and its gate alongside
the restored 400 and grant/revoke round-trip cases.

The 501 stub at grant-user-user and the never-called /group/* routes stay
deleted, as does puter.perms.revokeUser - puter.fs.unshare() replaces it and
falls back to live grants when no share row exists.

* fix(perms): let a write grant satisfy a read check on apps and subdomains

`apps-of-user:<uuid>:write` covers managing the user's apps, which includes
reading them, but nothing said so to the permission system. Prefix implication
only widens the other way — an `apps-of-user:<uuid>` grant covers both modes —
so a scan for `:read` missed a `:write` grant, and `puter.perms.check('apps')`
reported an app holding write as holding nothing. A batched request would then
prompt again for access already granted.

Adds the read-from-write exploder for both namespaces, mirroring
`fs-access-levels`. The widening runs one way only, and does not cross into
another user's namespace; both are covered by tests.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): answer the app-root-dir check without provisioning it

`/auth/request-app-root-dir` conflates two questions: may the caller claim its
root directory, and where is it. The second provisions `AppData/<uid>` on first
ask, so a caller that only wanted the first — `puter.perms.check('appRootDir')`
— created a directory by asking about it.

Adds `check: true`, which runs the same actor guard and stops at the answer.
A caller that may not claim it still gets the 403, so the flag can't widen
anything. Existing callers are unaffected: without it the route behaves exactly
as before.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): put request() and check() on one path

`request` dispatched to the old per-task methods while `check` asked the
permission tables, so the two answered different questions about the same
access. Concretely, before this: `request('folder', { access: 'write' })`,
`'apps'`, `'subdomains'`, `'appData'` and `'permission'` prompted every time,
whether or not the access was held — which the docs said they wouldn't;
`check('folder')` reported false for a folder the app could read through an ACL
grant that no `fs:` string names, so a batch prompted for it needlessly; a
batch entry for `'appRootDir'` skipped the post-grant retry the single call
does, resolving `undefined` after a grant that had in fact succeeded; and an
N-entry batch made N permission reads plus 2N `whoami` calls.

Both now run the same pipeline — resolve the permission strings, read what is
held once, prompt for the remainder, resolve each entry — with per-resource
hooks for the parts only that resource can answer. So a batch costs one
permission read and one `whoami`, a check reports exactly what a request would
skip the prompt for, and `'folder'` uses the same stat-or-permission reading in
both.

Also:

- A resource is looked up as an own property, so `request('constructor')` is
  the permission string it always was rather than a TypeError.
- A permission read that fails no longer decides anything: `request` falls
  through to the prompt it would have raised anyway, `check` throws. Before,
  `check('appRootDir')` folded a failed check into "not granted", which is what
  the documentation says must not happen.
- Drops `requestFolder`, `requestApps`, `requestSubdomains` and
  `requestAppRootDir`. They were added in this branch and immediately deprecated
  — never shipped, and `request()` no longer needs to route through them. The 22
  methods that did ship keep their exact behaviour, prompting without consulting
  what is held, which the suite now asserts alongside the new behaviour.
- Documents `'appRootDir'`, which was a supported resource in every overload and
  in `PermsResource` but named in none of the docs.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): act on review of the three preceding commits

- `src/puter-js/test/perms.test.js` still called `requestApps`,
  `requestFolder`, `requestSubdomains` and `requestAppRootDir`, which the
  previous commit removed. Four cases in the interactive browser harness threw.
  Pointed at `request(...)` instead.

- `request('appRootDir', …)` made two round trips where the shipped method
  makes one: a read-only probe, then the call that names the directory. A
  request is going to claim it either way, so the claim is now the check, and
  the entry it returns carries through to the result. `check` keeps the
  read-only mode, which is the reason that mode exists. Matters because the
  route sits on the FS_SIGN bucket, shared with signed-URL minting.

- `requestPermission` is one of the shipped methods, and the previous commit's
  message was wrong to say all 22 keep their exact behaviour: it forwards to
  `request`, so it now settles a permission the caller already holds instead of
  prompting for it. The value can differ, not just the prompt count — a user who
  would have clicked Deny on a re-prompt used to get `false`. It is the more
  honest answer (the app does hold the access, and denying a re-prompt never
  took it away), but it is a change, and the suite assertion had been switched
  to an unheld permission, which hid it. Asserted both ways instead, in the unit
  tests and the API suite.

- An entry that names no permission no longer rides a grant given for the other
  entries in the same call. Unreachable today — every resource either names one
  or reports itself held — but nothing pinned it.

- Reverted three type-union reformats in `LegacyFSController.ts` that a
  formatter had folded into the app-root-dir commit. That file was not
  prettier-clean to begin with; reformatting it is somebody else's change.

- Docs and types: `Perms.md`'s `appRootDir` row now matches `request.md`'s,
  `check.md` says that a `true` is per entry and a batch still prompts if any
  one entry is missing, and `types.js` no longer names `requestFolder` /
  `requestAppData` in prose that ships in the generated declarations.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-20 23:48:08 -07:00
ZHOU7821 943adae622 docs: use absolute link to router page in workers create docs (#3615) 2026-08-21 13:30:30 +07:00
Daniel Salazar 65572dcf23 fix: nicer sms card fallback (#3621) 2026-08-20 23:12:09 -07:00
ZHOU7821andZHOU7821 b3923f86da docs: fix typo in workers description (#3614)
Co-authored-by: ZHOU7821 <[email protected]>
2026-08-21 12:26:20 +07:00
Nariman Jelveh 729114bd09 Prioritize app-center in recommendations
Reordered the recommended app list so app-center appears earlier and dev-center is kept at the end. This keeps the default app suggestions aligned with the expected homepage/app-center placement without changing behavior beyond ordering.
2026-08-20 19:16:03 -07:00
Nariman Jelveh da1c85d3c5 Add Meetings to recommended app defaults
Include `meetings` in `RECOMMENDED_APP_NAMES` so it appears in the default recommended apps set alongside other core productivity tools.
2026-08-20 17:16:39 -07:00
Nariman Jelveh 66410d7f1f Revert "Let a browser send the device fingerprint header it is offered"
This reverts commit fef1337b2b.
2026-08-20 17:10:55 -07:00
Nariman Jelveh 76bd05f228 Dashboard: let the trial be a detail line, not an alarm
The Usage card gives each of its three columns a meter and two lines. The
plan column had a boxed, tinted, bordered paragraph instead — and at the
~230px a third of that card comes to, thirty words of it wrapped into a
five-line ribbon standing twice as tall as the storage and resources meters
beside it. The box is the shape we use for the dunning banner. A trial is a
date you should know about, not a fault, and it was reading as the loudest
thing on the card.

So it takes the slot its neighbours use for "n% of 100 GB": a quiet hint-
coloured line under the badge. All three columns now measure the same.

The copy stopped repeating itself while it was there. The badge said "ends
Sep 20" and the sentence beneath it said "ends on September 20, 2026" — two
elements, one fact, and neither of them answering the question someone
actually has, which is how much room is left to decide. The badge now
carries "31 days left" and the line under it carries the consequence:
"First charge Sep 20 unless you cancel." Billing keeps the full sentence and
the opt-out, and Manage is directly below.
2026-08-20 16:59:07 -07:00
Nariman Jelveh fef1337b2b Let a browser send the device fingerprint header it is offered
`fingerprint.ts` reads the device fingerprint from a request body or, for
authenticated requests with no body to carry it, from
`x-puter-device-fingerprint`. That header is documented there as "the
header the GUI may send the device fingerprint on for non-signup
requests", and the middleware has always honoured it.

A browser could never actually send it. The CORS allowlist in
`#installCors` does not include it, so the preflight comes back without
it in `Access-Control-Allow-Headers`, the browser abandons the request,
and `fetch` rejects before anything reaches the server. Nothing in tree
sends the header today, which is why this went unnoticed: the first
client to try it sees every call fail with a network error rather than a
readable status.

Adds the header to the allowlist, and a preflight test asserting the
browser is allowed to ask for it.
2026-08-20 12:40:48 -07:00
404oops 58794f4f76 feat: add MetaProvider for Muse Spark models and integrate with ChatCompletionDriver (#3616)
- Implemented MetaProvider to interface with Meta's Muse Spark models.
- Integrated MetaProvider into ChatCompletionDriver, allowing for dynamic model selection.
- Added unit and integration tests for MetaProvider to ensure functionality and reliability.
- Updated documentation to include Meta as a supported vendor for AI models.
2026-08-20 15:09:56 -04:00
Nariman Jelveh 9809e428b5 Update shared label in email template
Adjust the share-notification email copy to reference the current "Shared" files section instead of "Shared with me," in both HTML and plain-text variants. This keeps onboarding messaging aligned with the UI.
2026-08-19 20:05:56 -07:00
Nariman Jelveh eaddd1d44e redesign: share notification emails (#3612)
* redesign: share notification emails

The share digests were an unstyled fragment: a 520px div with a couple
of paragraphs, no preview line, no small-screen or dark-mode handling,
and an html-only body that left every text-only client reading a machine
down-conversion of the markup.

Both digests (`file_shared_with_you` and `file_shared_invite`) now render
through one layout: a 600px table column that goes full width under
600px, every color written inline so a client that drops `<style>` still
gets the design, with the stylesheet carrying only what inline CSS can't
express — the small-screen and dark-mode overrides. The shares are a
tinted panel of hairline-separated rows rather than loose text, the call
to action is a cell-padded button Outlook can draw, and the reason for
receiving plus the unsubscribe sit below the card where they belong. No
images at all, so nothing depends on an asset a client may refuse.

Templates may now carry a `text` part, and both digests do: it goes out
beside the HTML as multipart/alternative, which is what plain-text
clients, screen readers and spam scoring prefer.

Copy: a preview line that adds to the subject instead of repeating it, a
heading before the list, and an honest unsubscribe label — the link
silences notification mail account-wide, not just shares. The invite now
says what to do with the address and that an existing account can claim
the shares by confirming it.

Also drops `share_by_username` and `share_by_email`, unreferenced since
the digests replaced them.

* fix: widen the template map to the interface before compiling

`Object.entries` over the literal map yields a union with a distinct
type per template, and only the share digests carry a `text` part —
reading it off the union is an error the build tsconfig hides
(`noCheck: true`) and `npm run typecheck` catches.
2026-08-19 18:49:11 -07:00
Reynaldi Chernando 53b792c52c connect to worker's kv with cli (#3613) 2026-08-20 08:36:46 +07:00
Daniel Salazar 2dd7073034 fix: vscode webdav (#3610) 2026-08-19 14:04:56 -07:00
Daniel Salazar a552ce0b87 fix: better metrics for cache hit rates (#3609)
* fix: better metrics for cache hit rates

* chore: type + docs
2026-08-19 11:38:14 -07:00
Neal Shah c04827fd3f add dynamic worker create event (#3608) 2026-08-19 14:20:15 -04:00
Daniel Salazar 9debdffc5d fix: email send frequency + ai ordering (#3604) 2026-08-18 18:13:55 -07:00
Nariman JelvehandJuan Castro 57b4cbab66 feat(dashboard): rewrite Files-tab sharing as a from-scratch modal (#3598)
* feat(dashboard): rewrite Files-tab sharing as a from-scratch modal

The Files tab's Share… action opened UIWindowShare, a desktop UIWindow,
which doesn't fit the dashboard. Replace it with UIShareModal, built on
the same overlay pattern as UIItemPropertiesModal: a centered card on
desktop, a bottom sheet on mobile, styled with the dashboard's design
tokens (dark mode included).

Wiring follows the onShowProperties precedent: generate_file_context_menu
gains an optional onShare hook that TabFiles supplies; every other caller
falls back to the desktop share window unchanged.

Feature parity with UIWindowShare — grant by email/username with an
access level, list who has access (owner, direct, and inherited-from-
ancestor grants), change a grant's mode, revoke — plus:

- revoke confirmation inlined into the row instead of a stacked UIAlert
- real form semantics: Enter submits, button disabled until input,
  spinner while in flight, API errors keep the typed name for correction
- dialog a11y: focus moves in on open and back on close, Tab is trapped,
  Escape closes (first Escape cancels an open revoke confirmation),
  status messages announce via aria-live
- backdrop close keyed on where the press started, so a text-selection
  drag out of the input can't dismiss the dialog
- name-derived avatar colors and a loading state for the first fetch

Also fixes a double-encoding bug (shared by UIWindowShare) where mode
labels ran through html_encode twice, rendering "Can edit &amp; share".

* fix(dashboard): keep keyboard focus inside the share modal across re-renders

Changing a grant's mode, revoking, or Escape-canceling the inline
confirmation re-renders the access list (or disables the focused
control), which dropped focus onto <body> - outside the dialog's Tab
trap and invisible to screen readers. Focus is now restored explicitly
after each of those actions: to the same holder's control when it
survives, else to the dialog itself (tabindex=-1), and the Tab trap
steps back into the cycle when focus rests on the dialog container.
A failed grant likewise returns focus to the recipient input.

* fix(dashboard): give share modal controls 44px touch targets

On touch devices the close (32x32) and revoke (30x30) buttons and the
26px-tall per-row mode select were below the 44px minimum. On coarse
pointers the icon buttons now hit-test at 44x44 through a centered
pseudo-element (visuals unchanged, and the row spacing absorbs the
overhang without overlapping neighbors), and the row select gets
taller padding. Fine-pointer rendering is untouched.

* fix(dashboard): clear the iOS home indicator in the share bottom sheet

The app opts into viewport-fit=cover, so on notched phones the
bottom-docked sheet extends to the physical screen edge and its last
row sat under the home indicator. The sheet body's bottom padding now
adds env(safe-area-inset-bottom), matching how the codebase pads other
bottom-docked surfaces.

* fix(share): say 'Updated access' when changing a grant's mode

Changing someone's access level reported 'Shared with X', the same
message as a fresh grant, which reads as if a new share happened.
Both share dialogs now confirm mode changes with a dedicated
'Updated access for X' message. The UIWindowShare call also drops the
html_encode() around the recipient - i18n() already encodes
interpolations, so the wrapper double-encoded.

* fix(dashboard): align the share dialog's accessible name with the properties modal

The dialog's aria-label read 'name - Share' with an em dash while the
sibling properties modal uses plain 'name Properties'; screen readers
announce the dash as noise. Use the same name-then-noun pattern.

* fix(dashboard): ellipsize the share input's placeholder when it overflows

At phone widths the 'Add people by email or username' placeholder was
clipped mid-letter; text-overflow: ellipsis truncates it cleanly.

* fix(dashboard): raise the share selects' chevron contrast in light mode

The hardcoded slate-400 stroke measured ~2.6:1 against light surfaces,
below the 3:1 minimum for non-text indicators. Light mode now uses
slate-500 (~4.7:1); dark mode keeps slate-400, which already clears
5:1 there.

* fix(dashboard): stop Files-tab shortcuts from firing behind the share modal

The document-level keydown.tabfiles handler kept running while the
share (or item-properties) modal was open. With focus on any of the
modal's buttons or selects, Enter and Space were preventDefault-ed
before they could activate the control, arrows could not drive the
mode selects, and letter typeahead was hijacked into row typesearch —
while Enter opened the selected row behind the overlay, Delete moved
it to Trash, and Cmd+A/C/X/V acted on the hidden list. Yield the
keyboard to the modal for as long as one is up; its own handlers
already cover Escape and Tab.

* fix(dashboard): keep the on-screen keyboard down when the share sheet opens

Autofocusing the recipient input popped the keyboard over the bottom
sheet the moment it opened, hiding the access list before the user had
chosen what to do — the same reason the revoke flow already focuses
the dialog instead of the input. On touch-primary devices give the
dialog container initial focus (which also anchors the Tab trap);
desktop keeps the input autofocus.

* fix(dashboard): give the share sheet's confirm and submit buttons 44px touch height

The inline revoke confirmation's Cancel/Remove pair rendered ~29px
tall on touch — small targets 6px apart where one of the two is
destructive — and the submit button was fixed at 38px. Grow both to
44px under pointer: coarse, matching the standard the modal's icon
buttons already meet. The touch block moves below the confirm-button
base rules it now overrides, since a media query adds no specificity
and source order decides.

* fix(dashboard): name the person in the share rows' accessible labels

Every grant row's mode select announced as bare 'Access level' and
every revoke button as 'Remove access', so a screen reader user
tabbing the list could not tell whose grant a control changes. Carry
the holder in the aria-label ('Access level for alice' / 'Remove
access for alice'); the visible UI and the revoke tooltip stay as
they were. i18n() encodes the interpolated string as a whole, quotes
included, so the labels stay attribute-safe for any holder name.

* fix(dashboard): keep the share status region in the accessibility tree

* fix(dashboard): give the share modal's add-row controls 44px touch height

* refactor(gui): extract the share dialogs' pure logic into tested modules

* feat(dashboard): adopt the shared mode helpers and handle pending invitations

---------

Co-authored-by: Juan Castro <[email protected]>
2026-08-18 17:34:40 -07:00
Juan Fernando Castro 5e1645d40e 🔧 PUT-1524: Sanity check file sharing api (#3603)
* fix: sharing answers "who can reach this" from the grants, not the index

Two ways a share listing could name access that was no longer there.

A recipient's listing paired the flat permission read with the list of
permissions it asked for by array index. That read drops misses and
dedupes its keys, so the two are not positional: one entry's live grant
vouched for another entry that had none, and listShared() kept
publishing a withdrawn item's name, size and signed thumbnail URL to
someone who could no longer open it. Read the permission off the value
instead.

getShares() had the same gap from the owner's side, with no liveness
check at all — a grant withdrawn through /auth/revoke-user-user or an
ACL mode change left the index row behind, and the owner was told
someone could reach a file they could not. Checked against the grants
now, one batched read per distinct holder. Pending invites are not
subject to it: they have no grant yet, which is the point of them.

Also covers the access-token actor, which reaches the same reach bound
as an app through a different arm of the ACL check. No behavior change
there — it was correct and untested.

* test: pin that a revoke crosses regions

A revoke reaches a peer region as a replicated SQL delete plus three
invalidation events, one per cache the region owns: the u2u row cache,
the flat view and the scan generation. The delete alone changes nothing
there, and each cache has a different consequence if its event is lost
— 20s for the generation, 5 minutes for the row cache, and forever for
the flat view, whose grant-path entries carry no expiry.

Nothing covered the whole path end to end. The store tests prove each
event is emitted and applied; this proves the result, which is that the
recipient stops being able to read. It asserts the read still succeeds
after the SQL delete alone, so the test also records why replication is
not sufficient on its own.

* test: pin that a leaked uuid buys no access

A masked share path hides which folder an item sits in; it was never the
thing deciding who may open it. Nothing checked that at the route level,
so the guarantee rested on unit tests of the resolver alone.

Reads one shared file through its masked path, then tries the sibling
four ways: the shared uuid with the sibling's name, the sibling's own
uuid, a `..` back out of the root, and the owner's real path.

Worth knowing about this one: it's mutation-checked. Removing the head !== root.name guard in sharePathMask.ts fails it with reachable: /testuser/55dd54c0…/share-http-1df8933e.txt. I verified that specifically because two tests I wrote earlier in this chunk passed with their guards broken — both were vacuous, and I deleted them rather than commit false assurance.

* docs: say what listShared's total actually counts

`total` counts the shares recorded for you; items are filtered after the
page is read, so a withdrawn grant leaves the count higher than anything
paging will yield. The page description already explained the short-page
behaviour, but the field read as an exact count and the example printed
it as one.

The test pins the gap it describes: two shares, one withdrawn outside
the index, one item listed and a total of two.

* fix: keep an undelivered broadcast event instead of dropping it

The outbound queue was cleared before the send, so a peer that timed out
took its events with it. Most were survivable — a lost cache invalidation
heals when the entry's TTL lapses. A revoke's flat-perm invalidation is
not: grant-path entries carry no expiry, so a peer went on serving a
withdrawn grant until something else wrote that key.

Failed sends now go back on the queue, which the existing flush timer
retries. Anything queued since wins over the retry, and the queue is
bounded at 10,000 with the oldest dropped first, so a peer that stays
down cannot grow it without limit. Each retry is signed at send time, so
it is not rejected against the replay window.

* fix: stop plus-addressing from deciding who a share reaches
2026-08-18 15:51:23 -07:00
b795b219a8 ✨ PUT-1497: Share file link sharing and notifications (#3595)
* refactor(share): move share notifications into their own service

* feat(share): invite an address with no account, and email it

* feat(share): surface pending invites in the SDK and share dialog

* fix(share): unreachable revoke confirmation, and double-encoded labels

* feat(share): budget share announcements, group them, and let people block senders

Sharing had one defence against noise: a 15-minute quiet window per (sender,
recipient) pair, which dropped the second share rather than folding it in.
Twenty senders each under their own window could still bury someone, and there
was no way to make one of them stop.

Announcements are now budgeted on two axes through the existing sliding-window
limiter: 1 per 15 minutes and 20 per day from one sender, and 10 per hour /
50 per day to one recipient from anyone. Over budget the share still succeeds
and the recipient's notification is still brought up to date — only the
interruption is dropped. Invite email to an address with no account is budgeted
the same way, keyed on a hash of the canonical address.

Notifications now fold across senders: a new share rewrites the notification
the recipient hasn't dismissed, so "alice and bob shared 5 items with you"
replaces a stack of five. The record is written even when suppressed, so the
count is right whenever they next look.

Blocking is a new `user_block` table with enforcement in ShareService: a blocked
sender's share is refused with `recipient_not_accepting_shares`, spends no
quota, and writes no row, and their unclaimed invite is dropped when the address
is confirmed. Existing access is untouched — that is what revoke is for.
Managed from a Blocked people card in the dashboard's Security tab.

Also publishes the sharing limits, including the ones already on this branch
that were never documented.

* fix(share): name the item in share email, instead of 'an item'

* fix(share): make the invite lifecycle canonical, authorized, and race-safe

* refactor(email): drop EmailClient.isConfigured; callers read config.email

* feat(share): batch share email into a per-recipient digest, durably

* docs(share): document the share error codes; steady the disk migration tests

* fix(share): log why a digest wasn't sent, and recover orphaned ones

* feat(share): email recipients about shares by default, with a way to decline

Share email was off unless a deployment opted in, which meant an account
holder was told about a share in the app only. It is now on unless
`share_email_notifications` is set to false.

The reason it defaulted off was that nobody could decline. So this also
honors `user.unsubscribed` — the account-wide opt-out the /unsubscribe page
already writes and app feedback already respects, which share email ignored —
and the digest carries that link. Sharing and the in-app notification are
unaffected by it; only the mail stops.

The link is composed in the template around an interpolated uuid rather than
passed pre-built: Handlebars escapes interpolated values, so a whole URL came
out as `user_uuid&#x3D;…`, which browsers decode but link scanners and older
mail clients need not.

* fix(share): count every shared file in the digest, not just the first

* feat(share): let a recipient refuse shares from everyone

Blocking answered "not from this person" but had no answer to "not from
anyone", so the only way to stop a stream of unwanted shares was to name
each sender after they had already reached you.

Stored as a key in the user row's existing `metadata` blob rather than a
column: the share path already holds the recipient's row by the time it
asks, so reading it costs nothing, and a one-bit preference doesn't earn
a migration per dialect. `updateMetadata` merges and refreshes the cached
row, so the switch bites on the very next share.

Refusing everyone reports the same code as refusing one person — which of
the two it is is the recipient's business, not the sender's. Enforced at
both moments the per-sender block is: when the share is issued, and when
a pending invite is claimed. The per-sender list is untouched while the
blanket switch is on, so turning it off restores what it hid.

`GET /share/blocks` now carries `all`; `POST`/`DELETE` take `{ all: true }`
beside the existing `{ username }`. Managed from the same Blocked people
card in the dashboard's Security tab.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(share): keep the digest sweep off a window that still has a timer

The sweep treated an entry as orphaned the moment its window closed, which
is also the moment the node that armed it fires. Claiming an entry is only
exclusive among flushers that can see each other's deletes, so the two
could each claim a share of the same digest and both send. It now waits
out a grace period first, which costs a genuinely stranded digest that
much delay and nothing else.

Both digest listings were capped at 200 with no word when they hit it — a
truncated flush sends a digest that undercounts and reads as complete.
The cap is named and logged.

Also: `#emailHolder` still described share email as off by default, which
it stopped being; the config doc said the batch window defaults to 60s
when it is 90; and the two tests that need several calls inside one window
were racing a 50ms window across four sequential round trips, so they
failed under full-suite load rather than on the behaviour they cover.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(share): stop double-encoding the recipient in two dialog messages

`i18n()` encodes what it returns, replacements included, so encoding the
recipient first showed the entities to anyone whose address or username
contains one. Same pattern already fixed two lines above.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(notification): widen the mysql shown/acknowledged columns

Both arrived from the v1 schema as `tinyint(1)`, where they were flags.
The backend rework changed the writes to a unix second; sqlite (`INTEGER`)
and postgres (`bigint`) took it and mysql did not, so on mysql every
`markShown` and `markAcknowledged` has failed with
ER_WARN_DATA_OUT_OF_RANGE and left the column NULL. Dismissing a
notification never stuck — the unacknowledged count never moved and one
already delivered came back on every reconnect.

No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a
legacy `1` keeps meaning "yes" once widened. Guarded on the current type,
because changing a column type copies the table and this directory
replays on every boot.

Not reachable from the test suite — it runs against sqlite and postgres,
both of which already have the right type. Verified by hand against mysql:
`/notif/mark-read` and `/notif/mark-ack` now persist, and a dismissed
share notification is no longer the one a later share folds into.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-18 09:36:33 -07:00
Reynaldi Chernando 401b6066b3 Use gpt 5.6 luna as default for docs (#3602) 2026-08-18 09:36:23 -07:00
Daniel Salazar ec3c33fea6 fix: misc hardening fixes (#3600) 2026-08-18 00:06:58 -07:00
Anany Singh 95d797b047 feat(workers): expose app_uid in puter.workers.get and puter.workers.list (#3596) 2026-08-18 00:05:17 -07:00
Daniel Salazar bd06e88185 feat: bring back referrals PUT-1463 (#3599) 2026-08-17 23:49:09 -07:00
Nariman Jelveh cabedc30b0 Require login on app landing pages
Prevent first-time visitors from being treated as temporary users when they open an app landing page such as `/app/<name>` or `/desktop/app/<name>`. These routes now always go through the real login/signup flow.
2026-08-17 20:11:29 -07:00
Daniel Salazar f81b76c1d1 fix: webdav connectivity fixes for macos (#3597) 2026-08-17 19:02:55 -07:00
404oops 9604cc3f23 feat(ai-chat): overhaul provider model catalogs to match current APIs (#3586) 2026-08-17 14:06:44 -04:00
2c852bf6b3 ✨ PUT-1412 File sharing backend api (#3553)
* refactor(permissions): drop hardcoded group permission map for a flat default

* test(drivers): assert credential-gate intent instead of a 403 proxy

* fix(permissions): report whether a revoke removed anything and persist the linked grant row before the flat view

* feat(permissions): replicate permission invalidations across regions

* feat(share): extend the share table into an index of active shares

* feat(share): query and maintain active shares in ShareStore

* feat(users): add a batched lookup by email

* fix(cache): apply cache updates broadcast from peer regions

* fix(permissions): scope a revoke to the issuer that granted it

* feat(share): add ShareService with a per-day share limit

A share is two writes that belong together: the permission grant, which
authorizes access, and a share row, which makes it listable and ties it to an
fsentry so it dies with the file. Nothing else grants fs:* to a user.

Authorization reuses canManagePermission — an owner satisfies it through the
is-owner implicator, a delegate through an explicit manage:fs:<uid> grant. An
owner may clear any issuer's share of their node; anyone else only the ones
they issued, or their own access. Self-revoke skips the manage gate but still
requires `see`, so it cannot be used to probe for files.

The per-day limit counts shares created rather than live rows, so revoking and
re-sharing cannot recycle a slot, and changing an existing share's mode is not
new reach and does not spend budget. Tunable via share_daily_limit.

* feat(share): expose sharing over HTTP

POST /share, POST /share/revoke, GET /share/shared-with-me, GET /share/shares.
The controller was registered but entirely commented out.

Recipients × items fan out concurrently — every pair is a distinct
(holder, entry) key, so none of them contend — bounded by
runWithConcurrencyLimitSettled, which returns results index-aligned with the
input for the per-pair outcome list. Responses carry usernames only, never
internal ids, and the 404-not-403 rule is preserved so a failed call cannot
confirm a file the caller could not otherwise see. Notifications are fired off
the response path; a share must not fail over its own notification.

Per-request caps on recipients and items bound one call's fan-out; the daily
limit bounds the total.

* feat(share): keep recipients consistent when a shared item changes

* fix(fs): stop listing issuer homes at the filesystem root

* fix(acl): serialize concurrent mode changes on one node and pin app containment on shared paths

* fix(fs): expire signed URLs over entries the signer doesn't own

signFile defaults to a ~317k-year TTL and verifySignature checks only uid,
expires and signature — never the ACL. A recipient who ever signed a shared
file therefore held a permanent, revocation-proof URL to its bytes: revoking
the share did nothing to it.

signEntry now takes the acting user and drops to NON_OWNER_SIGNATURE_TTL_SECONDS
(1 hour) when the signer is not the entry's owner. Owners keep the permanent
default, so no existing client changes behavior.

The signature-authenticated directory listing bounds its children
unconditionally: that route has no session actor, and a signature proves
possession rather than ownership, so a recipient holding a short-lived
directory signature could otherwise mint permanent URLs for every child.

A bounded window is not revocation — the durable fix is a per-entry signature
epoch folded into the HMAC and bumped on any permission change.

* refactor(permissions): drop the unused permission-issuer lookup

listUserPermissionIssuers and its store method listUserPermissionIssuerIds
existed to synthesize the filesystem root from the home directories of everyone
who had granted the caller a permission. That listing is gone — it advertised
folders readdir then refused to open — and the share index answers "who shared
with me" directly, so nothing wants them back.

One removed test only asserted that the call returned an array; the other
covered readLinkedUserUserPerms round-tripping and is kept, rewritten without
the issuer lookup.

* fix(share): return the created share, not just an acknowledgement

* feat(puter.js): add file sharing to puter.fs

share(), unshare(), listShared() and getShares() on puter.fs, following the
existing FS operation shape: positional and options-object forms through
defineOperation, JSDoc overloads as the published signature, relative paths
resolved against the app's root directory.

A bare recipient string is read as an email when it contains @ and as a
username otherwise. Sharing an item with someone who already has it replaces
their access rather than stacking a second grant, so raising read to write is
one more call.

Adds a sharing suite to the API runner, which passes unchanged on node,
browser and workerd. Documents all four methods with runnable examples, and
corrects the FS overview callout that told readers one user cannot read
another's files — true before this, not after.

* feat(gui): add a Shared folder for items others shared with you

A sidebar entry listing everything other users have shared with you, backed by
puter.fs.listShared().

The path is the sentinel `puter://shared` rather than /<user>/Shared: this is a
query, not a directory, and a path-shaped value could collide with a folder
someone actually creates. refresh_item_container and update_window_path both
branch on it to skip the stat there is no fsentry for, and the listing swaps
readdir for listShared.

Entries render at their real paths under their owners' directories — the item
container already preferred an explicit fsentry.path over joining onto the
container, so nothing else had to change. Each carries who shared it and at
what level, which the context menu reads next.

* feat(gui): share items from the context menu

A sharing dialog shaped like its neighbours — options object, HTML-string
template, jQuery wiring, delegating to UIWindow() — with a recipient field, a
read/edit/share dropdown, and the current access list with revoke buttons.

Reached from a new "Share…" context menu entry, which is hidden on items shared
*with* you: re-sharing needs manage, so the dialog would only surface an error.

Those items get "Remove from Shared" in place of Delete. Delete moves an item
to *your* trash, which for someone else's file means moving their data out of
their tree — FSService refuses it, and the user saw a bare 403. Removing your
own access is what the action was reaching for, so that is what it now does.

* fix(share): withdraw what a removed recipient re-shared

* feat(share): report access inherited from a parent folder

* fix(gui): load the puter.js bundle the server configured

* refactor(gui): extract the action icon set into a helper

* feat(gui): surface Shared in the file browser

* feat(gui): manage access from the share dialog

* test(share): cover access inherited from a parent folder

* fix(share): keep downstream access from surviving a delegate who leaves

* fix(gui): name the real owner in the share dialog

* fix(gui): page through every shared item instead of the first 50

* feat(share): return item metadata with a share

* fix(share): invalidate a holder's cache when the entry is deleted

* fix(gui): treat items inside a shared folder as someone else's

* feat(permissions): let manage inherit down the filesystem tree

Access already reached descendants through the ancestor chain while authority did not, so someone trusted to manage a shared folder could re-share the folder but nothing inside it, and could not see who had access to a file within it.

A manage-inherits-from-ancestor implicator resolves it in the permission layer, beside is-owner, so every caller agrees rather than just ShareService. It consults only the immediate parent — resolving that re-enters one level up, making a chain of depth d cost d checks rather than d².

That makes two cascade gaps reachable, both fixed here. A revoke now walks the subtree, since a grant on a descendant can rest on authority held at the folder. And it stops at a delegate whose authority survives another issuer, because what they granted was never theirs to lose.

Also pins that manage is not transitive: granting it needs manage:manage:fs:<uid>, which only the owner holds, so delegation is one level deep by construction.

* fix(gui): offer sharing inside a folder you manage

The menus encoded "manage does not inherit" and would now hide an action that works. The Shared listing records each root's mode; the menus resolve a child's by longest matching ancestor, loading on demand so a deep link or restored window works too.

* fix(share): make the daily share limit hold under concurrency

* test(share): cover concurrency, measure cost, and name cases for what they verify

* fix(gui): import the ownership helpers the item menu calls

The single-item context menu handler calls is_owned_by_me and
shared_mode_for, but the imports were only ever added to
generate_file_context_menu.js — so every right-click on an item threw a
ReferenceError before the menu could build, and the non-owner Delete
gating never ran.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): authorize before resolving the recipient

share() looked up the recipient in parallel with the entry, before the
manage check — and the two failures carried different error codes. Any
verified user with a real entry uid could probe arbitrary emails and
usernames for account existence, at no quota cost. Resolve the entry,
authorize, and only then resolve the recipient: an unauthorized caller
now sees the identical safe 404 whether or not the recipient exists.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): broadcast permission row-cache invalidations to peer regions

Every publishCacheKeys call for the u2u, u2a, and access-token row
caches omitted broadcast, so a revoke only cleared the mutating
region's Redis. A peer region applied the replicated generation bump,
re-scanned, read the deleted row from its own still-warm 5-minute row
cache, and re-warmed the flat view from it — revoked access outlived
the revoke by the row-cache TTL instead of the intended 60-second
bound. CacheReplicationService already consumes these events; the
emits were just never sent.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(fs): refuse to rename an entry owned by another user

remove and move both refuse to act on an entry the caller does not
own, even when the ACL allows the write — rename had no such guard, so
a write-mode share recipient could rename the owner's file, or the
shared folder itself, rewriting the owner's whole subtree's paths.
rename now takes the acting user and applies the same policy.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): decide a flat delete from the primary, not a lagging replica

revokeUserUserPermission deletes the SQL grant, then only drops the
flat KV entry once no issuer still grants the permission. That
remaining-check read through the row cache the delete had just
invalidated, straight to a replica — under any lag the deleted row
reappeared, the flat delete was skipped, and the stale rows were
re-cached for another five minutes. Grant-path flat entries carry no
TTL, so the holder kept working access with zero SQL rows behind it,
invisible to every listing. The check now reads the primary and
re-warms the cache with what it actually saw.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): keep a failed remote flat-invalidation from crashing the process

The outer.permission.flatInvalidated applier was fire-and-forget with
no catch, and it awaits a KV delete — one transient KV error while
applying a peer region's revoke became an unhandled rejection, which
is process-fatal under default Node. Its sibling appliers were already
guarded; this one now logs and moves on, leaving the entry to the next
invalidation or its TTL, same as a lost event.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): revoke every requested item and recipient, not just the first

revokeShare destructured only the first recipient and first item while
the parsers accept arrays up to the request caps — unshare({items:
[a, b, c]}) returned success having revoked only a, leaving access the
caller believes is gone. Revoke now fans out over every (recipient,
item) pair exactly like POST /share, reports per-pair outcomes, and
sums the revoked count; the response stays backward compatible.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): only a confirmed email designates a recipient

Recipient resolution by email accepted unconfirmed accounts, so
pre-registering someone else's address (unconfirmed) was enough to
receive shares meant for them once no confirmed account held it.
An email now only resolves to an account that has confirmed it;
username shares are unaffected.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): accept tilde-rooted paths like the FS routes do

The SDK resolves relative paths to ~/..., but the share routes never
expanded the tilde — a ~-prefixed string was read as a uid and every
relative-path call 404'd. Item parsing now treats ~ as path-shaped and
expands it to the actor's home with the same helper the legacy FS
routes use, on share, revoke, and the shares listing.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): walk a directory revoke by parent linkage, not path prefix

listByFsentrySubtree matched descendants with fsentry_id = ? OR path
LIKE ?, which has two problems: fsentries.path is lazily backfilled
and NULL on old rows, so those descendants' shares silently survived a
directory revoke, and the OR'd predicates forced a scan of every
active share. A recursive CTE over parent_id — the same shape the
lineage resolver already uses — covers every descendant and runs on
idx_parentId_name.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(gui): give each item its own share dialog

single_instance keyed the dialog on the app id alone, so opening
Share… on a second file focused the first file's dialog — typing a
recipient there granted access to the wrong file, with only the title
hinting at it. The dialog is now instanced per path: same item
refocuses, different item opens fresh. Also stops pre-encoding the
title, which UIWindow encodes again.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(permissions): let manage answer a write check

A manage grant let its holder re-share a folder but not work in it: the
ACL mode family stops at write, and the fs exploder had no rule for the
narrowest mode, so `manage:fs:<uid>` never satisfied `fs:<uid>:write`.

Fold manage into the candidate list for every non-manage mode, in both
the access-token branch and the scan branch, and give `write` an (empty)
exploder rule so the manage arm is emitted for it too.

* fix(fs): authorize restructuring by write on the parent

* fix(fs): let a share recipient work inside a shared folder

rename, remove and move refused outright when the entry belonged to
someone else, so a recipient with write could neither delete nor rename
anything inside a folder shared with them. The GUI compounded it by
hiding Delete for any item it did not own.

Authorize the three by ACL write on the entry's parent. For an owner
that is the same answer; for a recipient it grants the inside of a
shared folder and withholds the folder itself, whose parent is the
owner's private tree.

Deleting sends the item to its owner's trash rather than the deleter's,
so it leaves the recipient's view without leaving the owner's account
and without changing hands. A move may not otherwise carry someone
else's entry out of their tree.

* fix(fs): give a new entry to the owner of the folder it lands in

A file a share recipient added to a shared folder was recorded as
theirs while living in the owner's tree, so a subtree could hold rows
belonging to several people — and the storage it consumed was checked
against the writer while being counted against the owner.

Take the owner from the parent row at every insert, charge the
allowance to that owner, and hand a moved entry over to the tree it
moves into. An entry now always belongs to whoever owns the directory
holding it.

* feat(fs): address shared entries as ~/share/<uid>

A recipient could read the owner's whole path off any shared entry —
where they keep the file and what sits beside it, neither of which the
share is about.

Give shares their own namespace. `~/share/<entry-uid>/rel/path` resolves
to the real path on the way in, and outgoing paths are rewritten to it
on the way out. Entries the actor owns pass through untouched, so no
existing client contract moves.

* revert(fs): mask only the directory bar, not the addressing

fe535d598 made `~/share/<uid>` the actual address for every shared
entry. That reached far past the intent: item names became uuids, the
Shared views rendered uuids instead of filenames, and navigation
addressed entries through a namespace nothing else understood.

Put real paths back everywhere — responses, the share listing, and
request handling — and do the masking where it was wanted, in the
window's directory bar. A recipient sees `Shared › Contents › sub`
while every crumb keeps the real path it navigates to.

The share listing now carries the entry's name, content type, owner and
a signed thumbnail. A share row has no fsentry behind it for a client
to stat, and the stored thumbnail is an `s3://bucket/key` URI that no
client can render and none should see.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix: path obfuscation, webdav + small ui stuff

* test(share): assert the masked share path by its exact shape

The substring check tripped on the scratch files' own names, which start
with `sharing-`; the exact-equality assertion on `/<owner>/<uid>/<name>`
already proves nothing above the share leaks.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): rename policy, shared-view guards, webdav share parent, quota/bucket invariants

- Rename: a directly-shared FILE renames with write on it; a shared
  folder root stays fixed (its name is the owner's tree structure).
  GUI can_rename mirrors the backend, guards all editor entry points.
- Up from a share root goes to the Shared view on both surfaces; the
  Shared view gets a single crumb, a disabled Up button, and refuses
  drops, New/Paste, uploads and ctrl+V everywhere (it is a query, not
  a directory).
- WebDAV: PROPFIND on /owner/uuid answers as a virtual collection
  holding the share root (ACL-gated; 404 for strangers).
- Storage allowance override no longer crosses user boundaries: a
  recipient's plan cannot raise the owner's cap.
- Overwrites stay in the bucket the entry already lives in instead of
  repointing to the handling server's bucket and stranding the old
  object.
- manage mode documented as implying write (matches enforcement);
  share dialog label now "Can edit & share". Documented that fs socket
  events are owner-only.
- Sidebar: saved orders gain the Shared entry once the user has shares.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): close review findings, and retire /auth/grant-user-user

Security
- Recipient writes no longer echo the owner's real path back. The pending-write
  event and the upload-progress meta both go to the *acting* user, so a write
  into a shared folder handed out the layout above the share root.
- `maskerFor` failed open: the first caller fixed the actor, and one that ran
  before the request knew who was acting pinned `undefined` — after which every
  path published unmasked. It now adopts the first real actor and rebuilds for
  a different one.
- `/auth/grant-user-user` returns 501. It wrote user-to-user grants straight to
  the permission tables with no share row, so nothing could list or cascade a
  revoke over them — and the new `manage` mode meant a delegate could reach it
  for the owner's files. Filesystem access goes through `/share`; nothing else
  is meant to pass between two users. Undocumented (its docs page was never
  written) and no callers. Revoking is untouched, so grants made before this
  can still be withdrawn.
- `#assertCanManage` asks about the permission actually being granted rather
  than a fixed `read`, so sharing at `manage` is refused at the share layer
  instead of by `grantUserUserPermission` two levels down.
- `listSharedWithMe` checks the grants, not just the index: withdrawing access
  any other way left the row publishing name, size and a signed thumbnail URL.
- Moving an item into another tree now retires its shares. Grants are keyed on
  uuid, so they followed it and left the new owner with recipients they never
  agreed to.

Correctness
- `acl.check` gets the real path again, not the masked one. ACL matches on the
  path string, and a mask hides the `AppData/<appUid>` shape it needs.
- "Leave this share" works for a grant that predates the index; the fallback
  scoped the delete to the caller as issuer, which can never match.
- `move`/`copy` reject a name with a slash or a `.`/`..` segment, as `rename`
  already did. It matters more here: a move into the owner's Trash skips the
  destination write check.
- Descendant walks scope by path, not by owner, so rows predating the
  one-owner-per-subtree invariant aren't orphaned by their parent's deletion.

Performance
- Index `user_to_user_permissions(permission)`. Retiring a node's grants is a
  prefix match, but the primary key is (issuer, holder, permission), so it was
  a full scan.
- Retirement is coalesced and chunked. `remove()` emits one event per
  descendant, so deleting a directory fired one unindexed lookup per file, all
  at once and unawaited.
- `listReaching` is a plain `fsentry_id IN (...)` on `idx_share_fsentry`. It
  runs behind every file write, and the join-plus-OR it replaced was
  unindexable.

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-08-17 09:08:23 -04:00
Ayush tiwari fc861da89f feat: add Gemini 3.7 Flash (#3587) 2026-08-16 23:51:37 -07:00
Parman Mohammadalizadeh facb844747 fix: derive express subdomain offset from the configured root domain (#3575)
Express derives `req.subdomains` by dropping a fixed number of labels
from the right of the hostname, defaulting to 2. A deployment whose
`domain` has more than two labels therefore reads its own root domain as
an active subdomain, so the user-site redirect sends the root origin to
the static hosting domain, which sends it back. Self-hosting docs
recommend exactly that shape (`puter.example.com`).

Set the offset from the label count of `config.domain`. Two-label
domains keep the express default, so existing deployments are unchanged.

Fixes #3561
2026-08-16 23:48:20 -07:00
Yashwarhade8010 3b791bb334 docs: update web search examples to gpt-5.6-luna (#3589) 2026-08-16 23:45:02 -07:00
Daniel Salazar a64a444a17 feat: batch delete for flush + metering fixes (#3593) 2026-08-16 15:26:33 -07:00
Daniel Salazar d3c9828207 fix: metering gui maths (#3592) 2026-08-16 14:57:29 -07:00
Daniel Salazar 87180b1081 fix: metering (#3591) 2026-08-16 14:19:05 -07:00
Daniel Salazar d22b656df8 fix: metering issues with addons (#3588) 2026-08-15 22:06:51 -07:00
Nariman Jelveh 19cd5f4476 feat: add BytePlus ModelArk providers (chat, image, video) (#3498)
* feat: add BytePlus ModelArk chat provider

Adds BytePlus ModelArk as a provider for the puter-chat-completion
driver, following the MiniMax/ZAI providers as reference per
doc/contributing-apis.md.

- OpenAI-compatible endpoint at ark.ap-southeast.bytepluses.com/api/v3
  (apiBaseUrl config selects the region)
- Static catalog of 16 chat models (Seed 2.x/1.x incl. vision, GLM,
  DeepSeek, GPT-OSS) with limits and per-token pricing from the
  official docs
- Passes Ark's thinking/response_format/stop params through custom;
  normalizes reasoning_content to reasoning
- Bare deepseek-v4-* names stay with the first-party DeepSeek
  provider; BytePlus only claims prefixed aliases
- Offline unit tests (mocked SDK against a real test server) plus an
  env-gated integration test

* feat: add BytePlus image and video providers

Extends the BytePlus ModelArk integration to the puter-image-generation
and puter-video-generation drivers, reusing the same services.byteplus
API key and regional apiBaseUrl as the chat provider.

Image (Seedream/SeedEdit via OpenAI-compatible /images/generations):
- dola-seedream-5-0-pro (pixel-tier pricing + billed input images from
  the 2nd on), seedream-5-0-lite, 4-5, 4-0, and seededit-3-0-i2i
- quality tiers 1K/1.5K/2K; aspect ratios resolve to Ark's documented
  pixel sizes; explicit WxH passes through with Ark's bounds enforced

Video (Seedance via Ark's async /contents/generations/tasks + polling):
- Seedance 2.0 / 2.0 Fast / 2.0 Mini / 1.5 Pro / 1.0 Pro / 1.0 Pro Fast
  (2.5 is priced but its API isn't live yet, so it's excluded)
- per-video-token billing from usage.completion_tokens, with per-second
  estimates feeding the credit cap; audio vs silent rates for 1.5 Pro
- first/last frame and reference-image inputs; generate_audio param
  added to IGenerateVideoParams

Pricing and capabilities hardcoded from the official docs (ModelArk
pages 1544106, 1330310, 1520757, 1521309, 1541523). Offline unit tests
mock the SDK / global fetch; integration tests are env-gated on
PUTER_TEST_AI_BYTEPLUS_API_KEY.

* fix: correct BytePlus catalogs and validation against the live API

Verified the three BytePlus providers against ModelArk with a real key;
these are the mismatches that surfaced.

- Drop seededit-3-0-i2i-250628. Ark reports it as Shutdown and every
  request 404s. Its now-unreachable image-to-image branches in the
  provider go with it.
- seedream-4-5 and the 5.0 series enforce a 3,686,400 pixel minimum, so
  they only accept the 2K tier. Mark them 2k-only and snap an
  unsupported tier up to the nearest allowed one, which also keeps the
  aspect-ratio table from mapping to a sub-minimum size.
- glm-4-7 has a 204,800 token context, not 256K.
- Guard the actor in the image provider like the video provider does.
- Round a sub-minimum video duration up to the shortest supported clip
  instead of reporting it as insufficient funds.
- Gate video resolution on the model's own dimensions; the dims table is
  shared across a family and accepts more than any one model does.

* Tighten BytePlus AI provider handling

Extract shared reasoning-content normalization for OpenAI-style chat providers, and harden BytePlus image/video behavior. This updates image tier and size validation, normalizes aspect ratios and input image refs, prevents mismatched BytePlus key/base URL fallback config, makes video resolution matching case-insensitive, and rejects excess reference images instead of silently truncating them. Tests were expanded to cover the new BytePlus request and validation paths.
2026-08-15 21:08:17 -07:00
404oops fc56075115 Merge pull request #3584 from rupesh0001-tech/rupesh
feat(ai-chat): add grok-4.6 model to xai provider
2026-08-15 21:22:01 +02:00
404oops e1998e39ab Increase max_tokens limit to 500,000 2026-08-15 21:20:45 +02:00
Daniel Salazar fb7968a1c7 fix: metering hardening; handle burst of unfinished ai requests (#3585) 2026-08-15 10:52:29 -07:00
rupesh0001-tech c88eeb3868 feat(ai-chat): add grok-4.6 model to xai provider
- Add grok-4.6 model definition with pricing and context specs
- Add aliases x-ai/grok-4.6 and grok-4.6-latest
- Add unit tests for alias resolution and metering calculation

Closes #3562
2026-08-15 20:32:22 +05:30
Reynaldi ChernandoandCopilot Autofix powered by AI 9e25ce1401 [PUT-1478] Add KV commands to CLI (#3579)
* Add KV commands to CLI

* docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-08-15 11:43:30 +07:00
Daniel Salazar f15d835eeb fix: restrict openai and anthropic compatible endpoints to be subscription (#3583) 2026-08-14 18:55:15 -07:00
Daniel SalazarandClaude Fable 5 27c4506e02 feat: usage rendered as credits on a 50c base, free allowance doubled (#3569)
* feat: usage rendered as credits on a 50c base, free allowance doubled

Usage is now displayed in credits: a configurable creditsPerDollar rate
(default 2,000, making the free tier's 50c allowance an even
1,000-credit base) ships with the metering usage response, and the
dashboard's usage cards and per-API table show raw credit numbers
through one shared formatter. The dashboard budget math is also fixed —
capacity is spend plus server-netted remainder, so held purchased
credit can never render as negative usage. The registered-user free
allowance doubles to 50c, tier display names move to Basic/Plus/Pro,
and dead referral-promise strings leave the English dictionary.

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: the credits display rate exists only in deployment config

No default in code and no rate on any wire the plan surfaces read: a
deployment that doesn't configure creditsPerDollar renders usage in
dollars, exactly as before credits existed. The usage endpoint sends
the rate only when configured, and nothing in code or comments states
what any deployment's rate is.

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: metering reports pre-scaled credits, never raw amounts

The usage, per-app usage, and cost-catalogue endpoints multiply every
monetary field by the config multiplier (renamed creditMultiplier)
before responding, and flag the unit; the multiplier itself never
ships. Counts, units, and byte figures pass through untouched. With no
multiplier configured the endpoints report raw amounts and clients
render dollars, as before credits existed.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: absent addon fields stay absent when scaling to credits

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 16:29:01 -07:00
Neal Shah 6a7c09f54e blacklist dynamic workers (#3582) 2026-08-14 18:35:34 -04:00
Reynaldi Chernando 6254714484 deprecate gemini 2 flash and lite (#3581)
* deprecate gemini 2 flash and lite

* fix test
2026-08-14 17:27:48 -04:00
Neal Shah efa2cb4441 add more extensibility to workers driver (#3565) 2026-08-14 17:24:36 -04:00
Nariman Jelveh 4bc969b99e Update RecommendedAppsService.ts 2026-08-14 14:05:44 -07:00
Reynaldi Chernando 65f6d38432 update gpt 5.6 luna and terra pricing (#3580) 2026-08-14 16:38:13 -04:00
Daniel SalazarandClaude Fable 5 d641f6f7a1 tests: count only this test's dynamo reads in the block-window assertion (#3576)
The spy sits on the test server's shared dynamo client, so a raw call
count also picks up background work and the async tail of earlier tests
in the file — which made the block-window test flakily report a third
read. Each test runs in its own random namespace, so filtering the
spy's calls to that namespace removes the cross-talk without loosening
the assertion.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 11:05:25 -07:00
Daniel SalazarandClaude Fable 5 00828b9998 fix: concurrency limiter heals legacy INCR keys instead of failing open (#3574)
The zset rework of the redis concurrent backend (#3529) kept the same
key names the old INCR counter used. Against a leftover string key every
zset command in the acquire MULTI fails WRONGTYPE while the trailing
EXPIRE still succeeds — so steady traffic refreshes the stale key
forever, and the failed zcard result turned into NaN, which admitted
every caller unbounded. Release then errored WRONGTYPE on each request
(the '[concurrent] release failed' log storm).

Surface per-command MULTI errors instead of coercing them to NaN, and on
WRONGTYPE delete the legacy key and re-acquire against a clean zset.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 01:40:05 -07:00
Daniel SalazarandClaude Fable 5 126ec09556 docs: publish rate limits and quotas, and make limit changes move the docs (#3568)
Developer-facing reference for every rate limit, concurrency cap,
quota, and error shape — an advanced page, since puter.js already turns
the common failures into prompts. AGENTS.md and CONTRIBUTING.md now
carry the rule that a PR moving any of these numbers updates the page
in the same PR: an undisclosed limit is one developers discover as a
service failure.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 00:41:01 -07:00
jelveh 25a5799088 Update RecommendedAppsService.ts 2026-08-14 00:38:05 -07:00
Daniel SalazarandClaude Fable 5 ea2f9967a6 fix: over-quota batch uploads surface the storage prompt instead of failing quietly (#3566)
* fix: over-quota batch uploads surface the storage prompt instead of failing quietly

Partial batch failures now carry each item's code/status, and when every
failed item failed the same way the shared code/status is hoisted onto
the rejection itself — so an upload that exceeds the storage quota
rejects with storage_limit_reached/413 and the SDK's upload handler
shows the free-up-space prompt. A partial failure is also no longer
misread as the signed-batch endpoint being unavailable.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: out-of-storage copies prompt to upgrade the same way uploads do

Any fs operation the server refuses with 413 storage_limit_reached now
surfaces the upgrade prompt — the check that lived inline in upload's
error handler moves to a shared helper wired into the operation
scaffold's reject path, so copy/move/mkdir/rename get it too. The
desktop's copy/paste suppresses its generic alert for that code, since
the SDK dialog already explains the refusal and carries the fix.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 00:26:07 -07:00
Daniel Salazar ba9b885a5a Revert "feat: dashboard app search continues into the App Center (#3567)" (#3572)
This reverts commit 6cc241ac31.
2026-08-13 21:00:02 -07:00
Daniel SalazarandClaude Fable 5 6cc241ac31 feat: dashboard app search continues into the App Center (#3567)
A search that matches none of the user's own apps now puts the same
query to the App Center catalogue and renders the hits under the empty
state — debounced, filtered to apps the user doesn't already have, and
launchable in place.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-13 20:48:03 -07:00
Nariman Jelveh 7f8e24f06d fix: keep "Add an app" in the grid while searching
Typing in the Apps tab's search dropped the tail tile — deliberately, on the
grounds that a query asks which apps you HAVE and a tile matching nothing has
no business in the answer. That reads the search too narrowly: looking for an
app you don't have is the same gesture as looking for one you do, and the
moment the search comes up short is exactly when the way to GET an app is
worth having on screen.

So the tile is pushed unconditionally now, and a query that matches nothing no
longer replaces the grid with a bare "No apps match your search" — the line
moves above a grid holding just that tile. Keeping it a real grid, rather than
parking a tile inside the empty state, is also what keeps it reachable: arrows
and Enter only walk tiles inside .myapps-page, and the tail tile carries
tabindex -1 for the pager's roving focus.

The no-apps-at-all state is untouched: with nothing installed and nothing
typed, the empty state still stands in for the whole grid.
2026-08-13 18:53:59 -07:00
Nariman Jelveh 9b6627b1ea fix: an app's background helper must stay private to it
`launchApp(name, args, { background: true })` hid the child's window and
nothing else, so the instance was hidden but not private: it kept its
taskbar item, lit the running dot on its dashboard tile, and — because
the dashboard enforces one instance per app — WAS what clicking that tile
opened. Minimize the parent, click the child's tile, and Puter handed the
user the window the parent was mid-conversation with; the app, which only
ever meant that instance for its caller, said so on screen. Showing it
also dropped the never-shown marker, so from then on the helper outlived
its launcher and owned the tile.

Hidden now means unreachable. `user_facing_windows` is the one rule: of
an app's windows, the ones that are the user's. Everything the user
drives filters through it — the tile's reopen and its running dot and
Quit item, the taskbar item's click, window list and show/hide/close-all,
the file row's switcher and open-file dot, and the dashboard's
Back/Forward — so a helper is never focused, restored, listed, or counted
as running. With none of the user's own left, the tile simply launches a
fresh instance and the helper keeps serving whoever launched it. Paths
that act on the APP rather than on a window it happens to have open —
uninstall, and a launcher's close taking its helpers down with it —
deliberately keep the unfiltered list.

The taskbar item is now what a hidden launch EARNS by becoming visible,
rather than something it advertises from the start; makeWindowVisible
adds it (and fires the dots event) before showing, since focusWindow
marks an item active and needs it to exist. Windows stamp data-in_taskbar
when they take their place in data-open-windows, so the close path only
decrements a count it actually joined — otherwise a helper closing beside
the user's own window would zero the count and strip the item off a
window still open. For the same reason a helper's close no longer pops
the dashboard URL: it never claimed an entry (the push only happens for a
window created visible), and consuming one would minimize the user's
window of the same app.

is_unseen_background_window gains the other half of its own definition:
launched hidden AND launched BY another app. An app that is always
windowless (`background` on the app record) starts hidden too, but when
the user opens it there is nobody it is serving — it is the instance
their tile has to find, or every click would start another one they
cannot see.
2026-08-13 18:38:13 -07:00
Nariman Jelveh 7576e0b03a fix: route dashboard file menus by input modality, not touch capability
A mouse right-click on a file row in the dashboard Files tab opened the
mobile context-menu sheet (centered over the row, nowhere near the
cursor) on any touch-capable laptop or desktop, because phone/tablet
detection used navigator.maxTouchPoints > 0. The same check applied the
.touch-device class, degrading mouse selection and drag on those
machines.

Route by how the menu was invoked (the row's tracked pointer type) plus
isTouchPrimaryDevice() — coarse pointer and no hover — which is false on
touch-capable laptops but true on iPads whose UA claims macOS. Also let
sidebar-folder and background menus accept taphold on such iPads, where
they were previously unreachable (taphold dismissed, and iOS never fires
native contextmenu).
2026-08-13 17:46:58 -07:00
Nariman Jelveh d980a9f590 fix: don't invert full-color icons in the dashboard context menu 2026-08-13 17:33:14 -07:00
Nariman Jelveh d18ea1adb5 fix: a background app must not outlive the app that launched it
An app launched with `background: true` gets a real window from the moment it
starts, just hidden. Nothing ever took it down: when the app that launched it
closed, the child kept running with no way to reach it and no reason to be
there. On the dashboard the only sign was a running dot on a tile the user had
never lit up, and clicking that tile did nothing at all.

Three things were wrong, and all three had to go:

A window nobody has seen now dies with its launcher. UIWindow's close path
closes hidden children the closing app launched, keyed on a marker stamped at
creation. makeWindowVisible drops that marker the first time the window becomes
visible — showing itself with `puter.ui.showWindow()`, or the user showing it —
because from then on the window is the user's, and keeps running.

The dashboard tile is a real handle again. focusExistingAppWindow only routed
MINIMIZED windows through showWindow(); a hidden one fell through to
focusWindow(), which leaves it invisible while handing it the keyboard. With no
taskbar in dashboard mode the tile is the only handle on a background app, so
that click had nowhere else to go. It now asks whether the window is on screen
at all. The Files tab's row-click had the same one-line defect.

And a background instance can't take a tile from the user's own session: an
on-screen window wins, then a window the user has seen, then a hidden one.

Removing the child then hit a crash of its own: ExecService's `remove` handler
dereferenced the launcher's iframe to say goodbye, and the launcher was already
gone. Throwing there aborts jQuery's remove() itself, so the window stayed in
the DOM — running dot and all. That one also hit anyone closing a background app
from the taskbar after its launcher had closed.

The predicates behind all of this live in one helper, window_visibility.js, with
showWindow() reading the same `hidden, not minimized` rule it spelled out inline
before.
2026-08-13 14:36:08 -07:00
Nariman Jelveh 08f075d774 Update RecommendedAppsService.ts 2026-08-13 11:45:28 -07:00
Nariman Jelveh 664de0016b fix: a hidden window must not hold the keyboard
Launching an app to serve another one took focus away from the app the user
was working in, and they had to click it to get it back. Typing a guest name
into Calendar's event editor — which launches Contacts to read the address
book — dropped the user out of the field mid-word.

Four things conspired, all of them variations on "a window nobody can see is
treated as the window in front":

`window-active` was hardcoded into every window's markup, including windows
created hidden. focusWindow() — the thing that normally grants that class,
and strips it from everyone else — is already skipped for a hidden window, so
two windows ended up claiming it. It now follows the same rule as the
window_stack push a few lines above: visible windows only.

Both "focus the window once its IPC attaches" sites read that class to decide,
so a hidden child window pulled focus a beat after launching. Neither focuses
a window with data-is_visible="0" any more.

makeWindowInvisible left `window-active` on the window it had just hidden.
Since focusWindow() disables pointer events on every other app's iframe, an
app calling puter.ui.hideWindow() on itself left whoever launched it both
unfocused AND unclickable until the next click. It now hands activation back
to the top of the window stack, the way closing a window does, and drops
itself out of the activation order until it is shown again.

showWindow only ever un-minimized: for a window hidden by hideWindow() (which
keeps its geometry and writes no data-orig-*) every branch left it hidden and
stamped NaN geometry on it from the missing attributes. It now routes such a
window to makeWindowVisible. That is what makes the taskbar item a real
handle on a background app — clicking the item calls showWindow on the group.
2026-08-13 09:59:37 -07:00
Nariman Jelveh e273431f14 feat: let an app launch another app in the background
`puter.ui.launchApp(name, args)` had no way to say "I need this app's API,
not its window". That matters because we create and show an app's window
before the app's own code runs, so an app launched purely to serve another
one cannot avoid appearing on screen: the best it can do is call
`puter.ui.hideWindow()` once it boots, which reads as a window flashing
open and shut. In dashboard mode it was worse than a flash — the child
maximized into the tab and minimized its parent behind it, so asking a
service app a question took the user's app away from them.

So `launchApp` now accepts `background: true`, and the window starts
hidden. The app is otherwise entirely normal: it keeps its taskbar item,
so a user can see that it is running, show it, or close it, and it can
show itself with `puter.ui.showWindow()` whenever it has something to say.
Only a literal `true` counts, since the flag arrives over IPC from another
app.

The decision now lives in one predicate, `starts_hidden(app_info, options)`,
which folds this together with the existing app-level `background` flag and
is used everywhere the old flag was read — including the dashboard's
minimize-the-parent branch. `show_in_taskbar` deliberately still keys on the
app-level flag alone: an app that is always windowless has nothing to put in
the taskbar, while a background *launch* should stay visible there.

Existing callers are unaffected: with `background` unset, both paths
evaluate exactly as they did.
2026-08-13 09:47:35 -07:00
Daniel Salazar 8ed8feed4b fix: don't let an aborted upload take down the process (#3557) 2026-08-13 01:02:17 -07:00
Daniel Salazar 22f5bf5429 fix: duplicate emails (#3556) 2026-08-12 22:00:48 -07:00
Nariman Jelveh 198184f986 fix: drop the feedback dialog's rate limit entirely
Reopening still failed intermittently: any call the backoff refused looks
exactly like the user closing the dialog, so an app's "Send feedback"
button just did nothing, with nothing to see anywhere.

Rate limiting this dialog is not worth that. The other modals an app can
open with no user gesture — requestPermission, alert, prompt — have no
rate limit either, so one dialog at a time is the only rule left, and a
refusal now says so in the console.
2026-08-12 17:46:05 -07:00
Nariman Jelveh 026ae46530 fix: let an app reopen the feedback dialog after the user closes it
The abuse guard counted every dismissal that sent nothing as a strike
against the app, so closing the dialog blocked the app's next
showFeedbackDialog() for 10s, then 60s, then for the rest of the page's
life. The key is the app uuid, so relaunching the app did not clear it,
and a blocked call replies sent:false — the app just sees nothing happen.

Key the guard off how fast the app comes back instead. A reopen within a
second of the app's last dialog activity is machine-paced and gets the
same escalating backoff; attempts made while backed off count as activity
too, so a loop cannot wait out a tier and start from zero strikes. Any
human-paced reopen clears the app's record, so the tiers are only reached
by an app that reopens at machine speed several times running.

One dialog at a time, and a successful send clearing the record, are
unchanged.
2026-08-12 17:35:54 -07:00
Nariman Jelveh 0615c3aa3c feat: add "Show hidden" to the dashboard's folder context menu
The dashboard's files tab filtered out dot-files unconditionally, so
there was no way to reach them the way the desktop Explorer allows.
Add the same "Show hidden" toggle next to Refresh, driven by the
shared show_hidden_files preference so the two stay in sync, and dim
revealed rows with item-revealed like the Explorer does.

The socket-driven item.added path applies the same rule, so a hidden
file created elsewhere can't appear in a view that filtered its
siblings out.
2026-08-12 17:26:04 -07:00
Nariman Jelveh 748461aabb fix: show the new folder's row in the dashboard before mkdir answers (#3555)
* fix: show the new folder's row in the dashboard before mkdir answers

Both New Folder paths in the dashboard Files tab awaited mkdir before
drawing anything, so on a slow connection the button appeared to do
nothing for seconds. The toolbar path then did a full renderDirectory on
top of that -- a second round-trip, and with the default eventual
consistency the new folder could miss the listing entirely, leaving the
rename editor unopened.

Both now go through one createFolderInstant(), which renders the row
first: a locally predicted name (mirroring the backend's " (N)" dedupe
convention), a temporary uid, and the name editor already open. mkdir is
still asked for the plain "New Folder" path with rename: true, so the
server keeps owning deduplication; the prediction is only what we draw in
the meantime, and the row corrects itself if the two disagree. On failure
the row is withdrawn and the error is shown rather than swallowed.

Renaming is the next thing the user does and it needs the real uid, so
rename() awaits the create promise parked on the row. Everything else
that acts on the item -- open, menus, drag -- sits out while the row is
pending. mkdir's item.added comes back to the originating client (the
event carries no original_client_socket_id) and cannot match a temporary
uid, so _creatingItem still suppresses it, now as a counter: as a flag,
one create finishing uncovered another still in flight.

Two things found on the way:

- directory_depth_limit_exceeded had no translation key, so that alert
  (including the pre-existing desktop create_folder path) rendered the
  raw slug.
- The empty-directory notice is now restorable without refetching, since
  a withdrawn create has to put it back. It carries its own class because
  the loading overlay shares the same container.

Verified end to end against a dev backend with mkdir slowed to 3s:
toolbar and context-menu paths, rename and Escape before the response,
two overlapping creates, and a rejected mkdir.

* fix: don't draw the instant new-folder row while a listing is rebuilding

renderDirectory() clears .files before its readdir resolves, so a row
drawn during a load survives the clear and is left behind in whatever
directory the load lands on. Clicking the new-folder button while a
folder is still opening produced a row for `<old dir>/New Folder` sitting
in the new directory's listing, with the rename editor open on it —
clicking it navigated elsewhere and renaming it renamed a folder the user
wasn't looking at.

Drop the click while a rebuild is in flight, the same way renderDirectory
already drops navigation clicks that arrive while it renders.

* fix: stop drops onto a not-yet-created folder row from mangling files

A row drawn ahead of its mkdir carries a predicted path, but it was still
registered as a live drop target. Dropping a file on it called
move_items() with a destination that does not exist yet, and move treats
a non-existent destination as a rename target — so the dragged file was
silently renamed to "New Folder" instead of moving into the folder.
Reproduced against a slow mkdir: victim.txt became a 5-byte file named
"New Folder".

Sit the row out of the jQuery UI droppable (drop and the spring-load
hover) until the real fsentry lands, matching the guards already on
opening it, its menus and dragging it. The native-file dragster drop gets
the same guard: it uploads into the same predicted path.
2026-08-12 17:16:44 -07:00
Daniel Salazar 78cfe0c0f6 metrics: add metering buffer store coutns (#3554) 2026-08-12 15:55:03 -07:00
Nariman Jelveh 81f9f00fc0 fix: stop app feedback dialog clicks from focusing the iframe beneath
Clicking inside the feedback overlay (e.g. the textarea) reached
initgui's global mousedown -> focusWindow path: mouseover_window is
computed geometrically and is blind to the overlay, so focusWindow
focused the app window's iframe underneath — stealing keyboard focus
from the textarea and forwarding the click into the app.

Suppress it the established way (same fix as the dashboard app-group
panel): set window.mouseover_window = undefined in the overlay's own
mousedown handler, which runs before the document-level one; undefined
is the only value initgui's guard skips.
2026-08-12 15:00:32 -07:00
Nariman Jelveh 556380cf16 fix: keep app feedback send button blue on hover
The generic .app-feedback-btn:hover:not(:disabled) rule out-ranked the
.app-feedback-btn-primary:hover override (the :disabled argument inside
:not() counts toward specificity), so hovering Send repainted it with
the light --afb-hover background under white text. Strengthen the
primary hover selector so the blue gradient wins.
2026-08-12 13:42:11 -07:00
Daniel Salazar 908ec23eda fix: sanitize user data (#3552) 2026-08-12 12:08:54 -07:00
Juan Fernando Castro d5ae5a0049 🔧 PUR-1072: Flatten driver permissions to hardcoded values (#3545)
* refactor(permissions): drop hardcoded group permission map for a flat default

* test(drivers): assert credential-gate intent instead of a 403 proxy
2026-08-12 12:08:38 -07:00
Daniel Salazar 419d0aaa89 doc: update bounty doc 2 (#3551) 2026-08-12 09:56:34 -07:00
Daniel Salazar c3a0c28705 docs: update bug bountu (#3550) 2026-08-12 09:55:13 -07:00
Nariman Jelveh 7ceb2090b7 feat: app user feedback system (#3546)
* feat: app user feedback system

Add puter.ui.showFeedbackDialog(), letting users send feedback to an
app's developer. In the app environment the Puter desktop renders the
dialog; on a third-party website a puter.com popup hosts it. The message
is stored in a new app_feedback table and emailed to the app owner's
confirmed email — it never passes through the app's own code.

Feedback is strictly opt-in per app via a new apps.feedback_enabled
column (a real column, not an app-metadata key, so Dev Center's
whole-blob metadata saves can't silently erase it), settable through the
existing puter.apps.update path (feedbackEnabled).

Backend follows the layered stack: AppFeedbackStore (durable count
queries) -> AppFeedbackService (opt-in check, message normalization,
abuse caps, best-effort owner email) -> AppFeedbackController
(POST /app-feedback, GET /app-feedback/target). New app-user-feedback
email template uses the escaping-safe nl2br triple-stash.

Defensive by design:
- requireUserActor blocks app tokens, so feedback can't be submitted
  programmatically; guiOriginOnly keeps cross-origin pages out.
- App identity comes only from the validated IPC sender (desktop) or the
  browser-attested opener origin (popup), never from message contents.
- The send-feedback popup action is in NON_AUTH_POPUP_ACTIONS, so it
  never delivers a token to the opener.
- Layered limits: route rate limits, plus DB-count caps that fail closed
  when the limiter backend is down, plus a per-app daily owner-email cap.
- Owner email is fully best-effort: an unconfigured transport,
  unconfirmed/unsubscribed/suspended owner, or send failure never fails
  the request or blocks storage.
- The dialog and SDK method are resolve-only and always settle, so a
  caller is never left hanging.

Migrations for sqlite/mysql/postgres, puter.js types, docs, backend
tests (sqlite + postgres), and a Playwright e2e spec are included.

* feat: add feedback control to the dashboard app-drawer

Surface the feedback dialog directly from the app window's chrome in
dashboard mode: apps that opt in (apps.feedbackEnabled) get a "Send
Feedback" button in the dashboard app-drawer, next to minimize/close.
It opens the same UIWindowAppFeedback dialog, targeting this app by uid.

The control is only rendered when the app opted in — feedback_enabled is
threaded from the launched app's metadata into the window options — and
the dialog still re-checks opt-in server-side, so a stale flag can't send
anywhere. Reuses the existing .dashboard-app-drawer-btn styling and the
app_feedback_title i18n string, so no new CSS or strings.

Adds e2e coverage: the control appears and opens the dialog for an
opted-in app, and is absent for an app that hasn't opted in.

* feat: enable app feedback by default in Dev Center

New apps created in Dev Center now have feedbackEnabled set on creation,
so users can send the developer feedback without any extra setup. A "User
Feedback" toggle in the app's settings lets developers turn it off (and
back on); it's wired into the save payload, the dirty-state tracking, and
the reset-to-original path like the neighboring toggles.

The Save update omits feedbackEnabled unless the toggle is present, and
the backend leaves an omitted field untouched, so the default survives
the create-then-save flow Dev Center runs. Add an SDK apps-suite guard
covering that round-trip (create-on -> unrelated update keeps it -> can
be turned off).

* fix: feedback modal polish + share sender email

Address four issues with the app feedback UI:

- Dashboard app-drawer: the extra "feedback" control pushed the close
  button past the drawer's derived width and clipped it. A `has-feedback`
  modifier widens the surface by one button + gap so all three controls
  fit. The control's glyph is now a message bubble with text lines, which
  reads more clearly at 14px than the previous bare speech bubble.

- The feedback dialog is no longer a UIWindow. It's a from-scratch
  overlay modal in the spirit of the dashboard modals (uninstall,
  add-app): a fixed scrim + centered card with self-contained,
  theme-aware color tokens (light default + dark override), a bottom-sheet
  layout on narrow screens, backdrop/Escape close, and an entrance
  transition. This renders consistently across the three contexts it's
  opened from (desktop app-IPC, dashboard drawer, standalone popup), so
  the callers no longer pass UIWindow-specific window_options.

- Feedback now shares the sender's email (not just their username) with
  the developer so they can respond: the owner email sets Reply-To to the
  sender and shows the address in the body — but only when the sender's
  email is verified (an unverified address could be anyone's, so it's
  never used as a reply target). EmailClient.send gains an optional
  replyTo. The dialog note now says the email will be shared.

Tests: e2e updated for the new modal (7 pass); backend feedback suite
covers the verified/unverified sender-email split (sqlite + postgres);
EmailClient + GUI unit suites pass; type-check clean.

* fix: resolve 'app-'-prefixed app names in feedback target lookup

APP_NAME_REGEX allows names beginning with "app-" (e.g. the seeded
app-center), but resolveTargetApp's startsWith('app-') heuristic sent
those to a uid-only lookup with no name fallback, so feedback for such
apps 403'd even when enabled. Use AppStore.resolveApp (uid, then name)
like the rest of the codebase.

* fix: make feedback daily caps fail closed under concurrent submissions

The per-user and per-user-per-app caps were check-then-insert and the
per-app email cap was count-then-send, so parallel requests (or multiple
nodes, or the route limiter failing open) could all read a stale
under-cap count and push past every limit — the exact scenario the
DB-backed caps exist to stop.

Now the user caps recount after the insert (own row included) and roll
the row back with 429 if a burst breached them, and the email cap claims
its slot (email_sent=1) before sending, recounts, and releases the slot
if over cap or if the send fails.

* docs: disclose the Dev Center's feedback-on-by-default in SDK docs/types

The Dev Center deliberately creates apps with feedbackEnabled (see
32c950de5), but the SDK docs, apps.d.ts, and AppFeedbackService's class
doc all described feedback as strictly opt-in / default-false with no
qualification — so a Dev Center developer reading them would wrongly
conclude feedback is off for their app. State the Dev Center behavior
alongside the API default, and correct the update-path docs: an omitted
feedbackEnabled leaves the current value unchanged rather than
defaulting to false.

* fix: don't mint a user-app token as a side effect of the feedback popup

Every embedded_in_popup boot ran the user-app token exchange, and the
exchange is a write: /auth/get-user-app-token bootstraps an app row for
the opener origin, grants flag:app-is-authenticated (what makes the
site count as connected to the account), and creates its AppData dir.
So merely opening — or immediately cancelling — a send-feedback popup
recorded a user<->site relationship the read-only feedback flow never
needs: the server resolves the feedback target from the attested origin
without any of it.

Gate the exchange behind runsUserAppTokenExchange(action) in all three
popup paths that mint (main postAuthActions exchange, temp-user signup
success, manual signup fallback). request-permission keeps the exchange
since grants are written against the app row it bootstraps.

* fix: refuse feedback when the deployment cannot deliver it

With no email transport configured (the common self-hosted default),
submissions were stored in app_feedback — a table with no read path
beyond the abuse-cap COUNTs — the owner email was silently skipped, and
the sender was still shown 'Feedback sent. Thank you!'. The developer
never learns the feedback exists while the user believes it was
delivered.

Gate acceptsFeedback on clients.email.isConfigured so the pre-flight
reports enabled:false (the dialog shows its 'not accepting feedback'
pane) and submit returns 403 instead of swallowing messages. Owner-level
store-without-email cases (unconfirmed owner email, per-app email cap
overflow) keep their existing deliberate semantics.

* fix: settle showFeedbackDialog instead of hanging on older host GUIs

In the app environment showFeedbackDialog awaited an IPC reply with no
capability check. A host GUI that predates this feature (self-hosted
Puter running the live js.puter.com SDK) has no handler for the message
and never replies, so the promise documented as 'never rejects' also
never resolved.

The GUI now advertises the IPC dialogs it can answer via a
puter.gui_features param on the app iframe URL, and the SDK resolves
false when 'feedback-dialog' isn't listed. A reply timeout could not
substitute: legitimate replies only arrive when the user closes the
dialog, so any timeout would false-negative while the user is typing.
Older SDKs ignore the extra param.

* fix: guard app-triggered feedback dialog against desktop-lockout loops

The showFeedbackDialog IPC handler had no re-entry or abuse guard, and
the dialog it opens is a full-viewport overlay above the taskbar and
every window — so 'while (true) await puter.ui.showFeedbackDialog()'
kept the desktop permanently covered (for signed-out users, the same
loop spams the full-page signup window instead). Every dismissal just
settled the promise and let the app immediately reopen it.

Allow one dialog at a time, and back off reopens per app after each
dismissal that sent nothing: 10s, then 60s, then blocked until page
reload. A successful send resets the backoff, and user-initiated paths
(dashboard drawer) are unaffected since they don't go through IPC.

* fix: carry feedback_enabled in suggested/recommended app summaries

launch_app uses options.app_obj verbatim when provided, and the
suggested-apps launch paths (open_item.js, UIWindowSearch.js,
UIDesktop.js) pass summaries from toAppSummary — which omitted
feedback_enabled. So an opted-in editor launched by opening a .txt file
showed no Send Feedback control in the dashboard drawer, while the same
app launched from the Apps tab (full puter.apps.get object) did.

* fix: gate feedback dialog Cancel/X on an in-flight submit

Escape and backdrop clicks were already ignored while the POST was
pending, but the X, Close, and Cancel buttons weren't — clicking one
mid-send settled the promise false and tore down the overlay while the
submission still landed server-side: the developer got the email, the
app was told sent=false, and a user who resubmitted 'the failed one'
sent a duplicate and burned a daily-cap slot.

Apply the same !sending gate to the buttons and disable them visually
while the send is in flight.

* fix: reject feedback origins longer than the source_origin column

readTargetParam accepted values up to 3000 chars but the raw origin is
stored verbatim into source_origin VARCHAR(2048) (MySQL/Postgres), so a
2049-3000 char origin passed every validation and then blew up the
INSERT with an HTTP 500 on Postgres/strict MySQL — or was silently
truncated on non-strict MySQL, corrupting the abuse-forensics value the
column exists for. Cap the param at the column size.

* fix: make the feedback dialog's privacy note match what is shared

The note unconditionally said 'Your email address will be shared with
the developer so they can respond', but AppFeedbackService shares the
username always and the email only when it exists and is verified — an
unverified or temp-user sender was promised a reply path that never
materializes, and nobody was told about the username.

Show 'username and email' when the signed-in user's email is verified,
and 'username' otherwise.

* fix: stop HTML-escaping email subject lines

Subjects were compiled with default Handlebars escaping, so the
app-user-feedback subject rendered a title like "Bob's App & Games" as
"Bob&#x27;s App &amp; Games" — literal entities in the recipient's mail
client. Subjects are plain-text headers, not HTML; compile them with
noEscape. Header safety is unaffected: the transport encodes newlines
and free-form values collapse whitespace upstream.

* docs: state exactly what showFeedbackDialog shares with the developer

The doc claimed the dialog 'tells the user their username will be
shared', while the dialog's note talked only about the email address and
the implementation shares the username always plus the email (as
Reply-To) only when verified. Describe the actual disclosure: username
always, email when verified.

* docs: document the COOP false-resolve limitation of showFeedbackDialog

Under COOP the popup's opener link is severed, so the SDK deliberately
resolves false while the popup stays open and the user can still submit
(a feedback submission has no server read-back the way a permission
grant does). The documented contract ('resolves to true if the user
submitted feedback') was silently wrong on cross-origin-isolated pages —
state the limitation in the doc and the SDK jsdoc: false means 'not
confirmed', not 'not sent'.

* feat: point the feedback email footer at the Dev Center

The footer told developers to turn feedback off via a puter.apps.update
one-liner, but the toggle lives in the Dev Center app settings — and
Dev Center is where apps get feedback enabled by default in the first
place. Reword it to 'manage it in the Dev Center' with a link built
from config.origin (like app_link) so it holds on self-hosted
deployments.

* test: cover the feedback service and store layers directly

The service owns every feedback business rule — target resolution,
eligibility, message normalization, the durable caps, and the owner-email
preconditions — but was only reachable through the controller's tests. Give
it and the store their own suites so a regression names the layer it broke.

Service coverage adds the branches the route tests could not reach: a
blocked origin resolving to null rather than surfacing a 403, owners who are
suspended or unsubscribed, length measured after normalization, the 24h cap
window boundary, subject-header injection via the app title, and the email
links being rooted at config.origin.

The two describes already labelled `AppFeedbackService ...` move out of the
controller test, which keeps only the caller-facing promise that a failed
send still returns success.

* refactor: prep the feedback cap error once instead of via a factory

Both throw sites are in one call and only one can ever run, so a plain
const reads the same and drops a function that existed only to defer a
constructor.

* Require verified users for app feedback API

Add `requireVerified: true` to both app feedback routes (`GET /target` and `POST /`) in `AppFeedbackController`. This tightens access control so only verified user accounts can fetch feedback targets or submit app feedback.
2026-08-12 09:49:24 -07:00
jelveh ea63c3093e feat(dashboard): add Publish as website and Properties to folder menu
The Files tab's folder-background context menu (empty listing area,
sidebar folders, breadcrumb segments) now offers Publish as website and
Properties for the folder the menu was opened on, matching the desktop's
folder menus. Both are omitted at the filesystem root and in Trash, and
Publish is omitted inside trashed folders.

The properties modal can now resolve an entry by path when the caller
has no uid, which the folder-background menu doesn't.

The publish gate (account creation + email confirmation) was duplicated
in three places, so it moves to a publish_as_website helper that all
call sites now share.
2026-08-12 08:50:03 -07:00
Daniel Salazar e8236a100c chore: bump versions (#3548) 2026-08-12 01:18:17 -07:00
Daniel Salazar 20b3b88e39 metering: big fixes to metering + jsdoc types (#3547)
Changes are:
- global egress metering
- remove file egress cost
- introduce file op cost for the per request cost s3 has
- enforce fs read/download etc to through 402 when out of usage; allow for subdomains
- enforce kv metering when out of usage through 402; allow for workers
- jsdoc as source of truth for puter.js types
- kv driver caching for get and batchget operations with decreased costs
2026-08-12 01:06:04 -07:00
Chen, Ting-An cbae39a08f fix(i18n): add Traditional Chinese app management strings (#3541)
Translate the app discovery, app request, desktop shortcut, and app folder controls that currently fall back to English in the Traditional Chinese interface. Preserve the positional placeholders used by accessibility labels.
2026-08-11 21:18:37 -07:00
Daniel Salazar afad3e37bf fix: server health tolerance for primary read (#3542) 2026-08-11 01:19:09 -07:00
Daniel Salazar a6b2161b2c fix: misc sec fixes (#3540) 2026-08-10 22:24:44 -07:00
Neal Shah 9e864e9574 PUT-1432 (#3539) 2026-08-10 23:33:20 -04:00
Daniel Salazar 79d4201f12 fix: rate limits, AI routing, and a type-check gate (#3529)
- declare rate + concurrency limits on every route and driver that lacked one
- add acquireConcurrent for websocket connections and the DAV mount
- bucket AI models by identity key only; keep resold duplicates of any vendor
- skip recently-failed provider routes; cap the fallback chain at 3 attempts
- let full-access access tokens bind a worker to an app their own user owns
- cache resolved subscriptions so tiered limits don't add a round trip
2026-08-10 19:09:47 -07:00
Daniel Salazar 0c2d7dfa34 feat: add user created timestamp on whoami (#3538) 2026-08-10 19:08:50 -07:00
Neal Shah 2a6d2fea32 allow max-tokenless models on openrouter (#3536) 2026-08-10 18:40:10 -04:00
Juan Fernando Castro 1dec5e90c4 docs: add requestAppData() to Perms sidebar (#3534) 2026-08-10 11:38:09 -07:00
Neal Shah 566298d89c make context length on togetherAI more lenient (#3533) 2026-08-10 13:14:33 -04:00
Daniel Salazar 6bb5eacf61 fix: metering kv spamc (#3532) 2026-08-10 08:36:56 -07:00
Reynaldi Chernando 390a83c860 Fix puter.js docs and types drift 10 aug (#3531) 2026-08-10 12:47:18 +07:00
Nariman Jelveh 8f23bf7b75 fix(gui): hover the add-an-app options with the accent wash, not grey
The rows already answer hover with a --select-ring border; filling them
with the grey --dashboard-hover under that accent ring read as two
systems disagreeing. The weak select tint keeps the hover in one voice
and stays under the icon chip's stronger tint in both themes.
2026-08-09 18:23:07 -07:00
Nariman Jelveh aa85dddd38 Refresh Add App AI description copy
Update the English `add_app_ai_desc` translation string in the GUI to a more direct, enthusiastic message: “Describe the app you want and AI will build it!”.
2026-08-09 18:21:31 -07:00
Nariman Jelveh 8284f35b93 fix(gui): honor godmode file_paths launches now that /apps returns booleans
The godmode gate in ExecService.launchApp checked `godmode === 1`, but
GET /apps/:name serializes the flag as a boolean, so the check always
failed and file_paths launches (e.g. Dev Center opening a worker's
source in the editor, #2218) silently dropped the file. Accept both
shapes, matching the existing guard in launch_app.js.

Also fix the two issues hiding behind that gate:
- puter.apps.get returns `uid`, not `uuid`, so the /sign call for the
  target app sent app_uid undefined — no write grant and no user-app
  token for the launched app. Use `uid` with a `uuid` fallback.
- The closeApp IPC handler had the same `godmode === 1` comparison,
  preventing godmode apps from closing other apps' windows.
2026-08-09 18:06:58 -07:00
Nariman Jelveh ccec3ce39b fix(gui): open add-an-app choices through the deep-link intro, not a + tile morph
60686643b made App Center and the AI builder morph out of the + tile the
user clicked, which answers where the window came from but not the
question these two options exist to answer: where the APP went. The grid
never showed it arriving, so the user still ended a launch with nothing
to find, no tile for minimize to fly back to, and no lesson learned.

Choosing either option now plays exactly what an /app/<name> landing
plays (beginDeepLinkLaunch): when the app already has a tile the intro
travels to it, plays the click flourish, and the window grows out of the
app's own slot; when it doesn't, the tile is INSTALLED first — the slot
opens, the progress stroke draws, the icon springs in — and only then
does the window morph out of the tile that just arrived. Same beats,
same exposure decay, same interruptibility, same duplicate-launch claim.

The app info is prefetched in parallel and handed to launch_app, the
same contract as initgui's landing path, so the intro (which may need to
draw the arriving tile from it) never costs a second round-trip.

This also reverts 60686643b's anchor plumbing (UIWindow's
dashboard_tile_el option and the dashboard_tile_in_view export): the
intro installs a real tile for the by-app-name morph to find, so no
stand-in anchor is needed.
2026-08-09 18:02:39 -07:00
Nariman Jelveh 60686643b3 fix(gui): launch from the add-an-app modal with the tile morph, not a fade
Choosing App Center or the AI builder opened the window with the plain
fade while every other launch in the Apps tab grows out of the icon that
was clicked — the one launch that most needs to say where it came from
read as the app appearing from nowhere.

The morph resolved its anchor by app name, which cannot work here: the
whole point of these two options is installing an app you do not have,
and an app with no tile has no name to find. So the launcher may name its
anchor element (dashboard_tile_el), and TabApps resolves it once before
launching so the click flourish and the window's half can never disagree
about which tile the launch came from: the app's own tile when it already
has one on screen — minimize flies the window back there, so that is
where opening should come from — and otherwise the + tile itself.

dashboard_tile_in_view's visibility test is now a named helper it shares
with the new path, so an anchor on an offscreen pager page is still
rejected and still falls through to the fade.
2026-08-09 17:49:50 -07:00
Nariman Jelveh 61bfcbdc83 feat(gui): add an add-an-app tile at the end of the Apps grid
The Apps grid showed what you have and offered no way to get more. A
dashed plus tile now rides at the tail of every page of it: after every
app, on the last page, and nowhere else. It wears the same tile skeleton
as an app so hover, focus, and arrow keys treat it identically, but
carries no app name or group id — which is what keeps it out of the
saved order, the running dots, and the deep-link tile lookups. It cannot
be picked up, is never a drop target (so no app can land past it), has
no context menu, and sits still while the apps jiggle in reorder mode.
Search results leave it out: a query asks which apps you have.

Clicking it asks how you would like to get an app: find one in the App
Center, have one built by the AI builder, or request one that does not
exist yet. The request is a form in the same card rather than the stock
Contact Us window — a titled pane one step deeper, whose Back arrow
returns to the options with the draft intact. It posts the same
/contactUs request that window did; a failure is said inside the form,
where the request is still there to retry, rather than in an alert
window the next click on the dashboard would bury.
2026-08-09 17:37:09 -07:00
Nariman Jelveh c8985b621d Add copy-link support for app tiles
Extract app tile URL generation into a shared helper and use it for a new dashboard context menu action that copies the tile's destination. This keeps copied links aligned with the existing open-in-new-tab behavior and adds regression coverage for app, external, encoded, and folder tiles.
2026-08-09 16:54:18 -07:00
Nariman Jelveh 42299a64ed Update recommended apps list
Add 'blockarena' and remove 'pretty-tiles', 'galaxy-troops', and 'blend-fruits' from the recommended apps list.
2026-08-09 11:20:40 -07:00
jelveh 344db5cb0b Remove vault from recommended apps
Drop `vault` from the recommended apps list so it no longer appears in app recommendations.
2026-08-08 11:54:56 -07:00
jelveh 66824f607d feat(gui): install an app onto the grid when its landing is what installs it
A /app/<name> landing for an app the dashboard didn't list opened with
a plain fade and left the app NOWHERE: the grid had been fetched before
the launch granted the permission that installedApps reports, so Back
found no tile to minimize into and the user landed on a dashboard
without the app they were just inside. The round trip that teaches
"windows are inflated tiles" broke on exactly the landing where a new
user is forming that model.

Now the landing SAYS what it does. The app joins the grid at the tail
(where new apps land) with a tile drawn from the landing's own app-info
prefetch -- only a confirmed-real app ever materializes, with its real
icon -- held invisible until the intro has revealed the grid and
travelled to the tile's page, and then it is INSTALLED, in the grammar
every app store taught: the slot opens (the folder well's own surface)
with the icon dim inside it, a progress stroke draws clockwise around
the slot, and on completion the icon springs to full color and size as
the label names it. Only then does the flourish play and the window
grow out of the tile. The arrival is the one beat that never decays:
per-app news delivered at most once per app, not a repeated lesson.
Minimize now has a target, and the app is simply there afterward.

The regression surface is guarded on every edge: the parked-invisible
state is re-applied by renderApps itself so mid-intro re-renders can't
strip it, and settleDeepLinkLaunch -- which the landing's finally
always reaches -- force-reveals it, so no path leaves an invisible
tile. A _pendingInstalls overlay (the mirror of _removedLocal) keeps a
refresh fetched before the grant lands from evicting the tile, and
retires itself once installedApps confirms the app or the user
uninstalls it again. The order is never saved on the splice, so the
default-position append can't freeze a custom order the user never
made. Failed or absent app info adds nothing; installed-app landings
are untouched; animations-off and reduced-motion users skip the
choreography but still get the tile, silently. The progress ring is a
transient element (the tile's ::before is the well, its ::after the
running dot) drawn as an SVG dashoffset sweep -- no Houdini dependency
-- and the app-info prefetch now asks for the 128px icons every other
tile already uses.
2026-08-08 11:25:48 -07:00
jelveh 66dedf3082 feat(gui): let the app drawer's intro flash retire once learned
The control drawer on headless dashboard apps opened itself on EVERY
window open, forever -- 2.6 seconds of tray over the app's own top
edge to teach a lesson (the controls live in this tongue) that is
learned in a few exposures. Same doctrine as the dashboard's
deep-link intro: pedagogy that knows when to step aside.

A per-account kv counter records deliveries, once per window: a flash
played while the user could see it, or the user opening the drawer
themselves (hover, tap, focus) -- the stronger proof, since the
drawer is a headless app's only chrome and must never decay out of a
user's awareness. After three, new windows keep the bare tongue and
the flash stops; hover, tap, and focus expansion never decay.

The attach is synchronous and the stored count is not: the first
drawer of a session races the read against a short cap and defaults
to teaching -- a hung read must not hold the intro hostage, and every
failure mode (unparseable value, failed read, timed-out read, failed
write) errs toward teaching once more, never toward never teaching.
The resolved count is cached and kept current in memory, so later
windows decide synchronously and a session's own not-yet-written
increments still count. kv.incr keeps two devices from losing an
increment, and the pre-read caps the key once the lesson is learned.

If the user beats a still-deciding flash to the drawer, the flash is
skipped rather than played over their hand -- its auto-collapse would
shut a drawer they are actively using. A flash in a hidden tab still
defers to first reveal (it rides requestAnimationFrame, as before)
and an unseen one is never counted.
2026-08-08 10:25:26 -07:00
Daniel Salazar 95da802b13 fix: metering hardening (#3527) 2026-08-08 06:10:01 -07:00
Daniel Salazar 0491d5fea7 fix: mcp better uploads (#3528) 2026-08-08 06:09:49 -07:00
Juan Fernando CastroandDaniel Salazar d202be10a9 feat: let apps use another app's data with user consent (#3516)
* feat(perms): add cross-app app-data permission vocabulary

* feat(perms): sweep app grants by permission prefix

* feat(perms): resolve and withdraw cross-app data grants

* feat(kv): support an authorized namespace override and per-key privacy

* feat(kv): gate cross-app KV access behind app-data grants

* feat(fs): allow cross-app AppData access and require a scope to delete

* feat(auth): accept permission lists and gate app-data grants

* feat(perms): add requestAppData to the puter.js SDK

* feat(gui): carry permission lists through the IPC and popup transports

* feat(gui): describe cross-app data requests in the consent dialog

* docs: document requestAppData and per-entry KV privacy

* perf(perms): sweep cross-app grants only for origin-bootstrapped apps

* fix(gui): stop double-encoding cross-app consent text

* fix(perms): close three gaps in cross-app grant enforcement

* fix(kv): meter and batch the per-entry privacy probe

* fix(perms): resolve app identifiers and scopes more strictly in the SDK

* test(perms): cover the cross-app consent flow end to end

* fix: small missing token resolution for app

also adds the same exclusion for the batchPut api, small change

* fix: make resolved actor optional

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-08-08 04:04:07 -07:00
jelveh 2c17476c07 fix(gui): folder pages, live drag-out, and one merge pulse in My Apps
Three fixes to the My Apps folders:

A folder big enough to need it now paginates like the grid outside
(iOS folders page too) instead of growing a scrollbar: pages of
cols x 3 tiles in a scroll-snap scroller with dots below, wheel and
swipe to flip, and a drag held at the card's edge flips folder pages
so reordering works across them. The card's height stays constant
across pages, and a deep-link landing on an app that lives on a later
folder page travels there before the launch morph.

Dragging an app out of the open folder no longer drops it blind:
after a beat held outside the card, the app leaves the folder, the
card closes behind it, and the SAME drag carries on over the grid --
placeholder shuffle, edge page flips, even a drop into another
folder -- so the app lands where the user watched themselves put it.
A quick drop during the beat still lands beside the folder, and
swinging back inside within the beat cancels cleanly.

The folder well's fill played twice on nearly every merge: the rest
countdown anchored at first contact with the target, so the hand
decelerating INTO the tile always read as movement when the dwell
elapsed, and the deliberate refill-from-empty replayed the fill. The
countdown now re-anchors on the move events themselves -- rest is
only visible at its edges -- so the fill that completes is the one
started by the last movement, played exactly once.
2026-08-08 01:28:09 -07:00
Nariman Jelveh 06d94534e8 feat(gui): iOS-style app folders in the My Apps grid (#3525)
* feat(gui): iOS-style app folders in the My Apps grid

Drag one app onto another and let it settle: a well opens under the
target and the drop makes a folder of the two. Dropping onto an existing
folder joins it. A folder opens by growing out of its own icon into a
card over a blurred grid, where its apps can be launched, rearranged,
renamed, or carried back out.

Hovering a tile mid-drag means two things — "push over, I'm passing
through" and "swallow me" — and the tile is barely bigger than its icon,
so pixels can't separate them; motion does. The shuffle is held while
the folder offer stands and fires when the icon leaves the tile, or at
the drop, so a quick drop onto a neighbour still reorders exactly as it
did. The offer itself re-arms rather than cancelling on movement: the
last events of a drag are the ones carrying the icon onto the target and
nothing is dispatched while it rests, so a cancel-on-movement dwell
could never fire at all.

Folders are stored in their own kv key; the grid's ORDER stays entirely
in the existing saved app order, with a folder occupying its first
member's slot and its members contiguous. Every saved order therefore
stays valid with no migration, and an app whose installedApps page
failed to load keeps both its folder and its position. Folders never
nest, one that drops below two apps dissolves, and a corrupt kv value
degrades to "no folders" rather than a broken tab.

Elsewhere:
- Search looks THROUGH folders — a match the user then has to hunt for
  inside one is not an answer.
- Minimize morphs into the FOLDER when an app lives in one, and a
  /app/<name> landing opens the folder so the launch grows out of the
  icon where the app actually is.
- The uninstall FLIP keyed surviving tiles by app name, which a folder
  tile doesn't have; it now keys by identity.

New pure model in appGroups.js with tests; verified end to end in the
running dashboard (create, join, open, rename, reorder, eject, ungroup,
launch-from-folder) alongside plain reorder, search, and uninstall.

* fix(gui): keep the folder name field from inheriting input[type=text] sizing

style.css styles every input[type=text] with `width: 100%` and grows it to
`padding: 7px; border: 2px` on focus. `input.myapps-group-name` matches at the
same specificity, so it only wins the properties it actually declares — width
was never one of them, and the focus rule declared neither padding nor border
width. The name field therefore spanned the entire folder card (so its hover
and focus chip read as a full-width bar rather than the name) and grew 8px
taller the moment it was clicked, shoving the folder's app grid down.

Spell the three out, in both the resting and the focus rule — the same trap
.myapps-search already documents next door.

* fix(gui): size folder icon ghosts to the icon they stand on

Border-box only reaches a folder's icon through `.dashboard * { box-sizing }`,
and every ghost cloned from one is appended to <body>, outside that rule: the
drag ghost, the click-time launch flourish, and the open/minimize morph ghosts
all fall back to content-box, where .myapps-group-icon's 5px padding is added
to the 56px slot. Each ghost rendered 66px square and 5px off, so it visibly
popped at exactly the moment it was supposed to sit flush on the real icon.

State box-sizing on the rule itself so a clone carries it wherever it lands.

* fix(gui): stop a closing folder from swallowing the next click

_closeGroup drops the open class and leaves the overlay in place for
GROUP_PANEL_CLOSE_MS so the card can recede into its tile. The scrim is
`position: fixed; inset: 0` and still hit-testable for that whole quarter
second, so a click on the grid during it landed on the outgoing overlay — whose
handler only re-runs _closeGroup, now a no-op. Shutting a folder and reaching
straight for an app did nothing.

Take the outgoing overlay out of hit-testing; it has no interactive job left.

* fix(gui): keep a folder name typed right up to the moment it closes

The name box commits on blur, and every exit that goes through a pointer blurs
it while the folder is still open — so clicking outside, or launching an app
from inside, keeps what was typed. Escape does not: _closeGroup clears
_openGroupId first and only then moves focus to the tile below (or removes the
card outright), so the blur arrives with no open folder to rename and
_renameGroup drops it. A brand-new folder opens with its name selected for
exactly this edit, so "type Games, press Escape" — the obvious way to dismiss
a dialog — was the path most likely to lose it.

Commit the pending name on the way out, before the folder id is gone.

* fix(gui): close an open folder when the Apps tab is re-entered

The dashboard hides an inactive section and calls onActivate on the way back
in; there is no deactivate hook, so a folder left open survives the round trip
and greets the user still open over a grid they walked away from. Worse,
onActivate's focusSearch then lands the caret in the search box behind the
folder's scrim, and typing filters the grid the card is covering — a modal with
the keyboard pointed outside it.

Shut the folder as the tab comes back: returning to the tab is returning to
the grid.

* fix(gui): move focus into a folder when it opens

Opening a folder called .focus({ preventScroll: true }) on a jQuery
object. jQuery's .focus() shorthand reads a lone non-function argument
as event DATA and binds a handler with it, so it never moved focus:
the folder opened modal over the grid with focus still on the tile
behind its scrim, where Tab walked away through the inert grid instead
of cycling inside the dialog — and the object it bound as a handler
threw a TypeError on that tile's every subsequent focus.

Focus the DOM node instead, as every other focus call in this file
already does.

* fix(gui): stop renaming a folder from swallowing the click that commits it

The folder's name box commits on blur, and blur fires on the PRESS —
before the click that press belongs to. Committing re-rendered, and the
re-render replaced every tile in the open folder, so by the time the
click was dispatched the tile under the pointer was detached and the
delegated handler never saw it. Typing a name and then tapping an app
in the folder — the path a brand-new folder puts the user on — renamed
the folder and did nothing else; the app only opened on a second click.

Rebuild the folder's contents only when they actually differ from what
is on screen. A rename doesn't change them, so nothing is detached, and
a background refresh no longer throws away hover/focus either. Tiles
that survive get their drag resting-rects cleared, since the card can
have moved under them since the rects were taken.

* fix(gui): keep Enter in a folder's name box from leaving the folder

Committing the name with Enter blurred the box, which left focus on
<body> — outside a dialog that is marked aria-modal and that traps Tab
on its own subtree. The next Tab therefore walked off through the inert
grid the folder is covering, exactly what the trap exists to prevent.

Step out onto the folder's first app instead; the same blur still
commits the name. The keystroke is stopped at the box because the
grid's document-level key handler reads Enter on a focused tile as
"launch it", and would otherwise have taken the focus move as its cue
to open an app the user never asked for.

* fix(gui): stop an open folder clipping its own uninstall badges

The folder's grid scrolls, so it clips anything outside its padding box
— and reorder mode's uninstall badge deliberately overhangs the top-left
corner of every tile. The top row's badges therefore rendered as flat
tabs rather than circles, on the one surface where they are a touch
user's only way to uninstall an app they have filed away.

Give the scroller 9px of top padding for the overhang to sit in and take
it straight back off as margin, so the card and everything in it stays
exactly where it was.

* fix(gui): hold page edge-flips while a folder merge is being offered

A tile in the pager's last column sits inside the 60px edge-flip zone, so
resting a dragged icon on it — the folder-making gesture — armed the edge
dwell alongside the merge dwell, and the page flipped out from under the
very folder the user was watching form (the merge target scrolls away
mid-offer, and the drop then resolves against its stale resting rect).
Worst on phones, where 72px tiles overlap the zone across the whole last
column.

A live merge offer now holds the edge flip: entering the zone arms no
dwell while an offer stands, and an offer that arrives during a running
dwell is re-checked at the flip (a resting pointer fires no event that
could clear the timer). Carrying the icon off the tile withdraws the offer
and the hold with it, so deliberate flips — resting in the bare edge
gutter — behave as before.

* fix(gui): make Escape cancel a folder rename instead of saving it

Escape pressed mid-edit fell through to the folder's close handler, and
closing commits whatever the name box holds — so the one key every inline
rename uses for "never mind" stored the abandoned half-typed name. Escape
in the box now puts the stored name back and steps out to the folder's
tiles, exactly the cancel Finder/Explorer taught; with nothing left to
cancel (name untouched) it falls through and closes the folder as before,
so a second press still exits.

* fix(gui): refill the folder well when the merge countdown restarts

The merge dwell pins its stillness anchor at first contact with the tile,
and a hand decelerating INTO a target routinely covers more than the
7px allowance between that moment and the first tick — so the countdown
quietly restarts. The well's fill, tuned to the same 460ms, had already
completed by then and just sat there half-open: a drop it seemed to
promise a folder for actually reordered. Restart the fill with the
countdown, so what the well shows is always the countdown that is
actually running.

* fix(gui): keep keyboard focus inside an open folder across its edits

The folder card is a modal dialog, but three of its flows stranded focus
on <body>, where Tab walks the inert grid behind the scrim:

- Remove from Folder rebuilds the card's grid after the context menu has
  dropped focus, so nothing inside the dialog holds it. The rebuild now
  hands focus back to the same app's tile (or the first) whenever it finds
  focus on <body> — never stealing from an uninstall modal, which holds it
  legitimately.
- The same eject dissolving the folder re-renders the grid right after
  _closeGroup's focus hand-back, replacing the very tile it chose. The
  ejected app's own tile — where the user is looking — takes focus instead.
- A click on the card's empty space focused nothing at all. The card now
  carries tabindex=-1 so such clicks land on it (no visible ring; a focus
  ring around the whole card would misread plumbing as selection), and the
  Tab trap wraps from there instead of stepping off through the scrim.

* fix(gui): size the folder name box to the name it holds

The box sat at the browser's default ~20ch regardless of content: a short
name floated in a hover pill far wider than the word, and a 40-character
one clipped while the card had room to spare. field-sizing: content hugs
the text between a 90px floor and the card's width, iOS-style; engines
without the property keep the default box exactly as before.
2026-08-08 00:28:03 -07:00
Neal Shah aea828ff12 add worker create event and new query methods (#3524) 2026-08-07 17:51:37 -04:00
Daniel Salazar c24f0fe08b fix: mcp fs copy, move, rename (#3517)
* fix: mcp fs copy, move, rename

* feat: more mcp tools for kv
2026-08-07 17:14:40 -04:00
jelveh 085ac116c7 fix(gui): keep app landings through signup, and unstack the session picker
Signing up from a direct /app/<name> landing redirected to '/' on
success, silently dropping the app the URL asked for — the deep link's
launch (and its dashboard intro) never happened. Signup now mirrors
login's pathname-preserving redirect, but only for app-landing routes
(/app/<name>, /desktop/app/<name>): every other route keeps the
historical '/', notably /action/signup, where returning to the same
path would just show the signup form again. The query string stays
dropped, matching login's credential-leak hygiene. This covers all
three signup entry points reachable from a landing: the login cover's
"Sign up", the session picker's "Create Account", and the
must_login_or_signup fallback.

The session picker (UIWindowSessionList) also left itself on top of
the cover windows its two links open, hiding their username fields:
"Create Account" tried to close the picker via the LOGIN window's
c2a selector (which matches nothing in the picker), and "Log Into
Another Account" never closed it at all. Both now close the picker —
in the reload flows only: the no-reload (popup) flows keep it open,
where it doubles as the fallback UI when the login/signup window is
abandoned mid-flow.

Picking an account was already correct (location.reload() keeps the
landing URL); with these fixes all three picker paths, plain login,
and signup all return to the app landing, where the boot replays the
launch and its intro.
2026-08-07 11:51:44 -07:00
jelveh 5538a5d24b feat(gui): let the deep-link intro step aside for users who know it
The intro exists to teach the dashboard's spatial model; once learned it
would only tax every bookmarked landing. Two mechanisms remove that tax:

- Interruptible: any real user input (pointer/key/wheel, isTrusted only)
  during the intro skips the remaining choreography and launches at
  once — it wakes the in-flight beat sleep, so the skip is immediate.
  Input never cancels the launch itself and is never swallowed;
  whatever it was doing (typing a search, clicking a tile) still
  happens.

- Exposure decay: after 3 delivered — or deliberately skipped — intros
  the beats collapse and the sequence plays in one breath, exactly like
  a warm tile click. Counted per account in kv (kv.incr, atomic across
  devices; capped at the threshold so the key stops changing) rather
  than per device: the lesson lives in the user's head and the account
  follows them, while localStorage would also bleed between accounts on
  a shared browser. Only real exposures count: a delivered flourish or
  an active skip with the grid on screen — hidden tabs, timeouts, and
  no-tile landings teach nothing and don't count. The animated page
  flip is exempt from decay: it isn't a repeated lesson, it's live
  wayfinding to where this app lives, and its settle is needed anyway
  to put the tile in view for the morph. Every failure mode (slow or
  failed kv read) errs toward teaching once more, never toward never
  teaching.

Also bounds the intro's wait on the app-list load by the same deadline
as the tile wait — fetch has no timeout of its own, so a stalled
installedApps request used to hold the deep-link launch hostage
indefinitely.
2026-08-07 11:23:47 -07:00
jelveh 14c14fb5c5 feat(gui): animate the pager flip when a deep-linked app lives on a later page
The deep-link intro used to flip to the tile's page instantly behind the
grid's load-fade, so landings on off-page apps woke up on page N with
nothing but the pager dots hinting a move happened. Now the grid always
reveals on its first page, holds the grid beat, and then visibly travels
to the tile's page before the tile's flourish — the journey shows where
the app lives, which is also where minimize will put it back.

Smooth scrollTo has no reliable completion event across engines, so the
travel uses a settle allowance (DEEP_LINK_INTRO_FLIP_SETTLE_MS, same
pattern as the drag code's DRAG_FLIP_SETTLE_MS): the scroll's ~450ms
plus a rest so the landing reads before the tile pops. First-page apps
skip the flip entirely and are unaffected; the hidden-page bail-out is
re-checked after the flip settles.
2026-08-07 11:12:03 -07:00
jelveh 41fc8d890b feat(gui): play the tile click→morph→open intro on /app/<name> landings
A direct landing on /app/<name> now plays the same sequence a real
Apps-tab tile click does — grid appears, a beat, the tile's icon ghost
pops out of its slot, a beat, and the window morphs out of the tile —
so the landing shows the user what is being opened and where minimize
puts it back.

TabApps.beginDeepLinkLaunch waits (3s cap) for the tile to be genuinely
visible (list loaded, render done, pager flipped to the tile's page,
load-fade revealed, icon painted), paces the beats, and claims the app
in _launchingApps so a click mid-intro can't spawn a duplicate. The
launch's app-info fetch is prefetched in parallel so the intro never
delays the app's own round-trip. No tile, animations off, reduced
motion, or a background tab (hidden pages throttle timers and defer
rendering) all skip straight to the immediate plain-fade launch.

Also fixes the tab title sticking as the app's name after closing a
deep-linked app: the landing's replaceState('/') committed the
dashboard's history entry while the page still carried the server's
app-name title, and Chrome shows an entry's stored title when close/
Back traverses onto it — document.title is now reset before the
replaceState so the entry is stamped with the dashboard's own title.
2026-08-07 10:34:21 -07:00
Reynaldi Chernando 45bdc46df0 Remove deprecated models 7 aug (#3522) 2026-08-07 23:20:50 +07:00
jelveh 82ca390bf4 fix(gui): create-in-folder via sidebar right-click renders in target dir
Right-clicking a sidebar/breadcrumb folder in the dashboard Files tab and
choosing New > Folder (or any file type) created the item in the right-
clicked folder correctly, but the UI inserted the new row and started the
inline rename in whatever directory was currently open — so the item
appeared to be created in the wrong place until a refresh.

Now, when the creation target isn't the directory on screen, navigate to
the target and run the select + rename flow there; same-directory
creation keeps the incremental insert.
2026-08-07 08:49:13 -07:00
Neal Shah edfd67a83c fix PUT-1444 (#3515) 2026-08-06 17:11:04 -04:00
404oops ff16ec8900 Add Neuralwatt support (#3509) 2026-08-06 17:01:33 -04:00
Daniel Salazar c7edfc0c74 perf: improve metering perf + add extra compat for monthly charges (#3513)
closes PUT-1445 and PUT-1446
2026-08-06 10:26:47 -07:00
velzie 09f30d693c Peer: allow anontoken in place of authentication (#3501) 2026-08-06 02:03:53 -04:00
Reynaldi Chernando a9ec15d121 Update NPM readme (#3510) 2026-08-05 17:07:02 -07:00
Daniel Salazar eb53c842dd fix: oidc issues with cache (#3512)
closes #3497
closes #3502
2026-08-05 17:05:31 -07:00
Daniel Salazar 1be5ce45f7 feat: support dekstop app linking again (#3511) 2026-08-05 14:16:02 -07:00
Neal Shah 294055e055 PUT-1436, PUT-1435, PUT-1371 (#3507) 2026-08-05 17:11:50 -04:00
jelveh 036008d8a7 fix: keep uninstalled recommended apps from resurrecting in the Apps tab
Uninstall only revokes permissions, but the recommended launch list is a
global hardcoded set that knows nothing about per-user revokes — so an
uninstalled recommended app's tile came back on every reload. Persist
uninstalled app names in kv (dashboard_removed_apps) and filter only the
recommended merge against them; installedApps is never filtered, so a
genuinely (re)installed app always shows.
2026-08-05 11:01:14 -07:00
Daniel Salazar 6df292f495 fix: PUT-1429 (#3508) 2026-08-05 01:17:34 -07:00
Daniel Salazar 40c8d09f05 fix: PUT-1401 (#3504) 2026-08-04 16:37:33 -07:00
meridah7 4a211b9092 docs: document the provider option and refresh the vendor list in chat() (#3505) 2026-08-04 16:22:03 -07:00
meridah7 24fc0b0e5f fix(ai-chat): infron default model is dropped by the aggregator alias rule (#3506)
* fix(ai-chat): use a resolvable default model for the infron provider

* test(ai-chat): assert the infron default model is in the served catalog
2026-08-04 16:21:55 -07:00
Nariman Jelveh f58772ec33 feat: swap browser favicon to the focused app's icon in dashboard 2026-08-04 14:16:32 -07:00
Neal Shah eb830799c0 add limit bypass (#3500)
* add limit bypass

* update test
2026-08-03 19:43:59 -04:00
jelveh 8df72ef343 fix: dashboard app drawer icon popping in late
The drawer resolved its own icon URL (a differently-sized variant under a
different URL than the dashboard tiles use), so its <img> fetched cold and
the icon popped in mid-intro. Reuse the bitmap an Apps-tab tile or Home-tab
recent already decoded — instant from the image cache; when nothing is
rendered yet (deep links), fade the icon in on load instead of popping, and
fall back to the generic app icon on a failed fetch.
2026-08-03 11:50:43 -07:00
jelveh 63b2fc34d9 feat: highlight pasted items in dashboard files tab
Pasted (copied or cut) items now land selected, the same treatment
uploads get. copy_clipboard_items resolves with the created items'
paths instead of firing and forgetting, moveClipboardItems returns
each move response's authoritative final path (with Keep Both the
landed name differs from the source's), and both dashboard paste
entry points refresh with strong consistency and hand the new paths
to selectUploadedRows. Pastes into a folder that isn't rendered match
no rows and the highlight is a no-op.
2026-08-03 06:52:40 -07:00
Nariman Jelveh 33d7324a24 feat: 'Keep Both' option in name-conflict dialogs (#3496)
Pasting or moving onto an existing name only offered Replace or
Cancel. Add a macOS-style middle ground: Keep Both retries the
operation with dedupe_name, landing the item as "name (1).ext" and
leaving the existing item untouched.

- puter-js move.js now forwards dedupeName to the wire (copy already
  did; move dropped it).
- All four conflict dialogs offer the new button: copy_clipboard_items,
  copy_items and move_items in helpers.js, and the dashboard's
  moveClipboardItems. Multi-item selections show
  Replace / Replace all / Keep Both / Skip.
- New keep_both translation key (other locales fall back to English).
2026-08-03 06:39:18 -07:00
Nariman Jelveh 62be61642d fix: phantom name conflict when pasting onto a just-renamed path (#3495)
Renaming (or moving) an entry left the OLD path's cache key serving
the pre-update entry for the full 60s TTL: updateEntry invalidated
only keys derived from the updated row, whose path is already the new
one. Pasting an item under the freed name then hit the stale cache in
the collision check and reported a conflict for a file that no longer
exists — and accepting the Replace it offered deleted the renamed file,
since the stale entry carries its uuid.

Read the pre-update entry when the patch changes the path and
invalidate its keys alongside the new ones. The move flow was shielded
by the outer.gui.item.moved cache-invalidation handler at the
controller layer; rename had no such band-aid, and fixing the store
covers every caller regardless of which events fire.
2026-08-02 23:53:59 -07:00
Nariman Jelveh 8cebbcf3be fix: thumbnails breaking after copy — duplicate the S3 object per copy (#3494)
A copied fsentry kept its source's thumbnail pointer verbatim, so both
rows shared one S3 thumbnail object. fs.remove.node deletes the
pointed-to object, so the first removal among the sharers — e.g. the
remove performed by a replace-on-copy — broke every other sharer's
thumbnail. FSService already emits fs.copy.node for exactly this
reason (its doc comment describes the duplication), but the thumbnails
extension never subscribed to it.

Add the missing handler: S3-copy the thumbnail to a freshly minted key
and repoint the copied row, under the same only-keys-we-minted trust
rule as the read/remove paths. If the shared object is already gone
(pre-fix damage), null the pointer instead of leaving the row
advertising a thumbnail it doesn't have.
2026-08-02 23:22:24 -07:00
Nariman Jelveh 5c2a423658 fix: progress window covering paste conflict dialog; ghost row after Replace (#3493)
* fix: progress window covering paste conflict dialog; ghost row after Replace

Pasting a copied file onto a name collision buried the Replace/Cancel
dialog under a stuck 'Preparing...' progress window, and answering
Replace left a stale duplicate row in every client.

- helpers.js: copy_clipboard_items armed its delayed progress window
  with a 0ms timer (its siblings use 2s), so the window opened
  instantly over the dialog. Use 2s, and in all three of
  copy_clipboard_items / copy_items / move_items pause the timer while
  a conflict dialog (or the own-location / trash-deny alerts) is
  waiting for input, re-arming it after. A window that opens
  mid-operation now shows the current file instead of a stuck
  'Preparing...', and the trash-deny bail no longer leaks a timer that
  opened an orphan window after the operation ended.
- LegacyFSController: /copy and /move dropped the legacy 'overwritten'
  response field and never emitted item.removed for the entry an
  overwrite deleted, so clients kept a ghost row until re-listing.
  Resolve the entry before the operation, return it, and emit
  item.removed on success.
- helpers.js: copy_items read resp[0].overwritten but removed
  resp.overwritten (always undefined), and the data-uid cleanup
  selectors were unquoted — invalid CSS when a UUID starts with a
  digit. Fixed all three sites.

* test: pin the v1 overwrite/collision wire contract for move and copy

The collision tests asserted only statusCode 409, which is how the
item_with_same_name_exists code regressed to 'conflict' unnoticed and
broke every replace/skip prompt in the GUI. Assert the legacy code and
entry_name explicitly, and add controller tests for the overwrite path:
the replaced entry must ride along as 'overwritten' in the /copy and
/move responses and be announced via outer.gui.item.removed so clients
drop its row.
2026-08-02 22:17:10 -07:00
Nariman Jelveh 1a9f025db6 fix: surface name collisions when pasting instead of failing silently (#3492)
Cut+paste onto a folder containing an item with the same name failed
with no feedback: the dashboard's moveClipboardItems swallowed the
error into console.error and cleared the clipboard, and the rewritten
backend broke the v1 wire contract the GUI's conflict prompts key on.

- FSService: move() and copy() collisions again return
  item_with_same_name_exists + entry_name (the contract the write path
  already preserves) instead of a generic 'conflict' code. This also
  restores the desktop's existing Replace/Skip dialogs, which were
  silently broken against the new backend.
- Dashboard: moveClipboardItems now mirrors the desktop's move_items
  conflict flow (Replace / Replace all / Skip / Cancel, retry with
  overwrite) and surfaces other move errors in an alert.
- keyboard.js: the desktop's global Ctrl+V handler threw an uncaught
  TypeError on every paste in dashboard mode (its file container has
  no data-path); guard it — which also prevents a double-paste if
  dashboard containers ever gain one.
2026-08-02 21:33:01 -07:00
jelveh a4667073f5 Update RecommendedAppsService.ts 2026-08-02 20:55:33 -07:00
jelveh 23d251e482 fix: dashboard files list blanking during same-directory refresh
Re-rendering the directory already on screen (after an upload, sort
change, undo, etc.) used to clear the list and show a spinner before
the readdir round-trip, blanking the pane for the whole fetch. Keep
the current rows visible until the fresh listing arrives, then swap
the DOM in one pass and restore the scroll position. Navigation to a
different directory still clears immediately.
2026-08-02 20:30:20 -07:00
jelveh 0c9dfb03e2 Show action bar only in mobile select mode 2026-08-02 19:49:14 -07:00
Nariman Jelveh 6dc928e0c3 fix: file picker dialogs going behind parent window when dragged by titlebar 2026-08-02 17:38:25 -07:00
Daniel Salazar 84582ee33d fix: alarm channels (#3491) 2026-08-01 15:55:06 -07:00
Daniel Salazar 116d6e6663 tests: big test push for better coverage (#3490) 2026-08-01 14:29:59 -07:00
Daniel Salazar 7d0d44aef9 feat: slack alarms (#3489) 2026-08-01 13:12:41 -07:00
Daniel Salazar 3ac6c8532e fix: open ai cost (#3488) 2026-08-01 12:34:36 -07:00
Nariman Jelveh f6a50c4fb4 Update recommended apps list order
Reorders and updates the recommended apps list: removes 'butler', 'code', and 'traffic-tap-puzzle'; adds 'contacts', 'diagram', and 'basketball-tap' (relocated from end); adjusts overall ordering of several apps.
2026-08-01 12:26:44 -07:00
Daniel Salazar c8113d7514 fix: auth message popups (#3487)
* fix: auth message popups

* remove v1 auth
2026-08-01 10:45:16 -07:00
jelveh 6159bc9ae1 Polish dashboard settings button styling
Update `.dashboard-settings-card .button` to use dashboard theme tokens for text, background, and borders, and add clearer hover/focus/active states with subtle elevation and transitions. Also override disabled styles in this context so dark mode no longer shows hard-coded light greys from the base `.button:disabled` rule.
2026-08-01 00:27:27 -07:00
Daniel Salazar 3a11d0d1f1 feat: sandboxed app workers (#3486) 2026-08-01 00:02:06 -07:00
jelveh 535cf5753e Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-31 14:14:11 -07:00
jelveh cee4a6ccce Improve app reorder mode on touch devices 2026-07-31 14:14:04 -07:00
Daniel Salazar 1f1f95c2f8 fix: auth me for local dev (#3484) 2026-07-31 14:04:59 -07:00
jelveh f600e857b0 GUI: touch reorder mode for the dashboard Apps tab
Long-press-to-drag can't be made reliable on touch: touch-action is
consulted at gesture start, so the pager's pan-x claims the finger
before a drag can begin (worst on iOS). Replace it with an explicit
edit mode - a cog button (touch-primary devices only) enters it, tiles
jiggle and drag on first movement, iOS-style x badges uninstall, and
Done exits. Drops still persist immediately, same as desktop.

Also raise the drag ghost above the window z-index bands; it was
rendering invisibly behind the fullpage dashboard window.
2026-07-31 13:46:17 -07:00
Neal ShahandDaniel Salazar 08d1708378 change cors auth path (#3464)
* change cors auth path

* undo oidc ref changes

* scary OIDC state changes

* fix: bad cors signin

* puterjs changes

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-31 01:55:01 -04:00
Nariman Jelveh f1bc065730 GUI: fix dev-server port-retry crash under Express 5 (#3482)
Express 5's app.listen wraps the listen callback in once() and also
invokes it on 'error', with the error as the first argument — at which
point server.address() is null, so the startup log threw a TypeError
and killed the process before the EADDRINUSE handler could try the
next port. Bail out of the callback when it receives an error and let
the 'error' listener own the retry.

Verified: with 4000 occupied the server now logs the retry and comes
up on 4001; with the port free it binds 4000 as before.
2026-07-30 18:11:12 -07:00
Nariman Jelveh b55782f066 GUI: add npm start --server=<domain> to run the GUI against a remote backend (#3481)
- npm start --server=puter.com (or -- --server=...) skips the local backend
  and serves the bundled GUI locally, pointed at the remote server's API.
  Bare domains resolve to https://api.<domain>; full origins are used
  verbatim. gui_origin points at the remote origin so /whoarewe, login,
  anti-csrf, socket.io, and builtin apps hit the real backend (CORS-open).
- --extensions=<dir>[;<dir>...] bundles out-of-tree GUI extension
  directories (sugar for PUTER_GUI_EXTENSION_PATHS). Their imports resolve
  as if the files lived in src/gui/src/extensions, with the extension's own
  files taking precedence, and bare imports fall back to the repo-root
  node_modules.
- Fix the bit-rotted dev-server: Express 5 wildcard routes, pass gui()
  params (previously called with none), inject the service_script shim,
  load bundle.min.js + bundle.min.css in prod mode, serve /sdk, and
  properly await the webpack build (it previously resolved immediately).
2026-07-30 16:29:54 -07:00
jelvehandClaude Fable 5 2c9ae3489e Suggested apps: rank registered apps above the editor fallback
For extensions with no intentional built-in mapping (doc, docx, and
every other unmapped type), suggestionsForExtension fell back to
['editor'], and #resolveForExtension always placed built-ins ahead of
apps from app_filetype_association. Since suggested[0] drives the GUI's
double-click open path and /open_item, a .docx defaulted to opening as
plain text in editor even when a word processor explicitly registered
the extension.

Tag the unknown-extension result as a fallback and order third-party
filetype-association apps ahead of it. Intentional mappings (code, txt,
md, images, pdf, media) keep built-ins in the head slot as before, and
the editor guess still appears as a last-resort option.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-30 13:51:01 -07:00
Daniel Salazar b28b41c90b otel: better telemetry numbers (#3480) 2026-07-30 13:23:16 -07:00
Nariman Jelveh f34c4dc335 Apps: normalize filetype associations to bare lowercase extensions (#3479)
* Apps: normalize filetype associations to bare lowercase extensions

Suggested-apps lookups match app_filetype_association rows against the
bare lowercase extension ('docx'), but writes stored whatever the
developer typed. Rows like '.docx' never matched, so those apps
silently dropped out of Open With suggestions.

AppStore now canonicalizes on write (trim, lowercase, strip leading
dots, dedupe, drop empties) and tolerates the dotted legacy form on
read: getAppsByFiletype normalizes the requested extension, matches
both 'docx' and '.docx', and dedupes apps associated under both forms.
Cache invalidation keys are normalized the same way. Existing dotted
rows work without a data migration.

* Update apps tests for extension canonicalization

Adjust apps API tests to match current normalization behavior for `filetypeAssociations`: extension values are stored as lowercase bare extensions (e.g. `.txt` -> `txt`), while MIME types remain unchanged. Added inline comments in both test suites to document this expected remap.
2026-07-30 13:13:14 -07:00
Daniel Salazar 5a157197b6 fix: PUT-1398 (#3478) 2026-07-30 01:42:40 -07:00
Daniel Salazar 8a711e0254 driver controller change (#3477)
* fix: launch app

* driver controller change
2026-07-30 00:41:29 -07:00
jelveh a22321daa2 Dashboard: closing a launched app goes home, not into its launcher
An app launched by another app minimizes its launcher (iOS-style
takeover). Closing the child then popped onto the launcher's history
entry, and the popstate handler restored it — so quitting an app threw
the user into a different full-screen app they had not asked for, zoom
animation and all. Closing an app should go home.

Mark the launcher when it is minimized FOR a child
(data-minimized_for_child). If it still carries that mark when the child
closes, the close's single history hop lands on the dashboard instead:
the entry is rewritten to the dashboard's route and the launcher is left
minimized, still running behind its tile's running dot. No extra history
traversal — the joint session history an app's iframe shares is exactly
what pop_dashboard_app_url's watchdog exists to survive, so the hop
count is unchanged.

Back is untouched and still returns to the launcher — that is the
navigation gesture, whereas close dismisses. Any restore (Back, tile
click, Forward) clears the mark, so quitting a child after the user has
brought the launcher back leaves the launcher alone.
2026-07-29 22:04:10 -07:00
jelveh fbf1bd18b1 Dashboard: an app launched by another app takes over the tab
In dashboard mode apps are full-tab experiences, but an app launched by
another app (puter.ui.launchApp) opened as a floating titlebar window
over its full-tab parent: the parent stayed on screen around its edges,
couldn't be raised (focusWindow never raises stay_on_top windows), and a
child with no Apps-tab tile had no switcher to come back to once
minimized.

Give child launches the same treatment as tile launches — maximized, so
they get the headless chrome and control drawer — and minimize the
parent behind them, iOS-style. State only: the parent's /app/<name>
history entry already sits beneath the child's, so Back from the child
(and the child's close, which consumes its own entry) lands on the
parent's entry and the existing popstate handler restores it. The parent
keeps running while hidden, so parent/child IPC is unaffected.

Explorer keeps its windowed form, background apps don't minimize their
parent, and an explicit `maximized` option still wins. Desktop mode is
untouched: both changes are gated on is_dashboard_mode, so an app
launching an app there still gets a floating child over a visible
parent.
2026-07-29 21:36:47 -07:00
jelveh dc371fc4eb Windows: keep dialogs and their apps in the stay-on-top z band
focusWindow re-raised a focused window's parent (and children) with the
bare z counter. For a file dialog owned by a fullpage/dashboard app that
DEMOTED the app out of the 99999999+ stay-on-top band it was created in,
burying app and dialog under every other open app window — opening a
file picker from an app stacked over another (e.g. an app launched from
Dev Center) made both vanish beneath the window below. The dialog itself
had the same flaw: raised into the plain counter band, under every app.

Raise each window within its own stacking band instead: stay-on-top
windows (and windows hanging off one through parent_uuid, walked up the
chain) get 99999999 + counter; everything else keeps the bare counter —
so desktop stacking arithmetic is unchanged. Same demotion family as the
showWindow restore fix in #3427.
2026-07-29 21:11:21 -07:00
jelveh 97e4e78df4 Dashboard files: drop the root crumb from the breadcrumb
Every path rendered as "> Puter > user > ...", but the root crumb is
noise on anything beneath it. Render it only at the root itself, where
it's all there is to show (the Up button can still reach /, and an
empty breadcrumb bar there would strand the user). Each crumb keeps
its leading caret, including the first.
2026-07-29 20:12:22 -07:00
jelveh 007649296b Keyboard: stop swallowing Enter when nothing claims it
The global Enter handler preventDefault'ed unconditionally, then
returned without acting unless a launch-menu item, context-menu item,
or selected item existed. That suppressed the browser's native Enter
activation on whatever was focused — a focused link or button (e.g. a
dashboard sidebar item, a dialog button) did nothing on Enter.

Move preventDefault/stopPropagation into the branches that actually
handle the key; otherwise fall through so native activation runs. The
handled paths are unchanged: Enter still opens the selected item,
launch-menu entry, and context-menu entry.
2026-07-29 19:40:16 -07:00
jelveh 9392aed402 Dashboard: restyle keyboard focus ring on sidebar controls
Sidebar items (and the user/collapse/close buttons) had no
:focus-visible style, so keyboard focus painted the UA's default blue
ring — hard-edged and bleeding outside the item. Use the dashboard's
own selection ring instead, tucked inside the control so it hugs the
rounded corners. :focus-visible only, so mouse clicks never paint it.
2026-07-29 19:40:06 -07:00
jelveh 32e417fe79 Dashboard files: clicking an open file returns to its window
In dashboard mode there is no taskbar, so a minimized app window's only
switcher was its Apps-tab tile — invisible from the Files tab. Opening a
file, minimizing, and clicking the file again launched a second instance
of the app, stranding the first (and any unsaved edits) somewhere
unreachable.

Make the file row itself the switcher:

- launch_app stamps the opened file's uid on the window (data-file_uid;
  the signature's uid wins so shortcuts resolve to their target)
- open_item (dashboard mode) restores/focuses an existing window that
  has the file open instead of launching a duplicate, mirroring the
  Apps-tab tile's single-instance behavior — keyed by file, not app, so
  opening a different file still gets its own instance
- re-clicks while a launch's fetches are still in flight are swallowed
  (same idea as TabApps._launchingApps, keyed by file uid, TTL'd so a
  failed launch can't swallow clicks forever)
- dashboard file opens now default to maximized, so they get the same
  headless full-tab chrome + control drawer as tile launches instead of
  a floating titlebar window
- rows show a dot while their file is open in a (possibly minimized)
  window — under the name in grid view, inline after it in list view —
  driven by the existing dashboard-app-windows-changed event

Desktop mode is untouched: the reuse branch and maximized default are
gated on is_dashboard_mode, and opening the same file twice there still
creates two windows as before.
2026-07-29 19:03:33 -07:00
Daniel Salazar 0e1f617401 fix: errors for bad keys (#3475) 2026-07-29 15:58:53 -07:00
Felix-Ayush bd6efaaa4e fix(puter-js): ignore peer offers for unknown connections (#3473)
createAnswer ran outside the connection null-check, so stale or unknown
offer ids threw on the signalling socket. Guard the whole offer path and
return the #message promise from onmessage so callers can observe errors.
2026-07-29 15:58:39 -07:00
Nariman Jelveh 2b6dc528ce Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-29 15:48:21 -07:00
Nariman Jelveh 9368ad57b6 Preserve UI state across tab visibility refresh 2026-07-29 15:47:59 -07:00
Felix-Ayush e97d338f3a fix(ai-chat): preserve max_tokens 0 for Alibaba, DeepSeek, and Claude (#3472)
These providers used `max_tokens || default`, so an explicit zero was
replaced by the fallback. Use nullish coalescing so 0 is forwarded.
2026-07-29 18:38:31 -04:00
Nariman Jelveh a87a517b82 Anchor file row menu to button edge
File row overflow menus in the dashboard now open anchored to the ⋯ button (below it with right-edge alignment) instead of pointer position, improving placement consistency. UIContextMenu now supports `position.right` for right-edge pinning, and the ⋯ trigger keeps an active visual state while its menu is open so hover styling doesn’t drop when the cursor moves onto the menu.
2026-07-29 15:25:51 -07:00
Nariman Jelveh dbb52171c8 Refine dashboard file row hover styling 2026-07-29 15:05:21 -07:00
Daniel Salazar 6c020318f9 perf: traces and perf improvements (#3471) 2026-07-29 14:35:15 -07:00
Nariman Jelveh e16994db3b Fix I-beam cursor on dashboard file rows in Safari
Safari ignores user-select: none for cursor styling, showing a text I-beam over file rows. Setting cursor: default explicitly fixes this and is inherited by child elements.
2026-07-29 14:18:50 -07:00
Daniel Salazar 8491a0b55d fix: limit api pointing to apps only (#3470) 2026-07-29 14:03:27 -07:00
Felix-Ayush 710e4da4c0 fix(puter-js): size img2txt data URIs by decoded bytes (#3457)
Match speech2txt/speech2speech and reject on decoded payload size, not
data-URI string length. Base64 expands ~4/3, so string-length checks
falsely rejected valid images under the 10MB limit.
2026-07-29 12:21:21 -07:00
Felix-Ayush fd1f4e2ac2 fix(puter-js): forward temperature 0 and max_tokens 0 in ai.chat (#3455)
Truthy checks dropped legitimate zero values, so deterministic sampling
never reached the driver. Match the stream option and use !== undefined.
2026-07-29 12:21:15 -07:00
Kobi Hikri 70bf03a6e7 ci: attach provenance and SBOM attestations to the published image (#3463) 2026-07-29 12:21:02 -07:00
Felix-Ayush 6f90d7a199 fix(ai-chat): forward temperature 0 and max_tokens 0 to providers (#3456)
Completions-family providers used truthy spreads, so temperature: 0 and
max_tokens: 0 were stripped before the SDK call. Align with Responses
providers that already use !== undefined.
2026-07-29 11:49:36 -07:00
Daniel Salazar f0b1947c54 feat: make uninstall also revoke + dashboard css changes (#3469)
* feat: make uninstall also revoke

* fix: wording for free trial
2026-07-29 10:40:43 -07:00
Daniel Salazar 92193ff11f chore: kill collector (#3466) 2026-07-28 19:02:49 -07:00
Daniel Salazar f9396b7283 fix: thumbnail res (#3465) 2026-07-28 17:31:01 -07:00
Sauhbagya PrasadandDaniel Salazar 7cf3e03433 Replaced Nginx with Caddy (#3378)
* Replace nginx with Caddy

* updated caddy location

* seprate logic for local and with domain caddy setup

* updated install

* updated install scripts

new ps1 script with caddy, and removed instances of nginx

* fix Caddy Host routing, restore TLS/SELinux/healthcheck, update docs

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-28 14:52:13 -07:00
Neal Shah 0a61b4e78b add fixes 1 (#3458)
* add fixes 1

* change socketio test

* clarify that success/error callbacks are legacy

* fix wisp issues
2026-07-28 17:34:12 -04:00
Reynaldi Chernando 1db4324c02 remove deprecated ai models (#3444)
* remove deprecated ai models

* test

* fix tests
2026-07-28 12:41:04 -07:00
Daniel Salazar 69529ee0db perf: try to improve app opens speed (#3462)
* perf: try to improve app opens speed

* make /rao non blocking to allow faster conn swap
2026-07-28 11:12:44 -07:00
Daniel Salazar 7a7ff47442 chore: bump version (#3461) 2026-07-27 23:59:51 -07:00
Daniel Salazar 148771a82b fix: performance degradations in app fetching + puter-js module parent class (#3460)
* fix: centralize module methods in parent class

* fix: performance degradations in app fetching
2026-07-27 23:41:51 -07:00
Daniel Salazar 5bad6a972f fix: puter-js cleanup around docs and ai resolution (#3459)
* fix: puter-js cleanup around docs and ai resolution

* more jsdoc stuff

* fix: docs

* fix: doc types + misc hardening

* fix: harden auth
2026-07-27 22:31:05 -07:00
Luke Oliff 8bb7d59527 feat: add Speechify TTS driver (puter-tts) (#3453)
* feat: add Speechify TTS driver

Adds a SpeechifyTTSProvider under the ai-tts driver (mirrors the xAI/
ElevenLabs REST-provider shape), registered in TTSDriver alongside the
existing providers. Wires puter-js txt2speech with provider: "speechify"
support, default voice/model, and the speechify-tts driver alias.

Every outbound request sets Speechify-Caller: puter; base URL is
https://api.speechify.ai only; default model is simba-3.2.

* fix: replace placeholder voice IDs with real simba-3.2/simba-english voices

DEFAULT_VOICE and the starter voice catalog used invalid IDs (henry,
cliff, kristy, george, aria) that the real API rejects with 400. Swapped
in confirmed-real voices from a live GET /v1/voices call: geffen_32,
dominic_32, harper_32, hugh_32, imogen_32, and alec for the
simba-english model-override test.

* fix: purge remaining placeholder voice IDs from client, docs, and types

The previous fix only covered the backend provider — the client-side
default in tts.js, its test, the txt2speech docs page, and the ai.d.ts
type comment all still referenced the invalid henry/cliff/kristy/george/
aria set. All replaced with the live-verified voices (geffen_32 default).
Also corrects the docs link to docs.speechify.ai.
2026-07-27 17:04:11 -07:00
Neal Shah 8732494442 dead code cleanup (#3441) 2026-07-27 17:51:15 -04:00
Daniel Salazar cade75ac9b fix: PUT-1385 PUT-1382 PUT-1381 PUT-1380 PUT-1379 (#3454) 2026-07-27 11:42:40 -07:00
meridah7 9abc9d1a19 feat: add Infron AI chat provider (#3433)
Adds Infron (https://infron.ai) as an aggregator provider for the
puter-chat-completion driver, following the OpenRouter provider as
reference per doc/contributing-apis.md.

- OpenAI-compatible gateway at llm.onerouter.pro/v1
- Dynamic model catalog (kv-cached 15 min, authenticated fetch),
  filtered to chat-capable LLM entries
- Bills the gateway-reported authoritative cost when present
  (usage: {include: true}); falls back to catalog per-token pricing
- Registered as an aggregator so its aliases never shadow
  first-party providers
- Offline unit tests (mocked SDK/axios against a real test server)
  plus an env-gated integration test
2026-07-27 08:38:04 -07:00
Nariman Jelveh f3fd8a30da Rework permission requests: new dialog, working popup flow for websites (#3447)
* Rework permission requests: new dialog, working popup flow for websites

- Replace the UIWindow-based permission prompt with a standalone top-layer
  <dialog> (responsive, light/dark, app/site identity, input protection)
- Implement puter.ui.requestPermission for env=web: opens the GUI's
  /action/request-permission popup with pinned origin/source/msg_id,
  popup-closed detection, and a check-permissions polling fallback for
  crossOriginIsolated openers
- Move the GUI's request-permission action into postAuthActions so
  signed-out users sign in first; identify the app by opener origin,
  correlate responses with original_msg_id, close the popup after answering
- Always respond from the IPC handler so the SDK promise can't hang;
  normalize the result to a strict boolean (a failed grant no longer
  resolves truthy)
- Accept origin in /auth/grant-user-app and /auth/revoke-user-app,
  mirroring grant-dev-app (fixes puter.perms.grantOrigin/revokeOrigin)
- Add Playwright e2e coverage for both the desktop and popup flows;
  update e2e harness for the auth_token_v2 localStorage key
- Update types and docs (perms request methods now work on websites)

* Harden permission request flows

- Grace period before treating popup close as denial: the GUI posts the
  decision then closes the popup, and postMessage delivery is not ordered
  relative to `closed` becoming true, so a grant could race to a false
- Unique popup window name per request so window.open name-reuse can't
  hijack a still-pending request's popup
- request-permission action always answers the requester and closes the
  popup, even when app resolution or the dialog throws
- Permission dialog: refuse unidentifiable requesters, allowlist icon URL
  schemes, and time out the grant request into the retryable error path
- Validate app_uid/origin/permission types and length in grant-user-app
  and revoke-user-app
- Tests: revoke-by-origin and input-validation backend tests; e2e for
  dialog dedup, unsupported permissions, and the consent-dialog path

* Sign in first in the requestPermission fixture's email flow on the web

In env=web the site has no auth token, so whoami threw 401 immediately
and the email button appeared to do nothing. Sign in via popup first,
matching the real third-party flow; env=app already has a token and is
unaffected.

* Point the requestPermission fixture at the real api subdomain

The SDK sends credentialed CORS requests; the GUI host doesn't answer
with Access-Control-Allow-Credentials, so whoami (and any authed call)
from the fixture origin failed CORS and looped through retries.

* Fix /auth/list-permissions schema mismatches

The endpoint's queries referenced columns that don't exist:
user_to_app_permissions stores a numeric app_id FK (not app_uid), and
user_to_user_permissions uses holder_user_id (not target_user_id) —
every call 500'd. Join apps to expose the app's uid and use the real
column names.

Replace the catch-either-branch test (which documented the breakage
instead of failing on it) with real assertions covering all three
sections of the response.

* Identify permission requesters by origin only

The request-permission action took `app_uid` straight from the query
string and used it as the grant target whenever the origin was absent or
unresolvable. Now that /auth/grant-user-app accepts `origin` and prefers
`app_uid` when both arrive, the displayed identity and the grant target
could diverge; with only `app_uid` in the URL the dialog rendered with an
empty name, so a link could produce a bare "Allow" prompt for an unnamed
requester. Resolve the uid from the origin alone, and let the server
resolve it from that same origin when the client lookup fails.

Also give the no-gesture consent popup a unique window name. UI.js does
this on the direct path because window.open() reuses a window with a
matching name, but the PuterDialog fallback opened under the default
'Puter' — the same name sign-in uses, so a consent click could navigate
an in-progress sign-in popup away.

And guard the IPC responder: an app that closes its own window while the
dialog is up leaves target_iframe.contentWindow null.

* Keep the permission popup from signing the site in

The popup loads the GUI with embedded_in_popup=true, so it ran the
sign-in token exchange and posted puter.token to the opener before the
user answered the prompt. A site that called requestPermission() walked
away holding a user-app token for the account even when the user pressed
"Don't Allow" — and because the SDK's global puter.token handler feeds
event.data.token into setAuthToken() without looking at `success`, a
failed exchange posted token: null and wiped a token the site already
had. Keep running the exchange (it bootstraps the app row the grant
needs and caches host_app_uid) but leave the token in the popup. A site
that wants credentials still has to call signIn().

Escape on the SDK's consent dialog left the caller pending forever.
PuterDialog wired its Cancel and close buttons but not the <dialog>'s
native cancel event, so the browser dismissed the dialog and nothing
reported it: no dialog, no popup, no answer. Route cancel to the same
handler. Programmatic close() fires only `close`, so launching the popup
— which closes this dialog — is unaffected, and the implicit-auth flow
stops hanging on Escape too.

Serialize the permission dialogs. showModal() makes the whole document
inert rather than just the requesting app's window (which is what the
UIWindow it replaced did), and the dedup map only coalesced identical
requests, so an app asking for permissions in a loop stacked one modal
per request and walled the user off from the desktop — including from
the app doing it. Prompts now queue and open one at a time, and each
caller still gets its own decision.

Identify apps by more than their title. `title` is free-form text the
author picks and is not unique, so it was the whole identity of a prompt
an app titled "Puter Settings" could raise; the registered `name` is
unique and format-restricted, so show it underneath. Give the name line
the unicode-bidi isolation the origin line already had, since escaping
leaves bidi overrides intact.

Stop the dialog from answering over its own in-flight grant: a dismissal
while the POST was outstanding resolved false for a permission the server
was committing. Ignore dismissals while granting, and time-box the
request with AbortController (AbortSignal.timeout isn't everywhere) so a
hung network can't leave a modal no one can close. Fail closed on the
remaining paths that could reject or prompt uselessly — showModal()
throwing under <iframe sandbox>, and a requester known only by app_name,
whose Allow the server would always reject. Pass the error string to
.text() unencoded so translations containing an apostrophe don't render
&#39;.

The e2e suite covers all of it; each new test fails without its fix.

* Restrict the permission popup flow to third-party websites

requestPermission's new web path ran in every environment with a window,
including env='gui' — so a permission_denied driver retry inside the
Puter GUI would open a popup to the Puter origin from the desktop itself
and try to grant the permission to a phantom app for Puter's own origin.
Resolve false everywhere except env='web', the previous behavior.

* Settle the permission dialog when a failed grant has no dialog left

The cancel handler is preventDefault'd, but close requests can't be
suppressed forever: Chrome's close watcher lets a repeated Esc skip
cancel and force-close the dialog while the grant POST is in flight.
The close handler defers to that grant on purpose — but if the grant
then failed, fail_grant re-enabled buttons on a closed dialog and the
promise never settled, leaving the requesting app waiting forever.
Settle as a denial when the dialog is no longer open.

Also add regression tests for this and for the popup-flow env guard
(routing the CDN SDK URL to the local build, since the prod-built GUI
loads its SDK from js.puter.com).

* Withhold the auth token on the popup's first-visit paths

Keeping the token inside the permission popup only covered the plain
token exchange. Two other popup paths mint a user-app token and posted
it to the opener unconditionally: first-visit temp-user creation, and
the manual signup shown when temp users are refused. Both sit on the
path a brand-new visitor takes — the audience the website popup flow
exists for — so a site that asked about one permission and was denied
still walked away holding a token, for a temp account or a real one.
The SDK's global puter.token handler feeds whatever arrives straight
into setAuthToken(), so posting it is the whole of it.

Move the rule into util/popupAuth.js and consult it at every site that
posts the token, so the next token path has one place to ask.

The first-visit path also left the prompt itself unreachable: it waits
on the spinner promise, which only resolves when the spinner was up for
under 2s. End that wait for any action that keeps the popup open;
sign-in still closes the window as before.

The e2e test fails without the fix — the site holds a token after the
user presses "Don't Allow".

* Poll for the decision when the popup's opener is severed

crossOriginIsolated was the test for "the popup can't message us back",
but being isolated also requires COEP. A site sending COOP: same-origin
on its own still has its opener relationship severed when it opens the
Puter popup, and took the watch-the-window path instead — where the
detached proxy reports closed === true on the first tick, so
requestPermission resolved false about a second after the popup opened,
while the user was still reading the dialog. Their "Allow" then had
nowhere to go. Treat an already-closed popup as severed and poll.

Pin the expected event.source before those early returns. popupWindow
was assigned after them, so for the whole consent-dialog wait — as long
as the user takes to click Continue — the handler accepted a decision
from any window on the GUI origin. A forged answer is only advisory
since the grant is written server-side, but the check may as well hold.

Settle instead of rejecting when the consent dialog can't be appended:
document.body is null in a <head> script, and the throw both rejected a
promise documented to resolve to a boolean and left the message listener
behind.

* Key the dialog dedup by the identity its gate accepts

The gate treats an empty app_uid as absent and falls through to the
origin; the dedup key used ?? and kept the empty string, so two requests
from different origins would collide on one key and share a single
decision. No caller can produce a blank uid today — server uids are
never empty and the IPC path's empty attribute is stopped by the gate —
but the two lines have to agree.

* Close the gaps the permission-request flow left open

Seven defects found reviewing the new permission flow end to end, each
reproduced against a running server before being fixed.

Security:

- `cross_origin_isolated=true` bypassed `deliversTokenToOpener` entirely.
  That branch is checked first, mints a user-app token, publishes it via
  `/login/set` and returns — so one query parameter on a
  request-permission URL skipped the prompt and handed the opener a token
  through the unauthenticated `/login/wait`. Gate it with the same rule.

- Grant/revoke by `origin` could land on an unrelated app. An origin with
  no app row synthesises `app-<uuidv5>`, and the permission services
  resolve their identifier as uid *or name* — and the uuid namespace is a
  source constant, so the string is computable offline and registrable as
  an app name. Resolve origins to a uid that names a real app row.

- A website's host was elided on the right, hiding the registrable domain
  that says who is asking. Elide it from the left, as the sibling rule
  already intended.

- A grant whose response was lost (client-side abort, dropped reply) left
  the row committed while the dialog reported a denial. Withdraw it when
  the user then answers "Don't Allow".

Correctness:

- `pollDecision` needs the site's own token, which a permission popup
  deliberately never delivers, so a signed-out cross-origin-isolated site
  burned the full five-minute timeout before answering. Answer at once
  when there is nothing to poll with.

- `getUserAppToken` reports failure by returning null, and three callers
  read `.app_uid` off it. Guard all three, keep the first-visit spinner
  promise settling on its failure paths, and dispatch the `login` event on
  the manual-signup path so `postAuthActions` runs at all — a user who
  signed up inside a permission popup got a blank window and the site got
  no answer.

- Time-box the lookups that run while a request holds the dialog queue's
  slot: they have no timeout of their own, and a stall (not a failure)
  wedged every later permission request in the page.

Also harden the grant/revoke input validation the PR introduced — it
skipped `extra`/`meta`, so a non-object faulted *after* the row was
written, and its length cap was 16x the column it lands in — stop a
non-URL `origin` from throwing past the answer-and-close, and drop the CSS
left behind by the deleted dialog.

* Close three gaps left in the permission-request flow

Each was reproduced first — the squatting grant against a running server,
the COOP timing in a real browser — and each fix was then confirmed by
reverting it and watching the new test fail.

Security: the dialog could name one site and grant to another.

The squatter guard added for grant/revoke by `origin` only ran when
`app_uid` was absent, and the dialog sends both — so `app_uid` won and the
guard never applied. `getAppUIDFromOrigin` returns the synthetic
`app-<uuidv5(origin)>` for any origin with no app row of its own, and the
grant endpoint resolves `app_uid` as uid *or name*, so the grant landed on
whoever registered an app under that computed name (the format allows it,
and the namespace is a source constant). A link like
`/action/request-permission?origin=https://a-site-you-trust.example` named
that site in the prompt while "Allow" handed the permission elsewhere.

Fixed on both sides of the wire. The action now sends the origin alone —
no uid resolved in the browser is safe to forward, whatever its source —
and a supplied `origin` now decides the target on the server even when an
`app_uid` travels beside it: the origin is what the prompt showed the
user, so it is what the grant has to follow.

Correctness: a COOP-only site was answered before the user decided.

7efc0c0b routed a severed opener to `pollDecision` by treating an
already-closed popup as severed, but that reads `closed` synchronously
after `window.open()` — before the navigation whose response headers cause
the severing has committed. Measured in Chromium: `closed` is false at
0ms and true by 200ms. So a site sending COOP: same-origin without COEP
still took the watch-the-window path, and `requestPermission` resolved
false 1.1s after the click while the dialog was still on screen. The
"Allow" that followed committed a grant the site had been told it did not
get. Tell the two apart by when the close lands, and keep the in-flight
message's grace period on both branches — an answer already on its way
outranks whatever the close is taken to mean.

Correctness: a grant that timed out was not withdrawn.

`grant_may_have_committed` was only set in the fetch's `catch`, which
cannot run until the timeout's timer callback returns — and that callback
already calls `fail_grant`, which settles the dialog as a denial outright
when the dialog was force-closed mid-grant. The reconciliation was
skipped in exactly the case it exists for. Record the unknown outcome in
the timer, where the timeout already means the request left the browser.

Also require a `token` from the user-app exchange rather than just a
non-null body: an HTTP failure (a blocked origin, a 5xx) returns the
parsed *error* body, which is truthy, so the guard added for this missed
it — handing the opener an `undefined` token, and prompting for a grant
whose app row was never bootstrapped.

Known limitation, now more reachable: a severed opener cannot signal a
denial at all, since nothing is written for one, so those sites wait out
the poll timeout before receiving false. A grant still resolves promptly.

* Deliver the uncertain-grant withdrawal from a closing popup

The permission dialog reconciles a denial after an uncertain grant by
firing a revoke in the background, but the popup flow posts the answer
and closes the window right after settling — and a plain fetch is
cancelled with its document, so the withdrawal never reached the server
and the user was told "denied" while the grant stayed live. Send it
with keepalive so the browser delivers it independently of the popup,
and cover the popup flow with a regression test (the existing
withdrawal tests only exercise the desktop flow, where the GUI
outlives the dialog).

Also make the popup boot's getAppUIDFromOrigin guard functional: the
helper reports failure by resolving to a null/undefined uid, not by
throwing, so the catch never engaged and a failed lookup clobbered
window.host_app_uid with undefined despite the comment claiming the
token exchange's value was kept.

* Keep the ai-chat model-map build inside the server lifecycle

onServerStart fired #buildModelMap without awaiting or tracking it, so
the network fetches it does (notably Ollama auto-discovery, which is
enabled by default and doomed on any machine without a local Ollama)
kept running after server.shutdown() resolved. In vitest that let the
provider's console.error land during worker teardown, which surfaces as
"Closing rpc while onUserConsoleLog was pending" — the unhandled error
that intermittently fails CI (last seen attributed to
WispController.test.ts). A rejection in the detached chain would also
have been an unhandled rejection.

Track the promise, catch and log rejections, and await it from
onServerShutdown so no provider I/O or logging outlives the server.
Also disable Ollama auto-discovery in setupTestServer's defaults —
every test server was firing a pointless model-list fetch at localhost.

* Settle requestPermission on the launch paths that could still throw

Three gaps left by the permission-request rework, each verified against a
live stack before and after the fix.

The IPC handler normalises a non-object `options` so it can always reply,
but `typeof null === 'object'` let null through the guard; reading
`.permission` off it threw out of the message listener before any reply,
so `puter.ui.requestPermission(null)` hung forever in env=app while
env=web answered false for the same input.

In the env=web branch only the consent-dialog path was wrapped, even
though its own catch reasons that this resolves to a boolean for every
other caller. A `window.open` refused by throwing rather than by
returning null escaped the launch branch and rejected instead.

The perms docs were flipped to platforms: [websites, apps], but every
entry point except `request()` reads the signed-in user's identity
first, so on a signed-out site they reject with Unauthorized and never
prompt — the permission popup deliberately does not sign the site in.
Document the sign-in precondition on those pages.

* Measure a permission's width after the rewrite that decides it

The new grant validation capped `permission` at 255 to match the column
it lands in, but it measured the caller's raw string. `fs:/path:mode` is
rewritten to `fs:<uuid>:mode` before storage, so what lands in the column
is ~44 characters however deep the path is. Granting access to a deeply
nested file therefore returned 400 even though the identical target
granted by uuid returned 200 and stored 44 characters — and because a 4xx
is read as an outright refusal, the permission dialog showed its
retryable error and could never succeed on retry.

Bound the request body only against absurd input, and enforce the column
width in the permission service on the rewritten string, before the app
is resolved so an oversized permission still refuses ahead of a missing
app. Covered both ways: a rewritten-short permission is accepted, and one
that no rewriter shortens is still refused.

* Require a registered app when a dev-app grant names an origin

The user-app handlers resolve a caller-supplied origin through
#registeredAppUidFromOrigin because appUidFromOrigin synthesises
app-<uuidv5(origin)> for an origin with no app row, and the permission
services resolve their identifier as uid-or-name — so the synthetic uid,
derived from a published namespace constant and computable offline,
lands on whoever registered an app under that literal name. The dev-app
handlers were left resolving the raw synthetic uid.

That leg matters at least as much: a dev-app grant is scanned with the
issuer's authority for anyone running as that app, so a squatted grant
hands over the granting user's permission. Verified against the store —
the grant landed on the squatter rather than rejecting.

Without a squatter the synthetic uid resolves to nothing and these
already 404, so the guard costs the legitimate case nothing; a test
covers a registered origin still resolving to its app.

Also assert that revoke accepts the same oversized-but-rewritten
permission grant does, since the dialog's withdrawal of an uncertain
grant depends on that symmetry.

* Revoke the row a user-app grant actually wrote

`app-root-dir:<uid>:<mode>` is a pseudo-permission: its rewriter resolves
it to a real `fs:<root_uid>:<mode>` only while a user-app permission row
is being written, and resolves to a match-nothing sentinel at all other
times so a scan can't match through the fs path.

Revoke shared that rewrite but not the flag, so it aimed the DELETE at
the sentinel: it removed nothing and reported success while the fs
permission stayed live. The permission dialog withdraws a grant whose
outcome it couldn't confirm through exactly this path, so a user who
answered "Don't Allow" after a dropped grant response kept the access
they had just refused.

Grant and revoke now share one rewrite helper. It also has to work for a
caller outside a request scope — an internal job, or a direct unit test —
where `Context.set` has nothing to set the flag on; an empty scope reads
the same as no scope, so it only makes the flag settable.

* Elide a long host from the left, as its own rule intends

The identity line is the only thing on the permission dialog naming the
requester, so a host too long for the dialog has to lose its front, not
its tail: the registrable domain is the part that says who is asking.

`direction: rtl` was there for that, but paired with
`unicode-bidi: plaintext` it does nothing — plaintext takes the base
direction from the content's own first strong character, which for any
Latin host is LTR, so the ellipsis went back on the right. Measured in
the real dialog, `account-security.paypal.com.verify-login.example`
rendered as `account-security.paypal.com.verify-l…`, reading as PayPal.

Isolating instead keeps the box anchored to the end of the text, and
still stops a bidi control character in the host from reordering
anything around it.

The test that covers this asserted the computed `direction` — the
property, not the outcome — so it passed throughout. It now measures
which characters are actually on screen, and that the host still reads
in source order.

* Tell a severed opener from a closed one by whether it can answer

The web popup flow decided which it was looking at by timing: a
`popup.closed` flip within 3s of `window.open()` was COOP severing the
opener, anything later was the user closing the window. Both halves
misfire, and both were reproduced against a real popup.

A COOP-only site whose popup navigation commits after the cutoff had its
severing read as a close, so the site was told "denied" about a second
later — while the prompt was still coming up. The Allow the user went on
to click then committed a grant the site had been told it did not get,
which is the failure the cutoff was introduced to prevent: how long a
navigation takes says nothing about whether the opener survived it.

The other way round, a signed-in site whose user dismissed the popup on
sight had that close read as severing, and fell back to polling for a
decision. A denial writes nothing to poll for, so the caller waited out
the full five-minute timeout instead of being answered.

So ask the question directly: the popup now announces itself to its
opener, which it can only do while the relationship is intact. Having
heard from it proves a later close is a real close and the answer is now;
never hearing from it means the prompt may be live in a window that
cannot answer, and the decision is read back from the server as before.
The announcement carries no token, and goes out before any sign-in gate —
a gate that delayed it would make abandoning sign-in look severed.

Measured: the popup announces itself ~290ms after opening, and a close
just after that is answered in ~1.3s rather than five minutes.

* Stop revoking a literal `*` after a dev-app revoke-all

The `*` arm of /auth/revoke-dev-app fell through: after
`revokeDevAppAll` it also ran `revokeDevAppPermission(…, '*')`, a
DELETE naming a row called literally `*` — which matches nothing —
plus a second `revoke` audit entry for the same action. The user-app
twin already if/elses its two arms; the dev-app handler now matches it,
and a test pins the parity: everything revoked, one audit row.

* Match the popup's messages against a canonical GUI origin

The web flow compared `event.origin` to `puter.defaultGUIOrigin` as raw
strings, but they are different kinds of value: the event carries the
browser's canonical origin serialization, while the configured origin is
whatever text was supplied — a trailing slash, an explicit default port,
or a stray path all name the same origin and all fail the comparison.

The mismatch doesn't read as a config error, it reads as the user's
answer: with every message from the popup dropped, the missing
`permissionPromptReady` makes the popup's close look like a severed
opener, and the missing decision leaves that path to answer on its own —
"denied", for a guest, moments after the user clicked Allow and the
grant committed.

Parse the configured origin once and compare canonical-to-canonical; the
popup URL is built from the same parsed origin, so a trailing slash no
longer yields a `//action/...` path either. A configured origin that
cannot parse could never have hosted the prompt, so it now denies up
front instead of opening a broken window. Pinned by an e2e test that
re-points the SDK at the same GUI through a trailing-slash origin and
expects the grant to be heard; it fails against the raw comparison.

* Take a permission popup's requester from the browser, not the link

`app_uid` was removed from the request-permission URL because the uid
names who receives the grant, so it has to come from the requesting
origin rather than from whoever built the link. Two other parameters
still carried exactly that identity, and the origin is the identity
twice over: it is the name the dialog attributes the request to, and it
is what the server resolves into the app the grant is written against.

`opener_origin` is believed on every popup boot, ahead of the referrer.
`origin` was the fallback in the request-permission block itself,
reached whenever there is no opener at all. Either one lets a bare link
raise a consent prompt in some other app's name — the token exchange
bootstraps an app row for whatever origin was typed, so the grant then
commits against it — while the user is looking at a domain the requester
does not control. That is the whole dialog defeated: a site can name
`docs.google.com` and have the Allow land on Google's app row.

So a permission popup now takes only an origin the browser vouches for:
`document.referrer`, or the opener's own reply to the `requestOrigin`
handshake. Neither can be forged to another origin. Nothing legitimate
relied on the parameters — the SDK sends neither, and the OIDC redirect
`opener_origin` exists for drops `action` too, so no permission flow can
arrive through one. The `origin` fallback only ever fired when there was
no opener, which is to say when there was no requester either; that path
now reports its denial as usual instead of prompting.

The referrer is safe here precisely because `action` is only
`request-permission` on the popup's first load, so it is always the
opener's. Restoring the action across an OIDC hop would break that — the
returning navigation's referrer is the identity provider — which is why
the round trip is withheld rather than repaired:

A popup that signs in through OIDC comes back to a redirect URI the
server hard-codes to `/action/sign-in`, so it returns believing it is a
plain sign-in popup. It posts `puter.token`, which the SDK's global
listener feeds straight into `setAuthToken()`, and never runs the action
it was opened for. For a permission prompt that is the exact outcome
`deliversTokenToOpener` exists to prevent: the site is signed in without
ever asking, the user is never shown the permission they were brought
there to decide, and the request resolves as a denial. Nothing in the
returned URL says what the popup was for, so it cannot recover on its
own. Until the redirect can carry the action back — and the opener
origin can be re-established from the handshake on return — a popup
whose purpose cannot survive the hop does not offer the hop. Email
sign-in stays in the window and is unaffected.

Pinned by e2e tests that spoof each parameter and expect the prompt to
name the real opener, or not to appear at all. The long-hostname test
drove the dialog through `origin=`, which no longer produces one, so it
now asserts the CSS elision contract directly while the popup test
asserts the real flow applies the `perm-dialog-entity-host` class that
contract keys on.

* Refuse an origin the grant could never have named

The identity line elides a long host from the left, keeping the
registrable domain visible, which it does with `direction: rtl`. That is
sound for a host — every character in one resolves left-to-right, so the
string reads in source order — but it is not sound for arbitrary text,
whose neutral and RTL runs can render in an order they were not written
in. On the one line of this dialog whose whole job is saying who is
asking, that is the wrong thing to be lenient about.

The entity resolver reached that state through its own fallback: when
`new URL(origin)` threw it displayed the unparsed string and still marked
it as a host. An origin the server cannot parse cannot name a grant
target either — `AuthService#originFromUrl` rejects it, and rejects
non-http(s) schemes with it — so there was never anything to prompt
about. Deny at the gate, next to the existing "requester the grant can't
name" check, on the same test the server applies.

* Give the decision poll a deadline it can actually reach

`pollDecision` bounds itself with a five-minute budget, but it only reads
the clock between iterations and its `fetch` had no timeout of its own. A
request that never settles — a stalled connection, a proxy that accepts
and never answers — parks that `await` forever: the loop never comes
back round to check, `settle` is never called, and the caller's promise
stays pending for the life of the page with the message listener and
interval still attached. It is the only unconditional hang left in the
flow, and the least recoverable one, because the popup is already closed
on this branch and nothing the user does can rescue it. Each attempt now
gets ten seconds — long enough that a slow-but-working connection is
still heard, short enough that the deadline means something.

The request id changes for a related reason. It was `#messageID++`, a
small integer restarting at 1 on every page load, and `event.source` is
not pinned for as long as the no-gesture consent dialog waits for its
Continue click — a stretch of time the user paces. A permission popup
left open from before a reload posts this exact message shape to its
opener on the way out, and its counter value collides with a fresh
request's, settling it with the decision the user made about a different
permission. A random suffix makes the two impossible to confuse; the GUI
echoes the value back verbatim, which the loose comparison still handles.

* Answer the app when the signup gate throws

The requestPermission branch grew a `respond` helper so every exit tells
the app something and its promise settles instead of hanging. One exit
still doesn't: `await UIWindowSignup(...)` is outside the guarded region,
and `ipc_listener` has no outer catch, so a throw from the signup window
escapes the listener entirely — before any reply — and leaves the app
waiting forever. That is the precise failure the helper was added to rule
out. Treat it as the refusal it amounts to.

* Revert "Keep the ai-chat model-map build inside the server lifecycle"

This reverts commit 1b2482dcb8.
2026-07-27 08:37:41 -07:00
Daniel Salazar b2324cd212 chore: cleanup puter-js driver to match rest of changes (#3452) 2026-07-27 08:25:44 -07:00
jelveh a1c8ba4cdd Fix stuck grab cursor on aborted drag
When dragging is aborted by returning false from the sortable 'start' handler, the cursor plugin never fires its cleanup, leaving the body cursor stuck on 'grabbing'. Reset the body cursor explicitly on each early-return path.
2026-07-26 23:29:33 -07:00
jelveh 30461e7d95 Hide Trash from file explorer listing
Trash is accessible via the sidebar and should not appear as a row in the file explorer. Filters it out from directory listings and prevents socket events from re-adding it to the view.
2026-07-26 22:32:32 -07:00
jelveh 46253fb80a Disable hover during rubberband file select
Add a `rubberband-selecting` state on the dashboard while marquee selection is active in `TabFiles`, and remove it when drag ends. CSS now disables pointer events on surrounding files chrome (sidebar, directory tree, and header) during that state so hover highlights/tooltips don’t trigger when the cursor leaves the clamped selection area.
2026-07-26 22:23:54 -07:00
jelveh 29f2a4d9c9 Default file picker and sidebar to Home
Make Home consistently visible and selected in file navigation. The file tab now shows the Home directory entry and auto-selects Home (instead of Desktop) when no initial path is provided. In `UIWindow`, saved sidebar layouts are backfilled to include Home if older preferences are missing that entry, preventing users from losing quick access after upgrades.
2026-07-26 22:14:17 -07:00
jelveh e00e1e57da Centralize trash icon update logic
Extract duplicated trash icon update code into two shared helpers:

- `window.update_trash_icons(is_empty)` updates all trash icon surfaces (taskbar, desktop items, shortcuts, open window headers, Dashboard sidebar) in one place.
- `window.refresh_trash_state()` stats the trash, emits the socket event, and calls `update_trash_icons`.

Replace all scattered inline stat+emit+icon-update blocks across UIItem, UIWindow, UIDesktop, UIDashboard, keyboard, helpers, and generate_file_context_menu with calls to these helpers. Also syncs the Dashboard sidebar trash icon on load and after every destructive operation.
2026-07-26 21:51:07 -07:00
Nariman Jelveh 6668d778d2 feat: highlight uploaded items after upload
Uploads now land selected wherever they finish: the desktop, explorer
windows, file dialogs, and the dashboard Files tab. Selection is applied
from both completion channels — the batch response selects elements
already in the DOM, and items whose item.added socket event arrives
later are remembered (with a TTL) and selected on creation. The explorer
footer count and the open-file dialog's Open button are updated to
match, and disabled items (e.g. filtered out by a dialog's file-type
filter) are never selected.
2026-07-26 21:14:26 -07:00
Nariman Jelveh 7f3691aea3 Auth: land back on the /app/<name> the user came for after login/signup (#3435)
Landing on /app/<name> logged-out and authenticating used to dump the
user at the root dashboard on several paths, losing the app they came
for:

- OIDC login/signup only sent return_to for /desktop and /dashboard
  (and the backend whitelist only accepted those two), so OIDC from an
  app landing redirected to /.
- UIWindowSignup defaulted its post-success redirect to /, so password
  signup reached via the session list, ?action=signup, or in-app signup
  prompts (IPC.js) lost the app.
- OIDC error redirects always went to /?action=..., so a recovered
  attempt (e.g. account-not-found bounced to signup) also lost the app.

New helpers in src/gui/src/helpers/auth_redirect.js:
- get_auth_redirect_url(): stay on the page auth started from;
  /action/* pages go to /; strips action/auth_error/message/
  request_code so the reload doesn't re-open the auth window or pass
  auth params through to the app as launch args.
- get_oidc_return_to(): pathname when whitelistable, now including
  /app/<name> (trailing slash normalized).

Backend (OIDCController):
- Shared isWhitelistedReturnPath() accepts /desktop, /dashboard, and
  /app/<name> (charset mirrors APP_NAME_REGEX — no open redirect).
- buildErrorRedirectUrl() lands on the whitelisted originating page
  from the signed state's redirect_uri instead of always /.

Tested: 47/47 OIDC controller tests pass (4 new: return_to accepted/
rejected on start, success redirect to /app/<name>, error redirect
keeping /app/<name>); verified live on local dev for first-visit temp
user, password signup (incl. email-confirmation gate), and password
login — all land on /app/camera with the app open.
2026-07-26 20:33:46 -07:00
Nariman Jelveh 1e280601c6 fix: showSaveFilePicker from external websites (#3450)
Saving a file via puter.ui.showSaveFilePicker from a third-party website
(popup flow) consistently failed with a DataCloneError alert after
clicking Save, and saving over an existing filename showed a raw error
instead of the Replace/Cancel prompt.

Two bugs:

1. privacy_aware_path is a curried factory (world => fspath => ...), and
   initgui.js is the only module that imports it directly — so the popup
   save handler's privacy_aware_path(res.path) returned the inner
   function, which postMessage cannot structured-clone. Every other call
   site resolves the bare name to the correctly bound
   window.privacy_aware_path global, which is why only the external-site
   popup flow was broken. Use the global at the call site and import the
   factory under a distinct name so a bare call can't silently resolve
   to it again.

2. The v2 backend returns `conflict` for a same-name write, but the v1
   wire contract is `item_with_same_name_exists` + `entry_name`, which
   the GUI's save dialogs key on to offer the overwrite prompt. Restore
   the legacy code/field on the write-conflict error and carry
   HttpError.fields through the /batch per-op error serializer.

Verified end-to-end against a local backend: fresh save resolves the
caller's promise with the signed saved_file and closes the popup;
saving an existing name shows Replace/Cancel and Replace overwrites.
Backend suite shows no new failures.
2026-07-26 11:28:38 -07:00
Daniel Salazar 2262975785 fix: readdir response type (#3449) 2026-07-25 18:06:50 -07:00
Daniel Salazar e6e6e3ba9a chore: cleanup API driver calls PUT-1324 (#3448) 2026-07-25 18:05:36 -07:00
Daniel Salazar e85cd9d53d feat: readdir with depth (#3446) 2026-07-25 14:04:21 -07:00
Nariman Jelveh 828b5d9430 Fix popup file pickers rendering an empty explorer
The cover-page rule added for the auth windows forced `width: 100%` on
every `.window-cover-page .window-body`, including the popup file,
directory, and save pickers. That body is an item-container with
`overflow: auto`, so it establishes its own formatting context and may
not overlap the floated `.window-sidebar`. Unable to shrink to fit, it
dropped below the sidebar, pushing the file list and the Cancel/Open bar
outside the 700x400 popup and leaving the page background showing
through as grey.

`.window-body-filedialog { width: auto !important }` was meant to cover
this but loses on specificity, one class against two. Scope the auth
rule with `:not(.window-filedialog)` instead; that class is only set for
the three dialog kinds, so no auth or session cover page changes.
2026-07-25 13:02:15 -07:00
jelveh 3b1ba72ddd Share item-added updates across GUI shells
Extracted desktop `item.added` container update logic into a shared `apply_item_added_to_containers` helper and wired it into both `UIDesktop` and `UIDashboard`. The helper handles overwrite updates, new item insertion, icon refresh, and resorting for matching item containers. This fixes dashboard-hosted explorer/file dialogs not reflecting uploads until navigation.
2026-07-24 23:01:10 -07:00
jelveh 4245ac04ac Isolate upload dialog to prevent double uploads
Replace reuse of `#upload-file-dialog` with a dedicated hidden file input owned by `init_upload_using_dialog`. This avoids collisions with the dashboard Files tab’s `onchange` property handler, which could trigger duplicate concurrent uploads and "Entry already exists" races. The new flow stores the current target path once per dialog open, clears input value before click (so re-selecting the same file still fires `change`), and snapshots `FileList` to an array before clearing to keep async uploads reliable.
2026-07-24 18:26:43 -07:00
Daniel Salazar b8559c1221 chore: cleanup other puter-js modules to match new structure (#3440) 2026-07-24 17:42:06 -07:00
Daniel Salazar 6c4fa629a9 fix: allow root token to also call ai drivers (#3442) 2026-07-24 17:41:38 -07:00
jelveh e6531d4357 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-24 15:38:38 -07:00
jelveh 0c27571030 Dashboard files: compact drag ghost + spring-loaded nav controls
- Replace the full-tile drag ghost with a compact icon+name chip:
  stacked sheets and a count badge for multi-selections, dimmed
  source rows, same design in list and grid views.
- Holding a dragged item over back/forward/up or a breadcrumb
  segment now spring-navigates there (700ms dwell); cancelling
  the drag returns to the original directory.
2026-07-24 15:37:33 -07:00
Neal Shah 6e6d9d2869 buildXhr, dedup, retry (#3432)
* buildXhr, dedup, retry

* change retry intervals
2026-07-24 18:37:09 -04:00
Nariman Jelveh 34529b2e79 Dashboard files: replace solid-blue selection with tinted accent wash
Selected tiles/rows now use a subtle accent tint plus hairline ring
instead of a solid --select-color fill, so icons and labels keep their
natural colors in both themes. Also refines the grid tile's hover
'...' button (inset, hairline border, hover state), fixes the
light-mode selected-row text color that referenced a dark-mode-only
variable, and drops the opaque icon chip visible on tinted list rows.
2026-07-24 13:09:31 -07:00
Nariman Jelveh 7e4ac6542e Update TabApps.js 2026-07-24 12:42:17 -07:00
Daniel Salazar 3331dbd464 chore: make fs upload cleaner in puter.js (#3436) 2026-07-24 12:18:19 -07:00
Daniel Salazar 391b175a68 fix: remove deprecated claude models (#3438) 2026-07-24 10:54:01 -07:00
Neal Shah 19f35b4200 add opus 5 (#3437) 2026-07-24 13:49:20 -04:00
jelveh 9877bb79d0 Polish mobile dashboard sidebar drawer UX
Refactors the dashboard’s mobile sidebar into a modal-style drawer with a coordinated state helper, scrim overlay, and dedicated close button. Adds accessibility improvements (`aria-label`, `aria-expanded`) and stronger dismissal behavior (scrim tap, close button, Escape, and swipe-left with velocity/threshold handling). Updates mobile CSS for safe-area-aware positioning, improved touch targets/pressed states, frosted toggle styling, drawer shadows/transitions, and drag-state handling for smoother interactions.
2026-07-23 18:56:43 -07:00
Nariman Jelveh 2982e894b1 Update dashboard.css 2026-07-23 18:14:47 -07:00
Nariman Jelveh 10bdf701e0 Dashboard: start the minimize morph immediately, bookkeep the URL behind it
The minimize controls consumed the app's /app/<name> history entry
first and let the popstate handler do the visible hide. When the app's
iframe had stacked joint session-history entries, the pop only settles
at pop_dashboard_app_url's 400ms watchdog — so clicking minimize sat
still for ~half a second before the morph began.

Hide the window in the click frame instead (minimize_window, shared by
the head button, the context menu, and the control drawer) and demote
the pop to pure URL bookkeeping. The eager hide is safe against both
ways the pop settles: every hideWindow branch marks data-is_minimized
synchronously, and the popstate handler and the watchdog both already
skip minimized windows. The browser's Back button still minimizes
through the popstate handler as before.
2026-07-23 17:50:32 -07:00
Nariman Jelveh e764a10ee4 Dashboard: recover minimize/close when history.back() misses the app entry
Minimize (and close's URL cleanup) on headless dashboard app windows
work by consuming the app's /app/<name> history entry with
history.back() and letting the popstate handler do the actual hide.
But back() traverses the JOINT session history, which the app's iframe
shares: if the app navigated internally after load (SPA router,
redirects — and browsers differ on which iframe navigations stack
joint entries), the back() consumes the IFRAME's entry instead. The
parent's URL never changes, no popstate arrives, and the
dashboard_url_pop_pending latch stays stuck — every later minimize and
close becomes a silent no-op, and the stranded /app/<name> entry makes
later traversals resurrect windows the user already dismissed.

Fix with a watchdog after each back(): a successful pop is a
same-document traversal of the parent (app entries are pushState over
the dashboard route), so its popstate lands well inside 400ms and the
watchdog no-ops. If the URL still names the app by then, do what the
popstate would have done — minimize through the same lookup and
minimized-guard — and replaceState the stranded entry back to the
dashboard's own route (captured at push time, tab hash included) so no
stale entry survives.
2026-07-23 17:23:03 -07:00
Neal Shah 7e461e14a2 remove unneeded write origin append (#3434) 2026-07-23 19:09:23 -04:00
Daniel Salazar 48cc706ad3 feat: paginated fetching for all (#3431)
* feat: paginated fetching for all

* fix: metering top up gui reporting
2026-07-23 15:07:58 -07:00
Daniel Salazar 20ea616b59 chore: cleanup kv module (#3430) 2026-07-23 10:05:19 -07:00
Nariman Jelveh 73a7a38ea8 Dashboard: redesign app control pill as a top-edge drawer (#3427)
* Dashboard: redesign app control pill as a top-edge drawer

The floating control capsule over headless dashboard apps consumed
noticeable space, especially on mobile. Replace it with a drawer: a
slim glass tray flush with the top edge (icon + title + minimize +
close) whose tongue-shaped handle hangs from its bottom center. At
rest only the tongue peeks in from the edge; hovering it (mouse),
tapping it, or keyboard-focusing the drawer slides the tray down.
The collapse is a single GPU transform instead of the pill's
max-width squeezes, and the open-then-retract intro now visibly
retracts into the tongue, teaching where the controls live.

Hover-expansion is gated on pointerType === 'mouse': touch taps
synthesize a pointerenter right before click, which made the old
pill's toggle see an already-open state and shut it again.

Also fixes two window-stacking bugs surfaced while testing:

- showWindow demoted stay_on_top windows (every fullpage/dashboard
  app window, created in the 99999999+ z band) to a plain counter
  z-index on restore. focusWindow deliberately never raises
  stay_on_top windows, so after minimize/restore a single dashboard
  focus buried the app permanently. Restore now re-raises into the
  stay-on-top band.

- Pressing the drawer now focuses its window, as the titlebar it
  replaces did. The document-level activation handler works off
  mouseover_window, which only mousemove refreshes — a tap with no
  intervening mousemove (touch, restored windows) would re-raise
  the dashboard over the app.

* Dashboard: restored windows come back with the drawer collapsed

The restore re-played the open intro (expand + auto-retract). A restore
should bring back the app, not the chrome — the user already met the
controls on open, and they just USED them to minimize. showWindow now
forces the drawer shut instead, which also covers the Back-button
minimize path, where the window hides without touching drawer state.

* Dashboard drawer: left-align the title against the icon

The tray is asymmetric — one icon on the left, two buttons on the
right — so a title centered in its fixed box landed off the tray's
true center and read as misaligned against the centered tongue below.
Left-aligned, icon + title cluster as the identity on the left and
the controls cluster on the right.

* Dashboard drawer: morph the tongue into the tray instead of sliding

The drawer is now ONE glass surface that changes shape, rather than a
tray that slides in from off-screen with a handle below it. At rest
it's the tongue; opened, the same surface swells into the tray while
the grabber bar rides the morph and settles into a slim strip along
the tray's bottom edge as the dismiss handle — same handle, both
states. Content is revealed from the center outward through an inner
clipping layer (the surface itself can't be overflow:hidden without
clipping the toggle's touch halo) and condenses in/out of focus with
an opacity + blur + scale crossfade. Opening gets a soft overshoot
and settles; closing is quicker and lands without bounce.

The open width is DERIVED in CSS from the same part variables that
lay out the controls row, so the surface always fits its content
exactly and breakpoints (touch sizes, hidden title under 500px) only
override parts.

* Refine dashboard drawer open-state handle

Update the headless dashboard app drawer so the grabber bar disappears when the drawer is open and only the bottom dismiss strip remains interactive. This adjusts tray/strip sizing on desktop and mobile, adds opacity timing for smooth dissolve/reappear transitions during open/close, and aligns in-code docs/comments with the new behavior.
2026-07-22 22:11:39 -07:00
jelveh 5c3fab5579 Revert "Dashboard: fix minimized apps getting permanently buried under the dashboard (#3429)"
This reverts commit bbc86e7b70.
2026-07-22 22:11:15 -07:00
jelveh 695862c61f Increase dashboard directory icon size
Adjusts the directory icon width in the dashboard files section from 28px to 35px for improved visibility.
2026-07-22 21:45:09 -07:00
Nariman Jelveh bbc86e7b70 Dashboard: fix minimized apps getting permanently buried under the dashboard (#3429)
* Dashboard: fix minimized apps getting permanently buried under the dashboard

Fullpage/dashboard app windows are created stay_on_top in the
99999999+ z band, but showWindow's restore path demoted them to a
plain counter z-index — and since focusWindow deliberately never
raises stay_on_top windows, the demotion stuck: one dashboard focus
after a minimize/restore buried the app permanently (URL and tab
title still naming an app the user could no longer see). Restore now
re-raises stay-on-top windows into their band.

Also make the control pill focus its window on mousedown, as the
titlebar it replaces did. The document-level activation handler works
off mouseover_window, which only mousemove refreshes — a tap with no
intervening mousemove (touch, restored windows) would activate the
stale window and could re-raise the dashboard over the app. Deferred
a tick so it runs after that handler and wins.

* Dashboard: restored windows come back with the pill collapsed

The restore re-played the open intro (expand + auto-retract). A restore
should bring back the app, not the chrome — the user already met the
controls on open, and they just USED them to minimize. showWindow now
forces the pill shut instead, which also covers the Back-button
minimize path, where the window hides without touching pill state.

* Dashboard pill: left-align the title against the icon

The pill is asymmetric — one icon on the left, two buttons on the
right — so a title centered in its fixed box landed off the pill's
true center and read as misaligned. Left-aligned, icon + title
cluster as the identity on the left and the controls cluster on
the right.
2026-07-22 21:40:04 -07:00
Daniel Salazar e35b57068b test: add extra tests (#3428) 2026-07-22 21:13:44 -07:00
Daniel Salazar 5faed55076 fix: autoclaim app when making a subdomain (#3426) 2026-07-22 21:12:38 -07:00
Daniel Salazar 928d5fec16 chore: cleanup AI module for puter-js (#3423) 2026-07-22 18:18:54 -07:00
velzie ebcc4bb597 Add loopback support to puter.peer (#3425)
* allow setting port instead of invite code in puter.peer

* fix grabbing fallbackice under wrong response
2026-07-22 20:13:35 -04:00
Neal Shah 51541d3739 initial pjs helper stuff (#3421)
* initial pjs helper stuff

* move resolveReauth

* move location of PuterClient and rename to networkUtils
2026-07-22 18:48:11 -04:00
Nariman Jelveh 43b9200205 Dashboard: headless full-tab apps with a floating control pill (#3424)
* Dashboard: headless full-tab apps with a floating control pill

Maximized app windows in dashboard mode no longer render a titlebar —
the app covers the full tab. The head's controls move to lighter-weight
affordances:

- A floating pill overlaid top-center of the app (parent DOM, above the
  iframe): expanded it shows the app icon, title, minimize, and close;
  it opens expanded so first-time and deep-link users see it, then
  collapses to a subtle capsule that re-expands on hover, tap, or
  keyboard focus. Minimize routes through the same URL-entry consumption
  as the Back button; close tears the window down normally. Dark
  translucent glass so it reads over any app content; safe-area aware;
  bigger targets on coarse pointers; title hidden under 500px; respects
  prefers-reduced-motion.
- Apps-tab tiles double as the app switcher: a macOS-dock-style running
  dot marks tiles with a live (visible or minimized) window, and the
  tile context menu gains a Quit item for running apps. UIWindow
  broadcasts window open/close so the dots stay current.
- puter.ui.setWindowTitle also updates the pill title.

Only maximized, URL-owning app windows in dashboard mode go headless:
dialogs, explorer, non-maximized child windows, and everything on the
desktop keep their titlebars. dashboard.css's 29px head-height
compensation is scoped to windows that still have heads, so headless
app bodies fill the window exactly.

* Fix pill buttons swallowed by their own hover halo

The pill's invisible hover halo (an absolutely-positioned ::after with
negative inset) painted above the pill's static children, so real
pointer clicks hit-tested to the halo and never reached the minimize/
close buttons — while element.click() (which skips hit-testing) worked,
hiding the bug from programmatic checks. Stack the halo at z-index -1
(the pill is its own stacking context, so the outside ring still sits
above the app's iframe) and raise the interactive children explicitly.

* Review round: pill/dashboard edge-case fixes

- Latch pop_dashboard_app_url while its history.back() is in flight: a
  double-click on minimize (or a close racing a minimize) issued two
  back()s, and the second popped the dashboard's own entry — navigating
  clean out of the page. Any popstate or push settles the latch.
- Close a running app's windows when it is uninstalled: the tile is a
  headless app's only switcher, so a minimized instance would otherwise
  be stranded — impossible to restore or quit.
- puter.ui.setWindowTitle in dashboard mode now also updates the
  browser-tab title (while that app owns the URL) and the window's
  data-name, so a later restore re-applies the app's current title
  instead of its launch-time one. Desktop behavior unchanged.
- Nudge collapsed-pill contrast (opacity 0.55 → 0.65, border alpha
  0.16 → 0.22) so the capsule stays findable over dark app content.

* Adjust dashboard tile spacing and label sizing

Tweaks My Apps tile layout constants to match the real icon/label geometry: tile height is increased, label top margin is widened, and the running-status dot is moved slightly lower to avoid overlap during icon hover scaling. It also raises dashboard app pill title line-height to prevent descenders from being clipped when ellipsis overflow is active.

* Pill and running-dot visual polish

- Fixed-width pill: the title gets a fixed 150px centered box instead of
  a cap, so every app's pill is identical and the minimize/close buttons
  sit at the same screen position in every app — they're operated by
  muscle memory, like any titlebar's controls.
- Give the pill title line-height 1.35: overflow:hidden (needed for the
  ellipsis) also clips vertically, and at line-height 1 it cut the
  descenders off g/p/y.
- More air around the tile running dot: label margin 6px → 10px and the
  dot 2px lower (tile row height 78px → 82px to keep the label line in
  budget — the pager layout reads the CSS var, so it follows). The
  icon's 1.08x hover growth no longer touches the dot.

* Size the pill icon to balance the capsule

16px read as undersized next to the 26px buttons in the 36px capsule —
bump the expanded icon to 20px with a proportional (25%) radius, and
object-fit: cover in case an app icon isn't square. Collapsed size is
unchanged.
2026-07-22 14:28:06 -07:00
jelveh d5e9417d2d Dashboard apps own the URL: /app/<name> entries with Back-to-minimize
- Opening an app in dashboard mode pushes a real history entry at
  /app/<name>; Back minimizes the app (it keeps running), Forward
  restores it — or relaunches it if it was closed
- The minimize/close buttons consume the entry via history.back() so
  the address bar never names an app that is no longer on screen
- Landing directly on /app/<name> now boots the dashboard with the app
  open maximized in-page instead of the desktop; embeds, popups, and
  explicit fullpage boots keep the desktop flow, and fullpage_on_landing
  no longer opts a landing out of the dashboard
- Desktop mode no longer rewrites the URL at all (no /app/<name> or
  folder-path replaceState on focus, no /desktop resets on close or
  minimize); tab titles still track the focused window
2026-07-22 10:50:10 -07:00
jelveh dec205d959 Lighten tile-to-window restore animation
Update `showWindow` to use a shorter, window-only morph when restoring a minimized app from a dashboard tile. `morph_window_from_tile` now accepts options for `icon_half` and `duration`, gates icon ghost creation accordingly, and derives transition/cleanup timing from the configured duration so un-minimizing feels distinct from a full app launch.
2026-07-22 09:57:24 -07:00
jelveh d87f3464fc Support Cmd/Ctrl+click and adaptive bg color
Update the dashboard app tile click handler so Ctrl/Cmd+click opens apps in a new tab (matching the context menu behavior) while keeping existing target-link/app-name routing. Also switch the default body background to white and add a dark-mode media query to use a dark background color.
2026-07-21 20:33:27 -07:00
jelveh 25410cac92 Allow tile morph animations on phones
Removes `isMobile.phone` checks that previously disabled dashboard tile morph transitions. Window open/close morph selection and tile-launch ghost setup now run on phone the same way they do elsewhere, while still honoring animation-disable and reduced-motion conditions.
2026-07-21 19:13:48 -07:00
jelveh 66a1baef24 Preplay dashboard tile launch morph animation
Add click-time launch feedback for dashboard app tiles so users get immediate visual response while app startup is still loading. TabApps now starts and settles tile launch state around `launch_app`, logs launch failures, and keeps duplicate-launch protection.

UIWindow now tracks in-flight tile feedback with a WeakMap, introduces `begin_dashboard_tile_launch`/`settle_dashboard_tile_launch`, and updates `morph_window_from_tile` to avoid replaying the icon half when it already ran at click time. Ghost/icon timing and cleanup were adjusted so fallback and failed launches do not leave stale tile state.
2026-07-21 19:09:32 -07:00
jelveh 5c4df9cbd7 Revert "feat(gui): start the tile open-morph on click, before the launch fetches"
This reverts commit f89b3a888a.
2026-07-21 18:25:39 -07:00
jelveh f89b3a888a feat(gui): start the tile open-morph on click, before the launch fetches
A fresh launch has a server round-trip (app info, signing) between the
click and the window's creation, so the open morph used to start late —
on a slow connection the click gave no feedback at all. The two halves
of the morph are now decoupled:

- On click, begin_dashboard_tile_launch starts the icon's half
  immediately: the ghost enlarges in place and dissolves exactly as
  before, and the real icon stays hidden — the empty slot reads as
  'launching' — while the fetches run.
- When the window opens, morph_window_from_tile claims the click-time
  ghost instead of spawning a second one, runs the window half alone,
  and restores the icon when it lands. On a fast launch the halves
  overlap into one motion; on a slow one the window zooms out of the
  empty slot.
- settle_dashboard_tile_launch (in the launch's finally) puts the icon
  back whenever the window never claims it: launch failed, app opened
  in background, or the morph fell back to the plain fade. The launch
  also gained a .catch, so a failed launch logs instead of leaking an
  unhandled rejection.

The un-minimize path has no fetch and is untouched: it still spawns its
own ghost inside the morph.
2026-07-21 17:58:05 -07:00
Nariman Jelveh bf545dda25 Dashboard: open apps maximized in-page, with minimize and single-instance behavior (#3419)
* feat(gui): open dashboard apps maximized in-page with minimize + single instance

Apps launched from the dashboard's Apps tab now open as maximized Puter
windows in the same page (via launch_app) instead of a new browser tab.
External website shortcuts still open in a new tab since arbitrary sites
can't reliably be iframed.

Dashboard app windows keep their minimize button (fullpage mode normally
hides it since there's no taskbar). With no taskbar item to animate
toward, hideWindow now hides the window in place and flags it with
data-minimized_in_place; showWindow un-hides it via that flag. An
explicit flag is used because data-orig-* can't distinguish the two
minimize paths - drag/maximize handlers set those attrs too.

Tiles are single-instance: clicking a tile un-hides a minimized
instance or focuses a visible one instead of launching a duplicate, and
launches in flight swallow repeat clicks.

Also fixes .window-body-app height in dashboard mode: fullpage mode
sizes it to 100% assuming no titlebar, which clipped the bottom 29px of
every app.

* feat(gui): iOS-style minimize-to-icon morph in dashboard mode

Minimizing a window in dashboard mode now zooms it into its app's tile
on the Apps tab — but only when the user can see that tile (Apps section
active AND the tile on the pager page currently in view); otherwise the
existing hide-in-place fade is kept, as on phones and with
prefers-reduced-motion.

The animation is a two-layer morph on one 450ms cubic-bezier(0.32,
0.72, 0, 1) path: the live window scales onto the tile's icon box
(transform + opacity only — no per-frame iframe reflow) while its corner
radius grows toward the icon squircle, and an enlarged clone of the icon
flies congruently with it, cross-fading in as the window fades out and
landing exactly on the real icon's slot (hidden until the handoff), so
the window reads as becoming the icon.

Window geometry is never touched: only inline transform/opacity/radius
are set and restored byte-identical when the animation ends, and
showWindow()'s existing minimized-in-place path restores as before —
including a reopen mid-animation.

* fix(gui): keep the minimize-morph ghost icon square (contain-fit)

The icon clone previously mapped rect-onto-rect over the window, which
stretched the square icon anamorphically. A covering square was tried
and jittered: it protruded past the card's short edge and the halo
popped in mid-crossfade.

Contain-fit resolves both: the ghost scales uniformly (always square),
sized to the window's short edge and centered — its side tracks the
card's short edge exactly for the whole flight (both interpolate
linearly between the same endpoints on the same curve), so the card's
rectangular flanks melt away around a steady, undistorted icon.

* feat(gui): add 'Open in new tab' to app tile context menu

Every tile's right-click menu now leads with Open in new tab, which
opens the app the way tiles did before in-page windows: /app/<name> in
a new browser tab (noopener), or the site link for external tiles.
Non-uninstallable apps previously had no menu at all; they now get this
single item, and uninstallable ones show it above a divider + Uninstall.

* feat(gui): reverse morph — dashboard tile icons zoom into their opening window

Opening an app in dashboard mode now runs the minimize morph backwards:
the window grows out of its tile's icon box (transform + opacity only,
radius relaxing from the 22% squircle to the resting value) while a clone
of the icon enlarges IN PLACE on its slot — capped at 2.5x, fully dissolved
by ~250% growth — as the window fades in underneath on the same 450ms
cubic-bezier(0.32, 0.72, 0, 1) path with the crossfade handoff mirrored
(window in at 80-240ms, ghost out at 120-320ms).

Both open paths get the morph, with the same visibility conditions as
minimize (Apps section active, tile on the pager page in view, not phone,
no reduced motion), falling back to the existing fades otherwise:

- fresh launch: TabApps passes window_options.morph_from_dashboard_tile,
  and UIWindow morphs instead of the 70ms opening fade. Since .window is
  display:none from the stylesheet until later in the pipeline, the hook
  shows it before measuring (nothing has painted yet, so neither the early
  show nor the hide-back on a declined morph can flash).
- un-minimize: showWindow's minimized-in-place branch morphs instead of
  fadeIn(150), keeping the z-index bump and 80ms focus timing.

A data-window_morphing flag guards against opposing morphs fighting over
the window's inline styles: minimize clicked mid-open-zoom (or a tile
click mid-minimize-zoom) falls back to the pre-morph fade behavior, and
each animation still restores every inline property byte-identical when
it ends.

* Update dashboard.css

* feat(gui): cap the open-morph ghost icon at 2x, dissolving in place

The tile icon no longer flies toward the window center nor grows to full
contain-fit (10x+ reads as a giant blurry sticker): it enlarges in place
on its own slot and its growth is capped so the fade-out completes right
around 200% of its size — the window alone carries the motion and the
rest of the growth.

* fix(gui): drop the tile :active press-shrink that stuttered the open morph

Clicking a tile launches the app and the open morph grows the icon in
place — the 0.95 press-shrink made every launch read as shrink-then-grow.
The icon now holds its hover scale through the click and enlarges
straight from there (the morph ghost is measured from the rendered rect,
so it picks up the hovered size seamlessly).
2026-07-21 17:41:45 -07:00
Neal Shah c706b4472d use max_tokens instead of hardcoded value 1000 (#3422) 2026-07-21 18:25:40 -04:00
Neal Shah 93b8041c3d don't allow prototype defined method to be called in drivers (#3413)
* don't allow prototype defined method to be called in drivers

* fix test
2026-07-21 18:01:52 -04:00
Daniel Salazar 89f9f9728f fix: PUT-1355 PUT-1351 PUT-1208 (#3420)
* fix: PUT-1355 PUT-1351 PUT-1208

* fix: dev center header
2026-07-21 14:17:33 -07:00
Daniel Salazar 645409eddb fix: misc hardening (#3418) 2026-07-21 13:55:00 -07:00
tamilselvanp57 58a763bcd1 Add Dev Center UI for viewing app users (#3349) (#3417)
Adds a 'View Users' link on the app Analytics tab that opens a modal
listing users who have authenticated into the app (username + email,
where the user has granted email:read to the app).

Consumes the existing puter.apps.get(...).users()/.getUsers() client
SDK method, backed by the app-telemetry driver's get_users - no new
backend work needed, per discussion on the issue.

Relates to #3349
2026-07-21 13:52:48 -07:00
Daniel Salazar fd91d72654 wip: better contributing docks (#3415) 2026-07-21 12:07:59 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 7fbf491e6f Update docs meta tags and description (#3416)
* Update docs meta tags and description

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-21 16:43:00 +07:00
Daniel Salazar a8833de9d5 fix: misc hardening (#3414) 2026-07-20 23:53:37 -07:00
Daniel Salazar 3a8b6394de feat: standardized api pagination (#3412) 2026-07-20 15:58:23 -07:00
Nariman Jelveh f5d656650f feat(email): accept custom message headers in sendRaw (#3411)
* feat(email): accept custom message headers in sendRaw

Lets callers set transport-level headers such as List-Unsubscribe /
List-Unsubscribe-Post. Also documents the puter-email driver's automatic
unsubscribe / report-abuse footer and the new `suppressed` result field
in the puter.js email module.

* feat(email): envelope override in sendRaw + suppressed in EmailSendResult type

Adds an optional `envelope` field to SendMailOptions so callers can set
transport recipients independently of the visible To/Cc headers (used by
the email-send driver for per-recipient deliveries), and adds the
`suppressed: string[]` field to the public EmailSendResult typing to
match the driver's documented result.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(email): document per-recipient best-effort delivery via `failed`

The email-send driver now attempts every recipient's private delivery
even when one fails, returning failed addresses in the result's
`failed` array instead of failing the whole call — so callers retry
only the failed subset and never re-mail delivered recipients. Adds
the field to the public EmailSendResult typing and module docs.
2026-07-20 11:55:46 -07:00
Daniel Salazar dd314da16d feat: require app or api tokens for ai api usages (#3407) 2026-07-20 08:34:44 -07:00
jelveh 47c1d4268f Fix overlapping context menu state on file rows 2026-07-19 20:08:00 -07:00
jelveh b9e87935cc Update dashboard.css 2026-07-19 20:00:18 -07:00
jelveh 73dd95bd41 Open Desktop by default in the Files tab 2026-07-19 19:53:52 -07:00
jelveh c5bc358ef2 Keep file rows visually active while their context menu is open 2026-07-19 19:51:38 -07:00
jelveh 57c9eb9e1a Fix grid badge drop shadow and hover overlap with item menu 2026-07-19 19:23:02 -07:00
jelveh a0c0544928 Remove extra top padding from files header 2026-07-19 19:15:51 -07:00
jelveh 6dcae81a56 Fix uninstall animation snapping after a prior FLIP slide 2026-07-19 18:57:04 -07:00
jelveh 09baee6381 Delay uninstall tile animation until after the modal closes 2026-07-19 18:48:06 -07:00
jelveh 68aab718a1 Add keyboard navigation to the Apps tab 2026-07-19 18:43:57 -07:00
jelveh 5d6b2f3265 Uninstall apps optimistically with a tile removal animation 2026-07-19 18:35:01 -07:00
jelveh 28730010c3 Fix inconsistent icon sizing in dashboard grid views 2026-07-19 12:24:21 -07:00
jelveh 817ece40ba Modernize folder icon yellows 2026-07-19 12:24:20 -07:00
jelveh 7c868e947a Fix Public folder icon grid and align glyph system 2026-07-19 12:07:10 -07:00
jelveh 66984cd3d6 Improve glyph sizing on system icons 2026-07-19 12:03:09 -07:00
jelveh 5fa508ba91 improve dashboard file list and grid styles 2026-07-19 11:48:46 -07:00
jelveh c1fca467bf Use bundled fallback for null app icons 2026-07-19 10:52:56 -07:00
jelveh e726c19311 Add compact grid option to Files view menu
Reworked the Files view toggle into a context menu with List, Compact Grid (`grid-sm`), and Grid modes, and refactored view handling into shared helpers (`isGridView`, `viewClass`, `setView`, `applyViewMode`) so grid behavior is applied consistently across rendering and drag previews. Added compact-grid CSS overrides to reduce tile, icon, text, and badge sizing while reusing existing grid layout behavior.
2026-07-19 09:48:31 -07:00
jelveh 6643642aa7 Fix item-more color for selected rows in dashboard
Ensure the item-more icon uses the correct sidebar item color when a dashboard file row is selected, including on hover.
2026-07-19 09:26:54 -07:00
jelveh 65dc3b5f4c Use thinner dashboard file nav icons
Switches the Files tab header navigation buttons to new thin arrow SVG assets and aligns icon styling with a lighter visual weight. It also updates several header action SVG paths (new folder, upload, list/grid, sort, select) and sets path action icon color to #5c5c5c so the controls look consistent.
2026-07-18 21:44:02 -07:00
jelveh c225fabbd4 Fix clipped app tiles and soften files header hover styling
Apps tab: tile rows only budget one label line, so two-line app names
made flexbox shrink the icon box into a landscape crop and clip the
label mid-glyph. Clamp labels to a single line and exempt the icon box
from flex shrinking.

Files tab: header hover states (nav arrows, breadcrumbs, action
buttons) now use a neutral #EEE chip and no longer recolor the glyph;
nav arrows also render at full opacity when enabled.
2026-07-18 21:27:24 -07:00
jelveh fdcc2e301b Fix My Apps tile icon and label overflow
Prevent app tile icons from shrinking when labels overflow, and clamp tile labels to a single line so they fit within the fixed tile height. This keeps the grid layout stable while preserving full app names in the existing tooltip.
2026-07-18 20:54:09 -07:00
Nariman Jelveh 183ab0780d fix: verified Dashboard bug fixes (routing, files, apps, usage, mobile menu, CSS) (#3406)
* fix: don't double-refresh Dashboard tabs on browser back/forward

A single back/forward navigation fires both popstate and hashchange, so
handleRouteChange ran twice, calling the tab's onActivate twice — every
back/forward to the Apps tab issued duplicate /installedApps and
/get-launch-apps requests and re-rendered the tab twice. Track the last
handled URL and skip the second event.

* fix: store raw values in Dashboard file-row data attributes

renderItem passed html_encode()d strings to setAttribute, which stores
them literally — a file named "Tom & Jerry.txt" got
data-path="...Tom &amp; Jerry.txt". Every flow that reads the attribute
back (keyboard cut/copy/paste, delete-to-trash, drag-and-drop, selection
actions) then hit the server with the mangled path and failed with
"Entry not found". The socket item.renamed/item.updated handlers had the
same problem via jQuery .attr(), and item.moved/item.removed matched rows
with html_encode()d selector needles that can't match raw attributes.

Store raw values everywhere (matching the desktop convention and the
raw readers throughout TabFiles), match paths by comparison instead of
selector interpolation, and repair the html_encode(x) ?? fallback
expressions that could never take their right-hand side. Verified live:
copy/paste of a name containing '&' now succeeds.

* fix: escape folder names in Dashboard nav-history menu (stored XSS)

The back/forward taphold menus interpolated path.basename(history_item)
straight into UIContextMenu item html, which renders it verbatim. A
folder named with an HTML/script payload executed when the user opened
the history menu after visiting it. The desktop file manager already
encodes this value; match it with html_encode(). Verified live: the
payload now renders as inert text and its onerror never fires.

* fix: harden Dashboard hash routing against malformed and unknown tabs

Three routing defects, all reachable from a shareable URL:

- A malformed percent-sequence (e.g. `#100%`) made parseDashboardRoute's
  decodeURIComponent throw at module load, blanking the entire GUI. Guard
  the decode and fall back to the raw hash. Verified: `#100%` now boots.

- A hash whose tab segment contained a quote (`#foo"bar`) was interpolated
  into a jQuery selector that throws, leaving dashboard event handlers
  unbound. Resolve the route tab against the known tab set (falling back to
  Apps) before it ever reaches a selector. This also fixes an unknown hash
  activating the Apps section without its `.dashboard-content.apps` styling.

- Re-clicking the already-active sidebar tab pushed duplicate history
  entries, so Back became a no-op until pressed repeatedly. Only pushState
  when the hash actually changes.

Verified live for all three.

* fix: keep Dashboard file list footer and size cells in sync on live updates

Incremental updates only touched hidden data attributes, so the visible
UI drifted from the data:

- Adding a file via the socket (item.added) or creating one inserted a
  row but never called updateFooterStats, so the "N items · size" footer
  stayed frozen at the old count. Removals and moves had the same gap.
- item.updated (and the in-place update in UIDashboardFileItem) rewrote
  data-size/data-modified but not the on-screen .item-size/.item-modified
  cells, so a remote overwrite left a stale size (e.g. "5 B" for a file
  that had grown to 2 KB).

Refresh the footer after add/remove/move and repaint the size/modified
cells on update. Verified live: add, overwrite, and delete now all
reflect immediately.

* fix: Dashboard Files shift-click selection after navigation and byte formatting

Shift-click selection:
- renderDirectory cleared the .selected class but left
  window.latest_selected_item pointing at a now-detached row from the
  previous directory. The shift handler set shift_clicked=true, found the
  anchor's index as -1, skipped the range select, and onclick then
  early-returned on shift_clicked — so the first shift-click in a new
  directory selected nothing. Reset the anchor when it detaches, only take
  the range path when the anchor is still in the list, and treat a
  shift-click with no valid anchor as a plain select that becomes the
  anchor (onclick otherwise ignores clicks while Shift is held).

formatFileSize:
- Returned "NaN undefined" for missing/invalid sizes and "1.5 undefined"
  for terabyte-plus files (sizes array stopped at GB). Guard non-finite
  and non-positive input, clamp the unit index, and add TB/PB.

Verified live: first shift-click selects the item, a second shift-click
extends the range, and formatFileSize returns 0 B / 1.5 TB / 2 PB.

* fix: show a message instead of a blank pane when a Dashboard folder fails to open

renderDirectory empties the file list before awaiting readdir, so a
readdir rejection (permission lost, folder deleted from another session,
network error) left a completely blank pane with only a console error.
Render a centered "This folder couldn't be opened." message on the catch
path. Verified live: a simulated readdir failure shows the message,
clears the spinner, and leaves navigation working (next folder loads).

* fix: Dashboard Usage/Home tab data freshness, errors, and dead controls

TabUsage:
- The Upgrade/Manage button was shown unconditionally and its click did
  `new window.UIUpgradeAccount()`, which only exists on hosted puter.com —
  on self-hosted it threw a TypeError from a dead button. Hide the button
  when UIUpgradeAccount is absent and guard the click.
- The tab had no onActivate and bound its refresh to a nonexistent
  element, so usage numbers were frozen at page-load for the whole
  session. Add onActivate to refetch (verified: reactivating refetches).
- Resource names rendered the backend's `_dot_` escaping literally
  ("gemini-2_dot_5-flash"); un-escape for display and html_encode the cell.
- The two loaders had no error handling; a failed fetch left the tab
  blank with an unhandled rejection. Catch each and show a fallback.
- Guard capacity 0 so storage no longer renders "NaN%".

TabHome:
- Plan button kept saying "Manage →" after a subscription lapsed; reset it
  to "Upgrade →" in the free branch.
- Guard capacity 0 ("NaN%") on the storage card.
- "Your Plan ›" card header had the arrow affordance but no target and did
  nothing on click; point it at the Usage tab like its siblings.
- Returning to the tab fired both focus and visibilitychange, and
  refreshAndBroadcast both called refresh() and dispatched the event its
  own listener handles — up to 4 duplicate reloads per focus. Drop the
  direct call and coalesce the focus/visibility pair.

TabHome + TabApps:
- window.open(externalAppUrl, '_blank') lacked noopener, exposing the
  dashboard tab to reverse tabnabbing from an external app site. Add
  noopener,noreferrer.

Verified live: Usage and Home render without NaN or console errors, the
upgrade button is hidden on self-hosted, and 'Your Plan' now navigates.

* fix: multiple Dashboard CSS defects (dead rules, contrast, responsive)

- The desktop rule that hides the row ⋮ button ended two selectors with a
  comma before an @media block, so the whole construct was invalid and
  discarded — the ⋮ showed on every list-view row on desktop instead of
  deferring to right-click. Verified: it's now hidden (display:none).
- Native-drop dark-mode styles keyed off .window[data-color-scheme="dark"],
  an attribute never set anywhere; converted to @media (prefers-color-scheme:
  dark) like every other dark rule so they actually apply.
- .myapps-tile-label set white-space:nowrap after a 2-line -webkit-line-clamp,
  defeating the clamp so long app names clipped mid-glyph on one line;
  removed it.
- .dashboard-sidebar-separator was defined a second time with a contradicting
  box model (background line + full-width margin), rendering a doubled,
  edge-to-edge divider; removed the duplicate so the inset border rule stands.
- Grid-view .item-icon had a hardcoded background:white and border-radius:2px
  overriding its own border-radius:8px (glaring white tiles in dark mode,
  square-ish corners); use var(--dashboard-background) and keep 8px.
- .files-footer used hardcoded #666/#CCC on theme-variable backgrounds
  (dim in dark, near-invisible separator in light); use the text/muted vars.
- Fixed the .ui-droppable-over typo (jQuery UI emits ui-droppable-hover).
- The context-menu backdrop's "desktop transparent" rule used min-width:768px
  while every mobile rule uses max-width:768px, overlapping at exactly 768px
  (a common tablet width) — a non-dimming, invisible modal shield; bumped to
  769px.
- At 481-768px the fixed hamburger toggle sat on top of the Files directories
  column's first folder because .dashboard-content.files padding wins over the
  media-query padding; add top clearance to the directories column there.

Verified light-mode desktop is unchanged (directories padding still 16px, ⋮
hidden, footer colors resolve).

* fix: only offer Uninstall for Dashboard apps where it actually sticks

Uninstall was suppressed via a hardcoded 8-name allowlist that had drifted
out of sync with the backend's ~26 recommended apps. For the ~18 unlisted
recommended apps (Calculator, Code, the games, …) the menu offered
Uninstall, revokeApp resolved, the tile vanished — then get-launch-apps
re-added it on the next load and it reappeared, so the uninstall silently
reverted.

Compute uninstallability from the actual lists: an app is uninstallable
only if it's in the user's installedApps AND not in the recommended list
(and not a protected core app). Recommended apps — installed or not —
resurrect on reload, so Uninstall is hidden for them. Verified live: a
synthetic installed-not-recommended app shows Uninstall; a recommended
app shows none.

* fix: guard Dashboard Apps loads against stale overwrites and drag/error clobbering

loadApps only checked for an in-progress drag before its await, and its
catch wiped whatever was on screen. Three concurrency issues followed:

- A slow, older load resolving after a newer one could overwrite the
  newer app list (and clobber a reorder the user saved while the stale
  fetch was in flight). Tag each load with an increasing id and skip
  applying one only when a strictly newer load has already applied —
  gating on "already applied" (not "latest started") so the first load to
  resolve still populates the list for the pager's ResizeObserver.
- A drag that began while a load was awaiting could have the grid rebuilt
  out from under it; re-check the drag after the await.
- A transient re-fetch error replaced a working grid with "Failed to load
  apps"; only show that placeholder when nothing has loaded yet.

Verified live: a slow older load no longer clobbers a newer one, and the
grid still renders normally on activation.

* fix: page through all installed apps in the Dashboard instead of capping at 100

The /installedApps endpoint clamps limit to 100 and paginates, but the
Apps tab fetched a single page — a user with more than 100 installed apps
silently lost the alphabetically-last ones from both the grid and search,
with no way to launch or uninstall them from the dashboard. Loop pages
until a short one comes back (the common <100-app case still makes a
single request). Verified: the request now carries &page=1 and stops
after one page for a small account; the loop pulls all pages otherwise.

* fix: make the mobile Dashboard context menu handle submenus, disabled items, and positioning

The touch context-menu modal (used whenever maxTouchPoints > 0) had three
defects:

- Items with a submenu and no onClick ("New", "Open With") rendered as
  plain buttons that did nothing and didn't even close the menu — the two
  submenu-bearing actions were simply unreachable on touch. Drill into the
  submenu on tap, with a Back row to return.
- disabled items were rendered as active buttons and executed their
  onClick, so a folder's disabled "Paste Into Folder"/"Publish as website"
  ran anyway. Render them inert (disabled attribute + dimmed class + a
  click-handler guard), mirroring the desktop UIContextMenu.
- Non-touch devices that still route here (touchscreen laptops) got the
  modal pinned at a hardcoded left:300px, far from the tapped item on a
  wide screen. Center it over the target and clamp to the viewport.

Verified live (with maxTouchPoints forced): "Open With" opens its submenu
and Back returns; disabled "Paste Into Folder" is inert; the modal is
positioned near the item.

* fix: repair Apps-tab regressions from the Dashboard bug-fix pass

- A non-array /installedApps response (an error payload) was read as
  end-of-pagination, silently rendering the grid without any installed
  apps; fail the load so the explicit error state shows instead.
- A loadApps result that resolved mid-drag was discarded with no retry,
  freezing the grid on stale data for the rest of the visit; stash it
  and apply it after the drag ends, reconciled against the drag's final
  order so it can't undo a reorder whose save is still in flight.
- Uninstall disappeared for recommended/recent apps, removing the only
  UI path that revokes their permissions; offer it again and instead set
  expectations in the confirm modal (the tile stays for apps that
  get-launch-apps re-adds).
- init and the initial-route onActivate both fired a full load on open;
  share the in-flight load instead of issuing a duplicate request trio.

* fix: repair Files/routing regressions from the Dashboard bug-fix pass

- The no-anchor shift-click fallback also fired when shift-clicking the
  current anchor, collapsing an existing multi-selection to one item,
  and it ignored Ctrl/Cmd; only run it when there is genuinely no
  anchor, and keep the existing selection when Ctrl/Cmd is held.
- Unknown hash values (a stale bookmark, an in-page anchor) were
  coerced to 'apps' and yanked the user off their current tab with a
  refetch and autofocus; ignore them outright, both at boot and on
  hashchange, while still keeping untrusted hashes out of selectors.
- The item.updated socket handler re-implemented the row-refresh from
  TabFiles and had already drifted (trashed items showed their raw UID
  instead of metadata.original_name); extract a shared row updater and
  delegate to it.

* fix: repair Usage-tab regressions from the Dashboard bug-fix pass

- Hiding the plan button whenever window.UIUpgradeAccount was missing
  at check time could hide it for the whole session on hosted
  deployments that attach the script after dashboard init; keep
  re-checking for a while before concluding the install is self-hosted.
- init and the initial-route onActivate both refreshed usage on a
  direct #usage open; share the in-flight refresh instead of issuing
  duplicate request pairs.

* fix: repair Dashboard CSS regressions from the bug-fix pass

- Repairing the malformed item-more rule put display:none into effect
  for the first time, removing the visible-and-used desktop row '⋯'
  button; drop the rule instead (it was never in effect on any release).
- The desktop context-menu backdrop breakpoint (min-width: 769px) left
  fractional viewport widths between 768px and 769px with the mobile
  dimming; use the exact complement of the mobile max-width: 768px
  rules instead.

* fix: make Dashboard app uninstall honest about recents and resilient mid-flight

- Recently-opened apps were classified as sticky-removable, but the
  recent list is built from app-open history that a revoke doesn't
  touch, so their tiles reappeared on the next load — the very bug the
  uninstall gating set out to fix. Treat recents like recommended.
- The confirm modal claimed every staying tile was 'provided by Puter',
  which is false for third-party recents; describe the actual reason.
- Uninstall now marks in-flight loads stale so a fetch that started
  before the revoke can't resurrect the removed tile.
- A pagination failure after the first page no longer discards the
  pages already fetched — one flaky request among N used to turn the
  whole grid into 'Failed to load apps'.

* fix: keep the Dashboard Usage tab stable through transient failures

- A failed refresh no longer wipes an already-rendered usage table;
  the unavailable note only shows when there is nothing on screen yet
  (mirrors the Apps grid's error handling).
- The plan-button check now re-checks indefinitely with backoff instead
  of giving up after ~10s, so an arbitrarily late UIUpgradeAccount
  attachment can't leave a subscriber without the button all session.

* fix: Dashboard Files footer sync and Home refresh resilience

- The shared row updater now refreshes the footer's item-count/total
  line, which is computed from the data-size attributes it just wrote;
  a remote overwrite no longer leaves the row and footer disagreeing.
- Size/Modified are only written when the update payload carries them,
  so a minimal event can't zero out correct values on screen.
- A failed readdir now resets the footer instead of keeping the
  previous directory's counts over an empty pane.
- Home's focus-refresh caps its wait on refresh_user_data, so a whoami
  that never settles can't leave the plan/usage cards stale forever.

* fix: contain partial Apps loads and keep the uninstall flag current

- A partial installed-apps list (a page beyond the first failed) now
  renders only when the grid is empty — it must never replace a
  complete grid already on screen — and saveOrder refuses to persist
  while the list is partial, since overwriting the saved order with a
  truncated one would drop the missing apps' positions for good.
- Opening an app from the grid now flips its tile's uninstallable flag:
  the open puts it in the server-side recent list, so a revoke from
  that moment on leaves the tile in place, and the modal must not act
  on the stale load-time snapshot.
- The in-flight-load invalidation moved into _invalidateInFlightLoads()
  so the seq protocol lives in one place.

* fix: Usage-tab error guard, dead spinner, and poll decay

- The transient-error guard now tracks 'a table has rendered' instead
  of data length, so an account with legitimately zero usage doesn't
  get its empty table replaced by the unavailable note on a flaky
  refresh.
- update_usage_details animated a spinner element that exists nowhere
  in this tab and held every refresh open for an artificial 1s minimum;
  both halves of the dead control are now gone.
- The plan-button recheck backs off to a 60s heartbeat instead of
  polling at 2s forever on installs where UIUpgradeAccount never
  appears.

* fix: guard minimal file updates; tighten selection and socket hot paths

- The shared row updater only writes the fields an update payload
  actually carries, so a minimal event can't blank a row's name/path —
  the same hazard the size/modified guards already covered.
- The all-rows lookup in the selection handler now happens only under
  Shift instead of on every click, and the single-select block shared
  by the no-anchor shift-click and drag-handle paths lives in one
  selectSingle helper.
- item.removed and item.moved narrow by data-uid (O(1) selector) before
  comparing paths instead of scanning every row per socket event.

* fix: replace partial-load and uninstall-flag patches with convergent designs

The previous round's containment patches each spawned new corner cases
(a persistently failing page froze all refreshes; the partial-load
saveOrder bail silently discarded reorders; the tile-open flag flip
lost races with in-flight loads, server recording, and popup blockers).
Replace both mechanisms:

- A partial installed-apps fetch (later page failed) now always applies,
  supplemented with the apps already known from the previous list — the
  grid and search can't shrink from one flaky request, refreshes never
  freeze, and an empty grid still gets the fetched pages.
- saveOrder carries over saved names missing from the current list
  instead of refusing to persist, so no app's saved position is ever
  dropped; stale names are harmless (reconcileAppOrder ignores them)
  and preserve the position of apps that return.
- The load-time uninstallable snapshot is no longer patched client-side
  on tile open. Uninstall now revokes, optimistically removes the tile
  when expected to stick, and refetches — the grid converges to
  server-side truth instead of guessing at recents state.

* fix: don't blank filenames on metadata-only updates; unify range selection

- A payload carrying metadata but no name resolved to an empty display
  name and blanked the row's visible filename — skip the write when the
  resolved name is empty.
- The shift-range branch now goes through the same applySelection
  helper as the no-anchor and drag-handle paths (it takes the row set),
  removing the last hand-rolled copy of the selection bookkeeping.

* fix: rank-preserving saved-order merge; keep the pager put after uninstall

- The saved-order carryover appended missing apps' names at the tail,
  permanently demoting their saved grid positions — the very thing it
  claimed to protect. Replace it with mergeSavedOrder in appOrder.js
  (beside its tested siblings, with unit tests): each missing name
  keeps its rank among surviving names, so a drag during a partial-load
  session neither drops hidden apps' positions nor teleports them
  behind the dragged tile.
- The post-uninstall convergence refetch rendered without preservePage,
  snapping the pager back to page 1 and replaying the load fade; thread
  render options through loadApps so the background sync keeps the
  user's page.

* fix: keep hidden apps' saved ranks through the mid-drag stash path

- _applyPendingLoad reconciled a stashed load against the visible-only
  on-screen order, tail-appending any app returning to the grid and
  contradicting the order the drag had just saved — the rank-demotion
  defect resurfacing through one more path. It now prefers
  _savedOrderNames, the canonical saved list that saveOrder keeps
  merged with hidden apps at their ranks, over the possibly pre-drag kv
  snapshot the stashed load fetched.
- The post-uninstall optimistic render now also skips the load fade;
  it was the one in-place rebuild still hiding the grid behind the
  opacity-0 icon gate.
- mergeSavedOrder's rank rescan collapsed to a single forward
  insertion pointer (verified output-identical; the 20 unit tests pin
  the behavior).

* fix: gate stale kv-order snapshots on the latest local save

A drag that started and ended while a load was in flight escaped the
stash path entirely: the load applied normally and replayed its
pre-drag kv snapshot, visibly reverting the just-saved reorder (and
permanently clobbering it after the next save). Conversely, the stash
path unconditionally preferred the local record, discarding a genuinely
fresher order fetched from another window when the drag committed
nothing.

Record the load-seq boundary at each saveOrder; a resolving load (both
apply paths, via _resolveOrderNames) replays its fetched kv order only
when it was issued after the latest local save — otherwise the
canonical in-memory saved list wins. Also rewrites the _applyPendingLoad
header, which described the pre-rank-merge implementation.

* Simplify app uninstall tile handling

Remove the uninstallability flag and modal warning logic, and treat uninstall as a local UI removal after permission revoke. This avoids an immediate reload that could re-add recommended/recent apps, while stale in-flight loads are still invalidated. Also simplifies app loading by dropping unused render options from `loadApps`/`_fetchAndRenderApps` and related merge comments.

* fix: match raw item paths in move_items cleanup and shortcut re-point

Item rows store data-path/data-shortcut_to_path unencoded, so the
html_encode()d attribute selectors missed names containing & < > " '
and the destination-row exclusion guard could remove a legitimate row
just created by a concurrent item.moved handler. Compare raw values
case-insensitively instead.

* fix: bound the Usage plan-button retry poll

The UIUpgradeAccount retry chain decayed to a 60s heartbeat but never
stopped, leaving self-hosted installs polling forever and pinning the
captured $el_window. Give up after ~30s of decaying retries; onActivate
re-checks on every return to the tab, so late script loads still get
the button.

* fix: drop the recent-opens list from the Apps tab grid

Recents are open history, not installs: they resurrected uninstalled
apps' tiles and showed merely-visited sites as if installed. The grid
is now recommended + installedApps; anything the user actually uses
still appears because opening an app grants it a permission. Recents
still power the Home tab.
2026-07-18 20:39:57 -07:00
jelveh 81c69f243b Align sign-up email button with OIDC styling 2026-07-18 09:35:13 -07:00
jelveh 6a3ecaff2a improve and unify OIDC button styling 2026-07-18 09:18:21 -07:00
jelveh 70f343d6fe Defer taskbar creation in fullpage mode 2026-07-17 14:38:06 -07:00
jelveh 172aa01e1c Trim file context menu actions 2026-07-17 14:23:52 -07:00
jelveh cba6ac8184 Clamp and normalize window progress values 2026-07-17 14:14:50 -07:00
jelveh 492f3d235a Add active label variant to context menu height rule 2026-07-17 13:54:12 -07:00
jelveh d9a01a3748 Add responsive item properties modal to Dashboard 2026-07-17 13:21:15 -07:00
Nariman Jelveh aab21ef91d don't record app open in iframe env 2026-07-17 11:42:48 -07:00
Daniel Salazar 2479e6a064 fix: FS permissions (#3404) 2026-07-17 11:01:34 -07:00
jelveh 7cba26bb39 Revert "fix(dashboard): make app drag-to-reorder work on touch (iOS)"
This reverts commit af0dc06f49.
2026-07-17 08:53:46 -07:00
jelveh af0dc06f49 fix(dashboard): make app drag-to-reorder work on touch (iOS)
On iOS/WebKit the pager's `touch-action: pan-x` made horizontal touchmove
non-cancelable, so the moment a long-press drag moved the finger the
browser started a native page-pan and fired pointercancel, aborting the
reorder before it visibly began. Preventing pointermove doesn't stop
native scrolling on iOS, and the reordering `touch-action: none` rule was
applied only after the drag began — too late, since touch-action is
latched at gesture start.

Drop the explicit pan-x (back to auto) so horizontal touchmove is
cancelable, and install a non-passive touchmove listener that
preventDefaults only once reordering is armed (post long-press). Before
arming it's a no-op, so a quick swipe still flips pages natively.
2026-07-17 08:47:58 -07:00
Nariman Jelveh 14900514b9 Dashboard: drag-to-reorder apps in the Apps tab (#3401)
* feat(dashboard): drag-to-reorder apps in the Apps tab

Let users arrange their My Apps tiles in any order via drag-and-drop.
The order is stored per-user in puter.kv (`dashboard_apps_order`) as a
list of app names and re-applied on load; apps installed since the order
was saved are appended, and uninstalled ones are dropped.

Interaction:
- Mouse/pen: drag past a small threshold to reorder; a plain click still
  opens the app.
- Touch: long-press to pick up a tile (a quick swipe still flips pages).
- Dragging to a scroller edge dwells, then flips to the adjacent page so
  tiles can move across pages; the floating drag image lives on <body> so
  the pager's overflow can't clip it, and displaced tiles reflow with a
  FLIP animation.
- Escape (or losing the window) cancels and reverts; reordering is
  disabled while a search filter is active; prefers-reduced-motion is
  honored.

Also fixes a latent pager bug where programmatic page changes (arrows,
dots, wheel) didn't update the active page indicator, because those
scrolls don't reliably emit 'scroll' events; goToPage now tracks the
page eagerly.

The order-reconciliation logic is extracted into a pure module
(appOrder.js) with unit tests.

* fix(dashboard): ignore stray pointers and no-op drags

- Filter drag pointermove/up/cancel by pointerId so a second finger can't
  hijack or prematurely end an in-progress reorder.
- Only persist a new order when the drop actually changed it, so an
  accidental long-press or drop-in-place doesn't freeze the default
  ordering into a custom one.

* fix(dashboard): address review of app drag-to-reorder

Correctness:
- Preserve the touch long-press → Uninstall path. A long-press now *arms*
  reordering instead of immediately grabbing the tile; moving begins the
  drag, while holding still lets the native context menu fire (which
  cancels the pending pickup). This restores the only touch route to
  Uninstall that the first cut had removed.
- Only swallow the post-drag click when the drop actually reordered
  something, so a small pointer drift or a drop-in-place still opens the
  app like a plain click.
- Guard the pre-start "intent" window: cancel a pending pickup before any
  re-render (ResizeObserver / loadApps), and bail out of _beginDrag if the
  pressed tile was detached, so a layout change mid-press can't reinsert a
  stale node and persist a corrupted order.

Cleanup:
- Reuse reconcileAppOrder for the drop path instead of a duplicate helper.
- Read the saved order in parallel with the app-list fetches.
- Cache the reduced-motion MediaQueryList instead of re-querying per move.
- Fold the touch-scroll cancel into _endDrag(false); drop _abortDragIntent.

* feat(dashboard): smoother, more tolerant app reordering

Make drag-to-reorder feel like the iOS home screen instead of snapping
and flickering:

- Slow the reflow to 320ms with a gentle spring easing.
- Hit-test against each tile's *resting* box, not its live
  getBoundingClientRect. Mid-animation a tile sits between slots, so
  testing its live box was dropping it under the pointer and swapping it
  straight back — the source of the back-and-forth jitter.
- Require the dragged icon's centre to be well inside a tile before it
  becomes the drop target (a deadzone around each tile), so hovering a
  boundary no longer thrashes.
- Probe with the dragged icon's centre rather than the fingertip, and
  batch the FLIP into a single reflow so large grids stay smooth.
2026-07-17 07:57:05 -07:00
Daniel Salazar 7faaef8cf0 tests: more tests for puter.js (#3402) 2026-07-16 22:35:15 -07:00
Nariman Jelveh c188257f85 Session Manager: compact redesign + responsive modal (no UIWindow) (#3400)
* redesign: compact, scannable Dashboard Session Manager

The session list rendered each entry as a bulky card with a 5-row
Created/Last active/Expires/IP/Client key-value table and a large filled-red
Revoke button, so only a couple of sessions fit on screen and destructive
actions dominated the layout.

Reworked it into an icon-led, compact list:
- One row per session: a device/kind icon tile (laptop / phone / globe by OS,
  bolt for workers, key for API tokens, the real app icon for app sessions),
  the title + badges, and two tight meta lines (client · IP, then a fainter
  last-active · created · expires with absolute-time tooltips).
- Destructive actions are calm: per-row Revoke and the rename pencil are quiet
  and reveal on hover (always shown on touch); "Revoke all other sessions" is a
  subtle ghost button instead of a filled block competing with search.
- Current session gets a green accent, tint, and pill, and has no revoke button.
- Toolbar search gains an inline icon + focus ring; added a session count line;
  restyled kind badges; nested child sessions get a proper tree line.

Styling uses the existing dashboard design tokens. All behavior is preserved:
search filtering, parent/child tree with expand/collapse, inline rename
(optimistic + rollback), per-session revoke (access-token vs session routing),
revoke-all, current-session guard, and focus/interval refresh.

Adds ui_session_count_one / ui_session_count_other to en.js (other locales
fall back to en).

* refactor: render Session Manager as a responsive modal, not a UIWindow

Replaces the draggable UIWindow shell with a self-contained DOM modal:
- Backdrop + centered card on desktop (max 680px / 90vh); full-screen sheet
  on phones (<=640px). Fade/scale in, respects prefers-reduced-motion.
- Adds a header bar (title + close), backdrop-click and Escape to close, and
  a scroll-contained body.
- Confirmations no longer use UIAlert (itself a UIWindow). Revoke, revoke-all,
  and the rename-error path now use in-modal confirm/alert sheets, so there is
  no cross-window z-index juggling — important because this can be opened from
  a stay_on_top window (UIWindowCopyToken).

Drops the UIWindow and UIAlert imports entirely. All behavior is preserved:
search, parent/child tree + expand/collapse, inline rename (optimistic +
rollback), per-session revoke (access-token vs session routing), revoke-all,
current-session guard, and the focus/60s-interval refresh (cleaned up on close).
2026-07-16 20:48:54 -07:00
Nariman Jelveh 4543a645f2 fix: verified Dashboard bugs (10 separate fixes) (#3399)
* fix: prevent crash rendering website files without a workers array in Dashboard Files

TabFiles.renderItem accessed `file.workers.length` unguarded when deciding
whether to show the website badge. Directory entries that have a published
website but no `workers` array (the common case) made this throw
`Cannot read properties of undefined (reading 'length')`, so the row failed
to render — the file silently disappeared from the list (swallowed by the
Promise.allSettled batch) or was never added on socket-driven updates.

The sibling `is_worker` derivation on the same entry already guards with
`file.workers?.length`, and the desktop reference (UIItem.js) normalizes
`workers` to an array and shows the badge via `!is_worker`. Reuse the safe
`is_worker` flag here instead of touching `file.workers.length` directly.

* fix: correct Dashboard usage that showed negative spend for users with credits

The Usage tab and Home usage card derived month-to-date spend as
`monthUsageAllowance - allowanceInfo.remaining`. But the metering service
computes `remaining = remainingAllowance + remainingPurchasedCredits`
(MeteringService.getAllowedUsage), so `remaining` exceeds the monthly
allowance whenever a user holds purchased credits — making the displayed
spend and percentage negative (e.g. "-$9.90 used of $0.25", "-3960%").

Use the actual reported spend (`res.usage.total`, already returned by the
same endpoint) and clamp the percentage/bar to 100%. For a free user with no
credits and no overage this yields the identical value as before.

* fix: make Dashboard Files Download button work for a single selection

In mobile select mode the floating action bar appears with one item selected
(minCountForActionBar is 1) and shows an enabled Download button, but the
click handler only acted when 2+ rows were selected — so downloading a single
file did nothing.

Gate on `> 0` instead. window.zipItems already normalizes a single item to an
array, so the one-file case works. On desktop the bar only appears at 2+
selections, so this changes nothing there.

* fix: auto-fit column width in Dashboard Files no longer scales with row count

Double-clicking a column resize handle is meant to fit the column to its
longest cell. The per-row loop did `maxWidth = Math.max(maxWidth + 10, textWidth)`,
which forces `maxWidth` to grow by at least 10px on every row regardless of
content — so a folder with N files produced a column ~10*N px wide and
persisted that runaway width to KV storage.

Compare against the running max only (`Math.max(maxWidth, textWidth)`);
`textWidth` already includes cell padding.

* fix: clear empty-directory placeholder when a file is added incrementally

When a directory is empty, renderDirectory appends an absolutely-positioned
"No files in this directory." placeholder. If a file then appears via an
incremental path (socket item.added/item.moved -> UIDashboardFileItem), the
new row was inserted but the placeholder was never removed, so the "No files"
text overlapped the real item. insertAtSortedPosition can't clear it because
the placeholder has no `.item.row` class.

Remove the placeholder in UIDashboardFileItem before inserting the row,
matching what the in-app New Folder / New File flows already do.

* fix: stop leaking a document keydown handler per uninstall-modal dismissal

showUninstallModal bound `keydown.uninstall-modal` on document but only
detached it on the Escape and Confirm paths. Dismissing via Cancel or a
backdrop click called `close()`, which just removed the overlay and left the
handler attached (closing over a now-detached overlay). Each open→cancel
cycle stacked another document-level listener for the page lifetime.

Move the `.off('keydown.uninstall-modal')` into `close()` so every dismissal
path cleans up, and drop the now-redundant explicit detaches.

* fix: resolve 2FA setup promise on every close so Dashboard toggle can't freeze

UIWindow2FASetup only resolved its promise from the Done button and, in
on_before_exit, when setup had NOT succeeded. Clicking "Enable" sets
setup_succeeded=true and advances to the recovery-codes screen without
resolving; if the user then closes that screen via the backdrop or Escape
instead of Done, the promise never settles.

The Dashboard Security tab is the only caller: it disables the 2FA toggle and
`await`s this promise, so the hang leaves the toggle permanently disabled and
stuck in the pending state for the rest of the session. Resolve with
setup_succeeded on any exit; resolve_promise is idempotent so a prior
Done-resolve is unaffected.

* fix: refresh Dashboard tabs when navigating via browser back/forward

The sidebar-click and initial-route handlers both call the target tab's
onActivate (its refresh hook), but handleRouteChange — wired to hashchange
and popstate — only swapped the active classes. Since tab switches use
history.pushState (which doesn't fire popstate/hashchange), the only way to
reach handleRouteChange is browser back/forward, and doing so left tabs like
Home/Apps showing stale data.

Look up the tab by id and call onActivate after switching, matching the
sidebar-click path.

* fix: refresh Home "Recently used" apps instead of freezing them for the session

loadRecentApps only hit /get-launch-apps when window.launch_apps.recent was
empty. Once populated (typically at page load), every subsequent init/onActivate
/focus just re-rendered the same cached array, so apps launched during the
session never showed up under "Recently used" until a hard reload.

Drop the empty-list guard so the list refetches on the same triggers that
already refresh the usage cards. The fetch can only make the list fresher.

* fix: sync window.user.otp when enabling 2FA from the Dashboard

The disable path clears window.user.otp (in UIWindowDisable2FA), but the
enable path updated only the card's DOM, never the in-memory flag. TabSecurity
html() reads user.otp to decide the toggle/badge state, so after enabling 2FA
a re-render of the Security tab from the cached window.user showed it as
disabled for an account that actually had 2FA on.

Set window.user.otp = enabling on success so both directions stay consistent.
2026-07-16 20:03:38 -07:00
jelveh 75d3b07e0f fix: phantom shortcut badge on files created via dashboard New menu
renderItem() showed the shortcut badge whenever file.is_shortcut !== 0.
Directory listings return is_shortcut as 0, but the puter.fs.upload()
result used to incrementally render newly-created items omits the field,
so undefined !== 0 was true and every new file got a phantom shortcut
badge. Normalize is_shortcut to 0/1 before using it.
2026-07-16 19:05:28 -07:00
Nariman JelvehandClaude Opus 4.8 9da32c73cb Restore Files tab in dashboard (#3397)
* Restore Files tab in dashboard

Reinstates the Files tab (TabFiles) that was removed in 973d046d, bringing
back the dashboard file browser: directory navigation, list/grid views,
sorting, drag-and-drop, context menus, upload, and trash support.

- Restore src/gui/src/UI/Dashboard/TabFiles.js (identical to pre-removal)
- Re-wire UIDashboard.js: tab registration, initial file path, item socket
  handlers, and route-based file navigation
- Re-add file-tab CSS in dashboard.css and the selection-area rule in style.css
- Re-add dashboard-mode item creation hooks in helpers.js move_items
- Re-add #files/<path> deep-link parsing in initgui.js parseDashboardRoute,
  while preserving Apps as the default dashboard tab

* Only remove stale rows when moving items

* Fix Dashboard Files tab bugs

- Escape filenames in renderItem to prevent stored XSS via HTML in file
  names (matches UIItem); raw name is still preserved for display/rename.
- Read clone data-id from the nested .row in drag drop handlers so
  multi-select drag moves all selected items, not just the grabbed one.
- Resolve sidebar drop target from the element's data-path so dragging
  onto Public/Home works (they aren't keyed in user.directories); fix the
  matching UID-vs-path comparison in the drag-out handler.
- Set data-is_worker/data-worker_url from is_worker itself instead of an
  always-true "!== undefined" check, so worker context-menu behavior only
  applies to actual workers.
- Give delete-confirmation buttons explicit values so permanent delete
  works in non-English locales.
- Refresh the view after a context-menu move-paste, mirroring Ctrl+V.
- Guard the item.updated socket handler against the client's own echo,
  matching the other handlers and UIDesktop.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* Prevent Files tab lockup when readdir fails

renderDirectory sets renderingDirectory=true then awaits puter.fs.readdir
with no error handling. readdir rejects on any backend error (permission,
deleted directory, network), leaving renderingDirectory stuck true so the
re-entry guard blocks all future navigation and the spinner never clears.
Catch the rejection and reset state.

* Isolate render failures so one bad entry can't lock the Files tab

renderItem parsed file.metadata with an unguarded JSON.parse, but metadata
is client-writable and stored verbatim, so '', undefined, or malformed
values throw (item_icon.js guards the identical call). Combined with
Promise.all, one bad entry rejected the whole render and left the tab
frozen. Guard the parse and use Promise.allSettled for per-item isolation.

* Clear selection when clicking empty area in Files tab

The empty-area click handler is an arrow function, so `this` was the
TabFiles object rather than the container element, making `e.target === this`
always false. Only clicks landing exactly on `.files` deselected. Use
e.currentTarget so clicks on the container background deselect as intended.

* Avoid stacking image-preview document click handlers

showImagePreview bound a click.imagepreview handler without removing a
prior one. It is re-invoked during image arrow-navigation while a preview
is open, so identical document handlers accumulated until close. Remove
any existing handler before binding.

* Sort live-inserted items by display name

insertAtSortedPosition compared the new item's raw file.name against
existing rows' data-name (the display name). For trashed items, name is
the UID while the real name is in metadata.original_name, so newly trashed
items landed in the wrong position in Trash until a full re-render. Compare
on the new row's data-name to match sortFiles and the existing rows.

* Use display name in in-place item update

UIDashboardFileItem's in-place update path set data-name and the shown
name from raw file.name, unlike renderItem which uses
metadata.original_name || file.name. An update to an already-visible
trashed item replaced its name with the UID and corrupted data-name
(which sort and type-to-search read). Compute the display name to match.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-16 18:57:19 -07:00
Daniel Salazar 102cef523f fix: app id whitelisting (#3398) 2026-07-16 18:03:34 -07:00
Daniel SalazarandClaude Fable 5 0e1be72f92 test: tests for puter.js (#3396)
* test: tests for puter.js

* fix: ship lockfile for coverage devDeps; tolerate missing base coverage

npm ci failed on CI because package.json gained the babel/istanbul
devDependencies without the matching package-lock.json update. Also make
the coverage workflow's base leg best-effort so a base ref that predates
the coverage script reports without the comparison column instead of
failing the run.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-16 15:52:37 -07:00
Neal Shah ef1afb9cc1 add just app telementry (#3387) 2026-07-16 14:04:54 -04:00
Neal Shah 0462ddd6f5 add support for step-up sessions (#3395)
* add support for step-up sessions

* update step up session
2026-07-16 13:58:49 -04:00
Neal Shah 8d7e70cde4 add kimi k3 (#3394) 2026-07-16 12:47:44 -04:00
Daniel Salazar 52e481128f wip: puter js tests structure (#3393) 2026-07-15 19:09:35 -07:00
Daniel Salazar c237e2433f fix: clean apps on subdomain deletion (#3392) 2026-07-15 13:38:53 -07:00
tamilselvanp57 a4bb57bb79 Fix static hosting domain fallback for self-hosted custom domains (#3382) 2026-07-15 11:45:10 -07:00
Daniel Salazar 861cf5efb8 feat: email api wip (#3391) 2026-07-15 11:30:16 -07:00
Daniel Salazar cf34f9fea9 fix: sanitize get user response (#3390)
* fix: sanitize get user response

* fix: app creation in dev center
2026-07-15 02:02:06 -07:00
jelveh 92a6d30e26 Add calendar to recommended apps 2026-07-14 23:04:24 -07:00
Neal ShahandDaniel Salazar 23e8705b03 Implement workers without cloudflare for local dev testing (#3389)
* typeify subdomains wip

* initial (untested) logic for LocalWorkerService

* Make it work, add lifecycle expiry since workers are process heavy in current implementation

* fix type errors

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-15 01:01:42 -04:00
Daniel Salazar 3240a4670a fix: admin gates (#3386) 2026-07-14 12:58:04 -07:00
Daniel Salazar 3bf83d40e9 fix: ui signup order (#3385) 2026-07-13 18:47:11 -07:00
Neal Shah c3368f553e Add GUI button for custom OIDC (#3384) 2026-07-13 20:32:37 -04:00
velzie 27def94d8b feat: support cross-origin-isolated login (#3338) 2026-07-13 17:36:56 -04:00
Neal Shah 2c139929ac Check if user is suspended on WebDAV basic auth and others (#3369) 2026-07-13 02:46:28 -04:00
Neal Shah b957f797da add support for ndjson logging (#3381)
* add support for ndjson logging

* move install location
2026-07-13 02:43:56 -04:00
jelveh 393500d565 Add calculator to recommended apps
Include 'calculator' in RECOMMENDED_APP_NAMES within RecommendedAppsService.ts so the calculator app is included in the recommended apps list.
2026-07-11 23:04:39 -07:00
jelveh a108837e82 Revert "Add delayed centered spinner for apps tab"
This reverts commit 859e88753f.
2026-07-11 15:21:59 -07:00
jelveh 859e88753f Add delayed centered spinner for apps tab
Show a subtle centered loading spinner for the Apps dashboard when fetching apps if the request is slow. Implements a 1.5s delay before showing the spinner and guarantees a minimum 1.5s visible time to avoid blinking; the spinner is only shown when the container is empty (so background refreshes keep existing tiles). Adds a finish helper to clear the spinner timer and await the minimum visible time before rendering results or errors. Also adds CSS for the spinner element and its animations.
2026-07-11 15:16:04 -07:00
Daniel Salazar 5160b44c4b fix: missing OIDC error messages (#3377)
* fix: missing OIDC error messages

* fix: card verification user prefill out for subs
2026-07-11 08:18:13 -07:00
Daniel Salazar 17a9485fb2 Add spans to drivers and high frequency paths (#3376) 2026-07-10 23:15:09 -07:00
Daniel Salazar d3c44f3d6c fix: cache healthcheck in mem for each node (#3375) 2026-07-10 17:41:09 -07:00
Neal Shah d49ba5281d add support for ignoring specific issues in healthcheck (#3374)
* add support for ignoring specific issues in healthcheck

* add mark-degraded to healthcheck
2026-07-10 17:08:03 -04:00
jelveh c107c80bd8 Open dashboard links at #home anchor 2026-07-10 13:22:58 -07:00
Daniel Salazar cf90e775bf fingerprint: use device info for rate limit too (#3372) 2026-07-09 20:22:56 -07:00
Daniel Salazar 0a46096c41 fix: ai rate limits (#3371) 2026-07-09 16:35:03 -07:00
Daniel Salazar 289a44ae79 fix: OIDC state (#3370) 2026-07-09 14:47:34 -07:00
Neal Shah 827e9bc343 fix possible XSS entrypoints (#3368) 2026-07-09 17:19:55 -04:00
Daniel Salazar f2ffaeb823 fix: add phone errors into kv for debugging (#3367) 2026-07-09 12:25:17 -07:00
Neal Shah 6f594f4b60 add 5.6 models (#3366) 2026-07-09 14:46:57 -04:00
jelveh 326be042f1 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-08 22:33:20 -07:00
jelveh 5b824fb745 Extract resolveTileDisplay and broaden app search
Move logic that determines a tile's displayed title and target link into resolveTileDisplay (use hostname/index_url for external apps) and have buildTileHtml consume it. Expose data-target-link on tiles. Update app search to match the displayed title, raw title, name, and uid so queries find website-shortcut tiles and external apps consistently (matching Home tab behavior).
2026-07-08 22:33:13 -07:00
Reynaldi Chernando cf6cb126b5 fix grok 4.5 release date (#3365) 2026-07-09 12:30:39 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 302df1edd2 update user pays model docs (#3364)
* update user pays model docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-09 08:54:04 +07:00
jelveh e212509ed3 Avoid clipping scaled icons in horizontal scroller
Add a negative top margin and matching top padding to the horizontal scroller to prevent first-row icons from being clipped when they scale on hover. Overflow-x:auto previously caused vertical clipping of scaled icons; margin-top:-8px and padding-top:8px bleed the clip edge upward so scaled icons remain visible without changing scroll behavior.
2026-07-08 18:35:57 -07:00
Daniel Salazar d3e3755d6d chore: cleanup uneeded ai messages (#3363) 2026-07-08 17:55:49 -07:00
jelveh 948ce6e529 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-08 17:51:38 -07:00
jelveh d117dc231e Implement paged apps launcher UI 2026-07-08 17:50:34 -07:00
Daniel Salazar eefa53e5bd fix: misc auth bug fixes (#3362) 2026-07-08 17:50:22 -07:00
Neal Shah db89fbfc52 add grok 4.5 (#3360) 2026-07-08 14:44:05 -04:00
Daniel Salazar 97ec5d115c fix: double counting purchased credit usage (#3359) 2026-07-07 21:43:09 -07:00
Daniel Salazar 722a6cf50f fix: phone retry limit messagin (#3358) 2026-07-07 21:05:41 -07:00
Neal Shah d039dcfee5 fix appTelemetry (#3357) 2026-07-07 20:10:13 -04:00
Daniel Salazar d09aa11f2e feat: add whatsapp support? (#3356) 2026-07-07 16:13:18 -07:00
jelveh 8a2264546d Always use hash for dashboard tabs
Previously the dashboard used a clean root URL for the 'apps' tab and hashes for other tabs. This change makes tab links and history entries consistently use a hash (e.g. #apps, #home) by always generating href as `#<id>` and pushing `#<section>` to history. Simplifies URL handling and ensures the Apps tab also reflects its hash in the address bar.
2026-07-07 15:16:59 -07:00
0801e1dc44 feat: add signup disable config and fix telemetry startup (#3319)
* feat: add signup disable config and fix telemetry startup

* fix: nits for config spread and 403 checks

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-07 13:52:29 -07:00
Daniel Salazar a04d5fa5e0 fix: package-lock (#3355) 2026-07-07 13:43:15 -07:00
e1280e8cc7 Add custom icon support for web links (#3325)
* feat: add custom web link icon support and context menu option

Introduce the ability to change and display custom icons for .weblink files. Added a "Change Icon" context menu item that invokes the changeWeblinkIcon helper, and updated the icon rendering logic to use getWeblinkIcon instead of the default link icon. This allows users to personalize web link appearance in the file manager.

* Fix security and robustness issues in web link custom icons

 # UIWindowSearch and require the icon data URL body to be pure base64 in
  #isValidWeblinkIcon. A shared/downloaded .weblink is untrusted input and
 # the previous prefix-only check let a crafted icon break out of the
 # <img src> attribute.
 # byte-signature MIME sniffing and capping stored icon size (was storing
 # full-resolution images as base64).
 # without picking a file.
 # weblinks no longer fetches every file's contents on each render.
 # path instead of the target weblink).
 # label, add the AGPL license header, and drop redundant stored icon
 # copies.

* Keep SVG weblink icons as vectors instead of rasterizing

Rasterizing a picked SVG to a 256px PNG threw away its scalability for no
real benefit: an SVG rendered in <img> runs in script-free static mode, and
the base64-body validation already prevents attribute injection. Store SVGs
as data:image/svg+xml;base64 (content-sniffed, size-capped), and keep
rasterizing raster formats and oversized SVGs to bound stored size.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-07 13:41:37 -07:00
Aman Rai 70884c2e30 fix: rename start-gui script to start:gui (#3346) 2026-07-07 12:09:49 -07:00
Reynaldi Chernando 917e95e3d7 Add nano banana 2 lite (#3336) 2026-07-07 12:08:36 -07:00
jelvehandClaude Opus 4.8 27222bbb0d Let ctrl/cmd+f use browser find while dashboard is open
The global ctrl/cmd+f shortcut opened UIWindowSearch and preventDefault'd
the keystroke, blocking the browser's native find. Skip that interception
while a dashboard window is open so ctrl/cmd+f falls through to the browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-07 09:43:34 -07:00
jelveh 3720034464 Make Apps the default dashboard tab
Treat the Apps tab as the default (root URL) instead of Home. UIDashboard now reads an initial route from window.dashboard_initial_route?.tab and falls back to 'apps' for unknown/absent routes. Sidebar and section active class logic and tab href/hash generation were updated so 'apps' uses the clean root URL while all other tabs (including home) use #tab. The history.pushState logic was adjusted accordingly. parseDashboardRoute in initgui.js was updated to default to 'apps' and its comment clarified the new behavior.
2026-07-07 09:31:43 -07:00
jelveh de6cab1f9a Reduce myapps tile icon size to 56px
Update .myapps-tile-icon in src/gui/src/css/dashboard.css to change width and height from 60px to 56px, adjusting tile icon sizing for improved layout and spacing.
2026-07-06 21:04:32 -07:00
jelveh 2d1fbc2611 Auto-focus My Apps search on activation
Call focusSearch when the Apps dashboard is activated and add a focusSearch helper. The helper imports isTouchPrimaryDevice and skips focusing on touch-primary devices to avoid showing the on-screen keyboard. It retries up to 12 times (30ms interval) until the search input is visible to handle cases where the dashboard is briefly hidden while entering full-page mode.
2026-07-06 20:10:44 -07:00
jelveh 0afc70218f Use input[type=search] and normalize styling 2026-07-06 20:03:48 -07:00
jelveh 8d3abbc5f0 Increase myapps tile icon size to 60px
Update .myapps-tile-icon width and height from 52px to 60px to improve icon visibility and better match the updated UI design. Other styles (overflow, background, transitions) are left unchanged.
2026-07-06 19:47:17 -07:00
jelveh 7ec14bc7db Add audio-editor and ai-image-project to recommended apps 2026-07-06 19:42:20 -07:00
jelveh 2843010a97 Increase myapps tile size to 100px 2026-07-06 19:37:46 -07:00
jelveh ad4e21ee61 Open external apps via index_url from Apps tab
When an app is external and provides an index_url, use its hostname as the tile title and store the index_url in a new data-target-link attribute. Update the click handler to open the target link in a new tab for external apps, and fall back to opening the internal /app/{name} page for others. This aligns Apps tab behavior with the Home tab and ensures external apps navigate to their website.
2026-07-06 19:21:34 -07:00
Nariman Jelveh 4de39ab6f2 Mark external apps and use the flag for dashboard titles (#3354)
* Mark external apps and use the flag for dashboard titles

An app with no owner_user_id (null/empty) isn't owned by a Puter user —
it's an external, origin-bootstrapped app whose uuid/name/title are all the
opaque app-… id. The dashboard used to detect these client-side by comparing
uuid/name/title and a name.startsWith('app-') check.

Expose an authoritative `external` flag from the API instead:

- /installedApps and /get-launch-apps now return `external`, derived from
  owner_user_id, and no longer leak the raw owner id.
- The dashboard (Home + Apps tabs) shows the index_url hostname for external
  apps based on `external` rather than the uuid/name/title heuristic.

Add/extend extension tests to cover the `external` flag and non-leak.

* Use hostname for opaque external app titles

Detect opaque external app IDs (when name === title === uid/uuid) and replace the displayed title with the hostname from index_url. Adds a uid/uuid fallback and tightens the external-app check in both the Apps and Home dashboard tabs so they behave consistently; preserves target_link for Home entries.

* Add external flag for apps without owner

Expose an `external` property in AppSummary (toAppSummary in RecommendedAppsService.ts) to mark apps that are not owned by a Puter user. The flag is set when `owner_user_id` is null or an empty string, allowing clients to identify origin-bootstrapped/external apps.
2026-07-06 18:00:56 -07:00
Nariman Jelveh fbb646b8eb Return index_url from /installedApps for dashboard app titles (#3352)
The dashboard derives a hostname title for anonymous (app-…) apps whose
uuid/name/title are all identical, falling back to the host of index_url.
That worked for launch/recent apps but not for apps that come only from
/installedApps, because the endpoint didn't return index_url.

Add apps.index_url to the SELECT (alongside apps.icon, relying on the
functional dependency on the apps.id primary key) and assert it in the
extension test.
2026-07-06 16:46:31 -07:00
Neal Shah 46f75ecd6f add type for puter.app.authenticated (#3353) 2026-07-06 18:38:30 -04:00
Daniel Salazar f68caa3bbd feat: allow optional card fallback for failed phone verifications (#3351) 2026-07-06 14:59:55 -07:00
jelveh 0109bf5b72 Use index_url hostname for anonymous app title
Detect anonymous apps whose name/title/uid are identical opaque ids (starting with 'app-') and, when index_url is available, replace the displayed title with the index_url hostname (matching the Home tab behavior). Also persist index_url into the mapped app object and make title a mutable variable to support the change.
2026-07-06 14:43:39 -07:00
Nariman Jelveh c69fb52d5c Make the dashboard the default interface at root (#3345)
* feat: make the dashboard the default interface at root

The dashboard now loads at / (with /dashboard kept as an alias), and the
desktop moves to a new /desktop route. An "Open Desktop" button pinned at
the bottom of the dashboard sidebar switches to the desktop.

Root URLs that carry a desktop-only flow keep booting the desktop:
auth popups (?embedded_in_popup=), app deep links (?app=), direct
downloads (?download=), fullpage mode (?puter.fullpage=), and iframe
embeds.

Desktop-context redirects are retargeted so desktop users stay on the
desktop: logout, the QR "continue on phone" link, the auth cover-page
logo click, and window close/hide URL resets now use /desktop. OIDC
login/signup gains a strictly whitelisted return_to param (/desktop or
/dashboard) so social login returns to the interface it started from.

Also: the dashboard strips ?ref= like the desktop does, ?auth_token=
cleanup preserves the path and tab hash, the logged-out session list
gets the wallpaper back, ?download= also works at /desktop, and the GUI
dev server serves /desktop and /dashboard.

* Add Open Desktop bento card and CSS

Replace the previous desktop-switch UI with a full-width "Open Desktop" bento card that links to /desktop. Removes the pinned sidebar Open Desktop button and the old desktop-switch button/handler, and adds new CSS variables and styles for the desktop card (icon background, shadow, surface/header colors), hover shadow and arrow transition, and layout/padding adjustments.

Changed files: TabHome.js (removed old switch, added anchor card), UIDashboard.js (removed sidebar button), dashboard.css (new desktop color vars and styles; removed legacy desktop-switch/button styles).

* Remove box-shadow from myapps search input

Delete the box-shadow declarations from input.myapps-search::placeholder in dashboard.css. This simplifies the placeholder styling (removes subtle and light shadow layers), likely to reduce visual noise or fix rendering inconsistency.

* Show index_url hostname for opaque app IDs

Only replace an app's title with the hostname from index_url when the app appears to use an opaque id. Add a check for names starting with 'app-' so the hostname is used only if title, name and uuid are identical and the name looks like an opaque app id. This avoids overwriting meaningful user-facing names that may coincidentally match the uuid and clarifies the intent in the comment.

* Update dashboard.css
2026-07-06 13:22:27 -07:00
Neal Shah 3a26f1f1c2 add admin moderation events table (#3348) 2026-07-06 11:49:09 -04:00
Daniel Salazar f1b94a5b69 fix: otel upgrade bump (#3328) 2026-07-06 08:42:25 -07:00
Neal Shah e3fce9191a emit puter.app.authenticated event (#3337) 2026-07-06 11:23:27 -04:00
Nariman Jelveh 2c0a1b9bbb Revert "feat: add sign-in flow for crossOriginIsolated contexts (#3253)" (#3335)
This reverts commit 6fff1fd75f.
2026-06-30 23:32:27 -07:00
jelveh 0d7417f858 Add 'contacts' to recommended apps
Include 'contacts' in the RECOMMENDED_APP_NAMES array so the Contacts app appears in the recommended apps list. Updated src/backend/services/apps/RecommendedAppsService.ts.
2026-06-30 19:08:11 -07:00
Neal Shah 2eee098c66 Add suspension reasons (#3331) 2026-06-30 18:09:00 -04:00
Neal Shah 890d4b3c16 remove temperature from sonnet-5 (#3330) 2026-06-30 17:18:29 -04:00
Neal Shah af9a7c6345 add claude sonnet 5 (#3329) 2026-06-30 17:08:42 -04:00
velzie 6fff1fd75f feat: add sign-in flow for crossOriginIsolated contexts (#3253)
* poc

* route login key through broadcastservice

* cleanup

* security validation and prevent additional webhook sends

* block double-broadcast from redis
2026-06-30 20:17:53 +01:00
velzie e6b7b08bda Add forceRelay option to Peer API and document raw peerconnection property (#3313)
* add forceRelay options

* document peerconnection
2026-06-30 12:28:49 -04:00
Daniel Salazar f4c3cbd53d feat: add logs for failed sms (#3318) 2026-06-30 06:50:47 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI c0c2f35266 Update docs model drift (#3326)
* Update docs model drift

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* standardize

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-30 16:55:42 +07:00
Nikhil Kumar SinghandDevblaze14 ccea980753 feat: add Ideogram 4.0 Together AI image model (#3309)
Co-authored-by: Devblaze14 <[email protected]>
2026-06-29 15:33:01 -04:00
Neal Shah 2bddebd576 add ability to suspend apps (#3322) 2026-06-29 13:06:08 -04:00
Neal Shah b9866ddbeb add suspended_at col (#3321) 2026-06-29 09:51:00 -04:00
Mochammad Fadhlan Al-Ghiffari 6fc79a7184 fix(docs): highlight code blocks after client-side navigation (#3287) (#3288)
* fix(docs): highlight all code blocks after client-side navigation

The docs site highlights code on full page load via an inline
hljs.highlightAll(), which targets the 'pre code' selector and so
highlights every fenced block, including those without a language tag.

The client-side navigation path (pathchange handler) instead selected
only code[class^="language"], so fenced blocks written without a
language hint were left unhighlighted after in-app navigation while
appearing correctly on a hard reload.

Align the pathchange handler with highlightAll() by selecting 'pre code'
and skipping blocks hljs has already processed (data-highlighted=yes),
keeping the pass idempotent across repeated pathchange events.

Fixes #3287

* fix(docs): hoist hljs.configure and scope highlight to docs content

Address review nits on the pathchange highlight pass:
- Move hljs.configure({ ignoreUnescapedHTML: true }) out of the per-element
  loop since it is global config and only needs to run once.
- Scope the selector to '.docs-content pre code' so the re-highlight pass
  only touches the swapped-in docs body and leaves code blocks elsewhere
  on the page untouched.
2026-06-28 23:27:07 +07:00
Nariman Jelveh 50039571b6 Reorder recommended apps and add vault
Adjust RECOMMENDED_APP_NAMES ordering: place 'editor', 'camera', and 'recorder' immediately after 'builder' and move 'app-center' accordingly. Also add 'vault' to the recommended apps list. This changes the resolved ordering of recommended apps at call time.
2026-06-27 17:55:52 -07:00
Daniel Salazar 1216f20567 fix: package-lock issue (#3316) 2026-06-27 14:33:11 -07:00
Daniel Salazar 0752ea761c feat: remove stych, add prelude client (#3312) 2026-06-27 14:07:52 -07:00
Daniel Salazar 98461088f8 fix: decrease puts for unused stamps (#3310) 2026-06-26 12:33:04 -07:00
Neal Shah d4a3224aa3 force locale en-US (#3311) 2026-06-26 15:31:58 -04:00
Daniel Salazar 197ce6faff fix: LegacyFSController (#3308) 2026-06-25 14:47:29 -07:00
MRB Labs c98e293879 fix: unlink OIDC providers when user updates email address (#3307) 2026-06-25 14:46:41 -07:00
Miika Kuisma ebbe382596 tweak context menu's text labels (#3306) 2026-06-25 09:46:05 -07:00
Neal Shah 01d28eb2c5 Await phone verification event (#3305) 2026-06-24 20:08:30 -04:00
Daniel Salazar 7d1c2e7f0a fix: fsService copy (#3304) 2026-06-24 16:32:34 -07:00
Daniel Salazar 22b499af20 fix: cleanup fs responses (#3303) 2026-06-24 14:35:58 -07:00
Neal Shah 7208f86723 bind btoa and atob in emulated worker globalspace (#3302)
* bind btoa and atob in emulated worker globalspace

* Create working directory for workers
2026-06-24 15:56:26 -04:00
Neal Shah 2974a8b01c store app owner ID even when worker is deployed from root ctx (#3301) 2026-06-24 15:21:48 -04:00
Daniel Salazar 7f05a707bd feat: improve kv store incr/decr costs (#3300) 2026-06-24 11:57:36 -07:00
Miika Kuismaandjelveh 1d056253ef contextmenu design tweak (#3298)
* contextmenu design tweak

* icon gap fix (mobile context menu web component)

* fix submenu item padding

* add dark theme support for puter.ui.contextMenu and puter.ui.setMenubar (works only with puter.env === 'web')

* Add theme test & refactor context menu items

Extract shared mediaMenuItems to remove duplication and simplify context menu calls. Add a light-theme context menu handler and a "Run theme test" button that executes runContextMenuThemeTest(), which programmatically renders menus to verify the theme attribute and .puter-theme-dark class for dark, light, and default cases (mirrors existing Playwright assertions). Update testContextMenu and testContextMenuDark to use the new item factory and clean up rendered menus after checks. UI text is updated to show pass/fail results.

---------

Co-authored-by: jelveh <[email protected]>
2026-06-24 11:30:16 -07:00
Neal Shah a22dc5facb fix whoami user token check (#3299) 2026-06-24 14:28:02 -04:00
Neal Shah aeafd4ac79 txt2img megaupdate (#3294)
* Update OpenAIImageProvider with txt2img with input source

* Grok image provider mega update

* make input_images universal
2026-06-24 10:24:09 -04:00
Daniel Salazar 14861df6b2 fix: bad boot error (#3295) 2026-06-24 00:33:38 -07:00
Daniel Salazar 34a7595a6d fix: bad empty file error handling (#3293) 2026-06-23 16:13:41 -07:00
Daniel Salazar b2108c0216 fix: update email for supprot (#3292) 2026-06-23 15:43:12 -07:00
Philip Ifeanyi 5a5a449210 feat: enhance playground UI with page title and style adjustments (#3289) 2026-06-23 22:45:18 +07:00
2547 changed files with 623937 additions and 70563 deletions

No files matched your search

+7 -2
View File
@@ -2,9 +2,9 @@
# docker-compose.full.yml up -d`. None of the defaults below are safe for
# anything beyond a local laptop test.
# ── Public-facing ports (nginx) ---------------------------------------
# ── Public-facing ports (Caddy) ---------------------------------------
HTTP_PORT=80
# HTTPS_PORT=443 # uncomment after you enable TLS in nginx/nginx.conf
# HTTPS_PORT=443 # uncomment after you enable TLS in caddy/Caddyfile
# ── MariaDB ------------------------------------------------------------
MARIADB_ROOT_PASSWORD=replace-with-strong-password
@@ -16,3 +16,8 @@ MARIADB_PASSWORD=replace-with-strong-password
S3_ACCESS_KEY=puter
S3_SECRET_KEY=replace-with-strong-secret
S3_BUCKET=puter-local
# ── Inbound SMTP (compose profile `smtp`) ------------------------------
# Host port mapped to the receiver's 2525. 25 is what an MX record means;
# many hosting providers block it.
SMTP_PORT=25
+72
View File
@@ -0,0 +1,72 @@
# Default owner, including backend and SDK areas without a specialist below.
# The last matching rule takes precedence.
- @Salazareo
# GUI
/src/gui/ @jelveh @Salazareo
# Documentation, developer site, and CLI
/doc/ @reynaldichernando @Salazareo
/src/docs/ @reynaldichernando @Salazareo
/src/dev-center/ @reynaldichernando @Salazareo
/src/cli/ @reynaldichernando @Salazareo
# Backend AI
/src/backend/controllers/puterai/ @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/ai-\*/ @404oops @ProgrammerIn-wonderland @Salazareo
# Backend: AI, hosting, workers, networking, and WebDAV
/src/backend/controllers/hosting/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/webdav/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/wisp/ @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/util/ @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/workers/ @ProgrammerIn-wonderland @Salazareo
/src/backend/services/localworker/ @ProgrammerIn-wonderland @Salazareo
/src/backend/stores/subdomain/ @ProgrammerIn-wonderland @Salazareo
/extensions/workerSandbox\* @ProgrammerIn-wonderland @Salazareo
/src/worker/ @ProgrammerIn-wonderland @Salazareo
/src/worker-types/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/peer/ @ProgrammerIn-wonderland @Salazareo
# Backend: sharing, permissions, and teams
/src/backend/controllers/share/ @jfcastro92 @Salazareo
/src/backend/controllers/team/ @jfcastro92 @Salazareo
/src/backend/services/share/ @jfcastro92 @Salazareo
/src/backend/services/permission/ @jfcastro92 @Salazareo
/src/backend/services/acl/ @jfcastro92 @Salazareo
/src/backend/services/team/ @jfcastro92 @Salazareo
/src/backend/services/apps/AppPermissionService\* @jfcastro92 @Salazareo
/src/backend/stores/share/ @jfcastro92 @Salazareo
/src/backend/stores/permission/ @jfcastro92 @Salazareo
/src/backend/stores/group/ @jfcastro92 @Salazareo
/src/backend/stores/team/ @jfcastro92 @Salazareo
# SDK: AI, hosting, workers, and networking
/src/puter-js/src/modules/ai/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/hosting/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Workers* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/networking/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/lib/networkUtils* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/ai.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/hosting.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/workers.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/net.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Peer\* @ProgrammerIn-wonderland @Salazareo
# SDK: sharing and permissions
/src/puter-js/src/modules/perms/ @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/share* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/unshare* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/getShares* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/listShared* @jfcastro92 @Salazareo
/src/puter-js/tests/api/suites/perms.suite.ts @jfcastro92 @Salazareo
/src/puter-js/tests/api/suites/sharing.suite.ts @jfcastro92 @Salazareo
+20
View File
@@ -0,0 +1,20 @@
version: 2
updates:
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
@@ -17,12 +17,12 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
node-version: '26'
cache: 'npm'
- name: Install dependencies
@@ -47,3 +47,13 @@ jobs:
PUTER_TEST_AI_MOONSHOT_API_KEY: ${{ secrets.PUTER_TEST_AI_MOONSHOT_API_KEY }}
PUTER_TEST_AI_ZAI_API_KEY: ${{ secrets.PUTER_TEST_AI_ZAI_API_KEY }}
PUTER_TEST_AI_ELEVENLABS_API_KEY: ${{ secrets.PUTER_TEST_AI_ELEVENLABS_API_KEY }}
# Realtime conversations through the SDK against live OpenAI. Builds
# the SDK bundle these tests load itself, so it runs even when the
# step above fails; the tests skip themselves without the key.
- name: Run live realtime conversation tests
if: ${{ !cancelled() }}
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/convoLive.test.ts
env:
CI: 'true'
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
+171 -39
View File
@@ -1,79 +1,211 @@
name: Backend Tests
# Always triggers on PRs so it can be a required check; the `changes` job below
# decides whether backend/extension paths actually changed (test-only changes
# match the same globs, since tests are co-located with the code), and `test`
# still reports success when they didn't. Pushes to main produce the coverage
# PRs compare against. puter.js SDK changes are covered by puterjs-tests.yaml.
on:
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- 'src/docs/**'
push:
branches: [main]
permissions:
contents: read
pull-requests: write
# Reads main's coverage artifact for the PR comparison.
actions: read
jobs:
test:
changes:
runs-on: ubuntu-latest
outputs:
backend: ${{ steps.filter.outputs.backend }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
id: filter
with:
filters: |
backend:
- 'src/backend/**'
- 'extensions/**'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'tsconfig.build.json'
- '.github/workflows/backend-tests.yaml'
# `tsconfig.build.json` builds with `noCheck: true`, so nothing else in CI
# runs the type checker and a missing export compiles to `undefined`,
# surfacing only when the call is finally reached at runtime. This diffs
# against tools/typecheck-baseline.json and fails only on *new* errors.
#
# The consuming repo runs an equivalent gate, but only once someone bumps the
# submodule pointer — too late to keep the error off this repo's default
# branch. Hence a gate here, on this repo's own pull requests.
typecheck:
needs: changes
if: github.event_name == 'pull_request' && needs.changes.outputs.backend == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '26'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Type check (new errors only)
run: npm run typecheck
# The suite is CPU-bound, so it's split across runners. Each shard writes a
# blob report (results plus coverage) that the `coverage` job merges.
test-shard:
needs: changes
if: needs.changes.outputs.backend == 'true'
name: test shard ${{ matrix.shard }}/${{ strategy.job-total }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- ref: ${{ github.base_ref }}
artifact: base
- ref: ${{ github.head_ref }}
artifact: pr
shard: [1, 2, 3, 4]
steps:
- name: Checkout ${{ matrix.ref }}
uses: actions/checkout@v4
- name: Checkout
uses: actions/checkout@v7
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha || github.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
node-version: '26'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run backend tests with coverage
run: npm run test:backend -- --coverage
run: >-
npm run test:backend --
--coverage
--shard=${{ matrix.shard }}/${{ strategy.job-total }}
--reporter=default
--reporter=blob
env:
CI: 'true'
- name: Upload blob report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: backend-blob-${{ matrix.shard }}
path: .vitest-reports/
include-hidden-files: true
retention-days: 1
# Required check, so its PR name is pinned. A skipped job would satisfy it,
# so this always runs and fails on any shard that didn't pass.
test:
needs: [changes, test-shard]
if: ${{ !cancelled() }}
name: test (${{ github.event_name == 'push' && 'main' || 'pr' }})
runs-on: ubuntu-latest
steps:
- name: Check shard results
env:
CHANGES_RESULT: ${{ needs.changes.result }}
BACKEND_CHANGED: ${{ needs.changes.outputs.backend }}
SHARDS_RESULT: ${{ needs.test-shard.result }}
run: |
if [ "$CHANGES_RESULT" != "success" ]; then
echo "changes job: $CHANGES_RESULT"
exit 1
fi
if [ "$BACKEND_CHANGED" != "true" ]; then
echo "No backend changes."
exit 0
fi
if [ "$SHARDS_RESULT" != "success" ]; then
echo "test shards: $SHARDS_RESULT"
exit 1
fi
coverage:
needs: [changes, test-shard]
if: ${{ !cancelled() && needs.test-shard.result == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '26'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Download blob reports
uses: actions/download-artifact@v8
with:
pattern: backend-blob-*
path: .vitest-reports
merge-multiple: true
- name: Merge shard coverage
run: npx vitest run --config src/backend/vitest.config.ts --merge-reports --coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: backend-coverage-${{ matrix.artifact }}
name: backend-coverage-${{ github.event_name == 'push' && 'main' || 'pr' }}
path: src/backend/coverage/
retention-days: 14
retention-days: ${{ github.event_name == 'push' && 30 || 14 }}
report-coverage:
needs: test
if: always()
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download PR coverage
uses: actions/download-artifact@v4
with:
name: backend-coverage-pr
path: src/backend/coverage
- name: Download base coverage
uses: actions/download-artifact@v4
with:
name: backend-coverage-base
path: coverage-base
# Latest main coverage from this repo's own runs; a fork can't publish
# under this repository id.
- name: Download main coverage
id: base
if: github.event_name == 'pull_request'
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
run: |
id=$(gh api "repos/$REPO/actions/artifacts?name=backend-coverage-main&per_page=20" --jq '
[.artifacts[]
| select(.expired | not)
| select(.workflow_run.head_branch == "main")
| select(.workflow_run.head_repository_id == .workflow_run.repository_id)
][0].id // empty')
if [ -z "$id" ]; then
echo "No main coverage artifact; reporting without a comparison."
exit 0
fi
gh api "repos/$REPO/actions/artifacts/$id/zip" > coverage-base.zip
unzip -q coverage-base.zip -d coverage-base
echo "found=true" >> "$GITHUB_OUTPUT"
- name: Report coverage on PR
if: github.event_name == 'pull_request'
uses: davelosert/vitest-coverage-report-action@v2
with:
json-summary-path: src/backend/coverage/coverage-summary.json
json-final-path: src/backend/coverage/coverage-final.json
json-summary-compare-path: coverage-base/coverage-summary.json
json-summary-compare-path: ${{ steps.base.outputs.found == 'true' && 'coverage-base/coverage-summary.json' || '' }}
@@ -0,0 +1,25 @@
name: Dependabot auto-merge
on: pull_request
permissions:
contents: write
pull-requests: write
jobs:
automerge:
runs-on: ubuntu-latest
if: github.actor == 'dependabot[bot]'
steps:
- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v3
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
- name: Enable auto-merge for patch updates
if: steps.metadata.outputs.update-type == 'version-update:semver-patch'
run: gh pr merge --auto --squash "$PR_URL"
env:
PR_URL: ${{ github.event.pull_request.html_url }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+10 -8
View File
@@ -45,30 +45,30 @@ jobs:
fi
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
@@ -82,18 +82,20 @@ jobs:
type=raw,value=main,enable=${{ github.event_name == 'push' || inputs.push_main }}
- name: Build and push Docker image
uses: docker/build-push-action@v5
uses: docker/build-push-action@v7
with:
platforms: linux/amd64,linux/arm64
context: .
push: true
provenance: mode=max
sbom: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Sync README to Docker Hub
uses: peter-evans/dockerhub-description@v4
uses: peter-evans/dockerhub-description@v5
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
+3 -3
View File
@@ -11,12 +11,12 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '24'
node-version: '26'
cache: 'npm'
- name: Install dependencies
+3 -3
View File
@@ -9,7 +9,7 @@ jobs:
update-release-pr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v7
- name: Setup Git
run: |
git config user.name github-actions
@@ -20,7 +20,7 @@ jobs:
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
branch: release
- uses: actions/checkout@v4
- uses: actions/checkout@v7
if: steps.find-pr.outputs.number == ''
with:
ref: release
@@ -31,7 +31,7 @@ jobs:
git reset --hard main
- name: Create/Update PR
if: steps.find-pr.outputs.number == ''
uses: peter-evans/create-pull-request@v6
uses: peter-evans/create-pull-request@v8
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: Update release branch
+48
View File
@@ -0,0 +1,48 @@
name: Notify HeyPuter (dev)
on:
workflow_dispatch:
inputs:
puter_ref:
description: "puter branch, tag or SHA"
type: string
default: "main"
permissions:
contents: read
jobs:
authorize:
if: github.repository == 'HeyPuter/puter'
runs-on: ubuntu-latest
steps:
- name: Require maintain or admin
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ACTOR: ${{ github.triggering_actor }}
run: |
ROLE=$(gh api "repos/$GITHUB_REPOSITORY/collaborators/$ACTOR/permission" --jq .role_name)
if [ "$ROLE" != "maintain" ] && [ "$ROLE" != "admin" ]; then
echo "::error::$ACTOR has role '$ROLE'; maintain or admin required"
exit 1
fi
notify:
needs: authorize
runs-on: ubuntu-latest
# Holds HEYPUTER_DISPATCH_TOKEN; limited to main so branch copies of this file can't read it.
environment: heyputer-dispatch
steps:
- name: Trigger heyputer dev deploy
env:
PUTER_REF: ${{ inputs.puter_ref }}
TOKEN: ${{ secrets.HEYPUTER_DISPATCH_TOKEN }}
run: |
jq -nc --arg ref "$PUTER_REF" \
'{event_type: "puter-dev-requested", client_payload: {puter_ref: $ref}}' |
curl -fsS -X POST \
-H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/HeyPuter/heyputer/dispatches \
-d @-
echo "Requested \`$PUTER_REF\`" >> "$GITHUB_STEP_SUMMARY"
+2 -1
View File
@@ -8,6 +8,7 @@ on:
jobs:
notify:
runs-on: ubuntu-latest
environment: heyputer-dispatch
steps:
- name: Trigger heyputer build
run: |
@@ -15,4 +16,4 @@ jobs:
-H "Authorization: token ${{ secrets.HEYPUTER_DISPATCH_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/HeyPuter/heyputer/dispatches \
-d '{"event_type":"puter-main-updated","client_payload":{"puter_ref":"main"}}'
-d '{"event_type":"puter-main-updated","client_payload":{"puter_ref":"main"}}'
+214
View File
@@ -0,0 +1,214 @@
name: Puter.js API Tests
# Always triggers so it can be a required PR check; the `changes` job below
# decides whether puter.js-relevant paths actually changed (the suites live
# in src/puter-js/tests, so test-only changes are covered by the same glob,
# plus the pieces the runners depend on: the worker preamble, the in-memory
# test env in testUtil, and the backend vitest config the API-tests config
# extends), and every other job is skipped — not absent — when they didn't,
# which still satisfies a required check. Backend and extension changes run
# backend-tests.yaml instead.
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
changes:
runs-on: ubuntu-latest
outputs:
puterjs: ${{ steps.filter.outputs.puterjs }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
id: filter
with:
filters: |
puterjs:
- 'src/puter-js/**'
- 'src/worker/**'
- 'src/backend/testUtil.ts'
- 'src/backend/vitest.config.ts'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/puterjs-tests.yaml'
# The JSDoc in src/puter-js/src is the source of truth for the SDK's public
# types. The declarations shipped to npm are generated from it at build time
# and never committed, so what needs guarding is the JSDoc itself: this
# generates the declarations and type-checks the published surface without
# skipLibCheck, which is how broken re-exports used to go unnoticed.
#
# Its own job rather than a step in `test`: that one builds bundles and
# installs a browser, and this needs neither.
types:
needs: changes
if: needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '26'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Check the puter.js JSDoc produces declarations that type-check
run: npm run check:puterjs:types
test:
needs: changes
if: needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: '26'
cache: 'npm'
- name: Install dependencies
run: npm ci
# The runners exercise the built artifacts: the SDK bundle
# (src/puter-js/dist) for node + browser, and the worker preamble
# (src/worker/dist) for workerd.
- name: Build puter.js SDK and worker preamble
run: npm run build:workerLib
- name: Install Playwright chromium
run: npx playwright install --with-deps chromium
- name: Run puter.js API tests (node, browser, workerd)
run: npm run test:puterjs
env:
CI: 'true'
# SDK coverage on both refs, reported as a PR comment — same shape as
# backend-tests.yaml. This run uses the istanbul-instrumented bundle
# (the `test` job above keeps exercising the production build).
# `needs.changes` gates the steps below, not this job, on purpose: a
# job-level `if` on a matrix job skips the whole job as one unexpanded
# check ("coverage (${{ matrix.artifact }})", literally, since there's no
# matrix context to fill it in) instead of two real per-artifact checks —
# confusing to read, and unusable if this ever gets pinned as a required
# check. Letting the matrix always expand keeps the check names stable.
coverage:
needs: changes
# Overrides the default matrix naming, which would otherwise bake the
# base/head branch name (matrix.ref) into the check name — making it a
# different string on every PR and impossible to pin as a required check.
name: coverage (${{ matrix.artifact }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
# The base branch lives in this repo; for a fork PR, the head
# repo's copy of it can be arbitrarily stale.
- ref: ${{ github.base_ref }}
repository: ${{ github.repository }}
artifact: base
- ref: ${{ github.head_ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
artifact: pr
steps:
- name: Checkout ${{ matrix.ref }}
if: needs.changes.outputs.puterjs == 'true'
uses: actions/checkout@v7
with:
ref: ${{ matrix.ref }}
repository: ${{ matrix.repository }}
- name: Set up Node.js
if: needs.changes.outputs.puterjs == 'true'
uses: actions/setup-node@v7
with:
node-version: '26'
cache: 'npm'
- name: Install dependencies
if: needs.changes.outputs.puterjs == 'true'
run: npm ci
- name: Install Playwright chromium
if: needs.changes.outputs.puterjs == 'true'
run: npx playwright install --with-deps chromium
# The base leg is best-effort — it only enriches the PR comment with
# a comparison, and the base ref may predate the coverage script.
- name: Run puter.js API tests with coverage
if: needs.changes.outputs.puterjs == 'true'
continue-on-error: ${{ matrix.artifact == 'base' }}
run: npm run test:puterjs:coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: needs.changes.outputs.puterjs == 'true' && !cancelled()
uses: actions/upload-artifact@v7
with:
name: puterjs-coverage-${{ matrix.artifact }}
path: src/puter-js/coverage/coverage-*.json
retention-days: 14
if-no-files-found: ignore
report-coverage:
needs: [changes, coverage]
if: always() && needs.changes.outputs.puterjs == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v7
# Missing when the PR leg failed; `coverage (pr)` already reports that.
- name: Download PR coverage
id: pr-coverage
continue-on-error: true
uses: actions/download-artifact@v8
with:
name: puterjs-coverage-pr
path: src/puter-js/coverage
# May not exist (base ref without the coverage script, or a failed
# base run) — then report without the comparison column.
- name: Download base coverage
id: base-coverage
if: steps.pr-coverage.outcome == 'success'
continue-on-error: true
uses: actions/download-artifact@v8
with:
name: puterjs-coverage-base
path: coverage-base
- name: Report coverage on PR
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome == 'success'
uses: davelosert/vitest-coverage-report-action@v2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
json-final-path: src/puter-js/coverage/coverage-final.json
json-summary-compare-path: coverage-base/coverage-summary.json
- name: Report coverage on PR (no base comparison)
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome != 'success'
uses: davelosert/vitest-coverage-report-action@v2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
json-final-path: src/puter-js/coverage/coverage-final.json
+2 -2
View File
@@ -45,14 +45,14 @@ jobs:
fi
- name: Log in to GitHub Package Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Create alias tag
env:
+4
View File
@@ -66,9 +66,13 @@ AGENTS.md
coverage/
.vitest-reports/
*.log
undefined
servers.json
config.*.json
volatile/
# Local build override
docker-compose.override.yml
+2 -1
View File
@@ -1,4 +1,5 @@
{
"tabWidth": 4,
"singleQuote": true
"singleQuote": true,
"plugins": ["prettier-plugin-jsdoc"]
}
+200 -63
View File
@@ -1,85 +1,41 @@
# AGENTS.md
Guidance for AI coding agents working in this repository. Read this before making changes. FOLLOW GUIDANCE AS CLOSELY AS POSSIBLE. If you think the guidance is wrong, raise an issue or flag a maintainer — don't just do what you think is right. This is the source of truth for how we write code, tests, and docs in this repo.
Guidance for AI coding agents working in this repository, and the source of truth for how we write code, tests, and docs here. Read it before making changes. FOLLOW GUIDANCE AS CLOSELY AS POSSIBLE. If you think a rule is wrong, raise an issue or flag a maintainer instead of doing what you think is right.
## Documentation Index
Use these as the source of truth before exploring further:
- [README.md](README.md) — project overview and quickstart.
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), and extensions.
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), actors, and extensions.
- [doc/contributing-apis.md](doc/contributing-apis.md) — adding and maintaining public APIs end to end (backend surface → puter.js → types → docs → tests). Follow it for any API work.
- [doc/pagination.md](doc/pagination.md) — the one pagination convention for list APIs (limit/cursor/offset/includeTotal, envelope shape, cursor semantics).
- [doc/alarms.md](doc/alarms.md) — raising alarms and picking a severity (what pages, what only gets recorded), plus the config that routes them.
- [doc/self-hosting.md](doc/self-hosting.md) — running Puter outside hosted infra.
- [CONTRIBUTING.md](CONTRIBUTING.md) — testing, security, AI-assisted code, PR conventions, Boy Scout Rule.
- [CONTRIBUTING.md](CONTRIBUTING.md) — contributor entry point and PR conventions.
- [SECURITY.md](SECURITY.md) — how to report vulnerabilities (do not file them publicly).
- [BUG-BOUNTY.md](BUG-BOUNTY.md) — bounty program scope.
- [TRADEMARK.md](TRADEMARK.md) — trademark usage.
---
## Backend
## Repo-wide conventions
The backend is organized as a layered stack inspired by Controller–Service–Repository with dependency injection. Every layer only depends on the layers beneath it, and `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires the whole thing together.
These apply everywhere — backend, puter.js, and GUI.
### Layers (top → bottom)
### Language & files
| Layer | Lives in | Responsibility |
| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Controllers** | [src/backend/controllers/](src/backend/controllers/) | Route handlers. Parse + validate input, apply per-route gates (auth, subdomain, rate limit, body parsers via `RouteOptions`), call services, format responses. |
| **Drivers** | [src/backend/drivers/](src/backend/drivers/) | Optional. RPC-style handlers exposed over `/drivers/*`. Thin shells that validate RPC inputs and call into services/stores. |
| **Services** | [src/backend/services/](src/backend/services/) | Business logic. Assume the caller is already authenticated/authorized — services do not run auth gates themselves. |
| **Stores** | [src/backend/stores/](src/backend/stores/) | Persistence. Wraps clients with the domain shape services consume (rows, entities, KV namespaces). |
| **Clients** | [src/backend/clients/](src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, dynamo, email, event bus). Knows protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | Flat, typed config object every layer receives at construction. |
Each layer receives the layers beneath it through its constructor. Dependencies are explicit and traceable from `PuterServer`.
### Cross-layer rules
- **Don't reach across layers.** Controllers do not poke clients directly; services do not register routes. If you want to, the abstraction is wrong — fix the abstraction.
- **Don't call sideways within a layer for code reuse.** If two services need the same logic, lift it into a util/helper. Services should not depend on other services for shared code.
- **Prefer explicit arguments over `Context` (ALS).** Reach for [Context](src/backend/core/context.ts) only when the value is genuinely request-scoped and would otherwise thread through many layers. Today it's used sparingly — mostly for `actor` and `req`.
### Extensions
Extensions live in [extensions/](extensions/) and parallel the layered stack. They are for **non-crucial parts of the system** — things Puter still works without if removed. If a feature is load-bearing for clients, it belongs in core, not in an extension (see [whoami](extensions/whoami.ts) as the cautionary example).
The `extension` global ([src/backend/extensions.ts](src/backend/extensions.ts)) exposes:
- `extension.registerClient/Store/Service/Driver/Controller(name, Class)` for first-class additions.
- `extension.on(event, handler)` and `extension.get/post/put/delete/patch/use(path, opts?, handler)` for the lightweight common case. `opts` is the same `RouteOptions` controllers use.
- `extension.import('client' | 'store' | 'service' | 'controller' | 'driver')` returns a lazy proxy to instantiated objects. `extension.config` exposes live config.
### Language & file conventions
- **Modules:** We transpile and build as needed — write ES modules, not CommonJS.
- **TypeScript preferred for new files.** Existing JS is fine; convert opportunistically when you're already touching a file.
- **Reuse types before inventing them.** Search for an existing type first; extend it if close. Only define a new type when nothing fits.
- **Make new types findable.** Co-locate them with the layer/module that owns them, export from the obvious entry point, and use a descriptive `PascalCase` name. Don't hide types in random files where future readers won't grep them.
- **Naming:** `camelCase` for variables/functions, `PascalCase` for classes and for files containing a class (`AuthService.ts`, `KVStoreDriver.ts`).
- Write ES modules, not CommonJS — we transpile and build as needed.
- TypeScript preferred for new files in the backend and extensions; convert existing JS there opportunistically when you're already touching a file. The GUI and puter.js are plain JavaScript — don't introduce TypeScript files in them.
- In plain-JS files, typing with JSDoc (`@type`/`@param`/`@returns`, `@typedef` for shared shapes, using the TypeScript type system) is encouraged. API types must not be `unknown` or untyped `...args`: spell out the real parameter and return shapes, unless the value passes through transparently to an upstream layer that owns its type. puter.js has stricter rules ([below](#types)).
- Make new types findable: descriptive `PascalCase` names, exported from the obvious entry point. A type used from many places goes in the owning module's `types.ts` (e.g. [src/backend/controllers/types.ts](src/backend/controllers/types.ts)); a type with a single consumer can stay next to it.
- Naming: `camelCase` for variables and functions, `PascalCase` for classes and files containing a class (`AuthService.ts`). **Prefer `camelCase` for all new code**, including parameters and internal object properties. `snake_case` is reserved for external contracts that already use it: wire/JSON keys, stable API error codes, public option names that already ship as `snake_case`, and established `puter_*` namespaced fields. Don't rename those (it's a breaking change), and don't add new `snake_case` beside them; a camelCase local can carry a value into a wire key (`{ operation_id: operationId }`).
### Comments
Keep comments light. Prefer self-documenting code — clear names, small functions, obvious flow. Add a comment **only** when:
Prefer self-documenting code. Comment only when the _why_ is non-obvious or a usage detail would trip the next reader. **A comment is a sentence or two**: one or two lines of `//`, a few of `/** ... */` JSDoc. State the constraint and stop. Don't restate the code, narrate history ("used to swallow the failure, which meant…"), argue the design, or explain parameters the signature already shows. Reasoning that needs paragraphs belongs in a design doc.
- The _why_ is non-obvious (a hidden constraint, a workaround, a subtle invariant).
- A non-trivial usage detail would otherwise trip the next reader.
Don't restate what the code already says. Don't write comments that reference the current task or PR or version of something — those rot.
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the _why_ in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
**Use plain ASCII `-` in comment section dividers, not box-drawing characters.** Write `// -- Section name --`, not `// ── Section name ──`. The `─` (U+2500) character renders inconsistently in some editors and diff tools and adds nothing over `-`.
### Tests
When adding new behavior (function, endpoint, driver method, branch of logic), add a test for it.
- Use Vitest for unit and integration tests. Test files go next to the code they test, named `*.test.ts` or `*.test.js`.
- **Mock data, not methods.** Stub the inputs (fixtures, fake rows, sample payloads) rather than mocking the function under test or the layer beneath it. Over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service), not within the same layer you're testing.
- Prefer test server over mocking deps. We have existing test utilities for running full test server to boot deps based on config. (see [src/backend/testUtil.ts](src/backend/testUtil.ts)).
- **Hit a real database / real client where reasonable.** Integration shapes catch the things unit tests with mocks miss.
- **Regression tests for bug fixes.** A test that fails before your fix and passes after is the cheapest insurance against the bug coming back.
- If something is genuinely hard to test (UI animation, third-party glue), skip it but say so in the PR.
Don't reference the current task, PR, or version; those rot. **No ticket references** (`PUT-1234`, `// fix for FOO-99`) in code, comments, or test names: describe the why in domain terms. Section dividers use plain ASCII (`// -- Section --`), never box-drawing characters.
### Security & privacy
@@ -91,9 +47,190 @@ Before opening a PR, scan the diff for:
When in doubt, return less. Auth-, permission-, or data-export-related changes deserve an explicit callout in the PR description.
Commit messages and PR descriptions never credit anyone or mention a bug report. For security fixes, state what the change does mechanically ("scoped tokens do xyz now"), not the vulnerability or how it could be abused.
### Reuse and abstraction
- **Search before you build.** Before writing a type, helper, cache/TTL map, retry, lock, pagination, validator, serializer, error mapper, host/email normalizer, or provider adapter, grep `src/backend/util/`, the owning layer, and `extensions/` for one that exists, and extend it instead of adding a parallel copy. If you find two copies, consolidate them or flag it. Before opening the PR, check the diff for new helpers that duplicate existing ones.
- **One path per operation.** Single-item variants call the batch version. Legacy and v2 endpoints share one service method. A second entry point (OIDC, WebDAV, admin, cron) calls the same service as the first instead of re-implementing it.
- **No abstraction without a second caller.** No base class, interface, factory, registry, decorator, or option bag with one implementation. No wrappers that only forward, no interface methods nobody calls, no methods only tests call.
- **Variants are data.** Providers or models that differ only in constants (base URL, prices, key names) are entries in a table read by one implementation, not one file each.
### Hot paths
- No per-row queries or awaits in loops: use the store's batch method (add one if missing) or [src/backend/util/concurrency.ts](src/backend/util/concurrency.ts).
- Bound everything: queries get `LIMIT`/pagination, in-memory maps get eviction, outbound calls get a timeout and a size cap.
- Build clients, catalogs, and config-derived lookups once, not per request.
- Listeners on fleet-wide events (`fs.write.file`, route lifecycle) return before any I/O for requests they don't concern.
### Working rules of thumb
- **Run it, don't just compile it.** "It type-checks" is not "it works." Exercise the code path end-to-end at least once.
- **Read the neighbors before writing.** Match the shape of similar things already in the tree. If you genuinely think the existing pattern is wrong, raise it — don't quietly diverge.
- **Boy Scout Rule, proportional to the change.** Fix the obvious typo, dead import, or missing typehint in files you're already touching. Don't ride a refactor along with a bug fix — that just makes review harder.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't defend in review is not.
- **Read the neighbors before writing.** Match the shape of similar things already in the tree. If you think the existing pattern is wrong, raise it — don't quietly diverge.
- **Test new behavior.** Every new function, endpoint, driver method, or logic branch gets a test; every bug fix gets a regression test that fails before the fix. If something is genuinely hard to test, skip it but say so in the PR.
- **Boy Scout Rule, proportional to the change.** In files you're already touching, fix the typo, the dead import, the missing test, the bit you had to read twice. Don't ride a refactor along with a bug fix.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't have written, debugged, or defended in review is not.
- **A limit change is not done until the docs change with it.** Every rate limit, concurrency cap, quota, and allowance in the code is published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) — an undisclosed limit is one developers discover as a service failure. If a PR moves any of these numbers, the same PR updates that page. The numbers live in `src/backend/controllers/fs/limits.ts` (filesystem), `src/backend/drivers/util/aiLimits.ts` (all AI drivers), per-driver `rateLimit`/`concurrent` configs, and `src/backend/data/subPolicies/` (free-tier allowances).
---
## Local development
To test a local Docker build, use the gitignored `docker-compose.override.yml` described in [Self-Hosting → Building from source](doc/self-hosting.md#building-from-source-instead-of-pulling). Keep local build settings there, not in `docker-compose.yml`.
When budget checks block unrelated local API tests, you may set `"unlimitedMetering": true` without further approval. Merge it into the gitignored `config.json` (`npm start`) or `puter/config/config.json` (Docker) and restart Puter. It covers every account on that instance, guests included, and usage is still recorded. Keep normal settings when testing budget or subscription enforcement, and leave shared defaults unchanged.
---
## Backend
Controllers → drivers (optional) → services → stores → clients, all configured from `IConfig`. Each layer depends only on the layers beneath it and receives them through its constructor; `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires them. [doc/architecture.md](doc/architecture.md) is the full reference.
A public API can be a controller or a driver. Prefer a controller when you need fine-grained control over routes and gates; [doc/contributing-apis.md](doc/contributing-apis.md#controller-or-driver) has the decision guide.
Layer rules:
- **Don't reach across layers.** Controllers don't poke clients; services don't register routes; lower layers never import from controllers. If you want to, the abstraction is wrong; fix the abstraction.
- **Gates run at the edge.** Controllers and drivers validate input and apply auth gates; services assume the caller is already authenticated and authorized.
- **Don't call sideways within a layer for code reuse.** Two services needing the same logic means a util/helper, not a service-to-service dependency.
- **A table's store owns every query against it.** Services, controllers, and extensions don't run raw SQL/redis against a table that has a store; add the store method so cache invalidation stays with the write.
- **Drivers and controllers stay thin.** Domain rules a second entry point needs (signup, app CRUD, worker deploy, AI routing/billing) live in a service, not in the first driver or controller that needed them.
- **Prefer explicit arguments over [`Context`](src/backend/core/context.ts) (ALS).** Use it only for request-scoped values that would otherwise thread through many layers; today that's mostly `actor` and `req`.
### Extensions
[extensions/](extensions/) is for **non-crucial parts of the system**: Puter must still work with the extension removed. If core needs to call it, it belongs in core ([whoami](extensions/whoami.ts) is the cautionary example). Inside an extension, follow the same layered structure, unless it only needs a few route handlers, in which case the `extension.get/post/...` helpers are enough on their own. The `extension` API is in [doc/architecture.md](doc/architecture.md#extension-api).
### Backend tests
- Vitest; test files sit next to the code they test (`*.test.ts` / `*.test.js`). Run with `npm run test:backend`.
- **Mock data, not methods.** Stub inputs (fixtures, fake rows, payloads), not the function under test or the layer beneath it — over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service).
- **Prefer the test server over mocking deps.** `setupPuterTestEnv()` in [src/backend/testUtil.ts](src/backend/testUtil.ts) boots a fully in-memory backend; hit a real database/client shape where reasonable — integration shapes catch what mocked unit tests miss.
- **Test code, not docs.** Never write a test that reads a file under `src/docs/` and asserts on its wording or numbers. Docs are kept in step by the PR (see the limits rule above) and checked in review; a test that greps a markdown page fails on every rewording and verifies nothing about behavior.
---
## puter.js (the SDK)
[src/puter-js/](src/puter-js/) is the public SDK. It ships live from `https://js.puter.com/v2/` with no version pinning, so every change reaches every existing app at once; see [Rule zero](doc/contributing-apis.md#rule-zero-dont-break-callers).
Layout: SDK modules in [src/puter-js/src/modules/](src/puter-js/src/modules/) (one file or directory per area — `FileSystem/`, `kv/`, `ai/`, …), shared plumbing in [src/puter-js/src/lib/](src/puter-js/src/lib/), generated (gitignored) type declarations in `src/puter-js/types/`, API tests in [src/puter-js/tests/api/](src/puter-js/tests/api/), UI e2e tests in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/), developer docs in [src/docs/](src/docs/).
Every SDK change carries all five of these, or the PR is incomplete. [doc/contributing-apis.md](doc/contributing-apis.md) has the steps for each.
1. **Backward compatibility**, unless a maintainer explicitly signs off on a break ([rules](doc/contributing-apis.md#maintaining-an-existing-api)).
2. **Tests** in the cross-platform API suite, or the Playwright harness for desktop-rendered `puter.ui.*` ([steps](doc/contributing-apis.md#6-tests)).
3. **Docs** in [src/docs/src/](src/docs/src/) that match the implementation exactly ([steps](doc/contributing-apis.md#5-docs), [writing rules](#developer-docs)).
4. **Types**, per the rules below.
5. **Error handling**: `{ message, code }` with stable `snake_case` codes, as in [modules/kv/](src/puter-js/src/modules/kv/) ([steps](doc/contributing-apis.md#3-puterjs)).
### Types
puter.js source is plain JavaScript typed via JSDoc, and **the JSDoc is the source of truth for the SDK's types**. `src/puter-js/types/` is `tsc --emitDeclarationOnly` output: the SDK build (`npm run build` in `src/puter-js`) generates it, the npm tarball ships it, and git ignores it. Never commit or hand-edit it. `npm run check:puterjs:types` generates it and type-checks the result without `skipLibCheck`; run it after any type change. The one hand-written declaration file is [src/puter-js/index.d.ts](src/puter-js/index.d.ts): it decides what is public and re-exports the generated names, so name a new type there if consumers should be able to import it.
- Public (exposed) methods carry JSDoc types for parameters and return value, with one `@overload` block per accepted call form; those overloads _are_ the published signature. Typing private helpers is at your discretion; annotate where it helps the next reader.
- Tag members `@internal`, not `@private`, to strip them from the generated declarations.
- Declarations must match runtime behavior exactly. A wrong type is worse than a missing one.
Declare a shape where it belongs and reference it elsewhere with an `import(...)` type. A shape more than one file needs lives in the module's `types.js`; one shared across modules lives in [src/puter-js/src/lib/types.js](src/puter-js/src/lib/types.js); one with a single consumer can stay next to it:
```js
/** @typedef {import('./types.js').KVOptConfig} KVOptConfig */
```
Use `@typedef {Object}` + `@property` for any shape whose fields need documenting; it's the only JSDoc form that carries a per-field doc comment into the generated declarations. Use the inline form for small internal shapes with nothing to say per field, and prefer `unknown` over `*`:
```js
/** @typedef {{ key: string; value: unknown }} KVEntry */
```
---
## Developer docs
[src/docs/src/](src/docs/src/) is the published puter.js documentation: one reference page per method (`<Area>/<method>.md`, [steps](doc/contributing-apis.md#5-docs)) and recipes in [src/docs/src/recipes/](src/docs/src/recipes/).
### Page structure
The reference has two kinds of page:
- **API overview pages** (`<Area>.md`, such as [KV.md](src/docs/src/KV.md)) give a quick top-level picture of what the API is for and what it can do, with short examples and a list of its functions that links every function page in the area. Leave out the details. The first sentence of the intro says what the API is about. This is a judgment call too: when the API depends on a Puter concept the reader may not know, such as Teams, the first sentence can name it and the sentences after it explain it.
- **Function pages** (`<Area>/<method>.md`) hold the details: parameters, defaults, return values, errors, limits, and edge cases.
An overview page answers "what is this API and is it what I need?", and a function page answers "how exactly does this call behave?". A change to one function usually only touches that function's page, so a new `kv.list()` option goes in `KV/list.md` without a matching update to `KV.md`. Don't copy a function's options, limits, or edge cases onto the overview page for completeness. Where a detail goes is a judgment call, not a hard rule: update the overview when a change affects how someone would describe the API as a whole.
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation.
**Sidebar order.** Order the API areas in the sidebar by how often developers use them, most common first. Overview stays first and Objects stays last. Inside an area, order the pages by use too, and keep pages that make up one feature next to each other, such as `share()`, `unshare()`, `listShared()`, and `listSharedByMe()`. A group goes where its most-used page would go. A helper that only supports another function, such as `listModels()` for `chat()`, goes by its own use. Put create, list, get, update, and delete in that order, and constants such as `MAX_KEY_SIZE` last. The function list on the area's overview page uses the sidebar's order. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
### Function page layout
A function page opens with one sentence that says what the function does, in its own paragraph, such as "Given a prompt, generate an image using AI." Put any more detail in the paragraphs after it.
After the intro, a function page has these `##` sections, in this order:
1. Syntax
2. Parameters
3. Return value
4. Errors, if the function has any
5. Sections for specific topics, if the page needs them, with `## Model-specific features` last
6. Examples
[chat.md](src/docs/src/AI/chat.md) follows this layout.
- **Options as a table.** Document an options object as a table with Option, Type, and Description columns. Give each option its own row, even when two options go together, such as `width` and `height`. Order the rows by how often developers use them, most common first, without group rows. In chat.md that's the core options, then reasoning, tools, response handling, request control, and output.
- **Common options in the table, model-specific behavior in its own section.** A page whose function works with many models or providers, such as [chat.md](src/docs/src/AI/chat.md) and [txt2img.md](src/docs/src/AI/txt2img.md), splits its options this way:
- **The options table** under Parameters lists the options that work with every model, and describes each one the way it works for every model.
- **`## Model-specific features`**, the last section before Examples, covers everything else. It has a `###` for each model family or provider, named after it, such as "OpenAI models" or "Cloudflare models". Under it go the options only some models accept, and the common options a model handles differently, such as a narrower range, other allowed values, or an option it ignores.
- **Write there only what's different.** A `###` lists an option only when that model or provider adds it or handles it differently. An option that works the same everywhere stays out, even if the provider supports it, and a model or provider with nothing different gets no `###`.
- **Model-specific features is the only place for model-specific details.** Every limit, format, allowed value, default, price, or quirk that applies to some models goes there, in that model's or provider's `###`, and nowhere else. The options table and topic sections (such as Image inputs or Video input) describe what's true for every model and link to Model-specific features for the rest. Don't add a table or sentence there that compares providers or repeats a model's limits. Each copy is another source of truth to keep in sync, and copies drift when a model changes.
- **Link the two.** Directly under the options table, put `Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).` A row whose option behaves differently on some models can also end with `See [Model-specific features](#model-specific-features).`
- **A long topic gets its own `##` section.** When explaining an option or feature takes more than a sentence or two, such as function calling or output dimensions, give it a section between Errors and Examples and link to it from the options table. This keeps the reference sections short.
- **Usage patterns are examples.** A way of using the API, such as stopping a stream, goes in a runnable example that the relevant section links to ("See the [stop a stream example](/playground/ai-chat-stop-stream/)."), not in a section of its own.
- **Each example has a `###` heading** under `## Examples`, so every example shows in the table of contents.
### Writing docs
These apply to reference pages and recipes alike.
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
- **Document Puter as it runs on puter.com.** The docs are for developers building on production Puter. Leave out behavior that only happens on a self-hosted or differently configured server, such as a provider that isn't set up there.
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
- **State the rule, then its consequence, in short sentences.** No metaphor in place of a rule. Name a concrete action ("you can create any files and folders inside it") over an abstract permission ("yours to organize freely"). Cut a sentence that only repeats the next one.
- **Describe behavior directly.** No quoted inner-voice phrases. Write "`isNew` is `true` when the person did not have access before", not "this tells 'I just gave this to someone' from 'they already had it'".
- **Keep references easy to follow.** Name what a list or code block belongs to in the sentence directly above it. Name a thing once and keep that name. Introduce an actor as "a user" before giving them a role: "A user can create a team. The user who creates the team becomes its owner."
- **Sentence mechanics.** Never start a sentence with inline code: write "To store data, use the [`set()`](/KV/set/) method." Link every inline mention of an API to its page, every time. No em dashes, and no colons or semicolons mid-sentence; a colon that introduces a code block is fine.
- **No `##` or `###` headings inside tabs.** The table of contents lists every `##` and `###` heading on the page, including the ones in a hidden tab (an `example-content` block switched by the `example-group` pills), so its links lead to content the reader can't see. Put the heading above the tabs, or use `####` inside a tab, which the table of contents skips.
- **Show the correct way.** No "don't" and no wrong-way-first examples, in recipes and playground examples alike. Mention a wrong approach only when developers already hit it in practice, and state how it fails as a fact: "`add()` reads a plain object as a path map, so the call rejects with …".
### Recipes
Recipes are tested, copy-pasteable patterns for a task, one layer above the per-method reference, often combining several APIs. Agents find them through the `## Recipes` section of `llms.txt`.
- **Files.** One `.md` per recipe in [src/docs/src/recipes/](src/docs/src/recipes/), with `title`, `description`, `tags`, and optional `order` front matter; they're picked up automatically. Tags must exist in [src/docs/src/recipe-tags.js](src/docs/src/recipe-tags.js) or the build fails. Quote a `description` that contains a colon followed by a space. Every top-level `.md` in that directory is built as a recipe, so keep other markdown out of it.
- **Title.** Short, Title Case, starts with a verb, and leaves out "to Your App" ("Add Team Support", "Deploy Workers Programmatically"). The slug is the title in kebab case, with no API-name prefix. In `llms.txt` an agent sees only the title and description before deciding to open a recipe, so word them to match the task the reader is trying to do.
- **Description.** Written for someone who has never heard of Puter. Start with "Learn how to", put the action first, and say what the reader's app gains: "Learn how to add file sharing to your app with Puter.js, so users can share files in their Puter account with other people." No API names, no mechanism, no list of example cases, and no "not just …" contrast.
- **Intro.** Explain the situation and why you'd do this, tied to something the reader already knows ("`puter.kv` is your database in Puter.js"). No "Use this when" bullet list.
- **Structure.** Get to the first working call as early as possible, then add validation, rate limits, and sign-in after it. Explain a Puter concept the first time it appears, such as what `me.puter` is or that `puter.workers.exec()` opens the Puter sign-in window. Split recipes by what the reader needs, not by technique, and give a family of recipes a shared tag.
- **Reading order.** A recipe reads top to bottom. Each code block uses only functions and variables from blocks above it, so the reader never has to search further down the page to understand it. Start with a small version that works, and have each section add one case. When a section changes a function from earlier, show the whole new version of that function there. If the code is built up over several sections, end with a `## Full Code` block that has all of it.
- **Link to the reference for full option lists.** Mention only the options the recipe uses, then link to the function page for the rest ("All the sort options are listed on [`puter.fs.readdir()`](/FS/readdir/)."). A copied list of options, rules, or scopes goes stale when the reference changes.
- **Limits.** Mention a limit only when it's the reason for a pattern the recipe uses, such as one wildcard subscription instead of one per key, and link to [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) or the function page for the number. Leave out limits that code following the recipe never reaches.
- **Code.** Platform-neutral: just the code, with top-level `await`. No `<script>` or `<html>` wrapper, no IIFE.
- **Tradeoffs.** Justify a choice with one scenario the reader can picture right away, in about two short paragraphs, or fold each point into the section where it matters. The description states the benefit only.
- **Notes.** Leave out `## Notes` by default. Use it only to point to an architecturally different approach. A limit the recipe needs goes in the section it applies to, and other facts belong on the linked reference page.
---
## GUI
[src/gui/](src/gui/) is the Puter desktop: deliberately plain JavaScript + jQuery with HTML-string templates. Don't introduce a UI framework or a new rendering pattern.
**Conformity over novelty**: match the existing design and code structure even where you'd choose differently. A visually or structurally divergent addition is a defect even when it works.
- **Reuse existing UI primitives before writing new ones.** Windows and dialogs are `UIWindow*` functions in [src/gui/src/UI/](src/gui/src/UI/); generic pieces already exist (`UIAlert`, `UIPrompt`, `UIContextMenu`, `UINotification`, `UIPopover`, widgets in [UI/Components/](src/gui/src/UI/Components/)). A new window should read like its neighbors: an async function taking an options object, composing an HTML string, wiring behavior with jQuery, delegating to `UIWindow(...)`.
- **Match the visual language.** Use existing CSS classes (`button`, `button-primary`, window chrome, form styles) and copy the layout patterns of neighboring windows; new styles go in [src/gui/src/css/](src/gui/src/css/) following existing conventions. Verify anything positional (menus, overlays, z-index) on both desktop and mobile viewports.
- **i18n every user-facing string.** No hardcoded UI text — use `i18n('key')` and add the key to [src/gui/src/i18n/translations/en.js](src/gui/src/i18n/translations/en.js).
- **Shared logic goes in helpers/services.** Reusable non-UI logic belongs in [src/gui/src/helpers/](src/gui/src/helpers/) or [src/gui/src/services/](src/gui/src/services/), not copy-pasted between windows.
- **Tests.** Vitest is wired for the GUI (`src/**/*.test.js`; see [appOrder.test.js](src/gui/src/UI/Dashboard/appOrder.test.js) for the shape). Extract pure logic into functions and test those. Desktop behavior driven through puter.js (`puter.ui.*`) is covered by the Playwright harness in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/) — add a spec there when you change how the desktop renders SDK-driven UI.
+66 -37
View File
@@ -6,33 +6,56 @@ We at **Puter** are committed to maintaining a secure experience for our users a
The following are in scope for this program:
* **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
* **`puter.com`**
* **`api.puter.com`**
- **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
- **`puter.com`**
- **`api.puter.com`**
Out-of-scope:
* Third-party services, applications, or libraries not maintained by Puter.
* Social engineering attacks (e.g., phishing against staff).
* Denial of Service (DoS), spam, or volumetric attacks.
* Physical security issues.
- Third-party services, applications, or libraries not maintained by Puter.
- Social engineering attacks (e.g., phishing against staff).
- Denial of Service (DoS), spam, or volumetric attacks.
- Physical security issues.
## Eligibility
We pay bounties for issues that threaten user security and privacy, or the security and health of the platform. We appreciate every report, but hardening suggestions and correctness bugs are not eligible, even when we fix them. Please open a GitHub issue for those.
A report must show how an attacker can directly perform the action against another account or the system. It is not eligible if it:
- Starts from access the attacker should not already have: a stolen session or token, the victim's password, a root or full-access token, or script execution on `puter.com`.
- Needs the victim to do more than ordinary use (opening a link or an app), such as publishing their own secrets, sending a verification link to an address they don't control, or approving a prompt that accurately states what it grants.
- Gives the attacker nothing they couldn't already get through a supported API.
- Only reproduces in a non-default or development setup (SQLite, a `custom` OIDC provider, an unset outbound proxy, cookies or config planted by hand). Findings in the open-source code must affect `puter.com` or a default self-hosted install.
**Shared state.** Sharing files, KV entries, or cross-app KV access means the sharer trusts the sharee. Exploits that depend on that trust, such as a sharee misusing access the sharer granted, may be treated as hardening or lower severity.
Hardening we ship in a public PR does not make similar code paths vulnerabilities. They are judged by these same rules.
## Known Non-Issues (Please Check Before Submitting)
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
* **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we *do* evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
* **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
* **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
* **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
* **App metadata or app user-count "leaks".** This information is currently public by design.
* **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
* **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
* **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
* **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
* **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
* **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
* **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
- **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is out of scope. This explicitly includes **missing `Content-Security-Policy: sandbox` on any endpoint that serves a file inline** — `/token-read`, `/file`, `/fs/read`, WebDAV, and any MIME type the `applyInlineContentSecurity` allow-list does not cover (`text/xsl`, `image/svg+xml`, `application/xhtml+xml`, …). Script execution on the API origin has nothing to reach: there is no session cookie, no `localStorage`, and no ambient credential there. WebDAV additionally has to return an accurate MIME type for real WebDAV clients. The one exception we _do_ evaluate on its merits is anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host — if your report does not name a specific credential that is actually reachable from the API origin, it is this non-issue.
- **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
- **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
- **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
- **App metadata or app user-count "leaks".** This information is currently public by design. This covers **private apps** too: private apps are an unreleased, admin-only beta for paid listings in the Puter App Center. Their `index_url` and other metadata are not secret, the flag is not user-settable, and no production app relies on it. Reports that a private app's metadata is reachable through a sibling route (taskbar items, `/get-launch-apps`, `index_url` drift on the hosting gate, etc.) are normalization work, not vulnerabilities.
- **Worker hot-reload driven by a share recipient.** Sharing a worker's source file with write access is how you let a collaborator change that worker's code and have it deploy live. The redeploy running as the file owner, with the owner's worker credentials, is the intended behavior of that grant — the owner chose to hand out write access to executable code. If you do not want that, do not share the source file with write access.
- **Re-claiming a site or worker name its owner deleted.** A hosted site or worker name that its owner deletes is treated as abandoned by that owner. If one of their apps still points at it, whoever claims the name next serves that app's `index_url` and receives what the app is launched with, and a new app on the same origin may share the old one's identity. Guarding app identities against being recycled this way is ongoing hardening, not a security boundary, so new ways to re-claim an abandoned name, or to inherit an app's tokens or data through one, are not eligible.
- **Share-mode downgrades that do not cascade.** Demoting a delegate does not retroactively revoke the grants that delegate already issued downstream. Shares are independent grants once issued, not a live-evaluated chain. Revoking the downstream grant is a separate, supported action.
- **Metering, credit-hold, and quota inaccuracies.** Credit holds and quota checks are up-front estimates, and actual usage is billed afterwards. Overspend that is bounded and still billed is a correctness bug, including concurrent requests that exceed a hold or quota by some margin; the per-account concurrency and rate-limit gates are the control we rely on. Report these only if you can use a service without it being billed, or overspend without bound.
- **Existence-only oracles.** A response that tells "exists but forbidden" apart from "not found" for a path or ID the caller already supplied, without revealing anything else about it, is hardening.
- **Session and credential lifecycle around 2FA enrollment.** Enrolling in 2FA is not a credential-revocation event: existing authenticated sessions stay valid, and outstanding password-reset links stay valid. Likewise, account-management actions behind the `/user-protected/*` gate are password-gated by design, not TOTP-gated. A recovery code that stops being accepted fails _closed_ — it denies access rather than granting it, so it is a bug report, not a vulnerability.
- **Group permissions** (`/auth/revoke-user-group` and siblings). The group-permission surface is unfinished and not exposed anywhere in the product. Findings there are not reachable by any user.
- **Missing Subresource Integrity (SRI) on external scripts/styles.** Accepted as hardening, not as a vulnerability, absent a demonstrated compromise of the CDN in question.
- **General "token in a URL" / token-lifetime / token-storage designs** — signed directory URLs exposing children, a write signature implying read, app tokens outliving a web session, or the session token being held in `localStorage` on `puter.com`. These are current intended behaviors. Shorter-lived tokens with a refresh flow are on the roadmap; their absence is not a vulnerability on its own. Show a reproducible attack that does not already presuppose script execution on `puter.com`.
- **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
- **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
- **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
- **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
- **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
- **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
@@ -40,37 +63,43 @@ If you believe you have a **genuinely new** exploit chain that defeats one of th
To participate, you must:
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see *Proof of Reproduction* below).
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see _Proof of Reproduction_ below).
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
5. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
6. **Check the Known Non-Issues list**: Reports matching an item in the "Known Non-Issues" section above are not eligible and will be closed.
6. **Check Eligibility and Known Non-Issues**: Reports that fall outside "Eligibility" or match an item in "Known Non-Issues" above are not eligible and will be closed.
Reports that do not meet these guidelines may not be eligible for a reward.
## Proof of Reproduction
Reports must demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
Reports **MUST** demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
* Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
* The **observed** result versus the **expected** result.
* For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
* For server-side bugs: a runnable proof-of-concept.
- Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
- The **observed** result versus the **expected** result.
- For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
- For server-side bugs: a runnable proof-of-concept and screenshot or video showing the attack process. A unit test or in-process harness alone does not count; show it against a running instance.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and are generally not eligible**. If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and not eligible for bounty**.
If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting. Reports sent by automated agents are not eligible.
Please submit **one issue per report**. Bundled "audit packs" of many speculative findings will be declined; send each confirmed issue separately.
Please submit **one issue per report**.
Bundled "audit packs" of many speculative findings will not be eligible for bounty; send each confirmed issue separately.
Additionally please **DO NOT** submit too many issues at once; these will be treated as a bundled report, and will not be eligible for bounty.
## Reporting Process
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
Include:
* A description of the vulnerability
* Steps to reproduce
* Potential impact
* Suggested remediation (if available)
- A description of the vulnerability
- Steps to reproduce
- Potential impact
- Suggested remediation (if available)
- Paypal id where we can send the bounty (if eligible)
Send the full report in your first email. We don't reply to messages that only ask whether we run a bounty or where to report.
We aim to acknowledge receipt within **72 hours** and provide a resolution timeline.
@@ -78,13 +107,13 @@ We aim to acknowledge receipt within **72 hours** and provide a resolution timel
We offer monetary rewards based on the severity of the vulnerability, as determined by our internal assessment (using CVSS as a guide).
* **Critical: \$1,000 – \$2,000**
* **High: \$500 – \$1,000**
* **Medium: \$200 – \$500**
* **Low: \$50 – \$100**
- **Critical: \$1,000 – \$2,000**
- **High: \$500 – \$1,000**
- **Medium: \$200 – \$500**
- **Low: \$50 – \$100**
Non-security issues, suggestions, and best practices feedback are always welcome, but may not qualify for a reward.
If multiple researchers report the same issue, the bounty will be awarded to the first eligible report we receive.
If multiple researchers report the same issue, the bounty will be awarded to the first eligible report we receive. Issues we already know about, from internal review, an open ticket, or a fix already in a public commit or PR, count as duplicates.
## Payments Disclaimer
+1
View File
@@ -0,0 +1 @@
FOLLOW ./AGENTS.md
+11 -32
View File
@@ -1,42 +1,21 @@
# Contributing to Puter Backend
# Contributing to Puter
Thanks for contributing. These rules aren't strictly enforced — but following them makes every PR easier. If anything's unclear, ping a core maintainer or open the PR and ask.
Thanks for contributing. These rules aren't strictly enforced, but following them makes every PR easier. If anything's unclear, ping a core maintainer or open the PR and ask.
New to the backend? Start with [doc/architecture.md](doc/architecture.md).
- New to the backend? Start with [doc/architecture.md](doc/architecture.md).
- Code, test, and docs conventions for the whole repo live in [AGENTS.md](AGENTS.md). It's written for AI agents; the conventions apply to human contributors too.
- Adding or changing a public API (an endpoint, driver method, or puter.js method)? Follow [doc/contributing-apis.md](doc/contributing-apis.md): backward compatibility, [developer docs](src/docs/), types, and tests all move in the same PR.
- Found a vulnerability? Report it privately per [SECURITY.md](SECURITY.md).
---
## 1. Test it. Run it.
## Before you open a PR
Run the affected code path end-to-end before opening a PR. "It builds" is not "it works."
Check your diff against these sections of AGENTS.md:
Add tests for new behavior, endpoints, or bug fixes. If something's genuinely hard to test, say so in the PR.
## 2. Follow existing patterns
Match the shape of similar code already in the repo. [doc/architecture.md](doc/architecture.md) is the source of truth for layers, wiring, and naming. If you think a pattern is wrong, raise it — don't quietly diverge.
## 3. Don't expose system or user information
Scan your diff for stray logs, debug routes, internal paths, secrets, tokens, or user data in errors/responses. When in doubt, return less. Flag any auth, permission, or data-export changes in the PR description.
For private security reports, see [SECURITY.md](SECURITY.md).
## 4. AI-assisted code is fine — understood code is required
Don't commit code you couldn't have written, debugged, or defended yourself. Read the diff, run it, and be ready to explain it in review.
## 5. Update docs for API changes
If you change puter-js APIs (drivers or endpoints used by puter-js), update [developer docs](src/docs/). puter-js itself may also need updating.
## 6. Boy Scout Rule — leave it 1% better
![Boy Scout Rule](https://imgs.search.brave.com/DMmIWl5-NuZVtrR9kXBb06AKF8kturkgSW9UMb2-6m4/rs:fit:860:0:0:0/g:ce/aHR0cHM6Ly9sYXdz/b2Zzb2Z0d2FyZWVu/Z2luZWVyaW5nLmNv/bS9pbWFnZXMvbGF3/cy9ib3ktc2NvdXQt/cnVsZS5wbmc)
Fix the typo, the dead import, the missing test, the bit you had to read twice. Keep cleanup proportional to the change — no refactors riding along on bug fixes.
---
- [Working rules of thumb](AGENTS.md#working-rules-of-thumb): run it end to end, test new behavior and bug fixes, understand every line you commit (AI assistance is fine), leave touched files a bit better, and publish limit changes.
- [Reuse and abstraction](AGENTS.md#reuse-and-abstraction) and [hot paths](AGENTS.md#hot-paths).
- [Security & privacy](AGENTS.md#security--privacy): expose nothing the caller doesn't need.
## Opening a PR
+11 -3
View File
@@ -14,7 +14,7 @@
# config.default.json, so partial overrides work. Absent file = defaults.
# ---- Build stage ----
FROM node:24-slim AS build
FROM node:26-bookworm-slim AS build
WORKDIR /opt/puter
@@ -59,7 +59,7 @@ RUN set -e; \
wait $pjs_pid
# ---- Runtime stage (slim — no build tools) ----
FROM node:24-slim
FROM node:26-bookworm-slim
WORKDIR /opt/puter
@@ -73,16 +73,24 @@ COPY --from=build --chown=node:node /opt/puter .
RUN mkdir -p /etc/puter /var/puter && \
chown -R node:node /etc/puter /var/puter
# Fix outdated HOME_URL in /etc/os-release (see issue #4329)
# The base image may contain a reference to the old phoenix repository.
RUN if [ -f /etc/os-release ]; then \
sed -i 's|HOME_URL="https://github.com/HeyPuter/phoenix"|HOME_URL="https://github.com/HeyPuter/puter"|' /etc/os-release; \
fi
# Self-hosters mount their override at this exact path. The v2 loader
# deep-merges it over config.default.json (see backend/index.ts).
ENV PUTER_CONFIG_PATH=/etc/puter/config.json
ENV NODE_OPTIONS=--enable-source-maps
EXPOSE 4100
# The inbound SMTP receiver, when run from this image with the `smtp` profile.
EXPOSE 2525
USER node
HEALTHCHECK --interval=30s --timeout=3s --start-period=30s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
CMD ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/index.js"]
CMD ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/index.js"]
+6 -7
View File
@@ -10,12 +10,10 @@
·
<a href="https://apps.puter.com/">App Store</a>
·
<a href="https://builder.puter.com/">AI Builder</a>
·
<a href="https://developer.puter.com" target="_blank">Developers</a>
·
<a href="https://discord.com/invite/PQcx7Teh8u">Discord</a>
·
<a href="https://reddit.com/r/puter">Reddit</a>
·
<a href="https://twitter.com/HeyPuter">X</a>
</p>
@@ -50,6 +48,8 @@ npm start
```
**→** This should launch Puter at http://puter.localhost:4100
To run this checkout with Docker, follow [Building from source](./doc/self-hosting.md#building-from-source-instead-of-pulling). Create a local `docker-compose.override.yml` to select the local build; keeping these settings out of `docker-compose.yml` avoids conflicts when pulling updates and keeps local configuration out of pull requests.
<br/>
@@ -82,10 +82,7 @@ Puter is available as a hosted service at [**puter.com**](https://puter.com).
Connect with the maintainers and community through these channels:
- Bug report or feature request? Please [open an issue](https://github.com/HeyPuter/puter/issues/new/choose).
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
- Mastodon: [mastodon.social/@puter](https://mastodon.social/@puter)
- Security issues or abuse reports? [[email protected]](mailto:[email protected])
- Email maintainers at [[email protected]](mailto:[email protected])
@@ -120,7 +117,9 @@ This repository, including all its contents, sub-projects, modules, and componen
- [Korean / 한국어](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ko.md)
- [Malay / Bahasa Malaysia](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.my.md)
- [Malayalam / മലയാളം](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ml.md)
- [Dutch / Nederlands](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.nl.md)
- [Polish / Polski](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pl.md)
- [Odia / ଓଡ଼ିଆ](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.od.md)
- [Portuguese / Português](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pt.md)
- [Punjabi / ਪੰਜਾਬੀ](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.pa.md)
- [Romanian / Română](https://github.com/HeyPuter/puter/blob/main/doc/i18n/README.ro.md)
+73
View File
@@ -0,0 +1,73 @@
# Reverse proxy in front of Puter — mirrors what the prod ALB does:
# accepts every Host header, forwards to the Puter container, and lets
# the Puter app handle subdomain-based routing internally (api.*,
# site.*, app.*, dev.*, plus the per-user subdomains under those).
#
# To enable TLS:
# 1. Drop a wildcard fullchain.pem + privkey.pem into ./puter/tls/
# (see "Step 3 — TLS" in doc/self-hosting.md).
# 2. Uncomment the `:443` block at the bottom of this file, and swap the
# `:80` block for the redirect shown alongside it.
# 3. Uncomment the `443:443` port mapping under the `caddy` service in
# docker-compose.yml, and HTTPS_PORT in .env.
# 4. Set `"protocol": "https"` and `"pub_port": 443` in config.json.
{
# Certs are supplied by the operator, not issued by Caddy. Puter serves
# per-user sites and apps on dynamic subdomains (<name>.site.<domain>,
# <name>.app.<domain>) — only a DNS-01 wildcard cert covers those, and
# DNS-01 needs a provider plugin that isn't in the stock caddy image.
# `off` stops Caddy attempting ACME on boot (it would fail and leave the
# site unreachable) and stops it inventing its own http→https redirects.
auto_https off
}
# Shared handling, imported by the HTTP and HTTPS site blocks below so the
# two can't drift apart.
(puter_routes) {
# Rough size cap that mirrors prod ALB defaults; tune for your uploads.
# Puter chunks large uploads, so 1 GiB per request is plenty.
request_body {
max_size 1024MiB
}
# RustFS — see the `s3` service in docker-compose.yml. Browsers PUT/GET
# here for presigned-URL uploads / downloads. Matched on the `s3.`
# subdomain of whatever the install domain is, so signature verification
# works (Caddy preserves the original Host end-to-end) and HTTPS stays
# clean — no mixed content from a port-9000 host publish.
@s3 header_regexp Host ^s3\.
handle @s3 {
reverse_proxy s3:9000
}
# Everything else, on every Host, goes to Puter — which routes on that
# Host internally. Caddy forwards it unchanged and adds X-Forwarded-For
# / -Proto / -Host, which is what `trust_proxy` in config.json counts.
handle {
reverse_proxy puter:4100 {
# Stream responses through unbuffered — Puter uses SSE and
# socket.io. WebSocket upgrades need no config of their own;
# Caddy proxies them by default.
flush_interval -1
}
}
}
# ── HTTP (port 80) ─────────────────────────────────────────────────────
# A site address with no hostname is the catch-all: it answers for every
# Host, which is what Puter's subdomain routing requires.
:80 {
import puter_routes
}
# ── HTTPS (port 443) — uncomment after dropping certs in ./puter/tls/ ──
# Replace the `:80` block above with a redirect to force HTTPS everywhere:
#
# :80 {
# redir https://{host}{uri} permanent
# }
#
# :443 {
# tls /etc/caddy/tls/fullchain.pem /etc/caddy/tls/privkey.pem
# import puter_routes
# }
+5 -2
View File
@@ -5,9 +5,7 @@
"protocol": "http",
"domain": "puter.localhost",
"cookie_name": "puter_auth_token",
"jwt_secret": "dev-jwt-secret-change-me",
"jwt_secret_v2": "dev-jwt-secret-v2-change-me",
"allow_v1_tokens": true,
"url_signature_secret": "dev-url-signature-secret-change-me",
"allow_all_host_values": true,
"allow_no_host_header": true,
@@ -23,6 +21,11 @@
"default_user_group": "78b1b1dd-c959-44d2-b02c-8735671f9997",
"default_temp_group": "b7220104-7905-4985-b996-649fdcdb3c8f",
"storage_capacity": 104857600,
"disable_user_signup": false,
"teams_enabled": false,
"max_teams_per_user": 1,
"max_seats_per_team_free": 3,
"max_seats_per_team_paid": 40,
"strict_email_verification_required": false,
"gui_assets_root": "./src/gui",
"puterjs_root": "./src/puter-js/dist",
+230 -11
View File
@@ -7,15 +7,20 @@
// Each setting lives at exactly one canonical key — there are no fallback
// aliases. Values shown are illustrative, not production secrets.
//
// Keys consumed only by closed-source / hosted-prod extensions (clickhouse,
// cacheUpdateHandler, pages, prodMeteringAndBilling, …) are intentionally
// omitted.
// Keys consumed only by closed-source / hosted-prod extensions are
// intentionally omitted.
// ── Environment / identity ──────────────────────────────────────────
"config_name": "template",
// `dev` opens a browser on boot, skips blocked-email checks, and runs the
// dev-time webpack watcher; `prod` serves pre-built bundles.
"env": "dev",
// Console output format. `json` replaces the global console so every call
// emits one structured JSON line (level, timestamp, msg, and the active
// request's trace id) — one event per call, so a line-oriented log
// collector can't split stack traces across events and level filtering
// works. Unset (the default) leaves console output human-readable.
"log_format": "text",
"version": "0.0.0",
// Stable identity for this server node — used by pager alerts and
// graceful-shutdown coordination.
@@ -26,6 +31,10 @@
"port": 4100,
// Externally-visible port (set this when behind a reverse proxy on 80/443).
"pub_port": 4100,
// Idle keep-alive timeout in ms. Must stay above the idle timeout of any
// proxy in front, or the proxy reuses connections this server has closed
// and its clients see 502s.
"keep_alive_timeout": 920000,
"protocol": "http",
"domain": "puter.localhost",
// Fully-qualified externally-visible URL. Computed from protocol/domain/
@@ -67,14 +76,13 @@
// ── Auth / session ──────────────────────────────────────────────────
// ALWAYS replace these for any public install — `openssl rand -hex 64`.
// `jwt_secret` is legacy verify-only; new tokens sign with `jwt_secret_v2`.
"jwt_secret": "change-me",
"jwt_secret_v2": "change-me",
// Set false to retire v1 tokens entirely (ROLLOUT-1).
"allow_v1_tokens": true,
"url_signature_secret": "change-me",
"cookie_name": "puter_auth_token",
"min_pass_length": 6,
// When true, anonymous users must log in instead of creating temp or
// permanent accounts.
"disable_user_signup": false,
"allow_system_login": false,
"strict_email_verification_required": false,
"captcha": {
@@ -207,7 +215,9 @@
// "bootstrapTables": true,
"path": "",
// Credentials. NOTE: snake_case here, unlike `s3.s3Config` below.
// For dynamodb-local, any non-empty values work.
// For dynamodb-local, any non-empty values work. On AWS,
// `"useCredentialChain": true` uses the default credential chain
// instead of keys.
"aws": {
"access_key": "",
"secret_key": "",
@@ -246,6 +256,158 @@
}
},
// ── Inbound mail (separate process) ─────────────────────────────────
// Mailboxes addressed at one of your own domains. `secret` gates the
// ingress endpoint that files a message into the recipient's ~/.mail.
//
// `localServer` additionally runs an SMTP listener that accepts mail from
// other mail servers and hands each message to that endpoint. It runs as
// its own process - `npm run start:smtp`, or the `smtp` compose profile -
// and stores nothing itself. Without it, nothing delivers to the endpoint
// unless you point something else at it.
//
// The listener is unauthenticated by nature: anything on the internet may
// hand it a message for an address in `localDomains`. It performs no spam,
// SPF, DKIM or DMARC checks. Read the self-hosting guide before exposing
// it.
// "userEmail": {
// "secret": "a long random string",
// "localServer": true,
// "localDomains": ["example.com"],
// // Container-internal by default; publish 25 onto it from outside.
// "localPort": 2525,
// // Defaults to <api_base_url>/email/ingress. The endpoint is served
// // on the `api` subdomain, so when this points somewhere internal,
// // `localIngressHost` has to name that virtual host.
// "localIngressUrl": "http://puter:4100/email/ingress",
// "localIngressHost": "api.example.com",
// "localHostname": "mx.example.com",
// "localMaxRecipients": 50,
// // Concurrent connections. Each message is held in memory while it
// // is delivered, so this also bounds memory use.
// "localMaxClients": 20
// },
// ── Sharing ─────────────────────────────────────────────────────────
// Email a recipient who already has an account about a new share. On
// unless set to false; they opt out via the unsubscribe link, or by
// blocking a sender. An invite to an address with no account always goes.
// "share_email_notifications": false,
//
// New shares one account may create per UTC day. Default 200.
// "share_daily_limit": 200,
//
// How often a share may interrupt its recipient — the notification pushed
// to their screen and the email with it. Over budget the share still
// succeeds and their notification is still brought up to date; only the
// interruption is dropped. The recipient bounds are what stop many senders
// burying one person between them. A non-positive value removes a bound.
// "share_notify_limits": {
// "pairWindowSeconds": 900,
// "pairDaily": 20,
// "recipientHourly": 10,
// "recipientDaily": 50,
// "emailBatchSeconds": 90
// },
// ── Teams / teams ──────────────────────────────────────────────
// Off by default, and off is the state every existing install stays in.
// The tables ship either way and sit inert; this gates whether `/teams`
// is registered at all, so with it off the paths do not exist rather than
// existing and refusing. It is also the backout: turning it off removes
// the feature without touching data. Teams ships as an extension; on,
// the server refuses to boot unless one registers it.
"teams_enabled": false,
//
// Live teams one user may own. Soft-deleted ones do not count, so
// deleting yours frees the slot. Default 1.
"max_teams_per_user": 1,
//
// Seats one team may provision. Each seat is a real Puter account taking a
// name from the global username pool, so this is what bounds a team's blast
// radius. The owner's plan decides which of the two applies; defaults are 3
// and 40. A plan registered with `teamSeatCap` uses that instead. Lowering
// a cap below a team's current seat count blocks new provisioning and
// disables nobody. Deployments upgrading from the old flat default of 50:
// without a billing extension every owner reads as free, so set
// `max_seats_per_team` to keep the old behavior.
"max_seats_per_team_free": 3,
"max_seats_per_team_paid": 40,
//
// One flat cap for every team, whatever the owner pays. Setting it
// overrides both of the above, so a deployment that does not sell seats can
// ignore plans entirely. Unset by default — leave it that way to keep the
// free cap meaningful.
// "max_seats_per_team": 50,
//
// Staged rollout: only these email domains may create a team, and the tab
// renders for them even with `gui_params.teams_ui` off. Members of an
// existing team always pass. Unset means everyone, gated by `teams_ui`.
// "teams_allowed_email_domains": ["puter.com"],
// ── Notifications ───────────────────────────────────────────────────
// How long a notification is kept, in days from creation. Acknowledged or
// not, a row past this is swept. Off by default; any value above 0 arms
// the sweep.
// "notificationRetentionDays": 14,
//
// Ms a live notification waits for open tabs to acknowledge it.
// "notificationAckTimeoutMs": 5000,
// ── Alarms / alerting ───────────────────────────────────────────────
// Where system alarms go. Severity is the routing decision — each
// transport takes everything at or above its own `minSeverity`:
//
// critical — an unhandled server error; pages on-call.
// error — pages as well; prefer critical or warning.
// warning — look at it today; no page.
// info — a record in the chat channel only.
//
// Both transports are off unless enabled, so a self-hosted node just
// logs its alarms to the console.
"pager": {
// Severity for call sites that don't pick one. Default "critical".
"defaultSeverity": "critical",
// Retier or silence an alarm without a deploy. Keys are alarm ids,
// or a prefix ending in `*`; the exact id wins over a pattern, and
// the longest matching pattern wins among patterns. Values are a
// severity or "mute". This is applied last, so it overrides both the
// call site and any known-error rule.
"severityOverrides": {
// "cronMonitor:*": "info",
// "http_500:GET:/some/flapping/route:*": "mute"
},
"pagerduty": {
"enabled": false,
"routingKey": "",
// Lowest severity that reaches PagerDuty. Default "warning",
// which keeps `info` out of the paging system entirely.
"minSeverity": "warning"
},
// Slack incoming webhook — the low-noise destination for everything
// that shouldn't page.
"slack": {
"enabled": false,
"webhookUrl": "",
// Optional; defaults to the channel the webhook was created for.
"channel": "#alerts",
"username": "puter-alarms",
// Severity window posted to Slack. The ceiling defaults to
// "info" when PagerDuty is configured — what pages belongs in
// the pager, not in chat — and to "critical" when Slack is the
// only transport.
"minSeverity": "info",
"maxSeverity": "info",
// Don't repost the same alarm id within this window. The first
// occurrence always posts; the next post that gets through
// reports how many occurrences piled up. 0 disables throttling.
"repeatThrottleMs": 900000
}
},
// ── Rate limiting ───────────────────────────────────────────────────
// `memory` for single-node, `redis` for multi-node (default), `kv` for
// dynamo-backed counters.
@@ -265,11 +427,17 @@
"apiURL": ""
},
"gemini": { "apiKey": "" },
// Meta Model API (Muse Spark). `muse-spark-1.2-contributor`
// trades a ~12x discount for Meta training on its prompts and
// completions, so it is only served to callers that name it.
"meta": { "apiKey": "" },
"groq": { "apiKey": "" },
"deepseek": { "apiKey": "" },
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
// MiniMax. One key powers chat and video generation (H3, Hailuo);
// video calls go to the same host, beside the chat API's `/v1`.
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
@@ -278,6 +446,28 @@
"apiKey": "",
"apiBaseUrl": "https://openrouter.ai/api/v1"
},
"infron": {
"apiKey": "",
"apiBaseUrl": "https://llm.onerouter.pro/v1"
},
// BytePlus ModelArk. One key powers chat (Seed/GLM/DeepSeek), image
// generation (Seedream) and video generation (Seedance); `apiBaseUrl`
// selects the region; see
// https://docs.byteplus.com/en/docs/ModelArk/1330310 for options.
"byteplus": {
"apiKey": "",
"apiBaseUrl": "https://ark.ap-southeast.bytepluses.com/api/v3"
},
// Kling AI. One API key (not the legacy AccessKey/SecretKey pair)
// powers video generation, image generation and TTS; `apiBaseUrl`
// defaults to https://api-singapore.klingai.com.
"kling": { "apiKey": "" },
"zai": { "apiKey": "" },
"alibaba": { "apiKey": "" },
"hoonify": {
"apiKey": "",
"apiBaseUrl": "https://api.hoonify.ai/v1"
},
"together-ai": { "apiKey": "" },
// Local Ollama. `enabled: false` skips the auto-probe at startup
// (otherwise Puter logs ECONNREFUSED on every boot when no Ollama
@@ -299,7 +489,6 @@
"xai-image-generation": { "apiKey": "" },
// ─ Video generation ─
"openai-video-generation": { "apiKey": "" },
"together-video-generation": { "apiKey": "" },
"gemini-video-generation": { "apiKey": "" },
@@ -311,11 +500,14 @@
"defaultVoiceId": "",
"speechToSpeechModelId": ""
},
// Both AWS providers also take `"useCredentialChain": true` in place
// of keys, like `dynamo.aws`.
"aws-polly": {
"access_key": "",
"secret_key": "",
"region": "us-west-2"
},
"speechify": { "apiKey": "" },
"aws-textract": {
"access_key": "",
"secret_key": "",
@@ -373,6 +565,33 @@
],
// ── Metering ────────────────────────────────────────────────────────
// When true, all metering checks pass — no per-actor limits enforced.
"unlimitedMetering": false
// When true, every account resolves to an unlimited policy: usage is still
// recorded, but nothing is ever refused for lack of budget. This is the
// setting for a deployment with no way to buy more — without it, accounts
// are held to the free monthly allowance and start getting 402s from the AI
// surfaces, file transfers and KV once they pass it.
"unlimitedMetering": false,
// Whether an account that has spent its whole allowance is refused the
// operations that spend it — file transfers, KV calls. Recording is
// unaffected either way. `workers` extends the same refusal to
// worker-driven calls, which are exempt by default because a deployed
// worker has nowhere to surface a payment prompt. `subscriptions` covers
// the separate question of surfaces reserved for paid plans (the
// vendor-compatible AI endpoints): turn it off on a deployment with no
// paid plans, or every account is refused them.
// "meteringEnforcement": { "enabled": true, "workers": false, "subscriptions": true },
// Fleet-wide spend rate, in micro-cents per minute, past which metering
// raises the `metering:excessiveGlobalUsageRate` alarm. Omit it (the
// default) to leave the check off: the only useful value is a multiple of
// what this deployment's normal traffic costs, so it has to be measured
// rather than guessed, and a stale number here alarms on healthy growth.
// "maxGlobalUsagePerMinute": 200000000
// How many full months past its own a monthly usage record is kept before
// it expires. Defaults to 3; set to 0 to keep every month's records
// forever. Changing it, including to 0, applies to records first written
// after the change; records already stamped keep their expiry.
// "meteringRetentionMonths": 3
}
+145
View File
@@ -0,0 +1,145 @@
# Alarms
`clients.alarm` is the one way code reports that something is wrong. It
de-dupes by alarm id, counts occurrences, and routes each alarm to alert
transports by **severity**.
```ts
this.clients.alarm.create(
`metering_write_failed:${userUuid}`, // de-dupe key
'Metering write failed', // what a human reads
{ userUuid, appId, error }, // context fields
'info', // severity
);
```
## Severity is the routing decision
| Severity | Meaning | Goes to |
| ---------- | ------------------------------------------------------ | ----------- |
| `critical` | An unhandled server error. Someone gets woken up. | Pager |
| `error` | Same urgency as critical; prefer one of the other two. | Pager |
| `warning` | Worth a look today. Nobody is paged. | Pager (low) |
| `info` | A record of something expected-but-notable. | Chat |
Each transport declares the severity window it accepts, so the value a call
site passes is what decides where the alarm lands. The two windows don't
overlap by default: anything that pages lives in the paging system, and chat
is the record of what didn't. The bar for `critical` is
deliberately high: an unhandled 5xx out of the HTTP error handler is the main
thing that still pages. Anything a human can look at tomorrow is `warning`,
and anything that's just worth recording is `info`.
Omitting the severity takes `pager.defaultSeverity` (itself `critical`), so
pass one explicitly unless you really mean "page someone".
### Choosing one
- Did the server fail to do its job in a way nobody expected? → `critical`
- Is a background job, rate, or dependency degraded, or has an account spent
far past its credits? → `warning`
- Is this a user doing something notable (tripping an abuse heuristic)?
→ `info`
- Did one of *our own* limits reject a caller — a rate limit, a concurrency
cap, a quota? → don't alarm at all. The limit doing its job is not an
event; the 429 is the whole signal, and alarming on it only produces noise
proportional to traffic. An *upstream provider* rate-limiting us is the
opposite case and still alarms (`upstream_rate_limited`, `info`) — that one
is not something we chose. The exception is a free model: nothing is billed,
nothing is actionable, and the throttling is the price of the model, so the
AI chat driver marks those `noAlarm` and only paid models still record.
`noAlarm` is the code-level mute: an `HttpError` carrying it skips the
terminal gate entirely, no matter its status or code. It is for failures the
call site *already knows* are expected and traffic-proportional — reach for it
there, not as a way to quiet an alarm you haven't diagnosed
(`severityOverrides` below is the knob for that).
An extension whose signals are all one tier can default its own local
`raiseAlarm` helper to that tier instead of repeating it at every call site —
see [extensions/cronMonitor](../../../extensions/cronMonitor/index.js).
## One incident per occurrence, unless you say otherwise
Alarms always de-dupe *in process* — repeats of an id bump its occurrence
count rather than creating a second alarm. What that means for the pager is a
separate decision, and by default every occurrence opens its own PagerDuty
incident: two failed scans an hour apart are two things that happened, and
closing one shouldn't hide the other.
Pass `{ dedup: true }` as a fifth argument when repeats of the id really are
one recurring fault, and they collapse onto a single incident carrying the
occurrence count:
```ts
this.clients.alarm.create(alarmId, message, fields, 'critical', {
dedup: true,
});
```
The HTTP error handler uses it: its id is route + error signature, so a hot
loop of the same crash is one incident with N occurrences instead of N pages.
Dedup and Slack's repeat throttle mean a responder sees only the latest
occurrence, so the handler attaches the `HttpError`'s `fields` to the alarm as
`details` — put what a responder needs (which upstreams failed, and how)
there. Everything in `fields` is also returned to the client in the response
body, so it has to be safe to show the caller.
Reach for it anywhere else only when the id is that specific — otherwise a
per-request alarm can flood the pager.
## Configuration
Everything lives under `pager` in config (see
[config.template.jsonc](../config.template.jsonc) for the annotated version).
Both transports are off unless enabled, so a self-hosted node just logs
alarms to the console.
```jsonc
"pager": {
"defaultSeverity": "critical",
"severityOverrides": { "cronMonitor:*": "info" },
"pagerduty": { "enabled": true, "routingKey": "…", "minSeverity": "warning" },
"slack": {
"enabled": true,
"webhookUrl": "…",
"channel": "#alerts",
"minSeverity": "info",
"maxSeverity": "info",
"repeatThrottleMs": 900000,
},
}
```
Slack's `maxSeverity` defaults to `info` whenever PagerDuty is configured, and
to `critical` when it isn't — a node with only a webhook still sees
everything. Raise it to have chat mirror the paging tiers as well.
### Retiering without a deploy
`severityOverrides` is the escape hatch for an alarm that turns out to be
noisier or more serious than its call site assumed. Keys are alarm ids or a
prefix ending in `*`; the exact id beats a pattern, and the longest matching
prefix wins among patterns. Values are a severity, or `mute` to drop the
alarm before any transport sees it.
It is applied *after* the call site's severity and any known-error rule, so
config always has the last word.
### Repeat throttling
Two levels, and they stack.
The client itself backs off before a repeat is reported at all: every
occurrence is reported up to a small burst, then one per interval. A fault
recurring hundreds of times a second still counts every occurrence, but it
does not write a log line or build an alert payload for each one.
The count lasts only as long as the client remembers the alarm. One that
hasn't recurred for 24 hours is forgotten, and the client holds at most 1000
at once, dropping the least recently raised first. Either way the next
occurrence is a new alarm, with its count and burst starting over.
Below that, the chat transport won't repost the same alarm id within
`repeatThrottleMs` (default 15 minutes). The first occurrence always posts,
and the next one that gets through reports how many piled up in between —
so a hot loop reads as one message with a count, not a wall of them.
+27 -16
View File
@@ -1,6 +1,6 @@
# Backend Architecture
Loosely inspired by the Controller–Service–Repository pattern with dependency injection. The backend is organized as a stack of layers where each layer only depends on the layers beneath it, and `PuterServer` ([src/backend/server.ts](../src/backend/server.ts)) instantiates each layer in order and hands the instances down to the next.
Loosely inspired by the Controller–Service–Repository pattern with dependency injection. The backend is a stack of layers; each depends only on the layers beneath it and receives them through its constructor from `PuterServer` ([src/backend/server.ts](../src/backend/server.ts)), which instantiates the layers in order. The rules for working within the layers are in [AGENTS.md](../AGENTS.md#backend).
## Layers
@@ -31,8 +31,6 @@ block-beta
style CFG fill:#334155,stroke:#1e293b,color:#fff
```
Each layer only depends on the layers beneath it, and every dependency is injected through the constructor by `PuterServer`. Extensions sit alongside this stack and can register into any layer — see [Extensions](#extensions) below.
| Layer | Lives in | Responsibility |
| --- | --- | --- |
| **Controllers** | [src/backend/controllers/](../src/backend/controllers/) | Route handlers. Parse + validate input, apply per-route gates (auth, subdomain, rate limit, body parsers — see `RouteOptions`), call into services, format responses. |
@@ -42,7 +40,7 @@ Each layer only depends on the layers beneath it, and every dependency is inject
| **Clients** | [src/backend/clients/](../src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, dynamodb, email, event bus, …). Knows protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | The flat, typed config object every layer receives at construction. |
Each layer receives the layers beneath it through its constructor, so dependencies are explicit and traceable from `PuterServer`. A controller does not reach into a client directly; if it needs one, the right move is usually a service.
Extensions sit alongside this stack and can register into any layer; see [Extensions](#extensions).
## Entry point: `PuterServer`
@@ -55,17 +53,25 @@ Each layer receives the layers beneath it through its constructor, so dependenci
## Context (ALS)
We use [`Context`](../src/backend/core/context.ts) — backed by `AsyncLocalStorage` — to carry per-request state without threading it through every function signature. It is used **sparingly**, mostly for `actor` and `req`. The request-context middleware opens a scope per request after the auth probe runs; anything inside a request handler can call `Context.get('actor')` / `Context.get('req')` instead of plumbing it as an argument.
[`Context`](../src/backend/core/context.ts), backed by `AsyncLocalStorage`, carries per-request state without threading it through every function signature. The request-context middleware opens a scope per request after the auth probe runs, so anything inside a request handler can call `Context.get('actor')` / `Context.get('req')`. It's used sparingly, mostly for `actor` and `req`; see the [Context rule](../AGENTS.md#backend).
Prefer explicit arguments. Reach for `Context` only when the value is truly request-scoped and would otherwise need to thread through many layers.
## Actors
An [`Actor`](../src/backend/core/actor.ts) is who a request is acting as. Its two app fields are not interchangeable: `app` is the app the actor carries *directly* (an app-under-user token), while `effectiveApp` is the app it ultimately acts as — its own, or the one that issued its access token. An access-token actor has no `app` of its own, so a gate that reads `app` answers "no app" for it and falls open.
`makeActor` resolves `effectiveApp` once at construction, and `assertResolvedActor` at the request edge rejects any actor that skipped it — so reading `effectiveApp` needs no fallback. Reach for `app` only where the direct app genuinely is the question, and say why in a comment.
- `actor.effectiveApp` — which app is acting. This is the default; nearly every scoping rule, namespace key and attribution wants it.
- `isAppActor(actor)` — a direct app actor specifically. Use it where a token must *not* be treated as its issuer, e.g. the permission scanners that read an app's own grant rows.
- `isPlainUserActor(actor)` — the account acting through nothing at all; `isAccountContext(actor)` for the wider "the account's own reach", which also admits a full-access token.
## Extensions
Extensions live alongside core ([packages/puter/extensions/](../extensions/)) and parallel the layered stack. They are meant for **non-crucial parts of the system** — things Puter still works without if removed.
Extensions live in [extensions/](../extensions/), parallel the layered stack, and hold the parts of the system Puter still works without. [AGENTS.md](../AGENTS.md#extensions) has the rule for what belongs in one.
- **Good extensions**: [thumbnails](../extensions/thumbnails.ts), [serverInfo](../extensions/serverInfo.ts), [devWatcher](../extensions/devWatcher.ts) — opt-in features cleanly bolted on.
- **Should probably be core**: [metering](../extensions/metering.ts), [appTelemetry](../extensions/appTelemetry.ts) — clients now expect these to be present, so the "extension" framing is misleading.
- **Shouldn't have been an extension**: [whoami](../extensions/whoami.ts) — it's load-bearing for every authenticated client. Keep this one in mind as a cautionary example when deciding whether something belongs in an extension.
- **Shouldn't have been an extension**: [whoami](../extensions/whoami.ts) — it's load-bearing for every authenticated client.
### Extension API
@@ -88,17 +94,22 @@ import { extension } from '@heyputer/backend/src/extensions';
const services = extension.import('service');
extension.get('/healthcheck/deep', { subdomain: 'api', adminOnly: true }, async (_req, res) => {
res.json({ ok: await services.health.runDeepCheck() });
res.json(await services.health.getStatus());
});
extension.on('user.signup', (_key, data) => {
console.log('new user', data.user.username);
extension.on('puter.signup.success', (_key, data) => {
console.log('new user', data.username);
});
```
## Conventions
The `app.recommended` event lets extensions customize desktop recommendations.
Its `appNames` array starts as a fresh copy of the default ordered names on each
call. Listeners may mutate or replace it, including setting it to an empty array.
Async listeners are awaited before names are resolved to app summaries; names
that do not exist are skipped.
- **TypeScript preferred** in new code where feasible. Existing JS is fine; convert opportunistically when you're already touching a file.
- **`camelCase`** for variable/function names; **`PascalCase`** for classes and for files that contain a class (`AuthService.ts`, `KVStoreDriver.ts`).
- **Deduplicate**. If two services need the same logic, lift it into a util/helper rather than calling sideways across the same layer — services should not depend on other services for code reuse.
- **Don't reach across layers.** Controllers do not poke clients directly; services do not register routes. If you find yourself wanting to, that's usually a signal the abstraction is wrong.
```ts
extension.on('app.recommended', (_key, data) => {
data.appNames = ['editor', 'camera'];
});
```
+106
View File
@@ -0,0 +1,106 @@
# Contributing APIs
How to add a new public API to Puter, and how to maintain one that already exists. "Public API" means anything applications can call: HTTP endpoints, `/drivers/*` methods, and the puter.js methods that wrap them. This guide is written for human contributors and AI agents alike — [AGENTS.md](../AGENTS.md) defers to it for API work.
Companion docs: [architecture.md](architecture.md) (backend layering), [pagination.md](pagination.md) (list APIs), [src/puter-js/tests/api/README.md](../src/puter-js/tests/api/README.md) (SDK test environment).
## Rule zero: don't break callers
puter.js is served live from `https://js.puter.com/v2/` with no version pinning — every app in existence picks up your change the moment it deploys, and the backend endpoints underneath have the same property. Assume every observable behavior (parameter handling, response fields, error codes, ordering) has someone depending on it.
Every change is backward compatible unless a maintainer has explicitly agreed to a break beforehand.
## Core or extension?
The first decision is where the API lives. If **nothing in core will call it, prefer an extension** over wiring it into core. [AGENTS.md](../AGENTS.md#extensions) has the rule and its test; [architecture.md](architecture.md#extension-api) documents the `extension` API.
## Adding a new API
Work through all seven steps; the PR is complete when every one is.
### 1. Design the surface first
- Sketch the signature, options, return shape, and error cases before writing code. Find the two or three most similar existing APIs and match their conventions.
- New parameter and field names follow the [naming rule](../AGENTS.md#language--files) (`camelCase`).
- Anything returning a list follows the [pagination convention](pagination.md): `limit`/`cursor` in, `{ items, cursor, total }` envelope out.
- Prefer an options object over a growing list of positional parameters, but keep the common case callable with a single argument where siblings do.
### 2. Backend
Follow the layered stack ([architecture.md](architecture.md), [layer rules](../AGENTS.md#backend)): a controller or driver at the edge, business logic in a service, persistence in a store. Return exactly what the caller needs and no more — every response field you ship is permanent — and use stable `snake_case` error codes.
#### Controller or driver?
Both are supported ways to define an API. **Prefer a controller when you need fine-grained control** — URL shape, HTTP verbs, per-route gates, response and streaming formats. A **driver** fits when the API is a set of RPC methods implementing one of the named interfaces on `/drivers/*` (`puter-kvstore`, `puter-chat-completion`, …) and the generic driver plumbing — one call envelope, interchangeable implementations, per-method policies — covers what you need.
- **Controller:** extend `PuterController` ([src/backend/controllers/types.ts](../src/backend/controllers/types.ts)) and declare routes with the `@Controller(prefix)` class decorator plus `@Get`/`@Post`/… method decorators ([src/backend/core/http/decorators.ts](../src/backend/core/http/decorators.ts)), each taking `(path, routeOptions)` — or override `registerRoutes(router)` imperatively. Core controllers register in [src/backend/controllers/index.ts](../src/backend/controllers/index.ts); extensions use `extension.registerController(...)` or the plain route helpers.
- **Driver:** extend `PuterDriver` ([src/backend/drivers/types.ts](../src/backend/drivers/types.ts)) and mark it with the `@Driver(interfaceName, opts)` decorator ([src/backend/drivers/decorators.ts](../src/backend/drivers/decorators.ts)), which also declares its policies. Core drivers register in [src/backend/drivers/index.ts](../src/backend/drivers/index.ts); extensions use `extension.registerDriver(...)`.
#### Middleware and gates
- **Controller routes** (and extension routes — same options) take [`RouteOptions`](../src/backend/core/http/types.ts): auth gates (`requireAuth`, `requireUserActor`, `noUserSession`, `adminOnly`, `allowedAppIds`, and the access-token controls), `subdomain` routing, per-route `rateLimit`, body parsers, and arbitrary extra `middleware`. The auth flavors are subtle and default-deny — read the JSDoc on each field before picking.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), `requireSubscription`, `requireReputation`, and `noUserSession`. The `/drivers/call` surface enforces them.
- **Paying accounts only:** `requireSubscription`, as a route option on a controller endpoint or in the per-method `@Driver({ requireSubscription })` block on a driver (`/drivers/call` is one shared route, so a driver's requirement can't live in route options). `true` accepts any plan that isn't free, so a plan an extension registers counts without core naming it; an array of policy ids (`['business', 'pro']`) accepts only those; `false` or leaving it out means no requirement. An empty array is a boot error, since it would read as subscribers-only while admitting everyone. The check is a map lookup on the metering service's cached per-actor subscription. It asks which plan the account is on; `requireCredits` asks whether it has budget left. Deployments with no paid plans (self-hosted installs) turn it off with `meteringEnforcement.subscriptions: false`.
- **Trusted-enough accounts only:** `requireReputation`, as a route option or in the per-method `@Driver({ requireReputation })` block. Opt-in; implies `requireAuth`. The value names a tier, and `reputationGate.tiers` in config sets the score that tier takes, so a deployment can retune it without editing the surface. A tier the running config doesn't define is inert and everyone passes, so an install that doesn't score its accounts never turns traffic away on a score it never computed. `reputationGate.enabled: false` stops every declared gate at once. Denials are a bare 403 `reputation_required` that names neither the score nor the tier. Nothing in the tree declares one yet.
- **A specific verified factor:** `requireVerified` (email, and only under `strict_email_verification_required`), `requirePhoneVerified`, `requireCardVerified`. Opt-in, and the factor must actually have been verified, unlike the default-on gate that turns away accounts still carrying a pending verification the abuse harness asked for. `requireAnyVerified: ['phone', 'card']` is the OR of the last two: any listed factor verified at any point passes, as does a paid plan when `card` is listed (a paying account already has a card on file). Otherwise only the factors the deployment can verify are asked for (an SMS provider configured, a card gate an extension reports on); with none verifiable the gate is inert rather than locking the route on a self-hosted install. A denial carries the first verifiable factor's code plus `factors` (the verifiable ones, in the route's order), so a client can lead with one flow and offer the other. `verifiedFactorGate.enabled: false` in config stops every declared gate at once, e.g. during an SMS-provider outage.
- **Metered spend:** an endpoint that spends metered resources on the caller's behalf (moving file content, object-store requests, anything else the account is billed for) declares `requireCredits: true`, which turns an account with no budget left away with a 402 before the handler runs. Endpoints that only describe or delete things deliberately don't: an account that has run out still has to see what it has, clear it, and reach its billing pages. Drivers have no route options, so they call `assertActorHasCredits` themselves ([src/backend/services/metering/enforcement.ts](../src/backend/services/metering/enforcement.ts)) — see `KVStoreDriver`, which does it once for every method.
Provider sign-in requires the account's authenticator or a recovery code when 2FA is enabled, including signup callbacks that resolve an existing account. The five-minute challenge uses the same single-use `/login/otp` and `/login/recovery-code` verification as password sign-in. The GUI carries it in a URL fragment and removes it before initialization. A session and successful popup-return proof are issued only after verification; cancelling retains the popup's signed retry context without marking it authenticated.
Disabling 2FA through `/user-protected/disable-2fa` requires both password or OIDC revalidation and either the current six-digit authenticator `code` or an unused recovery code in `code`. Recovery codes are consumed atomically. Missing or incorrect codes leave the secret and remaining recovery codes intact. A team's requirement for 2FA still prevents disabling it.
Email changes through `/user-protected/change-email` require the password or OIDC revalidation cookie, plus a six-digit `code` from the authenticator when 2FA is enabled. Missing or malformed codes return `400 bad_request`; incorrect codes return `401 code_mismatch`. Both include `factor: 'otp'`. Confirmation links are bound to that verified request. A 2FA-enabled account must request a new link if its existing link was issued without second-factor verification.
### 3. puter.js
- Add the method to the matching module in [src/puter-js/src/modules/](../src/puter-js/src/modules/), matching the calling conventions of its siblings (promise-returning; positional shortcut plus options form where that's the local pattern).
- Validate cheap preconditions client-side and throw `{ message, code }` objects; pass backend errors through unchanged rather than swallowing or re-wrapping them.
### 4. Types
Type the method in JSDoc on the implementation, following the [puter.js type rules](../AGENTS.md#types) (overloads, where shapes live, `index.d.ts`), then run `npm run check:puterjs:types`. Declarations are generated from the JSDoc, so there is nothing to keep in sync by hand.
### 5. Docs
- Add the method page at [src/docs/src/](../src/docs/src/)`<Area>/<method>.md` — frontmatter (`title`, `description`, `platforms`), syntax, parameters, return value, and at least one runnable example — and update the area overview (`<Area>.md`). Copy the structure of an existing page.
- The docs are the contract users code against: signatures, defaults, and return shapes match the implementation exactly.
### 6. Tests
- **Backend:** see [backend tests](../AGENTS.md#backend-tests).
- **SDK:** add cases to [src/puter-js/tests/api/suites/](../src/puter-js/tests/api/suites/)`<area>.suite.ts` (register new suites in `suites/index.ts`; there is no globbing). One suite runs on node, browser, and workerd via `npm run test:puterjs`; never write per-platform tests. The runners execute the built bundle, so run `npm run build:workerLib` first or the suite silently tests stale code.
- **Desktop-rendered UI** (`puter.ui.*`): add a Playwright spec per [src/puter-js/TESTING.md](../src/puter-js/TESTING.md).
### 7. Security pass
Run the [security & privacy](../AGENTS.md#security--privacy) check on the diff, and confirm the auth gates are present.
## Maintaining an existing API
Changes are **additive by default**:
- Existing call signatures keep working, including both positional and options-object forms where a method supports them. Say in the PR how existing callers are unaffected.
- New parameters are optional, with defaults that reproduce the old behavior exactly.
- Never rename, repurpose, or remove existing parameters, response fields, or error codes. Don't change types, ordering guarantees, or which fields are present when.
- New behavior that could surprise existing callers goes behind an opt-in flag.
- Docs, types, and tests move in the same PR as the behavior; a signature change with stale docs is a bug.
- Bug fixes come with a regression test that fails before the fix. Be suspicious of fixes that change observable behavior: someone may depend on the bug. When in doubt, ask a maintainer.
### Breaking changes
Rare and deliberate. In order: explicit maintainer sign-off, a documented migration path, and a rollout plan — typically new surface added first, old surface deprecated, old surface removed much later, if ever. Never break as a side effect of a refactor.
### Deprecating
The old surface keeps working. Mark it `@deprecated` in the type declarations, note the replacement on its docs page, and stop using it in examples. Removal is a separate, maintainer-approved decision.
## Definition of done
- [ ] Backward compatible (or the break was explicitly approved)
- [ ] Right home: extension if core never calls it; layered structure either way
- [ ] puter.js method matches sibling conventions; errors are `{ message, code }` with stable codes
- [ ] Types updated and matching runtime behavior
- [ ] Docs page + area overview updated, with a runnable example
- [ ] Tests: backend + three-platform SDK suite (+ e2e for desktop-rendered UI)
- [ ] Security pass on the diff
- [ ] You ran it end to end
+147
View File
@@ -0,0 +1,147 @@
# Embedded app browser
`GET /embed/apps` on the deployment's main GUI origin renders the same app browser
as the dashboard's Apps tab, without the desktop or dashboard navigation.
It supports search, paging, saved folders, and saved ordering. It uses the same
installed-app and recommendation data as the dashboard, including the user's
removed recommendations. Editing, adding, reordering, renaming, and uninstalling
are disabled. Selecting an app opens it in a new browser tab at its own URL, the
same way the dashboard does, and the host page stays open. The app's token isn't
put in the URL: the app's SDK asks the embed for it once the tab loads, so the
embed must still be open then. If the iframe is removed first, the app falls back
to its normal sign-in. Apps that need the desktop (built-in and godmode apps,
private apps the user can't open, or a failed token request) open `/app/<name>`
on the GUI origin, cut off from the embed. Website shortcuts open their external
destination with no opener.
The embed follows the OS's light or dark setting. A host with a fixed theme adds
`theme=dark` or `theme=light` to the fragment (`#user=<uuid>&theme=dark`), since
Chromium doesn't pass an iframe's `color-scheme` on to the page inside it.
Apps opened from the embed carry `utm_source=puter.com&utm_medium=referral&utm_campaign=app_browser_embed`;
the desktop and dashboard send `utm_campaign=desktop` and `utm_campaign=dashboard`.
## Session and framing
The embed shows the GUI session only to the user the host page is signed in as.
The host names that user's `uuid` (from `puter.auth.getUser()`) in the iframe
URL's fragment, as `/embed/apps#user=<uuid>`. The embed reads the session, asks
`/whoami` which user it belongs to, and shows the signed-out view unless the two
match. So a host that names no user, a malformed one, or a user other than the
session's, gets the signed-out view, and so does a session the API refuses. A
failed `/whoami` check shows the error state. The fragment never reaches the
server and can only narrow what the embed shows: naming the session's own user
gives the same view as a direct visit to the GUI. Changing the fragment of a
loaded embed reloads it.
The embed reads the existing GUI session from local storage on its own origin.
If that storage holds no session, the embed gets the session from the GUI origin's session cookie through
`/get-gui-token`. This matters in Safari, which gives a framed origin separate
local storage under each top-level origin, so even a sibling subdomain's iframe
can't see the GUI's stored token. The session cookie is still sent, because the
host is on the same site. The embed doesn't write that token back to storage. It
does not create accounts, show a login dialog, or accept a token from its parent.
Being signed in on a subdomain alone does not create a session on the main GUI
origin.
Without a session it can show, the embed shows the recommended apps (`RecommendedAppsService`)
that the server put in the page, and nothing else: no installed apps, folders, or
saved order, and no puter.js. Selecting one opens it at the same URL a signed-in
launch uses, with no opener and no token on offer, so the app signs in on its own.
Apps that need the desktop open `/app/<name>`. If the page carries no recommended
apps, or a load fails, the embed displays an unavailable/error state.
The route's `Content-Security-Policy: frame-ancestors` allows the GUI origin and
the embedding page's origin when it is exactly one label under `config.domain`
with the scheme and port from `config.origin`. Subdomains the server answers
itself (`api`, `dav`, `js`, `www`, the native-app hosts, and the host of
`api_base_url`) and the user-content hosting domains (`static_hosting_domain`,
`private_app_hosting_domain`, and their `_alt` variants) are refused. For
example, `https://puter.com` permits `https://spreadsheet.puter.com` but not
`https://api.puter.com`, and `http://puter.localhost:4100` permits
`http://spreadsheet.puter.localhost:4100` but not
`http://someone.site.puter.localhost:4100`. The embedding page is read from the
iframe request's referrer, so the host must not suppress it (the browser default
`strict-origin-when-cross-origin` is enough). All ancestors in a nested iframe
chain must be allowed. Other GUI routes retain `X-Frame-Options: SAMEORIGIN`.
The embed accepts no query parameters. Requests with a query string redirect to
the same embed path so SDK bootstrap parameters cannot override the session or API
origin. The host needs to allow the GUI origin in its own `frame-src` CSP if it
sets one. If sandboxing the iframe, allow scripts, same-origin storage, popups,
and popups to escape the sandbox.
## Host integration
A first-party app that signs in with puter.js can place `<puter-user-menu>` from
the SDK in its header instead (see `src/puter-js/src/ui/components.md`). It opens
this embed with the signed-in user named and closes it when the app's sign-in
changes. The rest of this section is for a host that builds the iframe itself.
The iframe can be shown whether or not the user is signed in. Create it when
“More apps” is opened, naming the signed-in user in its fragment, and remove it
on close so reopening loads current apps and preferences. Remove it when the
host's own sign-in changes too, such as on sign-out, so an open embed doesn't
keep the previous user. The host owns the modal and close control;
there is no parent messaging API and no app list or credential is sent to it.
The app browser fills its iframe and responds to its size. It shows a spinner until its apps load.
This example assumes `puterOrigin` comes from the host app's deployment
settings. Translate the button and frame labels using the host app's
localization system.
```html
<button id="more-apps" type="button">More apps</button>
<dialog id="apps-dialog" aria-label="More apps">
<button id="close-apps" type="button">Close</button>
<div id="apps-frame"></div>
</dialog>
<style>
#apps-dialog { padding: 12px; width: min(900px, 90vw); }
#apps-frame iframe { width: 100%; height: min(650px, 75vh); border: 0; }
</style>
<script type="module">
const puterOrigin = 'https://puter.com'; // Use this deployment's configured origin.
const dialog = document.querySelector('#apps-dialog');
const container = document.querySelector('#apps-frame');
const trigger = document.querySelector('#more-apps');
trigger.addEventListener('click', async () => {
const url = new URL('/embed/apps', puterOrigin);
// Signed out, the embed shows the recommended apps.
const user = puter.auth.isSignedIn() ? await puter.auth.getUser().catch(() => null) : null;
if (user) url.hash = `user=${encodeURIComponent(user.uuid)}`;
const frame = document.createElement('iframe');
frame.title = 'Your apps';
frame.src = url.href;
container.replaceChildren(frame);
dialog.showModal();
});
document.querySelector('#close-apps').addEventListener('click', () => dialog.close());
dialog.addEventListener('close', () => {
container.replaceChildren();
trigger.focus();
});
</script>
```
## Implementation and checks
`createAppBrowser` in `src/gui/src/UI/Dashboard/AppBrowser.js` owns the shared UI
and data behavior; its `guestApps` option is the signed-out list. `TabApps.js`
injects desktop actions. `appsEmbed.js` supplies new-tab launching through
`launchAppInTab` (signed in) and `guestAppLaunchUrl` (signed out) in
`src/gui/src/helpers/appTabLaunch.js`, which build the same app URL as
`launch_app`, and loads only the embed entry, its SDK, and shared styles. `embedHostUser` in
`src/gui/src/helpers/embedHostUser.js` reads the host's user from the fragment, and
`embedHostTheme.js` beside it applies the host's theme.
The normal GUI webpack build emits `bundle.min.js` and `apps-embed.min.js`.
The `/embed/apps` route puts the recommended list in the page's config as
`guestApps`. No puter.js API or app-list endpoint changes are needed.
Run the GUI component/browser tests (with the SDK bundle built and Playwright
Chromium installed), and the backend route tests:
```sh
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
```
+111
View File
@@ -0,0 +1,111 @@
# Pagination convention for list APIs
Every list endpoint follows one wire contract. This document is the source of
truth for adding pagination to a new or existing list surface.
## Wire contract
Requests accept:
| Param | Type | Meaning |
| --- | --- | --- |
| `limit` | number | Maximum items per page. Each endpoint documents its cap and default. |
| `cursor` | string \| null | Opaque continuation token. `null` (or any presence of the key) requests the first page. |
| `offset` | number | Skip N items. Legacy/discouraged — see below. Cannot be combined with `cursor`. |
| `includeTotal` | boolean | Adds `total` to the response. |
Paginated responses are an envelope:
```json
{ "items": [...], "cursor": "…", "total": 123 }
```
- `cursor` is present only while more pages exist. Clients iterate until it
is absent.
- `total` is present only when the request set `includeTotal`.
- **Pages may be short.** Post-query filtering (TTL expiry, permission
checks) can shrink a page below `limit` — or even to zero — while a
`cursor` is still returned. Never use `items.length < limit` as an
end-of-list signal.
## Backward compatibility
Requests without pagination params keep returning the full result in the
legacy shape (bare array) forever — old clients never break.
The envelope trigger depends on the endpoint's history:
- Endpoints where `limit` pre-dates the convention (readdir, subdomain
`select`) return the envelope only when the request contains `cursor`
(including `null`) or `includeTotal`; `limit`/`offset`-only requests keep
the bare array.
- Endpoints where every pagination param is new (kv `list`, workers
`getFilePaths`) return the envelope when any pagination param is present.
## Cursors
Cursors are opaque base64-encoded JSON, produced and consumed only by the
backend (`src/backend/util/pagination.ts`). A cursor over a global sequence
(an auto-increment `id`) is sealed with `sealCursor`/`openCursor` instead, so
two pages can't be decoded into how fast the table grows; `openCursor` still
reads a plain cursor. What a cursor wraps is an implementation detail per
store:
- DynamoDB-backed lists wrap `LastEvaluatedKey`. A kv listing of another
namespace (cross-app or through a share handle) resumes after the last entry
it returned instead, since the scan can stop on a private one, and its cursor
is sealed to that listing and padded to one length.
- SQL-backed lists wrap a keyset position — `(sortValue, id)` of the last
row — and the query seeks past it (`WHERE (col, id) > (?, ?) ORDER BY col,
id`). Any SQL list gaining a cursor must have a deterministic `ORDER BY`
ending in a unique tiebreaker (`id`).
- The notification catch-up (`GET /events/fetch`) holds two keysets in one
sealed cursor: the last `id` it returned, and the last `(replaced_at, id)`
among rows up to that id. Each is its own indexed read, merged into one
page, so a row replaced after a client read it comes back from that cursor.
`from=end` with no cursor answers an empty page whose cursor holds the
slice's newest `id` and the slice's latest `(replaced_at, id)` as the
replacement mark.
Cursors that carry a sort also pin it: a request that passes a cursor plus a
conflicting sort is rejected with 400.
## Offset
SQL-backed endpoints support `offset` natively. DynamoDB-backed endpoints
(kv) emulate it by advancing past skipped items with `Select: COUNT` queries
— no item data is transferred, but read capacity is still consumed for
everything skipped, and the caller is metered for it. **Offset is supported
for parity, not recommended** — cost grows linearly with the offset, so it is
capped (kv: 5000). Use cursors.
## Totals (`includeTotal`)
- SQL: `SELECT COUNT(*)` with the same WHERE clause as the listing.
- DynamoDB: a `Select: COUNT` loop over the query (TTL-filtered), metered to
the caller. Cost is proportional to the total item count — request the
total on the first page only, not every page.
- Where visibility is decided per-actor after the query (protected apps in
the catalog listing), `total` approximates the visible set: it counts
non-protected plus caller-owned rows, and misses rows visible only through
explicit permission grants.
## fetchUntilFull (kv only)
DynamoDB applies `Limit` before its filter expression, so TTL-filtered pages
are structurally short. `fetchUntilFull: true` makes the backend keep
fetching (bounded number of continuation queries) until the page holds
`limit` items or the keyset is exhausted. Requires `limit`. If the bound is
hit, the response simply carries a cursor — still convention-legal.
## Adding pagination to a new endpoint
1. Use `encodeCursor`/`decodeCursor`/`normalizeLimit`/`normalizeOffset` from
`src/backend/util/pagination.ts` — `sealCursor`/`openCursor` when the
cursor wraps a global id.
2. Push equality filters into the query so pages and counts operate on the
true result set; only genuinely per-actor filtering may remain post-query
(short-pages rule covers it).
3. Fetch `limit + 1` rows to detect whether a next page exists (SQL), or use
`LastEvaluatedKey` (DynamoDB).
4. Keep the no-params request returning the legacy full result.
+80
View File
@@ -0,0 +1,80 @@
# Browser push client
Puter's desktop shows pushes through `/sw.js`. Core shows Settings → Notifications only when the `pushEnabled` gui param is `true`; core never sets it. The worker is registered only when push is turned on in the browser, and a load removes one left without a subscription. A backend extension that delivers pushes sets it with `services.homepage.setGuiParam('pushEnabled', true)` and can load its own code into the worker with `services.homepage.registerServiceWorkerScript(path)`.
Core subscribes the browser to a push service only when the server also publishes `pushWebPushKey` (see [Built-in Web Push client](#built-in-web-push-client)), and it has no `/push/*` routes of its own. The device, app, preference and subscription endpoints below come from that extension. On a deployment without one, such as most self-hosted servers, they answer core's plain 404 (`code: 'not_found'`), which `puter.notifications` reports as `push_unavailable` and the Notifications tab shows as unavailable. A device the extension can't find is `device_not_found`.
## Push payload
`sw.js` reads the push's JSON body. The message is its `puter` field, or `data.puter` for services that only carry string data, as an object or a JSON string:
```ts
{ uid?: string; title?: string; body?: string; icon?: string; group?: string;
target?: { app?: string; path?: string }; data?: unknown; channel?: string;
type?: 'challenge'; challenge?: string }
```
`uid` is the notification's uid, a later push with the same `group` replaces the one showing, and `target` and `data` reach the app when the user taps it.
When the browser replaces the subscription (`pushsubscriptionchange`), `sw.js` posts `{ type: 'puter-push-subscription-changed' }` to every open window on Puter's origin. The worker has no token or session, so the page's push client registers the new subscription.
## Receipts
Before a notification is pushed, the server asks the open clients whether they received it live. It sends `notif.receipt` with `{ uid }` on the user's sockets for the app that sent it, or on the desktop's sockets for Puter's own notifications. The SDK remembers the uids its page received through `onNotification()`, `onPush()` or the desktop's feed, and answers once its page has that notification, or with `{ received: false }` when it never gets it. The server waits up to 5 seconds (`notificationAckTimeoutMs`).
An answer carries `visibilityState` and `focused` as they were when the page received the notification. Pages never show a browser notification themselves; pushes do.
- An answer from a page that was visible and focused stops the push (`handled` on `notif.delivery.settled`). An older client that doesn't send `focused` counts while visible.
- Otherwise the notification is pushed, to every browser the user subscribed. A website's pushes go only to the browsers that subscribed through its own worker, never to Puter's `/sw.js`. An app inside the desktop pushes through `/sw.js`, which shows the app's name and icon.
- A client that can't answer, such as an older puter.js, doesn't hold the push back.
The ask reaches only the sockets in the asking region. So that other regions hear about a page too, a page that receives a notification live while visible and focused also sends `notif.handled` with `{ uid }` on the socket it came on. Core relays it as the node-local `notif.handled` event with the socket's user and app, at most 60 a minute per user; an extension that delivers pushes decides what to do with it.
## Registering
The registry is in [src/gui/src/helpers/pushClient.js](../src/gui/src/helpers/pushClient.js). A service script registers its client like this:
```js
service_script(async (api) => {
api.use()['push.registerPushClient']?.(client);
});
```
`use(id)` throws when nothing is registered under `id`, so the form above also runs on a Puter version without the registry and does nothing there. Any script that runs after the GUI bundle can call `use()['push.registerPushClient']?.(client)` the same way. Registering again replaces the client. Registering dispatches `push-client-registered` on `window`, and an open Notifications tab redraws.
With no client registered, the tab says push notifications aren't available in this browser.
### Built-in Web Push client
When the server also sets the `pushWebPushKey` gui param (a VAPID public key), core registers its own client ([webPushClient.js](../src/gui/src/helpers/webPushClient.js)) unless an extension registered one first. It uses `puter.notifications.enable()` / `disable()`, the same standard Web Push flow apps use on their own origins:
1. `GET /push/web/key` for the key, then `pushManager.subscribe()` on the `/sw.js` registration.
2. `POST /push/web/subscriptions { subscription }`. A subscription the server already has answers `{ status: 'active' }`. A new one answers `{ status: 'pending' }`, and the server sends it a push with `{ type: 'challenge', challenge }`.
3. The worker shows that push and posts `{ type: 'puter-push-challenge', challenge }` to the origin's windows; the page sends it to `POST /push/web/subscriptions/verify`. The device is registered once the page hands the confirmation back.
4. `DELETE /push/web/subscriptions { endpoint }` removes it.
Apps on their own origin do the same through `push-sw.js`, which they import into their own worker. A subscription made with an app token is the app's: its pushes are that app's notifications only.
## Interface
```ts
interface PushClient {
enable(): Promise<{ deviceUid: string }>;
disable(): Promise<void>;
status(): Promise<{
permission: 'default' | 'granted' | 'denied';
registered: boolean;
deviceUid?: string;
}>;
}
```
- `enable()` may ask for notification permission, so core calls it from a click: Turn on in the Notifications tab, Allow on a permission prompt that includes `notifications:push` while this browser is off, and the "Turn on notifications" button of the `/action/enable-push` popup and the matching desktop dialog (`UIEnablePushDialog`). Core also calls it without a click, from `helpers/pushSync.js`, when `Notification.permission` is already `granted`, `status()` reports the browser as not registered, the user has not turned push off in this browser, and an app holds `notifications:push`; the browser does not prompt then. The built-in client runs `puter.notifications.enable()`, which subscribes with the `/sw.js` registration and registers the browser through the Web Push flow above (`platform: 'webpush'`). It then resolves the `uid` of this browser's `webpush` device from `listDevices()`. It rejects when permission is refused or the browser wasn't registered.
- `disable()` runs `puter.notifications.disable()`, which sends `DELETE /push/web/subscriptions { endpoint }` and ends the browser's subscription. The tab calls it for Turn off, and for Remove on the device marked as this browser. Remove on any other device is `DELETE /push/devices/:uid`.
- `status()` reports the state without prompting. `permission` is `Notification.permission`. `registered` is true when this browser has a device registered for the signed-in user, and `deviceUid` is that device's `uid`.
After `enable()` resolves, the tab calls `POST /push/devices/:uid/test` and asks the user whether a notification showed up.
The tab also reads `GET /push/apps` and `GET`/`POST /push/preferences`.
Core keeps its own record of a browser the user turned off, `puter.push.browser-off:<user uuid>` in `localStorage`, written by Turn off and by Remove on this browser's device, and cleared when `enable()` resolves. The silent registration above skips a browser with that record, so a device removed here stays removed while one removed from another device, or a new sign-in in a browser that already allows notifications, registers again.
+250 -35
View File
@@ -1,6 +1,6 @@
# Self-Hosting Puter
`docker-compose.yml` brings up Puter **plus every external service it needs** — MariaDB, Valkey, DynamoDB-local, RustFS S3, nginx — wired together. Closest thing to a production deployment you can self-manage on a single host.
`docker-compose.yml` brings up Puter **plus every external service it needs** — MariaDB, Valkey, DynamoDB-local, RustFS S3, Caddy — wired together. Closest thing to a production deployment you can self-manage on a single host.
## One-line installer (recommended)
@@ -8,11 +8,21 @@
curl -fsSL https://raw.githubusercontent.com/HeyPuter/puter/main/install.sh | sh
```
Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker-compose.yml` from the OSS repo, and runs `docker compose up -d`. Re-running is safe — it won't overwrite existing config (set `PUTER_FORCE=1` to rotate). Use the manual steps below if you want to inspect or tweak each step yourself.
Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker-compose.yml` + `caddy/Caddyfile` from the OSS repo, and runs `docker compose up -d`. Re-running is safe — it won't overwrite existing config (set `PUTER_FORCE=1` to rotate). Use the manual steps below if you want to inspect or tweak each step yourself.
## Requirements
- **Docker** with the `compose` plugin.
- **Docker** with the `compose` plugin. This is the one thing you have to install
yourself — the installer checks for it and stops with an explanation rather
than installing it for you.
- **macOS / Windows:** [Docker Desktop](https://docs.docker.com/get-docker/).
Both ship the `compose` plugin.
- **Linux:** [Docker Engine](https://docs.docker.com/engine/install/) plus the
[`docker-compose-plugin`](https://docs.docker.com/compose/install/) package.
- Confirm before running anything else: `docker compose version` should print a
version, not `command not found`.
- If Docker is installed but the shell can't see it, make sure your user is in
the `docker` group (Linux) or that Docker Desktop is actually running.
- A **domain** with DNS access — you need a wildcard record (`*.your-domain.com` → server IP). Puter routes by subdomain (`api.<domain>`, `site.<domain>`, `app.<domain>`).
- Optional: **TLS certs** (or `certbot` to grab them — see Step 3).
@@ -20,7 +30,7 @@ Generates secrets, writes `.env` + `puter/config/config.json`, downloads `docker
| Container | Image | Role |
| --------------- | ------------------------ | ---------------------------------------------------------- |
| `puter-nginx` | `nginx:1.27-alpine` | Reverse proxy on 80 (and 443 if TLS); forwards to Puter |
| `puter-caddy` | `caddy:2.11-alpine` | Reverse proxy on 80 (and 443 if TLS); forwards to Puter |
| `puter` | `ghcr.io/heyputer/puter` | The app |
| `puter-mariadb` | `mariadb:11` | SQL database — schema applied automatically on first boot |
| `puter-valkey` | `valkey/valkey:8-alpine` | Redis-compatible cache + rate-limiter |
@@ -35,6 +45,12 @@ Optional services (compose profile `ai`, opt-in):
| `puter-ollama` | `ollama/ollama` | Local LLM provider (CPU; GPU passthrough opt-in) |
| `puter-ollama-init` | `ollama/ollama` | One-shot — pulls the default model (`tinyllama`) on first boot |
Optional services (compose profile `smtp`, opt-in):
| Container | Image | Role |
| ------------ | ------------------------ | -------------------------------------------------------- |
| `puter-smtp` | `ghcr.io/heyputer/puter` | Receives mail over SMTP and files it into user mailboxes |
State lives under `./puter/data/<service>/`.
---
@@ -47,7 +63,6 @@ State lives under `./puter/data/<service>/`.
MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
@@ -65,7 +80,7 @@ S3_SECRET_KEY=$S3_SECRET_KEY
S3_BUCKET=puter-local
EOF
mkdir -p puter/config puter/data puter/tls
mkdir -p puter/config puter/data puter/data/caddy puter/tls
cat > puter/config/config.json <<EOF
{
"domain": "puter.localhost",
@@ -78,9 +93,7 @@ cat > puter/config/config.json <<EOF
"private_app_hosting_domain": "app.puter.localhost",
"private_app_hosting_domain_alt": "dev.puter.localhost",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"allow_v1_tokens": true,
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
@@ -125,6 +138,10 @@ cat > puter/config/config.json <<EOF
"ollama": { "enabled": false }
},
"meteringEnforcement": {
"subscriptions": false
},
"trust_proxy": 1
}
EOF
@@ -134,21 +151,32 @@ Replace `puter.localhost`, `site.puter.localhost`, `host.puter.localhost`, `dev.
Why these knobs:
- `jwt_secret` + `jwt_secret_v2` — Puter signs new auth tokens with `jwt_secret_v2` (v2 token format, `kid: 'v2'` JWT header). `jwt_secret` is verify-only and lets tokens minted before this version (cookies still in users' browsers, stored API tokens) keep working. `allow_v1_tokens: true` keeps that fallback enabled. Fresh installs can leave it as-is; future versions will flip the flag to `false` and retire `jwt_secret` entirely once legacy tokens have drained.
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start=gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `jwt_secret_v2` — the HMAC secret Puter signs and verifies auth tokens with (`kid: 'v2'` JWT header). The pre-v2 token format is retired: a token signed with the old `jwt_secret` no longer verifies, and holders are asked to sign in again. If you are upgrading from a release that had `jwt_secret`, drop it from your config — it is ignored.
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start:gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
- `meteringEnforcement.subscriptions: false` — a few surfaces (the OpenAI/Anthropic-compatible AI endpoints) are declared paid-plan-only. A self-hosted install has no paid plans, so every account sits on the free policy and those routes would answer `402 subscription_required` for everyone. Turning the plan gates off opens them; nothing else about metering changes.
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
- `s3.s3Config.forcePathStyle: true` — RustFS / MinIO / fauxqs need path-style URLs (`<endpoint>/<bucket>`). Real AWS S3 wants virtual-hosted (`<bucket>.<endpoint>`) — drop this flag (or set `false`) when you swap to real S3.
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. nginx routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
- `trust_proxy: 1` — nginx terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the nginx container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (nginx). Bump to `2` if you put Cloudflare in front of nginx; never set `true` (it trusts every hop and makes XFF forgeable).
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. Caddy routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
- **Bucket CORS** — the browser uploads file bytes straight to RustFS with a cross-origin `PUT` to the presigned URL (Dev Center deploy, `puter.fs.upload`). That only works if the bucket answers the `OPTIONS` preflight with `Access-Control-Allow-*` headers. The `s3-init` container applies those rules on every boot (`put-bucket-cors`, origins `*` — the signed URL is the credential, and the SDK sends no cookies). If deploys fail with `No 'Access-Control-Allow-Origin' header is present`, re-run it: `docker compose run --rm s3-init`. Verify with:
```bash
curl -sk -X OPTIONS "https://s3.<domain>/<bucket>/test" \
-H "Origin: https://<domain>" \
-H "Access-Control-Request-Method: PUT" \
-H "Access-Control-Request-Headers: content-type" -D - -o /dev/null | grep -i access-control
```
You should see `access-control-allow-origin`, `allow-methods: GET, HEAD, PUT, POST, DELETE`, `allow-headers: *`, and `max-age: 3600`.
- `trust_proxy: 1` — Caddy terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the Caddy container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (Caddy). Bump to `2` if you put Cloudflare in front of Caddy; never set `true` (it trusts every hop and makes XFF forgeable).
> If you ever change `MARIADB_PASSWORD` after first boot, `.env` alone won't update MariaDB — its credentials are baked into `./puter/data/mariadb/` on first init. Either rotate the password inside MariaDB by hand or `docker compose down && rm -rf ./puter/data/mariadb` to start fresh.
## Step 2 — Point DNS at the server \[Optional\]
In your DNS provider, add records for the main domain plus the subdomains Puter and nginx route on (`api.*`, `site.*`, `app.*`, `s3.*`):
In your DNS provider, add records for the main domain plus the subdomains Puter and Caddy route on (`api.*`, `site.*`, `app.*`, `s3.*`):
```
A puter.localhost → <your server's public IP>
@@ -163,7 +191,7 @@ A dev.puter.localhost → <your server's public IP>
A *.dev.puter.localhost → <your server's public IP>
```
The wildcards are required — Puter routes via subdomains (`api.*`, `app.*`, etc.) and nginx routes browser S3 traffic via `s3.*` to RustFS.
The wildcards are required — Puter routes via subdomains (`api.*`, `app.*`, etc.) and Caddy routes browser S3 traffic via `s3.*` to RustFS.
## Step 3 — TLS (recommended for public installs) \[Optional\]
@@ -182,13 +210,15 @@ sudo certbot certonly --manual --preferred-challenges dns \
The cert needs to cover `*.puter.localhost` so that `s3.puter.localhost` (browser S3 endpoint), plus Puter's own `api.*` / `app.*` subdomains, all validate.
> **Why not Caddy's automatic HTTPS?** Caddy issues certs over ACME by itself, but only for hostnames it knows up front, and only over HTTP-01 with the stock image. Puter serves every user's site and app on a subdomain it invents at runtime (`<name>.site.<domain>`, `<name>.app.<domain>`), which needs a **wildcard** cert — and wildcards require DNS-01, which requires a DNS-provider plugin that isn't in `caddy:2.11-alpine`. So `caddy/Caddyfile` sets `auto_https off` and reads the cert you supply below. If you'd rather have Caddy manage certs, build an image with the plugin for your DNS provider (`xcaddy build --with github.com/caddy-dns/<provider>`) and swap the `tls` line for a `tls { dns <provider> … }` block.
Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
**Wire nginx to use them:**
**Wire Caddy to use them:**
1. Open [nginx/nginx.conf](../nginx/nginx.conf), uncomment **both** `# server { listen 443 ssl … }` blocks (one for `s3.*`, one for the catch-all).
2. (Optional) Replace the body of the port-80 blocks with `return 301 https://$host$request_uri;` to force HTTPS everywhere.
3. In [docker-compose.yml](../docker-compose.yml), uncomment the `443:443` port mapping under the `nginx` service.
1. Open [caddy/Caddyfile](../caddy/Caddyfile) and uncomment the `# :443 { … }` block at the bottom.
2. (Optional but recommended) Replace the plain `:80 { import puter_routes }` block with the `redir` version shown alongside it, to force HTTPS everywhere.
3. In [docker-compose.yml](../docker-compose.yml), uncomment the `443:443` port mapping under the `caddy` service.
4. In `.env`, uncomment `HTTPS_PORT=443`.
5. In `config.json`, switch:
```json
@@ -196,14 +226,14 @@ Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
```
…and update the S3 public endpoint:
```json
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.local", ... } }
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.localhost", ... } }
```
## Running behind your own reverse proxy
The bundled `puter-nginx` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Caddy, Traefik, HAProxy, a cloud load balancer, another nginx). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
The bundled `puter-caddy` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Traefik, nginx, HAProxy, a cloud load balancer, another Caddy). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
You can keep the bundled nginx as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
You can keep the bundled Caddy as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
**The rules that actually matter** (getting any of these wrong is what causes the redirect loops and "Invalid Host header" failures people hit):
@@ -219,7 +249,7 @@ You can keep the bundled nginx as the single hop your proxy talks to, or bypass
- `X-Forwarded-For` — the real client IP (rate limiting + audit logs).
- `Upgrade` / `Connection` — passed through for WebSocket / socket.io upgrades, or the realtime connection silently fails.
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled nginx if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled Caddy if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
6. **Route the wildcard to Puter.** Your proxy must forward `*.<domain>` (covering `api.*`, `app.*`, and `s3.<domain>`) and `*.site.<domain>` (and any other hosting domains) to Puter — same wildcard DNS as Step 2. Puter does the per-subdomain routing internally; the proxy just needs to hand it the traffic with the Host intact.
@@ -227,6 +257,8 @@ With those in place the rest of the stack is unchanged — `docker compose up -d
## Step 4 — Bring it up
Both [docker-compose.yml](../docker-compose.yml) and [caddy/Caddyfile](../caddy/Caddyfile) need to sit next to your `.env` — clone the repo, or copy those two paths out of it. The Caddyfile is bind-mounted read-only; if it's missing, Docker creates a *directory* at that path and `puter-caddy` dies with "not a directory".
```bash
docker compose up -d
```
@@ -241,15 +273,15 @@ Healthy startup:
```
[config] override from /etc/puter/config.json
[mysql] running migrations from /opt/puter/dist/src/backend/clients/database/migrations/mysql: 2 file(s)
[mysql] running migrations from /opt/puter/dist/src/backend/clients/database/migrations/mysql: <n> file(s)
[mysql] applied mysql_mig_1.sql (...)
[mysql] applied mysql_mig_2.sql (9 statements)
...
```
Then open **<https://puter.local>** (or `http://` if you skipped TLS). Login is `admin` — the temp password is printed once in the puter container logs on first boot:
Then open **<https://puter.localhost>** (or `http://` if you skipped TLS). Login is `admin`; its temporary password is printed in the puter container logs on every boot until you change it:
```bash
docker compose logs puter | grep tmp_password
docker compose logs puter | grep "password for admin"
```
Change it in Settings after first login.
@@ -294,6 +326,18 @@ Used for password resets, email confirmation, and notifications. Without it thos
To require email confirmation before login, also set `"strict_email_verification_required": true`.
To read the mail yourself instead of sending it, point the transport at a local SMTP sink — [MailHog](https://github.com/mailhog/MailHog) catches everything and serves it at `http://localhost:8025`. Publish both ports; a container that only exposes them is unreachable from the host:
```sh
docker run -d -p 1025:1025 -p 8025:8025 mailhog/mailhog
```
```json
"email": { "from": "\"Puter\" <[email protected]>", "host": "127.0.0.1", "port": 1025, "secure": false, "ignoreTLS": true }
```
Share email is on once a transport exists: a recipient who already has an account is emailed as well as notified in-app, and they opt out with the unsubscribe link the mail carries. Set `"share_email_notifications": false` to keep those in the app only — an address with no account is still emailed, since there is nothing else to reach them with.
### Sign in with Google (or another OIDC provider)
```json
@@ -360,14 +404,56 @@ Full provider list (chat, image, video, TTS, OCR) is in the template.
### Per-user storage quota
Default is 100 MB per user.
Storage is unlimited by default (bounded by host disk). To cap each user:
```json
"storage_capacity": 5368709120, // 5 GB
"is_storage_limited": true
```
Set `is_storage_limited: false` for unlimited (bounded by host disk).
`storage_capacity` defaults to 100 MB.
### Usage metering and budgets
Puter meters what an account costs to serve — bytes sent back, object-store
requests, KV capacity, AI tokens — against a monthly budget, and refuses the
operations that spend it once that budget is gone. The refusal is a `402` with
code `insufficient_funds`; reads that only describe things, and every kind of
deletion, stay available so an account can always see what it has and clear it.
On a self-hosted install this is almost certainly not what you want. There is
nowhere to buy more, so accounts are held to the free monthly allowance
(US$0.50 of measured cost for a registered account, roughly 2.5 GiB of
downloads) and start being refused after that. Turn it off:
```json
"unlimitedMetering": true
```
Every account then resolves to an unlimited policy. Usage is still recorded, so
the dashboard still shows what is being consumed; nothing is ever refused for
lack of budget.
To keep the budgets but stop them blocking anything — recording only:
```json
"meteringEnforcement": { "enabled": false }
```
Calls driven by a deployed worker are exempt from enforcement by default,
because a worker has no prompt to show and nobody watching it fail. Set
`"meteringEnforcement": { "workers": true }` to include them.
Monthly usage records expire on their own once they're old enough to be of no
further use — `meteringRetentionMonths` full months past the month they
belong to, 3 by default. Set it to `0` to keep every month's records forever:
```json
"meteringRetentionMonths": 6
```
Changing it, including to `0`, applies to records first written after the
change; records already stamped keep their expiry.
### Captcha on signup / login
@@ -379,9 +465,19 @@ Built-in proof-of-work captcha — no external service needed.
`difficulty` is one of `easy` / `medium` / `hard`.
### Block disposable email TLDs
### Disable new signups
Only enforced when `env: "prod"`.
Force visitors to log in with an existing account instead of creating a
temporary or permanent one.
```json
"disable_user_signup": true
```
### Block disposable email domains
Addresses on these domains (suffix match, so subdomains too) can't sign up or be
set as an account's email.
```json
"blockedEmailDomains": ["mailinator.com", "tempmail.com", "guerrillamail.com"]
@@ -428,14 +524,125 @@ Without `--profile ai`, the `ollama` containers stay down and Puter (with `enabl
For GPU acceleration (NVIDIA), uncomment the `deploy:` block under the `ollama` service in [docker-compose.yml](../docker-compose.yml). Requires `nvidia-container-toolkit` on the host.
## Optional: receiving mail over SMTP
Puter can give each account a mailbox at a domain you control. Two pieces are
involved, and they are configured together under `userEmail`:
- an **ingress endpoint** (`POST /email/ingress`) that files a message into the
recipient's `~/.mail`. It exists whenever `userEmail.secret` is set.
- the **`puter-smtp` receiver**, which accepts mail from other mail servers and
hands each message to that endpoint. It runs as its own process, behind the
`smtp` compose profile.
> **Read this before you expose it.** An inbound mail server is unauthenticated
> by nature: anyone on the internet may hand it a message for any address in
> `localDomains`. Puter performs **no** spam filtering and **no** SPF, DKIM or
> DMARC checks — whatever is delivered is stored in the recipient's mailbox and
> counts against their storage. If that is not acceptable, put a filtering relay
> in front of it and point `localDomains` at that instead.
1. Configure it in `puter/config/config.json`:
```json
"userEmail": {
"secret": "a long random string",
"localServer": true,
"localDomains": ["example.com"]
}
```
`secret` is shared between the two halves; generate it with
`openssl rand -hex 32`. `localDomains` lists the domains this server accepts
mail for — anything else is refused at `RCPT TO`, which is what stops it
being used as an open relay. It has no default, and the receiver refuses to
start without it.
The receiver posts to `<api_base_url>/email/ingress` unless you set
`localIngressUrl`. The endpoint is served on the `api` subdomain, so if you
point it at an internal address to skip the reverse proxy, name that virtual
host too or the request will 404:
```json
"localIngressUrl": "http://puter:4100/email/ingress",
"localIngressHost": "api.example.com"
```
`localIngressHost` defaults to the host of `api_base_url`, which is usually
what you want.
2. Bring it up with the `smtp` profile:
```bash
docker compose --profile smtp up -d
docker compose logs -f puter-smtp
```
3. Point DNS at it:
```dns
mx.example.com. A 203.0.113.10
example.com. MX 10 mx.example.com.
```
Also set a `PTR` record for the address that resolves back to
`mx.example.com` — many senders refuse mail from a server without one.
Inbound TCP port 25 must reach the host. **Most cloud providers block port
25 by default and residential ISPs block it permanently**; if you cannot get
it unblocked, this will not work no matter how it is configured. The
container itself binds 2525 (it runs unprivileged) and compose publishes
`${SMTP_PORT:-25}` onto it, so you can move the host port if you need to.
A message is held in memory while it is being delivered, so peak memory is
bounded by `localMaxClients` (default 20) times the 25 MiB message ceiling.
Lower `localMaxClients` on a small host. There is no queue and no retry
schedule — if delivery fails, the sending server is told to try again later and
retrying is its job.
Mail is stored as `message/rfc822` objects under `~/.mail/objects/<date>/` and
read back through `puter.email` in the SDK.
Every setting is documented in
[config.template.jsonc](../config.template.jsonc). Two deliberate limits, both
in the name of keeping the self-hosted path simple: it does not offer STARTTLS,
so mail arrives in the clear, and it applies no per-sender rate limiting. Put a
filtering relay in front if either matters to you.
Note this is a *receiving* server only: it never sends, and it must not be used
for authenticated submission. Outbound transactional mail is the separate
`email` block above.
## Building from source instead of pulling
If you want to test local Dockerfile changes against the full stack, uncomment the `build:` block in [docker-compose.yml](../docker-compose.yml) under the `puter` service, change `pull_policy: always` → `pull_policy: never`, then:
To run a local build against the full stack, use a source checkout and complete
the configuration steps above. Create `docker-compose.override.yml` in the
repository root, next to [docker-compose.yml](../docker-compose.yml):
```yaml
services:
puter:
pull_policy: never
build:
context: .
```
If that file already exists, merge these settings into its `puter` service.
Compose merges the override automatically: `build.context: .` builds this
checkout, and `pull_policy: never` stops it pulling the published image. Git
ignores the override, so local build settings stay out of pull requests and
don't conflict with `docker-compose.yml` when you pull updates.
From the repository root, build and start the stack:
```bash
docker compose up -d --build
```
Run the same command after changing or pulling source code to rebuild the image.
To return to the published image, remove the local build settings (or the override
file if those are its only settings), then run `docker compose up -d`.
---
## Managing running backend
@@ -460,14 +667,22 @@ Migrations re-apply idempotently across pulls. Volumes are preserved.
## Troubleshooting
**Site loads but I get "Bad Gateway" / nginx errors.**
**`missing required command: docker` — the installer stops immediately.**
Docker isn't on your `PATH` yet. Puter self-hosting is a Docker Compose stack, so
nothing can run without it. Install [Docker Desktop](https://docs.docker.com/get-docker/)
on macOS/Windows or [Docker Engine](https://docs.docker.com/engine/install/) plus
`docker-compose-plugin` on Linux, then confirm `docker compose version` works in
the same shell and re-run the installer. On Linux, a fresh Docker install usually
needs a new login session before your user picks up the `docker` group.
**Site loads but I get a 502 / "Bad Gateway" from Caddy.**
The puter container failed to come up. `docker compose logs puter` will tell you which dependency rejected it (most often DB password mismatch between `.env` and `config.json`).
**Login screen says "admin password not set".**
First-boot temp password is logged once. Find it: `docker compose logs puter | grep "tmp_password"`. After login, change it in Settings.
**Can't find the admin password.**
It's logged on every boot until you change it: `docker compose logs puter | grep "password for admin"`. After login, change it in Settings.
**Healthcheck reports unhealthy but the site works.**
The healthcheck hits `puter.localhost:4100/test` from inside the container. If you changed `domain` or `port`, the check still uses defaults. The site itself is fine.
The compose healthcheck requests `http://localhost:4100/` from inside the container. If you changed `port` or set `allow_all_host_values: false`, the check fails while the site itself is fine.
**Nothing resolves at `puter.example.com` after DNS changes.**
DNS propagates slowly. `dig puter.example.com` and `dig api.puter.example.com` should both return your server IP. If not, give it 5–60 minutes.
+74 -23
View File
@@ -2,7 +2,7 @@
# Self-hosted Puter — full stack.
#
# Brings up Puter + every external service it needs:
# - nginx : reverse proxy (mirrors prod ALB; handles TLS + Host fan-out)
# - caddy : reverse proxy (mirrors prod ALB; handles TLS + Host fan-out)
# - valkey : redis-compatible cache / rate-limiter backend
# - mariadb : SQL database (Puter applies its schema on first boot)
# - dynamo : DynamoDB-local (KV store; Puter creates the table itself)
@@ -23,7 +23,8 @@
#
# Production:
# - Always replace the default passwords / S3 keys / Puter secrets.
# - Front Puter with TLS-terminating reverse proxy (Caddy / nginx).
# - Enable TLS on the bundled Caddy (see caddy/Caddyfile) or front the
# stack with your own TLS-terminating proxy.
# - Move state-bearing volumes to a backed-up location.
services:
@@ -118,7 +119,7 @@ services:
RUSTFS_SECRET_KEY: ${S3_SECRET_KEY:-puter-secret-change-me}
volumes:
- ./puter/data/s3:/data:z
# Internal-only — browsers reach RustFS via nginx (`s3.<domain>`),
# Internal-only — browsers reach RustFS via Caddy (`s3.<domain>`),
# which preserves the Host header for S3 signature validation and
# rides the same TLS termination as Puter. Uncomment to also expose
# 9000 directly on the host for `aws-cli` / debugging.
@@ -138,8 +139,10 @@ services:
start_period: 5s
s3-init:
# One-shot container that creates the `puter-local` bucket on first
# boot. Exits 0 once the bucket exists; stays exited 0 thereafter.
# One-shot container that creates the `puter-local` bucket and
# applies its CORS rules on every boot (put-bucket-cors is
# idempotent). Without CORS, browser uploads to presigned URLs
# fail the preflight check.
image: amazon/aws-cli:latest
container_name: puter-s3-init
depends_on:
@@ -162,6 +165,26 @@ services:
echo "creating bucket $$bucket"
aws --endpoint-url "$$endpoint" s3 mb "s3://$$bucket"
fi
# Browser uploads (Dev Center deploy, puter.fs.upload) PUT to
# presigned URLs cross-origin, so the bucket must answer the
# OPTIONS preflight. Origins are open because the presigned URL
# itself is the credential and the SDK sends no cookies.
cat > /tmp/cors.json <<'EOF'
{
"CORSRules": [
{
"AllowedOrigins": ["*"],
"AllowedMethods": ["GET", "HEAD", "PUT", "POST", "DELETE"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag", "x-amz-request-id"],
"MaxAgeSeconds": 3600
}
]
}
EOF
aws --endpoint-url "$$endpoint" s3api put-bucket-cors \
--bucket "$$bucket" --cors-configuration file:///tmp/cors.json
echo "bucket CORS configured"
restart: "no"
# ── Optional: local LLM ───────────────────────────────────────────
@@ -229,15 +252,8 @@ services:
puter:
image: ghcr.io/heyputer/puter:main
pull_policy: always
# Uncomment to build from this directory instead of pulling the published
# image. Also flip pull_policy to `never` so compose doesn't overwrite
# your local build by re-pulling :latest.
# build:
# context: .
# # buildx-only: cross-compile to both archs in a single push
# # platforms:
# # - linux/amd64
# # - linux/arm64
# For local builds, create a docker-compose.override.yml;
# see doc/self-hosting.md#building-from-source-instead-of-pulling.
container_name: puter
restart: unless-stopped
depends_on:
@@ -249,7 +265,7 @@ services:
condition: service_started
s3-init:
condition: service_completed_successfully
# Internal-only: nginx reaches it on the compose network. Uncomment
# Internal-only: Caddy reaches it on the compose network. Uncomment
# to also expose port 4100 directly on the host (useful for debugging).
# ports:
# - "4100:4100"
@@ -270,23 +286,58 @@ services:
retries: 3
start_period: 30s
nginx:
image: nginx:1.27-alpine
container_name: puter-nginx
# Inbound mail. Behind the `smtp` profile, so it only starts when asked for:
# docker compose --profile smtp up -d
# Accepts SMTP from other mail servers and hands each message to the ingress
# endpoint, which files it into the recipient's mailbox. Needs `userEmail`
# configured — see doc/self-hosting.md#optional-receiving-mail-over-smtp.
puter-smtp:
profiles: ["smtp"]
image: ghcr.io/heyputer/puter:main
pull_policy: always
container_name: puter-smtp
restart: unless-stopped
command: ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/smtp/index.js"]
depends_on:
puter:
condition: service_healthy
ports:
# The container runs unprivileged and binds 2525; 25 is what an MX record
# means, and many providers block it.
- "${SMTP_PORT:-25}:2525"
environment:
PUID: 1000
PGID: 1000
volumes:
# The same config.json the backend reads; only `userEmail` applies here.
- ./puter/config:/etc/puter:z
caddy:
image: caddy:2.11-alpine
container_name: puter-caddy
restart: unless-stopped
depends_on:
puter:
condition: service_started
ports:
- "${HTTP_PORT:-80}:80"
# Uncomment when you enable TLS in nginx/nginx.conf:
# Uncomment when you enable TLS in caddy/Caddyfile:
# - "${HTTPS_PORT:-443}:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro,z
# TLS certs (fullchain.pem + privkey.pem). Read-only inside.
- ./puter/tls:/etc/nginx/tls:ro,z
- ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro,z
# TLS certs (wildcard fullchain.pem + privkey.pem). Read-only inside.
- ./puter/tls:/etc/caddy/tls:ro,z
# Caddy's own state — the cert store it would use if you swap this
# config over to ACME. Kept under ./puter/data with everything else
# so one directory is the whole backup.
- ./puter/data/caddy:/data:z
healthcheck:
test: ["CMD-SHELL", "wget -qO- --tries=1 --timeout=2 http://localhost/ || exit 1"]
# Hits Caddy's local-only admin API rather than proxying through to
# Puter: this reports whether the proxy itself is up and configured,
# and keeps working once the `:80` block becomes an HTTPS redirect.
# 127.0.0.1, not localhost — the admin endpoint is IPv4-only and
# busybox wget tries ::1 first.
test: ["CMD-SHELL", "wget -qO- --tries=1 --timeout=2 http://127.0.0.1:2019/config/ >/dev/null || exit 1"]
interval: 10s
timeout: 3s
retries: 5
+146 -142
View File
@@ -1,75 +1,92 @@
import type { Request, Response } from 'express';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import {
handleAppTelemetryUserCount,
handleAppTelemetryUsers,
} from './appTelemetry.ts';
interface CapturedResponse {
body: unknown;
}
const makeReq = (query: Record<string, unknown> = {}): Request =>
({ query }) as unknown as Request;
const makeRes = () => {
const captured: CapturedResponse = { body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
// `handleAppTelemetryUsers` calls `Context.get('actor')` for the permission
// gate. Wrap test calls in `runWithContext` so the ALS lookup resolves.
const callWithActor = async (
actor: { user: { uuid: string; id?: number }; app?: null } | undefined,
fn: () => Promise<void>,
) => runWithContext({ actor }, fn);
// Importing the module registers the `appTelemetry` driver into the shared
// extensionStore, so `setupTestServer` instantiates it on `server.drivers`.
import { AppTelemetryDriver } from './appTelemetry.ts';
let server: PuterServer;
let driver: AppTelemetryDriver;
// `get_users` reads `Context.get('actor')` for the ownership gate. Wrap
// actor-dependent calls in `runWithContext` so the ALS lookup resolves.
const callWithActor = <T>(
actor: { user: { uuid: string; id?: number } } | undefined,
fn: () => Promise<T>,
) => runWithContext({ actor }, fn);
beforeAll(async () => {
server = await setupTestServer();
driver = (server.drivers as unknown as Record<string, AppTelemetryDriver>)
.appTelemetry;
// Guard: the driver must have been wired onto the server. If this is
// undefined the extension didn't register, and every test below would
// fail with a confusing "cannot read property of undefined".
expect(driver).toBeInstanceOf(AppTelemetryDriver);
});
afterAll(async () => {
await server.shutdown();
await server?.shutdown();
});
describe('appTelemetry extension — handleAppTelemetryUsers', () => {
const seedOwnedApp = async (prefix: string) => {
const owner = await server.stores.user.create({
username: `${prefix}_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `${prefix}_${slug}`,
title: `${prefix} ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
return { owner, app: app! };
};
const seedUser = async (prefix: string, email: string | null = null) =>
server.stores.user.create({
username: `${prefix}_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email,
});
const grantAuthenticated = async (appId: number, userId: number) => {
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, 'flag:app-is-authenticated', null],
);
};
const grantEmailRead = async (
appId: number,
userId: number,
userUuid: string,
) => {
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, `user:${userUuid}:email:read`, null],
);
};
describe('appTelemetry driver — get_users', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(makeReq({}), res),
).rejects.toMatchObject({
await expect(driver.get_users({})).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('app_uuid'),
});
});
it('throws HttpError(400) for a non-numeric limit', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: 'app-anything', limit: 'banana' }),
res,
),
driver.get_users({ app_uuid: 'app-anything', limit: 'banana' }),
).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('limit'),
@@ -77,22 +94,14 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
});
it('throws HttpError(400) when offset exceeds the allowed maximum', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: 'app-anything', offset: 1_000_001 }),
res,
),
driver.get_users({ app_uuid: 'app-anything', offset: 1_000_001 }),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws HttpError(404) when the app cannot be found', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: 'app-does-not-exist' }),
res,
),
driver.get_users({ app_uuid: 'app-does-not-exist' }),
).rejects.toMatchObject({
statusCode: 404,
message: 'App not found',
@@ -100,25 +109,10 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
});
it('throws HttpError(403) when the caller does not own the app', async () => {
// Seed an owner user + an app belonging to them, then call the
// handler as a *different* user. The real permission service should
// reject since the caller has no `apps-of-user:<owner>:write` perm.
const owner = await server.stores.user.create({
username: `owner_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `app_${slug}`,
title: `App ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
// Seed an owner + their app, then call as a *different* user. The
// real permission service should reject: the caller has no
// `apps-of-user:<owner>:write`.
const { app } = await seedOwnedApp('owner');
const intruder = await server.stores.user.create({
username: `intruder_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
@@ -126,15 +120,11 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
email: null,
});
const { res } = makeRes();
await callWithActor(
{ user: { uuid: intruder.uuid, id: intruder.id as number } },
async () => {
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: app!.uid }),
res,
),
driver.get_users({ app_uuid: app.uid }),
).rejects.toMatchObject({
statusCode: 403,
message: 'Permission denied',
@@ -144,78 +134,92 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
});
it('returns an empty list for an owned app with no authenticated users', async () => {
const owner = await server.stores.user.create({
username: `owner2_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `app2_${slug}`,
title: `App2 ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
const { owner, app } = await seedOwnedApp('owner2');
const { res, captured } = makeRes();
await callWithActor(
const result = await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
async () => {
await handleAppTelemetryUsers(
makeReq({ app_uuid: app!.uid }),
res,
);
},
() => driver.get_users({ app_uuid: app.uid }),
);
expect(captured.body).toEqual([]);
expect(result).toEqual([]);
});
it('omits user_email for a user who did not grant email:read', async () => {
const { owner, app } = await seedOwnedApp('noemail');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(row.user).toBe(member.username);
expect(row.user_uuid).toBe(member.uuid);
// No grant → the field must be absent (not just null), so the email
// never leaks to the app owner.
expect(Object.prototype.hasOwnProperty.call(row, 'user_email')).toBe(
false,
);
});
it('returns user_email when the user granted email:read to the app', async () => {
const { owner, app } = await seedOwnedApp('withemail');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
await grantEmailRead(
app.id as number,
member.id as number,
member.uuid,
);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(row.user_uuid).toBe(member.uuid);
expect(row.user_email).toBe('[email protected]');
});
it('does not leak email granted to a *different* app', async () => {
const { owner, app } = await seedOwnedApp('appA');
const { app: otherApp } = await seedOwnedApp('appB');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
// Grant email:read against the OTHER app only.
await grantEmailRead(
otherApp.id as number,
member.id as number,
member.uuid,
);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(Object.prototype.hasOwnProperty.call(row, 'user_email')).toBe(
false,
);
});
});
describe('appTelemetry extension — handleAppTelemetryUserCount', () => {
describe('appTelemetry driver — user_count', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUserCount(makeReq({}), res),
).rejects.toMatchObject({ statusCode: 400 });
await expect(driver.user_count({})).rejects.toMatchObject({
statusCode: 400,
});
});
it('throws HttpError(404) for an unknown app_uuid', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUserCount(
makeReq({ app_uuid: 'app-not-here' }),
res,
),
driver.user_count({ app_uuid: 'app-not-here' }),
).rejects.toMatchObject({ statusCode: 404 });
});
it('returns { count: 0 } for an app with no authenticated users', async () => {
const owner = await server.stores.user.create({
username: `counto_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `appcount_${slug}`,
title: `AppCount ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
const { res, captured } = makeRes();
await handleAppTelemetryUserCount(
makeReq({ app_uuid: app!.uid }),
res,
);
expect(captured.body).toEqual({ count: 0 });
it('returns 0 for an app with no authenticated users', async () => {
const { app } = await seedOwnedApp('appcount');
await expect(driver.user_count({ app_uuid: app.uid })).resolves.toBe(0);
});
});
+124 -80
View File
@@ -1,11 +1,23 @@
import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import { PuterDriver } from '@heyputer/backend/src/drivers/types';
import type {
DriverConcurrentConfig,
DriverRateLimitConfig,
} from '@heyputer/backend/src/drivers/meta';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '@heyputer/backend/src/services/metering/consts';
import { extension } from '@heyputer/backend/src/extensions';
const clients = extension.import('client');
const stores = extension.import('store');
const services = extension.import('service');
// App-telemetry lets an app owner enumerate the users who have
// authenticated into their app. v1 shipped this as a driver on the
// `app-telemetry` interface (methods `get_users` / `user_count`) and
// puter-js's `puter.apps(...).getUsers()` still calls it that way
// (`puter.drivers.call('app-telemetry', 'app-telemetry', 'get_users', …)`).
// This is the v2 port of that driver — same interface/method/return shapes
// so existing puter-js callers work unchanged.
const DEFAULT_LIMIT = 100;
const MAX_LIMIT = 1000;
@@ -41,91 +53,123 @@ const parseIntParam = (
return parsed;
};
export const handleAppTelemetryUsers = async (
req: Request,
res: Response,
): Promise<void> => {
const { app_uuid } = req.query as Record<string, string>;
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
/**
* Driver exposing the `app-telemetry` interface.
*
* The `/drivers/call` permission gate checks
* `service:app-telemetry:ii:app-telemetry`, which every actor already holds via
* the blanket `service` grant (hardcoded-permissions.js +
* `default_implicit_user_app_permissions`). The real authorization — "is the
* caller the app owner?" — is enforced inside `get_users` below, exactly as v1
* did.
*/
export class AppTelemetryDriver extends PuterDriver {
readonly driverInterface = 'app-telemetry';
readonly driverName = 'app-telemetry';
readonly isDefault = true;
const safeLimit = parseIntParam(req.query.limit, {
key: 'limit',
min: 1,
max: MAX_LIMIT,
fallback: DEFAULT_LIMIT,
});
const safeOffset = parseIntParam(req.query.offset, {
key: 'offset',
min: 0,
max: MAX_OFFSET,
fallback: 0,
});
// Declaring nothing here would leave both methods on the generic
// 600/minute driver default, which does not fit a paginated scan that
// can ask for MAX_LIMIT rows at MAX_OFFSET. This is a dashboard read —
// nobody calls it in a loop.
readonly rateLimit: DriverRateLimitConfig = {
default: {
limit: 60,
window: 60_000,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 30,
[DEFAULT_TEMP_SUBSCRIPTION]: 10,
},
},
};
const app = await stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
readonly concurrent: DriverConcurrentConfig = {
default: {
limit: 5,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 2,
[DEFAULT_TEMP_SUBSCRIPTION]: 2,
},
},
};
// `apps-of-user:<uuid>:write` — the implicator keys on the owner's
// UUID, not the numeric id. Look up the owner explicitly. v1 got
// this for free because its entity-storage layer eager-joined the
// owner row; v2's AppStore.getByUid returns the raw row with only
// `owner_user_id` populated.
const ownerId = (app as { owner_user_id?: number }).owner_user_id;
if (!ownerId) throw new HttpError(404, 'App owner not found');
const owner = (await stores.user.getById(ownerId)) as {
uuid?: string;
} | null;
if (!owner?.uuid) throw new HttpError(404, 'App owner not found');
/** Users who have authenticated into the given app (owner-only). */
async get_users({
app_uuid,
limit,
offset,
}: {
app_uuid?: string;
limit?: unknown;
offset?: unknown;
} = {}): Promise<
Array<{ user: string; user_uuid: string; user_email?: string | null }>
> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
const actor = Context.get('actor');
const ownsApp = await services.permission
.check(actor!, `apps-of-user:${owner.uuid}:write`)
.catch(() => false);
if (!ownsApp) throw new HttpError(403, 'Permission denied');
const safeLimit = parseIntParam(limit, {
key: 'limit',
min: 1,
max: MAX_LIMIT,
fallback: DEFAULT_LIMIT,
});
const safeOffset = parseIntParam(offset, {
key: 'offset',
min: 0,
max: MAX_OFFSET,
fallback: 0,
});
const users = await clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
INNER JOIN ${clients.db.quoteIdentifier('user')} u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
[(app as Record<string, unknown>).id, safeLimit, safeOffset],
);
const app = await this.stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
res.json(
(users as Array<{ username: string; uuid: string }>).map((e) => ({
user: e.username,
user_uuid: e.uuid,
})),
);
};
// The `apps-of-user:<uuid>:write` implicator keys on the owner's
// UUID, not the numeric id. Look up the owner explicitly — the raw
// app row only carries `owner_user_id`. (v1 got the owner for free
// because its entity-storage layer eager-joined the owner row.)
const ownerId = (app as { owner_user_id?: number }).owner_user_id;
if (!ownerId) throw new HttpError(404, 'App owner not found');
const owner = await this.stores.user.getById(ownerId);
if (!owner?.uuid) throw new HttpError(404, 'App owner not found');
export const handleAppTelemetryUserCount = async (
req: Request,
res: Response,
): Promise<void> => {
const { app_uuid } = req.query as Record<string, string>;
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
// `/drivers/call` requires auth, so there is always an actor.
const actor = Context.get('actor')!;
let ownsApp = false;
try {
ownsApp = await this.services.permission.check(
actor,
`apps-of-user:${owner.uuid}:write`,
);
} catch {
// A failed check is a denial.
}
if (!ownsApp) throw new HttpError(403, 'Permission denied');
const app = await stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
// An email surfaces only for a user who granted *this app*
// `user:<their-uuid>:email:read` — the grant `puter.perms.requestEmail()`
// obtains and `whoami` honours. Authenticating alone doesn't share it.
const users = await this.stores.permission.listAppAuthenticatedUsers(
(app as { id: number }).id,
{ limit: safeLimit, offset: safeOffset },
);
return users.map((u) =>
u.emailShared
? { user: u.username, user_uuid: u.uuid, user_email: u.email }
: { user: u.username, user_uuid: u.uuid },
);
}
const [row] = (await clients.db.read(
`SELECT COUNT(*) AS n FROM user_to_app_permissions
WHERE permission = 'flag:app-is-authenticated' AND app_id = ?`,
[(app as Record<string, unknown>).id],
)) as Array<{ n: number }>;
/** Count of users who have authenticated into the given app. */
async user_count({
app_uuid,
}: { app_uuid?: string } = {}): Promise<number> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
res.json({ count: row?.n ?? 0 });
};
const app = await this.stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
extension.get(
'/app-telemetry/users',
{ subdomain: 'api', requireAuth: true },
handleAppTelemetryUsers,
);
return this.stores.permission.countAppUsers((app as { id: number }).id);
}
}
extension.get(
'/app-telemetry/user-count',
{ subdomain: 'api', requireAuth: true },
handleAppTelemetryUserCount,
);
extension.registerDriver('appTelemetry', AppTelemetryDriver);
+555
View File
@@ -0,0 +1,555 @@
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
} from 'vitest';
import { extensionStore } from '../src/backend/extensions.ts';
import type { IConfig } from '../src/backend/types';
import './devWatcher.ts';
type Lifecycle = {
onServerStart: () => Promise<void>;
onServerShutdown: () => Promise<void>;
};
// The extension registers itself on import; that registry entry is the
// only handle on the service class.
const DevWatcherService = extensionStore.services.devWatcher as unknown as new (
config: IConfig,
clients: unknown,
stores: unknown,
services: unknown,
) => Lifecycle;
const makeService = (config: Record<string, unknown>): Lifecycle =>
new DevWatcherService(config as unknown as IConfig, {}, {}, {});
let workdir: string;
const write = (relative: string, contents: string): string => {
const abs = path.join(workdir, relative);
writeFileSync(abs, contents);
return abs;
};
/** Poll until `predicate` holds or the budget runs out. */
const waitFor = async (
predicate: () => boolean,
label: string,
timeoutMs = 15_000,
): Promise<void> => {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (predicate()) return;
await new Promise((r) => setTimeout(r, 20));
}
throw new Error(`timed out waiting for ${label}`);
};
const logLines = (spy: { mock: { calls: unknown[][] } }): string[] =>
spy.mock.calls.map((call) => call.map(String).join(' '));
describe('devWatcher extension', () => {
let log: ReturnType<typeof vi.spyOn>;
let warn: ReturnType<typeof vi.spyOn>;
let error: ReturnType<typeof vi.spyOn>;
beforeAll(() => {
workdir = mkdtempSync(path.join(tmpdir(), 'devwatch-test-'));
});
afterAll(() => {
rmSync(workdir, { recursive: true, force: true });
delete (extensionStore.services as Record<string, unknown>).devWatcher;
});
beforeEach(() => {
log = vi.spyOn(console, 'log').mockImplementation(() => {});
warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
error = vi.spyOn(console, 'error').mockImplementation(() => {});
});
afterEach(() => {
log.mockRestore();
warn.mockRestore();
error.mockRestore();
});
const started = () =>
logLines(log).some((line) =>
line.startsWith('[devwatch] starting watchers from'),
);
// -- start gating -------------------------------------------------
it('does nothing when the server is not running in dev', async () => {
await makeService({ env: 'production' }).onServerStart();
expect(started()).toBe(false);
});
it('does nothing when devwatch is explicitly disabled', async () => {
await makeService({
env: 'dev',
devwatch: { enabled: false },
}).onServerStart();
expect(started()).toBe(false);
});
it('does nothing when the server opts out with no_devwatch', async () => {
await makeService({
env: 'dev',
no_devwatch: true,
devwatch: { enabled: true },
}).onServerStart();
expect(started()).toBe(false);
});
it('starts in dev even with no explicit devwatch block', async () => {
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
expect(started()).toBe(true);
await service.onServerShutdown();
});
it('starts outside dev when devwatch is explicitly enabled', async () => {
const service = makeService({
env: 'production',
devwatch: {
enabled: true,
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
expect(started()).toBe(true);
await service.onServerShutdown();
});
it('ignores a devwatch config that is not an object', async () => {
// Falls back to `{}`, so the dev-env rule decides — and the
// default webpack entries would be used, so pin it off instead.
await makeService({
env: 'production',
devwatch: 'yes',
}).onServerStart();
expect(started()).toBe(false);
});
it('only starts once even if the lifecycle hook fires again', async () => {
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await service.onServerStart();
expect(
logLines(log).filter((line) =>
line.startsWith('[devwatch] starting watchers from'),
),
).toHaveLength(1);
await service.onServerShutdown();
});
it('waits out the configured ready delay before resolving', async () => {
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [],
ready_delay_ms: 60,
},
});
const start = Date.now();
await service.onServerStart();
expect(Date.now() - start).toBeGreaterThanOrEqual(50);
await service.onServerShutdown();
});
// -- child commands -----------------------------------------------
it('spawns a command, line-buffers its output and flushes the tail', async () => {
write(
'chatty.js',
[
"process.stdout.write('one\\ntwo\\n');",
"process.stderr.write('bad thing\\n');",
// No trailing newline: only the stream `end` flushes it.
"process.stdout.write('tail-no-newline');",
].join('\n'),
);
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [
{
name: 'chatty',
directory: '.',
command: 'node',
args: ['chatty.js'],
},
],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch:chatty:1] tail-no-newline'),
),
'child stdout',
);
const lines = logLines(log);
expect(lines).toContain('[devwatch:chatty:1] one');
expect(lines).toContain('[devwatch:chatty:1] two');
expect(logLines(warn)).toContain('[devwatch:chatty:2] bad thing');
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch:chatty:exit] process exited'),
),
'child exit',
);
await service.onServerShutdown();
});
it('passes literal and computed env values to the child', async () => {
write(
'env.js',
'process.stdout.write(`STATIC=${process.env.STATIC_VALUE} ORIGIN=${process.env.FROM_CONFIG} MISSING=${process.env.BLOWS_UP}\\n`);',
);
const service = makeService({
env: 'dev',
origin: 'http://puter.localhost:4100',
devwatch: {
root: workdir,
commands: [
{
name: 'envtest',
directory: '.',
command: 'node',
args: ['env.js'],
env: {
STATIC_VALUE: 'literal',
FROM_CONFIG: ({
global_config,
}: {
global_config: Record<string, unknown> | null;
}) => String(global_config?.origin ?? ''),
// Reading through a null is the "config not
// loaded yet" shape the extension deliberately
// stays quiet about.
BLOWS_UP: () => {
const nothing = null as unknown as {
x: string;
};
return nothing.x;
},
NOISY: () => {
throw new Error('unexpected failure');
},
},
},
],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await waitFor(
() => logLines(log).some((l) => l.includes('STATIC=literal')),
'env output',
);
const line = logLines(log).find((l) => l.includes('STATIC=literal'))!;
expect(line).toContain('ORIGIN=http://puter.localhost:4100');
expect(line).toContain('MISSING=undefined');
// A null-property read is expected noise and stays silent; any
// other failure is reported.
const warnings = logLines(warn);
expect(
warnings.some((w) => w.includes('could not evaluate env function')),
).toBe(true);
expect(warnings.some((w) => w.includes('for BLOWS_UP'))).toBe(false);
expect(warnings.some((w) => w.includes('for NOISY'))).toBe(true);
await service.onServerShutdown();
});
it('kills a still-running child on shutdown', async () => {
write('forever.js', 'setInterval(() => {}, 1000);');
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [
{
name: 'forever',
directory: '.',
command: 'node',
args: ['forever.js'],
},
],
webpack: [],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await service.onServerShutdown();
expect(logLines(log)).toContain('[devwatch:forever] stopping process');
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch:forever:exit] process exited'),
),
'child killed',
);
});
// -- webpack watchers ---------------------------------------------
const makeWebpackProject = (
name: string,
configFile: string,
contents: string,
packageJson?: string,
): string => {
const dir = path.join(workdir, name);
rmSync(dir, { recursive: true, force: true });
writeFileSync(
path.join(
(() => {
const { mkdirSync } =
require('node:fs') as typeof import('node:fs');
mkdirSync(path.join(dir, 'src'), { recursive: true });
return dir;
})(),
'src/entry.js',
),
"console.log('hello');\n",
);
writeFileSync(path.join(dir, configFile), contents);
if (packageJson) {
writeFileSync(path.join(dir, 'package.json'), packageJson);
}
return name;
};
it('compiles a CommonJS webpack config and reports later rebuilds', async () => {
const dir = makeWebpackProject(
'cjs-project',
'webpack.config.cjs',
`module.exports = {
mode: 'development',
entry: './src/entry.js',
output: { path: __dirname + '/out', filename: 'bundle.js' },
};`,
);
let onConfigSaw: Record<string, unknown> | undefined;
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [
{
name: 'cjs',
directory: dir,
onConfig: (cfg: Record<string, unknown>) => {
onConfigSaw = cfg;
},
},
],
ready_delay_ms: 0,
},
});
await service.onServerStart();
expect(onConfigSaw).toBeDefined();
// The watcher context is anchored at <root>/<directory> so relative
// entries resolve regardless of the server's cwd.
expect(onConfigSaw!.context).toBe(path.join(workdir, dir));
// First build is silent by design; force a rebuild and assert the
// update line shows up.
await new Promise((r) => setTimeout(r, 500));
writeFileSync(
path.join(workdir, dir, 'src/entry.js'),
"console.log('hello again');\n",
);
await waitFor(
() =>
logLines(log).some((l) =>
l.includes('[devwatch] updated cjs using Webpack'),
),
'webpack rebuild',
);
await service.onServerShutdown();
});
it('resolves an ESM webpack config declared through package.json type', async () => {
const dir = makeWebpackProject(
'esm-project',
'webpack.config.js',
`export default {
mode: 'development',
entry: './src/entry.js',
output: { filename: 'bundle.js' },
};`,
JSON.stringify({ name: 'esm-project', type: 'module' }),
);
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [{ directory: dir }],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await service.onServerShutdown();
// Falling back to the directory as the display name is the
// documented behaviour when `name` is omitted.
expect(error).not.toHaveBeenCalled();
});
it('calls a config exported as a function and honours an explicit context', async () => {
const dir = makeWebpackProject(
'fn-project',
'webpack.config.cjs',
`module.exports = () => ({
mode: 'development',
context: 'src',
entry: './entry.js',
name: process.env.WEBPACK_MARKER,
output: { path: __dirname + '/out', filename: 'bundle.js' },
});`,
);
let seen: Record<string, unknown> | undefined;
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [
{
name: 'fn',
directory: dir,
env: { WEBPACK_MARKER: 'set-during-load' },
onConfig: (cfg: Record<string, unknown>) => {
seen = cfg;
},
},
],
ready_delay_ms: 0,
},
});
await service.onServerStart();
// Relative `context` resolves against <root>/<directory>.
expect(seen!.context).toBe(path.join(workdir, dir, 'src'));
// The entry's env map is applied while the config module runs, and
// restored afterwards.
expect(seen!.name).toBe('set-during-load');
expect(process.env.WEBPACK_MARKER).toBeUndefined();
await service.onServerShutdown();
});
it('reports a failing compilation instead of crashing the server', async () => {
const dir = makeWebpackProject(
'broken-project',
'webpack.config.cjs',
`module.exports = {
mode: 'development',
entry: './src/does-not-exist.js',
output: { path: __dirname + '/out', filename: 'bundle.js' },
};`,
);
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [{ name: 'broken', directory: dir }],
ready_delay_ms: 0,
},
});
await service.onServerStart();
await waitFor(
() =>
logLines(error).some((l) =>
l.includes('[devwatch] failed to update broken'),
),
'webpack failure',
);
expect(
logLines(error).some((l) =>
l.includes('[devwatch] error information: broken'),
),
).toBe(true);
await service.onServerShutdown();
});
it('fails loudly when a directory has no webpack config at all', async () => {
const dir = path.join('no-config');
const { mkdirSync } = await import('node:fs');
mkdirSync(path.join(workdir, dir), { recursive: true });
const service = makeService({
env: 'dev',
devwatch: {
root: workdir,
commands: [],
webpack: [{ name: 'missing', directory: dir }],
ready_delay_ms: 0,
},
});
await expect(service.onServerStart()).rejects.toThrow(
'could not find webpack config for: no-config',
);
await service.onServerShutdown();
});
});
+73 -11
View File
@@ -9,6 +9,7 @@ import {
vi,
} from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { extension } from '../src/backend/extensions.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleInstalledApps } from './installedApps.ts';
@@ -38,7 +39,7 @@ beforeAll(async () => {
});
afterAll(async () => {
await server.shutdown();
await server?.shutdown();
});
const seedUser = async () => {
@@ -51,13 +52,14 @@ const seedUser = async () => {
});
};
const seedApp = async (ownerUserId: number) => {
const seedApp = async (ownerUserId: number, overrides = {}) => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.app.create(
{
name: `iaapp_${slug}`,
title: `Installed App ${slug}`,
index_url: `https://example.com/${slug}`,
...overrides,
},
{ ownerUserId },
);
@@ -72,15 +74,6 @@ const grantInstalled = async (appId: number, userId: number) => {
};
describe('installedApps extension — handleInstalledApps', () => {
it('throws HttpError(401) when no actor is on the context', async () => {
const { res } = makeRes();
await expect(
runWithContext({ actor: undefined }, () =>
handleInstalledApps(makeReq({}), res),
),
).rejects.toMatchObject({ statusCode: 401 });
});
it('throws HttpError(400) when orderBy is not in the allowlist', async () => {
const user = await seedUser();
const { res } = makeRes();
@@ -132,9 +125,78 @@ describe('installedApps extension — handleInstalledApps', () => {
expect(list[0].uid).toBe(app!.uid);
expect(list[0].name).toBe(app!.name);
expect(list[0].title).toBe(app!.title);
// index_url is required so the dashboard can derive a hostname title
// for anonymous (app-…) apps.
expect(list[0].index_url).toBe(app!.index_url);
expect(Object.prototype.hasOwnProperty.call(list[0], 'iconUrl')).toBe(
true,
);
// An owned app is not external, and the raw owner id must not leak.
expect(list[0].external).toBe(false);
expect(
Object.prototype.hasOwnProperty.call(list[0], 'owner_user_id'),
).toBe(false);
});
it('pairs iconUrl with the direct subdomain URL for a generated icon', async () => {
const user = await seedUser();
// The resize pipeline leaves an http(s) icon column behind, which is
// what says the sized PNG is on the icons subdomain.
const app = await seedApp(user.id as number, {
icon: 'https://api.puter.com/app-icon/x',
});
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
const { protocol, static_hosting_domain } = extension.config;
expect(list[0].iconCdnUrl).toBe(
`${protocol}://puter-app-icons.${static_hosting_domain}/${app!.uid}-256.png`,
);
});
it('reports no subdomain URL for an app with no generated icon', async () => {
const user = await seedUser();
const app = await seedApp(user.id as number);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list[0].iconCdnUrl).toBeNull();
});
it('flags apps with no owner_user_id as external', async () => {
const user = await seedUser();
const slug = Math.random().toString(36).slice(2, 8);
// createFromOrigin bootstraps an app with owner_user_id = null.
const app = await server.stores.app.createFromOrigin(
`app-${slug}`,
`https://external-${slug}.example.com`,
);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list).toHaveLength(1);
expect(list[0].external).toBe(true);
expect(
Object.prototype.hasOwnProperty.call(list[0], 'owner_user_id'),
).toBe(false);
});
it('clamps page/limit to safe ranges (page>=1, 1<=limit<=100)', async () => {
+45 -47
View File
@@ -2,75 +2,73 @@ import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import { extension } from '@heyputer/backend/src/extensions';
import { getAppIconUrl } from '@heyputer/backend/src/util/appIcon.js';
import {
INSTALLED_APPS_ORDERS,
type InstalledAppsOrder,
} from '@heyputer/backend/src/stores/permission/PermissionStore.js';
import {
getAppIconCdnUrl,
getAppIconUrl,
} from '@heyputer/backend/src/util/appIcon.js';
const clients = extension.import('client');
const ALLOWED_ORDER_BY = [
'id',
'name',
'uid',
'title',
'installed_at',
] as const;
const ORDER_BY_FIELD_MAP: Record<string, string> = {
id: 'apps.id',
name: 'apps.name',
uid: 'apps.uid',
title: 'apps.title',
installed_at: 'installed_at',
};
const stores = extension.import('store');
export const handleInstalledApps = async (
req: Request,
res: Response,
): Promise<void> => {
const actor = Context.get('actor');
if (!actor?.user?.id) throw new HttpError(401, 'Authentication required');
// Behind `requireUserActor`, so there is always an actor.
const actor = Context.get('actor')!;
const orderBy = String(req.query.orderBy ?? 'installed_at');
if (!(ALLOWED_ORDER_BY as readonly string[]).includes(orderBy)) {
const orderBy = String(
req.query.orderBy ?? 'installed_at',
) as InstalledAppsOrder;
if (!INSTALLED_APPS_ORDERS.includes(orderBy)) {
throw new HttpError(
400,
`Invalid orderBy. Allowed: ${ALLOWED_ORDER_BY.join(', ')}`,
`Invalid orderBy. Allowed: ${INSTALLED_APPS_ORDERS.join(', ')}`,
);
}
const page = Math.max(Number(req.query.page) || 1, 1);
const limit = Math.min(Math.max(Number(req.query.limit) || 100, 1), 100);
const offset = (page - 1) * limit;
const orderByField = ORDER_BY_FIELD_MAP[orderBy];
const sortDirection = req.query.desc ? 'DESC' : 'ASC';
const installedApps = (await clients.db.read(
`SELECT
apps.name,
apps.uid,
apps.title,
apps.description,
apps.icon,
MIN(perm.dt) AS installed_at
FROM apps
LEFT JOIN user_to_app_permissions AS perm ON apps.id = perm.app_id
WHERE perm.user_id = ?
GROUP BY apps.id, apps.name, apps.uid, apps.title, apps.description
ORDER BY ${orderByField} ${sortDirection}
LIMIT ?
OFFSET ?`,
[actor.user.id, limit, offset],
)) as Array<Record<string, unknown>>;
const installedApps = await stores.permission.listAppsGrantedByUser(
actor.user.id!,
{ orderBy, descending: Boolean(req.query.desc), limit, offset },
);
const apiBaseUrl = extension.config.api_base_url as string | undefined;
res.json(
installedApps.map((app) => ({
...app,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
})),
installedApps.map((app) => {
// An app with no owner_user_id (null/empty) isn't owned by a Puter
// user — it's an "external" app. Derive a flag and don't leak the
// raw owner id to the client.
const { owner_user_id, ...rest } = app;
const external = owner_user_id == null || owner_user_id === '';
return {
...rest,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
// Direct subdomain URL for the client to try before iconUrl.
iconCdnUrl: getAppIconCdnUrl(app, extension.config),
external,
};
}),
);
};
extension.get(
'/installedApps',
{ subdomain: 'api', requireUserActor: true, allowFullAccessToken: true },
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: {
scope: 'installed-apps',
limit: 120,
window: 60_000,
key: 'user',
},
},
handleInstalledApps,
);
+203 -22
View File
@@ -8,6 +8,7 @@ import {
it,
vi,
} from 'vitest';
import { makeActor, type Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
@@ -48,7 +49,7 @@ beforeAll(async () => {
});
afterAll(async () => {
await server.shutdown();
await server?.shutdown();
});
const seedUser = async () => {
@@ -62,15 +63,6 @@ const seedUser = async () => {
};
describe('metering extension — handleMeteringUsage', () => {
it('throws HttpError(401) when no user actor is on the context', async () => {
const { res } = makeRes();
await expect(
runWithContext({ actor: undefined }, () =>
handleMeteringUsage(makeReq(), res),
),
).rejects.toMatchObject({ statusCode: 401 });
});
it('returns usage details merged with allowanceInfo for an authenticated user', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
@@ -88,21 +80,83 @@ describe('metering extension — handleMeteringUsage', () => {
(captured.body as Record<string, unknown>).allowanceInfo,
).toBeDefined();
});
describe('usage split across apps', () => {
const seedUsage = async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const metering = server.services.metering;
await metering.incrementUsage(
makeActor({ user: owner, app: { uid: 'app-mine' } }),
'kv:read',
1,
10,
);
await metering.incrementUsage(
makeActor({ user: owner, app: { uid: 'app-other' } }),
'kv:write',
1,
20,
);
await metering.incrementUsage(
makeActor({ user: owner }),
'kv:list',
1,
40,
);
await server.stores.meteringBuffer.flushCycle();
return owner;
};
const usageAs = async (actor: Actor) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleMeteringUsage(makeReq(), res),
);
return captured.body as {
usage: Record<string, unknown> & { total: number };
appTotals: Record<string, unknown>;
allowanceInfo: Record<string, unknown>;
};
};
it('reports only the calling app’s own usage to an app actor', async () => {
const owner = await seedUsage();
const body = await usageAs(
makeActor({ user: owner, app: { uid: 'app-mine' } }),
);
expect(body.usage.total).toBe(10);
expect(Object.keys(body.usage).sort()).toEqual([
'kv:read',
'total',
]);
expect(body.appTotals).toEqual({
'app-mine': { total: 10, count: 1 },
});
expect(body.allowanceInfo).toMatchObject({
remaining: expect.any(Number),
monthUsageAllowance: expect.any(Number),
});
});
it('still reports the whole account to the user', async () => {
const owner = await seedUsage();
const body = await usageAs(makeActor({ user: owner }));
expect(body.usage.total).toBe(70);
expect(body.usage).toHaveProperty('kv:read');
expect(body.usage).toHaveProperty('kv:write');
expect(body.usage).toHaveProperty('kv:list');
expect(body.appTotals).toMatchObject({
'app-mine': { total: 10, count: 1 },
'app-other': { total: 20, count: 1 },
});
});
});
});
describe('metering extension — handleMeteringUsageForApp', () => {
it('throws HttpError(401) when no user actor is on the context', async () => {
const { res } = makeRes();
await expect(
runWithContext({ actor: undefined }, () =>
handleMeteringUsageForApp(
makeReq({ params: { appIdOrName: 'any' } }),
res,
),
),
).rejects.toMatchObject({ statusCode: 401 });
});
it('throws HttpError(400) when no appId is supplied', async () => {
const user = await seedUser();
const { res } = makeRes();
@@ -132,6 +186,96 @@ describe('metering extension — handleMeteringUsageForApp', () => {
),
).rejects.toMatchObject({ statusCode: 404 });
});
describe('scoped to the calling app', () => {
const appUsageAs = async (actor: Actor, appIdOrName: string) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleMeteringUsageForApp(
makeReq({ params: { appIdOrName } }),
res,
),
);
return captured.body as { total: number };
};
const seedApps = async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const mine = `app-${uuidv4()}`;
const other = `app-${uuidv4()}`;
await server.services.metering.incrementUsage(
makeActor({ user: owner, app: { uid: mine } }),
'kv:read',
1,
10,
);
await server.services.metering.incrementUsage(
makeActor({ user: owner, app: { uid: other } }),
'kv:write',
1,
20,
);
await server.stores.meteringBuffer.flushCycle();
return {
owner,
mine,
other,
asMine: makeActor({ user: owner, app: { uid: mine } }),
};
};
it('lets an app read its own usage', async () => {
const { mine, asMine } = await seedApps();
expect((await appUsageAs(asMine, mine)).total).toBe(10);
});
it('refuses an app another app’s usage', async () => {
const { other, asMine } = await seedApps();
await expect(appUsageAs(asMine, other)).rejects.toMatchObject({
statusCode: 403,
});
});
it('refuses an app another app’s usage by name', async () => {
const { owner, other, asMine } = await seedApps();
const name = `other-${uuidv4()}`;
await server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[other, name, name, `https://${name}.example/`, owner.id],
);
await expect(appUsageAs(asMine, name)).rejects.toMatchObject({
statusCode: 403,
});
});
it('refuses an app the usage outside any app', async () => {
const { asMine } = await seedApps();
await expect(appUsageAs(asMine, 'os-global')).rejects.toMatchObject(
{ statusCode: 403 },
);
});
it('still lets the user read any of their apps', async () => {
const { owner, mine, other } = await seedApps();
const asUser = makeActor({ user: owner });
expect((await appUsageAs(asUser, mine)).total).toBe(10);
expect((await appUsageAs(asUser, other)).total).toBe(20);
});
it('lets the user read their usage outside any app', async () => {
const { owner } = await seedApps();
const asUser = makeActor({ user: owner });
await server.services.metering.incrementUsage(
asUser,
'kv:read',
1,
5,
);
await server.stores.meteringBuffer.flushCycle();
expect((await appUsageAs(asUser, 'os-global')).total).toBe(5);
});
});
});
describe('metering extension — handleMeteringGlobalUsage', () => {
@@ -165,4 +309,41 @@ describe('metering extension — handleMeteringAllCosts', () => {
// per request).
expect(costsA).toBe(costsB);
});
it('refreshes a warm catalogue at the UTC price boundary', async () => {
vi.useFakeTimers({ toFake: ['Date'] });
try {
vi.setSystemTime(new Date('2026-11-30T23:59:59Z'));
const before = makeRes();
await handleMeteringAllCosts(makeReq(), before.res);
vi.setSystemTime(new Date('2026-12-01T00:00:00Z'));
const after = makeRes();
await handleMeteringAllCosts(makeReq(), after.res);
expect(
(after.captured.body as { costs: unknown[] }).costs,
).not.toBe((before.captured.body as { costs: unknown[] }).costs);
} finally {
vi.useRealTimers();
}
});
// A scheduled price change has to reach the catalogue without a restart.
it('collects the catalogue again once the cache expires', async () => {
vi.useFakeTimers({ toFake: ['Date'] });
try {
vi.setSystemTime(new Date('2026-12-02T00:00:00Z'));
const a = makeRes();
const b = makeRes();
await handleMeteringAllCosts(makeReq(), a.res);
vi.setSystemTime(Date.now() + 10 * 60 * 1000 + 1);
await handleMeteringAllCosts(makeReq(), b.res);
const costsA = (a.captured.body as { costs: unknown[] }).costs;
const costsB = (b.captured.body as { costs: unknown[] }).costs;
expect(costsB).not.toBe(costsA);
expect(costsB).toEqual(costsA);
} finally {
vi.useRealTimers();
}
});
});
+156 -28
View File
@@ -3,22 +3,30 @@ import { HttpError } from '@heyputer/backend/src/core/http';
import {
controllersContainers,
driversContainers,
servicesContainers,
} from '@heyputer/backend/src/exports';
import { extension } from '@heyputer/backend/src/extensions';
import { GLOBAL_APP_KEY } from '@heyputer/backend/src/services/metering/consts';
import {
creditMultiplierFrom,
toCredits,
} from '@heyputer/backend/src/services/metering/utils';
import type { Request, Response } from 'express';
const services = extension.import('service');
const clients = extension.import('client');
const stores = extension.import('store');
// Cached on first request — the underlying cost catalogues are baked into
// driver/controller source so they only change on deploy.
// Catalogues change on deploy or on a scheduled price change, so the cache
// expires rather than living as long as the process.
const ALL_COSTS_TTL_MS = 10 * 60 * 1000;
let cachedAllCosts: Record<string, unknown>[] | null = null;
let allCostsExpireAt = 0;
function collectAllCosts(): Record<string, unknown>[] {
const all: Record<string, unknown>[] = [];
const collect = (
source: Record<string, unknown>,
kind: 'driver' | 'controller',
kind: 'driver' | 'controller' | 'service',
) => {
for (const [name, instance] of Object.entries(source)) {
const fn = (
@@ -43,44 +51,115 @@ function collectAllCosts(): Record<string, unknown>[] {
};
collect(driversContainers as Record<string, unknown>, 'driver');
collect(controllersContainers as Record<string, unknown>, 'controller');
// Services report the costs that aren't tied to one endpoint — egress,
// which is metered for every response there is.
collect(servicesContainers as Record<string, unknown>, 'service');
return all;
}
// -- Credits scaling ---------------------------------------------------
//
// When the deployment configures a credit multiplier, every monetary amount
// is scaled to display credits BEFORE it leaves the server — raw metered
// amounts are never reported. Counts, units, and byte figures pass through
// untouched. A response that was scaled says so (`unit: 'credits'`); without
// a configured multiplier amounts pass through raw and clients render them
// as dollars.
/** The per-usage-type records of a UsageByType map, costs scaled. */
const scaleUsageByType = (
usage: Record<string, unknown>,
multiplier: number,
): Record<string, unknown> => {
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(usage)) {
// `allowanceUsed` is the month's allowance-charged spend — monetary,
// scalar, scaled like the total.
if (key === 'total' || key === 'allowanceUsed') {
out[key] = toCredits(Number(value) || 0, multiplier);
} else if (
value &&
typeof value === 'object' &&
typeof (value as { cost?: unknown }).cost === 'number'
) {
out[key] = {
...(value as Record<string, unknown>),
cost: toCredits((value as { cost: number }).cost, multiplier),
};
} else {
out[key] = value;
}
}
return out;
};
export const handleMeteringUsage = async (
_req: Request,
res: Response,
): Promise<void> => {
const actor = Context.get('actor');
if (!actor?.user) throw new HttpError(401, 'Authentication required');
// Behind `requireAuth`, so there is always an actor.
const actor = Context.get('actor')!;
const [actorUsage, allowanceInfo] = await Promise.all([
services.metering.getActorCurrentMonthUsageDetails(actor),
services.metering.getAllowedUsage(actor),
]);
res.json({ ...actorUsage, allowanceInfo });
const multiplier = creditMultiplierFrom(extension.config);
if (!multiplier) {
res.json({ ...actorUsage, allowanceInfo });
return;
}
res.json({
usage: scaleUsageByType(
actorUsage.usage as unknown as Record<string, unknown>,
multiplier,
),
appTotals: Object.fromEntries(
Object.entries(actorUsage.appTotals).map(([appId, totals]) => [
appId,
{ ...totals, total: toCredits(totals.total, multiplier) },
]),
),
allowanceInfo: {
...allowanceInfo,
remaining: toCredits(allowanceInfo.remaining, multiplier),
monthUsageAllowance: toCredits(
allowanceInfo.monthUsageAllowance,
multiplier,
),
// Monetary addon fields scale; absent ones stay absent rather
// than materializing as NaN/null. purchasedStorage is bytes.
addons: Object.fromEntries(
Object.entries(allowanceInfo.addons ?? {}).map(([k, v]) => [
k,
(k === 'purchasedCredits' ||
k === 'consumedPurchaseCredits') &&
typeof v === 'number'
? toCredits(v, multiplier)
: v,
]),
),
unit: 'credits',
},
});
};
export const handleMeteringUsageForApp = async (
req: Request,
res: Response,
): Promise<void> => {
const actor = Context.get('actor');
if (!actor?.user) throw new HttpError(401, 'Authentication required');
// Behind `requireAuth`, so there is always an actor.
const actor = Context.get('actor')!;
let appId = String(req.params.appIdOrName ?? '');
if (!appId) throw new HttpError(400, 'appId parameter is required');
// If not a UUID-shaped app UID, look up by name
if (!appId.startsWith('app-')) {
const appRows = (await clients.db.read(
'SELECT `uid` FROM `apps` WHERE `name` = ? LIMIT 1',
[appId],
)) as Array<{ uid: string }>;
if (appRows.length > 0) {
appId = appRows[0].uid;
} else {
throw new HttpError(404, 'App not found');
}
// If not a UUID-shaped app UID or the global sentinel, look up by name.
// Which apps an actor may read is MeteringService's call, not this route's.
if (!appId.startsWith('app-') && appId !== GLOBAL_APP_KEY) {
const app = await stores.app.getByName(appId);
if (!app) throw new HttpError(404, 'App not found');
appId = app.uid;
}
const appUsage =
@@ -88,7 +167,18 @@ export const handleMeteringUsageForApp = async (
actor,
appId,
);
res.json(appUsage);
const multiplier = creditMultiplierFrom(extension.config);
if (!multiplier) {
res.json(appUsage);
return;
}
res.json({
...scaleUsageByType(
appUsage as unknown as Record<string, unknown>,
multiplier,
),
unit: 'credits',
});
};
export const handleMeteringGlobalUsage = async (
@@ -99,37 +189,75 @@ export const handleMeteringGlobalUsage = async (
res.json(globalUsage);
};
// First hit walks the registries; subsequent hits serve the in-memory cache.
// A hit walks the registries when the in-memory cache is missing or expired.
export const handleMeteringAllCosts = async (
_req: Request,
res: Response,
): Promise<void> => {
if (!cachedAllCosts) {
cachedAllCosts = collectAllCosts();
if (!cachedAllCosts || Date.now() >= allCostsExpireAt) {
allCostsExpireAt =
(Math.floor(Date.now() / ALL_COSTS_TTL_MS) + 1) * ALL_COSTS_TTL_MS;
const raw = collectAllCosts();
const multiplier = creditMultiplierFrom(extension.config);
// Price the catalogue in the same unit the usage endpoints report:
// per-unit credits when a multiplier is configured, raw otherwise.
cachedAllCosts = multiplier
? raw.map(({ ucentsPerUnit, costs_currency: _cc, ...rest }) =>
typeof ucentsPerUnit === 'number'
? {
...rest,
creditsPerUnit: toCredits(
ucentsPerUnit,
multiplier,
),
}
: rest,
)
: raw;
}
res.json({ costs: cachedAllCosts });
};
/** Dashboard reads over the per-actor KV aggregates. */
const USAGE_READ_LIMIT = {
scope: 'metering-usage',
limit: 120,
window: 60_000,
key: 'user' as const,
};
extension.get(
'/metering/usage',
{ subdomain: 'api', requireAuth: true },
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringUsage,
);
extension.get(
'/metering/usage/:appIdOrName',
{ subdomain: 'api', requireAuth: true },
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringUsageForApp,
);
extension.get(
'/metering/globalUsage',
{ subdomain: 'api', adminOnly: true },
{
subdomain: 'api',
adminOnly: true,
// Sums across every shard of the global aggregate. Admin-gated, so
// this is loop protection — but one accidental poll is an
// expensive minute.
rateLimit: {
scope: 'metering-global-usage',
limit: 10,
window: 60_000,
key: 'user',
},
},
handleMeteringGlobalUsage,
);
extension.get(
'/metering/allCosts',
{ subdomain: 'api', requireAuth: true },
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringAllCosts,
);
+15 -3
View File
@@ -71,9 +71,7 @@ describe('serverInfo extension — handleServerInfo', () => {
const statfsSpy = vi
.spyOn(fs.default, 'statfs')
.mockRejectedValue(new Error('statfs unsupported'));
const errSpy = vi
.spyOn(console, 'error')
.mockImplementation(() => {});
const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
const { res, captured } = makeRes();
await handleServerInfo({} as Request, res);
@@ -88,4 +86,18 @@ describe('serverInfo extension — handleServerInfo', () => {
statfsSpy.mockRestore();
errSpy.mockRestore();
});
it('reports an unknown CPU model when os.cpus() comes back empty', async () => {
const os = await import('node:os');
const cpusSpy = vi.spyOn(os.default, 'cpus').mockReturnValue([]);
const { res, captured } = makeRes();
await handleServerInfo({} as Request, res);
const cpu = captured.body!.cpu as Record<string, unknown>;
expect(cpu.model).toBe('Unknown');
expect(cpu.cores).toBe(0);
cpusSpy.mockRestore();
});
});
File diff suppressed because it is too large. Load diff
+314 -68
View File
@@ -1,18 +1,95 @@
import {
DeleteObjectCommand,
GetObjectCommand,
HeadObjectCommand,
PutObjectCommand,
S3Client,
} from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { extension } from '@heyputer/backend/src/extensions';
import type { FSEntryStore } from '@heyputer/backend/src/stores/fs/FSEntryStore';
import { isMissingObjectError } from '@heyputer/backend/src/stores/fs/S3ObjectStore';
import crypto from 'node:crypto';
import type { Readable } from 'node:stream';
import sharp from 'sharp';
const clients = extension.import('client');
const stores = extension.import('store');
type ThumbnailEntryStore = Pick<FSEntryStore, 'updateEntryThumbnailByUuid'>;
const MAX_THUMBNAIL_BYTES = 2 * 1024 * 1024;
const MAX_THUMBNAIL_PIXELS = 64e6;
// Namespace every object this extension writes. An fs object's key is its
// bare fsentry uuid and the default config points `thumbnailStore.name` at
// the same bucket as `s3_bucket`, so without a prefix of our own there is no
// way to tell a thumbnail we minted from any other object in the deployment.
const THUMBNAIL_KEY_PREFIX = 'thumbnails/';
const UUID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
const isUuid = (value: unknown): value is string =>
typeof value === 'string' && UUID_PATTERN.test(value);
// `thumbnails/<entry uuid>/<random uuid>`: the entry segment binds the object
// to the entry it was minted for, which is what writes and deletes check.
const mintThumbnailKey = (entryUuid: string): string =>
`${THUMBNAIL_KEY_PREFIX}${entryUuid}/${crypto.randomUUID()}`;
/**
* Extract the object key from a stored thumbnail pointer, or null when the
* pointer isn't one this extension minted.
*
* `fsentries.thumbnail` is writable through the FS API, so neither half of the
* stored string is trusted: the bucket is discarded (callers always pass their
* own) and the key must sit under {@link THUMBNAIL_KEY_PREFIX} as either
* `<uuid>` (minted before keys were bound to entries) or `<entry uuid>/<uuid>`.
* Honouring an arbitrary key would lend this extension's storage credentials to
* whatever object the caller named — in the shared-bucket layout that is every
* user's file, since an fs object's key is its fsentry uuid. Bare-uuid keys
* from before the prefix fail the check and are treated as absent; they are
* indistinguishable from a planted pointer, so there is nothing safer to do
* with them than stop signing them.
*/
const resolveThumbnailKey = (pointer: string): string | null => {
let key: string;
if (pointer.startsWith('s3://')) {
const rest = pointer.slice('s3://'.length);
const slash = rest.indexOf('/');
if (slash === -1) return null;
key = rest.slice(slash + 1);
} else {
let pathname: string;
try {
pathname = new URL(pointer).pathname;
} catch {
return null;
}
const segments = pathname.replace(/^\/+/, '').split('/');
segments.shift(); // bucket
key = segments.join('/');
}
if (!key.startsWith(THUMBNAIL_KEY_PREFIX)) return null;
const segments = key.slice(THUMBNAIL_KEY_PREFIX.length).split('/');
return segments.length <= 2 && segments.every(isUuid) ? key : null;
};
// The entry a resolved key was minted for. Null for an unbound key, whose
// holder can't be told apart from a row that copied or planted the pointer.
const thumbnailKeyOwner = (key: string): string | null => {
const segments = key.slice(THUMBNAIL_KEY_PREFIX.length).split('/');
return segments.length === 2 ? segments[0] : null;
};
// Whether a stored thumbnail names an object in our storage. Anything else is
// an external image URL the client supplied: shown as given, never copied or
// deleted.
const isStoragePointer = (pointer: string, bucketEndpoint: string): boolean =>
pointer.startsWith('s3://') ||
// Legacy format — remove after full migration
(pointer.startsWith('https') &&
pointer.includes(new URL(bucketEndpoint).hostname));
// S3 client + bucket config — lazily resolved after boot from config.
let s3Client: S3Client | null = null;
let s3PresignClient: S3Client | null = null;
@@ -62,13 +139,6 @@ function getPresignClient(): S3Client {
return resolveClients().presign;
}
function base64ParseDataUrl(dataURL: string) {
dataURL = dataURL.slice(5);
const mimeType = dataURL.split(';')[0];
const data = Buffer.from(dataURL.split(',')[1], 'base64');
return { mimeType, data };
}
// Strictly decode a data: URL and validate the decoded image. Encoded-string
// length lies about decoded byte count (whitespace, padding) and says nothing
// about pixel count — a 2MB PNG can decompress to hundreds of MB of raster.
@@ -95,16 +165,73 @@ async function decodeAndValidateThumbnail(
return { mimeType, data };
}
// A storage pointer a client hands back after a presigned upload is kept only
// if it names a key minted for this entry and an object within the size bound.
async function vetUploadedThumbnail(
pointer: string,
entryUuid: unknown,
deps: { s3: S3Client; bucketName: string },
): Promise<string | null> {
const key = resolveThumbnailKey(pointer);
if (!key || thumbnailKeyOwner(key) !== entryUuid) return null;
let size: number | undefined;
try {
const head = await deps.s3.send(
new HeadObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
size = head.ContentLength;
} catch (err) {
if (!isMissingObjectError(err)) {
console.warn(
'[thumbnails] failed to check uploaded thumbnail',
err,
);
}
return null;
}
if (typeof size !== 'number') return null;
if (size > MAX_THUMBNAIL_BYTES) {
// Only reachable on a store that doesn't enforce the signed length.
// The key is this entry's own, so removing it is safe.
try {
await deps.s3.send(
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
} catch (err) {
console.warn(
'[thumbnails] failed to remove oversized thumbnail',
err,
);
}
return null;
}
return `s3://${deps.bucketName}/${key}`;
}
// -- thumbnail.created -----------------------------------------------
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer.
// and replace the URL with an s3:// pointer bound to the entry.
export async function handleThumbnailCreated(
event: Record<string, unknown>,
deps: { s3: S3Client; bucketName: string },
deps: { s3: S3Client; bucketName: string; bucketEndpoint: string },
): Promise<void> {
const url = event.url;
if (typeof url !== 'string' || !url.startsWith('data:')) return;
if (typeof url !== 'string') return;
if (!url.startsWith('data:')) {
if (isStoragePointer(url, deps.bucketEndpoint)) {
event.url = await vetUploadedThumbnail(url, event.uuid, deps);
}
return;
}
const entryUuid = event.uuid;
if (!isUuid(entryUuid)) {
event.url = null;
return;
}
const decoded = await decodeAndValidateThumbnail(url);
if (!decoded) {
@@ -112,7 +239,7 @@ export async function handleThumbnailCreated(
return;
}
const key = crypto.randomUUID();
const key = mintThumbnailKey(entryUuid);
event.url = `s3://${deps.bucketName}/${key}`;
await deps.s3.send(
@@ -141,21 +268,24 @@ export const handleThumbnailUploadPrepare = async (
typeof item.contentType === 'string' ? item.contentType.trim() : '';
if (!contentType) continue;
if (item.size !== undefined) {
const size = Number(item.size);
if (
!Number.isFinite(size) ||
size < 0 ||
size > MAX_THUMBNAIL_BYTES
)
continue;
}
const size = item.size;
if (
typeof size !== 'number' ||
!Number.isInteger(size) ||
size < 0 ||
size > MAX_THUMBNAIL_BYTES
)
continue;
if (!isUuid(item.item_uid)) continue;
const key = crypto.randomUUID();
const key = mintThumbnailKey(item.item_uid);
const command = new PutObjectCommand({
Bucket: deps.bucketName,
Key: key,
ContentType: contentType,
// Signing covers `content-length`, so a body of any other size
// fails the signature instead of landing.
ContentLength: size,
});
item.uploadUrl = await getSignedUrl(presignClient, command, {
expiresIn: 900,
@@ -171,76 +301,172 @@ export const handleThumbnailRead = async (
s3Presign: S3Client;
bucketName: string;
bucketEndpoint: string;
db: { write: (sql: string, params: unknown[]) => Promise<unknown> };
fsEntry: ThumbnailEntryStore;
},
): Promise<void> => {
const thumb = entry.thumbnail;
if (typeof thumb !== 'string' || !thumb) return;
const presignClient = deps.s3Presign;
const signingDate = new Date(
Math.floor(Date.now() / 3_600_000) * 3_600_000,
);
if (thumb.startsWith('s3://')) {
const [bucket, key] = thumb.slice(5).split('/');
if (isStoragePointer(thumb, deps.bucketEndpoint)) {
const key = resolveThumbnailKey(thumb);
if (!key) {
// Not a pointer we minted — refuse to sign it rather than hand
// out a presigned read of whatever object it names.
entry.thumbnail = null;
return;
}
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: bucket, Key: key }),
{ expiresIn: 604800 },
);
} else if (
thumb.startsWith('https') &&
thumb.includes(new URL(deps.bucketEndpoint).hostname)
) {
// Legacy format — remove after full migration
const [bucket, key] = new URL(thumb).pathname.slice(1).split('/');
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: bucket, Key: key }),
{ expiresIn: 604800 },
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
{ expiresIn: 604800, signingDate },
);
} else if (thumb.startsWith('data')) {
// Inline data-URL migration: upload to S3 and update the DB entry.
const key = crypto.randomUUID();
const { mimeType, data } = base64ParseDataUrl(thumb);
const newUrl = `s3://${deps.bucketName}/${key}`;
// Move an inline thumbnail into storage. The row only takes the new
// pointer while it still holds this data URL, so of concurrent reads
// one wins and the rest discard their upload.
const uuid = entry.uuid ?? entry.uid;
if (!isUuid(uuid)) return;
// Same bar as an upload; anything else stays inline, unmigrated.
const decoded = await decodeAndValidateThumbnail(thumb);
if (!decoded) return;
const key = mintThumbnailKey(uuid);
await deps.s3.send(
new PutObjectCommand({
Bucket: deps.bucketName,
Key: key,
Body: data,
ContentType: mimeType,
Body: decoded.data,
ContentType: decoded.mimeType,
}),
);
// Best-effort async DB update
const uuid = entry.uuid ?? entry.uid;
if (uuid) {
deps.db
.write(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[newUrl, uuid],
)
.catch((err: unknown) =>
console.warn('[thumbnails] inline migration failed', err),
let migrated: boolean;
try {
migrated = await deps.fsEntry.updateEntryThumbnailByUuid(
uuid,
`s3://${deps.bucketName}/${key}`,
thumb,
);
} catch (err) {
// The row may or may not point at the object now, so keep it.
console.warn('[thumbnails] inline migration failed', err);
return;
}
if (!migrated) {
try {
await deps.s3.send(
new DeleteObjectCommand({
Bucket: deps.bucketName,
Key: key,
}),
);
} catch (err) {
console.warn(
'[thumbnails] failed to remove unused thumbnail',
err,
);
}
return;
}
entry.thumbnail = await getSignedUrl(
presignClient,
new GetObjectCommand({ Bucket: deps.bucketName, Key: key }),
{ expiresIn: 604800 },
{ expiresIn: 604800, signingDate },
);
}
};
export const handleFsRemoveNodeThumbnail = async (
payload: { target: Record<string, unknown> },
deps: { s3: S3Client },
export const handleFsCopyNodeThumbnail = async (
payload: { copy?: { thumbnail?: string | null; uuid?: string } | null },
deps: {
s3: S3Client;
bucketName: string;
bucketEndpoint: string;
fsEntry: ThumbnailEntryStore;
},
): Promise<void> => {
const thumbnailUrl = payload.target.thumbnail as string | undefined;
if (!thumbnailUrl || !thumbnailUrl.startsWith('s3://')) return;
const copy = payload.copy;
const thumbnailUrl = copy?.thumbnail;
if (!copy || !isUuid(copy.uuid) || typeof thumbnailUrl !== 'string') return;
if (!isStoragePointer(thumbnailUrl, deps.bucketEndpoint)) return;
const [bucket, key] = thumbnailUrl.slice(5).split('/');
await deps.s3.send(new DeleteObjectCommand({ Bucket: bucket, Key: key }));
// Same trust rule as the read and remove paths: only touch objects this
// extension minted.
const sourceKey = resolveThumbnailKey(thumbnailUrl);
if (!sourceKey) return;
// The copied row points at the SAME object as its source. Give the copy
// an object of its own, bound to it, so removing either entry leaves the
// other's thumbnail alone.
const newKey = mintThumbnailKey(copy.uuid);
try {
// Nothing larger than an upload may produce gets duplicated; such an
// object came in through an unbounded upload URL.
const head = await deps.s3.send(
new HeadObjectCommand({ Bucket: deps.bucketName, Key: sourceKey }),
);
if (
typeof head.ContentLength !== 'number' ||
head.ContentLength > MAX_THUMBNAIL_BYTES
) {
throw new Error('thumbnail exceeds the size bound');
}
// Read and rewrite rather than CopyObject: a copy source skips any path
// in the client's endpoint, so it can miss an object Bucket/Key reach.
const source = await deps.s3.send(
new GetObjectCommand({ Bucket: deps.bucketName, Key: sourceKey }),
);
// The object can be replaced after the HEAD; bound what gets buffered.
if ((source.ContentLength ?? 0) > MAX_THUMBNAIL_BYTES) {
(source.Body as Readable | undefined)?.destroy();
throw new Error('thumbnail exceeds the size bound');
}
const body = await source.Body?.transformToByteArray();
if (!body) throw new Error('thumbnail has no body');
await deps.s3.send(
new PutObjectCommand({
Bucket: deps.bucketName,
Key: newKey,
Body: body,
ContentType: source.ContentType,
}),
);
} catch (err) {
// The shared object is gone or oversized — drop the pointer rather
// than leave the row advertising a thumbnail it doesn't have.
await deps.fsEntry.updateEntryThumbnailByUuid(copy.uuid, null);
console.warn('[thumbnails] failed to duplicate thumbnail on copy', err);
return;
}
await deps.fsEntry.updateEntryThumbnailByUuid(
copy.uuid,
`s3://${deps.bucketName}/${newKey}`,
);
};
export const handleFsRemoveNodeThumbnail = async (
payload: { target: { thumbnail?: string | null; uuid?: string } },
deps: { s3: S3Client; bucketName: string; bucketEndpoint: string },
): Promise<void> => {
const { thumbnail: thumbnailUrl, uuid } = payload.target;
if (!thumbnailUrl || !isStoragePointer(thumbnailUrl, deps.bucketEndpoint))
return;
// The pointer decides which object gets deleted, and a key is visible to
// anyone who can list the entry holding it. Only a key minted for this
// entry is deleted; an unbound one may be shared or planted, so it stays.
const key = resolveThumbnailKey(thumbnailUrl);
if (!key || thumbnailKeyOwner(key) !== uuid) return;
await deps.s3.send(
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
};
extension.on(
@@ -249,6 +475,7 @@ extension.on(
await handleThumbnailCreated(event, {
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
});
},
);
@@ -275,16 +502,35 @@ extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
s3Presign: getPresignClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
db: clients.db,
fsEntry: stores.fsEntry,
});
});
// -- fs.copy.node ----------------------------------------------------
// A copied entry initially shares its source's thumbnail object; duplicate
// it so removing either entry can't break the other's thumbnail.
extension.on('fs.copy.node', async (_key, payload) => {
await handleFsCopyNodeThumbnail(
payload as {
copy?: { thumbnail?: string | null; uuid?: string } | null;
},
{
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
fsEntry: stores.fsEntry,
},
);
});
// -- fs.remove.node --------------------------------------------------
// Delete S3 thumbnail when the file is removed.
extension.on(
'fs.remove.node',
async (_key, payload: { target: Record<string, unknown> }) => {
await handleFsRemoveNodeThumbnail(payload, { s3: getClient() });
},
);
extension.on('fs.remove.node', async (_key, payload) => {
await handleFsRemoveNodeThumbnail(payload, {
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
});
});
+355
View File
@@ -0,0 +1,355 @@
import { EventEmitter } from 'node:events';
import { Readable } from 'node:stream';
import type { Request, Response } from 'express';
import { beforeEach, describe, expect, test, vi } from 'vitest';
const SECRET = 'ingress-secret';
// Only `extension` is stubbed; the secret comparison and the routing are the
// real code paths, and the fake service/store layer is the boundary.
const state = vi.hoisted(() => ({
config: { userEmail: { secret: 'ingress-secret' } } as Record<
string,
unknown
>,
routes: [] as Array<{
path: string;
handler: (req: unknown, res: unknown) => unknown;
}>,
users: new Map<string, Record<string, unknown>>(),
writes: [] as Array<{
userId: number;
path: string;
homeRegion?: string;
content?: unknown;
}>,
feedback: [] as Array<{ to: string; from: string | null; raw: Buffer }>,
takeFeedback: true,
}));
vi.mock('@heyputer/backend/src/extensions', () => ({
extension: {
get config() {
return state.config;
},
post(
path: string,
_opts: unknown,
handler: (req: unknown, res: unknown) => unknown,
) {
state.routes.push({ path, handler });
},
import(layer: string) {
if (layer === 'client') {
return {
event: {
async emitAndWait(
key: string,
data: {
to: string;
from: string | null;
raw: Buffer;
handled: boolean;
},
) {
if (key !== 'email.ingress.feedback') return;
state.feedback.push(data);
if (state.takeFeedback) data.handled = true;
},
},
};
}
if (layer === 'store') {
return {
user: {
getByUsername: async (username: string) =>
state.users.get(username) ?? null,
},
};
}
return {
fs: {
async write(
userId: number,
{
fileMetadata,
fileContent,
}: {
fileMetadata: { path: string };
fileContent: unknown;
},
_uploadTracker?: unknown,
_storageAllowanceMax?: number,
homeRegion?: string,
) {
state.writes.push({
userId,
path: fileMetadata.path,
homeRegion,
content: fileContent,
});
return { fsEntry: { path: fileMetadata.path } };
},
},
};
},
},
}));
import './index.js';
const permanent = (id: number, username: string) => ({
id,
username,
email: `${username}@example.com`,
password: '$2b$10$hash',
});
/** No password and no email is what makes an account temporary. */
const temp = (id: number, username: string) => ({
id,
username,
email: null,
password: null,
});
const ingress = async (
to: string,
over: { secret?: string; contentLength?: string | null } = {},
) => {
const route = state.routes.find((r) => r.path === '/email/ingress');
if (!route) throw new Error('route not registered');
const body = Buffer.from('From: [email protected]\r\n\r\nhello\r\n');
const req = Readable.from([body], { objectMode: false }) as unknown as {
headers: Record<string, string>;
query: Record<string, string>;
};
// `contentLength: null` sends no length header at all, which is the one
// framing the HTTP parser does not bound for us.
req.headers =
over.contentLength === null
? {}
: {
'content-length': over.contentLength ?? String(body.length),
};
req.query = {
SECRET: over.secret ?? SECRET,
to,
from: '[email protected]',
subject: 'hi',
};
let status = 200;
let ended = false;
// `end` takes (chunk, encoding) and no callback, the way the compression
// middleware leaves it, and emits `finish` — which is what the refusal
// paths hang their drain/destroy on. A stub that accepts a callback
// instead hides the bug that took the 413 path to production as a 500.
const emitter = new EventEmitter();
const res = {
status(code: number) {
status = code;
return res;
},
end(chunk?: unknown, encoding?: BufferEncoding) {
if (chunk) Buffer.byteLength(chunk as string, encoding);
ended = true;
emitter.emit('finish');
},
once(event: string, listener: () => void) {
emitter.once(event, listener);
return res;
},
} as unknown as Response;
await route.handler(req as unknown as Request, res);
return {
status,
ended,
drained: (req as unknown as Readable).readableFlowing,
destroyed: (req as unknown as Readable).destroyed,
};
};
beforeEach(() => {
state.users.clear();
state.writes.length = 0;
state.feedback.length = 0;
state.takeFeedback = true;
});
describe('temporary accounts cannot receive', () => {
test('mail for a temp account is accepted and dropped', async () => {
state.users.set('tempy', temp(1, 'tempy'));
const { status, ended, drained } = await ingress('[email protected]');
// Accepted, so the relay neither retries nor bounces to the sender.
expect(status).toBe(200);
expect(ended).toBe(true);
// The body is read off the wire rather than left unconsumed.
expect(drained).toBe(true);
expect(state.writes).toEqual([]);
});
test('mail for a permanent account still lands in the mailbox', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status } = await ingress('[email protected]');
expect(status).toBe(200);
expect(state.writes).toHaveLength(1);
expect(state.writes[0].userId).toBe(2);
expect(state.writes[0].path).toMatch(/^\/bob\/\.mail\/objects\//);
});
test('an unknown address is still rejected outright', async () => {
const { status } = await ingress('[email protected]');
expect(status).toBe(404);
expect(state.writes).toEqual([]);
});
});
describe('inbound mail is placed in the recipient home region', () => {
/** A permanent account with explicit placement columns. */
const placed = (
id: number,
username: string,
over: { home?: string | null; signup_server?: string | null },
) => ({
...permanent(id, username),
home: null,
signup_server: null,
...over,
});
test("uses the recipient's own home region when they have one", async () => {
state.users.set('bob', placed(2, 'bob', { home: 'frankfurt' }));
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('frankfurt');
});
test('falls back to the server that served their signup', async () => {
// Every account that predates the `home` column is this case.
state.users.set(
'bob',
placed(2, 'bob', { home: null, signup_server: 'london' }),
);
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('london');
});
test('falls back to the primary region when neither is set', async () => {
state.users.set('bob', placed(2, 'bob', {}));
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('oregon');
});
test('home wins over signup_server when both are set', async () => {
state.users.set(
'bob',
placed(2, 'bob', { home: 'sydney', signup_server: 'london' }),
);
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('sydney');
});
});
describe('refusals dispose of the body nobody read', () => {
test('an unknown recipient drains it and keeps the connection', async () => {
// The secret checked out and the length is under the cap, so the body
// was ours to read: discard it rather than resetting a usable socket.
const { status, drained, destroyed } =
await ingress('[email protected]');
expect(status).toBe(404);
expect(drained).toBe(true);
expect(destroyed).toBe(false);
});
test('a bad secret closes rather than reading the upload', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
secret: 'not-the-secret',
});
expect(status).toBe(403);
// Nothing authenticated this caller; their bytes are not worth reading.
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
test('no content-length is refused with 411', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
contentLength: null,
});
expect(status).toBe(411);
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
test('a declared length over the cap is refused with 413', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
contentLength: String(25 * 1024 * 1024 + 1),
});
expect(status).toBe(413);
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
});
describe('feedback addresses', () => {
test('are handed to a listener and reach no mailbox', async () => {
const { status, ended } = await ingress('[email protected]');
expect(status).toBe(200);
expect(ended).toBe(true);
expect(state.feedback).toHaveLength(1);
expect(state.feedback[0].to).toBe('[email protected]');
expect(state.feedback[0].from).toBe('[email protected]');
expect(state.feedback[0].raw.toString()).toContain('hello');
expect(state.writes).toEqual([]);
});
test('match the local part case-insensitively', async () => {
await ingress('[email protected]');
expect(state.feedback).toHaveLength(1);
});
test('fall through to normal routing when nobody takes them', async () => {
state.takeFeedback = false;
state.users.set('abuse', permanent(3, 'abuse'));
const { status } = await ingress('[email protected]');
expect(status).toBe(200);
expect(state.feedback).toHaveLength(1);
// The body was already read, so the buffered copy is what is stored.
expect(state.writes).toHaveLength(1);
expect(Buffer.isBuffer(state.writes[0].content)).toBe(true);
});
test('an unclaimed one with no such account is still a 404', async () => {
state.takeFeedback = false;
const { status } = await ingress('[email protected]');
expect(status).toBe(404);
});
test('only count on a Puter mail domain', async () => {
state.users.set('abuse', permanent(3, 'abuse'));
await ingress('[email protected]');
expect(state.feedback).toEqual([]);
});
test('are refused over the feedback size cap without being read', async () => {
const { status, destroyed } = await ingress('[email protected]', {
contentLength: String(5 * 1024 * 1024 + 1),
});
expect(status).toBe(413);
expect(destroyed).toBe(true);
expect(state.feedback).toEqual([]);
});
});
+161
View File
@@ -0,0 +1,161 @@
import { extension } from '@heyputer/backend/src/extensions';
import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
import type { Request } from 'express';
import {
MAX_MESSAGE_BYTES,
isPuterEmailAddress,
isTempUser,
puterEmailUsername,
storeInboxMessage,
} from '@heyputer/backend/src/services/email/mailbox.js';
// copied form peer secrets checker
const COMPARE_KEY = randomBytes(32);
const secretsEqual = (a: string, b: string): boolean =>
timingSafeEqual(
createHmac('sha256', COMPARE_KEY).update(a).digest(),
createHmac('sha256', COMPARE_KEY).update(b).digest(),
);
const USER_EMAIL_CONFIG = (extension.config as Record<string, unknown>)
.userEmail as { secret?: string; feedbackAddresses?: unknown } | undefined;
const INGRESS_SECRET = USER_EMAIL_CONFIG?.secret;
/**
* Local parts that take complaint feedback rather than naming a mailbox. The
* same names are reserved at signup, so no account can be handed them.
*/
const DEFAULT_FEEDBACK_ADDRESSES = ['fbl', 'abuse', 'postmaster'];
const FEEDBACK_ADDRESSES = new Set(
(Array.isArray(USER_EMAIL_CONFIG?.feedbackAddresses)
? USER_EMAIL_CONFIG.feedbackAddresses
: DEFAULT_FEEDBACK_ADDRESSES
).map((local) => String(local).toLowerCase()),
);
/**
* Feedback is handed over whole, so it is read into memory; a complaint report
* quotes one message and has no business near the mailbox ceiling.
*/
const MAX_FEEDBACK_BYTES = 5 * 1024 * 1024;
const isFeedbackAddress = (to: string): boolean =>
isPuterEmailAddress(to) &&
FEEDBACK_ADDRESSES.has(puterEmailUsername(to).toLowerCase());
const readBody = async (req: Request): Promise<Buffer> => {
const chunks: Buffer[] = [];
for await (const chunk of req) chunks.push(chunk as Buffer);
return Buffer.concat(chunks);
};
if (!INGRESS_SECRET) {
console.warn(
'[useremailrx] config.userEmail.secret unset - email ingress disabled',
);
} else {
extension.post(
'/email/ingress',
{
subdomain: 'api',
},
async (req, res) => {
// Answer first, then deal with the body nobody read. Touching the
// request before the response has flushed takes the response down
// with the connection and the caller sees a reset with no status,
// so this hangs off `finish` — and it has to be the event rather
// than `res.end(cb)`, because the compression middleware replaces
// `res.end` with an `(chunk, encoding)` signature that has no
// callback parameter, so a callback lands in `chunk` and dies in
// `Buffer.byteLength`.
//
// `drain` discards the rest of a body we were entitled to read and
// leaves the connection reusable. `close` is for bodies we do not
// want to read at all: an unauthenticated caller's, or one whose
// declared length is missing or over the cap.
const refuse = (status: number, how: 'drain' | 'close') => {
res.once('finish', () =>
how === 'drain' ? req.resume() : req.destroy(),
);
res.status(status).end();
};
const presented = req.query.SECRET;
if (
typeof presented !== 'string' ||
!secretsEqual(presented, INGRESS_SECRET)
) {
// Nothing has authenticated this caller, so refuse to spend
// bandwidth reading whatever they were uploading.
refuse(403, 'close');
return;
}
const size = Number(req.headers['content-length']);
if (!Number.isInteger(size) || size <= 0) {
refuse(411, 'close');
return;
}
if (size > MAX_MESSAGE_BYTES) {
refuse(413, 'close');
return;
}
const stores = extension.import('store');
const services = extension.import('service');
const to = String(req.query.to ?? '');
let content: Buffer | Request = req;
if (isFeedbackAddress(to)) {
if (size > MAX_FEEDBACK_BYTES) {
refuse(413, 'close');
return;
}
const raw = await readBody(req);
const event = {
to,
from:
typeof req.query.from === 'string'
? req.query.from
: null,
raw,
handled: false,
};
await extension
.import('client')
.event.emitAndWait('email.ingress.feedback', event, {});
if (event.handled) {
res.end();
return;
}
// Nobody took it: route it like any other message.
content = raw;
}
const user = await stores.user.getByUsername(
puterEmailUsername(to),
);
if (!user) {
refuse(404, 'drain');
return;
}
if (isTempUser(user)) {
if (content === req) req.resume();
res.end();
return;
}
await storeInboxMessage(services.fs, user, {
subject: req.query.subject,
content,
size: Buffer.isBuffer(content) ? content.length : size,
});
const { SECRET: _secret, ..._loggableQuery } = req.query;
// console.log('got email: ', puterUser, loggableQuery, size);
res.end();
},
);
}
+12
View File
@@ -0,0 +1,12 @@
{
"name": "emailrx",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "emailrx",
"version": "1.0.0"
}
}
}
+6
View File
@@ -0,0 +1,6 @@
{
"name": "emailrx",
"version": "1.0.0",
"main": "index.ts",
"description": "Internal service which handles routing puter emails to users"
}
+425 -12
View File
@@ -1,14 +1,10 @@
import type { Request, Response } from 'express';
import { installSeatStub } from '@heyputer/backend/testFixtures/seatStub';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { makeActor, type Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { configContainer } from '../src/backend/exports.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { handleWhoami } from './whoami.ts';
@@ -51,7 +47,7 @@ beforeAll(async () => {
});
afterAll(async () => {
await server.shutdown();
await server?.shutdown();
});
const seedUser = async () => {
@@ -65,6 +61,78 @@ const seedUser = async () => {
};
describe('whoami extension — handleWhoami', () => {
// The sidebar label needs this at boot, which is why it rides whoami
// rather than a call of its own.
describe('the team an account belongs to', () => {
// A seat is created, never adopted, so it must have no password.
const seedSeat = async () => {
const slug = Math.random().toString(36).slice(2, 8);
return server.stores.user.create({
username: `wseat_${slug}`,
uuid: uuidv4(),
password: null,
email: null,
});
};
const seatOf = async (teamName: string) => {
const seat = await seedSeat();
const team = {
uid: `wt-${Math.random().toString(36).slice(2, 9)}`,
name: teamName,
};
installSeatStub(server as never, [seat.id as number], {
teamUid: team.uid,
teamName: team.name,
});
return { seat, team };
};
it('names no team where teams is off', async () => {
const { seat } = await seatOf('Acme Corp');
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: seat.uuid, id: seat.id as number } } },
() => handleWhoami(makeReq(), res),
);
// Off, so the seat is never looked up.
expect((captured.body as { team?: unknown }).team).toBeUndefined();
});
it('says nothing for an account that is not a seat', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
expect(captured.body).not.toHaveProperty('team');
});
it('withholds it from an app actor', async () => {
// Same class as the phone number: a seat's employer is not an
// app's business.
const { seat } = await seatOf('Acme Corp');
const { res, captured } = makeRes();
await runWithContext(
{
actor: makeActor({
user: { uuid: seat.uuid, id: seat.id as number },
app: { uid: 'app-1' },
}),
},
() => handleWhoami(makeReq(), res),
);
expect(captured.body).not.toHaveProperty('team');
});
});
it('returns 401 when no actor is on the context', async () => {
const { res, captured } = makeRes();
@@ -92,6 +160,81 @@ describe('whoami extension — handleWhoami', () => {
expect(captured.body).toEqual({ error: 'User not found' });
});
describe("an app's view of the email", () => {
const appActorFor = async (user: { uuid: string; id: unknown }) => {
const app = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `we-${uuidv4()}`,
title: 'Email reader',
index_url: `https://${uuidv4()}.example/`,
},
{ ownerUserId: user.id as number },
);
return {
app,
actor: makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: app.uid, id: app.id },
}),
};
};
const whoamiAs = async (actor: Actor) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleWhoami(makeReq(), res),
);
return captured;
};
it('is withheld until the user grants it, then shown', async () => {
const user = await seedUser();
const { app, actor } = await appActorFor(user);
const before = (await whoamiAs(actor)).body as Record<
string,
unknown
>;
expect(before).not.toHaveProperty('email');
expect(before).not.toHaveProperty('unconfirmed_email');
await server.services.permission.grantUserAppPermission(
makeActor({ user: { uuid: user.uuid, id: user.id as number } }),
app.uid,
`user:${user.uuid}:email:read`,
);
const after = (await whoamiAs(actor)).body as Record<
string,
unknown
>;
expect(after.email).toBe(user.email);
expect(after.unconfirmed_email).toBe(user.email);
});
it('answers 404 for a missing account even when the check fails', async () => {
const check = vi
.spyOn(server.services.permission, 'check')
.mockRejectedValue(new Error('scan failed'));
try {
const captured = await whoamiAs(
makeActor({
user: { uuid: 'ghost-uuid', id: 99_999_998 },
app: { uid: 'app-ghost', id: 1 },
}),
);
expect(captured.statusCode).toBe(404);
expect(captured.body).toEqual({ error: 'User not found' });
} finally {
check.mockRestore();
}
});
});
it('returns full user details for a user actor', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
@@ -107,12 +250,61 @@ describe('whoami extension — handleWhoami', () => {
expect(body.email).toBe(user.email);
expect(body.is_temp).toBe(false);
expect(body.oidc_only).toBe(false);
// is_user_token is present (true) for user actors.
expect(body.is_user_token).toBe(true);
// Account creation time, in unix seconds.
expect(typeof body.created_ts).toBe('number');
expect(body.created_ts).toBeGreaterThan(0);
// `directories` is only sent to user actors — confirm it’s present.
expect(body.directories).toBeDefined();
// taskbar_items is only sent to user actors.
expect(body).toHaveProperty('taskbar_items');
});
it('reports the SMS-to-card fallback only once a send has opened it', async () => {
const user = await server.stores.user.create({
username: `wuser_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'hashedpw',
email: `${Math.random().toString(36).slice(2, 8)}@example.com`,
requires_phone_verification: true,
} as never);
const prev = configContainer.phone_verification_card_fallback;
configContainer.phone_verification_card_fallback = {
enabled: true,
} as never;
try {
const before = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), before.res),
);
// Phone-gated, but the user has attempts left — no offer yet.
expect(
before.captured.body as Record<string, unknown>,
).toMatchObject({ card_fallback_available: false });
// Exhausting SMS attempts stamps this flag; whoami is then the only
// thing that can still tell a reloading GUI about the offer, since
// further sends are rejected by the route's own rate limit.
await server.stores.kv.set({
key: `card-fallback-open:${user.id}`,
value: true,
});
const after = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), after.res),
);
expect(
after.captured.body as Record<string, unknown>,
).toMatchObject({ card_fallback_available: true });
} finally {
configContainer.phone_verification_card_fallback = prev;
}
});
it('only forwards allow-listed feature flags', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
@@ -130,30 +322,251 @@ describe('whoami extension — handleWhoami', () => {
expect(flags.payment_bypass).toBeUndefined();
});
it('strips desktop_bg_* and human_readable_age fields for app actors', async () => {
it('strips desktop_bg_*, created_ts and human_readable_age fields for app actors', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
await runWithContext(
{
actor: {
actor: makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
},
}),
},
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.app_name).toBe('app-test-actor');
// is_user_token is stripped for app actors.
expect(body.is_user_token).toBeUndefined();
expect(body.desktop_bg_url).toBeUndefined();
expect(body.desktop_bg_color).toBeUndefined();
expect(body.desktop_bg_fit).toBeUndefined();
expect(body.human_readable_age).toBeUndefined();
// Account age, in either form, is not exposed to apps.
expect(body.created_ts).toBeUndefined();
// Directories are user-only.
expect(body.directories).toBeUndefined();
});
describe('fields added by whoami.details listeners', () => {
const addAccountDetails = (
_key: unknown,
event: { details: Record<string, unknown> },
) => {
Object.assign(event.details, {
subscribed: true,
paid_storage: 1024,
subscription: { tier: 'pro', status: 'trialing' },
accountEligible: true,
});
};
const whoamiWithListener = async (
actor: Actor,
listener: typeof addAccountDetails = addAccountDetails,
) => {
const { res, captured } = makeRes();
server.clients.event.on('whoami.details', listener);
try {
await runWithContext({ actor }, () =>
handleWhoami(makeReq(), res),
);
} finally {
server.clients.event.off('whoami.details', listener);
}
return captured.body as Record<string, unknown>;
};
it('reach an app actor only as the documented subscribed / paid_storage', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
);
expect(body.subscribed).toBe(true);
expect(body.paid_storage).toBe(1024);
expect(body).not.toHaveProperty('subscription');
expect(body).not.toHaveProperty('accountEligible');
// Fields the handler built itself still ship.
expect(body.username).toBe(user.username);
expect(body.app_name).toBe('app-test-actor');
});
it('are withheld from an app reached through an access token it issued', async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const body = await whoamiWithListener(
makeActor({
user: owner,
accessToken: {
uid: 'tok-1',
issuer: makeActor({
user: owner,
app: { uid: 'app-issuer' },
}),
},
}),
);
expect(body.subscribed).toBe(true);
expect(body).not.toHaveProperty('subscription');
expect(body).not.toHaveProperty('accountEligible');
});
it('all reach a user actor', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({ user: { uuid: user.uuid, id: user.id as number } }),
);
expect(body).toMatchObject({
subscribed: true,
paid_storage: 1024,
subscription: { tier: 'pro', status: 'trialing' },
accountEligible: true,
});
});
it('cannot add taskbar_items for an app actor, which core omits', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
(_key, event) => {
event.details.taskbar_items = [{ name: 'injected' }];
},
);
expect(body).not.toHaveProperty('taskbar_items');
});
});
it('redacts tmp_password from metadata for user actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
hasDevAccountAccess: true,
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
const metadata = body.metadata as Record<string, unknown>;
expect(metadata.tmp_password).toBeUndefined();
// Other metadata keys still reach the user's own client.
expect(metadata.hasDevAccountAccess).toBe(true);
expect(body.hasDevAccountAccess).toBe(true);
});
it('never sends user metadata to app actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
});
const { res, captured } = makeRes();
await runWithContext(
{
actor: makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
},
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.metadata).toBeUndefined();
expect(JSON.stringify(body)).not.toContain('bootstrap-secret');
});
it('never exposes phone, card fingerprint or signup identity', async () => {
const user = await seedUser();
await server.stores.user.update(user.id as number, {
phone: '+15551234567',
card_fingerprint: 'fp_ABC123',
requires_phone_verification: false,
requires_card_verification: false,
});
// Guard against a vacuous assertion below: the columns really do hold
// the values we then expect never to see on the wire.
const stored = await server.stores.user.getById(user.id as number, {
cached: false,
force: true,
});
expect(stored?.phone).toBe('+15551234567');
expect(stored?.card_fingerprint).toBe('fp_ABC123');
for (const actor of [
makeActor({ user: { uuid: user.uuid, id: user.id as number } }),
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
]) {
const { res, captured } = makeRes();
await runWithContext({ actor }, () => handleWhoami(makeReq(), res));
const body = captured.body as Record<string, unknown>;
expect(body.phone).toBeUndefined();
expect(body.card_fingerprint).toBeUndefined();
expect(body.password).toBeUndefined();
expect(body.otp_secret).toBeUndefined();
expect(body.signup_ip).toBeUndefined();
// Not just absent as a key — the values must not appear anywhere
// in the payload (nested under metadata, taskbar items, …).
const serialized = JSON.stringify(body);
expect(serialized).not.toContain('5551234567');
expect(serialized).not.toContain('fp_ABC123');
// The verification flags, which the GUI acts on, still ship.
expect(body).toHaveProperty('requires_phone_verification');
expect(body).toHaveProperty('requires_card_verification');
}
});
it('scrubs sensitive keys added to metadata, and leaves the cached row intact', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
billing: { card_fingerprint: 'fp_NESTED', tier: 'pro' },
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
const metadata = body.metadata as Record<string, unknown>;
expect(metadata.tmp_password).toBeUndefined();
// Nested sensitive keys are removed too; siblings survive.
expect(metadata.billing).toEqual({ tier: 'pro' });
expect(JSON.stringify(body)).not.toContain('fp_NESTED');
// The scrub works on a copy — server-side state is untouched.
const fresh = await server.stores.user.getById(user.id as number, {
cached: false,
force: true,
});
expect(fresh?.metadata?.tmp_password).toBe('bootstrap-secret');
expect(
(fresh?.metadata?.billing as Record<string, unknown>)
?.card_fingerprint,
).toBe('fp_NESTED');
});
it('marks the user as oidc_only when password is null', async () => {
const slug = Math.random().toString(36).slice(2, 8);
const oidcUser = await server.stores.user.create({
+78 -114
View File
@@ -1,6 +1,10 @@
import { Context } from '@heyputer/backend/src/core';
import { extension } from '@heyputer/backend/src/extensions';
import { getTaskbarItems } from '@heyputer/backend/src/util/taskbarItems.js';
import { APP_ICON_SIZES } from '@heyputer/backend/src/util/appIcon.js';
import {
buildUserDetails,
scrubSensitive,
} from '@heyputer/backend/src/util/userDetails.js';
import type { Request, Response } from 'express';
import TimeAgo from 'javascript-time-ago';
import localeEn from 'javascript-time-ago/locale/en';
@@ -14,14 +18,11 @@ const timeago = (() => {
return new TimeAgo('en-US');
})();
// Allowlist of `config.feature_flags` keys safe to surface via /whoami.
// Anything not listed here stays server-side, so internal flags
// (payment_bypass, staff_only_*, etc.) cannot leak by accident. Add a
// flag here when, and only when, the client actually needs to read it.
const CLIENT_VISIBLE_FEATURE_FLAGS: ReadonlySet<string> = new Set([
'create_shortcut',
'download_directory',
'prompt_user_when_navigation_away_from_puter',
// What an app actor may receive from `whoami.details` listeners. Anything
// else a listener adds describes the account, and only ships to user actors.
const APP_VISIBLE_LISTENER_KEYS: ReadonlySet<string> = new Set([
'subscribed',
'paid_storage',
]);
export const handleWhoami = async (
@@ -34,125 +35,49 @@ export const handleWhoami = async (
return;
}
const isUser = !actor.app;
const isUser = !actor.effectiveApp;
const canReadEmail = async (uuid: unknown) => {
try {
return await services.permission.check(
actor,
`user:${uuid}:email:read`,
);
} catch {
return false;
}
};
// Started beside the row read: the actor carries the row's uuid.
const emailCheck = isUser ? null : canReadEmail(actor.user.uuid);
const user = await stores.user.getById(actor.user.id);
if (!user) {
res.status(404).json({ error: 'User not found' });
return;
}
const oidcOnly = user.password === null;
const ALLOWED_ICON_SIZES = new Set([16, 32, 64, 128, 256, 512]);
const rawIconSize =
typeof req.query?.icon_size === 'string'
? Number(req.query.icon_size)
: undefined;
const iconSize =
rawIconSize !== undefined && ALLOWED_ICON_SIZES.has(rawIconSize)
rawIconSize !== undefined && APP_ICON_SIZES.includes(rawIconSize)
? rawIconSize
: undefined;
const noIcons = !iconSize;
// Feature flags come from `config.feature_flags`. We only forward keys
// listed in CLIENT_VISIBLE_FEATURE_FLAGS so internal flags can't leak.
// Non-boolean values (e.g. `"true"` as a string) are coerced so the
// client never has to guess.
const rawFlags = extension.config.feature_flags ?? {};
const feature_flags: Record<string, boolean> = {};
for (const [k, v] of Object.entries(rawFlags)) {
if (CLIENT_VISIBLE_FEATURE_FLAGS.has(k)) {
feature_flags[k] = Boolean(v);
}
}
const details: Record<string, unknown> = {
username: user.username,
uuid: user.uuid,
email: user.email,
unconfirmed_email: user.email,
email_confirmed: user.email_confirmed || user.username === 'admin',
requires_email_confirmation: user.requires_email_confirmation,
phone: user.phone,
requires_phone_verification: user.requires_phone_verification,
requires_card_verification: user.requires_card_verification,
desktop_bg_url: user.desktop_bg_url,
desktop_bg_color: user.desktop_bg_color,
desktop_bg_fit: user.desktop_bg_fit,
is_temp: user.password === null && user.email === null,
oidc_only: oidcOnly,
taskbar_items: isUser
? await getTaskbarItems(
user,
{
clients,
stores,
services,
apiBaseUrl: String(extension.config.api_base_url ?? ''),
},
{ iconSize, noIcons },
)
: undefined,
otp: !!user.otp_enabled,
feature_flags,
human_readable_age: user.timestamp
? timeago.format(new Date(user.timestamp as string))
: null,
metadata: user.metadata,
hasDevAccountAccess: !!user.metadata?.hasDevAccountAccess,
};
// OIDC revalidate URL for password-less accounts
if (oidcOnly) {
try {
const provider = await services.oidc.getLinkedProviderForUser(
user.id as number,
);
if (provider) {
const origin = (extension.config.origin ?? '').replace(
/\/$/,
'',
);
details.oidc_revalidate_url = `${origin}/auth/oidc/${provider}/start?flow=revalidate&user_uuid=${encodeURIComponent(user.uuid)}`;
}
} catch {
// OIDC not configured
}
}
// Directories — only sent to user actors
if (isUser) {
const directories: Record<string, unknown> = {};
const nameToProp: Record<string, string> = {
desktop_uuid: `/${user.username}/Desktop`,
appdata_uuid: `/${user.username}/AppData`,
documents_uuid: `/${user.username}/Documents`,
pictures_uuid: `/${user.username}/Pictures`,
videos_uuid: `/${user.username}/Videos`,
trash_uuid: `/${user.username}/Trash`,
};
for (const k in nameToProp) {
directories[nameToProp[k]] = user[k];
}
details.directories = directories;
}
// Last activity
if (user.last_activity_ts) {
try {
details.last_activity_ts = Math.round(
new Date(user.last_activity_ts as string).getTime() / 1000,
);
} catch {
/* ignore parse error */
}
}
const details = await buildUserDetails(
user,
{ config: extension.config, clients, stores, services },
{ isUser, iconSize, noIcons: !iconSize },
);
details.human_readable_age = user.timestamp
? timeago.format(new Date(user.timestamp as string))
: null;
// Strip sensitive fields for app actors
if (!isUser) {
const canReadEmail = await services.permission
.check(actor, `user:${user.uuid}:email:read`)
.catch(() => false);
if (!canReadEmail) {
const emailReadable = await (user.uuid === actor.user.uuid
? emailCheck
: canReadEmail(user.uuid));
if (!emailReadable) {
delete details.email;
delete details.unconfirmed_email;
}
@@ -160,12 +85,24 @@ export const handleWhoami = async (
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.created_ts;
delete details.is_user_token;
delete details.metadata;
// An app has no business reading the code its user earns credit with.
delete details.referral_code;
}
if (actor.app) {
details.app_name = actor.app.uid;
const app = actor.effectiveApp;
if (app) {
details.app_name = app.uid;
}
// A key core left undefined isn't built, so a listener can't fill it for an app.
const builtKeys = isUser
? null
: new Set(
Object.keys(details).filter((key) => details[key] !== undefined),
);
try {
await clients.event.emitAndWait(
'whoami.details',
@@ -175,6 +112,13 @@ export const handleWhoami = async (
} catch {
/* best-effort */
}
if (builtKeys) {
for (const key of Object.keys(details)) {
if (!builtKeys.has(key) && !APP_VISIBLE_LISTENER_KEYS.has(key)) {
delete details[key];
}
}
}
const subscription = details.subscription as
| { offering?: Record<string, unknown> }
@@ -185,11 +129,31 @@ export const handleWhoami = async (
delete subscription.offering.price_id;
}
// Last word on what ships, after every listener has had its say.
scrubSensitive(details);
res.json(details);
};
extension.get(
'/whoami',
{ subdomain: 'api', requireAuth: true, allowUnconfirmed: true },
{
subdomain: 'api',
requireAuth: true,
allowUnconfirmed: true,
// The GUI polls this, and each call fans out to every `whoami`
// event listener — so it costs more than the response suggests.
//
// It is also the call everything else leans on to find out who it is
// talking to, so it rides along with unrelated work rather than
// arriving at its own pace: the ceiling has to clear whatever the
// busiest session is doing, not what a person clicks.
rateLimit: {
scope: 'whoami',
limit: 1_800,
window: 60_000,
key: 'user',
},
},
handleWhoami,
);
+15 -12
View File
@@ -11,7 +11,7 @@
# What this does, in order:
# 1. Checks that docker (with the compose plugin) exists.
# 2. Creates ./puter-selfhosted/ (override with $env:PUTER_DIR).
# 3. Downloads docker-compose.yml + nginx.conf from the OSS repo.
# 3. Downloads docker-compose.yml + caddy/Caddyfile from the OSS repo.
# 4. Generates fresh secrets and writes .env + puter/config/config.json.
# 5. Runs `docker compose up -d` and prints how to find the admin password.
#
@@ -23,7 +23,7 @@
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_PORT HTTP port for Caddy (default: 80)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
[CmdletBinding()]
@@ -77,10 +77,10 @@ if ($LASTEXITCODE -ne 0) {
# ── Step 2: install dir ─────────────────────────────────────────────
$null = New-Item -ItemType Directory -Force -Path $PuterDir
Set-Location $PuterDir
$null = New-Item -ItemType Directory -Force -Path 'puter/config', 'puter/data', 'puter/tls'
$null = New-Item -ItemType Directory -Force -Path 'puter/config', 'puter/data', 'puter/data/caddy', 'puter/tls'
Write-Log "install dir: $((Get-Location).Path)"
# ── Step 3: docker-compose.yml + nginx config ──────────────────────
# ── Step 3: docker-compose.yml + Caddy config ──────────────────────
Write-Log "downloading docker-compose.yml from $PuterUrl"
try {
Invoke-WebRequest -Uri "$PuterUrl/docker-compose.yml" -OutFile 'docker-compose.yml' -UseBasicParsing
@@ -88,17 +88,19 @@ try {
Die "could not fetch $PuterUrl/docker-compose.yml — $_"
}
Write-Log "downloading nginx/nginx.conf from $PuterUrl"
$null = New-Item -ItemType Directory -Force -Path 'nginx'
# The Caddyfile is domain-agnostic — it answers on every Host and leaves
# the subdomain routing to Puter — so there's nothing to template in.
Write-Log "downloading caddy/Caddyfile from $PuterUrl"
$null = New-Item -ItemType Directory -Force -Path 'caddy'
# If the path was previously auto-created as a directory by a failed
# `compose up`, remove it so we can write the file there.
if (Test-Path 'nginx/nginx.conf' -PathType Container) {
Remove-Item 'nginx/nginx.conf' -Recurse -Force
if (Test-Path 'caddy/Caddyfile' -PathType Container) {
Remove-Item 'caddy/Caddyfile' -Recurse -Force
}
try {
Invoke-WebRequest -Uri "$PuterUrl/nginx/nginx.conf" -OutFile 'nginx/nginx.conf' -UseBasicParsing
Invoke-WebRequest -Uri "$PuterUrl/caddy/Caddyfile" -OutFile 'caddy/Caddyfile' -UseBasicParsing
} catch {
Die "could not fetch $PuterUrl/nginx/nginx.conf — $_"
Die "could not fetch $PuterUrl/caddy/Caddyfile — $_"
}
# ── Step 4: secrets, .env, config.json ──────────────────────────────
@@ -121,7 +123,8 @@ if ($writeConfig) {
$envContent = @"
HTTP_PORT=$PuterPort
# HTTPS_PORT=443 # uncomment after enabling TLS (see doc/selfhosting/full-stack.md)
# HTTPS_PORT=443 # uncomment after enabling TLS in caddy/Caddyfile
# # (see "Step 3 — TLS" in doc/self-hosting.md)
MARIADB_ROOT_PASSWORD=$mariadbRootPw
MARIADB_DATABASE=puter
@@ -205,4 +208,4 @@ Write-Log " cd $PuterDir; docker compose logs -f puter"
Write-Log ''
Write-Log "open http://${PuterDomain}:${PuterPort} once the puter container is healthy."
Write-Log 'first-boot admin password is logged once — grab it with:'
Write-Log " cd $PuterDir; docker compose logs puter | Select-String password"
Write-Log " cd $PuterDir; docker compose logs puter | Select-String password"
+66 -19
View File
@@ -7,7 +7,8 @@
# What this does, in order:
# 1. Checks that docker (with the compose plugin), curl, and openssl exist.
# 2. Creates ./puter-selfhosted/ (override with PUTER_DIR=...).
# 3. Downloads docker-compose.yml from the OSS repo (raw.githubusercontent.com).
# 3. Downloads docker-compose.yml + caddy/Caddyfile from the OSS repo
# (raw.githubusercontent.com).
# 4. Generates fresh secrets and writes .env + puter/config/config.json.
# 5. Runs `docker compose up -d` and prints the first-boot admin password.
#
@@ -19,15 +20,17 @@
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_PORT HTTP port for Caddy (default: 80)
# PUTER_PROTOCOL public scheme: http | https (default: http)
# Set https when a TLS-terminating reverse proxy (Caddy,
# Traefik, nginx, a cloud LB) sits in front — Puter then
# builds https:// origins, S3 URLs, and redirects. See
# "Running behind your own reverse proxy" in doc/self-hosting.md.
# Set https once TLS is terminating in front of Puter —
# either the bundled Caddy with your certs (see "Step 3 —
# TLS" in doc/self-hosting.md) or your own proxy (Traefik,
# a cloud LB). Puter then builds https:// origins, S3
# URLs, and redirects. Leave it http and the installer
# serves plain HTTP on PUTER_PORT.
# PUTER_TRUST_PROXY number of reverse-proxy hops in front (default: 1)
# 1 = the bundled nginx (or a single external proxy);
# 2 = two hops (e.g. Cloudflare → your proxy → Puter).
# 1 = the bundled Caddy (or a single external proxy);
# 2 = two hops (e.g. Cloudflare → Caddy → Puter).
# PUTER_ENV prod | dev (default: prod)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
@@ -50,9 +53,29 @@ need() {
command -v "$1" >/dev/null 2>&1 || die "missing required command: $1"
}
check_docker() {
if command -v docker >/dev/null 2>&1; then
return 0
fi
echo >&2 ""
echo >&2 "[puter-install] missing required command: docker"
echo >&2 ""
echo >&2 "Docker is required to run Puter self-hosted."
echo >&2 ""
echo >&2 "Installation instructions:"
echo >&2 " - macOS (Apple Silicon & Intel): https://docs.docker.com/desktop/setup/install/mac-install/"
echo >&2 " (Docker Desktop is recommended. Alternatives: OrbStack, Colima)"
echo >&2 " - Linux: https://docs.docker.com/engine/install/"
echo >&2 " - Windows: https://docs.docker.com/desktop/setup/install/windows-install/"
echo >&2 ""
echo >&2 "After installing Docker, make sure it's running and try again."
echo >&2 ""
exit 1
}
# ── Step 1: dependency check ────────────────────────────────────────
log "checking dependencies"
need docker
check_docker
need curl
need openssl
docker compose version >/dev/null 2>&1 \
@@ -72,25 +95,27 @@ mkdir -p puter/config puter/data puter/tls
# bind-mount roots sidesteps the mismatch without guessing each image's
# internal UID. (Docker Desktop on macOS/Windows papers over this with
# its VM layer; native Linux docker on Debian/Alpine doesn't.)
mkdir -p puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter
chmod 0777 puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter
mkdir -p puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter puter/data/caddy
chmod 0777 puter/data/valkey puter/data/mariadb puter/data/dynamo puter/data/s3 puter/data/puter puter/data/caddy
log "install dir: $(pwd)"
# ── Step 3: docker-compose.yml + nginx config ──────────────────────
# ── Step 3: docker-compose.yml + Caddy config ──────────────────────
log "downloading docker-compose.yml from $PUTER_URL"
curl -fsSL "$PUTER_URL/docker-compose.yml" -o docker-compose.yml \
|| die "could not fetch $PUTER_URL/docker-compose.yml"
# nginx is mounted as `./nginx/nginx.conf:/etc/nginx/nginx.conf:ro` — if
# Caddy is mounted as `./caddy/Caddyfile:/etc/caddy/Caddyfile:ro,z` — if
# the host file is missing, docker silently creates a directory at that
# path and the mount fails with "not a directory" at container start.
log "downloading nginx/nginx.conf from $PUTER_URL"
mkdir -p nginx
# The Caddyfile is domain-agnostic — it answers on every Host and leaves
# the subdomain routing to Puter — so there's nothing to template in.
log "downloading caddy/Caddyfile from $PUTER_URL"
mkdir -p caddy
# If the path was previously auto-created as a dir by a failed `compose up`,
# remove it so curl can write the file.
[ -d nginx/nginx.conf ] && rmdir nginx/nginx.conf 2>/dev/null || true
curl -fsSL "$PUTER_URL/nginx/nginx.conf" -o nginx/nginx.conf \
|| die "could not fetch $PUTER_URL/nginx/nginx.conf"
[ -d caddy/Caddyfile ] && rmdir caddy/Caddyfile 2>/dev/null || true
curl -fsSL "$PUTER_URL/caddy/Caddyfile" -o caddy/Caddyfile \
|| die "could not fetch $PUTER_URL/caddy/Caddyfile"
# ── Step 4: secrets, .env, config.json ──────────────────────────────
write_config=1
@@ -113,7 +138,8 @@ if [ "$write_config" = "1" ]; then
cat > .env <<EOF
HTTP_PORT=$PUTER_PORT
# HTTPS_PORT=443 # uncomment after enabling TLS (see doc/selfhosting/full-stack.md)
# HTTPS_PORT=443 # uncomment after enabling TLS in caddy/Caddyfile
# # (see "Step 3 — TLS" in doc/self-hosting.md)
MARIADB_ROOT_PASSWORD=$MARIADB_ROOT_PASSWORD
MARIADB_DATABASE=puter
@@ -125,6 +151,11 @@ S3_SECRET_KEY=$S3_SECRET_KEY
S3_BUCKET=puter-local
EOF
# `meteringEnforcement.subscriptions` is off here because a self-hosted
# install has no paid plans to be on: every account resolves to the free
# policy, so leaving the plan gates armed would 402 the surfaces that
# declare one (the OpenAI/Anthropic-compatible AI endpoints) for everybody.
# The code default is on — this is a property of the install, not of Puter.
log "writing puter/config/config.json"
cat > puter/config/config.json <<EOF
{
@@ -185,11 +216,27 @@ EOF
"ollama": { "enabled": false }
},
"meteringEnforcement": {
"subscriptions": false
},
"trust_proxy": $PUTER_TRUST_PROXY
}
EOF
fi
# `https` only works if something in front is actually terminating it. The
# bundled Caddy does that from ./puter/tls once you enable its `:443` block
# (see "Step 3 — TLS" in doc/self-hosting.md); your own edge proxy is fine
# too. Neither in place means Puter hands out https:// URLs for an endpoint
# that only speaks HTTP — broken logins and mixed content.
if [ "$PUTER_PROTOCOL" = "https" ] && [ ! -f puter/tls/fullchain.pem ]; then
warn "PUTER_PROTOCOL=https but ./puter/tls/fullchain.pem is missing."
warn "The bundled Caddy is still serving plain HTTP on port $PUTER_PORT."
warn "Either drop a wildcard cert into ./puter/tls and uncomment the :443"
warn "block in caddy/Caddyfile, or terminate TLS in your own proxy."
fi
# ── Step 5: bring it up ─────────────────────────────────────────────
log "docker compose up -d"
docker compose up -d
-135
View File
@@ -1,135 +0,0 @@
# Reverse proxy in front of Puter — mirrors what the prod ALB does:
# accepts every Host header, forwards to the Puter container, and lets
# the Puter app handle subdomain-based routing internally (api.*,
# site.*, app.*, etc).
#
# To enable TLS:
# 1. Drop your fullchain.pem + privkey.pem into ./puter/tls/.
# 2. Uncomment the 443 server{} block below.
# 3. Update server_name to your domain (and wildcard subdomains).
worker_processes auto;
events {
worker_connections 4096;
}
http {
# Required for Puter's WebSocket / socket.io upgrades.
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
# Rough size cap that mirrors prod ALB defaults; tune for your
# uploads. Puter chunks large uploads, so 1 GiB per request is plenty.
client_max_body_size 1024m;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
proxy_buffering off;
server_tokens off;
upstream puter_backend {
server puter:4100;
keepalive 32;
}
upstream s3_backend {
# RustFS — see `s3` service in docker-compose.full.yml. Browsers
# PUT/GET here for presigned-URL uploads / downloads. Routed via
# the `s3.<domain>` subdomain so signature verification works
# (Host header preserved end-to-end) and so HTTPS stays clean
# (no mixed-content from a port-9000 host publish).
server s3:9000;
keepalive 32;
}
# ── HTTP (port 80) ─────────────────────────────────────────────
server {
listen 80;
listen [::]:80;
server_name ~^s3\.;
location / {
proxy_pass http://s3_backend;
proxy_http_version 1.1;
# Critical: preserve the original Host so RustFS validates
# the request against the same host the URL was signed for.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
# Note: when you enable TLS, replace this block with a redirect:
# return 301 https://$host$request_uri;
location / {
proxy_pass http://puter_backend;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
# ── HTTPS (port 443) — uncomment after dropping certs in ./puter/tls/ ─
# server {
# listen 443 ssl;
# listen [::]:443 ssl;
# http2 on;
# server_name ~^s3\.;
#
# ssl_certificate /etc/nginx/tls/fullchain.pem;
# ssl_certificate_key /etc/nginx/tls/privkey.pem;
# ssl_protocols TLSv1.2 TLSv1.3;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_session_cache shared:SSL:10m;
# ssl_session_timeout 10m;
#
# location / {
# proxy_pass http://s3_backend;
# proxy_http_version 1.1;
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# }
# }
#
# server {
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
# http2 on;
# server_name _;
#
# ssl_certificate /etc/nginx/tls/fullchain.pem;
# ssl_certificate_key /etc/nginx/tls/privkey.pem;
# ssl_protocols TLSv1.2 TLSv1.3;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_session_cache shared:SSL:10m;
# ssl_session_timeout 10m;
#
# location / {
# proxy_pass http://puter_backend;
# proxy_http_version 1.1;
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# proxy_set_header X-Forwarded-Host $host;
# proxy_set_header X-Forwarded-Port $server_port;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection $connection_upgrade;
# }
# }
}
+7463 -7588
View File
File diff suppressed because it is too large. Load diff
+40 -30
View File
@@ -1,6 +1,6 @@
{
"name": "puter.com",
"version": "2.5.1",
"version": "26.07",
"author": "Puter Technologies Inc.",
"license": "AGPL-3.0-only",
"description": "Desktop environment in the browser!",
@@ -10,16 +10,14 @@
"lib": "lib"
},
"devDependencies": {
"@babel/core": "^7.29.7",
"@eslint/js": "^9.35.0",
"@playwright/test": "^1.56.1",
"@stylistic/eslint-plugin": "^5.3.1",
"@types/better-sqlite3": "^7.6.13",
"@types/express": "^5.0.0",
"@types/mime-types": "^3.0.1",
"@typescript-eslint/eslint-plugin": "^8.46.1",
"@typescript-eslint/parser": "^8.46.1",
"@vitest/coverage-v8": "^4.0.14",
"@vitest/ui": "^4.0.14",
"babel-loader": "^10.1.1",
"babel-plugin-istanbul": "^8.0.0",
"chalk": "^4.1.0",
"clean-css": "^5.3.2",
"dotenv": "^16.4.5",
@@ -27,40 +25,44 @@
"eslint": "^9.35.0",
"eslint-config-prettier": "^10.1.8",
"eslint-plugin-prettier": "^5.5.5",
"eslint-rule-composer": "^0.3.0",
"globals": "^15.15.0",
"html-webpack-plugin": "^5.6.0",
"husky": "^9.1.7",
"license-check-and-add": "^4.0.5",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-reports": "^3.2.0",
"nodemon": "^3.1.0",
"prettier": "^3.8.3",
"simple-git": "^3.32.3",
"prettier-plugin-jsdoc": "^1.8.1",
"typescript": "^5.4.5",
"uglify-js": "^3.17.4",
"vite-plugin-static-copy": "^3.3.0",
"vitest": "^4.1.5",
"webpack": "^5.88.2",
"webpack-cli": "^5.1.1",
"yaml": "^2.8.1"
"webpack-cli": "^5.1.1"
},
"scripts": {
"test:backend": "npm run setupExtensions && vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "npm run setupExtensions && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run setupExtensions && npm run build:ts",
"test:backend": "npm run setupExtensions && npm run build:workerLib && vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "npm run setupExtensions && npm run build:workerLib && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"test:puterjs": "npm run setupExtensions && vitest run --config src/puter-js/tests/api/vitest.config.ts",
"test:puterjs:node": "npm run test:puterjs -- src/puter-js/tests/api/runners/node.test.ts",
"test:puterjs:browser": "npm run test:puterjs -- src/puter-js/tests/api/runners/browser.test.ts",
"test:puterjs:workerd": "npm run test:puterjs -- src/puter-js/tests/api/runners/workerd.test.ts",
"test:puterjs:coverage": "npm run build:workerLib:coverage && npm run setupExtensions && rm -rf src/puter-js/coverage && PUTER_COVERAGE=1 vitest run --config src/puter-js/tests/api/vitest.config.ts && node ./tools/puterjsCoverageReport.mjs",
"build:workerLib:coverage": "cd src/puter-js && npm run build:coverage && cd ../worker && npm run build",
"start:gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node ./tools/start.mjs",
"start:smtp": "node ./tools/start.mjs --smtp",
"dev": "npm start",
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
"check-translations": "node tools/check-translations.js",
"prepare": "husky",
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
"check:puterjs:types": "node tools/checkPuterjsTypes.mjs",
"typecheck": "node tools/typecheck.mjs",
"typecheck:update": "node tools/typecheck.mjs --update",
"setupExtensions": "node ./tools/extensionSetup.mjs"
},
"workspaces": [
"src/*",
"tools/*",
"experiments/js-parse-and-output"
"src/*"
],
"nodemonConfig": {
"ext": "js, json, mjs, jsx, svg, css",
@@ -70,23 +72,31 @@
]
},
"dependencies": {
"@ai-sdk/openai": "^3.0.25",
"@aws-sdk/client-s3": "^3.1020.0",
"@aws-sdk/s3-request-presigner": "^3.1028.0",
"@heyputer/putility": "^1.0.2",
"ai": "^6.0.73",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.6",
"open": "^10.1.0"
"libphonenumber-js": "1.13.15",
"miniflare": "^4.20260617.1",
"open": "^10.1.0",
"sharp": "^0.35.5"
},
"engines": {
"node": ">=24.0.0"
},
"allowScripts": {
"[email protected]": true,
"[email protected]": true,
"@google/genai": false,
"[email protected]": true
"bcrypt": true,
"better-sqlite3": true,
"browser-tabs-lock": true,
"classic-level": true,
"esbuild": true,
"fsevents": true,
"protobufjs": true,
"sharp": true,
"workerd": true
},
"overrides": {
"undici": "^7.30.0"
}
}
+135
View File
@@ -0,0 +1,135 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import type { PuterServer } from './server.ts';
import { createTestUser, setupTestServer } from './testUtil.ts';
/**
* Teams is optional, so core has to work when neither the service nor the store
* is registered, which is how this test server boots.
*/
describe('core without a teams registration', () => {
let server: PuterServer;
const makeUser = async (): Promise<{ id: number; username: string }> => {
const username = `nt_${Math.random().toString(36).slice(2, 10)}`;
const created = (await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
})) as unknown as { id: number };
return { id: created.id, username };
};
beforeAll(async () => {
server = await setupTestServer({
// Without this the allowlist check returns before asking Teams.
teams_allowed_email_domains: ['test.local'],
} as never);
});
afterAll(async () => {
await server?.shutdown();
});
it('has no teams in the registry', () => {
expect(server.services.team).toBeUndefined();
expect(server.stores.team).toBeUndefined();
});
it('refuses to boot when the config asks for teams', async () => {
await expect(
setupTestServer({ teams_enabled: true } as never),
).rejects.toThrow(/teams_enabled/);
});
it('deletes an account without asking what it was charged for', async () => {
const user = await makeUser();
await expect(
server.services.userAccount.cascadeDelete(user.id),
).resolves.not.toThrow();
expect(await server.stores.user.getById(user.id)).toBeFalsy();
});
it('answers the address-claim question signup asks', async () => {
const user = await makeUser();
const row = (await server.stores.user.getById(user.id))!;
// Nothing is a seat, so the holder is returned as-is.
await expect(
server.services.signup.signupEmailHolder(row.email as string, {
releaseSeat: true,
}),
).resolves.toMatchObject({ id: user.id });
});
it('lists what is shared with an account, folding in no team shares', async () => {
const user = await makeUser();
await expect(
server.services.share.listSharedWithMe({
user: { id: user.id },
} as never),
).resolves.toMatchObject({ items: [] });
});
it('refuses a share aimed at a team, rather than failing', async () => {
// A real home directory, so the refusal is the recipient's and not
// the entry's.
const username = `nt_${Math.random().toString(36).slice(2, 10)}`;
const creds = await createTestUser(server, {
username,
password: 'Nt-Pw-12345!',
});
const row = (await server.stores.user.getByUsername(username))!;
const user = { id: row.id, username };
const actor = { user: row } as never;
expect(creds).toBeTruthy();
// The recipient cannot exist, so this is a 404 and not a crash.
await expect(
server.services.share.share(actor, {
recipient: { teamHandle: 'nobody' },
mode: 'read',
path: `/${user.username}`,
} as never),
).rejects.toMatchObject({ statusCode: 404 });
});
it('serves the homepage shell with the teams tab off', async () => {
const user = await makeUser();
const row = (await server.stores.user.getById(user.id))!;
let html = '';
const res = { send: (out: string) => (html = out) };
await server.services.homepage.send(
{
req: { headers: {}, query: {}, get: () => undefined },
res,
actor: { user: row },
} as never,
{ title: 'Puter' } as never,
{} as never,
);
const params = JSON.parse(
/gui\((\{.*?\})\);/s.exec(html)![1].replaceAll('\\u003c', '<'),
);
expect(params.teams_ui).toBe(false);
});
});
@@ -0,0 +1,692 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
ALARM_IDLE_TTL_MS,
AlarmClient,
MAX_TRACKED_ALARMS,
} from './AlarmClient';
import type { IConfig, IPagerConfig } from '../../types';
import type { AlertPayload } from './types';
const pdEvent = vi.hoisted(() => vi.fn(async () => ({})));
vi.mock('@pagerduty/pdjs', () => ({ event: pdEvent }));
const makeClient = (pager: IPagerConfig = {}) =>
new AlarmClient({ serverId: 'test-node', pager } as unknown as IConfig);
/** Register a capturing handler in place of a real transport. */
const capture = (
client: AlarmClient,
minSeverity?: 'critical' | 'error' | 'warning' | 'info',
maxSeverity?: 'critical' | 'error' | 'warning' | 'info',
) => {
const seen: AlertPayload[] = [];
client.addAlertHandler(
async (alert) => {
seen.push(alert);
},
{ name: 'capture', minSeverity, maxSeverity },
);
return seen;
};
describe('AlarmClient severity routing', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('defaults to critical when the call site says nothing', () => {
const client = makeClient();
const seen = capture(client);
client.create('boom', 'everything is on fire');
expect(seen).toHaveLength(1);
expect(seen[0].severity).toBe('critical');
});
it('honours a configured default severity', () => {
const client = makeClient({ defaultSeverity: 'info' });
const seen = capture(client);
client.create('boom', 'everything is on fire');
expect(seen[0].severity).toBe('info');
});
it('skips handlers whose floor the alarm does not reach', () => {
const client = makeClient();
const paging = capture(client, 'warning');
const chat = capture(client, 'info');
client.create('rate-limit', 'a user hit a limit', {}, 'info');
expect(paging).toHaveLength(0);
expect(chat).toHaveLength(1);
});
it('skips handlers whose ceiling the alarm exceeds', () => {
const client = makeClient();
const chat = capture(client, 'info', 'info');
client.create('rate-limit', 'a user hit a limit', {}, 'info');
client.create('outage', 'everything is on fire', {}, 'critical');
expect(chat.map((alert) => alert.id)).toEqual(['rate-limit']);
});
it('retiers an alarm from config', () => {
const client = makeClient({
severityOverrides: { 'noisy:*': 'info' },
});
const paging = capture(client, 'warning');
const chat = capture(client, 'info');
client.create('noisy:thing', 'used to page', {}, 'critical');
expect(paging).toHaveLength(0);
expect(chat[0].severity).toBe('info');
});
it('mutes an alarm from config', () => {
const client = makeClient({
severityOverrides: { 'noisy:thing': 'mute' },
});
const chat = capture(client, 'info');
client.create('noisy:thing', 'not worth reporting');
client.create('noisy:thing', 'still not worth reporting');
expect(chat).toHaveLength(0);
});
it('lets config override a known-error rule', () => {
const client = makeClient({
severityOverrides: { 'known:thing': 'critical' },
});
client.setKnownErrors([
{
match: { id: 'known:thing' },
action: { type: 'severity', value: 'info' },
},
]);
const paging = capture(client, 'warning');
client.create('known:thing', 'known but escalated');
expect(paging[0].severity).toBe('critical');
});
it('still respects a no-alert known-error rule', () => {
const client = makeClient();
client.setKnownErrors([
{ match: { id: 'known:quiet' }, action: { type: 'no-alert' } },
]);
const chat = capture(client, 'info');
client.create('known:quiet', 'suppressed');
expect(chat).toHaveLength(0);
});
});
describe('AlarmClient alert payload', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
});
it('reports occurrence counts across repeats', () => {
const client = makeClient();
const seen = capture(client);
client.create('flap', 'first');
client.create('flap', 'second');
expect(seen[0]).toMatchObject({ repeatCount: 1, isRepeat: false });
expect(seen[1]).toMatchObject({ repeatCount: 2, isRepeat: true });
});
it('renders fields as strings and lifts the stack out of the error', () => {
const client = makeClient();
const seen = capture(client);
const error = new Error('kaboom');
client.create('boom', 'failed', { error, status: 500 }, 'critical');
expect(seen[0].fields.status).toBe('500');
expect(seen[0].trace).toBe(error.stack);
expect(seen[0].shortId).toMatch(/^[a-z]+-[a-z]+-[a-z]+$/);
});
it('keeps one failing handler from starving the others', async () => {
const client = makeClient();
client.addAlertHandler(
async () => {
throw new Error('transport down');
},
{ name: 'broken' },
);
const seen = capture(client);
client.create('boom', 'failed');
await Promise.resolve();
expect(seen).toHaveLength(1);
});
it('suppresses alarms once draining', () => {
const client = makeClient();
const seen = capture(client);
client.onServerPrepareShutdown();
client.create('boom', 'too late');
expect(seen).toHaveLength(0);
});
});
describe('AlarmClient transport registration', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('splits info to Slack and everything above it to PagerDuty', async () => {
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
slack: { enabled: true, webhookUrl: 'https://hooks.example/abc' },
});
await client.onServerStart();
client.create('quiet:thing', 'informational', {}, 'info');
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
expect(pdEvent).not.toHaveBeenCalled();
client.create('loud:thing', 'paging', {}, 'critical');
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(1));
// The pager has it; chat doesn't repeat it.
expect(fetchMock).toHaveBeenCalledTimes(1);
vi.unstubAllGlobals();
});
it('sends every severity to Slack when there is no pager', async () => {
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const client = makeClient({
slack: { enabled: true, webhookUrl: 'https://hooks.example/abc' },
});
await client.onServerStart();
client.create(
'loud:thing',
'nowhere else to send this',
{},
'critical',
);
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
vi.unstubAllGlobals();
});
it('lets config widen the Slack ceiling back out', async () => {
const fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
slack: {
enabled: true,
webhookUrl: 'https://hooks.example/abc',
maxSeverity: 'critical',
},
});
await client.onServerStart();
client.create('loud:thing', 'paging', {}, 'critical');
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(1));
vi.unstubAllGlobals();
});
it('gives each occurrence its own PagerDuty incident by default', async () => {
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
});
await client.onServerStart();
client.create('scan:failed', 'first', {}, 'warning');
client.create('scan:failed', 'second', {}, 'warning');
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
const keys = pdEvent.mock.calls.map(
([arg]: [{ data: { dedup_key: string } }]) => arg.data.dedup_key,
);
expect(keys[0]).not.toBe(keys[1]);
});
it('collapses repeats of a dedup alarm onto one incident', async () => {
const client = makeClient({
pagerduty: { enabled: true, routingKey: 'rk' },
});
await client.onServerStart();
const raise = () =>
client.create(
'http_500:POST:/notif/mark-ack:deadlock',
'HTTP 500 on POST /notif/mark-ack: deadlock',
{},
'critical',
{ dedup: true },
);
raise();
raise();
await vi.waitFor(() => expect(pdEvent).toHaveBeenCalledTimes(2));
const keys = pdEvent.mock.calls.map(
([arg]: [{ data: { dedup_key: string } }]) => arg.data.dedup_key,
);
expect(keys[0]).toBe('http_500:POST:/notif/mark-ack:deadlock');
expect(keys[1]).toBe(keys[0]);
});
it('skips transports that are enabled but not configured', async () => {
const client = makeClient({
pagerduty: { enabled: true },
slack: { enabled: true },
});
await client.onServerStart();
const seen = capture(client);
client.create('boom', 'nowhere to send this');
// Only the capturing handler is registered.
expect(seen).toHaveLength(1);
expect(pdEvent).not.toHaveBeenCalled();
});
});
describe('AlarmClient alarm registry', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
});
it('finds an alarm by its full id or its short id', () => {
const client = makeClient();
client.create('disk-pressure', 'nearly full');
const alarm = client.get('disk-pressure');
expect(alarm?.message).toBe('nearly full');
expect(client.get(alarm!.shortId)).toBe(alarm);
});
it('returns nothing for an id it has never seen', () => {
expect(makeClient().get('never-raised')).toBeUndefined();
});
it('forgets an alarm once cleared, under both ids', () => {
const client = makeClient();
client.create('disk-pressure', 'nearly full');
const shortId = client.get('disk-pressure')!.shortId;
client.clear('disk-pressure');
expect(client.get('disk-pressure')).toBeUndefined();
expect(client.get(shortId)).toBeUndefined();
});
it('keeps the short id of another alarm that shares it when one is cleared', () => {
const client = makeClient();
// These two ids hash to the same short id.
client.create('alarm:274', 'first');
const shortId = client.get('alarm:274')!.shortId;
client.create('alarm:643', 'second');
expect(client.get('alarm:643')!.shortId).toBe(shortId);
client.clear('alarm:274');
expect(client.get(shortId)?.id).toBe('alarm:643');
});
it('ignores a clear for an alarm that is not active', () => {
const client = makeClient();
expect(() => client.clear('never-raised')).not.toThrow();
});
it('abbreviates long ids in the log line but keeps the short id usable', () => {
const log = vi.spyOn(console, 'error').mockImplementation(() => {});
const client = makeClient();
const longId = 'a-very-long-alarm-identifier-that-wraps';
client.create(longId, 'noisy');
const shortId = client.get(longId)!.shortId;
expect(log).toHaveBeenCalledWith(
`[alarm] ACTIVE ${shortId} (${longId.slice(0, 20)}...) :: noisy`,
);
});
it('accumulates fields across repeats of the same alarm', () => {
const client = makeClient();
const seen = capture(client);
client.create('flap', 'first', { a: 1 });
client.create('flap', 'second', { b: 2 });
expect(seen[1].fields).toEqual({ a: '1', b: '2' });
expect(client.get('flap')?.occurrences).toHaveLength(2);
});
it('caps retained occurrences while still counting every repeat', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 500; i++) {
client.create('hot', `occurrence ${i}`, { i });
}
const alarm = client.get('hot')!;
expect(alarm.count).toBe(500);
expect(alarm.occurrences).toHaveLength(20);
expect(alarm.timestamps).toHaveLength(20);
// The window kept is the most recent one, not the oldest.
expect(alarm.occurrences[19].message).toBe('occurrence 499');
// Trimming history must not rewind what the transports are told.
expect(seen.at(-1)).toMatchObject({ repeatCount: 500, isRepeat: true });
});
it('names anonymous handlers by their registration order', () => {
const client = makeClient();
client.addAlertHandler(async () => {
throw new Error('down');
});
expect(() => client.create('boom', 'x')).not.toThrow();
});
it('only logs the drain notice once', () => {
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
log.mockClear();
const client = makeClient();
client.onServerPrepareShutdown();
// A second prepare must not restart the drain.
client.onServerPrepareShutdown();
client.create('a', 'x');
client.create('b', 'y');
const drainLogs = log.mock.calls.filter(
([message]) =>
message === '[alarm] suppressing alarm while draining',
);
expect(drainLogs).toHaveLength(1);
});
it('substitutes placeholders for an empty id and message', () => {
const client = makeClient();
const seen = capture(client);
client.create('', '');
expect(seen[0]).toMatchObject({
id: 'something-bad',
message: 'something bad happened',
});
});
});
describe('AlarmClient alarm eviction', () => {
beforeEach(() => {
vi.useFakeTimers();
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
});
afterEach(() => {
vi.useRealTimers();
});
it('forgets alarms that stay quiet past the idle window, under both ids', () => {
const client = makeClient();
const ids = Array.from(
{ length: 50 },
(_, i) => `metering_usage_dropped:user-${i}`,
);
for (const id of ids) client.create(id, 'usage dropped');
const shortIds = ids.map((id) => client.get(id)!.shortId);
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS + 1);
client.create('unrelated', 'the next alarm sweeps the registry');
expect(ids.filter((id) => client.get(id))).toEqual([]);
expect(shortIds.filter((shortId) => client.get(shortId))).toEqual([]);
expect(client.get('unrelated')).toBeDefined();
});
it('treats a recurrence after the idle window as a new alarm', () => {
const client = makeClient();
const seen = capture(client);
client.create('scan:failed', 'first');
client.create('scan:failed', 'second');
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS + 1);
client.create('scan:failed', 'back again');
expect(seen.at(-1)).toMatchObject({ repeatCount: 1, isRepeat: false });
// A fresh start keeps per-occurrence incident keys from colliding.
expect(seen.at(-1)!.dedupKey).not.toBe(seen[0].dedupKey);
});
it('still treats a repeat inside the window as a repeat after a sweep', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 10; i++) client.create('flap', 'same fault');
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS - 1000);
client.create('other', 'sweeps the registry');
client.create('flap', 'same fault');
// Occurrence 11 is past the burst, so nothing new goes out for it.
expect(seen.map((alert) => alert.id)).toEqual([
...Array(10).fill('flap'),
'other',
]);
expect(client.get('flap')?.count).toBe(11);
});
it('keeps an alarm that keeps recurring for longer than the window', () => {
const client = makeClient();
for (let i = 0; i < 6; i++) {
client.create('hot', 'still broken');
vi.advanceTimersByTime(ALARM_IDLE_TTL_MS / 2);
client.create(`noise:${i}`, 'sweeps the registry');
}
expect(client.get('hot')?.count).toBe(6);
});
it('drops the least recently raised alarm once full', () => {
const client = makeClient();
for (let i = 0; i < MAX_TRACKED_ALARMS; i++) {
client.create(`user:${i}`, 'per-user alarm');
vi.advanceTimersByTime(1);
}
// Raised again, so user:1 is now the one seen longest ago.
client.create('user:0', 'per-user alarm');
client.create('one-more', 'over the cap');
expect(client.get('user:1')).toBeUndefined();
expect(client.get('user:0')?.count).toBe(2);
expect(client.get('user:2')).toBeDefined();
expect(client.get('one-more')).toBeDefined();
});
it('never holds more than the cap', () => {
const client = makeClient();
const ids = Array.from(
{ length: MAX_TRACKED_ALARMS * 3 },
(_, i) => `user:${i}`,
);
for (const id of ids) client.create(id, 'per-user alarm');
const held = ids.filter((id) => client.get(id));
expect(held).toHaveLength(MAX_TRACKED_ALARMS);
expect(held[0]).toBe(`user:${MAX_TRACKED_ALARMS * 2}`);
});
});
describe('AlarmClient known-error rules', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
});
it('suppresses alerts for a matching rule', () => {
const client = makeClient();
const seen = capture(client);
client.setKnownErrors([
{ match: { id: 'noisy' }, action: { type: 'no-alert' } },
]);
client.create('noisy', 'ignore me');
client.create('noisy', 'ignore me again');
expect(seen).toHaveLength(0);
});
it('leaves alarms with a different id alone', () => {
const client = makeClient();
const seen = capture(client);
client.setKnownErrors([
{ match: { id: 'noisy' }, action: { type: 'no-alert' } },
]);
client.create('real', 'page me');
expect(seen).toHaveLength(1);
});
it('only matches when the message matches too', () => {
const rules = [
{
match: { id: 'timeout', message: 'upstream timed out' },
action: { type: 'no-alert' as const },
},
];
// Separate clients: suppression is recorded on the alarm, so a
// second occurrence of the same id would inherit it.
const matching = makeClient();
const matchingSeen = capture(matching);
matching.setKnownErrors(rules);
matching.create('timeout', 'upstream timed out');
expect(matchingSeen).toHaveLength(0);
const differing = makeClient();
const differingSeen = capture(differing);
differing.setKnownErrors(rules);
differing.create('timeout', 'something else entirely');
expect(differingSeen.map((alert) => alert.message)).toEqual([
'something else entirely',
]);
});
it('only matches when every named field matches', () => {
const client = makeClient();
const seen = capture(client);
client.setKnownErrors([
{
match: { id: 'http', fields: { status: 404 } },
action: { type: 'no-alert' },
},
]);
client.create('http', 'not found', { status: 404 });
client.create('http-2', 'server error', { status: 500 });
expect(seen.map((alert) => alert.id)).toEqual(['http-2']);
});
it('retiers a matching alarm to a lower severity', () => {
const client = makeClient();
const paging = capture(client, 'error');
const chat = capture(client, 'info');
client.setKnownErrors([
{
match: { id: 'flaky' },
action: { type: 'severity', value: 'info' },
},
]);
client.create('flaky', 'transient', {}, 'critical');
expect(paging).toHaveLength(0);
expect(chat).toHaveLength(1);
expect(chat[0].severity).toBe('info');
});
});
describe('AlarmClient repeat reporting', () => {
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'log').mockImplementation(() => {});
pdEvent.mockClear();
});
it('reports every occurrence up to the burst, then backs off', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 12; i++) client.create('flap', 'same fault');
expect(seen).toHaveLength(10);
expect(seen[9].repeatCount).toBe(10);
});
it('reports again once the interval is reached', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 500; i++) client.create('flap', 'same fault');
expect(seen).toHaveLength(11);
expect(seen[10].repeatCount).toBe(500);
});
it('keeps a fast-repeating fault out of the log after the burst', () => {
const client = makeClient();
capture(client);
const warn = vi.mocked(console.warn);
warn.mockClear();
for (let i = 0; i < 100; i++) client.create('flap', 'same fault');
// Occurrences 2-10 only; the first is an ACTIVE line on console.error.
expect(warn.mock.calls).toHaveLength(9);
});
it('skips the payload entirely when no transport accepts the severity', () => {
const client = makeClient();
const seen = capture(client, 'critical');
client.create('quiet', 'nobody is listening', {}, 'info');
expect(seen).toHaveLength(0);
});
});
+311 -112
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -20,64 +20,84 @@
import { event as pdEvent } from '@pagerduty/pdjs';
import { inspect } from 'node:util';
import { createHash } from 'node:crypto';
import type { IConfig } from '../../types';
import type { IConfig, PagerSeverity, SeverityRule } from '../../types';
import { PuterClient } from '../types';
import {
meetsMinSeverity,
resolveSeverityOverride,
withinMaxSeverity,
} from './severity';
import { createSlackAlertHandler } from './slack';
import type {
Alarm,
AlarmFields,
AlarmOptions,
AlertHandler,
AlertPayload,
KnownErrorRule,
} from './types';
export type {
Alarm,
AlarmFields,
AlarmOptions,
AlertHandler,
AlertPayload,
KnownErrorRule,
} from './types';
export type { PagerSeverity } from '../../types';
// -- Types ------------------------------------------------------------
export interface AlarmFields {
error?: Error;
[key: string]: unknown;
interface RegisteredHandler {
name: string;
/** Lowest severity this transport accepts. */
minSeverity: PagerSeverity;
/** Highest severity this transport accepts. */
maxSeverity: PagerSeverity;
handler: AlertHandler;
}
interface AlarmOccurrence {
message: string;
fields: AlarmFields;
timestamp: number;
}
/** Severity used when neither the call site nor config picks one. */
const FALLBACK_SEVERITY: PagerSeverity = 'critical';
/**
* How many recent occurrences an alarm keeps. A hot alarm repeats for as long
* as the fault lasts, so retaining every occurrence means retaining every
* message and field set it was ever raised with, request bodies and actors
* included. The last few are what a human reads; the rest is only a count, and
* `count` keeps that.
*/
const OCCURRENCE_HISTORY_LIMIT = 20;
/**
* An alarm that hasn't recurred for this long is forgotten; its next occurrence
* is a new alarm with a fresh count. Kept far past the chat transport's repeat
* throttle so a slowly recurring fault still counts as one alarm.
*/
export const ALARM_IDLE_TTL_MS = 24 * 60 * 60 * 1000;
/**
* Most alarms held at once. Ids that embed a user or key can outrun the idle
* window; past this the least recently raised alarm is dropped.
*/
export const MAX_TRACKED_ALARMS = 1000;
interface Alarm {
id: string;
shortId: string;
message: string;
fields: AlarmFields;
error?: Error;
started: number;
timestamps: number[];
occurrences: AlarmOccurrence[];
severity?: PagerSeverity;
noAlert?: boolean;
}
// A fast-recurring fault repeats far faster than it is useful to report, and
// every report allocates a log line and a payload. Report every occurrence up
// to the burst, then one per interval; `count` still tracks all of them.
const REPEAT_REPORT_BURST = 10;
const REPEAT_REPORT_INTERVAL = 500;
export type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
export interface AlertPayload {
id: string;
message: string;
source: string;
severity: PagerSeverity;
custom?: Record<string, unknown>;
}
type AlertHandler = (alert: AlertPayload) => Promise<void>;
interface KnownErrorRule {
match: {
id: string;
message?: string;
fields?: Record<string, unknown>;
};
action: {
type: 'no-alert' | 'severity';
value?: PagerSeverity;
};
}
const shouldReportRepeat = (count: number): boolean =>
count <= REPEAT_REPORT_BURST || count % REPEAT_REPORT_INTERVAL === 0;
/** Keeps `info` alarms out of the paging system unless config says otherwise. */
const DEFAULT_PAGERDUTY_MIN_SEVERITY: PagerSeverity = 'warning';
/** Slack's ceiling once a pager exists: chat gets what doesn't page. */
const DEFAULT_SLACK_MAX_SEVERITY_WITH_PAGER: PagerSeverity = 'info';
// -- Helpers ----------------------------------------------------------
/**
* Deterministic short identifier derived from an alarm ID.
* Produces a readable 3-word slug like "amber-delta-fox".
* Deterministic short identifier derived from an alarm ID. Produces a readable
* 3-word slug like "amber-delta-fox".
*/
const WORD_POOL = [
'alpha',
@@ -169,13 +189,18 @@ function cleanFields(fields: AlarmFields): Record<string, string> {
// -- AlarmClient ------------------------------------------------------
/**
* Manages system alarms and dispatches alerts to external paging
* services (PagerDuty, or any registered handler).
* Manages system alarms and routes them to alert transports by severity.
*
* Severity is the routing decision: each transport declares the severity window
* it accepts, so `critical` pages on-call while `info` only reaches the chat
* channel. Config can retier or mute any alarm id after the fact — see
* `pager.severityOverrides` in {@link IConfig}.
*/
export class AlarmClient extends PuterClient {
/** Ordered by last occurrence: a repeat moves its alarm to the back. */
private alarms = new Map<string, Alarm>();
private aliases = new Map<string, Alarm>();
private alertHandlers: AlertHandler[] = [];
private alertHandlers: RegisteredHandler[] = [];
private knownErrors: KnownErrorRule[] = [];
private draining = false;
private drainLogged = false;
@@ -187,39 +212,8 @@ export class AlarmClient extends PuterClient {
// -- Lifecycle ----------------------------------------------------
override async onServerStart(): Promise<void> {
const pagerConf = this.config.pager;
if (!pagerConf?.pagerduty?.enabled) return;
const routingKey = pagerConf.pagerduty.routingKey;
if (!routingKey) {
console.warn(
'[alarm] PagerDuty enabled but no routingKey configured',
);
return;
}
const serverId = this.config.serverId;
this.alertHandlers.push(async (alert) => {
await pdEvent({
data: {
routing_key: routingKey,
event_action: 'trigger',
dedup_key: alert.id,
payload: {
summary: alert.message,
source: alert.source,
severity: alert.severity,
custom_details: {
...alert.custom,
server_id: serverId,
},
},
},
});
});
console.log('[alarm] PagerDuty handler registered');
const paging = this.registerPagerDuty();
this.registerSlack({ paging });
}
override onServerPrepareShutdown(): void {
@@ -228,22 +222,108 @@ export class AlarmClient extends PuterClient {
console.log('[alarm] entering drain mode — suppressing new alarms');
}
/** @returns Whether a working PagerDuty transport was registered. */
private registerPagerDuty(): boolean {
const pagerDutyConf = this.config.pager?.pagerduty;
if (!pagerDutyConf?.enabled) return false;
const routingKey = pagerDutyConf.routingKey;
if (!routingKey) {
console.warn(
'[alarm] PagerDuty enabled but no routingKey configured',
);
return false;
}
const serverId = this.config.serverId;
const minSeverity =
pagerDutyConf.minSeverity ?? DEFAULT_PAGERDUTY_MIN_SEVERITY;
this.addAlertHandler(
async (alert) => {
await pdEvent({
data: {
routing_key: routingKey,
event_action: 'trigger',
dedup_key: alert.dedupKey,
payload: {
summary: alert.message,
source: alert.source,
severity: alert.severity,
custom_details: {
...alert.custom,
server_id: serverId,
},
},
},
});
},
{ name: 'pagerduty', minSeverity },
);
console.log(
`[alarm] PagerDuty handler registered (min severity: ${minSeverity})`,
);
return true;
}
private registerSlack({ paging }: { paging: boolean }): void {
const slackConf = this.config.pager?.slack;
if (!slackConf?.enabled) return;
if (!slackConf.webhookUrl) {
console.warn('[alarm] Slack enabled but no webhookUrl configured');
return;
}
const minSeverity = slackConf.minSeverity ?? 'info';
// With a pager taking everything from `warning` up, chat is where the
// rest is recorded — reposting the paging tiers there only trains
// people to skim the channel. Without one, Slack is the only place an
// alarm can land, so it takes all of them.
const maxSeverity =
slackConf.maxSeverity ??
(paging ? DEFAULT_SLACK_MAX_SEVERITY_WITH_PAGER : 'critical');
this.addAlertHandler(
createSlackAlertHandler(slackConf, {
serverId: this.config.serverId,
}),
{ name: 'slack', minSeverity, maxSeverity },
);
console.log(
`[alarm] Slack handler registered (severity ${minSeverity}..${maxSeverity})`,
);
}
// -- Public API ---------------------------------------------------
/**
* Create or update an alarm. If the alarm ID already exists, the
* occurrence count is incremented and a repeat alert is dispatched.
* Create or update an alarm. If the alarm ID already exists, the occurrence
* count is incremented and a repeat alert is dispatched. An alarm idle for
* {@link ALARM_IDLE_TTL_MS} is forgotten, so its next occurrence is new.
*
* `severity` is the PagerDuty severity for this alarm; omit for the
* default 'critical'. Use 'info' / 'warning' for expected-but-worth-
* tracking signals (e.g. a user hitting a rate limit) so they record
* and de-dupe like any other alarm but don't page on-call.
* `severity` decides where the alarm lands:
*
* Critical — a real outage; pages on-call. Reserve it for unhandled server
* errors. error — pages on-call as well; prefer `critical` or `warning`.
* warning — worth a look soon, but nobody gets woken up. info — a record in
* the chat channel; never pages.
*
* Omit it to take `pager.defaultSeverity` (itself defaulting to
* 'critical'). Operators can retier or mute any alarm id from config
* afterwards, so the value here is the starting point, not the last word.
*
* Pass `{ dedup: true }` when repeats of this id are one recurring fault
* that should collapse into a single incident — see {@link AlarmOptions}.
*/
create(
id: string,
message: string,
fields: AlarmFields = {},
severity?: PagerSeverity,
opts: AlarmOptions = {},
): void {
if (this.draining) {
if (!this.drainLogged) {
@@ -253,9 +333,12 @@ export class AlarmClient extends PuterClient {
return;
}
this.evict();
const existing = this.alarms.get(id);
if (existing) {
this.alarms.delete(id);
this.alarms.set(id, existing);
this.recordOccurrence(existing, message, fields);
this.handleRepeat(existing);
return;
@@ -267,8 +350,12 @@ export class AlarmClient extends PuterClient {
message,
fields,
severity,
dedup: opts.dedup,
started: Date.now(),
timestamps: [Date.now()],
// `recordOccurrence` below stamps the first occurrence; seeding one
// here too would report every alarm as one occurrence ahead.
count: 0,
timestamps: [],
occurrences: [],
};
if (fields.error) alarm.error = fields.error;
@@ -276,6 +363,7 @@ export class AlarmClient extends PuterClient {
this.alarms.set(id, alarm);
this.aliases.set(alarm.shortId, alarm);
this.recordOccurrence(alarm, message, fields);
this.evict();
this.handleNew(alarm);
}
@@ -284,8 +372,7 @@ export class AlarmClient extends PuterClient {
const alarm = this.alarms.get(id);
if (!alarm) return;
this.alarms.delete(id);
this.aliases.delete(alarm.shortId);
this.forget(alarm);
console.log(`[alarm] CLEAR ${displayId(alarm)} :: ${alarm.message}`);
}
@@ -295,37 +382,84 @@ export class AlarmClient extends PuterClient {
}
/**
* Register an additional alert handler. Handlers are called for
* every alarm that isn't suppressed by a known-error rule.
* Register an additional alert handler. Handlers are called for every alarm
* that isn't suppressed by a known-error rule or muted by config, and whose
* severity falls inside the handler's own `minSeverity`..`maxSeverity`
* window (default: everything).
*/
addAlertHandler(handler: AlertHandler): void {
this.alertHandlers.push(handler);
addAlertHandler(
handler: AlertHandler,
opts: {
name?: string;
minSeverity?: PagerSeverity;
maxSeverity?: PagerSeverity;
} = {},
): void {
this.alertHandlers.push({
name: opts.name ?? `handler-${this.alertHandlers.length}`,
minSeverity: opts.minSeverity ?? 'info',
maxSeverity: opts.maxSeverity ?? 'critical',
handler,
});
}
/**
* Add rules that can suppress or adjust severity of known errors.
*/
/** Add rules that can suppress or adjust severity of known errors. */
setKnownErrors(rules: KnownErrorRule[]): void {
this.knownErrors = rules;
}
// -- Internals ----------------------------------------------------
/**
* Drop idle alarms, then the least recently raised ones past the cap. The
* map's order puts both at the front, so this stops at the first keeper.
*/
private evict(): void {
const now = Date.now();
for (const alarm of this.alarms.values()) {
const lastSeen = alarm.timestamps.at(-1) ?? alarm.started;
if (
this.alarms.size <= MAX_TRACKED_ALARMS &&
now - lastSeen < ALARM_IDLE_TTL_MS
) {
return;
}
this.forget(alarm);
}
}
private forget(alarm: Alarm): void {
this.alarms.delete(alarm.id);
// Short ids can collide; leave an alias that now names another alarm.
if (this.aliases.get(alarm.shortId) === alarm) {
this.aliases.delete(alarm.shortId);
}
}
private recordOccurrence(
alarm: Alarm,
message: string,
fields: AlarmFields,
): void {
const now = Date.now();
alarm.message = message;
alarm.fields = { ...alarm.fields, ...fields };
alarm.timestamps.push(Date.now());
alarm.count++;
if (fields.error) alarm.error = fields.error;
alarm.occurrences.push({
message,
fields,
timestamp: Date.now(),
});
alarm.timestamps.push(now);
alarm.occurrences.push({ message, fields, timestamp: now });
if (alarm.timestamps.length > OCCURRENCE_HISTORY_LIMIT) {
alarm.timestamps.splice(
0,
alarm.timestamps.length - OCCURRENCE_HISTORY_LIMIT,
);
alarm.occurrences.splice(
0,
alarm.occurrences.length - OCCURRENCE_HISTORY_LIMIT,
);
}
}
private applyKnownErrors(alarm: Alarm): void {
@@ -372,8 +506,10 @@ export class AlarmClient extends PuterClient {
private handleRepeat(alarm: Alarm): void {
this.applyKnownErrors(alarm);
if (!shouldReportRepeat(alarm.count)) return;
console.warn(
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.timestamps.length})`,
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.count})`,
);
if (alarm.noAlert) return;
@@ -381,25 +517,88 @@ export class AlarmClient extends PuterClient {
this.dispatchAlert(alarm);
}
/**
* Call-site severity, then any known-error rule (both already on the
* alarm), then the config override — so an operator always has the last
* word over what the code asked for.
*/
private resolveSeverity(alarm: Alarm): SeverityRule {
const base =
alarm.severity ??
this.config.pager?.defaultSeverity ??
FALLBACK_SEVERITY;
return (
resolveSeverityOverride(
alarm.id,
this.config.pager?.severityOverrides,
) ?? base
);
}
private hasHandlerFor(severity: PagerSeverity): boolean {
for (const { minSeverity, maxSeverity } of this.alertHandlers) {
if (
meetsMinSeverity(severity, minSeverity) &&
withinMaxSeverity(severity, maxSeverity)
) {
return true;
}
}
return false;
}
private dispatchAlert(alarm: Alarm): void {
const severity = alarm.severity ?? 'critical';
const resolved = this.resolveSeverity(alarm);
if (resolved === 'mute') {
if (!alarm.muteLogged) {
alarm.muteLogged = true;
console.log(`[alarm] MUTED by config ${displayId(alarm)}`);
}
return;
}
alarm.severity = resolved;
// Build nothing for a severity no transport accepts — the payload
// (cleaned fields plus the stack, twice) is the expensive part.
if (!this.hasHandlerFor(resolved)) return;
const fieldsClean = cleanFields(alarm.fields);
const repeatCount = alarm.count;
const id = alarm.id || 'something-bad';
const payload: AlertPayload = {
id: alarm.id || 'something-bad',
id,
// A de-duplicating transport should fold repeats of a recurring
// fault into one incident, but everything else is a fresh event
// each time it fires — the start time keeps occurrence numbering
// from colliding with an incident left open by an earlier boot.
dedupKey: alarm.dedup
? id
: `${id}#${alarm.started}.${repeatCount}`,
shortId: alarm.shortId,
message: alarm.message || alarm.id || 'something bad happened',
source: 'alarm',
severity,
severity: resolved,
fields: fieldsClean,
trace: alarm.error?.stack,
repeatCount,
isRepeat: repeatCount > 1,
custom: {
fields: fieldsClean,
trace: alarm.error?.stack,
repeat_count: alarm.timestamps.length,
repeat_count: repeatCount,
},
};
for (const handler of this.alertHandlers) {
for (const { name, minSeverity, maxSeverity, handler } of this
.alertHandlers) {
if (!meetsMinSeverity(resolved, minSeverity)) continue;
if (!withinMaxSeverity(resolved, maxSeverity)) continue;
handler(payload).catch((err) => {
console.error(`[alarm] alert handler failed: ${err?.message}`);
console.error(
`[alarm] ${name} alert handler failed: ${err?.message}`,
);
});
}
}
@@ -0,0 +1,76 @@
import { describe, expect, it, vi } from 'vitest';
import { meetsMinSeverity, resolveSeverityOverride } from './severity';
describe('meetsMinSeverity', () => {
it('accepts everything at or above the floor', () => {
expect(meetsMinSeverity('critical', 'warning')).toBe(true);
expect(meetsMinSeverity('error', 'warning')).toBe(true);
expect(meetsMinSeverity('warning', 'warning')).toBe(true);
expect(meetsMinSeverity('info', 'warning')).toBe(false);
});
it('lets an info floor take every severity', () => {
for (const severity of [
'critical',
'error',
'warning',
'info',
] as const) {
expect(meetsMinSeverity(severity, 'info')).toBe(true);
}
});
});
describe('resolveSeverityOverride', () => {
it('returns undefined without overrides or on no match', () => {
expect(resolveSeverityOverride('a:b', undefined)).toBeUndefined();
expect(
resolveSeverityOverride('a:b', { 'c:*': 'info' }),
).toBeUndefined();
});
it('matches an exact id', () => {
expect(
resolveSeverityOverride('cronMonitor:lowSignupRate', {
'cronMonitor:lowSignupRate': 'info',
}),
).toBe('info');
});
it('matches a prefix pattern', () => {
expect(
resolveSeverityOverride('cronMonitor:high_aiLogEntries', {
'cronMonitor:*': 'warning',
}),
).toBe('warning');
});
it('prefers the exact id over a prefix pattern', () => {
expect(
resolveSeverityOverride('cronMonitor:lowSignupRate', {
'cronMonitor:*': 'warning',
'cronMonitor:lowSignupRate': 'mute',
}),
).toBe('mute');
});
it('prefers the longest matching prefix', () => {
expect(
resolveSeverityOverride('abuse:card-verification:setup-failed', {
'abuse:*': 'info',
'abuse:card-verification:*': 'mute',
}),
).toBe('mute');
});
it('drops an unrecognized rule rather than guessing', () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
expect(
resolveSeverityOverride('a:b', {
'a:b': 'silent' as unknown as 'mute',
}),
).toBeUndefined();
expect(warn).toHaveBeenCalled();
warn.mockRestore();
});
});
+94
View File
@@ -0,0 +1,94 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { PagerSeverity, SeverityRule } from '../../types';
/** Ascending urgency. A transport takes everything at or above its floor. */
const RANK: Record<PagerSeverity, number> = {
info: 0,
warning: 1,
error: 2,
critical: 3,
};
const SEVERITIES = Object.keys(RANK) as PagerSeverity[];
export function isPagerSeverity(value: unknown): value is PagerSeverity {
return typeof value === 'string' && value in RANK;
}
/** True when `severity` is urgent enough for a transport whose floor is `min`. */
export function meetsMinSeverity(
severity: PagerSeverity,
min: PagerSeverity,
): boolean {
return RANK[severity] >= RANK[min];
}
/**
* True when `severity` is quiet enough for a transport whose ceiling is `max`.
* A ceiling is what keeps a chat transport from repeating everything the pager
* already delivered.
*/
export function withinMaxSeverity(
severity: PagerSeverity,
max: PagerSeverity,
): boolean {
return RANK[severity] <= RANK[max];
}
/**
* Look up the operator override for an alarm id. Exact ids win over prefix
* patterns (`cronMonitor:*`); among patterns the longest prefix wins, so a
* specific rule can carve an exception out of a broad one.
*/
export function resolveSeverityOverride(
id: string,
overrides: Record<string, SeverityRule> | undefined,
): SeverityRule | undefined {
if (!overrides) return undefined;
const exact = overrides[id];
if (exact !== undefined) return validRule(id, exact);
let bestLength = -1;
let best: SeverityRule | undefined;
for (const [pattern, rule] of Object.entries(overrides)) {
if (!pattern.endsWith('*')) continue;
const prefix = pattern.slice(0, -1);
if (!id.startsWith(prefix)) continue;
if (prefix.length <= bestLength) continue;
bestLength = prefix.length;
best = rule;
}
return best === undefined ? undefined : validRule(id, best);
}
/**
* A typo in the override map would otherwise silently mute or escalate an
* alarm, so an unrecognized value is dropped with a warning instead.
*/
function validRule(id: string, rule: SeverityRule): SeverityRule | undefined {
if (rule === 'mute' || isPagerSeverity(rule)) return rule;
console.warn(
`[alarm] ignoring invalid severity override "${rule}" for ${id} ` +
`(expected mute or one of ${SEVERITIES.join(', ')})`,
);
return undefined;
}
+206
View File
@@ -0,0 +1,206 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { buildSlackMessage, createSlackAlertHandler } from './slack';
import type { AlertPayload } from './types';
const alert = (over: Partial<AlertPayload> = {}): AlertPayload => ({
id: 'cronMonitor:high_aiLogEntries',
dedupKey: 'cronMonitor:high_aiLogEntries#1.1',
shortId: 'amber-delta-fox',
message: 'High AI log entries: 1200 in the last 10 minutes',
source: 'alarm',
severity: 'warning',
fields: { count: '1200', threshold: '1000' },
repeatCount: 1,
isRepeat: false,
...over,
});
describe('buildSlackMessage', () => {
it('renders severity, message and fields', () => {
const msg = buildSlackMessage(alert(), {
channel: '#alerts',
username: 'puter-alarms',
serverId: 'oregon',
});
expect(msg.text).toContain('[WARNING]');
expect(msg.text).toContain('High AI log entries');
expect(msg.channel).toBe('#alerts');
expect(msg.username).toBe('puter-alarms');
expect(msg.attachments[0].fields).toEqual([
{ title: 'count', value: '1200', short: true },
{ title: 'threshold', value: '1000', short: true },
]);
expect(msg.attachments[0].footer).toBe(
'amber-delta-fox • cronMonitor:high_aiLogEntries • oregon',
);
});
it('marks repeats with an occurrence count', () => {
const msg = buildSlackMessage(
alert({ repeatCount: 7, isRepeat: true }),
);
expect(msg.text).toContain('(x7)');
});
it('colours each severity differently', () => {
const colors = (['critical', 'error', 'warning', 'info'] as const).map(
(severity) =>
buildSlackMessage(alert({ severity })).attachments[0].color,
);
expect(new Set(colors).size).toBe(4);
});
it('puts the stack in a code block and keeps it out of the fields', () => {
const msg = buildSlackMessage(
alert({
fields: { error: 'Error: boom', path: '/api/x' },
trace: 'Error: boom\n at handler',
}),
);
expect(msg.attachments[0].text).toBe(
'```Error: boom\n at handler```',
);
expect(msg.attachments[0].fields.map((f) => f.title)).toEqual(['path']);
});
it('truncates long values and caps the field count', () => {
const fields: Record<string, string> = { long: 'x'.repeat(1000) };
for (let i = 0; i < 30; i++) fields[`f${i}`] = String(i);
const msg = buildSlackMessage(alert({ fields }));
expect(msg.attachments[0].fields.length).toBe(12);
expect(msg.attachments[0].fields[0].value).toHaveLength(400);
expect(msg.attachments[0].fields[0].value.endsWith('…')).toBe(true);
});
it('omits channel and username when unset', () => {
const msg = buildSlackMessage(alert());
expect(msg.channel).toBeUndefined();
expect(msg.username).toBeUndefined();
});
});
describe('createSlackAlertHandler', () => {
let fetchMock: ReturnType<typeof vi.fn>;
beforeEach(() => {
vi.useFakeTimers();
fetchMock = vi.fn(async () => ({ ok: true, status: 200 }));
vi.stubGlobal('fetch', fetchMock);
});
afterEach(() => {
vi.useRealTimers();
vi.unstubAllGlobals();
});
it('posts the payload to the webhook', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
channel: '#alerts',
});
await handler(alert());
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://hooks.example/abc');
expect(init.method).toBe('POST');
expect(JSON.parse(init.body).channel).toBe('#alerts');
});
it('throttles repeats of the same alarm and lets other ids through', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
await handler(alert());
await handler(alert({ repeatCount: 2, isRepeat: true }));
expect(fetchMock).toHaveBeenCalledTimes(1);
await handler(alert({ id: 'other:alarm' }));
expect(fetchMock).toHaveBeenCalledTimes(2);
vi.advanceTimersByTime(60_000);
await handler(alert({ repeatCount: 3, isRepeat: true }));
expect(fetchMock).toHaveBeenCalledTimes(3);
});
it('posts every occurrence when throttling is disabled', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 0,
});
await handler(alert());
await handler(alert({ repeatCount: 2, isRepeat: true }));
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('does not let a failed post consume the throttle slot', async () => {
fetchMock.mockResolvedValueOnce({ ok: false, status: 500 });
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
await expect(handler(alert())).rejects.toThrow('500');
await handler(alert());
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('releases the throttle slot when the post itself errors', async () => {
fetchMock.mockRejectedValueOnce(new Error('socket hang up'));
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
await expect(handler(alert())).rejects.toThrow('socket hang up');
await handler(alert());
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('evicts throttle entries that have aged out once the map fills', async () => {
const throttleMs = 60_000;
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: throttleMs,
});
// Fill to the high-water mark, then age every entry past the window.
for (let i = 0; i < 5000; i++) {
await handler(alert({ id: `aged:${i}` }));
}
vi.advanceTimersByTime(throttleMs);
await handler(alert({ id: 'fresh' }));
expect(fetchMock).toHaveBeenCalledTimes(5001);
// The aged ids were pruned, so they post again immediately.
await handler(alert({ id: 'aged:0' }));
expect(fetchMock).toHaveBeenCalledTimes(5002);
});
it('drops the oldest entry when the map is full of live ones', async () => {
const handler = createSlackAlertHandler({
webhookUrl: 'https://hooks.example/abc',
repeatThrottleMs: 60_000,
});
for (let i = 0; i < 5000; i++) {
await handler(alert({ id: `live:${i}` }));
}
// Nothing has aged out, so the oldest id makes room for the new one.
await handler(alert({ id: 'newcomer' }));
expect(fetchMock).toHaveBeenCalledTimes(5001);
await handler(alert({ id: 'live:0' }));
expect(fetchMock).toHaveBeenCalledTimes(5002);
// The newcomer is still throttled.
await handler(alert({ id: 'newcomer' }));
expect(fetchMock).toHaveBeenCalledTimes(5002);
});
});
+163
View File
@@ -0,0 +1,163 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { ISlackAlertConfig, PagerSeverity } from '../../types';
import { BoundedTtlMap } from '../../util/boundedTtlMap';
import type { AlertHandler, AlertPayload } from './types';
const REQUEST_TIMEOUT_MS = 5000;
export const DEFAULT_REPEAT_THROTTLE_MS = 15 * 60 * 1000;
/** Beyond this many tracked alarm ids, throttle state is pruned. */
const MAX_THROTTLE_ENTRIES = 5000;
const MAX_FIELDS = 12;
const MAX_FIELD_VALUE = 400;
const MAX_TRACE = 1500;
/** Values under this length sit two-per-row in the Slack attachment. */
const SHORT_FIELD_LENGTH = 40;
const STYLE: Record<PagerSeverity, { emoji: string; color: string }> = {
critical: { emoji: ':rotating_light:', color: '#d64545' },
error: { emoji: ':red_circle:', color: '#e08d4c' },
warning: { emoji: ':warning:', color: '#e0b84c' },
info: { emoji: ':information_source:', color: '#4c8de0' },
};
interface SlackField {
title: string;
value: string;
short: boolean;
}
export interface SlackMessage {
text: string;
channel?: string;
username?: string;
icon_emoji?: string;
attachments: Array<{
color: string;
fallback: string;
fields: SlackField[];
footer: string;
mrkdwn_in: string[];
text?: string;
}>;
}
function truncate(value: string, max: number): string {
return value.length <= max ? value : `${value.slice(0, max - 1)}…`;
}
/**
* Render an alert as an incoming-webhook payload: severity-coloured attachment,
* the alarm's fields as a table, and the stack (when there is one) as a code
* block. Split out from the handler so the formatting is testable on its own.
*/
export function buildSlackMessage(
alert: AlertPayload,
opts: { channel?: string; username?: string; serverId?: string } = {},
): SlackMessage {
const style = STYLE[alert.severity] ?? STYLE.info;
const repeat = alert.isRepeat ? ` (x${alert.repeatCount})` : '';
const headline = `${style.emoji} *[${alert.severity.toUpperCase()}]* ${alert.message}${repeat}`;
const fields: SlackField[] = [];
for (const [key, value] of Object.entries(alert.fields)) {
if (fields.length >= MAX_FIELDS) break;
// `error` is already rendered as the trace block below.
if (key === 'error') continue;
const rendered = truncate(value, MAX_FIELD_VALUE);
fields.push({
title: key,
value: rendered,
short: rendered.length <= SHORT_FIELD_LENGTH,
});
}
const footerParts = [alert.shortId, alert.id];
if (opts.serverId) footerParts.push(opts.serverId);
return {
text: headline,
...(opts.channel ? { channel: opts.channel } : {}),
...(opts.username ? { username: opts.username } : {}),
attachments: [
{
color: style.color,
fallback: `[${alert.severity}] ${alert.message}`,
fields,
footer: footerParts.join(' • '),
mrkdwn_in: ['text'],
...(alert.trace
? { text: '```' + truncate(alert.trace, MAX_TRACE) + '```' }
: {}),
},
],
};
}
/**
* Post alerts to a Slack incoming webhook. Repeats of the same alarm id are
* throttled so a hot loop doesn't flood the channel — the occurrence count on
* the next post that gets through tells the reader what they missed.
*/
export function createSlackAlertHandler(
conf: ISlackAlertConfig,
opts: { serverId?: string } = {},
): AlertHandler {
const webhookUrl = conf.webhookUrl as string;
const throttleMs = conf.repeatThrottleMs ?? DEFAULT_REPEAT_THROTTLE_MS;
// A zero or negative throttle expires every entry on arrival.
const lastPosted = new BoundedTtlMap<string, true>({
maxEntries: MAX_THROTTLE_ENTRIES,
ttlMs: throttleMs,
});
return async (alert) => {
if (!lastPosted.shouldEmit(alert.id)) return;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
try {
const res = await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(
buildSlackMessage(alert, {
channel: conf.channel,
username: conf.username,
serverId: opts.serverId,
}),
),
signal: controller.signal,
});
if (!res.ok) {
// A rejected post shouldn't leave the id marked as delivered.
lastPosted.delete(alert.id);
throw new Error(`Slack webhook returned ${res.status}`);
}
} catch (err) {
lastPosted.delete(alert.id);
throw err;
} finally {
clearTimeout(timer);
}
};
}
+105
View File
@@ -0,0 +1,105 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { PagerSeverity } from '../../types';
export interface AlarmFields {
error?: Error;
[key: string]: unknown;
}
export interface AlarmOccurrence {
message: string;
fields: AlarmFields;
timestamp: number;
}
export interface AlarmOptions {
/**
* Collapse repeats of this alarm id into a single incident on transports
* that de-duplicate (PagerDuty). Only right when the id already pins down
* one recurring fault — an uncaught error on a route, say — so that N
* occurrences really are one thing to fix. Off by default: every other
* alarm raises its own incident per occurrence.
*/
dedup?: boolean;
}
export interface Alarm extends AlarmOptions {
id: string;
shortId: string;
message: string;
fields: AlarmFields;
error?: Error;
started: number;
/**
* Every occurrence ever counted, including those aged out of the two lists
* below.
*/
count: number;
/**
* Timestamps of the most recent occurrences only — see
* `OCCURRENCE_HISTORY_LIMIT`.
*/
timestamps: number[];
/** The most recent occurrences only — see `OCCURRENCE_HISTORY_LIMIT`. */
occurrences: AlarmOccurrence[];
severity?: PagerSeverity;
noAlert?: boolean;
/** Set once a config mute has been logged, so it's reported only once. */
muteLogged?: boolean;
}
export interface AlertPayload {
id: string;
/**
* Key a de-duplicating transport groups by. Equal to `id` for alarms raised
* with `dedup`, and unique per occurrence for everything else.
*/
dedupKey: string;
/** Readable slug for the same alarm, for humans quoting it back. */
shortId: string;
message: string;
source: string;
severity: PagerSeverity;
/** Field values rendered as strings, ready to display. */
fields: Record<string, string>;
/** Stack of the attached error, when the alarm carried one. */
trace?: string;
/** Total occurrences of this alarm so far, including this one. */
repeatCount: number;
/** False for the first occurrence of an alarm id. */
isRepeat: boolean;
/** PagerDuty `custom_details` payload. */
custom?: Record<string, unknown>;
}
export type AlertHandler = (alert: AlertPayload) => Promise<void>;
export interface KnownErrorRule {
match: {
id: string;
message?: string;
fields?: Record<string, unknown>;
};
action: {
type: 'no-alert' | 'severity';
value?: PagerSeverity;
};
}
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,24 +17,9 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Optional ClickHouse client.
*
* ClickHouse is NOT a hard dependency. Core Puter runs without it — the app
* stats read path (open / unique-user counts) falls back to the primary SQL
* database. `this.clients.clickhouse` is therefore typed as optional and is
* `undefined` in default and self-hosted setups; callers MUST branch on its
* presence and fall back to SQL.
*
* A production deployment can register a real ClickHouse client via an
* extension (`extension.registerClient('clickhouse', client)`) to offload the
* analytics queries off the primary database and keep the stats path fast at
* scale. When registered, the instance flows into `this.clients.clickhouse`
* everywhere it's typed.
*
* Only the surface the stats path actually consumes is declared here. Extend
* this interface (don't widen to `any`) when a new query shape is needed.
*/
// Optional analytics client, registered by an extension. Only the surface the
// app-stats path consumes is declared; extend it rather than widening to `any`.
export interface ClickhouseQueryResult {
json<T = Record<string, unknown>>(): Promise<T[]>;
}
@@ -0,0 +1,330 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { DatabaseClientFactory } from './index.js';
import { MySQLDatabaseClient } from './MySQLDatabaseClient.js';
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
const config = (engine: string): IConfig =>
({ port: 0, extensions: [], database: { engine } }) as IConfig;
const sqlite = () => new SqliteDatabaseClient(config('sqlite'));
const mysql = () => new MySQLDatabaseClient(config('mysql'));
const postgres = () => new PostgresDatabaseClient(config('postgres'));
/**
* Minimal concrete client backed by fixture rows — exercises the shared helpers
* on `AbstractDatabaseClient` that the real engines inherit.
*/
class FixtureDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'fixture';
readonly reads: string[] = [];
readonly preads: string[] = [];
readonly writes: { query: string; params: unknown[] }[] = [];
constructor(
private readonly replicaRows: Record<string, unknown>[] | Error,
private readonly primaryRows: Record<string, unknown>[] = [],
private readonly replica = true,
) {
super(config('fixture'));
}
protected override hasReadReplica(): boolean {
return this.replica;
}
override async read(query: string): Promise<Record<string, unknown>[]> {
this.reads.push(query);
if (this.replicaRows instanceof Error) throw this.replicaRows;
return this.replicaRows;
}
override async pread(query: string): Promise<Record<string, unknown>[]> {
this.preads.push(query);
return this.primaryRows;
}
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
this.writes.push({ query, params });
return { insertId: 1, affectedRows: 1, anyRowsAffected: true };
}
}
describe('DatabaseClientFactory', () => {
it('picks sqlite when no engine is configured', () => {
expect(
new DatabaseClientFactory({ port: 0, extensions: [] } as IConfig),
).toBeInstanceOf(SqliteDatabaseClient);
});
it('refuses an unknown engine name', () => {
expect(() => new DatabaseClientFactory(config('cassandra'))).toThrow(
'Unknown database engine: cassandra',
);
});
});
describe('AbstractDatabaseClient — unimplemented surface', () => {
const bare = () => new AbstractDatabaseClient(config('none'));
it('refuses to run queries until a subclass implements them', async () => {
await expect(bare().read('SELECT 1')).rejects.toThrow(
'DatabaseClient.read() not implemented',
);
await expect(bare().pread('SELECT 1')).rejects.toThrow(
'DatabaseClient.pread() not implemented',
);
await expect(bare().write('DELETE FROM x')).rejects.toThrow(
'DatabaseClient.write() not implemented',
);
await expect(bare().batchWrite([])).rejects.toThrow(
'DatabaseClient.batchWrite() not implemented',
);
});
});
describe('AbstractDatabaseClient — replica-aware reads', () => {
it('avoids a primary read when the replica has rows in sequential mode', async () => {
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
expect(
await client.tryHardRead('SELECT 1', [], { race: false }),
).toEqual([{ id: 1 }]);
expect(client.preads).toEqual([]);
});
it.each([[], new Error('replica unavailable')])(
'falls back once in sequential mode',
async (replica) => {
const client = new FixtureDatabaseClient(replica, [{ id: 2 }]);
expect(
await client.tryHardRead('SELECT 1', [], { race: false }),
).toEqual([{ id: 2 }]);
expect(client.preads).toEqual(['SELECT 1']);
},
);
it('returns replica rows without waiting on the primary', async () => {
const client = new FixtureDatabaseClient([{ id: 1 }], [{ id: 2 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 1 },
]);
});
it('falls back to the primary when the replica has not caught up', async () => {
const client = new FixtureDatabaseClient([], [{ id: 2 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 2 },
]);
});
it('falls back to the primary when the replica read throws', async () => {
const client = new FixtureDatabaseClient(
new Error('replica unavailable'),
[{ id: 3 }],
);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 3 },
]);
});
it('surfaces the primary error when both reads fail', async () => {
const client = new FixtureDatabaseClient(new Error('replica down'));
client.pread = async () => {
throw new Error('primary down');
};
await expect(client.tryHardRead('SELECT 1')).rejects.toThrow(
'primary down',
);
});
it('issues a single read when there is no replica', async () => {
const client = new FixtureDatabaseClient([], [{ id: 2 }], false);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([]);
expect(client.reads).toEqual(['SELECT 1']);
expect(client.preads).toEqual([]);
});
});
describe('AbstractDatabaseClient — SQL generation', () => {
it('generates an INSERT with quoted columns and placeholders', async () => {
const client = new FixtureDatabaseClient([]);
await client.insert('user', { username: 'ada', email: null });
expect(client.writes).toEqual([
{
query: 'INSERT INTO `user` (`username`, `email`) VALUES (?, ?)',
params: ['ada', null],
},
]);
});
it('quotes each dotted segment and escapes embedded backticks', () => {
const client = sqlite();
expect(client.quoteIdentifier('user')).toBe('`user`');
expect(client.quoteIdentifier('db.user.id')).toBe('`db`.`user`.`id`');
expect(client.quoteIdentifier('user.*')).toBe('`user`.*');
expect(client.quoteIdentifier('we`ird')).toBe('`we``ird`');
});
it('quotes postgres identifiers with double quotes', () => {
const client = postgres();
expect(client.quoteIdentifier('db.user')).toBe('"db"."user"');
expect(client.quoteIdentifier('we"ird')).toBe('"we""ird"');
expect(client.quoteIdentifier('user.*')).toBe('"user".*');
});
it('renders booleans the way each engine expects', () => {
expect(sqlite().booleanLiteral(true)).toBe('1');
expect(sqlite().booleanValue(false)).toBe(0);
expect(postgres().booleanLiteral(true)).toBe('TRUE');
expect(postgres().booleanLiteral(false)).toBe('FALSE');
expect(postgres().booleanValue(true)).toBe(true);
});
it('picks the engine-specific ignore-conflict syntax', () => {
expect(sqlite().insertIgnoreInto('kv')).toBe(
'INSERT OR IGNORE INTO `kv`',
);
expect(sqlite().insertIgnoreSuffix()).toBe('');
expect(postgres().insertIgnoreInto('kv')).toBe('INSERT INTO "kv"');
expect(postgres().insertIgnoreSuffix()).toBe(' ON CONFLICT DO NOTHING');
expect(mysql().insertIgnoreInto('kv')).toBe('INSERT IGNORE INTO `kv`');
expect(mysql().insertIgnoreSuffix()).toBe('');
});
it('builds an upsert clause per engine', () => {
expect(mysql().upsertClause(['user_id'], ['value', 'dt'])).toBe(
'ON DUPLICATE KEY UPDATE `value` = ?, `dt` = ?',
);
expect(sqlite().upsertClause(['user_id', 'app'], ['value'])).toBe(
'ON CONFLICT(`user_id`, `app`) DO UPDATE SET `value` = ?',
);
expect(postgres().upsertClause(['user_id'], ['value'])).toBe(
'ON CONFLICT("user_id") DO UPDATE SET "value" = ?',
);
});
it('rejects an upsert with nothing to update', () => {
expect(() => sqlite().upsertClause(['user_id'], [])).toThrow(
'upsertClause requires at least one update column',
);
});
it('extracts JSON text using each engine dialect', () => {
expect(sqlite().jsonTextExtract('`metadata`', ['a', 'b'])).toBe(
"json_extract(`metadata`, '$.a.b')",
);
expect(mysql().jsonTextExtract('`metadata`', ['a'])).toBe(
"JSON_UNQUOTE(JSON_EXTRACT(`metadata`, '$.a'))",
);
expect(postgres().jsonTextExtract('"metadata"', ['a', 'b'])).toBe(
`"metadata" #>> ARRAY['a', 'b']`,
);
});
it('escapes quotes inside JSON path segments', () => {
expect(sqlite().jsonTextExtract('`m`', ["it's"])).toBe(
"json_extract(`m`, '$.it''s')",
);
});
it('coalesces expressions and rejects an empty list', () => {
expect(sqlite().nullCoalesce('`a`', '`b`', '0')).toBe(
'COALESCE(`a`, `b`, 0)',
);
expect(() => sqlite().nullCoalesce()).toThrow(
'nullCoalesce requires at least one expression',
);
});
it('only postgres needs a RETURNING clause to learn the insert id', () => {
expect(postgres().returningIdClause()).toBe(' RETURNING id');
expect(sqlite().returningIdClause()).toBe('');
expect(mysql().returningIdClause()).toBe('');
});
it('stamps a timestamp from unix seconds in each dialect', () => {
expect(sqlite().timestampFromUnixSql()).toBe(
"datetime(?, 'unixepoch')",
);
expect(mysql().timestampFromUnixSql()).toBe('FROM_UNIXTIME(?)');
expect(postgres().timestampFromUnixSql()).toBe('to_timestamp(?)');
});
it('reads a timestamp column back as unix seconds in each engine', () => {
expect(sqlite().unixSecondsSql('`t`')).toBe(
"CAST(strftime('%s', `t`) AS INTEGER)",
);
expect(mysql().unixSecondsSql('`t`')).toBe('UNIX_TIMESTAMP(`t`)');
expect(postgres().unixSecondsSql('`t`')).toBe(
'EXTRACT(EPOCH FROM `t`)::bigint',
);
});
it('reads the database clock as epoch milliseconds in each dialect', () => {
expect(sqlite().nowMsExpr()).toBe(
"CAST((julianday('now') - 2440587.5) * 86400000 AS INTEGER)",
);
expect(mysql().nowMsExpr()).toBe(
"(TIMESTAMPDIFF(MICROSECOND, '1970-01-01 00:00:00', UTC_TIMESTAMP(6)) DIV 1000)",
);
expect(postgres().nowMsExpr()).toBe(
'CAST(EXTRACT(EPOCH FROM clock_timestamp()) * 1000 AS BIGINT)',
);
});
it('describes a stored timestamp the way each driver reads it back', () => {
const seconds = Date.UTC(2026, 9, 11, 8, 5, 9) / 1000;
expect(sqlite().timestampValue(seconds)).toBe('2026-10-11 08:05:09');
expect(mysql().timestampValue(seconds)).toEqual(
new Date(seconds * 1000),
);
expect(postgres().timestampValue(seconds)).toEqual(
new Date(seconds * 1000),
);
});
it('falls back to `otherwise` for an engine with no explicit choice', () => {
const client = new FixtureDatabaseClient([]);
expect(
client.case({ sqlite: 'a', mysql: 'b', otherwise: 'fallback' }),
).toBe('fallback');
expect(client.case({ fixture: 'exact', otherwise: 'fallback' })).toBe(
'exact',
);
});
});
it('builds an insert for batchWrite and shares it with insert', async () => {
const client = new FixtureDatabaseClient([]);
const entry = client.insertEntry('widget', { name: "reader's", enabled: true });
expect(entry.values).toEqual(["reader's", true]);
expect(entry.statement).toContain('VALUES (?, ?)');
await client.insert('widget', { name: "reader's", enabled: true });
expect(client.writes).toEqual([{ query: entry.statement, params: entry.values }]);
});
+248 -44
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,13 +17,23 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import type { IConfig } from '../../types';
import { Span } from '../../util/span.js';
import { PuterClient } from '../types';
import { compareMigrationFilenames } from './migrationFilenames.js';
export interface WriteResult {
insertId: number | bigint;
affectedRows: number;
anyRowsAffected: boolean;
/**
* Warnings the statement raised; nonzero means a value may have been stored
* differently from how it was sent. Absent when the engine can't vouch that
* a clean insert stored exactly what was sent, on a fresh id.
*/
warningStatus?: number;
}
export interface BatchEntry {
@@ -33,12 +43,18 @@ export interface BatchEntry {
type SqlJsonPath = readonly [string, ...string[]];
/** The slice of a pooled connection that migrations need. */
export interface MigrationConnection {
query(sql: string): Promise<unknown>;
release(): void;
}
/**
* Base database client. Subclasses must override every method that throws here.
*
* Do not instantiate directly — use the factory exported from
* `clients/database/index.ts` which picks the right implementation
* based on `config.database.engine`.
* `clients/database/index.ts` which picks the right implementation based on
* `config.database.engine`.
*/
export class AbstractDatabaseClient extends PuterClient {
/** Short name used by `case()` to pick engine-specific values. */
@@ -52,9 +68,7 @@ export class AbstractDatabaseClient extends PuterClient {
// Abstract interface — subclasses MUST override
// ------------------------------------------------------------------
/**
* Execute a read query. Returns an array of row objects.
*/
/** Execute a read query. Returns an array of row objects. */
async read(
_query: string,
_params: unknown[] = [],
@@ -63,9 +77,9 @@ export class AbstractDatabaseClient extends PuterClient {
}
/**
* Read that prefers the primary database (useful when read-replicas
* may have replication lag). In single-node setups this is identical
* to `read()`.
* Read that prefers the primary database (useful when read-replicas may
* have replication lag). In single-node setups this is identical to
* `read()`.
*/
async pread(
_query: string,
@@ -74,16 +88,12 @@ export class AbstractDatabaseClient extends PuterClient {
throw new Error('DatabaseClient.pread() not implemented');
}
/**
* Execute a write query (INSERT / UPDATE / DELETE).
*/
/** Execute a write query (INSERT / UPDATE / DELETE). */
async write(_query: string, _params: unknown[] = []): Promise<WriteResult> {
throw new Error('DatabaseClient.write() not implemented');
}
/**
* Execute multiple write statements in a single transaction.
*/
/** Execute multiple write statements in a single transaction. */
async batchWrite(_entries: BatchEntry[]): Promise<void> {
throw new Error('DatabaseClient.batchWrite() not implemented');
}
@@ -100,60 +110,166 @@ export class AbstractDatabaseClient extends PuterClient {
tableName: string,
data: Record<string, unknown>,
): Promise<WriteResult> {
const entry = this.insertEntry(tableName, data);
return this.write(
entry.statement + this.returningIdClause(),
entry.values,
);
}
/** An insert that can join other writes in a transaction. */
insertEntry(tableName: string, data: Record<string, unknown>): BatchEntry {
const cols = Object.keys(data);
const values = Object.values(data);
const sql =
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
return {
statement:
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`,
values: Object.values(data),
};
}
/** Whether `read()` goes to a separate replica rather than the primary. */
protected hasReadReplica(): boolean {
return false;
}
/**
* Like `read()` but falls back to the primary when read-replicas are
* in use. Subclasses may override with replica-aware logic; the
* default delegates to `pread()`.
* The replica leg of `tryHardRead`. Override when `read()` falls back to
* the primary by itself, or a replica failure reads the primary twice.
*/
protected replicaRead(
query: string,
params: unknown[],
): Promise<Record<string, unknown>[]> {
return this.read(query, params);
}
/**
* Like `read()`, but an empty or failed replica read falls back to the
* primary, so replication lag can't hide a row that was just written. By
* default the primary read starts alongside the replica one; `race: false`
* reads the primary only after the replica returns empty or fails. Without
* a replica this is a single `read()`.
*/
@Span('db.tryHardRead', (query: string) => ({ 'db.statement': query }))
async tryHardRead(
query: string,
params: unknown[] = [],
{ race = true }: { race?: boolean } = {},
): Promise<Record<string, unknown>[]> {
const primary = this.pread(query, params);
primary.catch(() => {});
if (!this.hasReadReplica()) return this.read(query, params);
const primary = race ? settle(this.pread(query, params)) : null;
try {
const rows = await this.read(query, params);
if (rows.length > 0) {
return rows;
}
const rows = await this.replicaRead(query, params);
if (rows.length > 0) return rows;
} catch {
// replica failed — fall through to primary
}
return primary;
const result = await (primary ?? settle(this.pread(query, params)));
if ('error' in result) throw result.error;
return result.value;
}
/**
* Like `tryHardRead()` but throws when the result set is empty.
* Apply the `.sql` files named `<engineName>*` in each configured migration
* directory, in filename order. Files must be idempotent: nothing records
* which ones already ran. A failure aborts startup. `transactional` wraps
* each file in BEGIN/COMMIT, for engines whose DDL can roll back.
*/
async requireRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const rows = await this.tryHardRead(query, params);
if (rows.length === 0) {
throw new Error(`required read returned no rows: ${query}`);
protected async applyMigrationPaths(
connect: () => Promise<MigrationConnection>,
splitStatements: (contents: string) => string[],
{ transactional }: { transactional: boolean },
): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const tag = `[${this.engineName}]`;
const conn = await connect();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) =>
f.endsWith('.sql') &&
f.startsWith(this.engineName),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`${tag} migration path is unreadable: ${dir}`,
{
cause: e,
},
);
}
if (files.length === 0) {
console.log(`${tag} no migrations in ${dir}`);
continue;
}
console.log(
`${tag} running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const contents = readFileSync(
resolvePath(dir, file),
'utf8',
);
const statements = splitStatements(contents);
if (transactional) await conn.query('BEGIN');
try {
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`${tag} failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
if (transactional) await conn.query('COMMIT');
} catch (e) {
if (transactional) {
try {
await conn.query('ROLLBACK');
} catch {
// the original failure is the one worth surfacing
}
}
throw e;
}
console.log(
`${tag} applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
return rows;
}
/**
* Return the value from `choices` that matches the current engine.
*
* Usage:
* ```
* db.case({ sqlite: "datetime('now')", mysql: 'NOW()', otherwise: 'NOW()' })
* ```
*
* db.case({
* sqlite: "datetime('now')",
* mysql: 'NOW()',
* otherwise: 'NOW()',
* });
*
* If the engine name isn't present in `choices`, falls back to
* `choices.otherwise`.
@@ -231,6 +347,30 @@ export class AbstractDatabaseClient extends PuterClient {
});
}
/**
* Whole `periodSeconds` buckets since the epoch, from a unix-seconds
* column. MySQL's `/` yields a decimal, which would split a bucket.
*/
periodBucket(secondsExpression: string, periodSeconds: number): string {
return this.case({
mysql: `(${secondsExpression} DIV ${periodSeconds})`,
otherwise: `(${secondsExpression} / ${periodSeconds})`,
});
}
/**
* Counts each (`userExpression`, bucket) pair once. `COUNT(DISTINCT a, b)`
* is MySQL-only, so the pair folds into one number — injective while the
* bucket stays under the multiplier, which hourly reaches in the 2980s.
*/
countDistinctUserPeriods(
userExpression: string,
secondsExpression: string,
periodSeconds: number,
): string {
return `COUNT(DISTINCT ${userExpression} * 10000000 + ${this.periodBucket(secondsExpression, periodSeconds)})`;
}
nullCoalesce(...expressions: readonly string[]): string {
if (expressions.length === 0) {
throw new Error('nullCoalesce requires at least one expression');
@@ -245,7 +385,71 @@ export class AbstractDatabaseClient extends PuterClient {
});
}
/**
* A timestamp column value from one bound parameter in unix seconds, stored
* the way `CURRENT_TIMESTAMP` would store that instant.
*/
timestampFromUnixSql(): string {
return this.case({
sqlite: "datetime(?, 'unixepoch')",
postgres: 'to_timestamp(?)',
otherwise: 'FROM_UNIXTIME(?)',
});
}
/**
* A timestamp column as unix seconds, computed by the database: the mysql
* driver reads a stored UTC timestamp as local time, so parsing it in JS
* can be off by the server's UTC offset.
*/
unixSecondsSql(column: string): string {
return this.case({
postgres: `EXTRACT(EPOCH FROM ${column})::bigint`,
mysql: `UNIX_TIMESTAMP(${column})`,
otherwise: `CAST(strftime('%s', ${column}) AS INTEGER)`,
});
}
/**
* The database's own clock as epoch milliseconds, so nodes whose clocks
* disagree still agree on it. The mysql form ignores the session time zone,
* so a DST change can't shift it.
*/
nowMsExpr(): string {
return this.case({
sqlite: "CAST((julianday('now') - 2440587.5) * 86400000 AS INTEGER)",
postgres:
'CAST(EXTRACT(EPOCH FROM clock_timestamp()) * 1000 AS BIGINT)',
otherwise:
"(TIMESTAMPDIFF(MICROSECOND, '1970-01-01 00:00:00', UTC_TIMESTAMP(6)) DIV 1000)",
});
}
/**
* What a read of a timestamp column holding `seconds` returns: sqlite gives
* back the UTC text it stored, mysql2 and pg a `Date` read in node's time
* zone (so this matches when node and the database session share one).
*/
timestampValue(seconds: number): Date | string {
const date = new Date(seconds * 1000);
return this.case<Date | string>({
sqlite: date.toISOString().slice(0, 19).replace('T', ' '),
otherwise: date,
});
}
protected sqlStringLiteral(value: string): string {
return `'${value.replaceAll("'", "''")}'`;
}
}
/** Await later without an unhandled rejection in the meantime. */
const settle = async <T>(
promise: Promise<T>,
): Promise<{ value: T } | { error: unknown }> => {
try {
return { value: await promise };
} catch (error) {
return { error };
}
};
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,8 +17,94 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { compareMigrationFilenames } from './MySQLDatabaseClient.js';
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import {
MySQLDatabaseClient,
compareMigrationFilenames,
} from './MySQLDatabaseClient.js';
interface FakeConnection {
query: ReturnType<typeof vi.fn>;
execute: ReturnType<typeof vi.fn>;
beginTransaction: ReturnType<typeof vi.fn>;
commit: ReturnType<typeof vi.fn>;
rollback: ReturnType<typeof vi.fn>;
release: ReturnType<typeof vi.fn>;
}
interface FakePool {
poolConfig: Record<string, unknown>;
connectionHandlers: ((conn: unknown) => void)[];
connections: FakeConnection[];
calls: { sql: string; values?: unknown[] }[];
ended: boolean;
endError: unknown;
endThrows: unknown;
respond: (sql: string, values?: unknown[]) => unknown;
on: (event: string, handler: (conn: unknown) => void) => FakePool;
promise: () => { getConnection: () => Promise<FakeConnection> };
end: (cb: (err?: unknown) => void) => void;
}
// mysql2 is the wire boundary: stub the driver's pool so we can assert on
// the exact SQL, parameters and connection choreography the adapter emits.
const { createPoolMock, createdPools } = vi.hoisted(() => {
const createdPools: FakePool[] = [];
const createPoolMock = vi.fn((poolConfig: Record<string, unknown>) => {
const pool: FakePool = {
poolConfig,
connectionHandlers: [],
connections: [],
calls: [],
ended: false,
endError: null,
endThrows: null,
respond: () => [[], undefined],
on(event, handler) {
if (event === 'connection') {
pool.connectionHandlers.push(handler);
}
return pool;
},
promise() {
return {
getConnection: async () => {
const run = async (sql: string, values?: unknown[]) => {
pool.calls.push({ sql, values });
return pool.respond(sql, values);
};
const conn: FakeConnection = {
query: vi.fn(run),
execute: vi.fn(run),
beginTransaction: vi.fn(async () => {}),
commit: vi.fn(async () => {}),
rollback: vi.fn(async () => {}),
release: vi.fn(),
};
pool.connections.push(conn);
return conn;
},
};
},
end(cb) {
if (pool.endThrows) throw pool.endThrows;
pool.ended = true;
cb(pool.endError ?? undefined);
},
};
createdPools.push(pool);
return pool;
});
return { createPoolMock, createdPools };
});
vi.mock('mysql2', () => ({ createPool: createPoolMock }));
describe('compareMigrationFilenames', () => {
it('orders numbered migrations numerically, not lexically', () => {
@@ -93,3 +179,694 @@ describe('compareMigrationFilenames', () => {
expect([...sorted].sort(compareMigrationFilenames)).toEqual(sorted);
});
});
// ── Replica read failover ───────────────────────────────────────────
//
// `read()` normally goes to the replica batcher; when the replica side is
// degraded (batcher load-shed or a transient connection error) and a real
// replica is configured, the read retries once on the primary read batcher.
// The write batcher is left unset, so a failover that reached it would throw.
type Batcher = { execute: ReturnType<typeof vi.fn> };
const makeClient = (opts: {
replica: Batcher;
primary: Batcher;
multiNode?: boolean;
}) => {
const client = new MySQLDatabaseClient({
database: { engine: 'mysql' },
} as IConfig);
// Bypass onServerStart (which would connect to a real database) and
// inject the batchers directly. Configuration enum: SINGLE=0, REPLICA=1.
Object.assign(client as unknown as Record<string, unknown>, {
dbReplica: opts.replica,
dbPrimaryRead: opts.primary,
configuration: opts.multiNode === false ? 0 : 1,
});
return client;
};
const codedError = (code: string) => {
const err = new Error(code) as Error & { code: string };
err.code = code;
return err;
};
describe('MySQLDatabaseClient.read — replica failover', () => {
it('fails over to the primary on batcher load-shed errors', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
};
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
const client = makeClient({ replica, primary });
await expect(client.read('SELECT 1')).resolves.toEqual([{ ok: 1 }]);
expect(primary.execute).toHaveBeenCalledTimes(1);
});
it('fails over on transient connection errors', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('ECONNRESET')),
};
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
const client = makeClient({ replica, primary });
await expect(client.read('SELECT 1')).resolves.toEqual([{ ok: 1 }]);
});
it('rethrows deterministic SQL errors without touching the primary', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('ER_PARSE_ERROR')),
};
const primary = { execute: vi.fn() };
const client = makeClient({ replica, primary });
await expect(client.read('SELEC oops')).rejects.toMatchObject({
code: 'ER_PARSE_ERROR',
});
expect(primary.execute).not.toHaveBeenCalled();
});
it('does not fail over in single-node configuration', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
};
const primary = { execute: vi.fn() };
const client = makeClient({ replica, primary, multiNode: false });
await expect(client.read('SELECT 1')).rejects.toMatchObject({
code: 'dbBatchFailed',
});
expect(primary.execute).not.toHaveBeenCalled();
});
});
// -- Driver-level behaviour (stubbed mysql2 pool) ---------------------
const mysqlConfig = (
database: Partial<NonNullable<IConfig['database']>> = {},
): IConfig =>
({
port: 0,
extensions: [],
database: { engine: 'mysql', ...database },
}) as IConfig;
const startClient = async (
database: Partial<NonNullable<IConfig['database']>> = {},
): Promise<MySQLDatabaseClient> => {
const client = new MySQLDatabaseClient(mysqlConfig(database));
await client.onServerStart();
return client;
};
// The batcher coalesces a batch into `stmt1;stmt2; SELECT 1`, so the driver
// must answer with one row-set per statement plus one for the SELECT 1.
const rowSets = (...sets: unknown[][]) => [[...sets, [{ 1: 1 }]], undefined];
// Writes go out one statement at a time, so the driver answers with just the
// result header.
const writeHeader = (header: Record<string, unknown>) => [header, undefined];
beforeEach(() => {
createdPools.length = 0;
createPoolMock.mockClear();
});
describe('MySQLDatabaseClient — pool construction', () => {
it('builds the primary pool from config with multi-statement support', async () => {
await startClient({
host: 'db.internal',
port: 3307,
user: 'puter',
password: 'hunter2',
database: 'puterdb',
});
expect(createPoolMock).toHaveBeenCalledTimes(1);
expect(createdPools[0].poolConfig).toEqual({
maxPreparedStatements: 900,
connectionLimit: 30,
maxIdle: 29,
idleTimeout: 60_000,
gracefulEnd: true,
enableKeepAlive: true,
host: 'db.internal',
port: 3307,
user: 'puter',
password: 'hunter2',
database: 'puterdb',
multipleStatements: true,
});
});
// mysql2 skips its idle reaper unless maxIdle < connectionLimit, and an
// unreaped connection is closed by the server under the pool's feet.
it('keeps the idle reaper on when a pool sets its own connection limit', async () => {
await startClient({
replica: { host: 'replica.internal', connectionLimit: 10 },
});
expect(createdPools[1].poolConfig).toMatchObject({
connectionLimit: 10,
maxIdle: 9,
idleTimeout: 60_000,
});
});
// mysql2 reads 0 as unlimited; a maxIdle of -1 would crash its reaper.
it('leaves maxIdle to mysql2 for an unlimited pool', async () => {
await startClient({
replica: { host: 'replica.internal', connectionLimit: 0 },
});
expect(createdPools[1].poolConfig).toMatchObject({
connectionLimit: 0,
});
expect(createdPools[1].poolConfig).not.toHaveProperty('maxIdle');
});
it('falls back to loopback defaults when the endpoint is unspecified', async () => {
await startClient();
expect(createdPools[0].poolConfig).toMatchObject({
host: '127.0.0.1',
port: 3306,
user: 'root',
password: '',
database: 'puter',
});
});
it('arms a server-side statement timeout on every new connection', async () => {
await startClient({ selectTimeoutMs: 12_000 });
const pool = createdPools[0];
expect(pool.connectionHandlers).toHaveLength(1);
const conn = { query: vi.fn() };
pool.connectionHandlers[0](conn);
expect(conn.query).toHaveBeenCalledWith(
'SET SESSION max_execution_time = 12000',
);
});
it('omits the statement timeout when configured to 0', async () => {
await startClient({ selectTimeoutMs: 0 });
expect(createdPools[0].connectionHandlers).toHaveLength(0);
});
it('shares one pool between reads and writes without a replica', async () => {
await startClient();
expect(createPoolMock).toHaveBeenCalledTimes(1);
});
it('creates a second pool when a read-replica is configured', async () => {
await startClient({
replica: { host: 'replica.internal', port: 3306 },
});
expect(createPoolMock).toHaveBeenCalledTimes(2);
expect(createdPools[1].poolConfig).toMatchObject({
host: 'replica.internal',
multipleStatements: true,
});
});
});
describe('MySQLDatabaseClient — query interface', () => {
it('sends the query and its parameters through to the driver', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => rowSets([{ id: 1, username: 'ada' }]);
await expect(
client.read('SELECT * FROM `user` WHERE `id` = ?', [1]),
).resolves.toEqual([{ id: 1, username: 'ada' }]);
expect(pool.calls).toEqual([
{
sql: 'SELECT * FROM `user` WHERE `id` = ?; SELECT 1',
values: [1],
},
]);
});
it('returns an empty array when the driver yields no row-set', async () => {
const client = await startClient();
createdPools[0].respond = () => [[], undefined];
await expect(client.read('SELECT 1')).resolves.toEqual([]);
await expect(client.pread('SELECT 1')).resolves.toEqual([]);
});
it('reads from the replica pool and primary-reads from the primary', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => rowSets([{ from: 'replica' }]);
await expect(client.read('SELECT 1')).resolves.toEqual([
{ from: 'replica' },
]);
await expect(client.pread('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
});
it.each([
{ method: 'pread', poolIndex: 0, maxTimeInQueue: 30 },
{ method: 'read', poolIndex: 1, maxTimeInQueue: 10 },
] as const)(
'$method: flushes lone reads at once until 8 are in flight, then waits $maxTimeInQueue ms and coalesces',
async ({ method, poolIndex, maxTimeInQueue }) => {
const client = await startClient({ replica: { host: 'replica' } });
const pool = createdPools[poolIndex];
const held: Array<() => void> = [];
pool.respond = (sql) =>
new Promise((resolve) => {
const sets = sql
.split(';')
.slice(0, -1)
.map((stmt) => [{ stmt }]);
held.push(() => resolve(rowSets(...sets)));
});
vi.useFakeTimers({
toFake: [
'setTimeout',
'clearTimeout',
'setImmediate',
'clearImmediate',
],
});
try {
const reads: Promise<unknown>[] = [];
for (let i = 0; i < 8; i++) {
reads.push(client[method](`SELECT ${i}`));
await vi.advanceTimersByTimeAsync(0);
}
expect(pool.calls).toHaveLength(8);
reads.push(client[method]('SELECT 8'));
await vi.advanceTimersByTimeAsync(maxTimeInQueue - 1);
reads.push(client[method]('SELECT 9'));
expect(pool.calls).toHaveLength(8);
await vi.advanceTimersByTimeAsync(1);
expect(pool.calls.map((c) => c.sql).at(-1)).toBe(
'SELECT 8;SELECT 9; SELECT 1',
);
for (const release of held.splice(0)) release();
const rows = await Promise.all(reads);
expect(rows.at(-2)).toEqual([{ stmt: 'SELECT 8' }]);
expect(rows.at(-1)).toEqual([{ stmt: 'SELECT 9' }]);
} finally {
vi.useRealTimers();
}
},
);
it.each([
{ setup: 'single node', replica: undefined, waits: true },
{ setup: 'replica', replica: { host: 'replica' }, waits: false },
])(
'$setup: 8 primary reads in flight hold back a lone read: $waits',
async ({ replica, waits }) => {
const client = await startClient(replica ? { replica } : {});
const held: Array<() => void> = [];
for (const pool of createdPools) {
pool.respond = (sql) =>
new Promise((resolve) => {
const sets = sql
.split(';')
.slice(0, -1)
.map((stmt) => [{ stmt }]);
held.push(() => resolve(rowSets(...sets)));
});
}
const sent = () => createdPools.flatMap((pool) => pool.calls);
vi.useFakeTimers({
toFake: [
'setTimeout',
'clearTimeout',
'setImmediate',
'clearImmediate',
],
});
try {
const reads: Promise<unknown>[] = [];
for (let i = 0; i < 8; i++) {
reads.push(client.pread(`SELECT ${i}`));
await vi.advanceTimersByTimeAsync(0);
}
reads.push(client.read('SELECT 8'));
await vi.advanceTimersByTimeAsync(0);
expect(sent()).toHaveLength(waits ? 8 : 9);
await vi.advanceTimersByTimeAsync(10);
expect(sent()).toHaveLength(9);
for (const release of held.splice(0)) release();
await Promise.all(reads);
} finally {
vi.useRealTimers();
}
},
);
it('sends a write on its own, outside a transaction', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => writeHeader({ affectedRows: 1 });
await client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
'ada',
1,
]);
expect(pool.calls).toEqual([
{
sql: 'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]);
expect(pool.connections[0].beginTransaction).not.toHaveBeenCalled();
expect(pool.connections[0].commit).not.toHaveBeenCalled();
});
it('maps the mysql result header onto the shared write result', async () => {
const client = await startClient();
createdPools[0].respond = () =>
writeHeader({ insertId: 42, affectedRows: 2 });
await expect(
client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
'ada',
1,
]),
).resolves.toEqual({
insertId: 42,
affectedRows: 2,
anyRowsAffected: true,
});
});
it('passes the warning count through, so callers can tell a clean write', async () => {
const client = await startClient();
createdPools[0].respond = () =>
writeHeader({ insertId: 9, affectedRows: 1, warningStatus: 1 });
await expect(
client.write('INSERT IGNORE INTO `user` (`username`) VALUES (?)', [
'ada',
]),
).resolves.toEqual({
insertId: 9,
affectedRows: 1,
anyRowsAffected: true,
warningStatus: 1,
});
});
it('reports no rows affected when the header omits the counters', async () => {
const client = await startClient();
createdPools[0].respond = () => writeHeader({});
await expect(client.write('DELETE FROM `user`')).resolves.toEqual({
insertId: 0,
affectedRows: 0,
anyRowsAffected: false,
});
});
it('builds an INSERT with quoted identifiers and positional params', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => writeHeader({ insertId: 7, affectedRows: 1 });
await expect(
client.insert('user', { username: 'ada', email: null }),
).resolves.toMatchObject({ insertId: 7 });
expect(pool.calls[0]).toEqual({
sql: 'INSERT INTO `user` (`username`, `email`) VALUES (?, ?)',
values: ['ada', null],
});
});
});
describe('MySQLDatabaseClient — batchWrite transactions', () => {
it('runs every statement on one connection inside a transaction', async () => {
const client = await startClient();
const pool = createdPools[0];
await client.batchWrite([
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
{
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
values: ['s'],
},
]);
expect(pool.connections).toHaveLength(1);
const conn = pool.connections[0];
expect(conn.beginTransaction).toHaveBeenCalledTimes(1);
expect(conn.execute.mock.calls).toEqual([
['UPDATE `user` SET `x` = ?', [1]],
['DELETE FROM `sessions` WHERE `uuid` = ?', ['s']],
]);
expect(conn.commit).toHaveBeenCalledTimes(1);
expect(conn.rollback).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('rolls back and releases the connection when a statement fails', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = (sql) => {
if (sql.startsWith('DELETE')) throw new Error('constraint blew up');
return [[], undefined];
};
await expect(
client.batchWrite([
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
{ statement: 'DELETE FROM `user`', values: [] },
]),
).rejects.toThrow('constraint blew up');
const conn = pool.connections[0];
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.rollback).toHaveBeenCalledTimes(1);
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('still rethrows the original failure when the rollback also fails', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => {
throw new Error('statement blew up');
};
const original = client.batchWrite([
{ statement: 'UPDATE `user` SET `x` = ?', values: [1] },
]);
// The rollback is attached after the connection is handed out, so
// patch it once the adapter has acquired one.
await Promise.resolve();
pool.connections[0]?.rollback.mockRejectedValue(
new Error('rollback blew up'),
);
await expect(original).rejects.toThrow('statement blew up');
expect(pool.connections[0].release).toHaveBeenCalledTimes(1);
});
it('never acquires a connection for an empty batch', async () => {
const client = await startClient();
await client.batchWrite([]);
expect(createdPools[0].connections).toHaveLength(0);
});
});
describe('MySQLDatabaseClient — tryHardRead', () => {
it('issues exactly one query in single-node configuration', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => rowSets([{ id: 1 }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ id: 1 },
]);
expect(pool.calls).toHaveLength(1);
});
it('prefers replica rows when the replica has them', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => rowSets([{ from: 'replica' }]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'replica' },
]);
});
it('falls back to the primary when the replica is behind', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => rowSets([]);
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
});
it('falls back to the primary when the replica query throws', async () => {
const client = await startClient({ replica: { host: 'replica' } });
const [primary, replica] = createdPools;
primary.respond = () => rowSets([{ from: 'primary' }]);
replica.respond = () => {
throw new Error('replica down');
};
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
});
it('reads the primary once when the replica fails in a way read() fails over on', async () => {
const replica = {
execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')),
};
const primary = {
execute: vi.fn().mockResolvedValue([[{ from: 'primary' }]]),
};
const client = makeClient({ replica, primary });
await expect(client.tryHardRead('SELECT 1')).resolves.toEqual([
{ from: 'primary' },
]);
expect(primary.execute).toHaveBeenCalledTimes(1);
});
});
describe('MySQLDatabaseClient — migrations', () => {
let dir: string;
beforeEach(() => {
dir = mkdtempSync(join(tmpdir(), 'puter-mysql-mig-'));
});
afterEach(() => {
rmSync(dir, { recursive: true, force: true });
});
it('applies mysql migrations in numeric order, one statement at a time', async () => {
writeFileSync(
join(dir, 'mysql_mig_2.sql'),
'CREATE TABLE b (id INT);\nCREATE TABLE c (id INT);\n',
);
writeFileSync(
join(dir, 'mysql_mig_10.sql'),
'CREATE TABLE d (id INT);',
);
// Other engines' files and non-SQL noise must be ignored.
writeFileSync(join(dir, 'postgres_mig_1.sql'), 'CREATE TABLE nope ();');
writeFileSync(join(dir, 'mysql_notes.txt'), 'not sql');
await startClient({ migrationPaths: [dir] });
const pool = createdPools[0];
expect(pool.calls.map((c) => c.sql)).toEqual([
'CREATE TABLE b (id INT)',
'CREATE TABLE c (id INT)',
'CREATE TABLE d (id INT)',
]);
expect(pool.connections[0].release).toHaveBeenCalledTimes(1);
});
it('reports which file and statement index failed', async () => {
writeFileSync(
join(dir, 'mysql_mig_1.sql'),
'CREATE TABLE ok (id INT);\nCREATE TABLE bad (id INT);',
);
const client = new MySQLDatabaseClient(
mysqlConfig({ migrationPaths: [dir] }),
);
const cause = new Error('ER_PARSE_ERROR');
// The pool only exists once onServerStart runs, so arm the failure
// through the factory.
createPoolMock.mockImplementationOnce((poolConfig) => {
const pool = createPoolMock.getMockImplementation()!(poolConfig);
pool.respond = (sql: string) => {
if (sql.includes('bad')) throw cause;
return [[], undefined];
};
return pool;
});
await expect(client.onServerStart()).rejects.toThrow(
'[mysql] failed to apply mysql_mig_1.sql at statement 1',
);
});
it('fails loudly when a configured migration path is unreadable', async () => {
await expect(
startClient({ migrationPaths: [join(dir, 'does-not-exist')] }),
).rejects.toThrow('[mysql] migration path is unreadable');
});
it('skips a configured directory that holds no mysql migrations', async () => {
writeFileSync(join(dir, 'readme.md'), '# nothing here');
await startClient({ migrationPaths: [dir] });
expect(createdPools[0].calls).toEqual([]);
});
});
describe('MySQLDatabaseClient — pool lifecycle', () => {
it('closes both pools on shutdown when a replica is configured', async () => {
const client = await startClient({ replica: { host: 'replica' } });
await client.onServerShutdown();
expect(createdPools[0].ended).toBe(true);
expect(createdPools[1].ended).toBe(true);
});
it('leaves the pools open through prepare-shutdown; only onServerShutdown closes them', async () => {
vi.useFakeTimers();
try {
const client = await startClient();
await client.onServerPrepareShutdown();
expect(createdPools[0].ended).toBe(false);
// No timer closes the pools on its own — the layers above are
// still draining through them for as long as the process drains.
await vi.advanceTimersByTimeAsync(60_000);
expect(createdPools[0].ended).toBe(false);
await client.onServerShutdown();
expect(createdPools[0].ended).toBe(true);
} finally {
vi.useRealTimers();
}
});
it('surfaces a driver error raised while closing the pool', async () => {
const client = await startClient();
createdPools[0].endError = new Error('pool refused to close');
await expect(client.onServerShutdown()).rejects.toThrow(
'pool refused to close',
);
});
it('surfaces a driver error thrown synchronously by end()', async () => {
const client = await startClient();
createdPools[0].endThrows = new Error('end exploded');
await expect(client.onServerShutdown()).rejects.toThrow('end exploded');
});
});
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,35 +17,27 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { createPool, type Pool } from 'mysql2';
import { metrics } from '@opentelemetry/api';
import { createPool, ExecuteValues, type Pool } from 'mysql2';
import { Span } from '../../util/span.js';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { SQLBatcher } from './SQLBatcher.js';
import { isRetriableError } from './retriableErrors.js';
import { splitMysqlStatements } from './splitMysqlStatements.js';
import { compareMigrationFilenames } from './migrationFilenames.js';
import type { IConfig } from '../../types';
const RETRIABLE_ERROR_CODES = new Set([
'PROTOCOL_CONNECTION_LOST',
'PROTOCOL_SEQUENCE_TIMEOUT',
'PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR',
'ECONNRESET',
'ETIMEDOUT',
'EPIPE',
'ECONNREFUSED',
'EHOSTUNREACH',
'ENETUNREACH',
'EAI_AGAIN',
]);
const DEFAULT_SELECT_TIMEOUT_MS = 30_000;
const DEFAULT_CONNECTION_LIMIT = 30;
const IDLE_TIMEOUT_MS = 60_000;
const RETRIABLE_ERROR_MESSAGES = [
'Connection lost',
'read ECONNRESET',
'ETIMEDOUT',
];
const replicaFailoverCounter = metrics
.getMeter('puter-backend')
.createCounter('db.read.replica_failover', {
description:
'Reads that failed on the replica batcher and were retried on the primary',
});
export { compareMigrationFilenames };
export { compareMigrationFilenames } from './migrationFilenames.js';
type PoolConfig = Parameters<typeof createPool>[0];
@@ -61,9 +53,10 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
private replicaPool!: Pool;
private db!: SQLBatcher;
private dbReplica!: SQLBatcher;
/** Primary pool, SELECT-only: same rows as `db`, coalesced like the replica. */
private dbPrimaryRead!: SQLBatcher;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(config: IConfig) {
super(config);
@@ -85,7 +78,13 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
});
console.log('[mysql] connected to primary');
this.db = new SQLBatcher(this.primaryPool, 30, 5);
// Read batchers on one pool share the cap on immediate flushes.
const primaryReadFlushes = { count: 0 };
this.db = this.createPrimaryBatcher(this.primaryPool);
this.dbPrimaryRead = this.createPrimaryReadBatcher(
this.primaryPool,
primaryReadFlushes,
);
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
@@ -96,38 +95,30 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
this.configuration = Configuration.SINGLE;
}
this.dbReplica = new SQLBatcher(this.replicaPool, 10, 5);
this.dbReplica = this.createReplicaBatcher(
this.replicaPool,
this.configuration === Configuration.SINGLE
? primaryReadFlushes
: { count: 0 },
);
await this.runMigrations();
await this.applyMigrationPaths(
() => this.primaryPool.promise().getConnection(),
splitMysqlStatements,
{ transactional: false },
);
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
// Allow in-flight queries to drain before closing pools
const drainMs = 60_000;
console.log(
`[mysql] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools('drain').catch((e) =>
console.error('[mysql] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
// Pools stay open; onServerShutdown closes them after the layers
// above have drained.
console.log('[mysql] entering drain mode');
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools('shutdown');
}
@@ -135,24 +126,46 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// Query interface
// ------------------------------------------------------------------
// The db.* spans measure the logical query, including time queued in
// the SQLBatcher — the mysql2 auto-instrumentation only sees the
// coalesced multi-statement flush, so per-query latency lives here.
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.dbReplica.execute(query, params);
let result;
try {
result = await this.dbReplica.execute(query, params);
} catch (error) {
// Replica-side degradation (batcher load-shed or a transient
// connection failure) shouldn't fail reads while the primary is
// healthy. Deterministic errors (bad SQL) are rethrown — they
// would fail identically on the primary.
if (
this.configuration !== Configuration.REPLICA ||
!MySQLDatabaseClient.isFailoverWorthy(error)
) {
throw error;
}
replicaFailoverCounter.add(1);
result = await this.dbPrimaryRead.execute(query, params);
}
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
@Span('db.pread', (query: string) => ({ 'db.statement': query }))
override async pread(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.db.execute(query, params);
const result = await this.dbPrimaryRead.execute(query, params);
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
@@ -161,15 +174,20 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
const header = result[0] as {
insertId?: number;
affectedRows?: number;
warningStatus?: number;
};
const affectedRows = header.affectedRows ?? 0;
return {
insertId: header.insertId ?? 0,
affectedRows,
anyRowsAffected: affectedRows > 0,
warningStatus: header.warningStatus,
};
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(
entries: { statement: string; values: unknown[] }[],
): Promise<void> {
@@ -183,7 +201,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
await conn.beginTransaction();
try {
for (const { statement, values } of entries) {
await conn.execute(statement, values);
await conn.execute(statement, values as ExecuteValues);
}
await conn.commit();
} catch (err) {
@@ -195,100 +213,19 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
}
}
override async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
if (this.configuration === Configuration.SINGLE) {
return this.read(query, params);
}
// Run both reads in parallel — prefer replica when it returns rows,
// otherwise fall back to primary to handle replication lag.
const primaryPromise = this.db.execute(query, params);
try {
const replicaResult = await this.dbReplica.execute(query, params);
if (
Array.isArray(replicaResult?.[0]) &&
(replicaResult[0] as unknown[]).length > 0
) {
primaryPromise.catch(() => {}); // suppress unhandled rejection
return replicaResult[0] as Record<string, unknown>[];
}
} catch {
// fall through to primary
}
const primaryResult = await primaryPromise;
return (primaryResult?.[0] as Record<string, unknown>[]) ?? [];
protected override hasReadReplica(): boolean {
return this.configuration === Configuration.REPLICA;
}
// ------------------------------------------------------------------
// Migrations
// ------------------------------------------------------------------
/**
* Apply `.sql` files from each configured migration directory in order.
* Files within a directory are sorted lexically. Files MUST be
* idempotent — there is no per-file applied-state tracking. Failures
* abort startup so operators see schema problems loud.
*/
private async runMigrations(): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const conn = await this.primaryPool.promise().getConnection();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) => f.endsWith('.sql') && f.startsWith('mysql'),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[mysql] migration path is unreadable: ${dir}`,
{ cause: e },
);
}
if (files.length === 0) {
console.log(`[mysql] no migrations in ${dir}`);
continue;
}
console.log(
`[mysql] running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const filePath = resolvePath(dir, file);
const contents = readFileSync(filePath, 'utf8');
const statements = splitMysqlStatements(contents);
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`[mysql] failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
console.log(
`[mysql] applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
// No failover: `tryHardRead` falls back to the primary on an empty result
// or error (raced alongside by default, after the replica with `race: false`).
protected override async replicaRead(
query: string,
params: unknown[],
): Promise<Record<string, unknown>[]> {
const result = await this.dbReplica.execute(query, params);
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
// ------------------------------------------------------------------
@@ -296,112 +233,97 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// ------------------------------------------------------------------
private createPool(poolConfig: PoolConfig): Pool {
return createPool({
const connectionLimit =
poolConfig.connectionLimit ?? DEFAULT_CONNECTION_LIMIT;
const pool = createPool({
maxPreparedStatements: 900,
connectionLimit: 30,
connectionLimit,
// mysql2 only reaps idle connections when maxIdle < connectionLimit.
// Unreaped, they sit until the server's wait_timeout closes them,
// and the next statement sent on one fails. 0 means unlimited,
// and a negative maxIdle would crash the reaper.
...(connectionLimit > 1 && { maxIdle: connectionLimit - 1 }),
idleTimeout: IDLE_TIMEOUT_MS,
// Reaped connections say goodbye instead of dropping the socket.
gracefulEnd: true,
enableKeepAlive: true,
...poolConfig,
multipleStatements: true,
} as PoolConfig);
}
/** Reinitialize the primary pool (e.g. after a health-check failure). */
reinitPrimary(): void {
if (this.shutdownStarted) return;
const dbConf = this.config.database!;
const previous = this.primaryPool;
this.primaryPool = this.createPool({
host: dbConf.host ?? '127.0.0.1',
port: dbConf.port ?? 3306,
user: dbConf.user ?? 'root',
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
});
this.db = new SQLBatcher(this.primaryPool, 30, 5);
if (this.configuration === Configuration.SINGLE) {
this.replicaPool = this.primaryPool;
this.dbReplica = new SQLBatcher(this.primaryPool, 10, 5);
}
if (previous && previous !== this.primaryPool) {
this.closePool(previous, 'reinit:primary').catch(() => {});
}
}
/** Reinitialize the replica pool. */
reinitReplica(): void {
if (this.shutdownStarted || !this.config.database?.replica) return;
const previous = this.replicaPool;
this.replicaPool = this.createPool(this.config.database.replica);
this.dbReplica = new SQLBatcher(this.replicaPool, 10, 5);
if (
previous &&
previous !== this.replicaPool &&
previous !== this.primaryPool
) {
this.closePool(previous, 'reinit:replica').catch(() => {});
}
}
// ------------------------------------------------------------------
// Retry helpers (for health checks or resilient reads)
// ------------------------------------------------------------------
static isRetriableError(error: unknown): boolean {
const code = (error as { code?: string })?.code;
if (code && RETRIABLE_ERROR_CODES.has(code)) return true;
const msg = String((error as Error)?.message ?? '');
return RETRIABLE_ERROR_MESSAGES.some((m) => msg.includes(m));
}
async readWithRetry(
label: string,
operation: () => Promise<unknown[]>,
opts?: {
maxAttempts?: number;
baseBackoffMs?: number;
maxBackoffMs?: number;
jitterRatio?: number;
},
): Promise<unknown[]> {
const maxAttempts = opts?.maxAttempts ?? 3;
const baseBackoffMs = opts?.baseBackoffMs ?? 100;
const maxBackoffMs = opts?.maxBackoffMs ?? 500;
const jitterRatio = opts?.jitterRatio ?? 0.2;
let attempt = 1;
while (true) {
try {
return await operation();
} catch (error) {
if (this.shutdownStarted) throw error;
if (
attempt >= maxAttempts ||
!MySQLDatabaseClient.isRetriableError(error)
)
throw error;
const raw = baseBackoffMs * 2 ** (attempt - 1);
const capped = Math.min(maxBackoffMs, raw);
const window = Math.round(capped * jitterRatio);
const jitter =
window === 0
? 0
: Math.floor(Math.random() * (window * 2 + 1)) - window;
const delay = Math.max(0, capped + jitter);
console.warn(
`[${label}] transient mysql error (${(error as { code?: string })?.code ?? 'unknown'}); retry ${attempt + 1}/${maxAttempts} in ${delay}ms`,
// Server-side kill switch for runaway reads: MySQL applies
// max_execution_time to SELECT statements only, so this is
// write-safe. Without it, a stalled database turns reads into
// indefinite hangs that no client-side timeout ever converts
// into a failure. 0 disables.
const selectTimeoutMs = Math.floor(
Number(
this.config.database?.selectTimeoutMs ??
DEFAULT_SELECT_TIMEOUT_MS,
),
);
if (selectTimeoutMs > 0) {
pool.on('connection', (conn) => {
conn.query(
`SET SESSION max_execution_time = ${selectTimeoutMs}`,
);
await new Promise((r) => setTimeout(r, delay));
attempt++;
}
});
}
return pool;
}
/**
* One write per flush. Coalesced writes share a transaction, so unrelated
* callers hold each other's row locks until COMMIT and deadlock; a lone
* statement needs no BEGIN/COMMIT round trips.
*/
private createPrimaryBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxBatchSize: 1,
poolLabel: 'primary',
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
/** Reads that must see the primary, still coalesced. */
private createPrimaryReadBatcher(
pool: Pool,
inFlight: { count: number },
): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 30,
maxInFlightImmediate: 8,
inFlight,
maxBatchSize: 5,
poolLabel: 'primary',
readOnly: true,
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
private createReplicaBatcher(
pool: Pool,
inFlight: { count: number },
): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 10,
maxInFlightImmediate: 8,
inFlight,
maxBatchSize: 5,
poolLabel: 'replica',
readOnly: true,
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
/**
* Replica failures worth retrying on the primary: batcher load-shed or
* transient connection errors — never deterministic SQL errors.
*/
private static isFailoverWorthy(error: unknown): boolean {
const code = (error as { code?: string })?.code;
return code === 'dbBatchFailed' || isRetriableError(error);
}
// ------------------------------------------------------------------
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -18,7 +18,7 @@
*/
import type { FieldDef, QueryResult } from 'pg';
import { describe, expect, it } from 'vitest';
import { describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import { DatabaseClientFactory } from './index.js';
import {
@@ -28,6 +28,20 @@ import {
type PostgresPoolClient,
} from './PostgresDatabaseClient.js';
// Pass-through spy: the rewrite is pure, so its call count shows the memo.
const { rewriteSpy } = vi.hoisted(() => ({ rewriteSpy: vi.fn() }));
vi.mock('./preparePostgresSql.js', async (importOriginal) => {
const actual =
await importOriginal<typeof import('./preparePostgresSql.js')>();
return {
...actual,
preparePostgresSql: (sql: string) => {
rewriteSpy(sql);
return actual.preparePostgresSql(sql);
},
};
});
type QueryCall = {
text: string;
values?: unknown[];
@@ -167,6 +181,23 @@ describe('PostgresDatabaseClient', () => {
});
});
it('rewrites a repeated query once', async () => {
const pool = new RecordingPool();
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
const query = 'SELECT `name` FROM `apps` WHERE `id` = ?';
rewriteSpy.mockClear();
await client.read(query, [1]);
await client.pread(query, [2]);
await client.write(query, [3]);
expect(rewriteSpy).toHaveBeenCalledTimes(1);
expect(pool.calls.slice(-3).map((c) => c.text)).toEqual(
Array(3).fill('SELECT "name" FROM "apps" WHERE "id" = $1'),
);
});
it('normalizes int8 fields on read and primary read rows', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
@@ -357,4 +388,33 @@ describe('PostgresDatabaseClient', () => {
]);
expect(conn.released).toBe(true);
});
it('leaves the pool open through prepare-shutdown; only onServerShutdown closes it', async () => {
vi.useFakeTimers();
try {
const pool = new RecordingPool();
let ended = false;
pool.end = async () => {
ended = true;
};
const client = new PostgresDatabaseClient(
postgresConfig(),
() => pool,
);
await client.onServerStart();
await client.onServerPrepareShutdown();
expect(ended).toBe(false);
// No timer closes the pool on its own — the layers above are
// still draining through it for as long as the process drains.
await vi.advanceTimersByTimeAsync(60_000);
expect(ended).toBe(false);
await client.onServerShutdown();
expect(ended).toBe(true);
} finally {
vi.useRealTimers();
}
});
});
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -17,15 +17,14 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { Pool, type PoolConfig, type QueryResult } from 'pg';
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
import { Span } from '../../util/span.js';
import {
AbstractDatabaseClient,
type BatchEntry,
type WriteResult,
} from './DatabaseClient';
import { compareMigrationFilenames } from './migrationFilenames.js';
import { preparePostgresSql } from './preparePostgresSql.js';
import { splitPostgresStatements } from './splitPostgresStatements.js';
import type { IConfig } from '../../types';
@@ -61,6 +60,7 @@ enum Configuration {
}
const POSTGRES_INT8_OID = 20;
const PREPARED_SQL_CACHE_SIZE = 1000;
const INTEGER_TEXT_PATTERN = /^-?\d+$/u;
const normalizePostgresInt8 = (
@@ -137,7 +137,10 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
private replicaPool!: PostgresPool;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
/** Query text as written → its Postgres rewrite, which is pure. */
private readonly preparedSql = new BoundedTtlMap<string, string>({
maxEntries: PREPARED_SQL_CACHE_SIZE,
});
constructor(
config: IConfig,
@@ -164,35 +167,23 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
this.configuration = Configuration.SINGLE;
}
await this.runMigrations();
await this.applyMigrationPaths(
() => this.primaryPool.connect(),
splitPostgresStatements,
{ transactional: true },
);
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
const drainMs = 60_000;
console.log(
`[postgres] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools().catch((e) =>
console.error('[postgres] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
// Pools stay open; onServerShutdown closes them after the layers above
// have drained through them.
console.log('[postgres] entering drain mode');
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools();
}
@@ -214,6 +205,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return value;
}
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
@@ -222,6 +214,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return normalizePostgresRows(result);
}
@Span('db.pread', (query: string) => ({ 'db.statement': query }))
override async pread(
query: string,
params: unknown[] = [],
@@ -230,6 +223,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return normalizePostgresRows(result);
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
@@ -238,6 +232,9 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return mapPostgresWriteResult(result);
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(entries: BatchEntry[]): Promise<void> {
if (entries.length === 0) return;
@@ -258,97 +255,8 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
}
}
override async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
if (this.configuration === Configuration.SINGLE) {
return this.read(query, params);
}
const primaryPromise = this.query(this.primaryPool, query, params);
try {
const replicaResult = await this.query(
this.replicaPool,
query,
params,
);
if (replicaResult.rows.length > 0) {
primaryPromise.catch(() => {});
return normalizePostgresRows(replicaResult);
}
} catch {
// fall through to primary
}
const primaryResult = await primaryPromise;
return normalizePostgresRows(primaryResult);
}
private async runMigrations(): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const conn = await this.primaryPool.connect();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) =>
f.endsWith('.sql') && f.startsWith('postgres'),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[postgres] migration path is unreadable: ${dir}`,
{ cause: e },
);
}
if (files.length === 0) {
console.log(`[postgres] no migrations in ${dir}`);
continue;
}
console.log(
`[postgres] running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const filePath = resolvePath(dir, file);
const contents = readFileSync(filePath, 'utf8');
const statements = splitPostgresStatements(contents);
await conn.query('BEGIN');
try {
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`[postgres] failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
await conn.query('COMMIT');
} catch (e) {
await conn.query('ROLLBACK').catch(() => {});
throw e;
}
console.log(
`[postgres] applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
protected override hasReadReplica(): boolean {
return this.configuration === Configuration.REPLICA;
}
private createPool(dbConf: PostgresEndpointConfig): PostgresPool {
@@ -375,8 +283,12 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
query: string,
params: unknown[] = [],
): Promise<QueryResult> {
const prepared = preparePostgresSql(query);
return target.query(prepared.text, params);
let text = this.preparedSql.get(query);
if (text === undefined) {
text = preparePostgresSql(query).text;
this.preparedSql.set(query, text);
}
return target.query(text, params);
}
private async closeCurrentPools(): Promise<void> {
+270 -53
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -18,10 +18,20 @@
*/
import { metrics } from '@opentelemetry/api';
import {
POOL_ACQUIRE_TIMEOUT,
isNeverSentError,
isRetriableError,
isRolledBackError,
} from './retriableErrors.js';
const DEFAULT_MAX_QUEUE_SIZE = 1000;
const DEFAULT_FAILURE_THRESHOLD = 5;
const DEFAULT_COOLDOWN_MS = 5_000;
const DEFAULT_ACQUIRE_TIMEOUT_MS = 5_000;
const ACQUIRE_ATTEMPTS = 3;
const ITEM_RETRY_ATTEMPTS = 2;
const RETRY_BASE_BACKOFF_MS = 100;
const FALLBACK_RETRY_CONCURRENCY = 8;
const meter = metrics.getMeter('puter-backend');
@@ -53,33 +63,85 @@ const fallbackItemFailuresCounter = meter.createCounter(
'Per-item failures observed during SQLBatcher per-item retry',
},
);
const fallbackItemRetriesCounter = meter.createCounter(
'sql_batcher.fallback.item_retries',
{
description:
'Transient per-item failures retried during SQLBatcher fallback',
},
);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
export class SQLBatcher {
dbPool;
maxTimeInQueue;
maxInFlightImmediate;
maxBatchSize;
maxQueueSize;
failureThreshold;
cooldownMs;
poolLabel;
readOnly;
acquireTimeoutMs;
queue = [];
timeouts = [];
#flushTimer = null;
#flushTimerIsImmediate = false;
#inFlight;
#consecutiveFailures = 0;
#lastFailureAt = 0;
#metricAttrs;
/**
* @param {object} dbPool Mysql2 pool
* @param {object} [opts]
* @param {number} [opts.maxTimeInQueue] Ms an item may wait before flush
* @param {number} [opts.maxInFlightImmediate] While fewer flushes than this
* are in flight, a lone item flushes on the next turn instead of waiting
* maxTimeInQueue; past it, items wait and coalesce to spare connections
* @param {{ count: number }} [opts.inFlight] In-flight flush counter the
* cap reads; batchers sharing a pool can share one
* @param {number} [opts.maxBatchSize] Items coalesced per flush
* @param {number} [opts.maxQueueSize] Drop-oldest high-water mark
* @param {number} [opts.failureThreshold] Consecutive failures to open the
* breaker
* @param {number} [opts.cooldownMs] Breaker open duration after last
* failure
* @param {'primary' | 'replica'} [opts.poolLabel] Role label on metrics; in
* single-node setups the 'replica' batcher shares the primary pool, so
* this reflects the read/write role rather than a physical instance
* @param {boolean} [opts.readOnly] This batcher only ever carries SELECTs,
* so any transient failure is safe to retry
* @param {number} [opts.acquireTimeoutMs] Max wait for a pooled connection;
* 0 disables the bound
*/
constructor(
dbPool,
maxTimeInQueue = 20,
maxBatchSize = 50,
maxQueueSize = DEFAULT_MAX_QUEUE_SIZE,
failureThreshold = DEFAULT_FAILURE_THRESHOLD,
cooldownMs = DEFAULT_COOLDOWN_MS,
{
maxTimeInQueue = 20,
maxInFlightImmediate = 0,
inFlight = { count: 0 },
maxBatchSize = 50,
maxQueueSize = DEFAULT_MAX_QUEUE_SIZE,
failureThreshold = DEFAULT_FAILURE_THRESHOLD,
cooldownMs = DEFAULT_COOLDOWN_MS,
poolLabel = 'primary',
readOnly = false,
acquireTimeoutMs = DEFAULT_ACQUIRE_TIMEOUT_MS,
} = {},
) {
this.dbPool = dbPool;
this.maxTimeInQueue = maxTimeInQueue;
this.maxInFlightImmediate = maxInFlightImmediate;
this.#inFlight = inFlight;
this.maxBatchSize = maxBatchSize;
this.maxQueueSize = maxQueueSize;
this.failureThreshold = failureThreshold;
this.cooldownMs = cooldownMs;
this.poolLabel = poolLabel;
this.readOnly = readOnly;
this.acquireTimeoutMs = acquireTimeoutMs;
this.#metricAttrs = { pool: poolLabel };
}
async execute(sql, values) {
@@ -90,9 +152,13 @@ export class SQLBatcher {
return this;
}
#createPublicBatchError() {
// The public error is deliberately opaque (no SQL, no internals), but
// `reason` distinguishes the load-shed path for logs and callers:
// breakerOpen | queueOverflow | connAcquire.
#createPublicBatchError(reason) {
const error = new Error('Database operation failed');
error.code = 'dbBatchFailed';
error.reason = reason;
return error;
}
@@ -106,8 +172,8 @@ export class SQLBatcher {
async query(sql, values) {
if (this.#isBreakerOpen()) {
enqueueRejectedCounter.add(1);
throw this.#createPublicBatchError();
enqueueRejectedCounter.add(1, this.#metricAttrs);
throw this.#createPublicBatchError('breakerOpen');
}
const { promise, resolve, reject } = Promise.withResolvers();
@@ -117,8 +183,8 @@ export class SQLBatcher {
// to have already exceeded any caller-side timeout anyway.
while (this.queue.length >= this.maxQueueSize) {
const dropped = this.queue.shift();
dropped.reject(this.#createPublicBatchError());
enqueueDroppedCounter.add(1);
dropped.reject(this.#createPublicBatchError('queueOverflow'));
enqueueDroppedCounter.add(1, this.#metricAttrs);
}
this.queue.push({
@@ -131,55 +197,141 @@ export class SQLBatcher {
if (this.queue.length >= this.maxBatchSize) {
this.flush(this.queue.splice(0, this.maxBatchSize));
} else if (this.queue.length === 1) {
this.timeouts.push(
setTimeout(() => {
this.flush(this.queue.splice(0, this.queue.length));
}, this.maxTimeInQueue),
);
} else if (this.#flushTimer === null) {
this.#armFlushTimer();
}
return promise;
}
async flush(batch) {
const timeout = this.timeouts.shift();
if (timeout && !timeout._destroyed) {
clearTimeout(timeout);
// A lone flush holds a pooled connection for a full round trip, so only
// the first maxInFlightImmediate flushes skip the wait.
#armFlushTimer() {
const run = () => {
this.#flushTimer = null;
this.flush(this.queue.splice(0, this.queue.length));
};
this.#flushTimerIsImmediate =
this.#inFlight.count < this.maxInFlightImmediate;
this.#flushTimer = this.#flushTimerIsImmediate
? setImmediate(run)
: setTimeout(run, this.maxTimeInQueue);
}
#clearFlushTimer() {
if (this.#flushTimer === null) return;
if (this.#flushTimerIsImmediate) clearImmediate(this.#flushTimer);
else clearTimeout(this.#flushTimer);
this.#flushTimer = null;
}
// Bounded wait for a pooled connection. Without a bound, a stalled
// database turns every flush into an indefinite hang — nothing fails,
// so neither the breaker nor callers' own timeouts ever engage.
#getConnectionWithTimeout() {
const acquire = this.dbPool.promise().getConnection();
if (!this.acquireTimeoutMs) return acquire;
return new Promise((resolve, reject) => {
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
// A connection that arrives late must go back to the pool.
acquire.then(
(conn) => conn.release(),
() => {},
);
const error = new Error(
'Timed out acquiring database connection',
);
error.code = POOL_ACQUIRE_TIMEOUT;
reject(error);
}, this.acquireTimeoutMs);
acquire.then(
(conn) => {
if (timedOut) return;
clearTimeout(timer);
resolve(conn);
},
(err) => {
if (timedOut) return;
clearTimeout(timer);
reject(err);
},
);
});
}
// Acquisition failures never sent a statement, so retrying is always
// safe regardless of what the batch contains.
async #acquireConnection() {
let lastError;
for (let attempt = 1; attempt <= ACQUIRE_ATTEMPTS; attempt++) {
try {
return await this.#getConnectionWithTimeout();
} catch (error) {
lastError = error;
if (attempt < ACQUIRE_ATTEMPTS) {
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
}
}
}
throw lastError;
}
async flush(batch) {
this.#clearFlushTimer();
if (batch.length === 0) return;
const query = `${batch.map((b) => b.sql.replace(/;+\s*$/, '')).join(';')}; SELECT 1`; // SELECT 1 forces mysql2 to return array
const values = batch.map((b) => b.values ?? []).flat();
this.#inFlight.count++;
try {
await this.#flushBatch(batch);
} finally {
this.#inFlight.count--;
}
}
async #flushBatch(batch) {
let connection;
try {
connection = await this.dbPool.promise().getConnection();
connection = await this.#acquireConnection();
} catch (error) {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1);
this.#recordFlushFailure();
console.warn(
'SQLBatcher could not acquire connection for flush:',
error,
);
for (const b of batch) {
b.reject(this.#createPublicBatchError());
b.reject(this.#createPublicBatchError('connAcquire'));
}
return;
}
if (batch.length === 1 && !this.readOnly) {
await this.#flushLoneWrite(batch[0], connection);
return;
}
const query = `${batch.map((b) => b.sql.replace(/;+\s*$/, '')).join(';')}; SELECT 1`; // SELECT 1 forces mysql2 to return array
const values = batch.map((b) => b.values ?? []).flat();
// Run the coalesced multi-statement inside an explicit transaction so
// a single bad statement (e.g. a duplicate-key INSERT) rolls back the
// whole batch atomically, leaving us free to re-run each item
// individually below. Without this, MySQL would commit every
// statement up to the failure point and a per-item retry would
// misreport already-committed inserts as duplicate-key failures.
//
// A read-only batch has nothing to roll back, and the wrapper is not
// free: BEGIN and COMMIT are round trips of their own, which triples
// the cost of a read against a pool in another region.
const wrapInTransaction = !this.readOnly;
let batchSucceeded = false;
try {
await connection.beginTransaction();
if (wrapInTransaction) await connection.beginTransaction();
const [results, fields] = await connection.query(query, values);
await connection.commit();
if (wrapInTransaction) await connection.commit();
batchSucceeded = true;
this.#consecutiveFailures = 0;
for (let i = 0; i < batch.length; i++) {
@@ -187,10 +339,12 @@ export class SQLBatcher {
b.resolve([results[i], fields?.[i]]);
}
} catch (batchError) {
try {
await connection.rollback();
} catch (rollbackError) {
console.warn('SQLBatcher rollback failed:', rollbackError);
if (wrapInTransaction) {
try {
await connection.rollback();
} catch (rollbackError) {
console.warn('SQLBatcher rollback failed:', rollbackError);
}
}
console.warn(
'SQLBatcher batch failed; retrying items individually:',
@@ -202,12 +356,13 @@ export class SQLBatcher {
if (batchSucceeded) return;
// Per-item fallback. The transaction was rolled back so no statement
// committed; re-running each item independently produces clean
// success/failure outcomes for each caller. Concurrency is capped to
// avoid briefly saturating the pool when a large batch fails.
flushFailureCounter.add(1);
fallbackInvocationsCounter.add(1);
// Per-item fallback. Nothing from the batch is live — it was rolled
// back, or carried only SELECTs — so re-running each item
// independently produces clean success/failure outcomes for each
// caller. Concurrency is capped to avoid briefly saturating the pool
// when a large batch fails.
flushFailureCounter.add(1, this.#metricAttrs);
fallbackInvocationsCounter.add(1, this.#metricAttrs);
const settled = new Array(batch.length);
let cursor = 0;
@@ -216,17 +371,7 @@ export class SQLBatcher {
async () => {
while (cursor < batch.length) {
const i = cursor++;
const b = batch[i];
try {
settled[i] = {
ok: true,
value: await this.dbPool
.promise()
.query(b.sql, b.values ?? []),
};
} catch (error) {
settled[i] = { ok: false, error };
}
settled[i] = await this.#runItem(batch[i]);
}
},
);
@@ -246,7 +391,7 @@ export class SQLBatcher {
}
}
if (failureCount > 0) {
fallbackItemFailuresCounter.add(failureCount);
fallbackItemFailuresCounter.add(failureCount, this.#metricAttrs);
}
// Only escalate the breaker when the database itself looks unhealthy
@@ -259,4 +404,76 @@ export class SQLBatcher {
this.#consecutiveFailures++;
}
}
// A single statement is atomic under autocommit, so it skips BEGIN/COMMIT
// and their round trips. With no transaction to roll back, a failure is
// not re-run wholesale; only the per-item retry policy applies.
async #flushLoneWrite(item, connection) {
const r = await this.#runItem(item, connection);
if (r.ok) {
this.#consecutiveFailures = 0;
item.resolve(r.value);
return;
}
// An error the server answered with (duplicate key, bad SQL) means it
// is up; only failing to reach it counts toward the breaker.
if (isRetriableError(r.error)) {
this.#recordFlushFailure();
} else {
this.#consecutiveFailures = 0;
flushFailureCounter.add(1, this.#metricAttrs);
}
item.reject(r.error);
}
#recordFlushFailure() {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1, this.#metricAttrs);
}
// Run one item on its own, retrying transient failures with backoff.
// A read-only batcher may retry anything transient; a batcher that
// carries writes only retries failures where the statement provably
// did not apply — either it never reached the server, or the server
// rolled it back itself. A write that died mid-flight may have
// committed, and re-running that one would double-apply.
//
// Lock contention lands in the second group and is worth retrying rather
// than surfacing: an item is a single statement, so a deadlock victim has
// been fully undone, and the caller sees an unhandled 500 for what the
// database is telling us to just run again.
async #runItem(b, connection) {
let attempt = 0;
while (true) {
if (!connection) {
try {
connection = await this.#acquireConnection();
} catch (error) {
return { ok: false, error };
}
}
let error;
try {
return {
ok: true,
value: await connection.query(b.sql, b.values ?? []),
};
} catch (e) {
error = e;
} finally {
connection.release();
connection = undefined;
}
const canRetry = this.readOnly
? isRetriableError(error) || isRolledBackError(error)
: isNeverSentError(error) || isRolledBackError(error);
if (!canRetry || attempt >= ITEM_RETRY_ATTEMPTS) {
return { ok: false, error };
}
attempt++;
fallbackItemRetriesCounter.add(1, this.#metricAttrs);
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
}
}
}
@@ -0,0 +1,717 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it, vi } from 'vitest';
import { SQLBatcher } from './SQLBatcher.js';
const makeError = (code: string, message = code): Error & { code: string } => {
const error = new Error(message) as Error & { code: string };
error.code = code;
return error;
};
interface FakeConnection {
beginTransaction: ReturnType<typeof vi.fn>;
query: ReturnType<typeof vi.fn>;
commit: ReturnType<typeof vi.fn>;
rollback: ReturnType<typeof vi.fn>;
release: ReturnType<typeof vi.fn>;
}
// The batch flush sends one coalesced multi-statement (always suffixed
// `; SELECT 1`); fallback items send their original single statement.
const isBatchQuery = (sql: string) => sql.endsWith('; SELECT 1');
const makeConnection = (
onQuery: (sql: string, values: unknown[]) => unknown,
): FakeConnection => ({
beginTransaction: vi.fn(async () => {}),
query: vi.fn(async (sql: string, values: unknown[]) => onQuery(sql, values)),
commit: vi.fn(async () => {}),
rollback: vi.fn(async () => {}),
release: vi.fn(),
});
const makePool = (connection: FakeConnection | (() => Promise<FakeConnection>)) => {
const getConnection = vi.fn(async () =>
typeof connection === 'function' ? connection() : connection,
);
return {
pool: { promise: () => ({ getConnection }) },
getConnection,
};
};
// Happy-path onQuery: batch returns one result row-set per statement plus
// the trailing SELECT 1 row-set.
const happyBatch = (sql: string) => {
if (!isBatchQuery(sql)) throw new Error('unexpected fallback query');
const statements = sql.split(';').length - 1;
return [
Array.from({ length: statements + 1 }, (_, i) => [{ n: i }]),
undefined,
];
};
// Each query on this connection waits until the test releases it, so its
// flush stays in flight.
const makeHeldConnection = () => {
const held: Array<() => void> = [];
const conn = makeConnection(
(sql) =>
new Promise((resolve) => {
held.push(() => resolve(happyBatch(sql)));
}),
);
const releaseAll = () => {
for (const release of held.splice(0)) release();
};
const sent = () => conn.query.mock.calls.map(([sql]) => sql);
return { conn, releaseAll, sent };
};
const fakeFlushTimers = () =>
vi.useFakeTimers({
toFake: [
'setTimeout',
'clearTimeout',
'setImmediate',
'clearImmediate',
],
});
describe('SQLBatcher flush timing', () => {
it('flushes a lone query on the next turn while under the in-flight cap', async () => {
const conn = makeConnection(happyBatch);
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxTimeInQueue: 60_000,
maxInFlightImmediate: 2,
});
const pending = batcher.query('SELECT a', []);
expect(getConnection).not.toHaveBeenCalled();
await new Promise((resolve) => setImmediate(resolve));
expect(getConnection).toHaveBeenCalledTimes(1);
expect((await pending)[0]).toEqual([{ n: 0 }]);
});
it('holds a lone query for maxTimeInQueue without an in-flight cap', async () => {
fakeFlushTimers();
try {
const conn = makeConnection(happyBatch);
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxTimeInQueue: 30,
});
const pending = batcher.query('SELECT a', []);
await vi.advanceTimersByTimeAsync(29);
expect(getConnection).not.toHaveBeenCalled();
await vi.advanceTimersByTimeAsync(1);
expect(getConnection).toHaveBeenCalledTimes(1);
expect((await pending)[0]).toEqual([{ n: 0 }]);
} finally {
vi.useRealTimers();
}
});
it('batches queries from the same turn and sends later ones separately', async () => {
const conn = makeConnection(happyBatch);
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxInFlightImmediate: 8,
});
const sameTurn = [
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
batcher.query('SELECT c', []),
];
await new Promise((resolve) => setImmediate(resolve));
const nextTurn = batcher.query('SELECT d', []);
const results = await Promise.all([...sameTurn, nextTurn]);
expect(results.map((r) => r[0])).toEqual([
[{ n: 0 }],
[{ n: 1 }],
[{ n: 2 }],
[{ n: 0 }],
]);
expect(getConnection).toHaveBeenCalledTimes(2);
expect(conn.query.mock.calls.map(([sql]) => sql)).toEqual([
'SELECT a;SELECT b;SELECT c; SELECT 1',
'SELECT d; SELECT 1',
]);
});
it('gives a query enqueued during an in-flight flush its own flush while under the cap', async () => {
const { conn, releaseAll, sent } = makeHeldConnection();
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxInFlightImmediate: 2,
});
const turn = () => new Promise((resolve) => setImmediate(resolve));
const a = batcher.query('SELECT a', []);
await turn();
const b = batcher.query('SELECT b', []);
await turn();
expect(sent()).toEqual(['SELECT a; SELECT 1', 'SELECT b; SELECT 1']);
releaseAll();
expect((await a)[0]).toEqual([{ n: 0 }]);
expect((await b)[0]).toEqual([{ n: 0 }]);
});
it('holds a lone query for maxTimeInQueue at the in-flight cap and coalesces what follows', async () => {
fakeFlushTimers();
try {
const { conn, releaseAll, sent } = makeHeldConnection();
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxTimeInQueue: 30,
maxInFlightImmediate: 1,
});
const a = batcher.query('SELECT a', []);
await vi.advanceTimersByTimeAsync(0);
expect(sent()).toEqual(['SELECT a; SELECT 1']);
const b = batcher.query('SELECT b', []);
await vi.advanceTimersByTimeAsync(29);
const c = batcher.query('SELECT c', []);
expect(sent()).toHaveLength(1);
await vi.advanceTimersByTimeAsync(1);
expect(sent()).toEqual([
'SELECT a; SELECT 1',
'SELECT b;SELECT c; SELECT 1',
]);
releaseAll();
expect((await a)[0]).toEqual([{ n: 0 }]);
expect((await b)[0]).toEqual([{ n: 0 }]);
expect((await c)[0]).toEqual([{ n: 1 }]);
// Settled flushes free the cap again.
const d = batcher.query('SELECT d', []);
await vi.advanceTimersByTimeAsync(0);
expect(sent()).toHaveLength(3);
releaseAll();
expect((await d)[0]).toEqual([{ n: 0 }]);
} finally {
vi.useRealTimers();
}
});
it('shares the in-flight cap between batchers given one counter', async () => {
fakeFlushTimers();
try {
const { conn, releaseAll, sent } = makeHeldConnection();
const { pool } = makePool(conn);
const inFlight = { count: 0 };
const opts = {
readOnly: true,
maxTimeInQueue: 30,
maxInFlightImmediate: 1,
inFlight,
};
const first = new SQLBatcher(pool, opts);
const second = new SQLBatcher(pool, opts);
const own = new SQLBatcher(pool, { ...opts, inFlight: undefined });
const a = first.query('SELECT a', []);
await vi.advanceTimersByTimeAsync(0);
const b = second.query('SELECT b', []);
const c = own.query('SELECT c', []);
await vi.advanceTimersByTimeAsync(0);
expect(sent()).toEqual([
'SELECT a; SELECT 1',
'SELECT c; SELECT 1',
]);
await vi.advanceTimersByTimeAsync(30);
expect(sent()).toHaveLength(3);
expect(sent()[2]).toBe('SELECT b; SELECT 1');
releaseAll();
await Promise.all([a, b, c]);
expect(inFlight.count).toBe(0);
} finally {
vi.useRealTimers();
}
});
it('counts a flush as in flight until its per-item fallback settles', async () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
fakeFlushTimers();
try {
const held: Array<() => void> = [];
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
return new Promise((resolve) => {
held.push(() => resolve([[{ ok: sql }], undefined]));
});
});
const { pool } = makePool(conn);
const sent = () => conn.query.mock.calls.map(([sql]) => sql);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxTimeInQueue: 30,
maxInFlightImmediate: 1,
acquireTimeoutMs: 0,
});
const a = batcher.query('SELECT a', []);
await vi.advanceTimersByTimeAsync(0);
// The batch failed; the per-item retry of `a` is still running.
expect(sent()).toEqual(['SELECT a; SELECT 1', 'SELECT a']);
const b = batcher.query('SELECT b', []);
await vi.advanceTimersByTimeAsync(29);
expect(sent()).toHaveLength(2);
await vi.advanceTimersByTimeAsync(1);
expect(sent()).toEqual([
'SELECT a; SELECT 1',
'SELECT a',
'SELECT b; SELECT 1',
'SELECT b',
]);
for (const release of held.splice(0)) release();
expect((await a)[0]).toEqual([{ ok: 'SELECT a' }]);
expect((await b)[0]).toEqual([{ ok: 'SELECT b' }]);
} finally {
vi.useRealTimers();
warnSpy.mockRestore();
}
});
it('flushes a full batch at once past the cap and cancels the armed wait', async () => {
fakeFlushTimers();
try {
const { conn, releaseAll, sent } = makeHeldConnection();
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, {
readOnly: true,
maxBatchSize: 2,
maxTimeInQueue: 30,
maxInFlightImmediate: 1,
acquireTimeoutMs: 0,
});
const a = batcher.query('SELECT a', []);
await vi.advanceTimersByTimeAsync(0);
const b = batcher.query('SELECT b', []);
expect(vi.getTimerCount()).toBe(1);
const c = batcher.query('SELECT c', []);
expect(getConnection).toHaveBeenCalledTimes(2);
expect(vi.getTimerCount()).toBe(0);
// A later lone query waits its own full maxTimeInQueue.
await vi.advanceTimersByTimeAsync(10);
const d = batcher.query('SELECT d', []);
await vi.advanceTimersByTimeAsync(29);
expect(getConnection).toHaveBeenCalledTimes(2);
await vi.advanceTimersByTimeAsync(1);
expect(getConnection).toHaveBeenCalledTimes(3);
expect(sent()).toEqual([
'SELECT a; SELECT 1',
'SELECT b;SELECT c; SELECT 1',
'SELECT d; SELECT 1',
]);
releaseAll();
await Promise.all([a, b, c, d]);
} finally {
vi.useRealTimers();
}
});
it('cancels the pending next-turn flush when a batch fills up', async () => {
fakeFlushTimers();
try {
const conn = makeConnection(happyBatch);
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxBatchSize: 2,
readOnly: true,
maxInFlightImmediate: 8,
acquireTimeoutMs: 0,
});
const a = batcher.query('SELECT a', []);
expect(vi.getTimerCount()).toBe(1);
const b = batcher.query('SELECT b', []);
expect(vi.getTimerCount()).toBe(0);
const c = batcher.query('SELECT c', []);
expect(vi.getTimerCount()).toBe(1);
await vi.runAllTimersAsync();
await Promise.all([a, b, c]);
expect(vi.getTimerCount()).toBe(0);
expect(getConnection).toHaveBeenCalledTimes(2);
} finally {
vi.useRealTimers();
}
});
it('leaves no timer armed after a failed flush, a drop or a breaker rejection', async () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
fakeFlushTimers();
try {
const getConnection = vi.fn(async () => {
throw makeError('ECONNREFUSED');
});
const batcher = new SQLBatcher(
{ promise: () => ({ getConnection }) },
{
readOnly: true,
maxQueueSize: 1,
failureThreshold: 1,
cooldownMs: 60_000,
maxInFlightImmediate: 8,
},
);
const dropped = expect(
batcher.query('SELECT a', []),
).rejects.toMatchObject({ reason: 'queueOverflow' });
const failed = expect(
batcher.query('SELECT b', []),
).rejects.toMatchObject({ reason: 'connAcquire' });
expect(vi.getTimerCount()).toBe(1);
await vi.runAllTimersAsync();
await dropped;
await failed;
expect(vi.getTimerCount()).toBe(0);
await expect(batcher.query('SELECT c', [])).rejects.toMatchObject({
reason: 'breakerOpen',
});
expect(vi.getTimerCount()).toBe(0);
expect(batcher.queue).toHaveLength(0);
} finally {
vi.useRealTimers();
warnSpy.mockRestore();
}
});
});
describe('SQLBatcher', () => {
it('resolves each batched item with its own result', async () => {
const conn = makeConnection(happyBatch);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
const [a, b] = await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(a[0]).toEqual([{ n: 0 }]);
expect(b[0]).toEqual([{ n: 1 }]);
expect(conn.beginTransaction).toHaveBeenCalledTimes(1);
expect(conn.commit).toHaveBeenCalledTimes(1);
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('skips the transaction wrapper on a readOnly batcher', async () => {
const conn = makeConnection(happyBatch);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
const [a, b] = await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(a[0]).toEqual([{ n: 0 }]);
expect(b[0]).toEqual([{ n: 1 }]);
// BEGIN and COMMIT are round trips; a SELECT-only batch has nothing to
// roll back, so it must not pay for them.
expect(conn.beginTransaction).not.toHaveBeenCalled();
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('does not roll back a failed readOnly batch', async () => {
const conn = makeConnection((sql: string) => {
if (isBatchQuery(sql)) throw makeError('ER_LOCK_DEADLOCK');
return [[{ n: 0 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(conn.rollback).not.toHaveBeenCalled();
});
it('drops the oldest item with reason queueOverflow at the high-water mark', async () => {
const conn = makeConnection(() => [[{ n: 0 }], undefined]);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
maxQueueSize: 1,
});
const first = batcher.query('SELECT a', []);
const second = batcher.query('SELECT b', []);
await expect(first).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'queueOverflow',
});
await expect(second).resolves.toBeTruthy();
});
it('rejects with reason connAcquire after exhausting acquisition retries', async () => {
const getConnection = vi.fn(async () => {
throw makeError('ECONNREFUSED');
});
const pool = { promise: () => ({ getConnection }) };
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('SELECT a', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'connAcquire',
});
expect(getConnection).toHaveBeenCalledTimes(3);
});
it('bounds connection acquisition and rejects when the pool never answers', async () => {
const getConnection = vi.fn(
() => new Promise(() => {}), // pool never yields a connection
);
const pool = { promise: () => ({ getConnection }) };
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
acquireTimeoutMs: 30,
});
await expect(batcher.query('SELECT a', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'connAcquire',
});
expect(getConnection).toHaveBeenCalledTimes(3);
});
it('opens the breaker after consecutive failures and rejects with reason breakerOpen', async () => {
// Batch and fallback both fail with an ambiguous (non-retriable for
// writes) connection error, so no item gets through.
const conn = makeConnection(() => {
throw makeError('ECONNRESET');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
failureThreshold: 1,
cooldownMs: 60_000,
});
await expect(batcher.query('INSERT x', [])).rejects.toMatchObject({
code: 'ECONNRESET',
});
await expect(batcher.query('INSERT y', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'breakerOpen',
});
});
it('retries transient fallback failures when readOnly', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ECONNRESET');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
const result = await batcher.query('SELECT a', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('does not retry ambiguous failures on a batcher that carries writes', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
throw makeError('ECONNRESET');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('INSERT x', [])).rejects.toMatchObject({
code: 'ECONNRESET',
});
expect(fallbackAttempts).toBe(1);
});
it('retries never-sent failures even on a batcher that carries writes', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ECONNREFUSED');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
const result = await batcher.query('INSERT x', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('retries a deadlocked write instead of surfacing it', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ER_LOCK_DEADLOCK');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ER_LOCK_DEADLOCK');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
// The victim statement was rolled back by the server, so re-running it
// can't double-apply — the caller should never see the deadlock.
const result = await batcher.query('UPDATE notification SET x', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('gives up on a write that deadlocks past the retry budget', async () => {
const conn = makeConnection(() => {
throw makeError('ER_LOCK_DEADLOCK');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('UPDATE hot SET x', [])).rejects.toMatchObject(
{ code: 'ER_LOCK_DEADLOCK' },
);
});
it('sends a lone write as-is, outside a transaction', async () => {
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw new Error('unexpected batch query');
return [{ affectedRows: 1 }, undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxBatchSize: 1 });
const result = await batcher.query('UPDATE t SET x = ?', [1]);
expect(result[0]).toEqual({ affectedRows: 1 });
expect(conn.query).toHaveBeenCalledTimes(1);
expect(conn.query).toHaveBeenCalledWith('UPDATE t SET x = ?', [1]);
expect(conn.beginTransaction).not.toHaveBeenCalled();
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('runs a lone write the server rejects once, without tripping the breaker', async () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
const conn = makeConnection((sql) => {
if (sql === 'INSERT dup') throw makeError('ER_DUP_ENTRY');
return [{ affectedRows: 1 }, undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxBatchSize: 1,
failureThreshold: 1,
cooldownMs: 60_000,
});
await expect(batcher.query('INSERT dup', [])).rejects.toMatchObject({
code: 'ER_DUP_ENTRY',
});
expect(conn.query).toHaveBeenCalledTimes(1);
expect(conn.rollback).not.toHaveBeenCalled();
expect(warnSpy).not.toHaveBeenCalled();
// The server answered, so it is up: the next write goes through.
await expect(batcher.query('INSERT fine', [])).resolves.toBeTruthy();
warnSpy.mockRestore();
});
it('retries a lone write on a connection the server closed for inactivity', async () => {
let attempts = 0;
const conn = makeConnection(() => {
attempts++;
if (attempts === 1) {
throw makeError('ER_CLIENT_INTERACTION_TIMEOUT');
}
return [{ affectedRows: 1 }, undefined];
});
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxBatchSize: 1 });
const result = await batcher.query('INSERT x', []);
expect(result[0]).toEqual({ affectedRows: 1 });
expect(attempts).toBe(2);
expect(getConnection).toHaveBeenCalledTimes(2);
});
it('never retries deterministic row-level errors and does not escalate the breaker', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ER_DUP_ENTRY');
fallbackAttempts++;
if (sql === 'INSERT dup') throw makeError('ER_DUP_ENTRY');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
failureThreshold: 1,
cooldownMs: 60_000,
readOnly: true,
});
const dup = batcher.query('INSERT dup', []);
const fine = batcher.query('INSERT fine', []);
await expect(dup).rejects.toMatchObject({ code: 'ER_DUP_ENTRY' });
await expect(fine).resolves.toBeTruthy();
expect(fallbackAttempts).toBe(2);
// One fallback item succeeded, so the breaker must stay closed.
const conn2 = makeConnection(happyBatch);
// reuse same batcher/pool: next query must not be rejected upfront
conn.query.mockImplementation(conn2.query.getMockImplementation()!);
await expect(batcher.query('SELECT a', [])).resolves.toBeTruthy();
});
});
File diff suppressed because it is too large. Load diff
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -21,14 +21,20 @@ import { existsSync, mkdirSync, readFileSync } from 'fs';
import { basename, dirname, extname, join, resolve } from 'path';
import { createContext, runInContext } from 'vm';
import type { IConfig } from '../../types';
import { BoundedTtlMap } from '../../util/boundedTtlMap.js';
import { Span } from '../../util/span.js';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
const MIGRATIONS_DIR = resolve(__dirname, './migrations/sqlite');
const STATEMENT_CACHE_SIZE = 1000;
type SqliteDatabase = InstanceType<typeof import('better-sqlite3')>;
type SqliteStatement = import('better-sqlite3').Statement<unknown[]>;
/**
* Ordered list of [threshold_version, files[]] pairs.
* A database whose `user_version` is <= threshold_version will have
* the corresponding files applied.
* Ordered list of [threshold_version, files[]] pairs. A database whose
* `user_version` is <= threshold_version will have the corresponding files
* applied.
*/
const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[-1, ['0001_create-tables.sql', '0002_add-default-apps.sql']],
@@ -88,13 +94,60 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[52, ['0057_add_user_reputation.sql']],
[53, ['0058_add_phone_verification.sql']],
[54, ['0059_add_card_verification.sql']],
[55, ['0060_add_card_fingerprint.sql']],
[56, ['0061_add_suspended_at.sql']],
[57, ['0062_blocked-app-origins.sql']],
[58, ['0063_add_suspended_reason.sql']],
[59, ['0064_abuse-moderation-events.sql']],
[60, ['0065_app-feedback.sql']],
[61, ['0066_owned-email-unique.sql']],
[62, ['0067_share_entries.sql']],
[63, ['0068_referral-code-unique.sql']],
[64, ['0069_user-block.sql']],
[65, ['0070_drop-orphaned-default-groups.sql']],
[66, ['0071_share_issuer_index.sql']],
[67, ['0072_notification-scope.sql']],
[68, ['0073_notification-created-at.sql']],
[69, ['0074_add_user_home.sql']],
[70, ['0075_event-subscriptions.sql']],
[71, ['0076_event-handlers.sql']],
[72, ['0077_teams.sql']],
[73, ['0078_jct-user-group-pair-unique.sql']],
[74, ['0079_team-audit-and-group-shares.sql']],
[75, ['0080_kv-share-handles.sql']],
[76, ['0081_event-subscriptions-indexes.sql']],
[77, ['0082_temp-password-expiry.sql']],
[78, ['0083_team-directory.sql']],
[79, ['0084_share-anyone-with-link.sql']],
[80, ['0085_event-subscriptions-include-value.sql']],
[81, ['0086_subdomains-app-owner-cascade.sql']],
[82, ['0087_feedback-attachments.sql']],
[83, ['0088_apps-index-url.sql']],
[84, ['0089_team-require-2fa.sql']],
[85, ['0090_user-app-permission-index.sql']],
[86, ['0091_icloud-dot-significance.sql']],
[87, ['0092_share-entry-owner.sql']],
[88, ['0093_fsentries-parent-uid-index.sql']],
[89, ['0094_deleted-apps.sql']],
[90, ['0095_kv-share-handle-levels.sql']],
[91, ['0096_kv-share-handle-quota-indexes.sql']],
[92, ['0097_user-to-user-holder-permission-index.sql']],
[93, ['0098_notification-groups.sql']],
[94, ['0099_kv-share-handle-notify.sql']],
[95, ['0100_notification-send-key.sql']],
[96, ['0101_user-scoped-handlers.sql']],
[97, ['0102_notification-scope-id-index.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'sqlite';
// better-sqlite3 instance — set during onServerStart
private db!: InstanceType<typeof import('better-sqlite3')>;
private db!: SqliteDatabase;
/** Prepared statements by query text; SQLite re-prepares on schema change. */
private readonly statements = new BoundedTtlMap<string, SqliteStatement>({
maxEntries: STATEMENT_CACHE_SIZE,
});
constructor(config: IConfig) {
super(config);
@@ -116,12 +169,14 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
mkdirSync(dirname(dbPath), { recursive: true });
}
this.statements.clear();
this.db = new Database(dbPath);
await this.runMigrations(isNew);
}
override onServerShutdown(): void {
this.statements.clear();
if (this.db) {
this.db.close();
}
@@ -131,16 +186,14 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
// Query interface
// ------------------------------------------------------------------
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
query = this.transformQuery(query);
params = this.transformParams(params);
return this.db.prepare(query).all(...params) as Record<
string,
unknown
>[];
return this.prepare(query).all(
...this.transformParams(params),
) as Record<string, unknown>[];
}
override async pread(
@@ -151,30 +204,32 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
return this.read(query, params);
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
query = this.transformQuery(query);
params = this.transformParams(params);
const info = this.db.prepare(query).run(...params);
const info = this.prepare(query).run(...this.transformParams(params));
return {
insertId: info.lastInsertRowid,
affectedRows: info.changes,
anyRowsAffected: info.changes > 0,
// No warningStatus: SQLite raises none, but a new row can reuse a
// deleted row's rowid that other tables still name, so a clean
// insert doesn't prove the row is all the caller sent.
};
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(
entries: { statement: string; values: unknown[] }[],
): Promise<void> {
this.db.transaction(() => {
for (let { statement, values } of entries) {
statement = this.transformQuery(statement);
values = this.transformParams(values);
this.db.prepare(statement).run(...values);
for (const { statement, values } of entries) {
this.prepare(statement).run(...this.transformParams(values));
}
})();
}
@@ -183,6 +238,15 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
// SQLite-specific transforms
// ------------------------------------------------------------------
private prepare(query: string): SqliteStatement {
let statement = this.statements.get(query);
if (!statement) {
statement = this.db.prepare(this.transformQuery(query));
this.statements.set(query, statement);
}
return statement;
}
private transformQuery(query: string): string {
return query.replace(/now\(\)/gi, "datetime('now')");
}
+4 -4
View File
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -33,9 +33,9 @@ import { PostgresDatabaseClient } from './PostgresDatabaseClient';
import { SqliteDatabaseClient } from './SqliteDatabaseClient';
/**
* Factory class registered in `puterClients`. PuterServer calls
* `new DatabaseClientFactory(config)` — the constructor returns the
* concrete subclass selected by `config.database.engine`.
* Factory class registered in `puterClients`. PuterServer calls `new
* DatabaseClientFactory(config)` — the constructor returns the concrete
* subclass selected by `config.database.engine`.
*/
export const DatabaseClientFactory = class DatabaseClientFactory {
constructor(config: IConfig) {
@@ -1,4 +1,4 @@
/**
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
@@ -1060,6 +1060,7 @@ CREATE TABLE IF NOT EXISTS `user` (
`signup_server` varchar(255) DEFAULT NULL,
`metadata` json DEFAULT (json_object()),
`reputation` smallint DEFAULT '100',
`home` varchar(64) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uid` (`uuid`),
UNIQUE KEY `username` (`username`),
@@ -1151,6 +1152,7 @@ CALL _puter_add_col('user', 'signup_user_agent', '`signup_user_agent` varchar(51
CALL _puter_add_col('user', 'signup_origin', '`signup_origin` varchar(255) DEFAULT NULL');
CALL _puter_add_col('user', 'signup_server', '`signup_server` varchar(255) DEFAULT NULL');
CALL _puter_add_col('user', 'metadata', '`metadata` json DEFAULT (json_object())');
CALL _puter_add_col('user', 'home', '`home` varchar(64) DEFAULT NULL');
CALL _puter_add_col('user', 'reputation', '`reputation` smallint DEFAULT ''100''');
--
@@ -0,0 +1,46 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspended-at column. Mirrors SQLite migration 0061. `suspended_at` is when
-- the account was suspended, as unix seconds (NULL while not suspended) — the
-- timestamp sibling of the boolean `suspended` flag, indexed so the
-- signup-abuse harness can count an IP's recently-suspended accounts.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'suspended_at', '`suspended_at` bigint DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_user_suspended_at_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_suspended_at_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_suspended_at'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_suspended_at` (`suspended_at`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_suspended_at_index();
DROP PROCEDURE IF EXISTS _puter_add_user_suspended_at_index;
@@ -0,0 +1,36 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Admin-managed blocklist of app origins. Mirrors SQLite migration 0062.
-- An app whose `index_url` host (or a request origin) matches an entry is
-- denied access to Puter resources: it cannot obtain an app token and
-- already-issued app tokens are rejected on each request. `include_subdomains
-- = 1` also blocks every subdomain of `domain`. Enforced in AuthService via
-- AppOriginBlocklistService.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` lets the directory replay safely.
CREATE TABLE IF NOT EXISTS `blocked_app_origins` (
`id` INT NOT NULL AUTO_INCREMENT,
`domain` VARCHAR(255) NOT NULL,
`include_subdomains` TINYINT(1) NOT NULL DEFAULT 0,
`reason` TEXT DEFAULT NULL,
`created_by` VARCHAR(255) DEFAULT NULL,
`created_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_blocked_app_origins_domain` (`domain`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspension reason column. Mirrors SQLite migration 0063. Why an account was
-- suspended (NULL while not suspended) — companion to the boolean `suspended`
-- flag and `suspended_at` timestamp. Constrained at the application layer to a
-- fixed set of reasons (see extensions/admin suspension_reasons.js).
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations).
CALL _puter_add_col('user', 'suspended_reason', '`suspended_reason` VARCHAR(64) DEFAULT NULL');
@@ -0,0 +1,38 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Append-only log of admin moderation actions. Mirrors SQLite migration 0064.
-- Used to measure the abuse system's false-positive rate: `unsuspend` (admin
-- unblock) and `admin_create_user` are the false-positive signals; `suspend`
-- gives the denominator. Preserves history the `user` suspension columns lose
-- on unsuspend. `created_at` is unix seconds (matching `user.suspended_at`).
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` lets the directory replay safely.
CREATE TABLE IF NOT EXISTS `abuse_moderation_events` (
`id` INT NOT NULL AUTO_INCREMENT,
`action` VARCHAR(32) NOT NULL,
`target_user_id` BIGINT DEFAULT NULL,
`target_username` VARCHAR(255) DEFAULT NULL,
`admin_username` VARCHAR(255) DEFAULT NULL,
`reason` TEXT DEFAULT NULL,
`source` VARCHAR(64) DEFAULT NULL,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_abuse_moderation_events_created_at` (`created_at`),
KEY `idx_abuse_moderation_events_action` (`action`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,48 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- User-to-developer feedback for apps that opt in. Mirrors SQLite migration
-- 0065. Opt-in is the new `apps.feedback_enabled` column (developer-writable
-- through the regular `puter.apps.update` path). Each row of `app_feedback`
-- is one message a signed-in user submitted through the GUI feedback dialog;
-- a copy is emailed to the app owner unless the per-app daily email cap
-- suppressed it (`email_sent` records which). `app_uid` is denormalized
-- alongside `app_id` so rows stay attributable after an app is deleted.
-- `created_at` is unix seconds.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- always runs first) and the table uses `CREATE TABLE IF NOT EXISTS`, so the
-- directory can replay safely.
CALL _puter_add_col('apps', 'feedback_enabled', '`feedback_enabled` tinyint(1) DEFAULT ''0''');
CREATE TABLE IF NOT EXISTS `app_feedback` (
`id` INT NOT NULL AUTO_INCREMENT,
`uid` CHAR(36) NOT NULL,
`app_id` INT NOT NULL,
`app_uid` CHAR(40) NOT NULL,
`user_id` INT NOT NULL,
`message` TEXT NOT NULL,
`source_env` VARCHAR(16) DEFAULT NULL,
`source_origin` VARCHAR(2048) DEFAULT NULL,
`email_sent` TINYINT(1) NOT NULL DEFAULT 0,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uq_app_feedback_uid` (`uid`),
KEY `idx_app_feedback_app_created` (`app_id`, `created_at`),
KEY `idx_app_feedback_user_created` (`user_id`, `created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,128 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce "at most one account owns an email address". Mirrors SQLite
-- migration 0066.
--
-- `user.email` is deliberately not UNIQUE: several rows may legitimately hold
-- the same address while unconfirmed (admin-provisioned placeholders, signups
-- that were never confirmed, temp accounts on their way to becoming real). What
-- must never happen is two rows both *owning* an address — owning meaning the
-- row is confirmed, or holds a password and so can drive password recovery for
-- that inbox.
--
-- Signup, save-account, change-email, OIDC and admin provisioning each check for
-- an owner before writing, but a check and an insert are not one operation: two
-- requests can both read "free" and both write. This index is what actually
-- holds the invariant; the application checks just produce a nicer error most of
-- the time.
--
-- SQLite expresses that with a partial index. MySQL has none, so the predicate
-- lives in a generated column that evaluates to NULL for every row that does not
-- own its address — and NULLs do not collide in an InnoDB unique index, which is
-- exactly the "unlimited unconfirmed placeholders" behaviour we need.
--
-- The column is VIRTUAL, not STORED, on purpose: adding a stored generated
-- column rebuilds the table, while a virtual one is a metadata-only change and
-- the index that follows builds INPLACE. On a `user` table of any size with
-- read replicas attached, that is the difference between a routine change and an
-- outage. The ALGORITHM/LOCK clauses are spelled out so a server that cannot
-- honour them refuses the statement instead of quietly copying the table.
--
-- Matching is on the canonical address so provider aliases
-- (`[email protected]` vs `[email protected]`) collide. `clean_email` is
-- written on every modern write path; the COALESCE covers rows old enough to
-- predate the column. Run the `clean_email` backfill before this migration or
-- alias collisions among those rows go unnoticed.
--
-- Idempotent: both steps are guarded on INFORMATION_SCHEMA so the directory
-- replays safely.
--
-- If the index creation fails with ER_DUP_ENTRY, the DB already contains
-- duplicate owners. Collapse them first (admin → One-off Jobs → Collapse
-- Duplicate Emails) — there is no safe automatic merge of two accounts.
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
DELIMITER //
CREATE PROCEDURE _puter_add_owned_email()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND COLUMN_NAME = 'owned_email'
) THEN
ALTER TABLE `user`
ADD COLUMN `owned_email` VARCHAR(256)
CHARACTER SET latin1 COLLATE latin1_swedish_ci
GENERATED ALWAYS AS (
CASE
WHEN `email` IS NOT NULL
AND (`email_confirmed` = 1 OR `password` IS NOT NULL)
THEN COALESCE(`clean_email`, LOWER(`email`))
ELSE NULL
END
) VIRTUAL,
ALGORITHM=INSTANT;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_owned_email'
) THEN
ALTER TABLE `user`
ADD UNIQUE KEY `idx_user_owned_email` (`owned_email`),
ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_owned_email();
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
-- One Puter account per external identity. OIDCStore.link already assumes this
-- constraint exists — it catches the unique violation to tell "re-linking the
-- same account" apart from "this sub belongs to someone else" — but the table
-- never actually had it, so two concurrent first-time logins could each create
-- an account and each link the same sub. Subsequent logins then resolved to
-- whichever row came back first.
--
-- Dedupe `user_oidc_providers` before applying this: keep the lowest `id` per
-- (provider, provider_sub) and point it at the account the collapse job kept.
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
DELIMITER //
CREATE PROCEDURE _puter_add_oidc_sub_unique()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_oidc_providers'
AND INDEX_NAME = 'idx_user_oidc_provider_sub'
) THEN
ALTER TABLE `user_oidc_providers`
ADD UNIQUE KEY `idx_user_oidc_provider_sub` (`provider`, `provider_sub`);
END IF;
END//
DELIMITER ;
CALL _puter_add_oidc_sub_unique();
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
@@ -0,0 +1,106 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Grow `share` from a pending-email-invite table into the index of active
-- shares. See sqlite/0067_share_entries.sql for the column rationale.
--
-- Idempotent: columns go through _puter_add_col (from mig_1), indexes and
-- foreign keys through the guarded procedure below. There is no per-file
-- applied-state tracking, so every statement must tolerate a re-run.
CALL _puter_add_col('share', 'holder_user_id', '`holder_user_id` int unsigned DEFAULT NULL');
CALL _puter_add_col('share', 'fsentry_id', '`fsentry_id` int unsigned DEFAULT NULL');
CALL _puter_add_col('share', 'mode', '`mode` varchar(20) DEFAULT NULL');
CALL _puter_add_col('share', 'applied_at', '`applied_at` timestamp NULL DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
DELIMITER //
CREATE PROCEDURE _puter_add_share_index_constraints()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_holder` (`holder_user_id`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_fsentry'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_fsentry` (`fsentry_id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_entry_issuer'
) THEN
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_holder_entry_issuer`
(`holder_user_id`, `fsentry_id`, `issuer_user_id`);
END IF;
-- Retiring a deleted node's grants looks them up by permission text
-- (`fs:<uuid>` and `fs:<uuid>:%`); the subject lives in that text rather
-- than a column, so no foreign key can cascade it. The primary key is
-- (issuer, holder, permission), which never puts `permission` first, so
-- without this both the equality and the left-anchored LIKE degrade to a
-- full scan on the delete path. The column is ascii varchar(255), well
-- inside InnoDB's key limit, so it is indexed whole.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_to_user_permissions'
AND INDEX_NAME = 'idx_user_to_user_permissions_permission'
) THEN
ALTER TABLE `user_to_user_permissions`
ADD INDEX `idx_user_to_user_permissions_permission` (`permission`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_holder_user_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_holder_user_fk`
FOREIGN KEY (`holder_user_id`) REFERENCES `user` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
-- The cascade that retires a share with its file.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_fsentry_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_fsentry_fk`
FOREIGN KEY (`fsentry_id`) REFERENCES `fsentries` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END//
DELIMITER ;
CALL _puter_add_share_index_constraints();
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- One user refusing contact from another. See sqlite/0068_user-block.sql for
-- the rationale. `created_at` is unix seconds.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- so the directory can replay safely.
CREATE TABLE IF NOT EXISTS `user_block` (
`id` INT NOT NULL AUTO_INCREMENT,
`blocker_user_id` INT UNSIGNED NOT NULL,
`blocked_user_id` INT UNSIGNED NOT NULL,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_user_block_pair` (`blocker_user_id`, `blocked_user_id`),
CONSTRAINT `fk_user_block_blocker` FOREIGN KEY (`blocker_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_user_block_blocked` FOREIGN KEY (`blocked_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,61 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Widen `notification.shown` and `notification.acknowledged` to hold the unix
-- second the store has been writing to them since the backend rework.
--
-- Both arrived from the v1 schema as `tinyint(1)`, where they were flags. The
-- rework changed the writes to timestamps and sqlite (`INTEGER`) and postgres
-- (`bigint`) took them; only mysql was left too narrow, so every `markShown`
-- and `markAcknowledged` there fails with ER_WARN_DATA_OUT_OF_RANGE and the
-- column stays NULL. Dismissing a notification therefore never sticks, and one
-- already delivered is re-sent on every reconnect.
--
-- No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a legacy
-- `1` keeps meaning "yes" once widened.
--
-- Guarded on the current type rather than run unconditionally. Changing a
-- column type copies the table, and this one grows without bound — a migration
-- directory that replays on every boot must not rebuild it every time.
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
DELIMITER //
CREATE PROCEDURE _puter_widen_notification_stamps()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND COLUMN_NAME = 'shown'
AND DATA_TYPE = 'tinyint'
) THEN
ALTER TABLE `notification` MODIFY `shown` BIGINT DEFAULT NULL;
END IF;
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND COLUMN_NAME = 'acknowledged'
AND DATA_TYPE = 'tinyint'
) THEN
ALTER TABLE `notification` MODIFY `acknowledged` BIGINT DEFAULT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_notification_stamps();
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
@@ -0,0 +1,22 @@
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
-- `dangerous`. No migration creates them — they were added by hand back when
-- hardcoded permissions were keyed by group name. That map is now a flat
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
-- so a group nothing looks up grants nothing, and the rows are dead weight.
--
-- Guarded, and deliberately so: `user_to_group_permissions.group_id` and
-- `jct_user_group.group_id` both cascade on delete, so removing a group that
-- still carries permissions or members would silently revoke them from every
-- member. Only a group with neither is dropped. A group that survives this
-- migration has dependents and needs a deliberate decision, not a sweep --
-- query `user_to_group_permissions` by `group_id` to see what it holds.
--
-- The joins rather than `NOT IN` subqueries: MySQL will not read from the table
-- it is deleting from inside a subquery on the same statement.
DELETE g FROM `group` g
LEFT JOIN `user_to_group_permissions` p ON p.`group_id` = g.`id`
LEFT JOIN `jct_user_group` j ON j.`group_id` = g.`id`
WHERE JSON_UNQUOTE(JSON_EXTRACT(g.`extra`, '$.name'))
IN ('freeai', 'experimental', 'dangerous')
AND p.`group_id` IS NULL
AND j.`group_id` IS NULL;
@@ -0,0 +1,80 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Give `notification` a scope tuple: what the row is (`type`), the app it is
-- about (`app_uid`, NULL for platform), and who the recipient is in relation
-- to it (`audience`). Attribution lived inside the `value` JSON as two
-- free-form markers, `source` and `template`, that could disagree; these
-- columns replace them and are what a scoped list can be indexed on.
--
-- `app_uid` is char(40) to match `apps`.`uid`, which is `app-` plus a uuid.
--
-- Idempotent: columns go through `_puter_add_col` (mig_1), the index through
-- the guarded procedure. There is no per-file applied-state tracking, so every
-- statement here tolerates a re-run.
CALL _puter_add_col('notification', 'app_uid', '`app_uid` char(40) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
CALL _puter_add_col('notification', 'audience', '`audience` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT ''account''');
CALL _puter_add_col('notification', 'type', '`type` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT ''''');
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_scope_index()
BEGIN
-- Serves both the scoped list and the unacknowledged count.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_scope'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_scope`
(`user_id`, `audience`, `app_uid`, `acknowledged`);
END IF;
END //
DELIMITER ;
CALL _puter_add_notification_scope_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_scope_index;
-- Backfill from the markers the payload actually carried. `type = ''` is the
-- not-yet-classified state, so re-running matches nothing the first pass
-- claimed, and a row matching no marker keeps the defaults (`''` / 'account')
-- and reads as legacy.
UPDATE `notification`
SET `type` = 'share.received', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.template')) = 'file-shared-with-you';
UPDATE `notification`
SET `type` = 'share.claimed', `audience` = 'account', `app_uid` = NULL
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.template')) = 'file-shared-before-you-joined';
-- Worker rows carried only `source: 'worker'` and a title, so the app they
-- belong to is unrecoverable and they stay unattributed. The title prefix is
-- the sole surviving signal of which way the deploy went.
UPDATE `notification`
SET `audience` = 'developer',
`app_uid` = NULL,
`type` = CASE
WHEN JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.title')) LIKE 'Successfully deployed %'
THEN 'app.worker.deployed'
ELSE 'app.worker.deployFailed'
END
WHERE `type` = ''
AND JSON_UNQUOTE(JSON_EXTRACT(`value`, '$.source')) = 'worker';
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Index `notification`.`created_at`. The retention sweep selects by age alone
-- and by nothing else, so no existing index on the table narrows it — every
-- pass would otherwise scan the whole table to find the oldest few hundred
-- rows.
--
-- Idempotent: the guarded procedure, as mig_26. There is no per-file
-- applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_add_notification_created_at_index;
DELIMITER //
CREATE PROCEDURE _puter_add_notification_created_at_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND INDEX_NAME = 'idx_notification_created_at'
) THEN
ALTER TABLE `notification` ADD INDEX `idx_notification_created_at`
(`created_at`);
END IF;
END //
DELIMITER ;
CALL _puter_add_notification_created_at_index();
DROP PROCEDURE IF EXISTS _puter_add_notification_created_at_index;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Home region column. Mirrors SQLite migration 0074. Home region this
-- account's data belongs in — a node id, not a storage region. NULL for
-- accounts that predate the column, which fall back to `signup_server`.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations).
CALL _puter_add_col('user', 'home', '`home` VARCHAR(64) DEFAULT NULL');
@@ -0,0 +1,59 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Subscriptions that outlive the connection that made them. See
-- sqlite/0075_event-subscriptions.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_23. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `event_subscriptions` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`sub_id` varchar(80) NOT NULL,
`token` varchar(255) NOT NULL,
`owner_user_id` int unsigned NOT NULL,
`holder_user_id` int unsigned NOT NULL,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion. No foreign key: a row outlives the
-- app that made it, which is what keeps it revocable afterwards.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci
DEFAULT NULL,
`subject` varchar(4096) NOT NULL,
`anchor_uid` char(36) NOT NULL,
`anchor_path` varchar(4096) NOT NULL,
`match` varchar(1024) DEFAULT NULL,
`delivery` varchar(16) NOT NULL,
`ops` varchar(64) DEFAULT NULL,
`handler_name` varchar(128) DEFAULT NULL,
`targets` json NOT NULL,
`context` text,
`permission` varchar(32) NOT NULL,
`suspended_at` bigint DEFAULT NULL,
`suspended_reason` varchar(64) DEFAULT NULL,
`expires_at` bigint DEFAULT NULL,
`created_at` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_subscriptions_sub_id` (`sub_id`),
KEY `idx_event_subscriptions_token` (`token`),
KEY `idx_event_subscriptions_holder` (`holder_user_id`, `app_uid`),
KEY `idx_event_subscriptions_owner` (`owner_user_id`),
CONSTRAINT `fk_event_subscriptions_owner` FOREIGN KEY (`owner_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_event_subscriptions_holder` FOREIGN KEY (`holder_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,43 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions`.`anchor_uid` was sized for a filesystem node uuid.
-- A key-value subscription anchors on an app instead, and `apps`.`uid` is
-- `char(40)`, so the column has to hold one.
--
-- Idempotent: the guarded procedure, as mig_27. There is no per-file
-- applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_widen_event_subscriptions_anchor_uid;
DELIMITER //
CREATE PROCEDURE _puter_widen_event_subscriptions_anchor_uid()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'anchor_uid'
AND CHARACTER_MAXIMUM_LENGTH < 40
) THEN
ALTER TABLE `event_subscriptions`
MODIFY COLUMN `anchor_uid` varchar(40) NOT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_event_subscriptions_anchor_uid();
DROP PROCEDURE IF EXISTS _puter_widen_event_subscriptions_anchor_uid;
@@ -0,0 +1,37 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Named handlers an app deploys once. See sqlite/0076_event-handlers.sql for
-- the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_29. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `event_handlers` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`name` varchar(128) NOT NULL,
`source` mediumtext NOT NULL,
`source_hash` char(64) NOT NULL,
`created_at` bigint NOT NULL,
`updated_at` bigint NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_event_handlers_app_name` (`app_uid`, `name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,58 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Teams. See sqlite/0077_teams.sql for the column rationale.
-- No per-file applied-state tracking, so every statement must tolerate a re-run:
-- columns go through _puter_add_col, indexes through the guarded procedure below.
CALL _puter_add_col('group', 'kind', '`kind` varchar(20) DEFAULT NULL');
CALL _puter_add_col('group', 'name', '`name` varchar(255) DEFAULT NULL');
CALL _puter_add_col('group', 'handle', '`handle` varchar(64) DEFAULT NULL');
CALL _puter_add_col('group', 'deleted_at', '`deleted_at` timestamp NULL DEFAULT NULL');
CALL _puter_add_col('jct_user_group', 'org_owned', '`org_owned` tinyint(1) DEFAULT NULL');
CALL _puter_add_col('user', 'requires_password_change', '`requires_password_change` tinyint(1) NOT NULL DEFAULT ''0''');
DROP PROCEDURE IF EXISTS _puter_add_team_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_team_indexes()
BEGIN
-- Already case-insensitive: the table is utf8mb4_unicode_ci. NULLs stay distinct.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'group'
AND INDEX_NAME = 'idx_group_handle'
) THEN
ALTER TABLE `group` ADD UNIQUE INDEX `idx_group_handle` (`handle`);
END IF;
-- Composite, unlike the existing single-column `group_id` key.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'jct_user_group'
AND INDEX_NAME = 'idx_jct_user_group_group'
) THEN
ALTER TABLE `jct_user_group`
ADD INDEX `idx_jct_user_group_group` (`group_id`, `user_id`);
END IF;
END//
DELIMITER ;
CALL _puter_add_team_indexes();
DROP PROCEDURE IF EXISTS _puter_add_team_indexes;
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Deduplicate then constrain. See sqlite/0075 for why this is lossless.
-- Both steps sit behind the index check: mysql re-runs this file on every boot, and
-- guarding them together stops a rolling deploy deleting and indexing concurrently.
DROP PROCEDURE IF EXISTS _puter_add_membership_pair_index;
DELIMITER //
CREATE PROCEDURE _puter_add_membership_pair_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'jct_user_group'
AND INDEX_NAME = 'idx_jct_user_group_pair'
) THEN
-- Self-join because mysql refuses `NOT IN (SELECT ... FROM jct_user_group)` (1093).
DELETE dup FROM `jct_user_group` dup
JOIN `jct_user_group` keep
ON dup.`user_id` = keep.`user_id`
AND dup.`group_id` = keep.`group_id`
AND dup.`id` > keep.`id`;
ALTER TABLE `jct_user_group`
ADD UNIQUE INDEX `idx_jct_user_group_pair` (`user_id`, `group_id`);
END IF;
END//
DELIMITER ;
CALL _puter_add_membership_pair_index();
DROP PROCEDURE IF EXISTS _puter_add_membership_pair_index;
@@ -0,0 +1,84 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0076. Types follow mysql_mig_1; the guarded procedure follows mysql_mig_22.
CREATE TABLE IF NOT EXISTS `audit_team_membership` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`group_id` int unsigned DEFAULT NULL,
`group_id_keep` int unsigned NOT NULL,
`user_id` int unsigned DEFAULT NULL,
`user_id_keep` int unsigned NOT NULL,
`actor_user_id` int unsigned DEFAULT NULL,
`action` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`reason` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idx_audit_team_membership_group` (`group_id_keep`, `id`),
KEY `idx_audit_team_membership_user` (`user_id_keep`, `id`),
KEY `idx_audit_team_membership_group_fk` (`group_id`),
KEY `idx_audit_team_membership_user_fk` (`user_id`),
KEY `idx_audit_team_membership_actor_fk` (`actor_user_id`),
-- SET NULL, never CASCADE: deleting an account must not erase what was done to it.
CONSTRAINT `fk_audit_team_membership_group` FOREIGN KEY (`group_id`)
REFERENCES `group` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_audit_team_membership_user` FOREIGN KEY (`user_id`)
REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_audit_team_membership_actor` FOREIGN KEY (`actor_user_id`)
REFERENCES `user` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
CALL _puter_add_col('share', 'holder_group_id', '`holder_group_id` int unsigned DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_group_share_constraints;
DELIMITER //
CREATE PROCEDURE _puter_add_group_share_constraints()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_group'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_holder_group` (`holder_group_id`, `id`);
END IF;
-- Team shares leave `holder_user_id` NULL, so the existing unique index binds none.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_group_entry_issuer'
) THEN
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_holder_group_entry_issuer`
(`holder_group_id`, `fsentry_id`, `issuer_user_id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_holder_group_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_holder_group_fk`
FOREIGN KEY (`holder_group_id`) REFERENCES `group` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END//
DELIMITER ;
CALL _puter_add_group_share_constraints();
DROP PROCEDURE IF EXISTS _puter_add_group_share_constraints;
@@ -0,0 +1,68 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Opaque names for shared regions of a user's key-value namespace. See
-- sqlite/0080_kv-share-handles.sql for the column rationale.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- as mig_29. There is no per-file applied-state tracking, so a replay has to
-- be a no-op.
CREATE TABLE IF NOT EXISTS `kv_share_handles` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`handle` varchar(64) NOT NULL,
`owner_user_id` int unsigned NOT NULL,
`grantee_user_id` int unsigned NOT NULL,
-- Matches `apps`.`uid` exactly, charset included, so an equality against
-- one never falls back to a conversion. No foreign key: a handle outlives
-- the app it was minted against, which is what keeps it revocable.
`app_uid` char(40) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`key_prefix` varchar(1024) NOT NULL,
`permission` varchar(1024) NOT NULL,
`created_at` bigint NOT NULL,
`revoked_at` bigint DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_kv_share_handles_handle` (`handle`),
KEY `idx_kv_share_handles_owner` (`owner_user_id`, `id`),
CONSTRAINT `fk_kv_share_handles_owner` FOREIGN KEY (`owner_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_kv_share_handles_grantee` FOREIGN KEY (`grantee_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- A subscription made through a handle stores the grant string it was
-- authorized under rather than an access mode, and a grant string carries a
-- user uuid, an app uid and a key prefix. Guarded on the current length, as
-- mig_24: replaying a column change on a growing table rebuilds it every boot.
DROP PROCEDURE IF EXISTS _puter_widen_subscription_permission;
DELIMITER //
CREATE PROCEDURE _puter_widen_subscription_permission()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND COLUMN_NAME = 'permission'
AND CHARACTER_MAXIMUM_LENGTH < 1024
) THEN
ALTER TABLE `event_subscriptions` MODIFY `permission` varchar(1024) NOT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_subscription_permission();
DROP PROCEDURE IF EXISTS _puter_widen_subscription_permission;
@@ -0,0 +1,60 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- `event_subscriptions` grew three hot queries with no index behind them:
-- a handler's own rows, the expiry reaper and the suspended reaper. See
-- sqlite/0081_event-subscriptions-indexes.sql for the query shapes.
--
-- Idempotent: the guarded procedure, as mig_30/mig_34/mig_35. There is no
-- per-file applied-state tracking, so a replay has to be a no-op.
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_indexes;
DELIMITER //
CREATE PROCEDURE _puter_add_event_subscriptions_indexes()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND INDEX_NAME = 'idx_event_subscriptions_app_handler'
) THEN
ALTER TABLE `event_subscriptions`
ADD INDEX `idx_event_subscriptions_app_handler` (`app_uid`, `handler_name`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND INDEX_NAME = 'idx_event_subscriptions_expires'
) THEN
ALTER TABLE `event_subscriptions`
ADD INDEX `idx_event_subscriptions_expires` (`expires_at`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'event_subscriptions'
AND INDEX_NAME = 'idx_event_subscriptions_suspended'
) THEN
ALTER TABLE `event_subscriptions`
ADD INDEX `idx_event_subscriptions_suspended` (`suspended_at`, `id`);
END IF;
END//
DELIMITER ;
CALL _puter_add_event_subscriptions_indexes();
DROP PROCEDURE IF EXISTS _puter_add_event_subscriptions_indexes;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0082_temp-password-expiry.sql for the column rationale.
-- No per-file applied-state tracking, so the column goes through _puter_add_col.
CALL _puter_add_col('user', 'temp_password_expires_at', '`temp_password_expires_at` bigint DEFAULT NULL');
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0083_team-directory.sql for the column rationale.
-- No per-file applied-state tracking, so the column goes through _puter_add_col.
CALL _puter_add_col('group', 'directory_enabled', '`directory_enabled` tinyint(1) NOT NULL DEFAULT 0');
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- See sqlite/0084_share-anyone-with-link.sql for the column rationale.
-- No per-file applied-state tracking, so the column goes through _puter_add_col
-- and the index through a guarded procedure, as mysql_mig_34 does.
CALL _puter_add_col('share', 'anyone', '`anyone` tinyint(1) DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_share_anyone_index;
DELIMITER //
CREATE PROCEDURE _puter_add_share_anyone_index()
BEGIN
-- Every other row leaves `anyone` NULL, so this binds only link shares.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_anyone'
) THEN
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_anyone` (`fsentry_id`, `anyone`);
END IF;
END//
DELIMITER ;
CALL _puter_add_share_anyone_index();
DROP PROCEDURE IF EXISTS _puter_add_share_anyone_index;
Loaded 100 of 2547 files, more files were not shown because too many files have changed in this diff. Show more