Compare commits

...
Author SHA1 Message Date
Nariman Jelveh a290dd446f feat: puter.ui.setURLParams() for app deep links
Apps could read a deep link's params at launch but had no way to write
the browser URL, so app state was never shareable or bookmarkable.

setURLParams() lets an app own the query string of its own /app/<name>
URL. The path segment is always built from the window's own data-app,
never from app input, so an app can decorate its own URL but cannot make
the address bar name another app or a Puter route. Writes use
replaceState, so apps cannot mint history entries and Back keeps meaning
"leave the app".

Params Puter itself interprets on a page load are refused: a shareable
link is exactly where they would do damage (auth_token would turn an
app's "copy link" button into a session-fixation link, api_origin would
point token traffic elsewhere, embedded_in_popup diverts boot into the
popup-auth path, error_from_within_iframe makes the backend serve an
error page). The deny-list carries an audit rule, since these are read
from three different places: url_query_params, a direct location.search
parse during boot-mode selection, and req.query server-side.

Ownership is enforced GUI-side and re-checked when a paced write lands:
dashboard mode only, never explorer, the window must claim the URL and
not be minimized. Writes are coalesced per app name at 500ms, kept
under WebKit's history budget so a spamming app cannot exhaust what the
GUI needs for its own pushes.

Also fixes four pre-existing bugs this surfaced:

- An unguarded pushState aborted UIWindow() mid-build when a browser
  refused the write, leaving a headless dashboard window with no
  minimize or close control.
- A dashboard tile click focused an already-open app without re-claiming
  the URL, so the address bar kept naming the app the user just left.
- push_dashboard_app_url's dedup branch left dashboard_url_pop_pending
  set, so restoring an app inside the pop watchdog's window got it
  re-minimized.
- UIDesktop's posargs JSON.parse was unguarded, matching the guard
  initgui.js already had.
2026-07-26 20:00:07 -07:00
Nariman Jelveh 1e280601c6 fix: showSaveFilePicker from external websites (#3450)
Saving a file via puter.ui.showSaveFilePicker from a third-party website
(popup flow) consistently failed with a DataCloneError alert after
clicking Save, and saving over an existing filename showed a raw error
instead of the Replace/Cancel prompt.

Two bugs:

1. privacy_aware_path is a curried factory (world => fspath => ...), and
   initgui.js is the only module that imports it directly — so the popup
   save handler's privacy_aware_path(res.path) returned the inner
   function, which postMessage cannot structured-clone. Every other call
   site resolves the bare name to the correctly bound
   window.privacy_aware_path global, which is why only the external-site
   popup flow was broken. Use the global at the call site and import the
   factory under a distinct name so a bare call can't silently resolve
   to it again.

2. The v2 backend returns `conflict` for a same-name write, but the v1
   wire contract is `item_with_same_name_exists` + `entry_name`, which
   the GUI's save dialogs key on to offer the overwrite prompt. Restore
   the legacy code/field on the write-conflict error and carry
   HttpError.fields through the /batch per-op error serializer.

Verified end-to-end against a local backend: fresh save resolves the
caller's promise with the signed saved_file and closes the popup;
saving an existing name shows Replace/Cancel and Replace overwrites.
Backend suite shows no new failures.
2026-07-26 11:28:38 -07:00
Daniel Salazar 2262975785 fix: readdir response type (#3449) 2026-07-25 18:06:50 -07:00
Daniel Salazar e6e6e3ba9a chore: cleanup API driver calls PUT-1324 (#3448) 2026-07-25 18:05:36 -07:00
Daniel Salazar e85cd9d53d feat: readdir with depth (#3446) 2026-07-25 14:04:21 -07:00
Nariman Jelveh 828b5d9430 Fix popup file pickers rendering an empty explorer
The cover-page rule added for the auth windows forced `width: 100%` on
every `.window-cover-page .window-body`, including the popup file,
directory, and save pickers. That body is an item-container with
`overflow: auto`, so it establishes its own formatting context and may
not overlap the floated `.window-sidebar`. Unable to shrink to fit, it
dropped below the sidebar, pushing the file list and the Cancel/Open bar
outside the 700x400 popup and leaving the page background showing
through as grey.

`.window-body-filedialog { width: auto !important }` was meant to cover
this but loses on specificity, one class against two. Scope the auth
rule with `:not(.window-filedialog)` instead; that class is only set for
the three dialog kinds, so no auth or session cover page changes.
2026-07-25 13:02:15 -07:00
jelveh 3b1ba72ddd Share item-added updates across GUI shells
Extracted desktop `item.added` container update logic into a shared `apply_item_added_to_containers` helper and wired it into both `UIDesktop` and `UIDashboard`. The helper handles overwrite updates, new item insertion, icon refresh, and resorting for matching item containers. This fixes dashboard-hosted explorer/file dialogs not reflecting uploads until navigation.
2026-07-24 23:01:10 -07:00
jelveh 4245ac04ac Isolate upload dialog to prevent double uploads
Replace reuse of `#upload-file-dialog` with a dedicated hidden file input owned by `init_upload_using_dialog`. This avoids collisions with the dashboard Files tab’s `onchange` property handler, which could trigger duplicate concurrent uploads and "Entry already exists" races. The new flow stores the current target path once per dialog open, clears input value before click (so re-selecting the same file still fires `change`), and snapshots `FileList` to an array before clearing to keep async uploads reliable.
2026-07-24 18:26:43 -07:00
Daniel Salazar b8559c1221 chore: cleanup other puter-js modules to match new structure (#3440) 2026-07-24 17:42:06 -07:00
Daniel Salazar 6c4fa629a9 fix: allow root token to also call ai drivers (#3442) 2026-07-24 17:41:38 -07:00
jelveh e6531d4357 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-24 15:38:38 -07:00
jelveh 0c27571030 Dashboard files: compact drag ghost + spring-loaded nav controls
- Replace the full-tile drag ghost with a compact icon+name chip:
  stacked sheets and a count badge for multi-selections, dimmed
  source rows, same design in list and grid views.
- Holding a dragged item over back/forward/up or a breadcrumb
  segment now spring-navigates there (700ms dwell); cancelling
  the drag returns to the original directory.
2026-07-24 15:37:33 -07:00
Neal Shah 6e6d9d2869 buildXhr, dedup, retry (#3432)
* buildXhr, dedup, retry

* change retry intervals
2026-07-24 18:37:09 -04:00
Nariman Jelveh 34529b2e79 Dashboard files: replace solid-blue selection with tinted accent wash
Selected tiles/rows now use a subtle accent tint plus hairline ring
instead of a solid --select-color fill, so icons and labels keep their
natural colors in both themes. Also refines the grid tile's hover
'...' button (inset, hairline border, hover state), fixes the
light-mode selected-row text color that referenced a dark-mode-only
variable, and drops the opaque icon chip visible on tinted list rows.
2026-07-24 13:09:31 -07:00
Nariman Jelveh 7e4ac6542e Update TabApps.js 2026-07-24 12:42:17 -07:00
Daniel Salazar 3331dbd464 chore: make fs upload cleaner in puter.js (#3436) 2026-07-24 12:18:19 -07:00
Daniel Salazar 391b175a68 fix: remove deprecated claude models (#3438) 2026-07-24 10:54:01 -07:00
Neal Shah 19f35b4200 add opus 5 (#3437) 2026-07-24 13:49:20 -04:00
jelveh 9877bb79d0 Polish mobile dashboard sidebar drawer UX
Refactors the dashboard’s mobile sidebar into a modal-style drawer with a coordinated state helper, scrim overlay, and dedicated close button. Adds accessibility improvements (`aria-label`, `aria-expanded`) and stronger dismissal behavior (scrim tap, close button, Escape, and swipe-left with velocity/threshold handling). Updates mobile CSS for safe-area-aware positioning, improved touch targets/pressed states, frosted toggle styling, drawer shadows/transitions, and drag-state handling for smoother interactions.
2026-07-23 18:56:43 -07:00
Nariman Jelveh 2982e894b1 Update dashboard.css 2026-07-23 18:14:47 -07:00
Nariman Jelveh 10bdf701e0 Dashboard: start the minimize morph immediately, bookkeep the URL behind it
The minimize controls consumed the app's /app/<name> history entry
first and let the popstate handler do the visible hide. When the app's
iframe had stacked joint session-history entries, the pop only settles
at pop_dashboard_app_url's 400ms watchdog — so clicking minimize sat
still for ~half a second before the morph began.

Hide the window in the click frame instead (minimize_window, shared by
the head button, the context menu, and the control drawer) and demote
the pop to pure URL bookkeeping. The eager hide is safe against both
ways the pop settles: every hideWindow branch marks data-is_minimized
synchronously, and the popstate handler and the watchdog both already
skip minimized windows. The browser's Back button still minimizes
through the popstate handler as before.
2026-07-23 17:50:32 -07:00
Nariman Jelveh e764a10ee4 Dashboard: recover minimize/close when history.back() misses the app entry
Minimize (and close's URL cleanup) on headless dashboard app windows
work by consuming the app's /app/<name> history entry with
history.back() and letting the popstate handler do the actual hide.
But back() traverses the JOINT session history, which the app's iframe
shares: if the app navigated internally after load (SPA router,
redirects — and browsers differ on which iframe navigations stack
joint entries), the back() consumes the IFRAME's entry instead. The
parent's URL never changes, no popstate arrives, and the
dashboard_url_pop_pending latch stays stuck — every later minimize and
close becomes a silent no-op, and the stranded /app/<name> entry makes
later traversals resurrect windows the user already dismissed.

Fix with a watchdog after each back(): a successful pop is a
same-document traversal of the parent (app entries are pushState over
the dashboard route), so its popstate lands well inside 400ms and the
watchdog no-ops. If the URL still names the app by then, do what the
popstate would have done — minimize through the same lookup and
minimized-guard — and replaceState the stranded entry back to the
dashboard's own route (captured at push time, tab hash included) so no
stale entry survives.
2026-07-23 17:23:03 -07:00
Neal Shah 7e461e14a2 remove unneeded write origin append (#3434) 2026-07-23 19:09:23 -04:00
Daniel Salazar 48cc706ad3 feat: paginated fetching for all (#3431)
* feat: paginated fetching for all

* fix: metering top up gui reporting
2026-07-23 15:07:58 -07:00
Daniel Salazar 20ea616b59 chore: cleanup kv module (#3430) 2026-07-23 10:05:19 -07:00
Nariman Jelveh 73a7a38ea8 Dashboard: redesign app control pill as a top-edge drawer (#3427)
* Dashboard: redesign app control pill as a top-edge drawer

The floating control capsule over headless dashboard apps consumed
noticeable space, especially on mobile. Replace it with a drawer: a
slim glass tray flush with the top edge (icon + title + minimize +
close) whose tongue-shaped handle hangs from its bottom center. At
rest only the tongue peeks in from the edge; hovering it (mouse),
tapping it, or keyboard-focusing the drawer slides the tray down.
The collapse is a single GPU transform instead of the pill's
max-width squeezes, and the open-then-retract intro now visibly
retracts into the tongue, teaching where the controls live.

Hover-expansion is gated on pointerType === 'mouse': touch taps
synthesize a pointerenter right before click, which made the old
pill's toggle see an already-open state and shut it again.

Also fixes two window-stacking bugs surfaced while testing:

- showWindow demoted stay_on_top windows (every fullpage/dashboard
  app window, created in the 99999999+ z band) to a plain counter
  z-index on restore. focusWindow deliberately never raises
  stay_on_top windows, so after minimize/restore a single dashboard
  focus buried the app permanently. Restore now re-raises into the
  stay-on-top band.

- Pressing the drawer now focuses its window, as the titlebar it
  replaces did. The document-level activation handler works off
  mouseover_window, which only mousemove refreshes — a tap with no
  intervening mousemove (touch, restored windows) would re-raise
  the dashboard over the app.

* Dashboard: restored windows come back with the drawer collapsed

The restore re-played the open intro (expand + auto-retract). A restore
should bring back the app, not the chrome — the user already met the
controls on open, and they just USED them to minimize. showWindow now
forces the drawer shut instead, which also covers the Back-button
minimize path, where the window hides without touching drawer state.

* Dashboard drawer: left-align the title against the icon

The tray is asymmetric — one icon on the left, two buttons on the
right — so a title centered in its fixed box landed off the tray's
true center and read as misaligned against the centered tongue below.
Left-aligned, icon + title cluster as the identity on the left and
the controls cluster on the right.

* Dashboard drawer: morph the tongue into the tray instead of sliding

The drawer is now ONE glass surface that changes shape, rather than a
tray that slides in from off-screen with a handle below it. At rest
it's the tongue; opened, the same surface swells into the tray while
the grabber bar rides the morph and settles into a slim strip along
the tray's bottom edge as the dismiss handle — same handle, both
states. Content is revealed from the center outward through an inner
clipping layer (the surface itself can't be overflow:hidden without
clipping the toggle's touch halo) and condenses in/out of focus with
an opacity + blur + scale crossfade. Opening gets a soft overshoot
and settles; closing is quicker and lands without bounce.

The open width is DERIVED in CSS from the same part variables that
lay out the controls row, so the surface always fits its content
exactly and breakpoints (touch sizes, hidden title under 500px) only
override parts.

* Refine dashboard drawer open-state handle

Update the headless dashboard app drawer so the grabber bar disappears when the drawer is open and only the bottom dismiss strip remains interactive. This adjusts tray/strip sizing on desktop and mobile, adds opacity timing for smooth dissolve/reappear transitions during open/close, and aligns in-code docs/comments with the new behavior.
2026-07-22 22:11:39 -07:00
jelveh 5c3fab5579 Revert "Dashboard: fix minimized apps getting permanently buried under the dashboard (#3429)"
This reverts commit bbc86e7b70.
2026-07-22 22:11:15 -07:00
jelveh 695862c61f Increase dashboard directory icon size
Adjusts the directory icon width in the dashboard files section from 28px to 35px for improved visibility.
2026-07-22 21:45:09 -07:00
Nariman Jelveh bbc86e7b70 Dashboard: fix minimized apps getting permanently buried under the dashboard (#3429)
* Dashboard: fix minimized apps getting permanently buried under the dashboard

Fullpage/dashboard app windows are created stay_on_top in the
99999999+ z band, but showWindow's restore path demoted them to a
plain counter z-index — and since focusWindow deliberately never
raises stay_on_top windows, the demotion stuck: one dashboard focus
after a minimize/restore buried the app permanently (URL and tab
title still naming an app the user could no longer see). Restore now
re-raises stay-on-top windows into their band.

Also make the control pill focus its window on mousedown, as the
titlebar it replaces did. The document-level activation handler works
off mouseover_window, which only mousemove refreshes — a tap with no
intervening mousemove (touch, restored windows) would activate the
stale window and could re-raise the dashboard over the app. Deferred
a tick so it runs after that handler and wins.

* Dashboard: restored windows come back with the pill collapsed

The restore re-played the open intro (expand + auto-retract). A restore
should bring back the app, not the chrome — the user already met the
controls on open, and they just USED them to minimize. showWindow now
forces the pill shut instead, which also covers the Back-button
minimize path, where the window hides without touching pill state.

* Dashboard pill: left-align the title against the icon

The pill is asymmetric — one icon on the left, two buttons on the
right — so a title centered in its fixed box landed off the pill's
true center and read as misaligned. Left-aligned, icon + title
cluster as the identity on the left and the controls cluster on
the right.
2026-07-22 21:40:04 -07:00
Daniel Salazar e35b57068b test: add extra tests (#3428) 2026-07-22 21:13:44 -07:00
Daniel Salazar 5faed55076 fix: autoclaim app when making a subdomain (#3426) 2026-07-22 21:12:38 -07:00
Daniel Salazar 928d5fec16 chore: cleanup AI module for puter-js (#3423) 2026-07-22 18:18:54 -07:00
velzie ebcc4bb597 Add loopback support to puter.peer (#3425)
* allow setting port instead of invite code in puter.peer

* fix grabbing fallbackice under wrong response
2026-07-22 20:13:35 -04:00
Neal Shah 51541d3739 initial pjs helper stuff (#3421)
* initial pjs helper stuff

* move resolveReauth

* move location of PuterClient and rename to networkUtils
2026-07-22 18:48:11 -04:00
Nariman Jelveh 43b9200205 Dashboard: headless full-tab apps with a floating control pill (#3424)
* Dashboard: headless full-tab apps with a floating control pill

Maximized app windows in dashboard mode no longer render a titlebar —
the app covers the full tab. The head's controls move to lighter-weight
affordances:

- A floating pill overlaid top-center of the app (parent DOM, above the
  iframe): expanded it shows the app icon, title, minimize, and close;
  it opens expanded so first-time and deep-link users see it, then
  collapses to a subtle capsule that re-expands on hover, tap, or
  keyboard focus. Minimize routes through the same URL-entry consumption
  as the Back button; close tears the window down normally. Dark
  translucent glass so it reads over any app content; safe-area aware;
  bigger targets on coarse pointers; title hidden under 500px; respects
  prefers-reduced-motion.
- Apps-tab tiles double as the app switcher: a macOS-dock-style running
  dot marks tiles with a live (visible or minimized) window, and the
  tile context menu gains a Quit item for running apps. UIWindow
  broadcasts window open/close so the dots stay current.
- puter.ui.setWindowTitle also updates the pill title.

Only maximized, URL-owning app windows in dashboard mode go headless:
dialogs, explorer, non-maximized child windows, and everything on the
desktop keep their titlebars. dashboard.css's 29px head-height
compensation is scoped to windows that still have heads, so headless
app bodies fill the window exactly.

* Fix pill buttons swallowed by their own hover halo

The pill's invisible hover halo (an absolutely-positioned ::after with
negative inset) painted above the pill's static children, so real
pointer clicks hit-tested to the halo and never reached the minimize/
close buttons — while element.click() (which skips hit-testing) worked,
hiding the bug from programmatic checks. Stack the halo at z-index -1
(the pill is its own stacking context, so the outside ring still sits
above the app's iframe) and raise the interactive children explicitly.

* Review round: pill/dashboard edge-case fixes

- Latch pop_dashboard_app_url while its history.back() is in flight: a
  double-click on minimize (or a close racing a minimize) issued two
  back()s, and the second popped the dashboard's own entry — navigating
  clean out of the page. Any popstate or push settles the latch.
- Close a running app's windows when it is uninstalled: the tile is a
  headless app's only switcher, so a minimized instance would otherwise
  be stranded — impossible to restore or quit.
- puter.ui.setWindowTitle in dashboard mode now also updates the
  browser-tab title (while that app owns the URL) and the window's
  data-name, so a later restore re-applies the app's current title
  instead of its launch-time one. Desktop behavior unchanged.
- Nudge collapsed-pill contrast (opacity 0.55 → 0.65, border alpha
  0.16 → 0.22) so the capsule stays findable over dark app content.

* Adjust dashboard tile spacing and label sizing

Tweaks My Apps tile layout constants to match the real icon/label geometry: tile height is increased, label top margin is widened, and the running-status dot is moved slightly lower to avoid overlap during icon hover scaling. It also raises dashboard app pill title line-height to prevent descenders from being clipped when ellipsis overflow is active.

* Pill and running-dot visual polish

- Fixed-width pill: the title gets a fixed 150px centered box instead of
  a cap, so every app's pill is identical and the minimize/close buttons
  sit at the same screen position in every app — they're operated by
  muscle memory, like any titlebar's controls.
- Give the pill title line-height 1.35: overflow:hidden (needed for the
  ellipsis) also clips vertically, and at line-height 1 it cut the
  descenders off g/p/y.
- More air around the tile running dot: label margin 6px → 10px and the
  dot 2px lower (tile row height 78px → 82px to keep the label line in
  budget — the pager layout reads the CSS var, so it follows). The
  icon's 1.08x hover growth no longer touches the dot.

* Size the pill icon to balance the capsule

16px read as undersized next to the 26px buttons in the 36px capsule —
bump the expanded icon to 20px with a proportional (25%) radius, and
object-fit: cover in case an app icon isn't square. Collapsed size is
unchanged.
2026-07-22 14:28:06 -07:00
jelveh d5e9417d2d Dashboard apps own the URL: /app/<name> entries with Back-to-minimize
- Opening an app in dashboard mode pushes a real history entry at
  /app/<name>; Back minimizes the app (it keeps running), Forward
  restores it — or relaunches it if it was closed
- The minimize/close buttons consume the entry via history.back() so
  the address bar never names an app that is no longer on screen
- Landing directly on /app/<name> now boots the dashboard with the app
  open maximized in-page instead of the desktop; embeds, popups, and
  explicit fullpage boots keep the desktop flow, and fullpage_on_landing
  no longer opts a landing out of the dashboard
- Desktop mode no longer rewrites the URL at all (no /app/<name> or
  folder-path replaceState on focus, no /desktop resets on close or
  minimize); tab titles still track the focused window
2026-07-22 10:50:10 -07:00
jelveh dec205d959 Lighten tile-to-window restore animation
Update `showWindow` to use a shorter, window-only morph when restoring a minimized app from a dashboard tile. `morph_window_from_tile` now accepts options for `icon_half` and `duration`, gates icon ghost creation accordingly, and derives transition/cleanup timing from the configured duration so un-minimizing feels distinct from a full app launch.
2026-07-22 09:57:24 -07:00
jelveh d87f3464fc Support Cmd/Ctrl+click and adaptive bg color
Update the dashboard app tile click handler so Ctrl/Cmd+click opens apps in a new tab (matching the context menu behavior) while keeping existing target-link/app-name routing. Also switch the default body background to white and add a dark-mode media query to use a dark background color.
2026-07-21 20:33:27 -07:00
jelveh 25410cac92 Allow tile morph animations on phones
Removes `isMobile.phone` checks that previously disabled dashboard tile morph transitions. Window open/close morph selection and tile-launch ghost setup now run on phone the same way they do elsewhere, while still honoring animation-disable and reduced-motion conditions.
2026-07-21 19:13:48 -07:00
jelveh 66a1baef24 Preplay dashboard tile launch morph animation
Add click-time launch feedback for dashboard app tiles so users get immediate visual response while app startup is still loading. TabApps now starts and settles tile launch state around `launch_app`, logs launch failures, and keeps duplicate-launch protection.

UIWindow now tracks in-flight tile feedback with a WeakMap, introduces `begin_dashboard_tile_launch`/`settle_dashboard_tile_launch`, and updates `morph_window_from_tile` to avoid replaying the icon half when it already ran at click time. Ghost/icon timing and cleanup were adjusted so fallback and failed launches do not leave stale tile state.
2026-07-21 19:09:32 -07:00
jelveh 5c4df9cbd7 Revert "feat(gui): start the tile open-morph on click, before the launch fetches"
This reverts commit f89b3a888a.
2026-07-21 18:25:39 -07:00
jelveh f89b3a888a feat(gui): start the tile open-morph on click, before the launch fetches
A fresh launch has a server round-trip (app info, signing) between the
click and the window's creation, so the open morph used to start late —
on a slow connection the click gave no feedback at all. The two halves
of the morph are now decoupled:

- On click, begin_dashboard_tile_launch starts the icon's half
  immediately: the ghost enlarges in place and dissolves exactly as
  before, and the real icon stays hidden — the empty slot reads as
  'launching' — while the fetches run.
- When the window opens, morph_window_from_tile claims the click-time
  ghost instead of spawning a second one, runs the window half alone,
  and restores the icon when it lands. On a fast launch the halves
  overlap into one motion; on a slow one the window zooms out of the
  empty slot.
- settle_dashboard_tile_launch (in the launch's finally) puts the icon
  back whenever the window never claims it: launch failed, app opened
  in background, or the morph fell back to the plain fade. The launch
  also gained a .catch, so a failed launch logs instead of leaking an
  unhandled rejection.

The un-minimize path has no fetch and is untouched: it still spawns its
own ghost inside the morph.
2026-07-21 17:58:05 -07:00
Nariman Jelveh bf545dda25 Dashboard: open apps maximized in-page, with minimize and single-instance behavior (#3419)
* feat(gui): open dashboard apps maximized in-page with minimize + single instance

Apps launched from the dashboard's Apps tab now open as maximized Puter
windows in the same page (via launch_app) instead of a new browser tab.
External website shortcuts still open in a new tab since arbitrary sites
can't reliably be iframed.

Dashboard app windows keep their minimize button (fullpage mode normally
hides it since there's no taskbar). With no taskbar item to animate
toward, hideWindow now hides the window in place and flags it with
data-minimized_in_place; showWindow un-hides it via that flag. An
explicit flag is used because data-orig-* can't distinguish the two
minimize paths - drag/maximize handlers set those attrs too.

Tiles are single-instance: clicking a tile un-hides a minimized
instance or focuses a visible one instead of launching a duplicate, and
launches in flight swallow repeat clicks.

Also fixes .window-body-app height in dashboard mode: fullpage mode
sizes it to 100% assuming no titlebar, which clipped the bottom 29px of
every app.

* feat(gui): iOS-style minimize-to-icon morph in dashboard mode

Minimizing a window in dashboard mode now zooms it into its app's tile
on the Apps tab — but only when the user can see that tile (Apps section
active AND the tile on the pager page currently in view); otherwise the
existing hide-in-place fade is kept, as on phones and with
prefers-reduced-motion.

The animation is a two-layer morph on one 450ms cubic-bezier(0.32,
0.72, 0, 1) path: the live window scales onto the tile's icon box
(transform + opacity only — no per-frame iframe reflow) while its corner
radius grows toward the icon squircle, and an enlarged clone of the icon
flies congruently with it, cross-fading in as the window fades out and
landing exactly on the real icon's slot (hidden until the handoff), so
the window reads as becoming the icon.

Window geometry is never touched: only inline transform/opacity/radius
are set and restored byte-identical when the animation ends, and
showWindow()'s existing minimized-in-place path restores as before —
including a reopen mid-animation.

* fix(gui): keep the minimize-morph ghost icon square (contain-fit)

The icon clone previously mapped rect-onto-rect over the window, which
stretched the square icon anamorphically. A covering square was tried
and jittered: it protruded past the card's short edge and the halo
popped in mid-crossfade.

Contain-fit resolves both: the ghost scales uniformly (always square),
sized to the window's short edge and centered — its side tracks the
card's short edge exactly for the whole flight (both interpolate
linearly between the same endpoints on the same curve), so the card's
rectangular flanks melt away around a steady, undistorted icon.

* feat(gui): add 'Open in new tab' to app tile context menu

Every tile's right-click menu now leads with Open in new tab, which
opens the app the way tiles did before in-page windows: /app/<name> in
a new browser tab (noopener), or the site link for external tiles.
Non-uninstallable apps previously had no menu at all; they now get this
single item, and uninstallable ones show it above a divider + Uninstall.

* feat(gui): reverse morph — dashboard tile icons zoom into their opening window

Opening an app in dashboard mode now runs the minimize morph backwards:
the window grows out of its tile's icon box (transform + opacity only,
radius relaxing from the 22% squircle to the resting value) while a clone
of the icon enlarges IN PLACE on its slot — capped at 2.5x, fully dissolved
by ~250% growth — as the window fades in underneath on the same 450ms
cubic-bezier(0.32, 0.72, 0, 1) path with the crossfade handoff mirrored
(window in at 80-240ms, ghost out at 120-320ms).

Both open paths get the morph, with the same visibility conditions as
minimize (Apps section active, tile on the pager page in view, not phone,
no reduced motion), falling back to the existing fades otherwise:

- fresh launch: TabApps passes window_options.morph_from_dashboard_tile,
  and UIWindow morphs instead of the 70ms opening fade. Since .window is
  display:none from the stylesheet until later in the pipeline, the hook
  shows it before measuring (nothing has painted yet, so neither the early
  show nor the hide-back on a declined morph can flash).
- un-minimize: showWindow's minimized-in-place branch morphs instead of
  fadeIn(150), keeping the z-index bump and 80ms focus timing.

A data-window_morphing flag guards against opposing morphs fighting over
the window's inline styles: minimize clicked mid-open-zoom (or a tile
click mid-minimize-zoom) falls back to the pre-morph fade behavior, and
each animation still restores every inline property byte-identical when
it ends.

* Update dashboard.css

* feat(gui): cap the open-morph ghost icon at 2x, dissolving in place

The tile icon no longer flies toward the window center nor grows to full
contain-fit (10x+ reads as a giant blurry sticker): it enlarges in place
on its own slot and its growth is capped so the fade-out completes right
around 200% of its size — the window alone carries the motion and the
rest of the growth.

* fix(gui): drop the tile :active press-shrink that stuttered the open morph

Clicking a tile launches the app and the open morph grows the icon in
place — the 0.95 press-shrink made every launch read as shrink-then-grow.
The icon now holds its hover scale through the click and enlarges
straight from there (the morph ghost is measured from the rendered rect,
so it picks up the hovered size seamlessly).
2026-07-21 17:41:45 -07:00
Neal Shah c706b4472d use max_tokens instead of hardcoded value 1000 (#3422) 2026-07-21 18:25:40 -04:00
Neal Shah 93b8041c3d don't allow prototype defined method to be called in drivers (#3413)
* don't allow prototype defined method to be called in drivers

* fix test
2026-07-21 18:01:52 -04:00
Daniel Salazar 89f9f9728f fix: PUT-1355 PUT-1351 PUT-1208 (#3420)
* fix: PUT-1355 PUT-1351 PUT-1208

* fix: dev center header
2026-07-21 14:17:33 -07:00
Daniel Salazar 645409eddb fix: misc hardening (#3418) 2026-07-21 13:55:00 -07:00
tamilselvanp57 58a763bcd1 Add Dev Center UI for viewing app users (#3349) (#3417)
Adds a 'View Users' link on the app Analytics tab that opens a modal
listing users who have authenticated into the app (username + email,
where the user has granted email:read to the app).

Consumes the existing puter.apps.get(...).users()/.getUsers() client
SDK method, backed by the app-telemetry driver's get_users - no new
backend work needed, per discussion on the issue.

Relates to #3349
2026-07-21 13:52:48 -07:00
Daniel Salazar fd91d72654 wip: better contributing docks (#3415) 2026-07-21 12:07:59 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 7fbf491e6f Update docs meta tags and description (#3416)
* Update docs meta tags and description

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-21 16:43:00 +07:00
Daniel Salazar a8833de9d5 fix: misc hardening (#3414) 2026-07-20 23:53:37 -07:00
Daniel Salazar 3a8b6394de feat: standardized api pagination (#3412) 2026-07-20 15:58:23 -07:00
Nariman Jelveh f5d656650f feat(email): accept custom message headers in sendRaw (#3411)
* feat(email): accept custom message headers in sendRaw

Lets callers set transport-level headers such as List-Unsubscribe /
List-Unsubscribe-Post. Also documents the puter-email driver's automatic
unsubscribe / report-abuse footer and the new `suppressed` result field
in the puter.js email module.

* feat(email): envelope override in sendRaw + suppressed in EmailSendResult type

Adds an optional `envelope` field to SendMailOptions so callers can set
transport recipients independently of the visible To/Cc headers (used by
the email-send driver for per-recipient deliveries), and adds the
`suppressed: string[]` field to the public EmailSendResult typing to
match the driver's documented result.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(email): document per-recipient best-effort delivery via `failed`

The email-send driver now attempts every recipient's private delivery
even when one fails, returning failed addresses in the result's
`failed` array instead of failing the whole call — so callers retry
only the failed subset and never re-mail delivered recipients. Adds
the field to the public EmailSendResult typing and module docs.
2026-07-20 11:55:46 -07:00
Daniel Salazar dd314da16d feat: require app or api tokens for ai api usages (#3407) 2026-07-20 08:34:44 -07:00
jelveh 47c1d4268f Fix overlapping context menu state on file rows 2026-07-19 20:08:00 -07:00
jelveh b9e87935cc Update dashboard.css 2026-07-19 20:00:18 -07:00
jelveh 73dd95bd41 Open Desktop by default in the Files tab 2026-07-19 19:53:52 -07:00
jelveh c5bc358ef2 Keep file rows visually active while their context menu is open 2026-07-19 19:51:38 -07:00
jelveh 57c9eb9e1a Fix grid badge drop shadow and hover overlap with item menu 2026-07-19 19:23:02 -07:00
jelveh a0c0544928 Remove extra top padding from files header 2026-07-19 19:15:51 -07:00
jelveh 6dcae81a56 Fix uninstall animation snapping after a prior FLIP slide 2026-07-19 18:57:04 -07:00
jelveh 09baee6381 Delay uninstall tile animation until after the modal closes 2026-07-19 18:48:06 -07:00
jelveh 68aab718a1 Add keyboard navigation to the Apps tab 2026-07-19 18:43:57 -07:00
jelveh 5d6b2f3265 Uninstall apps optimistically with a tile removal animation 2026-07-19 18:35:01 -07:00
jelveh 28730010c3 Fix inconsistent icon sizing in dashboard grid views 2026-07-19 12:24:21 -07:00
jelveh 817ece40ba Modernize folder icon yellows 2026-07-19 12:24:20 -07:00
jelveh 7c868e947a Fix Public folder icon grid and align glyph system 2026-07-19 12:07:10 -07:00
jelveh 66984cd3d6 Improve glyph sizing on system icons 2026-07-19 12:03:09 -07:00
jelveh 5fa508ba91 improve dashboard file list and grid styles 2026-07-19 11:48:46 -07:00
jelveh c1fca467bf Use bundled fallback for null app icons 2026-07-19 10:52:56 -07:00
jelveh e726c19311 Add compact grid option to Files view menu
Reworked the Files view toggle into a context menu with List, Compact Grid (`grid-sm`), and Grid modes, and refactored view handling into shared helpers (`isGridView`, `viewClass`, `setView`, `applyViewMode`) so grid behavior is applied consistently across rendering and drag previews. Added compact-grid CSS overrides to reduce tile, icon, text, and badge sizing while reusing existing grid layout behavior.
2026-07-19 09:48:31 -07:00
jelveh 6643642aa7 Fix item-more color for selected rows in dashboard
Ensure the item-more icon uses the correct sidebar item color when a dashboard file row is selected, including on hover.
2026-07-19 09:26:54 -07:00
jelveh 65dc3b5f4c Use thinner dashboard file nav icons
Switches the Files tab header navigation buttons to new thin arrow SVG assets and aligns icon styling with a lighter visual weight. It also updates several header action SVG paths (new folder, upload, list/grid, sort, select) and sets path action icon color to #5c5c5c so the controls look consistent.
2026-07-18 21:44:02 -07:00
jelveh c225fabbd4 Fix clipped app tiles and soften files header hover styling
Apps tab: tile rows only budget one label line, so two-line app names
made flexbox shrink the icon box into a landscape crop and clip the
label mid-glyph. Clamp labels to a single line and exempt the icon box
from flex shrinking.

Files tab: header hover states (nav arrows, breadcrumbs, action
buttons) now use a neutral #EEE chip and no longer recolor the glyph;
nav arrows also render at full opacity when enabled.
2026-07-18 21:27:24 -07:00
jelveh fdcc2e301b Fix My Apps tile icon and label overflow
Prevent app tile icons from shrinking when labels overflow, and clamp tile labels to a single line so they fit within the fixed tile height. This keeps the grid layout stable while preserving full app names in the existing tooltip.
2026-07-18 20:54:09 -07:00
Nariman Jelveh 183ab0780d fix: verified Dashboard bug fixes (routing, files, apps, usage, mobile menu, CSS) (#3406)
* fix: don't double-refresh Dashboard tabs on browser back/forward

A single back/forward navigation fires both popstate and hashchange, so
handleRouteChange ran twice, calling the tab's onActivate twice — every
back/forward to the Apps tab issued duplicate /installedApps and
/get-launch-apps requests and re-rendered the tab twice. Track the last
handled URL and skip the second event.

* fix: store raw values in Dashboard file-row data attributes

renderItem passed html_encode()d strings to setAttribute, which stores
them literally — a file named "Tom & Jerry.txt" got
data-path="...Tom &amp; Jerry.txt". Every flow that reads the attribute
back (keyboard cut/copy/paste, delete-to-trash, drag-and-drop, selection
actions) then hit the server with the mangled path and failed with
"Entry not found". The socket item.renamed/item.updated handlers had the
same problem via jQuery .attr(), and item.moved/item.removed matched rows
with html_encode()d selector needles that can't match raw attributes.

Store raw values everywhere (matching the desktop convention and the
raw readers throughout TabFiles), match paths by comparison instead of
selector interpolation, and repair the html_encode(x) ?? fallback
expressions that could never take their right-hand side. Verified live:
copy/paste of a name containing '&' now succeeds.

* fix: escape folder names in Dashboard nav-history menu (stored XSS)

The back/forward taphold menus interpolated path.basename(history_item)
straight into UIContextMenu item html, which renders it verbatim. A
folder named with an HTML/script payload executed when the user opened
the history menu after visiting it. The desktop file manager already
encodes this value; match it with html_encode(). Verified live: the
payload now renders as inert text and its onerror never fires.

* fix: harden Dashboard hash routing against malformed and unknown tabs

Three routing defects, all reachable from a shareable URL:

- A malformed percent-sequence (e.g. `#100%`) made parseDashboardRoute's
  decodeURIComponent throw at module load, blanking the entire GUI. Guard
  the decode and fall back to the raw hash. Verified: `#100%` now boots.

- A hash whose tab segment contained a quote (`#foo"bar`) was interpolated
  into a jQuery selector that throws, leaving dashboard event handlers
  unbound. Resolve the route tab against the known tab set (falling back to
  Apps) before it ever reaches a selector. This also fixes an unknown hash
  activating the Apps section without its `.dashboard-content.apps` styling.

- Re-clicking the already-active sidebar tab pushed duplicate history
  entries, so Back became a no-op until pressed repeatedly. Only pushState
  when the hash actually changes.

Verified live for all three.

* fix: keep Dashboard file list footer and size cells in sync on live updates

Incremental updates only touched hidden data attributes, so the visible
UI drifted from the data:

- Adding a file via the socket (item.added) or creating one inserted a
  row but never called updateFooterStats, so the "N items · size" footer
  stayed frozen at the old count. Removals and moves had the same gap.
- item.updated (and the in-place update in UIDashboardFileItem) rewrote
  data-size/data-modified but not the on-screen .item-size/.item-modified
  cells, so a remote overwrite left a stale size (e.g. "5 B" for a file
  that had grown to 2 KB).

Refresh the footer after add/remove/move and repaint the size/modified
cells on update. Verified live: add, overwrite, and delete now all
reflect immediately.

* fix: Dashboard Files shift-click selection after navigation and byte formatting

Shift-click selection:
- renderDirectory cleared the .selected class but left
  window.latest_selected_item pointing at a now-detached row from the
  previous directory. The shift handler set shift_clicked=true, found the
  anchor's index as -1, skipped the range select, and onclick then
  early-returned on shift_clicked — so the first shift-click in a new
  directory selected nothing. Reset the anchor when it detaches, only take
  the range path when the anchor is still in the list, and treat a
  shift-click with no valid anchor as a plain select that becomes the
  anchor (onclick otherwise ignores clicks while Shift is held).

formatFileSize:
- Returned "NaN undefined" for missing/invalid sizes and "1.5 undefined"
  for terabyte-plus files (sizes array stopped at GB). Guard non-finite
  and non-positive input, clamp the unit index, and add TB/PB.

Verified live: first shift-click selects the item, a second shift-click
extends the range, and formatFileSize returns 0 B / 1.5 TB / 2 PB.

* fix: show a message instead of a blank pane when a Dashboard folder fails to open

renderDirectory empties the file list before awaiting readdir, so a
readdir rejection (permission lost, folder deleted from another session,
network error) left a completely blank pane with only a console error.
Render a centered "This folder couldn't be opened." message on the catch
path. Verified live: a simulated readdir failure shows the message,
clears the spinner, and leaves navigation working (next folder loads).

* fix: Dashboard Usage/Home tab data freshness, errors, and dead controls

TabUsage:
- The Upgrade/Manage button was shown unconditionally and its click did
  `new window.UIUpgradeAccount()`, which only exists on hosted puter.com —
  on self-hosted it threw a TypeError from a dead button. Hide the button
  when UIUpgradeAccount is absent and guard the click.
- The tab had no onActivate and bound its refresh to a nonexistent
  element, so usage numbers were frozen at page-load for the whole
  session. Add onActivate to refetch (verified: reactivating refetches).
- Resource names rendered the backend's `_dot_` escaping literally
  ("gemini-2_dot_5-flash"); un-escape for display and html_encode the cell.
- The two loaders had no error handling; a failed fetch left the tab
  blank with an unhandled rejection. Catch each and show a fallback.
- Guard capacity 0 so storage no longer renders "NaN%".

TabHome:
- Plan button kept saying "Manage →" after a subscription lapsed; reset it
  to "Upgrade →" in the free branch.
- Guard capacity 0 ("NaN%") on the storage card.
- "Your Plan ›" card header had the arrow affordance but no target and did
  nothing on click; point it at the Usage tab like its siblings.
- Returning to the tab fired both focus and visibilitychange, and
  refreshAndBroadcast both called refresh() and dispatched the event its
  own listener handles — up to 4 duplicate reloads per focus. Drop the
  direct call and coalesce the focus/visibility pair.

TabHome + TabApps:
- window.open(externalAppUrl, '_blank') lacked noopener, exposing the
  dashboard tab to reverse tabnabbing from an external app site. Add
  noopener,noreferrer.

Verified live: Usage and Home render without NaN or console errors, the
upgrade button is hidden on self-hosted, and 'Your Plan' now navigates.

* fix: multiple Dashboard CSS defects (dead rules, contrast, responsive)

- The desktop rule that hides the row ⋮ button ended two selectors with a
  comma before an @media block, so the whole construct was invalid and
  discarded — the ⋮ showed on every list-view row on desktop instead of
  deferring to right-click. Verified: it's now hidden (display:none).
- Native-drop dark-mode styles keyed off .window[data-color-scheme="dark"],
  an attribute never set anywhere; converted to @media (prefers-color-scheme:
  dark) like every other dark rule so they actually apply.
- .myapps-tile-label set white-space:nowrap after a 2-line -webkit-line-clamp,
  defeating the clamp so long app names clipped mid-glyph on one line;
  removed it.
- .dashboard-sidebar-separator was defined a second time with a contradicting
  box model (background line + full-width margin), rendering a doubled,
  edge-to-edge divider; removed the duplicate so the inset border rule stands.
- Grid-view .item-icon had a hardcoded background:white and border-radius:2px
  overriding its own border-radius:8px (glaring white tiles in dark mode,
  square-ish corners); use var(--dashboard-background) and keep 8px.
- .files-footer used hardcoded #666/#CCC on theme-variable backgrounds
  (dim in dark, near-invisible separator in light); use the text/muted vars.
- Fixed the .ui-droppable-over typo (jQuery UI emits ui-droppable-hover).
- The context-menu backdrop's "desktop transparent" rule used min-width:768px
  while every mobile rule uses max-width:768px, overlapping at exactly 768px
  (a common tablet width) — a non-dimming, invisible modal shield; bumped to
  769px.
- At 481-768px the fixed hamburger toggle sat on top of the Files directories
  column's first folder because .dashboard-content.files padding wins over the
  media-query padding; add top clearance to the directories column there.

Verified light-mode desktop is unchanged (directories padding still 16px, ⋮
hidden, footer colors resolve).

* fix: only offer Uninstall for Dashboard apps where it actually sticks

Uninstall was suppressed via a hardcoded 8-name allowlist that had drifted
out of sync with the backend's ~26 recommended apps. For the ~18 unlisted
recommended apps (Calculator, Code, the games, …) the menu offered
Uninstall, revokeApp resolved, the tile vanished — then get-launch-apps
re-added it on the next load and it reappeared, so the uninstall silently
reverted.

Compute uninstallability from the actual lists: an app is uninstallable
only if it's in the user's installedApps AND not in the recommended list
(and not a protected core app). Recommended apps — installed or not —
resurrect on reload, so Uninstall is hidden for them. Verified live: a
synthetic installed-not-recommended app shows Uninstall; a recommended
app shows none.

* fix: guard Dashboard Apps loads against stale overwrites and drag/error clobbering

loadApps only checked for an in-progress drag before its await, and its
catch wiped whatever was on screen. Three concurrency issues followed:

- A slow, older load resolving after a newer one could overwrite the
  newer app list (and clobber a reorder the user saved while the stale
  fetch was in flight). Tag each load with an increasing id and skip
  applying one only when a strictly newer load has already applied —
  gating on "already applied" (not "latest started") so the first load to
  resolve still populates the list for the pager's ResizeObserver.
- A drag that began while a load was awaiting could have the grid rebuilt
  out from under it; re-check the drag after the await.
- A transient re-fetch error replaced a working grid with "Failed to load
  apps"; only show that placeholder when nothing has loaded yet.

Verified live: a slow older load no longer clobbers a newer one, and the
grid still renders normally on activation.

* fix: page through all installed apps in the Dashboard instead of capping at 100

The /installedApps endpoint clamps limit to 100 and paginates, but the
Apps tab fetched a single page — a user with more than 100 installed apps
silently lost the alphabetically-last ones from both the grid and search,
with no way to launch or uninstall them from the dashboard. Loop pages
until a short one comes back (the common <100-app case still makes a
single request). Verified: the request now carries &page=1 and stops
after one page for a small account; the loop pulls all pages otherwise.

* fix: make the mobile Dashboard context menu handle submenus, disabled items, and positioning

The touch context-menu modal (used whenever maxTouchPoints > 0) had three
defects:

- Items with a submenu and no onClick ("New", "Open With") rendered as
  plain buttons that did nothing and didn't even close the menu — the two
  submenu-bearing actions were simply unreachable on touch. Drill into the
  submenu on tap, with a Back row to return.
- disabled items were rendered as active buttons and executed their
  onClick, so a folder's disabled "Paste Into Folder"/"Publish as website"
  ran anyway. Render them inert (disabled attribute + dimmed class + a
  click-handler guard), mirroring the desktop UIContextMenu.
- Non-touch devices that still route here (touchscreen laptops) got the
  modal pinned at a hardcoded left:300px, far from the tapped item on a
  wide screen. Center it over the target and clamp to the viewport.

Verified live (with maxTouchPoints forced): "Open With" opens its submenu
and Back returns; disabled "Paste Into Folder" is inert; the modal is
positioned near the item.

* fix: repair Apps-tab regressions from the Dashboard bug-fix pass

- A non-array /installedApps response (an error payload) was read as
  end-of-pagination, silently rendering the grid without any installed
  apps; fail the load so the explicit error state shows instead.
- A loadApps result that resolved mid-drag was discarded with no retry,
  freezing the grid on stale data for the rest of the visit; stash it
  and apply it after the drag ends, reconciled against the drag's final
  order so it can't undo a reorder whose save is still in flight.
- Uninstall disappeared for recommended/recent apps, removing the only
  UI path that revokes their permissions; offer it again and instead set
  expectations in the confirm modal (the tile stays for apps that
  get-launch-apps re-adds).
- init and the initial-route onActivate both fired a full load on open;
  share the in-flight load instead of issuing a duplicate request trio.

* fix: repair Files/routing regressions from the Dashboard bug-fix pass

- The no-anchor shift-click fallback also fired when shift-clicking the
  current anchor, collapsing an existing multi-selection to one item,
  and it ignored Ctrl/Cmd; only run it when there is genuinely no
  anchor, and keep the existing selection when Ctrl/Cmd is held.
- Unknown hash values (a stale bookmark, an in-page anchor) were
  coerced to 'apps' and yanked the user off their current tab with a
  refetch and autofocus; ignore them outright, both at boot and on
  hashchange, while still keeping untrusted hashes out of selectors.
- The item.updated socket handler re-implemented the row-refresh from
  TabFiles and had already drifted (trashed items showed their raw UID
  instead of metadata.original_name); extract a shared row updater and
  delegate to it.

* fix: repair Usage-tab regressions from the Dashboard bug-fix pass

- Hiding the plan button whenever window.UIUpgradeAccount was missing
  at check time could hide it for the whole session on hosted
  deployments that attach the script after dashboard init; keep
  re-checking for a while before concluding the install is self-hosted.
- init and the initial-route onActivate both refreshed usage on a
  direct #usage open; share the in-flight refresh instead of issuing
  duplicate request pairs.

* fix: repair Dashboard CSS regressions from the bug-fix pass

- Repairing the malformed item-more rule put display:none into effect
  for the first time, removing the visible-and-used desktop row '⋯'
  button; drop the rule instead (it was never in effect on any release).
- The desktop context-menu backdrop breakpoint (min-width: 769px) left
  fractional viewport widths between 768px and 769px with the mobile
  dimming; use the exact complement of the mobile max-width: 768px
  rules instead.

* fix: make Dashboard app uninstall honest about recents and resilient mid-flight

- Recently-opened apps were classified as sticky-removable, but the
  recent list is built from app-open history that a revoke doesn't
  touch, so their tiles reappeared on the next load — the very bug the
  uninstall gating set out to fix. Treat recents like recommended.
- The confirm modal claimed every staying tile was 'provided by Puter',
  which is false for third-party recents; describe the actual reason.
- Uninstall now marks in-flight loads stale so a fetch that started
  before the revoke can't resurrect the removed tile.
- A pagination failure after the first page no longer discards the
  pages already fetched — one flaky request among N used to turn the
  whole grid into 'Failed to load apps'.

* fix: keep the Dashboard Usage tab stable through transient failures

- A failed refresh no longer wipes an already-rendered usage table;
  the unavailable note only shows when there is nothing on screen yet
  (mirrors the Apps grid's error handling).
- The plan-button check now re-checks indefinitely with backoff instead
  of giving up after ~10s, so an arbitrarily late UIUpgradeAccount
  attachment can't leave a subscriber without the button all session.

* fix: Dashboard Files footer sync and Home refresh resilience

- The shared row updater now refreshes the footer's item-count/total
  line, which is computed from the data-size attributes it just wrote;
  a remote overwrite no longer leaves the row and footer disagreeing.
- Size/Modified are only written when the update payload carries them,
  so a minimal event can't zero out correct values on screen.
- A failed readdir now resets the footer instead of keeping the
  previous directory's counts over an empty pane.
- Home's focus-refresh caps its wait on refresh_user_data, so a whoami
  that never settles can't leave the plan/usage cards stale forever.

* fix: contain partial Apps loads and keep the uninstall flag current

- A partial installed-apps list (a page beyond the first failed) now
  renders only when the grid is empty — it must never replace a
  complete grid already on screen — and saveOrder refuses to persist
  while the list is partial, since overwriting the saved order with a
  truncated one would drop the missing apps' positions for good.
- Opening an app from the grid now flips its tile's uninstallable flag:
  the open puts it in the server-side recent list, so a revoke from
  that moment on leaves the tile in place, and the modal must not act
  on the stale load-time snapshot.
- The in-flight-load invalidation moved into _invalidateInFlightLoads()
  so the seq protocol lives in one place.

* fix: Usage-tab error guard, dead spinner, and poll decay

- The transient-error guard now tracks 'a table has rendered' instead
  of data length, so an account with legitimately zero usage doesn't
  get its empty table replaced by the unavailable note on a flaky
  refresh.
- update_usage_details animated a spinner element that exists nowhere
  in this tab and held every refresh open for an artificial 1s minimum;
  both halves of the dead control are now gone.
- The plan-button recheck backs off to a 60s heartbeat instead of
  polling at 2s forever on installs where UIUpgradeAccount never
  appears.

* fix: guard minimal file updates; tighten selection and socket hot paths

- The shared row updater only writes the fields an update payload
  actually carries, so a minimal event can't blank a row's name/path —
  the same hazard the size/modified guards already covered.
- The all-rows lookup in the selection handler now happens only under
  Shift instead of on every click, and the single-select block shared
  by the no-anchor shift-click and drag-handle paths lives in one
  selectSingle helper.
- item.removed and item.moved narrow by data-uid (O(1) selector) before
  comparing paths instead of scanning every row per socket event.

* fix: replace partial-load and uninstall-flag patches with convergent designs

The previous round's containment patches each spawned new corner cases
(a persistently failing page froze all refreshes; the partial-load
saveOrder bail silently discarded reorders; the tile-open flag flip
lost races with in-flight loads, server recording, and popup blockers).
Replace both mechanisms:

- A partial installed-apps fetch (later page failed) now always applies,
  supplemented with the apps already known from the previous list — the
  grid and search can't shrink from one flaky request, refreshes never
  freeze, and an empty grid still gets the fetched pages.
- saveOrder carries over saved names missing from the current list
  instead of refusing to persist, so no app's saved position is ever
  dropped; stale names are harmless (reconcileAppOrder ignores them)
  and preserve the position of apps that return.
- The load-time uninstallable snapshot is no longer patched client-side
  on tile open. Uninstall now revokes, optimistically removes the tile
  when expected to stick, and refetches — the grid converges to
  server-side truth instead of guessing at recents state.

* fix: don't blank filenames on metadata-only updates; unify range selection

- A payload carrying metadata but no name resolved to an empty display
  name and blanked the row's visible filename — skip the write when the
  resolved name is empty.
- The shift-range branch now goes through the same applySelection
  helper as the no-anchor and drag-handle paths (it takes the row set),
  removing the last hand-rolled copy of the selection bookkeeping.

* fix: rank-preserving saved-order merge; keep the pager put after uninstall

- The saved-order carryover appended missing apps' names at the tail,
  permanently demoting their saved grid positions — the very thing it
  claimed to protect. Replace it with mergeSavedOrder in appOrder.js
  (beside its tested siblings, with unit tests): each missing name
  keeps its rank among surviving names, so a drag during a partial-load
  session neither drops hidden apps' positions nor teleports them
  behind the dragged tile.
- The post-uninstall convergence refetch rendered without preservePage,
  snapping the pager back to page 1 and replaying the load fade; thread
  render options through loadApps so the background sync keeps the
  user's page.

* fix: keep hidden apps' saved ranks through the mid-drag stash path

- _applyPendingLoad reconciled a stashed load against the visible-only
  on-screen order, tail-appending any app returning to the grid and
  contradicting the order the drag had just saved — the rank-demotion
  defect resurfacing through one more path. It now prefers
  _savedOrderNames, the canonical saved list that saveOrder keeps
  merged with hidden apps at their ranks, over the possibly pre-drag kv
  snapshot the stashed load fetched.
- The post-uninstall optimistic render now also skips the load fade;
  it was the one in-place rebuild still hiding the grid behind the
  opacity-0 icon gate.
- mergeSavedOrder's rank rescan collapsed to a single forward
  insertion pointer (verified output-identical; the 20 unit tests pin
  the behavior).

* fix: gate stale kv-order snapshots on the latest local save

A drag that started and ended while a load was in flight escaped the
stash path entirely: the load applied normally and replayed its
pre-drag kv snapshot, visibly reverting the just-saved reorder (and
permanently clobbering it after the next save). Conversely, the stash
path unconditionally preferred the local record, discarding a genuinely
fresher order fetched from another window when the drag committed
nothing.

Record the load-seq boundary at each saveOrder; a resolving load (both
apply paths, via _resolveOrderNames) replays its fetched kv order only
when it was issued after the latest local save — otherwise the
canonical in-memory saved list wins. Also rewrites the _applyPendingLoad
header, which described the pre-rank-merge implementation.

* Simplify app uninstall tile handling

Remove the uninstallability flag and modal warning logic, and treat uninstall as a local UI removal after permission revoke. This avoids an immediate reload that could re-add recommended/recent apps, while stale in-flight loads are still invalidated. Also simplifies app loading by dropping unused render options from `loadApps`/`_fetchAndRenderApps` and related merge comments.

* fix: match raw item paths in move_items cleanup and shortcut re-point

Item rows store data-path/data-shortcut_to_path unencoded, so the
html_encode()d attribute selectors missed names containing & < > " '
and the destination-row exclusion guard could remove a legitimate row
just created by a concurrent item.moved handler. Compare raw values
case-insensitively instead.

* fix: bound the Usage plan-button retry poll

The UIUpgradeAccount retry chain decayed to a 60s heartbeat but never
stopped, leaving self-hosted installs polling forever and pinning the
captured $el_window. Give up after ~30s of decaying retries; onActivate
re-checks on every return to the tab, so late script loads still get
the button.

* fix: drop the recent-opens list from the Apps tab grid

Recents are open history, not installs: they resurrected uninstalled
apps' tiles and showed merely-visited sites as if installed. The grid
is now recommended + installedApps; anything the user actually uses
still appears because opening an app grants it a permission. Recents
still power the Home tab.
2026-07-18 20:39:57 -07:00
jelveh 81c69f243b Align sign-up email button with OIDC styling 2026-07-18 09:35:13 -07:00
jelveh 6a3ecaff2a improve and unify OIDC button styling 2026-07-18 09:18:21 -07:00
jelveh 70f343d6fe Defer taskbar creation in fullpage mode 2026-07-17 14:38:06 -07:00
jelveh 172aa01e1c Trim file context menu actions 2026-07-17 14:23:52 -07:00
jelveh cba6ac8184 Clamp and normalize window progress values 2026-07-17 14:14:50 -07:00
jelveh 492f3d235a Add active label variant to context menu height rule 2026-07-17 13:54:12 -07:00
jelveh d9a01a3748 Add responsive item properties modal to Dashboard 2026-07-17 13:21:15 -07:00
Nariman Jelveh aab21ef91d don't record app open in iframe env 2026-07-17 11:42:48 -07:00
Daniel Salazar 2479e6a064 fix: FS permissions (#3404) 2026-07-17 11:01:34 -07:00
jelveh 7cba26bb39 Revert "fix(dashboard): make app drag-to-reorder work on touch (iOS)"
This reverts commit af0dc06f49.
2026-07-17 08:53:46 -07:00
jelveh af0dc06f49 fix(dashboard): make app drag-to-reorder work on touch (iOS)
On iOS/WebKit the pager's `touch-action: pan-x` made horizontal touchmove
non-cancelable, so the moment a long-press drag moved the finger the
browser started a native page-pan and fired pointercancel, aborting the
reorder before it visibly began. Preventing pointermove doesn't stop
native scrolling on iOS, and the reordering `touch-action: none` rule was
applied only after the drag began — too late, since touch-action is
latched at gesture start.

Drop the explicit pan-x (back to auto) so horizontal touchmove is
cancelable, and install a non-passive touchmove listener that
preventDefaults only once reordering is armed (post long-press). Before
arming it's a no-op, so a quick swipe still flips pages natively.
2026-07-17 08:47:58 -07:00
Nariman Jelveh 14900514b9 Dashboard: drag-to-reorder apps in the Apps tab (#3401)
* feat(dashboard): drag-to-reorder apps in the Apps tab

Let users arrange their My Apps tiles in any order via drag-and-drop.
The order is stored per-user in puter.kv (`dashboard_apps_order`) as a
list of app names and re-applied on load; apps installed since the order
was saved are appended, and uninstalled ones are dropped.

Interaction:
- Mouse/pen: drag past a small threshold to reorder; a plain click still
  opens the app.
- Touch: long-press to pick up a tile (a quick swipe still flips pages).
- Dragging to a scroller edge dwells, then flips to the adjacent page so
  tiles can move across pages; the floating drag image lives on <body> so
  the pager's overflow can't clip it, and displaced tiles reflow with a
  FLIP animation.
- Escape (or losing the window) cancels and reverts; reordering is
  disabled while a search filter is active; prefers-reduced-motion is
  honored.

Also fixes a latent pager bug where programmatic page changes (arrows,
dots, wheel) didn't update the active page indicator, because those
scrolls don't reliably emit 'scroll' events; goToPage now tracks the
page eagerly.

The order-reconciliation logic is extracted into a pure module
(appOrder.js) with unit tests.

* fix(dashboard): ignore stray pointers and no-op drags

- Filter drag pointermove/up/cancel by pointerId so a second finger can't
  hijack or prematurely end an in-progress reorder.
- Only persist a new order when the drop actually changed it, so an
  accidental long-press or drop-in-place doesn't freeze the default
  ordering into a custom one.

* fix(dashboard): address review of app drag-to-reorder

Correctness:
- Preserve the touch long-press → Uninstall path. A long-press now *arms*
  reordering instead of immediately grabbing the tile; moving begins the
  drag, while holding still lets the native context menu fire (which
  cancels the pending pickup). This restores the only touch route to
  Uninstall that the first cut had removed.
- Only swallow the post-drag click when the drop actually reordered
  something, so a small pointer drift or a drop-in-place still opens the
  app like a plain click.
- Guard the pre-start "intent" window: cancel a pending pickup before any
  re-render (ResizeObserver / loadApps), and bail out of _beginDrag if the
  pressed tile was detached, so a layout change mid-press can't reinsert a
  stale node and persist a corrupted order.

Cleanup:
- Reuse reconcileAppOrder for the drop path instead of a duplicate helper.
- Read the saved order in parallel with the app-list fetches.
- Cache the reduced-motion MediaQueryList instead of re-querying per move.
- Fold the touch-scroll cancel into _endDrag(false); drop _abortDragIntent.

* feat(dashboard): smoother, more tolerant app reordering

Make drag-to-reorder feel like the iOS home screen instead of snapping
and flickering:

- Slow the reflow to 320ms with a gentle spring easing.
- Hit-test against each tile's *resting* box, not its live
  getBoundingClientRect. Mid-animation a tile sits between slots, so
  testing its live box was dropping it under the pointer and swapping it
  straight back — the source of the back-and-forth jitter.
- Require the dragged icon's centre to be well inside a tile before it
  becomes the drop target (a deadzone around each tile), so hovering a
  boundary no longer thrashes.
- Probe with the dragged icon's centre rather than the fingertip, and
  batch the FLIP into a single reflow so large grids stay smooth.
2026-07-17 07:57:05 -07:00
Daniel Salazar 7faaef8cf0 tests: more tests for puter.js (#3402) 2026-07-16 22:35:15 -07:00
Nariman Jelveh c188257f85 Session Manager: compact redesign + responsive modal (no UIWindow) (#3400)
* redesign: compact, scannable Dashboard Session Manager

The session list rendered each entry as a bulky card with a 5-row
Created/Last active/Expires/IP/Client key-value table and a large filled-red
Revoke button, so only a couple of sessions fit on screen and destructive
actions dominated the layout.

Reworked it into an icon-led, compact list:
- One row per session: a device/kind icon tile (laptop / phone / globe by OS,
  bolt for workers, key for API tokens, the real app icon for app sessions),
  the title + badges, and two tight meta lines (client · IP, then a fainter
  last-active · created · expires with absolute-time tooltips).
- Destructive actions are calm: per-row Revoke and the rename pencil are quiet
  and reveal on hover (always shown on touch); "Revoke all other sessions" is a
  subtle ghost button instead of a filled block competing with search.
- Current session gets a green accent, tint, and pill, and has no revoke button.
- Toolbar search gains an inline icon + focus ring; added a session count line;
  restyled kind badges; nested child sessions get a proper tree line.

Styling uses the existing dashboard design tokens. All behavior is preserved:
search filtering, parent/child tree with expand/collapse, inline rename
(optimistic + rollback), per-session revoke (access-token vs session routing),
revoke-all, current-session guard, and focus/interval refresh.

Adds ui_session_count_one / ui_session_count_other to en.js (other locales
fall back to en).

* refactor: render Session Manager as a responsive modal, not a UIWindow

Replaces the draggable UIWindow shell with a self-contained DOM modal:
- Backdrop + centered card on desktop (max 680px / 90vh); full-screen sheet
  on phones (<=640px). Fade/scale in, respects prefers-reduced-motion.
- Adds a header bar (title + close), backdrop-click and Escape to close, and
  a scroll-contained body.
- Confirmations no longer use UIAlert (itself a UIWindow). Revoke, revoke-all,
  and the rename-error path now use in-modal confirm/alert sheets, so there is
  no cross-window z-index juggling — important because this can be opened from
  a stay_on_top window (UIWindowCopyToken).

Drops the UIWindow and UIAlert imports entirely. All behavior is preserved:
search, parent/child tree + expand/collapse, inline rename (optimistic +
rollback), per-session revoke (access-token vs session routing), revoke-all,
current-session guard, and the focus/60s-interval refresh (cleaned up on close).
2026-07-16 20:48:54 -07:00
Nariman Jelveh 4543a645f2 fix: verified Dashboard bugs (10 separate fixes) (#3399)
* fix: prevent crash rendering website files without a workers array in Dashboard Files

TabFiles.renderItem accessed `file.workers.length` unguarded when deciding
whether to show the website badge. Directory entries that have a published
website but no `workers` array (the common case) made this throw
`Cannot read properties of undefined (reading 'length')`, so the row failed
to render — the file silently disappeared from the list (swallowed by the
Promise.allSettled batch) or was never added on socket-driven updates.

The sibling `is_worker` derivation on the same entry already guards with
`file.workers?.length`, and the desktop reference (UIItem.js) normalizes
`workers` to an array and shows the badge via `!is_worker`. Reuse the safe
`is_worker` flag here instead of touching `file.workers.length` directly.

* fix: correct Dashboard usage that showed negative spend for users with credits

The Usage tab and Home usage card derived month-to-date spend as
`monthUsageAllowance - allowanceInfo.remaining`. But the metering service
computes `remaining = remainingAllowance + remainingPurchasedCredits`
(MeteringService.getAllowedUsage), so `remaining` exceeds the monthly
allowance whenever a user holds purchased credits — making the displayed
spend and percentage negative (e.g. "-$9.90 used of $0.25", "-3960%").

Use the actual reported spend (`res.usage.total`, already returned by the
same endpoint) and clamp the percentage/bar to 100%. For a free user with no
credits and no overage this yields the identical value as before.

* fix: make Dashboard Files Download button work for a single selection

In mobile select mode the floating action bar appears with one item selected
(minCountForActionBar is 1) and shows an enabled Download button, but the
click handler only acted when 2+ rows were selected — so downloading a single
file did nothing.

Gate on `> 0` instead. window.zipItems already normalizes a single item to an
array, so the one-file case works. On desktop the bar only appears at 2+
selections, so this changes nothing there.

* fix: auto-fit column width in Dashboard Files no longer scales with row count

Double-clicking a column resize handle is meant to fit the column to its
longest cell. The per-row loop did `maxWidth = Math.max(maxWidth + 10, textWidth)`,
which forces `maxWidth` to grow by at least 10px on every row regardless of
content — so a folder with N files produced a column ~10*N px wide and
persisted that runaway width to KV storage.

Compare against the running max only (`Math.max(maxWidth, textWidth)`);
`textWidth` already includes cell padding.

* fix: clear empty-directory placeholder when a file is added incrementally

When a directory is empty, renderDirectory appends an absolutely-positioned
"No files in this directory." placeholder. If a file then appears via an
incremental path (socket item.added/item.moved -> UIDashboardFileItem), the
new row was inserted but the placeholder was never removed, so the "No files"
text overlapped the real item. insertAtSortedPosition can't clear it because
the placeholder has no `.item.row` class.

Remove the placeholder in UIDashboardFileItem before inserting the row,
matching what the in-app New Folder / New File flows already do.

* fix: stop leaking a document keydown handler per uninstall-modal dismissal

showUninstallModal bound `keydown.uninstall-modal` on document but only
detached it on the Escape and Confirm paths. Dismissing via Cancel or a
backdrop click called `close()`, which just removed the overlay and left the
handler attached (closing over a now-detached overlay). Each open→cancel
cycle stacked another document-level listener for the page lifetime.

Move the `.off('keydown.uninstall-modal')` into `close()` so every dismissal
path cleans up, and drop the now-redundant explicit detaches.

* fix: resolve 2FA setup promise on every close so Dashboard toggle can't freeze

UIWindow2FASetup only resolved its promise from the Done button and, in
on_before_exit, when setup had NOT succeeded. Clicking "Enable" sets
setup_succeeded=true and advances to the recovery-codes screen without
resolving; if the user then closes that screen via the backdrop or Escape
instead of Done, the promise never settles.

The Dashboard Security tab is the only caller: it disables the 2FA toggle and
`await`s this promise, so the hang leaves the toggle permanently disabled and
stuck in the pending state for the rest of the session. Resolve with
setup_succeeded on any exit; resolve_promise is idempotent so a prior
Done-resolve is unaffected.

* fix: refresh Dashboard tabs when navigating via browser back/forward

The sidebar-click and initial-route handlers both call the target tab's
onActivate (its refresh hook), but handleRouteChange — wired to hashchange
and popstate — only swapped the active classes. Since tab switches use
history.pushState (which doesn't fire popstate/hashchange), the only way to
reach handleRouteChange is browser back/forward, and doing so left tabs like
Home/Apps showing stale data.

Look up the tab by id and call onActivate after switching, matching the
sidebar-click path.

* fix: refresh Home "Recently used" apps instead of freezing them for the session

loadRecentApps only hit /get-launch-apps when window.launch_apps.recent was
empty. Once populated (typically at page load), every subsequent init/onActivate
/focus just re-rendered the same cached array, so apps launched during the
session never showed up under "Recently used" until a hard reload.

Drop the empty-list guard so the list refetches on the same triggers that
already refresh the usage cards. The fetch can only make the list fresher.

* fix: sync window.user.otp when enabling 2FA from the Dashboard

The disable path clears window.user.otp (in UIWindowDisable2FA), but the
enable path updated only the card's DOM, never the in-memory flag. TabSecurity
html() reads user.otp to decide the toggle/badge state, so after enabling 2FA
a re-render of the Security tab from the cached window.user showed it as
disabled for an account that actually had 2FA on.

Set window.user.otp = enabling on success so both directions stay consistent.
2026-07-16 20:03:38 -07:00
jelveh 75d3b07e0f fix: phantom shortcut badge on files created via dashboard New menu
renderItem() showed the shortcut badge whenever file.is_shortcut !== 0.
Directory listings return is_shortcut as 0, but the puter.fs.upload()
result used to incrementally render newly-created items omits the field,
so undefined !== 0 was true and every new file got a phantom shortcut
badge. Normalize is_shortcut to 0/1 before using it.
2026-07-16 19:05:28 -07:00
Nariman JelvehandClaude Opus 4.8 9da32c73cb Restore Files tab in dashboard (#3397)
* Restore Files tab in dashboard

Reinstates the Files tab (TabFiles) that was removed in 973d046d, bringing
back the dashboard file browser: directory navigation, list/grid views,
sorting, drag-and-drop, context menus, upload, and trash support.

- Restore src/gui/src/UI/Dashboard/TabFiles.js (identical to pre-removal)
- Re-wire UIDashboard.js: tab registration, initial file path, item socket
  handlers, and route-based file navigation
- Re-add file-tab CSS in dashboard.css and the selection-area rule in style.css
- Re-add dashboard-mode item creation hooks in helpers.js move_items
- Re-add #files/<path> deep-link parsing in initgui.js parseDashboardRoute,
  while preserving Apps as the default dashboard tab

* Only remove stale rows when moving items

* Fix Dashboard Files tab bugs

- Escape filenames in renderItem to prevent stored XSS via HTML in file
  names (matches UIItem); raw name is still preserved for display/rename.
- Read clone data-id from the nested .row in drag drop handlers so
  multi-select drag moves all selected items, not just the grabbed one.
- Resolve sidebar drop target from the element's data-path so dragging
  onto Public/Home works (they aren't keyed in user.directories); fix the
  matching UID-vs-path comparison in the drag-out handler.
- Set data-is_worker/data-worker_url from is_worker itself instead of an
  always-true "!== undefined" check, so worker context-menu behavior only
  applies to actual workers.
- Give delete-confirmation buttons explicit values so permanent delete
  works in non-English locales.
- Refresh the view after a context-menu move-paste, mirroring Ctrl+V.
- Guard the item.updated socket handler against the client's own echo,
  matching the other handlers and UIDesktop.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* Prevent Files tab lockup when readdir fails

renderDirectory sets renderingDirectory=true then awaits puter.fs.readdir
with no error handling. readdir rejects on any backend error (permission,
deleted directory, network), leaving renderingDirectory stuck true so the
re-entry guard blocks all future navigation and the spinner never clears.
Catch the rejection and reset state.

* Isolate render failures so one bad entry can't lock the Files tab

renderItem parsed file.metadata with an unguarded JSON.parse, but metadata
is client-writable and stored verbatim, so '', undefined, or malformed
values throw (item_icon.js guards the identical call). Combined with
Promise.all, one bad entry rejected the whole render and left the tab
frozen. Guard the parse and use Promise.allSettled for per-item isolation.

* Clear selection when clicking empty area in Files tab

The empty-area click handler is an arrow function, so `this` was the
TabFiles object rather than the container element, making `e.target === this`
always false. Only clicks landing exactly on `.files` deselected. Use
e.currentTarget so clicks on the container background deselect as intended.

* Avoid stacking image-preview document click handlers

showImagePreview bound a click.imagepreview handler without removing a
prior one. It is re-invoked during image arrow-navigation while a preview
is open, so identical document handlers accumulated until close. Remove
any existing handler before binding.

* Sort live-inserted items by display name

insertAtSortedPosition compared the new item's raw file.name against
existing rows' data-name (the display name). For trashed items, name is
the UID while the real name is in metadata.original_name, so newly trashed
items landed in the wrong position in Trash until a full re-render. Compare
on the new row's data-name to match sortFiles and the existing rows.

* Use display name in in-place item update

UIDashboardFileItem's in-place update path set data-name and the shown
name from raw file.name, unlike renderItem which uses
metadata.original_name || file.name. An update to an already-visible
trashed item replaced its name with the UID and corrupted data-name
(which sort and type-to-search read). Compute the display name to match.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-16 18:57:19 -07:00
Daniel Salazar 102cef523f fix: app id whitelisting (#3398) 2026-07-16 18:03:34 -07:00
Daniel SalazarandClaude Fable 5 0e1be72f92 test: tests for puter.js (#3396)
* test: tests for puter.js

* fix: ship lockfile for coverage devDeps; tolerate missing base coverage

npm ci failed on CI because package.json gained the babel/istanbul
devDependencies without the matching package-lock.json update. Also make
the coverage workflow's base leg best-effort so a base ref that predates
the coverage script reports without the comparison column instead of
failing the run.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-16 15:52:37 -07:00
Neal Shah ef1afb9cc1 add just app telementry (#3387) 2026-07-16 14:04:54 -04:00
Neal Shah 0462ddd6f5 add support for step-up sessions (#3395)
* add support for step-up sessions

* update step up session
2026-07-16 13:58:49 -04:00
Neal Shah 8d7e70cde4 add kimi k3 (#3394) 2026-07-16 12:47:44 -04:00
Daniel Salazar 52e481128f wip: puter js tests structure (#3393) 2026-07-15 19:09:35 -07:00
Daniel Salazar c237e2433f fix: clean apps on subdomain deletion (#3392) 2026-07-15 13:38:53 -07:00
tamilselvanp57 a4bb57bb79 Fix static hosting domain fallback for self-hosted custom domains (#3382) 2026-07-15 11:45:10 -07:00
Daniel Salazar 861cf5efb8 feat: email api wip (#3391) 2026-07-15 11:30:16 -07:00
Daniel Salazar cf34f9fea9 fix: sanitize get user response (#3390)
* fix: sanitize get user response

* fix: app creation in dev center
2026-07-15 02:02:06 -07:00
jelveh 92a6d30e26 Add calendar to recommended apps 2026-07-14 23:04:24 -07:00
Neal ShahandDaniel Salazar 23e8705b03 Implement workers without cloudflare for local dev testing (#3389)
* typeify subdomains wip

* initial (untested) logic for LocalWorkerService

* Make it work, add lifecycle expiry since workers are process heavy in current implementation

* fix type errors

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-07-15 01:01:42 -04:00
Daniel Salazar 3240a4670a fix: admin gates (#3386) 2026-07-14 12:58:04 -07:00
Daniel Salazar 3bf83d40e9 fix: ui signup order (#3385) 2026-07-13 18:47:11 -07:00
Neal Shah c3368f553e Add GUI button for custom OIDC (#3384) 2026-07-13 20:32:37 -04:00
velzie 27def94d8b feat: support cross-origin-isolated login (#3338) 2026-07-13 17:36:56 -04:00
Neal Shah 2c139929ac Check if user is suspended on WebDAV basic auth and others (#3369) 2026-07-13 02:46:28 -04:00
Neal Shah b957f797da add support for ndjson logging (#3381)
* add support for ndjson logging

* move install location
2026-07-13 02:43:56 -04:00
jelveh 393500d565 Add calculator to recommended apps
Include 'calculator' in RECOMMENDED_APP_NAMES within RecommendedAppsService.ts so the calculator app is included in the recommended apps list.
2026-07-11 23:04:39 -07:00
jelveh a108837e82 Revert "Add delayed centered spinner for apps tab"
This reverts commit 859e88753f.
2026-07-11 15:21:59 -07:00
jelveh 859e88753f Add delayed centered spinner for apps tab
Show a subtle centered loading spinner for the Apps dashboard when fetching apps if the request is slow. Implements a 1.5s delay before showing the spinner and guarantees a minimum 1.5s visible time to avoid blinking; the spinner is only shown when the container is empty (so background refreshes keep existing tiles). Adds a finish helper to clear the spinner timer and await the minimum visible time before rendering results or errors. Also adds CSS for the spinner element and its animations.
2026-07-11 15:16:04 -07:00
Daniel Salazar 5160b44c4b fix: missing OIDC error messages (#3377)
* fix: missing OIDC error messages

* fix: card verification user prefill out for subs
2026-07-11 08:18:13 -07:00
Daniel Salazar 17a9485fb2 Add spans to drivers and high frequency paths (#3376) 2026-07-10 23:15:09 -07:00
Daniel Salazar d3c44f3d6c fix: cache healthcheck in mem for each node (#3375) 2026-07-10 17:41:09 -07:00
Neal Shah d49ba5281d add support for ignoring specific issues in healthcheck (#3374)
* add support for ignoring specific issues in healthcheck

* add mark-degraded to healthcheck
2026-07-10 17:08:03 -04:00
jelveh c107c80bd8 Open dashboard links at #home anchor 2026-07-10 13:22:58 -07:00
Daniel Salazar cf90e775bf fingerprint: use device info for rate limit too (#3372) 2026-07-09 20:22:56 -07:00
Daniel Salazar 0a46096c41 fix: ai rate limits (#3371) 2026-07-09 16:35:03 -07:00
Daniel Salazar 289a44ae79 fix: OIDC state (#3370) 2026-07-09 14:47:34 -07:00
Neal Shah 827e9bc343 fix possible XSS entrypoints (#3368) 2026-07-09 17:19:55 -04:00
Daniel Salazar f2ffaeb823 fix: add phone errors into kv for debugging (#3367) 2026-07-09 12:25:17 -07:00
Neal Shah 6f594f4b60 add 5.6 models (#3366) 2026-07-09 14:46:57 -04:00
jelveh 326be042f1 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-08 22:33:20 -07:00
jelveh 5b824fb745 Extract resolveTileDisplay and broaden app search
Move logic that determines a tile's displayed title and target link into resolveTileDisplay (use hostname/index_url for external apps) and have buildTileHtml consume it. Expose data-target-link on tiles. Update app search to match the displayed title, raw title, name, and uid so queries find website-shortcut tiles and external apps consistently (matching Home tab behavior).
2026-07-08 22:33:13 -07:00
Reynaldi Chernando cf6cb126b5 fix grok 4.5 release date (#3365) 2026-07-09 12:30:39 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 302df1edd2 update user pays model docs (#3364)
* update user pays model docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-09 08:54:04 +07:00
jelveh e212509ed3 Avoid clipping scaled icons in horizontal scroller
Add a negative top margin and matching top padding to the horizontal scroller to prevent first-row icons from being clipped when they scale on hover. Overflow-x:auto previously caused vertical clipping of scaled icons; margin-top:-8px and padding-top:8px bleed the clip edge upward so scaled icons remain visible without changing scroll behavior.
2026-07-08 18:35:57 -07:00
Daniel Salazar d3e3755d6d chore: cleanup uneeded ai messages (#3363) 2026-07-08 17:55:49 -07:00
jelveh 948ce6e529 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-07-08 17:51:38 -07:00
jelveh d117dc231e Implement paged apps launcher UI 2026-07-08 17:50:34 -07:00
Daniel Salazar eefa53e5bd fix: misc auth bug fixes (#3362) 2026-07-08 17:50:22 -07:00
Neal Shah db89fbfc52 add grok 4.5 (#3360) 2026-07-08 14:44:05 -04:00
358 changed files with 43306 additions and 9055 deletions

No files matched your search

+11 -2
View File
@@ -1,10 +1,19 @@
name: Backend Tests
# Runs on backend and extension changes. Tests are co-located with the
# code (src/backend/**/*.test.ts, extensions/**/*.test.ts), so test-only
# changes match the same globs. puter.js SDK changes are covered by
# puterjs-tests.yaml instead.
on:
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- 'src/docs/**'
paths:
- 'src/backend/**'
- 'extensions/**'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/backend-tests.yaml'
permissions:
contents: read
+145
View File
@@ -0,0 +1,145 @@
name: Puter.js API Tests
# Runs on puter.js SDK changes — the suites live in src/puter-js/tests,
# so test changes are covered by the same glob — plus the pieces the
# runners depend on: the worker preamble, the in-memory test env in
# testUtil, and the backend vitest config the API-tests config extends.
# Backend and extension changes run backend-tests.yaml instead.
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'src/puter-js/**'
- 'src/worker/**'
- 'src/backend/testUtil.ts'
- 'src/backend/vitest.config.ts'
- 'tools/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/puterjs-tests.yaml'
permissions:
contents: read
pull-requests: write
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
# The runners exercise the built artifacts: the SDK bundle
# (src/puter-js/dist) for node + browser, and the worker preamble
# (src/worker/dist) for workerd.
- name: Build puter.js SDK and worker preamble
run: npm run build:workerLib
- name: Install Playwright chromium
run: npx playwright install --with-deps chromium
- name: Run puter.js API tests (node, browser, workerd)
run: npm run test:puterjs
env:
CI: 'true'
# SDK coverage on both refs, reported as a PR comment — same shape as
# backend-tests.yaml. This run uses the istanbul-instrumented bundle
# (the `test` job above keeps exercising the production build).
coverage:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- ref: ${{ github.base_ref }}
artifact: base
- ref: ${{ github.head_ref }}
artifact: pr
steps:
- name: Checkout ${{ matrix.ref }}
uses: actions/checkout@v4
with:
ref: ${{ matrix.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Install Playwright chromium
run: npx playwright install --with-deps chromium
# The base leg is best-effort — it only enriches the PR comment with
# a comparison, and the base ref may predate the coverage script.
- name: Run puter.js API tests with coverage
continue-on-error: ${{ matrix.artifact == 'base' }}
run: npm run test:puterjs:coverage
env:
CI: 'true'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: puterjs-coverage-${{ matrix.artifact }}
path: src/puter-js/coverage/coverage-*.json
retention-days: 14
if-no-files-found: ignore
report-coverage:
needs: coverage
if: always()
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download PR coverage
uses: actions/download-artifact@v4
with:
name: puterjs-coverage-pr
path: src/puter-js/coverage
# May not exist (base ref without the coverage script, or a failed
# base run) — then report without the comparison column.
- name: Download base coverage
id: base-coverage
continue-on-error: true
uses: actions/download-artifact@v4
with:
name: puterjs-coverage-base
path: coverage-base
- name: Report coverage on PR
if: steps.base-coverage.outcome == 'success'
uses: davelosert/vitest-coverage-report-action@v2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
json-final-path: src/puter-js/coverage/coverage-final.json
json-summary-compare-path: coverage-base/coverage-summary.json
- name: Report coverage on PR (no base comparison)
if: steps.base-coverage.outcome != 'success'
uses: davelosert/vitest-coverage-report-action@v2
with:
name: puter.js SDK
json-summary-path: src/puter-js/coverage/coverage-summary.json
json-final-path: src/puter-js/coverage/coverage-final.json
+2 -1
View File
@@ -1,4 +1,5 @@
{
"tabWidth": 4,
"singleQuote": true
"singleQuote": true,
"plugins": ["prettier-plugin-jsdoc"]
}
+113 -60
View File
@@ -8,6 +8,8 @@ Use these as the source of truth before exploring further:
- [README.md](README.md) — project overview and quickstart.
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), and extensions.
- [doc/contributing-apis.md](doc/contributing-apis.md) — adding and maintaining public APIs end to end (backend surface → puter.js → types → docs → tests). Follow it for any API work.
- [doc/pagination.md](doc/pagination.md) — the one pagination convention for list APIs (limit/cursor/offset/includeTotal, envelope shape, cursor semantics).
- [doc/self-hosting.md](doc/self-hosting.md) — running Puter outside hosted infra.
- [CONTRIBUTING.md](CONTRIBUTING.md) — testing, security, AI-assisted code, PR conventions, Boy Scout Rule.
- [SECURITY.md](SECURITY.md) — how to report vulnerabilities (do not file them publicly).
@@ -16,70 +18,21 @@ Use these as the source of truth before exploring further:
---
## Backend
## Repo-wide conventions
The backend is organized as a layered stack inspired by Controller–Service–Repository with dependency injection. Every layer only depends on the layers beneath it, and `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires the whole thing together.
These apply everywhere — backend, puter.js, and GUI.
### Layers (top → bottom)
### Language & files
| Layer | Lives in | Responsibility |
| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Controllers** | [src/backend/controllers/](src/backend/controllers/) | Route handlers. Parse + validate input, apply per-route gates (auth, subdomain, rate limit, body parsers via `RouteOptions`), call services, format responses. |
| **Drivers** | [src/backend/drivers/](src/backend/drivers/) | Optional. RPC-style handlers exposed over `/drivers/*`. Thin shells that validate RPC inputs and call into services/stores. |
| **Services** | [src/backend/services/](src/backend/services/) | Business logic. Assume the caller is already authenticated/authorized — services do not run auth gates themselves. |
| **Stores** | [src/backend/stores/](src/backend/stores/) | Persistence. Wraps clients with the domain shape services consume (rows, entities, KV namespaces). |
| **Clients** | [src/backend/clients/](src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, dynamo, email, event bus). Knows protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | Flat, typed config object every layer receives at construction. |
Each layer receives the layers beneath it through its constructor. Dependencies are explicit and traceable from `PuterServer`.
### Cross-layer rules
- **Don't reach across layers.** Controllers do not poke clients directly; services do not register routes. If you want to, the abstraction is wrong — fix the abstraction.
- **Don't call sideways within a layer for code reuse.** If two services need the same logic, lift it into a util/helper. Services should not depend on other services for shared code.
- **Prefer explicit arguments over `Context` (ALS).** Reach for [Context](src/backend/core/context.ts) only when the value is genuinely request-scoped and would otherwise thread through many layers. Today it's used sparingly — mostly for `actor` and `req`.
### Extensions
Extensions live in [extensions/](extensions/) and parallel the layered stack. They are for **non-crucial parts of the system** — things Puter still works without if removed. If a feature is load-bearing for clients, it belongs in core, not in an extension (see [whoami](extensions/whoami.ts) as the cautionary example).
The `extension` global ([src/backend/extensions.ts](src/backend/extensions.ts)) exposes:
- `extension.registerClient/Store/Service/Driver/Controller(name, Class)` for first-class additions.
- `extension.on(event, handler)` and `extension.get/post/put/delete/patch/use(path, opts?, handler)` for the lightweight common case. `opts` is the same `RouteOptions` controllers use.
- `extension.import('client' | 'store' | 'service' | 'controller' | 'driver')` returns a lazy proxy to instantiated objects. `extension.config` exposes live config.
### Language & file conventions
- **Modules:** We transpile and build as needed — write ES modules, not CommonJS.
- **TypeScript preferred for new files.** Existing JS is fine; convert opportunistically when you're already touching a file.
- **Reuse types before inventing them.** Search for an existing type first; extend it if close. Only define a new type when nothing fits.
- **Make new types findable.** Co-locate them with the layer/module that owns them, export from the obvious entry point, and use a descriptive `PascalCase` name. Don't hide types in random files where future readers won't grep them.
- **Naming:** `camelCase` for variables/functions, `PascalCase` for classes and for files containing a class (`AuthService.ts`, `KVStoreDriver.ts`).
- Write ES modules, not CommonJS — we transpile and build as needed.
- TypeScript preferred for new files in the backend and extensions; convert existing JS there opportunistically when you're already touching a file. The GUI and puter.js are plain JavaScript — don't introduce TypeScript files in them. In puter.js, type with JSDoc instead (see the puter.js section for what must be typed).
- **Reuse before adding.** Search for an existing type, helper, or implementation and extend it; only add a new one when nothing suitable exists.
- Make new types findable: descriptive `PascalCase` names, exported from the obvious entry point. A type used from many places belongs in the owning module's `types.ts` (e.g. [src/backend/controllers/types.ts](src/backend/controllers/types.ts)) rather than bloating a logic file; a type with a single consumer can stay next to it.
- Naming: `camelCase` for variables/functions, `PascalCase` for classes and files containing a class (`AuthService.ts`). **Prefer `camelCase` for all new code** — new local variables, parameters, functions, and internal object properties. `snake_case` is reserved for identifiers that are part of an external contract: wire/JSON keys sent to or received from the backend, stable API error codes, public option names that already ship as `snake_case`, and established `puter_*` namespaced fields. Don't rename those (it's a breaking change), but don't introduce new `snake_case` alongside them either — a camelCase local can carry a value into a `snake_case` wire key (`{ operation_id: operationId }`).
### Comments
Keep comments light. Prefer self-documenting code — clear names, small functions, obvious flow. Add a comment **only** when:
- The _why_ is non-obvious (a hidden constraint, a workaround, a subtle invariant).
- A non-trivial usage detail would otherwise trip the next reader.
Don't restate what the code already says. Don't write comments that reference the current task or PR or version of something — those rot.
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the _why_ in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
**Use plain ASCII `-` in comment section dividers, not box-drawing characters.** Write `// -- Section name --`, not `// ── Section name ──`. The `─` (U+2500) character renders inconsistently in some editors and diff tools and adds nothing over `-`.
### Tests
When adding new behavior (function, endpoint, driver method, branch of logic), add a test for it.
- Use Vitest for unit and integration tests. Test files go next to the code they test, named `*.test.ts` or `*.test.js`.
- **Mock data, not methods.** Stub the inputs (fixtures, fake rows, sample payloads) rather than mocking the function under test or the layer beneath it. Over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service), not within the same layer you're testing.
- Prefer test server over mocking deps. We have existing test utilities for running full test server to boot deps based on config. (see [src/backend/testUtil.ts](src/backend/testUtil.ts)).
- **Hit a real database / real client where reasonable.** Integration shapes catch the things unit tests with mocks miss.
- **Regression tests for bug fixes.** A test that fails before your fix and passes after is the cheapest insurance against the bug coming back.
- If something is genuinely hard to test (UI animation, third-party glue), skip it but say so in the PR.
Keep comments light; prefer self-documenting code. Comment only when the _why_ is non-obvious or a usage detail would trip the next reader. Use `//` for single lines and `/** ... */` JSDoc when it genuinely needs more — if a comment runs long, it's probably too long. Don't restate the code and don't reference the current task, PR, or version — those rot. **No ticket references** (`PUT-1234`, `// fix for FOO-99`) in code, comments, or test names; describe the why in domain terms, not project-management terms. Use plain ASCII `-` in comment section dividers (`// -- Section --`), never box-drawing characters.
### Security & privacy
@@ -94,6 +47,106 @@ When in doubt, return less. Auth-, permission-, or data-export-related changes d
### Working rules of thumb
- **Run it, don't just compile it.** "It type-checks" is not "it works." Exercise the code path end-to-end at least once.
- **Read the neighbors before writing.** Match the shape of similar things already in the tree. If you genuinely think the existing pattern is wrong, raise it — don't quietly diverge.
- **Boy Scout Rule, proportional to the change.** Fix the obvious typo, dead import, or missing typehint in files you're already touching. Don't ride a refactor along with a bug fix — that just makes review harder.
- **Read the neighbors before writing.** Match the shape of similar things already in the tree. If you think the existing pattern is wrong, raise it — don't quietly diverge.
- **Test new behavior.** Every new function, endpoint, driver method, or logic branch gets a test; every bug fix gets a regression test that fails before the fix. If something is genuinely hard to test, skip it but say so in the PR.
- **Boy Scout Rule, proportional to the change.** Fix the obvious typo or dead import in files you're already touching; don't ride a refactor along with a bug fix.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't defend in review is not.
---
## Backend
A layered stack with explicit dependency injection: each layer depends only on the layers beneath it, receives them through its constructor, and `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires the whole thing together. [doc/architecture.md](doc/architecture.md) is the full reference.
| Layer (top → bottom) | Lives in | Responsibility |
| -------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Controllers** | [src/backend/controllers/](src/backend/controllers/) | Route handlers: parse/validate input, per-route gates via `RouteOptions` (auth, subdomain, rate limit, body parsers), call services, format responses. |
| **Drivers** | [src/backend/drivers/](src/backend/drivers/) | Optional RPC-style handlers on `/drivers/*`; thin shells that validate inputs and call services/stores. |
| **Services** | [src/backend/services/](src/backend/services/) | Business logic. Assume the caller is already authenticated/authorized. |
| **Stores** | [src/backend/stores/](src/backend/stores/) | Persistence; wraps clients in the domain shapes services consume. |
| **Clients** | [src/backend/clients/](src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, email, event bus). Protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | Flat, typed config object every layer receives at construction. |
A public API can be exposed through either a controller or a driver — both are supported; prefer a controller when you need fine-grained control over routes and gates. [doc/contributing-apis.md](doc/contributing-apis.md) has the decision guide with links to the decorators and middleware for each.
Cross-layer rules:
- **Don't reach across layers.** Controllers don't poke clients; services don't register routes. If you want to, the abstraction is wrong — fix the abstraction.
- **Don't call sideways within a layer for code reuse.** Two services needing the same logic means a util/helper, not a service-to-service dependency.
- **Prefer explicit arguments over `Context` (ALS).** Reach for [Context](src/backend/core/context.ts) only for genuinely request-scoped values that would otherwise thread through many layers — today mostly `actor` and `req`.
### Extensions
[extensions/](extensions/) parallels the layered stack and is for **non-crucial parts of the system** — Puter still works with the extension removed. If core needs to call it, it belongs in core, not an extension (see [whoami](extensions/whoami.ts) as the cautionary example). The `extension` global ([src/backend/extensions.ts](src/backend/extensions.ts)) exposes:
- `extension.registerClient/Store/Service/Driver/Controller(name, Class)` for first-class additions.
- `extension.on(event, handler)` and `extension.get/post/put/delete/patch/use(path, opts?, handler)` for the lightweight common case. `opts` is the same `RouteOptions` controllers use.
- `extension.import('client' | 'store' | 'service' | 'controller' | 'driver')` — lazy proxy to instantiated core objects. `extension.config` exposes live config.
Follow the same layered structure inside an extension — unless it only needs a few route handlers, in which case the lightweight helpers are enough on their own.
### Backend tests
- Vitest; test files sit next to the code they test (`*.test.ts` / `*.test.js`). Run with `npm run test:backend`.
- **Mock data, not methods.** Stub inputs (fixtures, fake rows, payloads), not the function under test or the layer beneath it — over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service).
- **Prefer the test server over mocking deps.** `setupPuterTestEnv()` in [src/backend/testUtil.ts](src/backend/testUtil.ts) boots a fully in-memory backend; hit a real database/client shape where reasonable — integration shapes catch what mocked unit tests miss.
---
## puter.js (the SDK)
[src/puter-js/](src/puter-js/) is the public SDK. It ships live from `https://js.puter.com/v2/` with no version pinning — every existing app picks up changes immediately. Treat every observable behavior (signatures, response fields, error codes) as something a production app depends on.
Layout: SDK modules in [src/puter-js/src/modules/](src/puter-js/src/modules/) (one file or directory per area — `FileSystem/`, `KV.js`, `ai/`, …), shared plumbing in [src/puter-js/src/lib/](src/puter-js/src/lib/), hand-maintained type declarations in [src/puter-js/types/](src/puter-js/types/), API tests in [src/puter-js/tests/api/](src/puter-js/tests/api/), UI e2e tests in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/), developer docs in [src/docs/](src/docs/).
Language & typing: puter.js source is plain JavaScript — never TypeScript files — typed via JSDoc. Reference the hand-maintained declarations in [src/puter-js/types/](src/puter-js/types/) with `import(...)` types rather than re-declaring shapes:
```js
/** @typedef {import('../../types/modules/ai').ChatOptions} ChatOptions */
/** @type {ChatOptions} */
const options = { model: 'gpt-5-nano' };
```
When a shape only exists locally, declare it as an inline object literal — not the `@typedef {Object}` + `@property` list form — and use `unknown` over `*`:
```js
/** @typedef {{ key: string, value: unknown }} KVEntry */
```
Public (exposed) methods must carry JSDoc types — parameters and return value, matching the `.d.ts` declarations exactly. Unexposed/private helpers are typed at the contributor's discretion: annotate where it helps the next reader, and either way keep them clean.
Typing in JS files is encouraged: annotate with JSDoc `@type`/`@param`/`@returns` using the TypeScript type system, and define shared shapes with `@typedef`. API types must not be `unknown` or untyped `...args` — spell out the real parameter and return shapes; the only exception is values passed through transparently to an upstream layer that owns their type. For example:
```js
/**
* @typedef {{key:string, value: unknown}} KVEntry
*/
/** @type {KVEntry[]} */
let entries = [];
```
Every SDK change carries all five of the following — a puter.js PR missing one is incomplete:
1. **Backward compatibility.** Mandatory unless a maintainer explicitly signs off on a break. Existing call signatures keep working (including both positional and options-object forms where a method supports them); new parameters are optional with defaults that preserve old behavior; never rename or repurpose existing params, response fields, or error codes. New parameter names are `camelCase` (existing `snake_case` stays for compatibility). Say in the PR how existing callers are unaffected.
2. **Tests.** Add or extend a suite in [tests/api/suites/](src/puter-js/tests/api/suites/) (`<area>.suite.ts`; register new suites in `suites/index.ts` — no globbing). One suite runs unchanged on node, browser, and workerd via `npm run test:puterjs`; never write a per-platform test. The runners execute the **built** bundle — run `npm run build:workerLib` after SDK changes or the suite silently tests stale code. For `puter.ui.*` methods rendered by the desktop, use the Playwright e2e harness instead — see [src/puter-js/TESTING.md](src/puter-js/TESTING.md).
3. **Docs.** New or changed APIs update [src/docs/src/](src/docs/src/): the method page (`<Area>/<method>.md`, with frontmatter and a runnable example) and the area overview when the surface changes. Docs are the contract users code against — signatures, defaults, and return shapes must match the implementation exactly.
4. **Types.** Update [src/puter-js/types/modules/](src/puter-js/types/modules/)`<module>.d.ts` and re-export new types through `index.d.ts` / `types/puter.d.ts`. Declarations must match runtime behavior exactly — a wrong type is worse than a missing one.
5. **Error handling.** Reject/throw `{ message, code }` objects with stable `snake_case` codes, matching the existing modules (see `KV.js`). Validate cheap preconditions client-side before making the network call; pass backend errors through unchanged rather than swallowing or re-wrapping them. Error codes are API surface — changing one is a breaking change.
[doc/contributing-apis.md](doc/contributing-apis.md) walks the full lifecycle of adding an API across backend + SDK.
---
## GUI
[src/gui/](src/gui/) is the Puter desktop: deliberately plain JavaScript + jQuery with HTML-string templates. Don't introduce a UI framework or a new rendering pattern.
The guiding rule here is **conformity over novelty** — match the existing design and code structure even where you'd personally choose differently. A visually or structurally divergent addition is a defect even when it works.
- **Reuse existing UI primitives before writing new ones.** Windows and dialogs are `UIWindow*` functions in [src/gui/src/UI/](src/gui/src/UI/); generic pieces already exist (`UIAlert`, `UIPrompt`, `UIContextMenu`, `UINotification`, `UIPopover`, widgets in [UI/Components/](src/gui/src/UI/Components/)). A new window should read like its neighbors: an async function taking an options object, composing an HTML string, wiring behavior with jQuery, delegating to `UIWindow(...)`.
- **Match the visual language.** Use existing CSS classes (`button`, `button-primary`, window chrome, form styles) and copy the layout patterns of neighboring windows; new styles go in [src/gui/src/css/](src/gui/src/css/) following existing conventions. Verify anything positional (menus, overlays, z-index) on both desktop and mobile viewports.
- **i18n every user-facing string.** No hardcoded UI text — use `i18n('key')` and add the key to [src/gui/src/i18n/translations/en.js](src/gui/src/i18n/translations/en.js). Run `npm run check-translations` before opening the PR.
- **Shared logic goes in helpers/services.** Reusable non-UI logic belongs in [src/gui/src/helpers/](src/gui/src/helpers/) or [src/gui/src/services/](src/gui/src/services/), not copy-pasted between windows.
- **Tests.** Vitest is wired for the GUI (`src/**/*.test.js`; see [appOrder.test.js](src/gui/src/UI/Dashboard/appOrder.test.js) for the shape). Extract pure logic into functions and test those. Desktop behavior driven through puter.js (`puter.ui.*`) is covered by the Playwright harness in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/) — add a spec there when you change how the desktop renders SDK-driven UI.
+4 -2
View File
@@ -16,6 +16,8 @@ Add tests for new behavior, endpoints, or bug fixes. If something's genuinely ha
Match the shape of similar code already in the repo. [doc/architecture.md](doc/architecture.md) is the source of truth for layers, wiring, and naming. If you think a pattern is wrong, raise it — don't quietly diverge.
In plain-JS files, typing is encouraged via JSDoc `@type` annotations using the TypeScript type system, with `@typedef` for shared shapes. Don't type API surfaces as `unknown` or untyped `...args` unless the values are passed through transparently to an upstream layer that owns their type.
## 3. Don't expose system or user information
Scan your diff for stray logs, debug routes, internal paths, secrets, tokens, or user data in errors/responses. When in doubt, return less. Flag any auth, permission, or data-export changes in the PR description.
@@ -26,9 +28,9 @@ For private security reports, see [SECURITY.md](SECURITY.md).
Don't commit code you couldn't have written, debugged, or defended yourself. Read the diff, run it, and be ready to explain it in review.
## 5. Update docs for API changes
## 5. Adding or changing APIs
If you change puter-js APIs (drivers or endpoints used by puter-js), update [developer docs](src/docs/). puter-js itself may also need updating.
If you add or change a public API (an endpoint, driver method, or puter-js method), follow [doc/contributing-apis.md](doc/contributing-apis.md) — backward compatibility, [developer docs](src/docs/), types, and tests all move in the same PR.
## 6. Boy Scout Rule — leave it 1% better
+6
View File
@@ -16,6 +16,12 @@
// `dev` opens a browser on boot, skips blocked-email checks, and runs the
// dev-time webpack watcher; `prod` serves pre-built bundles.
"env": "dev",
// Console output format. `json` replaces the global console so every call
// emits one structured JSON line (level, timestamp, msg, and the active
// request's trace id) — one event per call, so a line-oriented log
// collector can't split stack traces across events and level filtering
// works. Unset (the default) leaves console output human-readable.
"log_format": "text",
"version": "0.0.0",
// Stable identity for this server node — used by pager alerts and
// graceful-shutdown coordination.
+108
View File
@@ -0,0 +1,108 @@
# Contributing APIs
How to add a new public API to Puter, and how to maintain one that already exists. "Public API" means anything applications can call: HTTP endpoints, `/drivers/*` methods, and the puter.js methods that wrap them. This guide is written for human contributors and AI agents alike — [AGENTS.md](../AGENTS.md) defers to it for API work.
Companion docs: [architecture.md](architecture.md) (backend layering), [pagination.md](pagination.md) (list APIs), [src/puter-js/tests/api/README.md](../src/puter-js/tests/api/README.md) (SDK test environment).
## Rule zero: don't break callers
puter.js is served live from `https://js.puter.com/v2/` with no version pinning — every app in existence picks up your change the moment it deploys, and the backend endpoints underneath have the same property. Assume every observable behavior (parameter handling, response fields, error codes, ordering) has someone depending on it.
Every change is backward compatible unless a maintainer has explicitly agreed to a break beforehand.
## Core or extension?
The first decision is where the API lives.
If the API is **not crucial to core functionality — nothing in core will call it — prefer an extension** over wiring it into core. Extensions live in [extensions/](../extensions/), parallel the core layered stack, and reach core through `extension.import(...)`:
```js
const services = extension.import('service');
const stores = extension.import('store');
extension.registerDriver('myFeature', MyFeatureDriver); // first-class driver
extension.post('/my-feature/frobnicate', opts, handler); // or plain routes
```
Follow the same layered structure inside the extension (driver/controller → service → store) unless it genuinely only needs a couple of route handlers — then the lightweight `extension.get/post/...` helpers are enough on their own.
The test is the direction of dependency: Puter must still work with the extension removed. The moment core needs to call your API, it belongs in core — see [whoami](../extensions/whoami.ts) for the cautionary example of a load-bearing "extension".
## Adding a new API
Work through all seven steps; the PR is complete when every one is.
### 1. Design the surface first
- Sketch the signature, options, return shape, and error cases before writing code. Find the two or three most similar existing APIs and match their conventions.
- New parameter and field names are `camelCase`. (Existing `snake_case` names stay where they already exist.)
- Anything returning a list follows the [pagination convention](pagination.md): `limit`/`cursor` in, `{ items, cursor, total }` envelope out.
- Prefer an options object over a growing list of positional parameters, but keep the common case callable with a single argument where siblings do.
### 2. Backend
Follow the layered stack ([architecture.md](architecture.md)): a controller or driver at the edge, business logic in a service, persistence in a store. The edge parses and validates input and applies gates; services assume the caller is already authorized. Return exactly what the caller needs and no more — every response field you ship is permanent — and use stable `snake_case` error codes.
#### Controller or driver?
Both are supported ways to define an API. **Prefer a controller when you need fine-grained control** — URL shape, HTTP verbs, per-route gates, response and streaming formats. A **driver** fits when the API is a set of RPC methods implementing one of the named interfaces on `/drivers/*` (`puter-kvstore`, `puter-chat-completion`, …) and the generic driver plumbing — one call envelope, interchangeable implementations, per-method policies — covers what you need.
- **Controller:** extend `PuterController` ([src/backend/controllers/types.ts](../src/backend/controllers/types.ts)) and declare routes with the `@Controller(prefix)` class decorator plus `@Get`/`@Post`/… method decorators ([src/backend/core/http/decorators.ts](../src/backend/core/http/decorators.ts)), each taking `(path, routeOptions)` — or override `registerRoutes(router)` imperatively. Core controllers register in [src/backend/controllers/index.ts](../src/backend/controllers/index.ts); extensions use `extension.registerController(...)` or the plain route helpers.
- **Driver:** extend `PuterDriver` ([src/backend/drivers/types.ts](../src/backend/drivers/types.ts)) and mark it with the `@Driver(interfaceName, opts)` decorator ([src/backend/drivers/decorators.ts](../src/backend/drivers/decorators.ts)), which also declares its policies. Core drivers register in [src/backend/drivers/index.ts](../src/backend/drivers/index.ts); extensions use `extension.registerDriver(...)`.
#### Middleware and gates
- **Controller routes** (and extension routes — same options) take [`RouteOptions`](../src/backend/core/http/types.ts): auth gates (`requireAuth`, `requireUserActor`, `noUserSession`, `adminOnly`, `allowedAppIds`, and the access-token controls), `subdomain` routing, per-route `rateLimit`, body parsers, and arbitrary extra `middleware`. The auth flavors are subtle and default-deny — read the JSDoc on each field before picking.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), and `noUserSession`. The `/drivers/call` surface enforces them.
### 3. puter.js
- Add the method to the matching module in [src/puter-js/src/modules/](../src/puter-js/src/modules/), matching the calling conventions of its siblings (promise-returning; positional shortcut plus options form where that's the local pattern).
- Validate cheap preconditions client-side and throw `{ message, code }` objects; pass backend errors through unchanged rather than swallowing or re-wrapping them.
### 4. Types
- Add or extend the declaration in [src/puter-js/types/modules/](../src/puter-js/types/modules/)`<module>.d.ts` and re-export new types through `index.d.ts`. Declarations must match the runtime exactly — optionality, defaults, and return types included.
### 5. Docs
- Add the method page at [src/docs/src/](../src/docs/src/)`<Area>/<method>.md` — frontmatter (`title`, `description`, `platforms`), syntax, parameters, return value, and at least one runnable example — and update the area overview (`<Area>.md`). Copy the structure of an existing page.
### 6. Tests
- **Backend:** colocated Vitest tests; prefer the in-memory test server (`setupPuterTestEnv` in [src/backend/testUtil.ts](../src/backend/testUtil.ts)) over mocking.
- **SDK:** add cases to [src/puter-js/tests/api/suites/](../src/puter-js/tests/api/suites/)`<area>.suite.ts` (register new suites in `suites/index.ts`). One suite runs on node, browser, and workerd via `npm run test:puterjs` — never write per-platform tests, and rebuild first with `npm run build:workerLib` since the runners execute the built bundle.
- **Desktop-rendered UI** (`puter.ui.*`): add a Playwright spec per [src/puter-js/TESTING.md](../src/puter-js/TESTING.md).
### 7. Security pass
Scan the diff before opening the PR: no internals leaked in errors or logs, no over-broad responses, auth gates present. Flag anything auth-, permission-, or data-export-related in the PR description.
## Maintaining an existing API
Changes are **additive by default**:
- New parameters are optional, with defaults that reproduce the old behavior exactly.
- Never rename, repurpose, or remove existing parameters, response fields, or error codes. Don't change types, ordering guarantees, or which fields are present when.
- New behavior that could surprise existing callers goes behind an opt-in flag.
- Docs, types, and tests move in the same PR as the behavior. A signature change with stale docs is a bug — the docs are the contract users code against.
- Bug fixes come with a regression test that fails before the fix. Be suspicious of fixes that change observable behavior: someone may depend on the bug. When in doubt, ask a maintainer.
### Breaking changes
Rare and deliberate. In order: explicit maintainer sign-off, a documented migration path, and a rollout plan — typically new surface added first, old surface deprecated, old surface removed much later, if ever. Never break as a side effect of a refactor.
### Deprecating
The old surface keeps working. Mark it `@deprecated` in the type declarations, note the replacement on its docs page, and stop using it in examples. Removal is a separate, maintainer-approved decision.
## Definition of done
- [ ] Backward compatible (or the break was explicitly approved)
- [ ] Right home: extension if core never calls it; layered structure either way
- [ ] puter.js method matches sibling conventions; errors are `{ message, code }` with stable codes
- [ ] Types updated and matching runtime behavior
- [ ] Docs page + area overview updated, with a runnable example
- [ ] Tests: backend + three-platform SDK suite (+ e2e for desktop-rendered UI)
- [ ] Security pass on the diff
- [ ] You ran it end to end
+100
View File
@@ -0,0 +1,100 @@
# Pagination convention for list APIs
Every list endpoint follows one wire contract. This document is the source of
truth for adding pagination to a new or existing list surface.
## Wire contract
Requests accept:
| Param | Type | Meaning |
| --- | --- | --- |
| `limit` | number | Maximum items per page. Each endpoint documents its cap and default. |
| `cursor` | string \| null | Opaque continuation token. `null` (or any presence of the key) requests the first page. |
| `offset` | number | Skip N items. Legacy/discouraged — see below. Cannot be combined with `cursor`. |
| `includeTotal` | boolean | Adds `total` to the response. |
Paginated responses are an envelope:
```json
{ "items": [...], "cursor": "…", "total": 123 }
```
- `cursor` is present only while more pages exist. Clients iterate until it
is absent.
- `total` is present only when the request set `includeTotal`.
- **Pages may be short.** Post-query filtering (TTL expiry, permission
checks) can shrink a page below `limit` — or even to zero — while a
`cursor` is still returned. Never use `items.length < limit` as an
end-of-list signal.
New request params are camelCase (`includeTotal`, `fetchUntilFull`).
Pre-existing snake_case params stay for compatibility.
## Backward compatibility
Requests without pagination params keep returning the full result in the
legacy shape (bare array) forever — old clients never break.
The envelope trigger depends on the endpoint's history:
- Endpoints where `limit` pre-dates the convention (readdir, subdomain
`select`) return the envelope only when the request contains `cursor`
(including `null`) or `includeTotal`; `limit`/`offset`-only requests keep
the bare array.
- Endpoints where every pagination param is new (kv `list`, workers
`getFilePaths`) return the envelope when any pagination param is present.
## Cursors
Cursors are opaque base64-encoded JSON, produced and consumed only by the
backend (`src/backend/util/pagination.ts`). What a cursor wraps is an
implementation detail per store:
- DynamoDB-backed lists wrap `LastEvaluatedKey`.
- SQL-backed lists wrap a keyset position — `(sortValue, id)` of the last
row — and the query seeks past it (`WHERE (col, id) > (?, ?) ORDER BY col,
id`). Any SQL list gaining a cursor must have a deterministic `ORDER BY`
ending in a unique tiebreaker (`id`).
Cursors that carry a sort also pin it: a request that passes a cursor plus a
conflicting sort is rejected with 400.
## Offset
SQL-backed endpoints support `offset` natively. DynamoDB-backed endpoints
(kv) emulate it by advancing past skipped items with `Select: COUNT` queries
— no item data is transferred, but read capacity is still consumed for
everything skipped, and the caller is metered for it. **Offset is supported
for parity, not recommended** — cost grows linearly with the offset, so it is
capped (kv: 5000). Use cursors.
## Totals (`includeTotal`)
- SQL: `SELECT COUNT(*)` with the same WHERE clause as the listing.
- DynamoDB: a `Select: COUNT` loop over the query (TTL-filtered), metered to
the caller. Cost is proportional to the total item count — request the
total on the first page only, not every page.
- Where visibility is decided per-actor after the query (protected apps in
the catalog listing), `total` approximates the visible set: it counts
non-protected plus caller-owned rows, and misses rows visible only through
explicit permission grants.
## fetchUntilFull (kv only)
DynamoDB applies `Limit` before its filter expression, so TTL-filtered pages
are structurally short. `fetchUntilFull: true` makes the backend keep
fetching (bounded number of continuation queries) until the page holds
`limit` items or the keyset is exhausted. Requires `limit`. If the bound is
hit, the response simply carries a cursor — still convention-legal.
## Adding pagination to a new endpoint
1. Use `encodeCursor`/`decodeCursor`/`normalizeLimit`/`normalizeOffset` from
`src/backend/util/pagination.ts`.
2. Push equality filters into the query so pages and counts operate on the
true result set; only genuinely per-actor filtering may remain post-query
(short-pages rule covers it).
3. Fetch `limit + 1` rows to detect whether a next page exists (SQL), or use
`LastEvaluatedKey` (DynamoDB).
4. Keep the no-params request returning the legacy full result.
+86
View File
@@ -50,6 +50,32 @@ const seedOwnedApp = async (prefix: string) => {
return { owner, app: app! };
};
const seedUser = async (prefix: string, email: string | null = null) =>
server.stores.user.create({
username: `${prefix}_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email,
});
const grantAuthenticated = async (appId: number, userId: number) => {
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, 'flag:app-is-authenticated', null],
);
};
const grantEmailRead = async (
appId: number,
userId: number,
userUuid: string,
) => {
await server.clients.db.write(
`INSERT INTO user_to_app_permissions (user_id, app_id, permission, extra) VALUES (?, ?, ?, ?)`,
[userId, appId, `user:${userUuid}:email:read`, null],
);
};
describe('appTelemetry driver — get_users', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
await expect(driver.get_users({})).rejects.toMatchObject({
@@ -117,6 +143,66 @@ describe('appTelemetry driver — get_users', () => {
expect(result).toEqual([]);
});
it('omits user_email for a user who did not grant email:read', async () => {
const { owner, app } = await seedOwnedApp('noemail');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(row.user).toBe(member.username);
expect(row.user_uuid).toBe(member.uuid);
// No grant → the field must be absent (not just null), so the email
// never leaks to the app owner.
expect(Object.prototype.hasOwnProperty.call(row, 'user_email')).toBe(
false,
);
});
it('returns user_email when the user granted email:read to the app', async () => {
const { owner, app } = await seedOwnedApp('withemail');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
await grantEmailRead(
app.id as number,
member.id as number,
member.uuid,
);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(row.user_uuid).toBe(member.uuid);
expect(row.user_email).toBe('[email protected]');
});
it('does not leak email granted to a *different* app', async () => {
const { owner, app } = await seedOwnedApp('appA');
const { app: otherApp } = await seedOwnedApp('appB');
const member = await seedUser('member', '[email protected]');
await grantAuthenticated(app.id as number, member.id as number);
// Grant email:read against the OTHER app only.
await grantEmailRead(
otherApp.id as number,
member.id as number,
member.uuid,
);
const [row] = (await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
() => driver.get_users({ app_uuid: app.uid }),
)) as Array<Record<string, unknown>>;
expect(Object.prototype.hasOwnProperty.call(row, 'user_email')).toBe(
false,
);
});
});
describe('appTelemetry driver — user_count', () => {
+36 -5
View File
@@ -70,7 +70,9 @@ export class AppTelemetryDriver extends PuterDriver {
app_uuid?: string;
limit?: unknown;
offset?: unknown;
} = {}): Promise<Array<{ user: string; user_uuid: string }>> {
} = {}): Promise<
Array<{ user: string; user_uuid: string; user_email?: string | null }>
> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
const safeLimit = parseIntParam(limit, {
@@ -105,16 +107,45 @@ export class AppTelemetryDriver extends PuterDriver {
.catch(() => false);
if (!ownsApp) throw new HttpError(403, 'Permission denied');
const appId = (app as { id: number }).id;
const users = (await this.clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
`SELECT u.id, u.username, u.uuid, u.email FROM user_to_app_permissions p
INNER JOIN ${this.clients.db.quoteIdentifier('user')} u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
[(app as { id: number }).id, safeLimit, safeOffset],
)) as Array<{ username: string; uuid: string }>;
[appId, safeLimit, safeOffset],
)) as Array<{
id: number;
username: string;
uuid: string;
email: string | null;
}>;
return users.map((e) => ({ user: e.username, user_uuid: e.uuid }));
// Only surface a user's email if *that user* granted this app the
// `user:<their-uuid>:email:read` permission — the same grant
// `puter.perms.requestEmail()` obtains and `whoami` honours. This is a
// per-user check keyed on the app (not the calling owner-actor): a
// user may have authenticated into the app without sharing their
// email. Resolve the whole page in one query.
const emailPermitted = new Set<number>();
if (users.length > 0) {
const permStrings = users.map((u) => `user:${u.uuid}:email:read`);
const placeholders = permStrings.map(() => '?').join(', ');
const grants = (await this.clients.db.read(
`SELECT user_id FROM user_to_app_permissions
WHERE app_id = ? AND permission IN (${placeholders})`,
[appId, ...permStrings],
)) as Array<{ user_id: number }>;
for (const g of grants) emailPermitted.add(g.user_id);
}
return users.map((e) =>
emailPermitted.has(e.id)
? { user: e.username, user_uuid: e.uuid, user_email: e.email }
: { user: e.username, user_uuid: e.uuid },
);
}
/** Count of users who have authenticated into the given app. */
+43
View File
@@ -158,6 +158,49 @@ describe('whoami extension — handleWhoami', () => {
expect(body.directories).toBeUndefined();
});
it('redacts tmp_password from metadata for user actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
hasDevAccountAccess: true,
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
const metadata = body.metadata as Record<string, unknown>;
expect(metadata.tmp_password).toBeUndefined();
// Other metadata keys still reach the user's own client.
expect(metadata.hasDevAccountAccess).toBe(true);
expect(body.hasDevAccountAccess).toBe(true);
});
it('never sends user metadata to app actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
});
const { res, captured } = makeRes();
await runWithContext(
{
actor: {
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
},
},
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.metadata).toBeUndefined();
expect(JSON.stringify(body)).not.toContain('bootstrap-secret');
});
it('marks the user as oidc_only when password is null', async () => {
const slug = Math.random().toString(36).slice(2, 8);
const oidcUser = await server.stores.user.create({
+5 -1
View File
@@ -65,6 +65,9 @@ export const handleWhoami = async (
}
}
const metadata = user.metadata ? { ...user.metadata } : user.metadata;
if (metadata) delete metadata.tmp_password;
const details: Record<string, unknown> = {
username: user.username,
uuid: user.uuid,
@@ -98,7 +101,7 @@ export const handleWhoami = async (
human_readable_age: user.timestamp
? timeago.format(new Date(user.timestamp as string))
: null,
metadata: user.metadata,
metadata,
hasDevAccountAccess: !!user.metadata?.hasDevAccountAccess,
};
@@ -162,6 +165,7 @@ export const handleWhoami = async (
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.is_user_token;
delete details.metadata;
}
if (actor.app) {
+1270 -73
View File
File diff suppressed because it is too large. Load diff
+14
View File
@@ -10,6 +10,7 @@
"lib": "lib"
},
"devDependencies": {
"@babel/core": "^7.29.7",
"@eslint/js": "^9.35.0",
"@playwright/test": "^1.56.1",
"@stylistic/eslint-plugin": "^5.3.1",
@@ -20,6 +21,8 @@
"@typescript-eslint/parser": "^8.46.1",
"@vitest/coverage-v8": "^4.0.14",
"@vitest/ui": "^4.0.14",
"babel-loader": "^10.1.1",
"babel-plugin-istanbul": "^8.0.0",
"chalk": "^4.1.0",
"clean-css": "^5.3.2",
"dotenv": "^16.4.5",
@@ -31,9 +34,13 @@
"globals": "^15.15.0",
"html-webpack-plugin": "^5.6.0",
"husky": "^9.1.7",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-reports": "^3.2.0",
"license-check-and-add": "^4.0.5",
"nodemon": "^3.1.0",
"prettier": "^3.8.3",
"prettier-plugin-jsdoc": "^1.8.1",
"simple-git": "^3.32.3",
"typescript": "^5.4.5",
"uglify-js": "^3.17.4",
@@ -46,6 +53,12 @@
"scripts": {
"test:backend": "npm run setupExtensions && vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "npm run setupExtensions && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"test:puterjs": "npm run setupExtensions && vitest run --config src/puter-js/tests/api/vitest.config.ts",
"test:puterjs:node": "npm run test:puterjs -- src/puter-js/tests/api/runners/node.test.ts",
"test:puterjs:browser": "npm run test:puterjs -- src/puter-js/tests/api/runners/browser.test.ts",
"test:puterjs:workerd": "npm run test:puterjs -- src/puter-js/tests/api/runners/workerd.test.ts",
"test:puterjs:coverage": "npm run build:workerLib:coverage && npm run setupExtensions && rm -rf src/puter-js/coverage && PUTER_COVERAGE=1 vitest run --config src/puter-js/tests/api/vitest.config.ts && node ./tools/puterjsCoverageReport.mjs",
"build:workerLib:coverage": "cd src/puter-js && npm run build:coverage && cd ../worker && npm run build",
"start:gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run setupExtensions && npm run build:ts",
@@ -78,6 +91,7 @@
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.6",
"miniflare": "^4.20260617.1",
"open": "^10.1.0"
},
"engines": {
@@ -17,8 +17,12 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { compareMigrationFilenames } from './MySQLDatabaseClient.js';
import { describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import {
MySQLDatabaseClient,
compareMigrationFilenames,
} from './MySQLDatabaseClient.js';
describe('compareMigrationFilenames', () => {
it('orders numbered migrations numerically, not lexically', () => {
@@ -93,3 +97,76 @@ describe('compareMigrationFilenames', () => {
expect([...sorted].sort(compareMigrationFilenames)).toEqual(sorted);
});
});
// ── Replica read failover ───────────────────────────────────────────
//
// `read()` normally goes to the replica batcher; when the replica side is
// degraded (batcher load-shed or a transient connection error) and a real
// replica is configured, the read retries once on the primary batcher.
type Batcher = { execute: ReturnType<typeof vi.fn> };
const makeClient = (opts: {
replica: Batcher;
primary: Batcher;
multiNode?: boolean;
}) => {
const client = new MySQLDatabaseClient({
database: { engine: 'mysql' },
} as IConfig);
// Bypass onServerStart (which would connect to a real database) and
// inject the batchers directly. Configuration enum: SINGLE=0, REPLICA=1.
Object.assign(client as unknown as Record<string, unknown>, {
dbReplica: opts.replica,
db: opts.primary,
configuration: opts.multiNode === false ? 0 : 1,
});
return client;
};
const codedError = (code: string) => {
const err = new Error(code) as Error & { code: string };
err.code = code;
return err;
};
describe('MySQLDatabaseClient.read — replica failover', () => {
it('fails over to the primary on batcher load-shed errors', async () => {
const replica = { execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')) };
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
const client = makeClient({ replica, primary });
await expect(client.read('SELECT 1')).resolves.toEqual([{ ok: 1 }]);
expect(primary.execute).toHaveBeenCalledTimes(1);
});
it('fails over on transient connection errors', async () => {
const replica = { execute: vi.fn().mockRejectedValue(codedError('ECONNRESET')) };
const primary = { execute: vi.fn().mockResolvedValue([[{ ok: 1 }]]) };
const client = makeClient({ replica, primary });
await expect(client.read('SELECT 1')).resolves.toEqual([{ ok: 1 }]);
});
it('rethrows deterministic SQL errors without touching the primary', async () => {
const replica = { execute: vi.fn().mockRejectedValue(codedError('ER_PARSE_ERROR')) };
const primary = { execute: vi.fn() };
const client = makeClient({ replica, primary });
await expect(client.read('SELEC oops')).rejects.toMatchObject({
code: 'ER_PARSE_ERROR',
});
expect(primary.execute).not.toHaveBeenCalled();
});
it('does not fail over in single-node configuration', async () => {
const replica = { execute: vi.fn().mockRejectedValue(codedError('dbBatchFailed')) };
const primary = { execute: vi.fn() };
const client = makeClient({ replica, primary, multiNode: false });
await expect(client.read('SELECT 1')).rejects.toMatchObject({
code: 'dbBatchFailed',
});
expect(primary.execute).not.toHaveBeenCalled();
});
});
@@ -19,31 +19,24 @@
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { metrics } from '@opentelemetry/api';
import { createPool, type Pool } from 'mysql2';
import { Span } from '../../util/span.js';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { SQLBatcher } from './SQLBatcher.js';
import { isRetriableError } from './retriableErrors.js';
import { splitMysqlStatements } from './splitMysqlStatements.js';
import { compareMigrationFilenames } from './migrationFilenames.js';
import type { IConfig } from '../../types';
const RETRIABLE_ERROR_CODES = new Set([
'PROTOCOL_CONNECTION_LOST',
'PROTOCOL_SEQUENCE_TIMEOUT',
'PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR',
'ECONNRESET',
'ETIMEDOUT',
'EPIPE',
'ECONNREFUSED',
'EHOSTUNREACH',
'ENETUNREACH',
'EAI_AGAIN',
]);
const DEFAULT_SELECT_TIMEOUT_MS = 30_000;
const RETRIABLE_ERROR_MESSAGES = [
'Connection lost',
'read ECONNRESET',
'ETIMEDOUT',
];
const replicaFailoverCounter = metrics
.getMeter('puter-backend')
.createCounter('db.read.replica_failover', {
description:
'Reads that failed on the replica batcher and were retried on the primary',
});
export { compareMigrationFilenames };
@@ -85,7 +78,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
});
console.log('[mysql] connected to primary');
this.db = new SQLBatcher(this.primaryPool, 30, 5);
this.db = this.createPrimaryBatcher(this.primaryPool);
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
@@ -96,7 +89,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
this.configuration = Configuration.SINGLE;
}
this.dbReplica = new SQLBatcher(this.replicaPool, 10, 5);
this.dbReplica = this.createReplicaBatcher(this.replicaPool);
await this.runMigrations();
}
@@ -135,15 +128,36 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// Query interface
// ------------------------------------------------------------------
// The db.* spans measure the logical query, including time queued in
// the SQLBatcher — the mysql2 auto-instrumentation only sees the
// coalesced multi-statement flush, so per-query latency lives here.
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.dbReplica.execute(query, params);
let result;
try {
result = await this.dbReplica.execute(query, params);
} catch (error) {
// Replica-side degradation (batcher load-shed or a transient
// connection failure) shouldn't fail reads while the primary is
// healthy. Deterministic errors (bad SQL) are rethrown — they
// would fail identically on the primary.
if (
this.configuration !== Configuration.REPLICA ||
!MySQLDatabaseClient.isFailoverWorthy(error)
) {
throw error;
}
replicaFailoverCounter.add(1);
result = await this.db.execute(query, params);
}
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
}
@Span('db.pread', (query: string) => ({ 'db.statement': query }))
override async pread(
query: string,
params: unknown[] = [],
@@ -153,6 +167,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
return (result[0] as Record<string, unknown>[]) ?? [];
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
@@ -170,6 +185,9 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
};
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(
entries: { statement: string; values: unknown[] }[],
): Promise<void> {
@@ -195,6 +213,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
}
}
@Span('db.tryHardRead', (query: string) => ({ 'db.statement': query }))
override async tryHardRead(
query: string,
params: unknown[] = [],
@@ -296,12 +315,53 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// ------------------------------------------------------------------
private createPool(poolConfig: PoolConfig): Pool {
return createPool({
const pool = createPool({
maxPreparedStatements: 900,
connectionLimit: 30,
enableKeepAlive: true,
...poolConfig,
multipleStatements: true,
} as PoolConfig);
// Server-side kill switch for runaway reads: MySQL applies
// max_execution_time to SELECT statements only, so this is
// write-safe. Without it, a stalled database turns reads into
// indefinite hangs that no client-side timeout ever converts
// into a failure. 0 disables.
const selectTimeoutMs = Math.floor(
Number(
this.config.database?.selectTimeoutMs ??
DEFAULT_SELECT_TIMEOUT_MS,
),
);
if (selectTimeoutMs > 0) {
pool.on('connection', (conn) => {
conn.query(
`SET SESSION max_execution_time = ${selectTimeoutMs}`,
);
});
}
return pool;
}
private createPrimaryBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 30,
maxBatchSize: 5,
poolLabel: 'primary',
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
private createReplicaBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 10,
maxBatchSize: 5,
poolLabel: 'replica',
readOnly: true,
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
/** Reinitialize the primary pool (e.g. after a health-check failure). */
@@ -317,11 +377,11 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
});
this.db = new SQLBatcher(this.primaryPool, 30, 5);
this.db = this.createPrimaryBatcher(this.primaryPool);
if (this.configuration === Configuration.SINGLE) {
this.replicaPool = this.primaryPool;
this.dbReplica = new SQLBatcher(this.primaryPool, 10, 5);
this.dbReplica = this.createReplicaBatcher(this.primaryPool);
}
if (previous && previous !== this.primaryPool) {
@@ -335,7 +395,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
const previous = this.replicaPool;
this.replicaPool = this.createPool(this.config.database.replica);
this.dbReplica = new SQLBatcher(this.replicaPool, 10, 5);
this.dbReplica = this.createReplicaBatcher(this.replicaPool);
if (
previous &&
@@ -351,11 +411,14 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// ------------------------------------------------------------------
static isRetriableError(error: unknown): boolean {
const code = (error as { code?: string })?.code;
if (code && RETRIABLE_ERROR_CODES.has(code)) return true;
return isRetriableError(error);
}
const msg = String((error as Error)?.message ?? '');
return RETRIABLE_ERROR_MESSAGES.some((m) => msg.includes(m));
/** Replica failures worth retrying on the primary: batcher load-shed
* or transient connection errors — never deterministic SQL errors. */
private static isFailoverWorthy(error: unknown): boolean {
const code = (error as { code?: string })?.code;
return code === 'dbBatchFailed' || isRetriableError(error);
}
async readWithRetry(
@@ -20,6 +20,7 @@
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { Pool, type PoolConfig, type QueryResult } from 'pg';
import { Span } from '../../util/span.js';
import {
AbstractDatabaseClient,
type BatchEntry,
@@ -214,6 +215,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return value;
}
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
@@ -222,6 +224,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return normalizePostgresRows(result);
}
@Span('db.pread', (query: string) => ({ 'db.statement': query }))
override async pread(
query: string,
params: unknown[] = [],
@@ -230,6 +233,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return normalizePostgresRows(result);
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
@@ -238,6 +242,9 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
return mapPostgresWriteResult(result);
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(entries: BatchEntry[]): Promise<void> {
if (entries.length === 0) return;
@@ -258,6 +265,7 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
}
}
@Span('db.tryHardRead', (query: string) => ({ 'db.statement': query }))
override async tryHardRead(
query: string,
params: unknown[] = [],
+158 -27
View File
@@ -18,10 +18,19 @@
*/
import { metrics } from '@opentelemetry/api';
import {
POOL_ACQUIRE_TIMEOUT,
isNeverSentError,
isRetriableError,
} from './retriableErrors.js';
const DEFAULT_MAX_QUEUE_SIZE = 1000;
const DEFAULT_FAILURE_THRESHOLD = 5;
const DEFAULT_COOLDOWN_MS = 5_000;
const DEFAULT_ACQUIRE_TIMEOUT_MS = 5_000;
const ACQUIRE_ATTEMPTS = 3;
const ITEM_RETRY_ATTEMPTS = 2;
const RETRY_BASE_BACKOFF_MS = 100;
const FALLBACK_RETRY_CONCURRENCY = 8;
const meter = metrics.getMeter('puter-backend');
@@ -53,6 +62,15 @@ const fallbackItemFailuresCounter = meter.createCounter(
'Per-item failures observed during SQLBatcher per-item retry',
},
);
const fallbackItemRetriesCounter = meter.createCounter(
'sql_batcher.fallback.item_retries',
{
description:
'Transient per-item failures retried during SQLBatcher fallback',
},
);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
export class SQLBatcher {
dbPool;
@@ -61,18 +79,43 @@ export class SQLBatcher {
maxQueueSize;
failureThreshold;
cooldownMs;
poolLabel;
readOnly;
acquireTimeoutMs;
queue = [];
timeouts = [];
#consecutiveFailures = 0;
#lastFailureAt = 0;
#metricAttrs;
/**
* @param {object} dbPool mysql2 pool
* @param {object} [opts]
* @param {number} [opts.maxTimeInQueue] ms an item may wait before flush
* @param {number} [opts.maxBatchSize] items coalesced per flush
* @param {number} [opts.maxQueueSize] drop-oldest high-water mark
* @param {number} [opts.failureThreshold] consecutive failures to open the breaker
* @param {number} [opts.cooldownMs] breaker open duration after last failure
* @param {'primary'|'replica'} [opts.poolLabel] role label on metrics; in
* single-node setups the 'replica' batcher shares the primary pool, so
* this reflects the read/write role rather than a physical instance
* @param {boolean} [opts.readOnly] this batcher only ever carries SELECTs,
* so any transient failure is safe to retry
* @param {number} [opts.acquireTimeoutMs] max wait for a pooled
* connection; 0 disables the bound
*/
constructor(
dbPool,
maxTimeInQueue = 20,
maxBatchSize = 50,
maxQueueSize = DEFAULT_MAX_QUEUE_SIZE,
failureThreshold = DEFAULT_FAILURE_THRESHOLD,
cooldownMs = DEFAULT_COOLDOWN_MS,
{
maxTimeInQueue = 20,
maxBatchSize = 50,
maxQueueSize = DEFAULT_MAX_QUEUE_SIZE,
failureThreshold = DEFAULT_FAILURE_THRESHOLD,
cooldownMs = DEFAULT_COOLDOWN_MS,
poolLabel = 'primary',
readOnly = false,
acquireTimeoutMs = DEFAULT_ACQUIRE_TIMEOUT_MS,
} = {},
) {
this.dbPool = dbPool;
this.maxTimeInQueue = maxTimeInQueue;
@@ -80,6 +123,10 @@ export class SQLBatcher {
this.maxQueueSize = maxQueueSize;
this.failureThreshold = failureThreshold;
this.cooldownMs = cooldownMs;
this.poolLabel = poolLabel;
this.readOnly = readOnly;
this.acquireTimeoutMs = acquireTimeoutMs;
this.#metricAttrs = { pool: poolLabel };
}
async execute(sql, values) {
@@ -90,9 +137,13 @@ export class SQLBatcher {
return this;
}
#createPublicBatchError() {
// The public error is deliberately opaque (no SQL, no internals), but
// `reason` distinguishes the load-shed path for logs and callers:
// breakerOpen | queueOverflow | connAcquire.
#createPublicBatchError(reason) {
const error = new Error('Database operation failed');
error.code = 'dbBatchFailed';
error.reason = reason;
return error;
}
@@ -106,8 +157,8 @@ export class SQLBatcher {
async query(sql, values) {
if (this.#isBreakerOpen()) {
enqueueRejectedCounter.add(1);
throw this.#createPublicBatchError();
enqueueRejectedCounter.add(1, this.#metricAttrs);
throw this.#createPublicBatchError('breakerOpen');
}
const { promise, resolve, reject } = Promise.withResolvers();
@@ -117,8 +168,8 @@ export class SQLBatcher {
// to have already exceeded any caller-side timeout anyway.
while (this.queue.length >= this.maxQueueSize) {
const dropped = this.queue.shift();
dropped.reject(this.#createPublicBatchError());
enqueueDroppedCounter.add(1);
dropped.reject(this.#createPublicBatchError('queueOverflow'));
enqueueDroppedCounter.add(1, this.#metricAttrs);
}
this.queue.push({
@@ -142,6 +193,61 @@ export class SQLBatcher {
return promise;
}
// Bounded wait for a pooled connection. Without a bound, a stalled
// database turns every flush into an indefinite hang — nothing fails,
// so neither the breaker nor callers' own timeouts ever engage.
#getConnectionWithTimeout() {
const acquire = this.dbPool.promise().getConnection();
if (!this.acquireTimeoutMs) return acquire;
return new Promise((resolve, reject) => {
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
// A connection that arrives late must go back to the pool.
acquire.then(
(conn) => conn.release(),
() => {},
);
const error = new Error(
'Timed out acquiring database connection',
);
error.code = POOL_ACQUIRE_TIMEOUT;
reject(error);
}, this.acquireTimeoutMs);
acquire.then(
(conn) => {
if (timedOut) return;
clearTimeout(timer);
resolve(conn);
},
(err) => {
if (timedOut) return;
clearTimeout(timer);
reject(err);
},
);
});
}
// Acquisition failures never sent a statement, so retrying is always
// safe regardless of what the batch contains.
async #acquireConnection() {
let lastError;
for (let attempt = 1; attempt <= ACQUIRE_ATTEMPTS; attempt++) {
try {
return await this.#getConnectionWithTimeout();
} catch (error) {
lastError = error;
if (attempt < ACQUIRE_ATTEMPTS) {
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
}
}
}
throw lastError;
}
async flush(batch) {
const timeout = this.timeouts.shift();
if (timeout && !timeout._destroyed) {
@@ -154,17 +260,17 @@ export class SQLBatcher {
let connection;
try {
connection = await this.dbPool.promise().getConnection();
connection = await this.#acquireConnection();
} catch (error) {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1);
flushFailureCounter.add(1, this.#metricAttrs);
console.warn(
'SQLBatcher could not acquire connection for flush:',
error,
);
for (const b of batch) {
b.reject(this.#createPublicBatchError());
b.reject(this.#createPublicBatchError('connAcquire'));
}
return;
}
@@ -206,8 +312,8 @@ export class SQLBatcher {
// committed; re-running each item independently produces clean
// success/failure outcomes for each caller. Concurrency is capped to
// avoid briefly saturating the pool when a large batch fails.
flushFailureCounter.add(1);
fallbackInvocationsCounter.add(1);
flushFailureCounter.add(1, this.#metricAttrs);
fallbackInvocationsCounter.add(1, this.#metricAttrs);
const settled = new Array(batch.length);
let cursor = 0;
@@ -216,17 +322,7 @@ export class SQLBatcher {
async () => {
while (cursor < batch.length) {
const i = cursor++;
const b = batch[i];
try {
settled[i] = {
ok: true,
value: await this.dbPool
.promise()
.query(b.sql, b.values ?? []),
};
} catch (error) {
settled[i] = { ok: false, error };
}
settled[i] = await this.#runFallbackItem(batch[i]);
}
},
);
@@ -246,7 +342,7 @@ export class SQLBatcher {
}
}
if (failureCount > 0) {
fallbackItemFailuresCounter.add(failureCount);
fallbackItemFailuresCounter.add(failureCount, this.#metricAttrs);
}
// Only escalate the breaker when the database itself looks unhealthy
@@ -259,4 +355,39 @@ export class SQLBatcher {
this.#consecutiveFailures++;
}
}
// Run one fallback item, retrying transient failures with backoff.
// A read-only batcher may retry anything transient; a batcher that
// carries writes only retries failures where the statement provably
// never reached the server — a write that died mid-flight may have
// committed, and re-running it would double-apply.
async #runFallbackItem(b) {
let attempt = 0;
while (true) {
let connection;
try {
connection = await this.#acquireConnection();
} catch (error) {
return { ok: false, error };
}
try {
return {
ok: true,
value: await connection.query(b.sql, b.values ?? []),
};
} catch (error) {
const canRetry = this.readOnly
? isRetriableError(error)
: isNeverSentError(error);
if (!canRetry || attempt >= ITEM_RETRY_ATTEMPTS) {
return { ok: false, error };
}
attempt++;
fallbackItemRetriesCounter.add(1, this.#metricAttrs);
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
} finally {
connection.release();
}
}
}
}
@@ -0,0 +1,239 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it, vi } from 'vitest';
import { SQLBatcher } from './SQLBatcher.js';
const makeError = (code: string, message = code): Error & { code: string } => {
const error = new Error(message) as Error & { code: string };
error.code = code;
return error;
};
interface FakeConnection {
beginTransaction: ReturnType<typeof vi.fn>;
query: ReturnType<typeof vi.fn>;
commit: ReturnType<typeof vi.fn>;
rollback: ReturnType<typeof vi.fn>;
release: ReturnType<typeof vi.fn>;
}
// The batch flush sends one coalesced multi-statement (always suffixed
// `; SELECT 1`); fallback items send their original single statement.
const isBatchQuery = (sql: string) => sql.endsWith('; SELECT 1');
const makeConnection = (
onQuery: (sql: string, values: unknown[]) => unknown,
): FakeConnection => ({
beginTransaction: vi.fn(async () => {}),
query: vi.fn(async (sql: string, values: unknown[]) => onQuery(sql, values)),
commit: vi.fn(async () => {}),
rollback: vi.fn(async () => {}),
release: vi.fn(),
});
const makePool = (connection: FakeConnection | (() => Promise<FakeConnection>)) => {
const getConnection = vi.fn(async () =>
typeof connection === 'function' ? connection() : connection,
);
return {
pool: { promise: () => ({ getConnection }) },
getConnection,
};
};
// Happy-path onQuery: batch returns one result row-set per statement plus
// the trailing SELECT 1 row-set.
const happyBatch = (sql: string) => {
if (!isBatchQuery(sql)) throw new Error('unexpected fallback query');
const statements = sql.split(';').length - 1;
return [
Array.from({ length: statements + 1 }, (_, i) => [{ n: i }]),
undefined,
];
};
describe('SQLBatcher', () => {
it('resolves each batched item with its own result', async () => {
const conn = makeConnection(happyBatch);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
const [a, b] = await Promise.all([
batcher.query('SELECT a', []),
batcher.query('SELECT b', []),
]);
expect(a[0]).toEqual([{ n: 0 }]);
expect(b[0]).toEqual([{ n: 1 }]);
expect(conn.beginTransaction).toHaveBeenCalledTimes(1);
expect(conn.commit).toHaveBeenCalledTimes(1);
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('drops the oldest item with reason queueOverflow at the high-water mark', async () => {
const conn = makeConnection(happyBatch);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
maxQueueSize: 1,
});
const first = batcher.query('SELECT a', []);
const second = batcher.query('SELECT b', []);
await expect(first).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'queueOverflow',
});
await expect(second).resolves.toBeTruthy();
});
it('rejects with reason connAcquire after exhausting acquisition retries', async () => {
const getConnection = vi.fn(async () => {
throw makeError('ECONNREFUSED');
});
const pool = { promise: () => ({ getConnection }) };
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('SELECT a', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'connAcquire',
});
expect(getConnection).toHaveBeenCalledTimes(3);
});
it('bounds connection acquisition and rejects when the pool never answers', async () => {
const getConnection = vi.fn(
() => new Promise(() => {}), // pool never yields a connection
);
const pool = { promise: () => ({ getConnection }) };
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
acquireTimeoutMs: 30,
});
await expect(batcher.query('SELECT a', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'connAcquire',
});
expect(getConnection).toHaveBeenCalledTimes(3);
});
it('opens the breaker after consecutive failures and rejects with reason breakerOpen', async () => {
// Batch and fallback both fail with an ambiguous (non-retriable for
// writes) connection error, so no item gets through.
const conn = makeConnection(() => {
throw makeError('ECONNRESET');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
failureThreshold: 1,
cooldownMs: 60_000,
});
await expect(batcher.query('INSERT x', [])).rejects.toMatchObject({
code: 'ECONNRESET',
});
await expect(batcher.query('INSERT y', [])).rejects.toMatchObject({
code: 'dbBatchFailed',
reason: 'breakerOpen',
});
});
it('retries transient fallback failures when readOnly', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ECONNRESET');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
readOnly: true,
});
const result = await batcher.query('SELECT a', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('does not retry ambiguous failures on a batcher that carries writes', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
throw makeError('ECONNRESET');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('INSERT x', [])).rejects.toMatchObject({
code: 'ECONNRESET',
});
expect(fallbackAttempts).toBe(1);
});
it('retries never-sent failures even on a batcher that carries writes', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ECONNRESET');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ECONNREFUSED');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
const result = await batcher.query('INSERT x', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('never retries deterministic row-level errors and does not escalate the breaker', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ER_DUP_ENTRY');
fallbackAttempts++;
if (sql === 'INSERT dup') throw makeError('ER_DUP_ENTRY');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
failureThreshold: 1,
cooldownMs: 60_000,
readOnly: true,
});
const dup = batcher.query('INSERT dup', []);
const fine = batcher.query('INSERT fine', []);
await expect(dup).rejects.toMatchObject({ code: 'ER_DUP_ENTRY' });
await expect(fine).resolves.toBeTruthy();
expect(fallbackAttempts).toBe(2);
// One fallback item succeeded, so the breaker must stay closed.
const conn2 = makeConnection(happyBatch);
// reuse same batcher/pool: next query must not be rejected upfront
conn.query.mockImplementation(conn2.query.getMockImplementation()!);
await expect(batcher.query('SELECT a', [])).resolves.toBeTruthy();
});
});
@@ -21,6 +21,7 @@ import { existsSync, mkdirSync, readFileSync } from 'fs';
import { basename, dirname, extname, join, resolve } from 'path';
import { createContext, runInContext } from 'vm';
import type { IConfig } from '../../types';
import { Span } from '../../util/span.js';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
const MIGRATIONS_DIR = resolve(__dirname, './migrations/sqlite');
@@ -136,6 +137,7 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
// Query interface
// ------------------------------------------------------------------
@Span('db.read', (query: string) => ({ 'db.statement': query }))
override async read(
query: string,
params: unknown[] = [],
@@ -156,6 +158,7 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
return this.read(query, params);
}
@Span('db.write', (query: string) => ({ 'db.statement': query }))
override async write(
query: string,
params: unknown[] = [],
@@ -172,6 +175,9 @@ export class SqliteDatabaseClient extends AbstractDatabaseClient {
};
}
@Span('db.batchWrite', (entries: unknown[]) => ({
'db.batch_size': entries.length,
}))
override async batchWrite(
entries: { statement: string; values: unknown[] }[],
): Promise<void> {
@@ -0,0 +1,78 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/** Error code set by the pool-acquisition timeout in SQLBatcher. */
export const POOL_ACQUIRE_TIMEOUT = 'POOL_ACQUIRE_TIMEOUT';
const RETRIABLE_ERROR_CODES = new Set([
'PROTOCOL_CONNECTION_LOST',
'PROTOCOL_SEQUENCE_TIMEOUT',
'PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR',
'ECONNRESET',
'ETIMEDOUT',
'EPIPE',
'ECONNREFUSED',
'EHOSTUNREACH',
'ENETUNREACH',
'EAI_AGAIN',
POOL_ACQUIRE_TIMEOUT,
]);
const RETRIABLE_ERROR_MESSAGES = [
'Connection lost',
'read ECONNRESET',
'ETIMEDOUT',
];
/**
* Failures where the statement provably never reached the server, so a
* retry can never double-apply it — safe even for writes. Anything that
* can occur after the statement was sent (resets, protocol drops) is
* deliberately excluded: the server may have committed before the
* connection died.
*/
const NEVER_SENT_ERROR_CODES = new Set([
'ECONNREFUSED',
'EHOSTUNREACH',
'ENETUNREACH',
'EAI_AGAIN',
POOL_ACQUIRE_TIMEOUT,
]);
const errorCode = (error: unknown): string | undefined =>
(error as { code?: string } | null)?.code;
/**
* Transient connection-level failures worth retrying — but only for
* statements that are safe to run twice (reads). Row-level errors
* (duplicate key, constraint violations) are deterministic and never
* match.
*/
export const isRetriableError = (error: unknown): boolean => {
const code = errorCode(error);
if (code && RETRIABLE_ERROR_CODES.has(code)) return true;
const msg = String((error as Error)?.message ?? '');
return RETRIABLE_ERROR_MESSAGES.some((m) => msg.includes(m));
};
export const isNeverSentError = (error: unknown): boolean => {
const code = errorCode(error);
return Boolean(code && NEVER_SENT_ERROR_CODES.has(code));
};
+30 -1
View File
@@ -43,6 +43,7 @@ import { once } from 'node:events';
import { Agent as httpsAgent } from 'node:https';
import { HttpError } from '../../core/http';
import type { IConfig, IDynamoConfig } from '../../types';
import { Span } from '../../util/span.js';
import { PuterClient } from '../types';
const LOCAL_DYNAMO_MEMORY_PREFIX = ':memory:';
@@ -151,6 +152,7 @@ export class DDBClient extends PuterClient {
await this.#localInitPromise;
}
@Span('ddb.get', (table: string) => ({ 'db.table': table }))
async get<T extends Record<string, unknown>>(
table: string,
key: T,
@@ -167,6 +169,7 @@ export class DDBClient extends PuterClient {
return client.send(command);
}
@Span('ddb.put', (table: string) => ({ 'db.table': table }))
async put<T extends Record<string, unknown>>(table: string, item: T) {
const command = new PutCommand({
TableName: table,
@@ -178,6 +181,9 @@ export class DDBClient extends PuterClient {
return client.send(command);
}
@Span('ddb.batchGet', (params: unknown[]) => ({
'db.batch_size': params.length,
}))
async batchGet(
params: { table: string; items: Record<string, unknown> }[],
consistentRead = false,
@@ -212,6 +218,9 @@ export class DDBClient extends PuterClient {
return client.send(command);
}
@Span('ddb.batchPut', (params: unknown[]) => ({
'db.batch_size': params.length,
}))
async batchPut(params: { table: string; item: Record<string, unknown> }[]) {
const consumedCapacityByTable = new Map<string, number>();
if (params.length === 0) {
@@ -318,6 +327,7 @@ export class DDBClient extends PuterClient {
};
}
@Span('ddb.del', (table: string) => ({ 'db.table': table }))
async del<T extends Record<string, unknown>>(table: string, key: T) {
const command = new DeleteCommand({
TableName: table,
@@ -329,6 +339,7 @@ export class DDBClient extends PuterClient {
return client.send(command);
}
@Span('ddb.query', (table: string) => ({ 'db.table': table }))
async query<T extends Record<string, unknown>>(
table: string,
keys: T,
@@ -336,7 +347,15 @@ export class DDBClient extends PuterClient {
pageKey?: Record<string, unknown>,
index = '',
consistentRead = false,
options?: { beginsWith?: { key: string; value: string } },
options?: {
beginsWith?: { key: string; value: string };
select?: 'COUNT';
filter?: {
expression: string;
values?: Record<string, unknown>;
names?: Record<string, string>;
};
},
) {
const keyExpressionParts = Object.keys(keys).map(
(key) => `#${key} = :${key}`,
@@ -367,6 +386,11 @@ export class DDBClient extends PuterClient {
expressionAttributeNames[`#${beginsKey}`] = beginsKey;
}
if (options?.filter) {
Object.assign(expressionAttributeValues, options.filter.values);
Object.assign(expressionAttributeNames, options.filter.names);
}
const command = new QueryCommand({
TableName: table,
...(!index ? {} : { IndexName: index }),
@@ -376,6 +400,10 @@ export class DDBClient extends PuterClient {
ConsistentRead: consistentRead,
...(!pageKey ? {} : { ExclusiveStartKey: pageKey }),
...(!limit ? {} : { Limit: limit }),
...(options?.filter
? { FilterExpression: options.filter.expression }
: {}),
...(options?.select ? { Select: options.select } : {}),
ReturnConsumedCapacity: 'TOTAL',
});
@@ -383,6 +411,7 @@ export class DDBClient extends PuterClient {
return client.send(command);
}
@Span('ddb.update', (table: string) => ({ 'db.table': table }))
async update<T extends Record<string, unknown>>(
table: string,
key: T,
+37 -12
View File
@@ -24,12 +24,20 @@ import type { IConfig } from '../../types';
import { PuterClient } from '../types';
import { EMAIL_TEMPLATES, type EmailTemplateName } from './templates';
// -- Types ------------------------------------------------------------
/** Attachment shape passed through to the underlying transport. */
export interface EmailAttachment {
filename: string;
content: Buffer | string;
contentType?: string;
encoding?: string;
}
// nodemailer doesn't ship TS types, so declare the subset we use.
interface NodemailerTransport {
sendMail: (options: SendMailOptions) => Promise<unknown>;
close?: () => void;
/** Subset of the transport's send result callers may care about. */
export interface SentMessageInfo {
messageId?: string;
accepted?: string[];
rejected?: string[];
[key: string]: unknown;
}
export interface SendMailOptions {
@@ -37,10 +45,18 @@ export interface SendMailOptions {
to: string;
cc?: string;
bcc?: string;
/** Optional transport recipients when they differ from visible headers. */
envelope?: {
from?: string;
to: string;
};
subject: string;
html?: string;
text?: string;
replyTo?: string;
attachments?: EmailAttachment[];
/** Extra message headers (e.g. List-Unsubscribe), passed to the transport. */
headers?: Record<string, string>;
}
export type EmailValidator = (email: string) => Promise<boolean> | boolean;
@@ -100,7 +116,8 @@ const DOMAIN_ALIASES: Record<string, string> = {
* - Policy + extensible validation (via `validate`)
*/
export class EmailClient extends PuterClient {
private transport: NodemailerTransport | null = null;
private transport: ReturnType<typeof nodemailer.createTransport> | null =
null;
private compiledTemplates: Partial<
Record<EmailTemplateName, CompiledTemplate>
> = {};
@@ -123,8 +140,7 @@ export class EmailClient extends PuterClient {
return;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this.transport = nodemailer.createTransport(emailConf as any);
this.transport = nodemailer.createTransport(emailConf);
console.log('[email] transport configured');
}
@@ -159,21 +175,30 @@ export class EmailClient extends PuterClient {
/**
* Raw send — bypasses the template system. Useful for one-off
* admin emails that don't warrant a named template.
*
* Returns the transport's send result, or `null` when no transport
* is configured (the send is a no-op in that case — callers that
* must not silently drop mail should check `isConfigured` first).
*/
async sendRaw(options: SendMailOptions): Promise<void> {
async sendRaw(options: SendMailOptions) {
if (!this.transport) {
console.warn(
'[email] attempted to send email without transport. If you need to send email, configure an SMTP transport in your config file (see docs for details). Email content:',
options,
);
return;
return null;
}
await this.transport.sendMail({
from: options.from ?? this.defaultFrom(),
return await this.transport.sendMail({
...options,
from: options.from ?? this.defaultFrom(),
});
}
/** Whether an SMTP transport is configured (sends are no-ops otherwise). */
get isConfigured(): boolean {
return this.transport !== null;
}
// -- Public API: clean / validate ---------------------------------
/**
+40 -18
View File
@@ -18,6 +18,7 @@
*/
import { extensionStore } from '../../extensions';
import { withSpan } from '../../util/span.js';
import { PuterClient } from '../types';
import {
EventListener,
@@ -99,26 +100,34 @@ export class EventClient extends PuterClient {
data: EventMap[T],
meta: EventMetadata,
) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey = (
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`
) as ListenKey;
const extensionListeners = extensionStore.events[matchKey];
const listeners = (this.#eventListeners[matchKey] || []).concat(
extensionListeners || [],
);
if (!listeners) continue;
for (const listener of listeners) {
try {
await listener(key, data, meta);
} catch (e) {
console.error('Error in event listener for event', key, e);
// Spanned because callers block on listeners — this is where time
// spent in extension hooks (e.g. `ip.validate`) gets attributed.
return withSpan('event.emitAndWait', { 'event.key': key }, async () => {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey = (
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`
) as ListenKey;
const extensionListeners = extensionStore.events[matchKey];
const listeners = (this.#eventListeners[matchKey] || []).concat(
extensionListeners || [],
);
if (!listeners) continue;
for (const listener of listeners) {
try {
await listener(key, data, meta);
} catch (e) {
console.error(
'Error in event listener for event',
key,
e,
);
}
}
}
}
});
}
/**
@@ -146,6 +155,19 @@ export class EventClient extends PuterClient {
this.#eventListeners[key] ?? (this.#eventListeners[key] = []);
listeners.push(callback as EventListener);
}
off<P extends ListenKey>(
key: P,
callback: (
key: MatchingEvents<P>,
data: EventMap[MatchingEvents<P>],
meta: EventMetadata,
) => Promise<void> | void,
) {
const listeners = this.#eventListeners[key];
if (!listeners) return;
const idx = listeners.indexOf(callback as EventListener);
if (idx !== -1) listeners.splice(idx, 1);
}
async #emitEvent<T extends keyof EventMap>(
listener: EventListener,
key: T,
+31 -20
View File
@@ -3,15 +3,16 @@
*
* Domain objects (User, Actor, FSEntry, Socket, ...) are typed as `unknown`
* here on purpose — pulling their real types in would couple this module to
* most of the backend and risk import cycles. Refine at the listener if
* you need narrowed access.
* most of the backend and risk import cycles. Refine at the listener if you
* need narrowed access.
*
* Conventions:
* - `*.validate` events carry an `allow` flag listeners flip to reject;
* they tend to grow listener-specific fields, so they accept extras via
* an index signature.
* - `outer.gui.*` events ride the `{ user_id_list, response }` envelope
* the SocketService fans out to user-scoped channels.
*
* - `*.validate` events carry an `allow` flag listeners flip to reject; they tend
* to grow listener-specific fields, so they accept extras via an index
* signature.
* - `outer.gui.*` events ride the `{ user_id_list, response }` envelope the
* SocketService fans out to user-scoped channels.
*/
import type {
@@ -148,13 +149,16 @@ export type EventMap = {
trail?: Array<string>;
/**
* Set by the abuse harness for flagged signups — the id under which the
* decision trail is persisted to KV (`abuse:trail:<id>`), shared back on
* the request for log / support correlation.
* decision trail is persisted to KV (`abuse:trail:<id>`), shared back
* on the request for log / support correlation.
*/
trail_id?: string;
/** Device signal forwarded verbatim from the signup request body. */
fingerprint?: string | null;
/** Set by the abuse harness — require SMS phone verification post-signup. */
/**
* Set by the abuse harness — require SMS phone verification
* post-signup.
*/
requires_phone_verification?: boolean;
/** Set by the abuse harness — require card verification post-signup. */
requires_card_verification?: boolean;
@@ -251,6 +255,7 @@ export type EventMap = {
fingerprint: string | null;
funding: string | null;
country: string | null;
customer_id: string | null;
[key: string]: unknown;
};
'user.card-verified': {
@@ -259,6 +264,7 @@ export type EventMap = {
fingerprint: string | null;
funding: string | null;
country: string | null;
customer_id: string | null;
};
'user.username-changed': {
user_id: number;
@@ -332,9 +338,13 @@ export type EventMap = {
};
// ---- Thumbnails ----
// The listener rewrites `thumbnail` in place (an s3:// key or legacy
// https:// URL becomes a signed URL) and falls back to `uuid`/`uid` to
// migrate inline data-URL thumbnails. Declared to match: the previous
// `uri` member was never emitted nor read.
'thumbnail.read': {
uri: string;
size?: string;
uuid?: string;
uid?: string;
thumbnail?: string | null;
};
'thumbnail.created': { url: string };
@@ -390,6 +400,8 @@ export type EventMap = {
// normalized path: `route.<method>.<path>.before|after|error|reject`. Same
// wildcard + veto semantics as the driver lifecycle above.
[K in `route.${string}`]: RouteLifecycleEvent;
} & {
[K in `pubsub.login.${string}`]: { authtoken: string };
};
/**
@@ -402,9 +414,9 @@ export type LifecyclePhase = 'before' | 'after' | 'error' | 'reject';
/**
* Payload for `driver.<iface>.<method>.<phase>` events.
*
* One shape across all phases; read `phase` (or the key suffix) to
* branch. `allow`/`rejectReason` are only meaningful on the `before` phase
* (emitted via `emitAndWait`).
* One shape across all phases; read `phase` (or the key suffix) to branch.
* `allow`/`rejectReason` are only meaningful on the `before` phase (emitted via
* `emitAndWait`).
*/
export type DriverMethodLifecycleEvent = {
phase: LifecyclePhase;
@@ -470,11 +482,10 @@ export type EventKey = keyof EventMap & string;
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
? | `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
@@ -1154,6 +1154,139 @@ describe('AuthController.handleLoginOtp + handleLoginRecoveryCode', () => {
});
});
// ── Step-up (elevation) ─────────────────────────────────────────────
describe('AuthController.handleElevate', () => {
it('password account: correct password mints the elevation cookie', async () => {
const { actor } = await makeUserAndActor();
const res = makeRes();
await controller.handleElevate(
makeReq({ password: 'correct-horse-battery' }, { actor }),
res,
);
expect(res.body).toMatchObject({ elevated: true });
expect(res.cookies.puter_elevated).toBeDefined();
expect(res.cookies.puter_elevated.opts).toMatchObject({ httpOnly: true });
// The minted cookie satisfies verifyStepUpSession for this same user.
const { verifyStepUpSession } = await import(
'../../core/http/middleware/stepUpSession.js'
);
const ok = verifyStepUpSession(
{
cookies: { puter_elevated: res.cookies.puter_elevated.value },
actor: { user: { uuid: actor.user.uuid } },
} as never,
{ tokenService: server.services.token },
);
expect(ok).toBe(true);
});
it('password account: wrong password → 401 password_mismatch', async () => {
const { actor } = await makeUserAndActor();
await expect(
controller.handleElevate(
makeReq({ password: 'nope' }, { actor }),
makeRes(),
),
).rejects.toMatchObject({
statusCode: 401,
legacyCode: 'password_mismatch',
});
});
it('2FA account: a live TOTP code elevates; a wrong code is rejected', async () => {
const { TOTP } = await import('otpauth');
const { createSecret } = await import(
'../../services/auth/OTPUtil.js'
);
const { user, actor } = await makeUserAndActor();
const { secret } = createSecret(user.username);
await server.stores.user.update(user.id, {
otp_enabled: 1,
otp_secret: secret,
});
// Reflect the enabled state on the actor the way the auth probe would.
const otpActor = {
user: { ...actor.user, otp_enabled: true },
} as never;
const totp = new TOTP({
issuer: 'puter.com',
label: user.username,
algorithm: 'SHA1',
digits: 6,
secret,
});
const res = makeRes();
await controller.handleElevate(
makeReq({ code: totp.generate() }, { actor: otpActor }),
res,
);
expect(res.body).toMatchObject({ elevated: true });
expect(res.cookies.puter_elevated).toBeDefined();
await expect(
controller.handleElevate(
makeReq({ code: '000000' }, { actor: otpActor }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 401 });
});
it('account with no password and 2FA off cannot elevate → 403', async () => {
const { user, actor } = await makeUserAndActor();
await server.stores.user.update(user.id, { password: null });
await expect(
controller.handleElevate(
makeReq({ password: 'anything' }, { actor }),
makeRes(),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'elevation_unavailable',
});
});
it('API clients (no cookie) get the token back to send as a header', async () => {
const { actor } = await makeUserAndActor();
const res = makeRes();
await controller.handleElevate(
makeReq({ password: 'correct-horse-battery' }, { actor }),
res,
);
expect(typeof (res.body as { token?: string }).token).toBe('string');
});
it('browser sessions (cookie-authed) do NOT get the raw token in the body', async () => {
const { actor } = await makeUserAndActor();
const res = makeRes();
// Mimic the browser: the resolved token IS the session cookie value.
// Cookie name must match `config.cookie_name` (puter_auth_token).
const req = {
...makeReq({ password: 'correct-horse-battery' }, { actor }),
token: 'session-cookie-value',
cookies: { puter_auth_token: 'session-cookie-value' },
};
await controller.handleElevate(req, res);
expect(res.body).toEqual({ elevated: true });
expect(res.cookies.puter_elevated).toBeDefined();
});
it('the elevation token is never honored as a main auth token', async () => {
const { user } = await makeUserAndActor();
const { signStepUpToken } = await import(
'../../core/http/middleware/stepUpSession.js'
);
const token = signStepUpToken(server.services.token, {
uuid: user.uuid,
});
const result = await server.services.auth.authenticate(token);
expect(result.actor).toBeUndefined();
});
});
// ── Logout ──────────────────────────────────────────────────────────
describe('AuthController.handleLogout', () => {
@@ -1591,9 +1724,49 @@ describe('AuthController.handleGetUserAppToken + handleCheckApp', () => {
);
const body = res.body as { token: string; app_uid: string };
expect(body.app_uid).toMatch(/^app-/);
// A bootstrap app row was created for that origin.
// A bootstrap app row was created for that origin. External
// origins have no hosted subdomain, so no owner is stamped.
const bootstrapped = await server.stores.app.getByUid(body.app_uid);
expect(bootstrapped).toBeTruthy();
expect(bootstrapped?.owner_user_id ?? null).toBeNull();
});
it('stamps the hosted-subdomain owner as the bootstrap app creator', async () => {
// Test config inherits `static_hosting_domain: site.puter.localhost`
// from config.default.json. The subdomain belongs to a different
// user than the visitor minting the token — the app must be owned
// by the site owner, not the visitor.
const ownerName = `so_${uuidv4().slice(0, 8)}`;
await controller.handleSignup(
makeReq({
username: ownerName,
email: `${ownerName}@test.local`,
password: 'correct-horse-battery',
}),
makeRes(),
);
const owner = await server.stores.user.getByUsername(ownerName);
expect(owner!.id).not.toBe(user.id);
const subdomain = `sd-${uuidv4().slice(0, 8)}`;
await server.stores.subdomain.create({
userId: owner!.id,
subdomain,
});
const res = makeRes();
await inCtx(actor, () =>
controller.handleGetUserAppToken(
makeReq(
{ origin: `https://${subdomain}.site.puter.localhost` },
{ actor },
),
res,
),
);
const body = res.body as { token: string; app_uid: string };
const bootstrapped = await server.stores.app.getByUid(body.app_uid);
expect(bootstrapped).toBeTruthy();
expect(bootstrapped?.owner_user_id).toBe(owner!.id);
});
});
@@ -2008,6 +2181,62 @@ describe('AuthController.handleSendConfirmPhone validation', () => {
);
});
it('persists the send failure to KV under the error_id with a 7-day expiry', async () => {
const { user, actor } = await makeUserAndActor();
const kvSet = vi.spyOn(server.stores.kv, 'set');
try {
await withPrelude(
stubPrelude({
createVerification: vi.fn(async () => {
throw new Error('network down');
}),
}),
async () => {
let thrown: HttpError | undefined;
try {
await controller.handleSendConfirmPhone(
makeReq({ phone: '+14155550123' }, { actor }),
makeRes(),
);
} catch (e) {
thrown = e as HttpError;
}
const errorId = thrown!.fields!.error_id as string;
const { res: record } = await server.stores.kv.get({
key: `sms-send-error:${errorId}`,
});
expect(record).toMatchObject({
reason: 'prelude_request_failed',
status: 502,
user_id: user.id,
user_uid: user.uuid,
detail: 'network down',
t: expect.any(Number),
});
const errorSet = kvSet.mock.calls.find(([arg]) =>
(arg as { key?: string }).key?.startsWith(
'sms-send-error:',
),
);
const { expireAt } = errorSet![0] as {
expireAt: number;
};
const nowSec = Math.floor(Date.now() / 1000);
expect(expireAt).toBeGreaterThan(
nowSec + 7 * 24 * 60 * 60 - 60,
);
expect(expireAt).toBeLessThanOrEqual(
nowSec + 7 * 24 * 60 * 60,
);
},
);
} finally {
kvSet.mockRestore();
}
});
it('forwards ip, device fingerprint, and user-agent to Prelude as signals', async () => {
const { actor } = await makeUserAndActor();
const createVerification = vi.fn(async () => ({ status: 'success' }));
@@ -3032,6 +3261,29 @@ describe('AuthController.handleConfirmEmail', () => {
original_client_socket_id: 's',
});
});
it('rejects "null" as a code when no confirmation code is stored', async () => {
const { user, actor } = await makeUserAndActor();
// A row with no stored code must never be confirmable: String(null)
// would otherwise equal a submitted "null" and confirm the email.
await server.stores.user.update(user.id, { email_confirm_code: null });
const res = makeRes();
await controller.handleConfirmEmail(
makeReq(
{ code: 'null', original_client_socket_id: 's' },
{ actor },
),
res,
);
expect(res.body).toEqual({
email_confirmed: false,
original_client_socket_id: 's',
});
const after = await server.stores.user.getById(user.id, {
force: true,
});
expect(after!.email_confirmed).toBeFalsy();
});
});
// ── Password recovery flow ──────────────────────────────────────────
+293 -71
View File
@@ -3,24 +3,25 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import bcrypt from 'bcrypt';
import type { Request, RequestHandler, Response } from 'express';
import crypto from 'node:crypto';
import { v4 as uuidv4 } from 'uuid';
import { v4 as uuidv4, validate as validateUuid } from 'uuid';
import validator from 'validator';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import type { HttpErrorOptions } from '../../core/http/HttpError.js';
@@ -28,6 +29,11 @@ import { HttpError } from '../../core/http/HttpError.js';
import { antiCsrf } from '../../core/http/middleware/antiCsrf.js';
import { generateCaptcha } from '../../core/http/middleware/captcha.js';
import { checkRateLimit } from '../../core/http/middleware/rateLimit.js';
import {
signStepUpToken,
STEP_UP_COOKIE_NAME,
stepUpCookieOptions,
} from '../../core/http/middleware/stepUpSession.js';
import {
createUserProtectedGate,
createWebSessionActorGate,
@@ -72,6 +78,9 @@ const SEND_PHONE_RATE_WINDOW_MS = 60 * 60_000;
// Once the threshold is crossed the fallback stays open this long, so the
// user can finish the card flow without racing the attempt counter's expiry.
const CARD_FALLBACK_OPEN_TTL_SECONDS = 24 * 60 * 60;
// How long a failed-SMS-send record stays readable by its error_id — long
// enough to cover the typical support round-trip.
const SMS_SEND_ERROR_TTL_SECONDS = 7 * 24 * 60 * 60;
const RESERVED_USERNAMES = new Set([
'admin',
'administrator',
@@ -99,17 +108,77 @@ const RESERVED_USERNAMES = new Set([
* Auth controller — login/logout, permission grants/revokes, session
* management, OTP, and permission checks.
*
* Routes are declared via decorators (@Get/@Post on each handler). The
* five `/user-protected/*` and `/user-protected/delete-own-user` routes
* also need a per-instance `createUserProtectedGate(...)` middleware
* built from `this.config / this.stores / this.services`, which can't
* live in a static decorator literal — those are wired imperatively in
* the `registerRoutes` override below. The override also re-runs the
* default decorator-walker logic so the rest of the routes register
* normally.
* Routes are declared via decorators (@Get/@Post on each handler). The five
* `/user-protected/*` and `/user-protected/delete-own-user` routes also need a
* per-instance `createUserProtectedGate(...)` middleware built from
* `this.config / this.stores / this.services`, which can't live in a static
* decorator literal — those are wired imperatively in the `registerRoutes`
* override below. The override also re-runs the default decorator-walker logic
* so the rest of the routes register normally.
*/
@Controller('')
export class AuthController extends PuterController {
@Post('/login/wait', {
subdomain: ['api'],
rateLimit: [
// A client will make a request to this every 10 seconds while waiting for the login to complete, so we allow a higher limit than the main /login endpoint.
{ scope: 'login-wait', limit: 100, window: 15 * 60_000, key: 'ip' },
],
})
async loginWait(req: Request, res: Response) {
const { session } = req.body;
// validate uuid to prevent ultra long key or listening on pubsub.login.*
if (!session || !validateUuid(session)) {
throw new HttpError(400, 'session is required.', {
legacyCode: 'bad_request',
});
}
const { resolve, promise } = Promise.withResolvers<void>();
let token: string | null = null;
const listener = (_key: string, value: { authtoken: string }) => {
token = value.authtoken;
resolve();
};
this.clients.event.on(`pubsub.login.${session}`, listener);
const timeout = new Promise<void>((resolve) =>
setTimeout(resolve, 10000),
);
await Promise.race([promise, timeout]);
this.clients.event.off(`pubsub.login.${session}`, listener);
if (!token) {
throw new HttpError(408, 'Request timeout.', {
legacyCode: 'request_timeout',
});
}
res.json({
auth_token: token,
});
}
@Post('/login/set', {
subdomain: ['api'],
})
async loginSet(req: Request, res: Response) {
const { session, auth_token } = req.body;
if (!session || !auth_token || !validateUuid(session)) {
throw new HttpError(400, 'session and auth_token are required.', {
legacyCode: 'bad_request',
});
}
this.clients.event.emit(
`pubsub.login.${session}`,
{
authtoken: auth_token,
},
{},
);
res.json({ success: true });
}
// -- Login -------------------------------------------------------
@Post('/login', {
@@ -855,8 +924,7 @@ export class AuthController extends PuterController {
is_temp: user!.password === null && user!.email === null,
ip:
(req?.headers?.['x-forwarded-for'] as
| string
| undefined) ||
string | undefined) ||
(
req as unknown as {
connection?: { remoteAddress?: string };
@@ -900,6 +968,11 @@ export class AuthController extends PuterController {
// a stale value would re-authenticate the next request.
res.clearCookie(this.config.cookie_name ?? 'puter_token');
res.clearCookie('puter_token_v2');
// Drop any step-up elevation too, so it can't reactivate on a shared
// machine.
res.clearCookie(STEP_UP_COOKIE_NAME, {
...(this.config.domain ? { domain: this.config.domain } : {}),
});
// Remove the session (fire-and-forget)
if (req.token) {
@@ -1004,7 +1077,13 @@ export class AuthController extends PuterController {
});
return;
}
if (String(user.email_confirm_code) !== String(code)) {
// Reject before comparing when no code is stored: `String(null)` would
// otherwise equal a submitted `"null"` and confirm the email without
// the real code.
if (
!user.email_confirm_code ||
String(user.email_confirm_code) !== String(code)
) {
res.json({
email_confirmed: false,
original_client_socket_id,
@@ -1059,12 +1138,14 @@ export class AuthController extends PuterController {
* Build the error thrown when a verification SMS can't be sent (a delivery
* failure, or a refused/blocked send). Mints a short `error_id`, writes a
* single greppable line tying that id to the real reason (so support can
* look it up in CloudWatch with the id the user quotes), and returns the
* look it up in CloudWatch with the id the user quotes), stores the same
* record in KV under `sms-send-error:<error_id>` for a week (the admin
* abuse page looks it up there without needing log access), and returns the
* `HttpError` with the id attached as `error_id` for the GUI to surface.
* The phone number is deliberately omitted from the log line (PII); the
* user + country are enough to correlate.
* The phone number is deliberately omitted from the log line and the KV
* record (PII); the user + country are enough to correlate.
*/
private smsSendError(
private async smsSendError(
statusCode: number,
clientMessage: string,
reason: string,
@@ -1075,7 +1156,7 @@ export class AuthController extends PuterController {
detail?: unknown;
},
options: HttpErrorOptions = {},
): HttpError {
): Promise<HttpError> {
const errorId = uuidv4();
const detail =
ctx.detail instanceof Error ? ctx.detail.message : ctx.detail;
@@ -1086,6 +1167,26 @@ export class AuthController extends PuterController {
`country=${ctx.country ?? ''}` +
(detail ? ` detail=${JSON.stringify(String(detail))}` : ''),
);
// Best-effort: the record backs a support lookup, so a KV failure
// must never mask the error actually being reported.
try {
const now = Math.floor(Date.now() / 1000);
await this.stores.kv.set({
key: `sms-send-error:${errorId}`,
value: {
reason,
status: statusCode,
user_id: ctx.userId ?? null,
user_uid: ctx.userUid ?? null,
country: ctx.country ?? null,
detail: detail != null ? String(detail) : null,
t: now,
},
expireAt: now + SMS_SEND_ERROR_TTL_SECONDS,
});
} catch (e) {
console.warn('[send-confirm-phone] error-record store failed:', e);
}
return new HttpError(statusCode, clientMessage, {
...options,
fields: { ...options.fields, error_id: errorId },
@@ -1219,7 +1320,7 @@ export class AuthController extends PuterController {
legacyCode: 'account_suspended',
});
if (!this.clients.prelude?.isConfigured())
throw this.smsSendError(
throw await this.smsSendError(
503,
'Phone verification is unavailable.',
'prelude_not_configured',
@@ -1254,7 +1355,7 @@ export class AuthController extends PuterController {
// (see PreludeClient / countries.ts). Avoids paying exorbitant per-SMS
// rates in low-revenue, high-fraud geographies.
if (!this.clients.prelude.isCountrySupported(parsed.country))
throw this.smsSendError(
throw await this.smsSendError(
400,
'Phone verification is not available for this country.',
'country_not_supported',
@@ -1305,7 +1406,7 @@ export class AuthController extends PuterController {
// its meaning lives in the extension (which sets it) and the GUI (which
// displays it), so no abuse semantics leak into the OSS repo.
if (abuseCheck.allowed === false)
throw this.smsSendError(
throw await this.smsSendError(
429,
'Phone verification is unavailable for this number right now.',
`not_allowed:${abuseCheck.reason ?? 'unspecified'}`,
@@ -1339,7 +1440,7 @@ export class AuthController extends PuterController {
expireAt: Math.floor(Date.now() / 1000) + 60 * 60,
});
} catch (e) {
throw this.smsSendError(
throw await this.smsSendError(
503,
'Could not start phone verification.',
'pending_store_failed',
@@ -1378,7 +1479,7 @@ export class AuthController extends PuterController {
result.status === 'blocked' ||
result.status === 'shadow_blocked'
) {
throw this.smsSendError(
throw await this.smsSendError(
429,
'Phone verification is temporarily unavailable for this number.',
`prelude_${result.status}`,
@@ -1395,7 +1496,7 @@ export class AuthController extends PuterController {
}
} catch (e) {
if (e instanceof HttpError) throw e;
throw this.smsSendError(
throw await this.smsSendError(
502,
'Could not send verification code.',
'prelude_request_failed',
@@ -1549,11 +1650,11 @@ export class AuthController extends PuterController {
/**
* Start card verification for the calling user. Pure mechanism: the
* endpoint emits `puter.card-verification.setup` and a payments
* extension fills in the client credentials — the OSS backend holds
* no provider knowledge or config. Phone verification (when required)
* must be completed first; the ordering is enforced here so a client
* can't skip the cheaper gate.
* endpoint emits `puter.card-verification.setup` and a payments extension
* fills in the client credentials — the OSS backend holds no provider
* knowledge or config. Phone verification (when required) must be completed
* first; the ordering is enforced here so a client can't skip the cheaper
* gate.
*/
@Post('/card-verification/setup', {
subdomain: ['api', ''],
@@ -1664,9 +1765,9 @@ export class AuthController extends PuterController {
}
/**
* Complete card verification. The client confirms the setup intent with
* the payment provider directly, then posts the resulting id here; the
* payments extension checks it (and applies its own abuse limits) via
* Complete card verification. The client confirms the setup intent with the
* payment provider directly, then posts the resulting id here; the payments
* extension checks it (and applies its own abuse limits) via
* `puter.card-verification.confirm`. On success the gate clears exactly
* like `/confirm-phone` clears the phone gate.
*/
@@ -1725,6 +1826,7 @@ export class AuthController extends PuterController {
fingerprint: null as string | null,
funding: null as string | null,
country: null as string | null,
customer_id: null as string | null,
};
try {
await this.clients.event?.emitAndWait(
@@ -1781,6 +1883,7 @@ export class AuthController extends PuterController {
fingerprint: confirmEvent.fingerprint,
funding: confirmEvent.funding,
country: confirmEvent.country,
customer_id: confirmEvent.customer_id,
} as never,
{},
);
@@ -2949,7 +3052,13 @@ export class AuthController extends PuterController {
let app = await this.stores.app.getByUid(app_uid);
if (!app && resolvedFromOrigin) {
app = await this.stores.app.createFromOrigin(app_uid, origin);
// Hosted-subdomain origins get the site owner stamped as the
// app's creator at bootstrap; external origins stay unowned.
const ownerUserId =
await this.services.auth.subdomainOwnerIdFromOrigin(origin);
app = await this.stores.app.createFromOrigin(app_uid, origin, {
ownerUserId,
});
}
if (!app) {
throw new HttpError(404, `App ${app_uid} does not exist`, {
@@ -3458,13 +3567,9 @@ export class AuthController extends PuterController {
@Get('/group/list', { subdomain: 'api', requireUserActor: true })
async handleGroupList(req: Request, res: Response): Promise<void> {
const userId = req.actor!.user.id!;
const groupStore = this.stores.group as unknown as {
listByOwner: (id: number) => Promise<unknown[]>;
listByMember: (id: number) => Promise<unknown[]>;
};
const [owned, member] = await Promise.all([
groupStore.listByOwner(userId),
groupStore.listByMember(userId),
this.stores.group.listGroupsWithOwner(userId),
this.stores.group.listGroupsWithMember(userId),
]);
res.json({
owned_groups: owned,
@@ -3528,6 +3633,91 @@ export class AuthController extends PuterController {
res.status(204).end();
}
// -- Step-up ("elevation"), wired below --------------------------
//
// Mints the second-factor cookie for a session that re-proves identity: a
// fresh TOTP code when 2FA is enabled, otherwise the account password.
// Privileged endpoints require it on top of the session, so a leaked session
// alone can't exercise them. Accounts with neither credential (no password
// and 2FA disabled) can't elevate.
async handleElevate(req: Request, res: Response): Promise<void> {
const user = await this.stores.user.getById(req.actor!.user.id!, {
force: true,
});
if (!user)
throw new HttpError(404, 'User not found.', {
legacyCode: 'not_found',
});
if (user.suspended)
throw new HttpError(403, 'Account suspended.', {
legacyCode: 'account_suspended',
});
if (user.otp_enabled) {
const code = req.body?.code;
if (!code)
throw new HttpError(400, 'code is required.', {
legacyCode: 'bad_request',
fields: { factor: 'otp' },
});
if (
!verifyOtp(
user.username,
user.otp_secret as string,
String(code),
)
)
throw new HttpError(401, 'Incorrect code.', {
legacyCode: 'code_mismatch' as never,
fields: { factor: 'otp' },
});
} else if (user.password) {
const password = req.body?.password;
if (!password || typeof password !== 'string')
throw new HttpError(400, 'Password is required.', {
legacyCode: 'password_required',
fields: { factor: 'password' },
});
const match = await bcrypt.compare(
password,
user.password as string,
);
if (!match)
throw new HttpError(401, 'Incorrect password.', {
legacyCode: 'password_mismatch',
fields: { factor: 'password' },
});
} else {
// Neither credential on file (e.g. an account that only ever
// authenticated through an external identity provider).
throw new HttpError(
403,
'This account has no credential to re-authenticate with. Set a password or enable two-factor authentication first.',
{ legacyCode: 'elevation_unavailable' as never },
);
}
const token = signStepUpToken(this.services.token, user as never);
res.cookie(
STEP_UP_COOKIE_NAME,
token,
stepUpCookieOptions(this.config),
);
// A browser reads its elevation back from the httpOnly cookie and never
// needs the raw value; handing it to page JS would put the second factor
// within reach of an XSS. API clients have no cookie jar, so they get the
// token to send back as `x-puter-elevation`. Both paths proved the same
// password/TOTP — this only avoids needless exposure, it isn't a gate.
const cookieName = this.config.cookie_name ?? 'puter_token';
const usedSessionCookie =
!!req.token && req.token === req.cookies?.[cookieName];
res.json(
usedSessionCookie ? { elevated: true } : { elevated: true, token },
);
}
// -- Delete own account (user-protected, wired below) ------------
//
// Purge S3 objects + fsentries first, then the user row. FK
@@ -3540,6 +3730,9 @@ export class AuthController extends PuterController {
res.clearCookie(this.config.cookie_name ?? 'puter_token');
res.clearCookie('puter_token_v2');
res.clearCookie('puter_revalidation');
res.clearCookie(STEP_UP_COOKIE_NAME, {
...(this.config.domain ? { domain: this.config.domain } : {}),
});
await this.#cascadeDeleteUser(userId);
res.json({ success: true });
}
@@ -3686,6 +3879,37 @@ export class AuthController extends PuterController {
(req, res) => this.handleDeleteOwnUser(req, res),
);
// Step-up. Served on the root origin (browser form posts same-origin)
// and on `api` (SDK/script clients, which have no cookie jar and send a
// bearer). Deliberately NOT cookie-gated: the password/TOTP in the body
// is the control — a stolen token alone can't satisfy it, and it's also
// what makes CSRF a non-issue. `requireUserActor` still keeps app and
// access-token actors out, so an access token can never mint an
// elevation for its issuer.
router.post(
'/auth/elevate',
{
subdomain: ['api', ''],
requireUserActor: true,
allowUnconfirmed: true,
rateLimit: [
{
scope: 'elevate',
limit: 10,
window: 15 * 60_000,
key: 'user',
},
{
scope: 'elevate-ip',
limit: 40,
window: 15 * 60_000,
key: 'ip',
},
],
},
(req, res) => this.handleElevate(req, res),
);
const webSessionGate = createWebSessionActorGate();
router.post(
@@ -3828,10 +4052,10 @@ export class AuthController extends PuterController {
}
/**
* Config-blocklist + extension-driven email validation.
* Config blocklist (suffix match on cleaned email) blocks first; then
* the `email.validate` event lets extensions (abuse) reject.
* Throws HttpError(400) on rejection.
* Config-blocklist + extension-driven email validation. Config blocklist
* (suffix match on cleaned email) blocks first; then the `email.validate`
* event lets extensions (abuse) reject. Throws HttpError(400) on
* rejection.
*/
async #validateEmail(email: string): Promise<void> {
if (
@@ -3876,9 +4100,9 @@ export class AuthController extends PuterController {
/**
* Per-IP enumeration clamp on `auth_id`-bearing auth requests. Separate
* from the route-level rate limit so a tighter ceiling applies only to
* the path that takes a uuid hint from the body — the normal login
* path stays at its more generous limit.
* from the route-level rate limit so a tighter ceiling applies only to the
* path that takes a uuid hint from the body — the normal login path stays
* at its more generous limit.
*/
async #checkAuthIdRateLimit(req: Request): Promise<void> {
const ip = req.ip || req.socket?.remoteAddress || 'unknown';
@@ -3896,12 +4120,11 @@ export class AuthController extends PuterController {
}
/**
* Extract the `auth_id` claim from a client-supplied reauth_token.
* Returns null when no token was supplied. Throws on invalid/expired
* tokens. The reauth_token is a server-signed JWT minted by the
* authProbe at 401 time — accepting only the signed envelope (vs. a
* raw UUID) means a leaked auth_id alone can't attach a session to
* an existing account.
* Extract the `auth_id` claim from a client-supplied reauth_token. Returns
* null when no token was supplied. Throws on invalid/expired tokens. The
* reauth_token is a server-signed JWT minted by the authProbe at 401 time —
* accepting only the signed envelope (vs. a raw UUID) means a leaked
* auth_id alone can't attach a session to an existing account.
*/
#extractAuthIdFromReauthToken(suppliedToken: unknown): string | null {
if (suppliedToken === undefined || suppliedToken === null) return null;
@@ -3915,18 +4138,17 @@ export class AuthController extends PuterController {
}
/**
* Enforce a verified `auth_id` against the user the credential flow
* has resolved. Caller has already extracted `auth_id` from the
* server-signed reauth_token (or from an OTP-flow JWT). When the GUI
* is forced through reauth, the 401 response embeds the reauth_token;
* the client echoes it back so we can confirm the second login lands
* on the same user row — critical for temp users, where a fresh
* signup would otherwise mint a new account and strand their files.
* Enforce a verified `auth_id` against the user the credential flow has
* resolved. Caller has already extracted `auth_id` from the server-signed
* reauth_token (or from an OTP-flow JWT). When the GUI is forced through
* reauth, the 401 response embeds the reauth_token; the client echoes it
* back so we can confirm the second login lands on the same user row —
* critical for temp users, where a fresh signup would otherwise mint a new
* account and strand their files.
*
* No `authId` supplied → no-op (normal login). Unknown `authId` →
* 404 (mirrors username-not-found, avoids being an enumeration
* oracle). Mismatch against the resolved user → 409
* (`auth_id_mismatch`).
* No `authId` supplied → no-op (normal login). Unknown `authId` → 404
* (mirrors username-not-found, avoids being an enumeration oracle).
* Mismatch against the resolved user → 409 (`auth_id_mismatch`).
*/
async #enforceAuthIdMatch(
req: Request,
@@ -19,6 +19,12 @@
import { Readable } from 'node:stream';
import type { Request, RequestHandler, Response } from 'express';
import { trace } from '@opentelemetry/api';
import {
BasicTracerProvider,
InMemorySpanExporter,
SimpleSpanProcessor,
} from '@opentelemetry/sdk-trace-base';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { actorUid, type Actor } from '../../core/actor.js';
@@ -289,6 +295,157 @@ describe('DriverController.#handleCall (via captured router)', () => {
).rejects.toMatchObject({ statusCode: 404 });
});
// A lifecycle hook *exists* on every driver (inherited from PuterDriver),
// so the old `typeof driver[method] === 'function'` dispatch would have
// invoked it. It must not be reachable over /drivers/call.
it.each([
'onServerStart',
'onServerShutdown',
'onServerPrepareShutdown',
'getReportedCosts',
'constructor',
'toString',
])('rejects the framework method %s with 404', async (method) => {
const actor = await makeUserActor();
const req = makeReq({ interface: 'puter-kvstore', method }, actor);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('admits a bare session ("root" token) actor on the AI drivers', async () => {
// Privileged ("godmode") apps call AI with the user's own session
// token, so the AI drivers no longer set `noUserSession`: the
// session actor reaches the permission scan (403 forbidden — no
// perms granted) instead of the credential-shape rejection.
const actor = await makeUserActor();
const req = makeReq(
{
interface: 'puter-chat-completion',
method: 'complete',
args: { messages: [] },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
it('admits app and access-token actors on the AI drivers (they fail later on permission, not credential shape)', async () => {
const base = await makeUserActor();
const delegatedActors: Actor[] = [
{ ...base, app: { uid: `app-${uuidv4()}` } },
{
...base,
accessToken: {
uid: `tok-${uuidv4()}`,
issuer: base,
fullAccess: true,
},
},
];
for (const actor of delegatedActors) {
const req = makeReq(
{
interface: 'puter-chat-completion',
method: 'complete',
args: { messages: [] },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
// Fresh actors hold no service:* perms, so the call still
// 403s — but at the permission scan, proving the
// credential-shape gate let the delegated actor through.
legacyCode: 'forbidden',
});
}
});
it('admits a user-scoped worker session on the AI drivers', async () => {
// Workers deployed without an app binding authenticate as a user
// actor whose session row is kind='worker' — they must keep their
// AI access (they fail later on permission here, not on
// credential shape).
const base = await makeUserActor();
const actor: Actor = {
...base,
session: { uid: `sess-${uuidv4()}`, kind: 'worker' },
};
const req = makeReq(
{
interface: 'puter-chat-completion',
method: 'complete',
args: { messages: [] },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
it('still allows bare session actors on drivers without the noUserSession flag', async () => {
// puter-kvstore doesn't set the flag; the session actor reaches
// the permission scan (403 forbidden — no perms granted) instead
// of being rejected for its credential shape.
const actor = await makeUserActor();
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key: 'x', value: 'y' },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
});
it('rejects with 403 when the actor lacks the service permission', async () => {
// A fresh user has no service:* perms → permission.check returns
// false → 403.
@@ -615,4 +772,65 @@ describe('DriverController.registerRoutes', () => {
});
});
describe('DriverController.#handleCall tracing', () => {
let routes: Captured;
const exporter = new InMemorySpanExporter();
const provider = new BasicTracerProvider({
spanProcessors: [new SimpleSpanProcessor(exporter)],
});
beforeAll(() => {
routes = captureRoutes(controller);
trace.setGlobalTracerProvider(provider);
});
afterAll(async () => {
await provider.shutdown();
trace.disable();
});
it('emits driver + downstream spans around a successful call', async () => {
const actor = await makeUserActor();
await server.stores.permission.setFlatUserPerm(
actor.user!.id!,
'service:puter-kvstore:ii:puter-kvstore',
{
permission: 'service:puter-kvstore:ii:puter-kvstore',
deleted: false,
issuer_user_id: actor.user!.id!,
} as never,
);
exporter.reset();
const res = makeRes();
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key: `k-${uuidv4()}`, value: 'v' },
},
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](req, res as unknown as Response, () => {}),
);
expect((res.body as { success: boolean }).success).toBe(true);
const names = exporter.getFinishedSpans().map((s) => s.name);
const driverSpan = exporter
.getFinishedSpans()
.find((s) => s.name === 'driver.puter-kvstore.set');
expect(driverSpan).toBeDefined();
expect(driverSpan!.attributes).toMatchObject({
driver: 'puter-kvstore',
'driver.method': 'set',
'driver.name': 'puter-kvstore',
});
// The call path exercises the permission gate and the KV store's
// dynamo writes — both should have produced spans of their own.
expect(names).toContain('permission.scan');
expect(names.some((n) => n.startsWith('ddb.'))).toBe(true);
});
});
void vi; // unused but reserved for future expansion
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import type { Request, Response } from 'express';
@@ -22,6 +23,7 @@ import { actorUid } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { Controller } from '../../core/http/decorators.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import { assertNotUserSession } from '../../core/http/middleware/gates.js';
import {
acquireDriverConcurrent,
checkDriverRateLimit,
@@ -30,6 +32,7 @@ import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { DriverMeta } from '../../drivers/meta.js';
import {
isDriverStreamResult,
resolveCallableMethods,
resolveDriverMeta,
resolveDriverMethodConcurrent,
resolveDriverMethodRateLimit,
@@ -37,6 +40,7 @@ import {
import type { PermissionService } from '../../services/permission/PermissionService.js';
import { PermissionUtil } from '../../services/permission/permissionUtil.js';
import type { WithLifecycle } from '../../types';
import { withSpan } from '../../util/span.js';
import { PuterController } from '../types.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
@@ -113,15 +117,23 @@ const translateProviderError = (err: unknown): unknown => {
@Controller('/drivers')
export class DriverController extends PuterController {
/** iface → Map<driverName, driverInstance> */
/** Iface → Map<driverName, driverInstance> */
#drivers = new Map<string, Map<string, DriverInstance>>();
/** iface → default driver name */
/** Iface → default driver name */
#defaults = new Map<string, string>();
/**
* driver instance → resolved meta. Cached so the per-call rate-limit
* lookup doesn't have to walk prototype chains on every request.
* Driver instance → resolved meta. Cached so the per-call rate-limit lookup
* doesn't have to walk prototype chains on every request.
*/
#meta = new WeakMap<DriverInstance, DriverMeta>();
/**
* Driver instance → the set of method names callable via `/drivers/call`.
* Resolved once at registration (server startup) via
* `resolveCallableMethods`; the request path only does a `Set.has` lookup.
* This is what stops framework/lifecycle methods (`onServerStart`, etc.)
* and `Object.prototype` members from being invoked by remote callers.
*/
#callableMethods = new WeakMap<DriverInstance, Set<string>>();
constructor(...args: ConstructorParameters<typeof PuterController>) {
super(...args);
@@ -200,14 +212,19 @@ export class DriverController extends PuterController {
);
}
const fn = driver[method];
if (typeof fn !== 'function') {
// Only methods in the pre-resolved callable set are dispatchable.
// This excludes framework/lifecycle hooks (onServerStart, etc.),
// inherited base methods, and Object.prototype members, none of
// which are part of any interface's RPC contract.
const callable = this.#callableMethods.get(driver);
if (!callable?.has(method)) {
throw new HttpError(
404,
`Method '${method}' not found on driver '${ifaceName}'`,
{ legacyCode: 'not_found' },
);
}
const fn = driver[method];
// Resolve the concrete driver name for permission keys, falling
// back through prototype metadata → instance field → requested name.
@@ -218,6 +235,20 @@ export class DriverController extends PuterController {
requestedDriver ??
'unknown';
const driverMeta = this.#meta.get(driver);
// Drivers flagged `noUserSession` refuse the bare
// account-session ("root") token: callers must present an app or
// worker token, or an API token minted from the dashboard. This is
// the per-driver counterpart of the `noUserSession` route option —
// `/drivers/call` is one shared route, so the flag has to live on
// the driver rather than in `RouteOptions`. Checked before the
// permission scan so a session-token caller always gets the
// credential-shape message, not a permission error.
if (driverMeta?.noUserSession) {
assertNotUserSession(req.actor);
}
if (req.actor) {
const permService = this.services.permission as unknown as
| PermissionService
@@ -256,7 +287,6 @@ export class DriverController extends PuterController {
// — we hook `res.finish` / `res.close` for that so streamed
// responses hold their slot until the stream drains, and aborted
// requests still give the slot back.
const driverMeta = this.#meta.get(driver);
const rateLimitSpec = resolveDriverMethodRateLimit(
driverMeta?.rateLimit,
method,
@@ -382,11 +412,22 @@ export class DriverController extends PuterController {
}
// Drivers read actor/context via the Context API — no drilled args.
// The span ends when the method returns; for streamed results that
// is stream start, not stream drain (same window the lifecycle
// events below report as durationMs).
const startedAt = Date.now();
let result;
try {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
result = await (fn as (...x: unknown[]) => any).call(driver, args);
result = await withSpan(
`driver.${ifaceName}.${method}`,
{
driver: ifaceName,
'driver.method': method,
'driver.name': resolved,
},
// eslint-disable-next-line @typescript-eslint/no-explicit-any
() => (fn as (...x: unknown[]) => any).call(driver, args),
);
} catch (e) {
this.clients.event?.emit(
`driver.${ifaceName}.${method}.error`,
@@ -487,6 +528,11 @@ export class DriverController extends PuterController {
// Cache the resolved meta so the request hot-path can read the
// per-method rate-limit spec without re-walking the prototype.
this.#meta.set(instance, meta);
// Resolve the callable RPC surface once, at startup. The request
// path checks membership against this set instead of reflecting on
// the live instance, so lifecycle hooks / inherited framework
// methods can never be dispatched.
this.#callableMethods.set(instance, resolveCallableMethods(instance));
// Register each alias pointing at the same instance. Legacy puter-js
// calls that pass a provider id in the `driver` slot (e.g. the TTS
// module sends `aws-polly` / `openai-tts` / `elevenlabs-tts` instead
@@ -0,0 +1,229 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
/**
* Route-level coverage for `GET /fs/readdir`. The unit tests call the handler
* directly, which cannot catch a route that was never registered or a gate that
* rejects the request — so this suite drives real HTTP over a listening server,
* and authenticates purely through `?auth_token=` (no request headers), which is
* the whole point of offering the read as a GET.
*/
describe('GET /fs/readdir over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const readdirUrl = (params: Record<string, string>) => {
const url = new URL('/fs/readdir', env.apiOrigin);
for (const [key, value] of Object.entries(params)) {
url.searchParams.set(key, value);
}
return url;
};
it('lists a directory authenticated only by a query token', async () => {
const { username, token } = env.users.user;
const response = await fetch(
readdirUrl({ path: `/${username}`, auth_token: token }),
);
expect(response.status).toBe(200);
const body = (await response.json()) as Array<{
name: string;
uuid: string;
}>;
expect(Array.isArray(body)).toBe(true);
// Default provisioned home directories.
expect(body.map((e) => e.name)).toContain('Documents');
for (const entry of body) {
expect(entry.uuid).toEqual(expect.any(String));
}
});
it('paginates via query params', async () => {
const { username, token } = env.users.user;
const response = await fetch(
readdirUrl({
path: `/${username}`,
auth_token: token,
limit: '2',
cursor: '',
includeTotal: 'true',
}),
);
expect(response.status).toBe(200);
const page = (await response.json()) as {
items: unknown[];
cursor?: string;
total?: number;
};
expect(page.items.length).toBe(2);
expect(typeof page.total).toBe('number');
expect(page.cursor).toEqual(expect.any(String));
});
it('rejects an unauthenticated request', async () => {
const { username } = env.users.user;
const response = await fetch(readdirUrl({ path: `/${username}` }));
expect(response.status).toBeGreaterThanOrEqual(400);
expect(response.status).toBeLessThan(500);
});
it('does not leak internal ids or storage columns over the wire', async () => {
const { username, token } = env.users.user;
const response = await fetch(
readdirUrl({ path: `/${username}`, auth_token: token }),
);
const raw = await response.text();
const body = JSON.parse(raw) as Array<Record<string, unknown>>;
expect(body.length).toBeGreaterThan(0);
// Checked per-entry rather than by scanning the raw body: the nested
// `associatedApp` payload legitimately carries its own `id` (v1 has
// always exposed it), so a substring scan would false-alarm.
for (const entry of body) {
for (const field of [
'id',
'parentId',
'userId',
'associatedAppId',
'bucket',
'bucketRegion',
'publicToken',
'fileRequestToken',
]) {
expect(entry).not.toHaveProperty(field);
}
}
// No user-identifying data (emails, owner records) anywhere.
expect(raw).not.toContain('@');
expect(raw).not.toMatch(/"(email|owner|user_id|userId)"/);
});
it('lists a nested subtree recursively over GET', async () => {
const { username, token } = env.users.user;
const base = `/${username}/Documents/http-recursive-${Date.now()}`;
for (const path of [base, `${base}/a`, `${base}/a/b`]) {
const created = await fetch(new URL('/fs/mkdir', env.apiOrigin), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ path, auth_token: token }),
});
expect(created.status).toBe(200);
}
const response = await fetch(
readdirUrl({
path: base,
auth_token: token,
recursive: 'true',
depth: '5',
}),
);
expect(response.status).toBe(200);
const page = (await response.json()) as {
items: Array<{ path: string }>;
};
expect(
page.items.map((e) => e.path.slice(base.length + 1)).sort(),
).toEqual(['a', 'a/b']);
});
it('mkdir does not return internal ids, storage columns or tokens', async () => {
const { username, token } = env.users.user;
const response = await fetch(new URL('/fs/mkdir', env.apiOrigin), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
path: `/${username}/Documents/http-mkdir-${Date.now()}`,
auth_token: token,
}),
});
expect(response.status).toBe(200);
const entry = (await response.json()) as Record<string, unknown>;
expect(entry.uuid).toEqual(expect.any(String));
expect(entry.isDir).toBe(true);
for (const field of [
'id',
'parentId',
'userId',
'associatedAppId',
'bucket',
'bucketRegion',
'publicToken',
'fileRequestToken',
]) {
expect(entry).not.toHaveProperty(field);
}
});
it('startBatchWrite does not expose storage internals', async () => {
const { username, token } = env.users.user;
const response = await fetch(
new URL('/fs/startBatchWrite', env.apiOrigin),
{
method: 'POST',
// Array body, so there is no `auth_token` field for the auth
// probe to read — authenticate via the header instead.
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify([
{
fileMetadata: {
path: `/${username}/Documents/http-signed-${Date.now()}.bin`,
size: 4,
},
},
]),
},
);
expect(response.status).toBe(200);
const [target] = (await response.json()) as Array<
Record<string, unknown>
>;
// What a client actually needs to upload.
expect(target!.sessionId).toEqual(expect.any(String));
expect(typeof target!.url).toBe('string');
// What it does not: where the bytes physically live.
for (const field of ['bucket', 'bucketRegion', 'objectKey']) {
expect(target).not.toHaveProperty(field);
}
});
it('still serves the POST form for existing callers', async () => {
const { username, token } = env.users.user;
const response = await fetch(new URL('/fs/readdir', env.apiOrigin), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ path: `/${username}`, auth_token: token }),
});
expect(response.status).toBe(200);
const body = (await response.json()) as Array<{ name: string }>;
expect(body.map((e) => e.name)).toContain('Documents');
});
});
+643 -36
View File
@@ -3,24 +3,25 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import type { Request, Response } from 'express';
import type { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
@@ -28,9 +29,9 @@ import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { FSController } from './FSController.js';
import type {
ClientSignedWriteResponse,
CompleteWriteRequest,
SignedWriteRequest,
SignedWriteResponse,
} from './requestTypes.js';
// ── Test harness ────────────────────────────────────────────────────
@@ -91,11 +92,15 @@ const makeReq = <B>(init: {
headers?: Record<string, string>;
actor: Actor;
user?: { id: number; username: string };
method?: string;
}): Request => {
return {
body: init.body ?? ({} as B),
query: init.query ?? {},
headers: init.headers ?? {},
// Handlers that serve both verbs (readdir) read params from `query` on
// GET and `body` otherwise.
...(init.method ? { method: init.method } : {}),
actor: init.actor,
// Some controller helpers fall back to `req.user` (set by the
// session middleware) for id / username before reading `req.actor`.
@@ -165,13 +170,16 @@ describe('FSController.startBatchWrites', () => {
const req = makeReq<SignedWriteRequest[]>({ body, actor });
await withActor(actor, () => controller.startBatchWrites(req, res));
const responses = captured.body as SignedWriteResponse[];
const responses = captured.body as ClientSignedWriteResponse[];
expect(responses).toHaveLength(2);
for (const r of responses) {
expect(r.sessionId).toEqual(expect.any(String));
expect(r.objectKey).toEqual(expect.any(String));
expect(r.bucket).toEqual(expect.any(String));
expect(r.uploadMode).toBe('single');
// Storage internals stay server-side: the presigned `url` already
// encodes bucket and key.
for (const field of ['bucket', 'bucketRegion', 'objectKey']) {
expect(r).not.toHaveProperty(field);
}
// In-memory mock S3 still returns a presigned-URL string for
// single-mode uploads — verify it's there but don't assert
// shape (varies by region/host config).
@@ -203,7 +211,7 @@ describe('FSController.startBatchWrites', () => {
const { res, captured } = makeRes();
const req = makeReq<SignedWriteRequest[]>({ body, actor });
await withActor(actor, () => controller.startBatchWrites(req, res));
const [response] = captured.body as SignedWriteResponse[];
const [response] = captured.body as ClientSignedWriteResponse[];
const session = await server.stores.fsEntry.getPendingEntryBySessionId(
response!.sessionId,
);
@@ -329,7 +337,8 @@ describe('FSController.completeBatchWrites', () => {
startRes.res,
),
);
const startResponses = startRes.captured.body as SignedWriteResponse[];
const startResponses = startRes.captured
.body as ClientSignedWriteResponse[];
expect(startResponses).toHaveLength(2);
// 2) Complete via the controller. Single-mode completion only
@@ -352,16 +361,29 @@ describe('FSController.completeBatchWrites', () => {
const responses = captured.body as Array<{
sessionId: string;
wasOverwrite: boolean;
fsEntry: { path: string; userId: number; isDir: boolean };
fsEntry: Record<string, unknown>;
}>;
expect(responses.map((r) => r.fsEntry.path).sort()).toEqual([
expect(responses.map((r) => r.fsEntry.path as string).sort()).toEqual([
`/${username}/Documents/c.txt`,
`/${username}/Documents/d.txt`,
]);
for (const response of responses) {
expect(response.wasOverwrite).toBe(false);
expect(response.fsEntry.userId).toBe(userId);
expect(response.fsEntry.isDir).toBe(false);
// Ownership is asserted against the store below — the response
// itself must not carry the owner, storage columns, or the
// capability tokens.
for (const field of [
'userId',
'id',
'parentId',
'bucket',
'bucketRegion',
'publicToken',
'fileRequestToken',
]) {
expect(response.fsEntry).not.toHaveProperty(field);
}
}
// The real fsentries were committed and are now resolvable.
@@ -392,7 +414,7 @@ describe('FSController.completeBatchWrites', () => {
),
);
const [firstResponse] = firstStart.captured
.body as SignedWriteResponse[];
.body as ClientSignedWriteResponse[];
const firstComplete = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
@@ -424,7 +446,7 @@ describe('FSController.completeBatchWrites', () => {
),
);
const [secondResponse] = secondStart.captured
.body as SignedWriteResponse[];
.body as ClientSignedWriteResponse[];
const secondComplete = makeRes();
await withActor(actor, () =>
@@ -464,20 +486,25 @@ describe('FSController.completeBatchWrites', () => {
start.res,
),
);
const [started] = start.captured.body as SignedWriteResponse[];
const [started] = start.captured.body as ClientSignedWriteResponse[];
// Actually upload 4096 bytes to the session's object key (simulating
// a client that PUTs more than it declared via the signed URL).
// a client that PUTs more than it declared via the signed URL). The
// storage location isn't in the response any more, so read it off the
// upload session — a real client just PUTs to the presigned URL.
const session = await server.stores.fsEntry.getPendingEntryBySessionId(
started!.sessionId,
);
const realBytes = Buffer.alloc(4096, 0x41);
await server.stores.s3Object.uploadFromServer(
{
bucket: started!.bucket,
objectKey: started!.objectKey,
bucket: session!.bucket!,
objectKey: session!.objectKey,
contentType: 'application/octet-stream',
body: realBytes,
contentLength: realBytes.byteLength,
},
started!.bucketRegion,
session!.bucketRegion!,
);
const complete = makeRes();
@@ -518,7 +545,7 @@ describe('FSController.completeBatchWrites', () => {
),
);
const [firstResponse] = firstStart.captured
.body as SignedWriteResponse[];
.body as ClientSignedWriteResponse[];
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
@@ -555,7 +582,7 @@ describe('FSController.completeBatchWrites', () => {
),
);
const [secondResponse] = secondStart.captured
.body as SignedWriteResponse[];
.body as ClientSignedWriteResponse[];
const emitSpy = vi.spyOn(server.clients.event, 'emit');
let updatedCall: (typeof emitSpy.mock.calls)[number] | undefined;
@@ -635,7 +662,7 @@ describe('FSController.completeBatchWrites', () => {
startA.res,
),
);
const [aResponse] = startA.captured.body as SignedWriteResponse[];
const [aResponse] = startA.captured.body as ClientSignedWriteResponse[];
const err = await withActor(b.actor, () =>
controller
@@ -696,6 +723,41 @@ describe('FSController.statEntry', () => {
expect(body.name).toBe('stat-me');
});
it('does not leak backend-internal fields to the client', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: `/${username}/Documents/no-leak` },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.statEntry(
makeReq({
body: { path: `/${username}/Documents/no-leak` },
actor,
}),
res,
),
);
const body = captured.body as Record<string, unknown>;
for (const field of [
'bucket',
'bucketRegion',
'userId',
'publicToken',
'fileRequestToken',
]) {
expect(body).not.toHaveProperty(field);
}
});
it('includes the subtree size when return_size is set on a directory', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
@@ -813,7 +875,11 @@ describe('FSController.readdirEntries', () => {
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
).rejects.toMatchObject({
statusCode: 400,
// Matches the legacy `/readdir` code the SDK moved off of.
legacyCode: 'dest_is_not_a_directory',
});
});
});
@@ -1510,6 +1576,546 @@ describe('FSController.readdirEntries sort + limit', () => {
});
});
// -- /readdir pagination envelope --
describe('FSController.readdirEntries pagination', () => {
const makeDocs = async (names: string[]) => {
const { actor } = await makeUser();
const username = actor.user!.username!;
for (const name of names) {
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: `/${username}/Documents/${name}` },
actor,
}),
makeRes().res,
),
);
}
return { actor, path: `/${username}/Documents` };
};
const readdir = async (
actor: Awaited<ReturnType<typeof makeUser>>['actor'],
body: Record<string, unknown>,
) => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdirEntries(makeReq({ body, actor }), res),
);
return captured.body;
};
it('keeps the bare array response for limit/offset requests', async () => {
const { actor, path } = await makeDocs(['a', 'b', 'c']);
const body = await readdir(actor, { path, limit: 2, offset: 1 });
expect(Array.isArray(body)).toBe(true);
expect((body as unknown[]).length).toBe(2);
});
it('returns the envelope when cursor is present (null = first page)', async () => {
const { actor, path } = await makeDocs(['a', 'b', 'c']);
const page = (await readdir(actor, { path, cursor: null })) as {
items: Array<{ name: string }>;
cursor?: string;
};
expect(page.items.map((e) => e.name)).toEqual(['a', 'b', 'c']);
expect(page.cursor).toBeUndefined();
});
it('pages through children with cursors in sort order', async () => {
const { actor, path } = await makeDocs(['d1', 'd2', 'd3', 'd4', 'd5']);
const names: string[] = [];
let cursor: string | null | undefined = null;
do {
const page = (await readdir(actor, {
path,
limit: 2,
cursor,
})) as { items: Array<{ name: string }>; cursor?: string };
names.push(...page.items.map((e) => e.name));
cursor = page.cursor;
} while (cursor);
expect(names).toEqual(['d1', 'd2', 'd3', 'd4', 'd5']);
});
it('respects descending sort across pages', async () => {
const { actor, path } = await makeDocs(['a', 'b', 'c', 'd']);
const first = (await readdir(actor, {
path,
limit: 2,
cursor: null,
sortBy: 'name',
sortOrder: 'desc',
})) as { items: Array<{ name: string }>; cursor?: string };
expect(first.items.map((e) => e.name)).toEqual(['d', 'c']);
const second = (await readdir(actor, {
path,
limit: 2,
cursor: first.cursor,
})) as { items: Array<{ name: string }>; cursor?: string };
expect(second.items.map((e) => e.name)).toEqual(['b', 'a']);
});
it('rejects a cursor that conflicts with the requested sort', async () => {
const { actor, path } = await makeDocs(['a', 'b', 'c']);
const first = (await readdir(actor, {
path,
limit: 1,
cursor: null,
sortBy: 'name',
})) as { cursor?: string };
await expect(
withActor(actor, () =>
controller.readdirEntries(
makeReq({
body: {
path,
limit: 1,
cursor: first.cursor,
sortBy: 'size',
},
actor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('reports total with includeTotal', async () => {
const { actor, path } = await makeDocs(['a', 'b', 'c']);
const page = (await readdir(actor, {
path,
limit: 1,
cursor: null,
includeTotal: true,
})) as { items: unknown[]; total?: number };
expect(page.items.length).toBe(1);
expect(page.total).toBe(3);
});
it('wraps the root listing in an envelope when asked', async () => {
const { actor } = await makeUser();
const page = (await readdir(actor, {
path: '/',
cursor: null,
includeTotal: true,
})) as { items: unknown[]; total?: number; cursor?: string };
expect(Array.isArray(page.items)).toBe(true);
expect(page.total).toBe(page.items.length);
expect(page.cursor).toBeUndefined();
});
});
// -- /readdir recursive (nested listing) --
describe('FSController.readdirEntries recursive', () => {
// Seed a nested tree under Documents/tree and return its base path.
// Relative depths: l1a/l1b = 1, l2a = 2, l3a = 3, l4a = 4.
const makeTree = async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const base = `/${username}/Documents/tree`;
const dirs = [
base,
`${base}/l1a`,
`${base}/l1b`,
`${base}/l1a/l2a`,
`${base}/l1a/l2a/l3a`,
`${base}/l1a/l2a/l3a/l4a`,
];
for (const path of dirs) {
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({ body: { path }, actor }),
makeRes().res,
),
);
}
return { actor, userId, base };
};
const readdir = async (
actor: Awaited<ReturnType<typeof makeUser>>['actor'],
body: Record<string, unknown>,
) => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdirEntries(makeReq({ body, actor }), res),
);
return captured.body;
};
const rel = (base: string, items: Array<{ path: string }>) =>
items.map((e) => e.path.slice(base.length + 1)).sort();
it('depth 1 returns only direct children (like a normal readdir)', async () => {
const { actor, base } = await makeTree();
const page = (await readdir(actor, {
path: base,
recursive: true,
depth: 1,
})) as { items: Array<{ path: string }>; cursor?: string };
expect(rel(base, page.items)).toEqual(['l1a', 'l1b']);
});
it('deeper levels appear as depth grows', async () => {
const { actor, base } = await makeTree();
const d2 = (await readdir(actor, {
path: base,
recursive: true,
depth: 2,
})) as { items: Array<{ path: string }> };
expect(rel(base, d2.items)).toEqual(['l1a', 'l1a/l2a', 'l1b']);
const d3 = (await readdir(actor, {
path: base,
recursive: true,
depth: 3,
})) as { items: Array<{ path: string }> };
expect(rel(base, d3.items)).toEqual([
'l1a',
'l1a/l2a',
'l1a/l2a/l3a',
'l1b',
]);
});
it('caps depth at 10 so a huge depth returns the whole subtree', async () => {
const { actor, base } = await makeTree();
const page = (await readdir(actor, {
path: base,
recursive: true,
depth: 9999,
})) as { items: Array<{ path: string }> };
expect(rel(base, page.items)).toEqual([
'l1a',
'l1a/l2a',
'l1a/l2a/l3a',
'l1a/l2a/l3a/l4a',
'l1b',
]);
});
it('pages through the whole subtree with cursors, no dupes or gaps', async () => {
const { actor, base } = await makeTree();
const seen: string[] = [];
let cursor: string | null | undefined = null;
do {
const page = (await readdir(actor, {
path: base,
recursive: true,
depth: 10,
limit: 2,
cursor,
})) as { items: Array<{ path: string }>; cursor?: string };
expect(page.items.length).toBeLessThanOrEqual(2);
seen.push(...page.items.map((e) => e.path));
cursor = page.cursor;
} while (cursor);
expect(
rel(
base,
seen.map((path) => ({ path })),
),
).toEqual(['l1a', 'l1a/l2a', 'l1a/l2a/l3a', 'l1a/l2a/l3a/l4a', 'l1b']);
});
it('counts the subtree with includeTotal', async () => {
const { actor, base } = await makeTree();
const page = (await readdir(actor, {
path: base,
recursive: true,
depth: 2,
cursor: null,
includeTotal: true,
})) as { items: unknown[]; total?: number };
expect(page.total).toBe(3); // l1a, l1b, l2a
});
it('enriches entries with type, thumbnail and associatedApp', async () => {
const { actor, base } = await makeTree();
const page = (await readdir(actor, {
path: base,
recursive: true,
depth: 1,
})) as {
items: Array<{
type?: unknown;
thumbnail?: unknown;
associatedApp?: unknown;
}>;
};
for (const item of page.items) {
expect(item.type).toBe('folder');
expect(item.thumbnail ?? null).toBeNull();
expect('associatedApp' in item).toBe(true);
}
});
it('gives files a MIME type and an associatedApp field', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const dir = `/${username}/Documents/files`;
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({ body: { path: dir }, actor }),
makeRes().res,
),
);
await withActor(actor, () =>
controller.touchEntry(
makeReq({ body: { path: `${dir}/pic.png` }, actor }),
makeRes().res,
),
);
const page = (await readdir(actor, {
path: dir,
recursive: true,
depth: 1,
})) as {
items: Array<{
name: string;
type?: unknown;
associatedApp?: unknown;
}>;
};
const file = page.items.find((e) => e.name === 'pic.png')!;
expect(String(file.type)).toContain('image/png');
expect('associatedApp' in file).toBe(true);
});
it('masks denials for app-under-user actors as a 404 (legacy parity)', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
...userActor,
app: { uid: `app-readdir-${uuidv4()}` },
};
// The user's Documents is outside the app's AppData subtree, so the
// app can't list it. Legacy `/readdir` masks this as a 404
// subject_does_not_exist rather than leaking a 403.
await expect(
withActor(appActor, () =>
controller.readdirEntries(
makeReq({
body: { path: `/${username}/Documents` },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({
statusCode: 404,
legacyCode: 'subject_does_not_exist',
});
});
it('rejects recursive listing at the root', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
controller.readdirEntries(
makeReq({
body: { path: '/', recursive: true },
actor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('does not leak another users identically-named subtree', async () => {
const { actor, base } = await makeTree();
// A second user with the same relative tree must not appear.
await makeTree();
const page = (await readdir(actor, {
path: base,
recursive: true,
depth: 10,
})) as { items: Array<{ path: string }> };
for (const item of page.items) {
expect(item.path.startsWith(`${base}/`)).toBe(true);
}
expect(page.items).toHaveLength(5);
});
});
// -- /readdir over GET (query params) --
describe('FSController.readdirEntriesViaGet', () => {
const seed = async (names: string[]) => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const dir = `/${username}/Documents/get-readdir`;
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({ body: { path: dir }, actor }),
makeRes().res,
),
);
for (const name of names) {
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({ body: { path: `${dir}/${name}` }, actor }),
makeRes().res,
),
);
}
return { actor, dir };
};
// Query strings carry every value as a string — that is the whole risk
// surface of this route, so tests pass strings the way a real URL would.
const getReaddir = async (
actor: Awaited<ReturnType<typeof makeUser>>['actor'],
query: Record<string, unknown>,
) => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdirEntriesViaGet(
makeReq({ query, actor, method: 'GET' }),
res,
),
);
return captured.body;
};
it('lists a directory from query params', async () => {
const { actor, dir } = await seed(['a', 'b', 'c']);
const body = await getReaddir(actor, { path: dir });
expect(Array.isArray(body)).toBe(true);
expect((body as Array<{ name: string }>).map((e) => e.name)).toEqual([
'a',
'b',
'c',
]);
});
it('honors string limit/offset like the POST form', async () => {
const { actor, dir } = await seed(['a', 'b', 'c']);
const body = (await getReaddir(actor, {
path: dir,
limit: '2',
offset: '1',
})) as Array<{ name: string }>;
expect(body.map((e) => e.name)).toEqual(['b', 'c']);
});
it('treats an empty cursor as the first page and returns the envelope', async () => {
const { actor, dir } = await seed(['a', 'b', 'c']);
const page = (await getReaddir(actor, { path: dir, cursor: '' })) as {
items: Array<{ name: string }>;
cursor?: string;
};
expect(page.items.map((e) => e.name)).toEqual(['a', 'b', 'c']);
expect(page.cursor).toBeUndefined();
});
it('pages through with a string cursor', async () => {
const { actor, dir } = await seed(['d1', 'd2', 'd3', 'd4', 'd5']);
const names: string[] = [];
let cursor: string | undefined = '';
do {
const page = (await getReaddir(actor, {
path: dir,
limit: '2',
cursor,
})) as { items: Array<{ name: string }>; cursor?: string };
names.push(...page.items.map((e) => e.name));
cursor = page.cursor;
} while (cursor);
expect(names).toEqual(['d1', 'd2', 'd3', 'd4', 'd5']);
});
it('coerces string includeTotal=true', async () => {
const { actor, dir } = await seed(['a', 'b', 'c']);
const page = (await getReaddir(actor, {
path: dir,
limit: '1',
cursor: '',
includeTotal: 'true',
})) as { items: unknown[]; total?: number };
expect(page.items.length).toBe(1);
expect(page.total).toBe(3);
});
it('coerces string recursive/depth and lists nested entries', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const base = `/${username}/Documents/get-tree`;
for (const path of [base, `${base}/a`, `${base}/a/b`]) {
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({ body: { path }, actor }),
makeRes().res,
),
);
}
const page = (await getReaddir(actor, {
path: base,
recursive: 'true',
depth: '2',
})) as { items: Array<{ path: string }> };
expect(
page.items.map((e) => e.path.slice(base.length + 1)).sort(),
).toEqual(['a', 'a/b']);
});
it('rejects a non-directory target with the legacy code', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const file = `/${username}/Documents/get-file.txt`;
await withActor(actor, () =>
controller.touchEntry(
makeReq({ body: { path: file }, actor }),
makeRes().res,
),
);
await expect(
withActor(actor, () =>
controller.readdirEntriesViaGet(
makeReq({ query: { path: file }, actor, method: 'GET' }),
makeRes().res,
),
),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'dest_is_not_a_directory',
});
});
it('does not expose internal ids, storage columns or tokens', async () => {
const { actor, dir } = await seed(['a']);
const body = (await getReaddir(actor, { path: dir })) as Array<
Record<string, unknown>
>;
const entry = body[0]!;
// Numeric primary keys and storage/token columns must never ship in a
// listing — entries are addressed by uuid.
for (const field of [
'id',
'parentId',
'userId',
'associatedAppId',
'bucket',
'bucketRegion',
'publicToken',
'fileRequestToken',
]) {
expect(entry).not.toHaveProperty(field);
}
expect(entry.uuid).toEqual(expect.any(String));
// No user-identifying data anywhere in the serialized payload.
const serialized = JSON.stringify(body);
expect(serialized).not.toContain('@');
expect(serialized).not.toMatch(/"(email|owner|user_id|userId)"/);
});
});
// ── /touch additional branches ──────────────────────────────────────
describe('FSController.touchEntry additional branches', () => {
@@ -2022,10 +2628,9 @@ describe('FSController associatedAppId entitlement gate', () => {
],
);
const row = (
await server.clients.db.read(
'SELECT id FROM apps WHERE uid = ?',
[uid],
)
await server.clients.db.read('SELECT id FROM apps WHERE uid = ?', [
uid,
])
)[0] as { id: number };
return { id: row.id };
};
@@ -2041,13 +2646,15 @@ describe('FSController associatedAppId entitlement gate', () => {
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path, size: 3, associatedAppId } }],
body: [
{ fileMetadata: { path, size: 3, associatedAppId } },
],
actor,
}),
startRes.res,
),
);
const [started] = startRes.captured.body as SignedWriteResponse[];
const [started] = startRes.captured.body as ClientSignedWriteResponse[];
const completeRes = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
+310 -62
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import Busboy from 'busboy';
@@ -40,13 +41,24 @@ import {
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import {
assertAccess as assertLegacyAccess,
fsEntryMimeType,
loadLegacyAssociatedApps,
signEntryThumbnail,
toLegacyEntry,
} from './legacyFsHelpers.js';
import type {
ClientCompleteWriteResponse,
ClientFSEntry,
ClientReaddirEntry,
ClientSignedWriteResponse,
ClientSignMultipartPartsResponse,
ClientWriteResponse,
CompleteWriteRequest,
CompleteWriteResponse,
SignedWriteRequest,
SignedWriteResponse,
SignMultipartPartsRequest,
SignMultipartPartsResponse,
WriteGuiMetadata,
WriteRequest,
WriteResponse,
@@ -60,7 +72,6 @@ import type {
ThumbnailUploadPrepareItem,
ThumbnailUploadPreparePayload,
} from './types.js';
import { toLegacyEntry } from './legacyFsHelpers.js';
class UploadProgressTracker implements UploadProgressTrackerLike {
total = 0;
progress = 0;
@@ -89,6 +100,8 @@ class UploadProgressTracker implements UploadProgressTrackerLike {
}
const MAX_THUMBNAIL_BYTES = 2 * 1024 * 1024;
// Hard cap on how many levels below the target a recursive readdir descends.
const MAX_READDIR_DEPTH = 10;
const DEFAULT_BATCH_ACL_CHECK_CONCURRENCY = 32;
const DEFAULT_BATCH_WRITE_SIDE_EFFECT_CONCURRENCY = 8;
@@ -106,7 +119,7 @@ export class FSController extends PuterController {
@Post('/startWrite', { subdomain: 'api', requireVerified: true })
async startWrite(
req: Request<RouteParams, null, SignedWriteRequest>,
res: Response<SignedWriteResponse>,
res: Response<ClientSignedWriteResponse>,
) {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
@@ -156,13 +169,15 @@ export class FSController extends PuterController {
}
}, 'emitStartWriteDirectoryEvents');
}
res.json(response);
res.json(
this.#withoutStorageInternals(this.#withClientFsEntry(response)),
);
}
@Post('/startBatchWrite', { subdomain: 'api', requireVerified: true })
async startBatchWrites(
req: Request<RouteParams, null, SignedWriteRequest[]>,
res: Response<SignedWriteResponse[]>,
res: Response<ClientSignedWriteResponse[]>,
) {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
@@ -252,13 +267,17 @@ export class FSController extends PuterController {
}
}, 'emitStartBatchWriteDirectoryEvents');
}
res.json(responses);
res.json(
responses.map((r) =>
this.#withoutStorageInternals(this.#withClientFsEntry(r)),
),
);
}
@Post('/completeWrite', { subdomain: 'api', requireVerified: true })
async completeWrite(
req: Request<RouteParams, null, CompleteWriteRequest>,
res: Response<CompleteWriteResponse>,
res: Response<ClientCompleteWriteResponse>,
) {
const userId = this.#getActorUserId(req);
const requestBody = this.#withGuiMetadata(req.body, req.body);
@@ -278,13 +297,18 @@ export class FSController extends PuterController {
},
requestBody.guiMetadata,
);
res.json({ ...response, fsEntry: writeResponse.fsEntry });
res.json(
this.#withRequiredClientFsEntry({
...response,
fsEntry: writeResponse.fsEntry,
}),
);
}
@Post('/completeBatchWrite', { subdomain: 'api', requireVerified: true })
async completeBatchWrites(
req: Request<RouteParams, null, CompleteWriteRequest[]>,
res: Response<CompleteWriteResponse[]>,
res: Response<ClientCompleteWriteResponse[]>,
) {
const userId = this.#getActorUserId(req);
const requests = Array.isArray(req.body)
@@ -319,7 +343,9 @@ export class FSController extends PuterController {
return { ...writeResponse, fsEntry: withSideEffects.fsEntry };
},
);
res.json(updatedResponse);
res.json(
updatedResponse.map((r) => this.#withRequiredClientFsEntry(r)),
);
}
@Post('/abortWrite', { subdomain: 'api', requireVerified: true })
@@ -341,20 +367,20 @@ export class FSController extends PuterController {
@Post('/signMultipartParts', { subdomain: 'api', requireVerified: true })
async signMultipartParts(
req: Request<RouteParams, null, SignMultipartPartsRequest>,
res: Response<SignMultipartPartsResponse>,
res: Response<ClientSignMultipartPartsResponse>,
) {
const userId = this.#getActorUserId(req);
const response = await this.services.fs.signMultipartParts(
userId,
req.body,
);
res.json(response);
res.json(this.#withoutStorageInternals(response));
}
@Post('/write', { subdomain: 'api', requireVerified: true })
async write(
req: Request<RouteParams, null, WriteRequest>,
res: Response<WriteResponse>,
res: Response<ClientWriteResponse>,
) {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
@@ -394,13 +420,13 @@ export class FSController extends PuterController {
response,
requestBody.guiMetadata,
);
res.json(updatedResponse);
res.json(this.#withRequiredClientFsEntry(updatedResponse));
}
@Post('/batchWrite', { subdomain: 'api', requireVerified: true })
async batchWrites(
req: Request<RouteParams, null, WriteRequest[]>,
res: Response<WriteResponse[]>,
res: Response<ClientWriteResponse[]>,
) {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
@@ -702,7 +728,9 @@ export class FSController extends PuterController {
);
},
);
res.json(updatedResponses);
res.json(
updatedResponses.map((r) => this.#withRequiredClientFsEntry(r)),
);
return;
}
@@ -822,7 +850,9 @@ export class FSController extends PuterController {
);
},
);
res.json(updatedResponses);
res.json(
updatedResponses.map((r) => this.#withRequiredClientFsEntry(r)),
);
}
// -- Read-side routes ------------------------------------------------
@@ -845,17 +875,135 @@ export class FSController extends PuterController {
await this.services.suggestedApps.getSuggestedApps(entry);
res.json({
...entry,
...this.#toClientEntry(entry),
...(subtreeSize !== undefined ? { size: subtreeSize } : {}),
});
}
/**
* Strip backend-internal fields before returning an entry to a client.
* Storage location (bucket/region), the owner's numeric id, and the
* capability-token columns are never used by clients and must not leak to
* callers who only hold `see`/`list` on the entry — a share recipient, or
* (with public folders enabled) any authenticated user. The legacy read
* path already curates its output; this does the same for the v2 routes.
*/
#toClientEntry(entry: FSEntry): ClientFSEntry {
// Allowlist, not a denylist: a denylist silently ships every column
// added to `fsentries` later. Omits the numeric primary keys (`id`,
// `parentId`, `associatedAppId`), the storage columns, the owning
// `userId`, and the `publicToken`/`fileRequestToken` capability tokens.
//
// Tolerant of partially-hydrated entries: write/mkdir paths return a
// freshly-built entry that hasn't been through a subdomain join.
const subdomains = entry.subdomains ?? [];
return {
uuid: entry.uuid,
uid: entry.uid ?? entry.uuid,
parentUid: entry.parentUid ?? null,
path: entry.path,
name: entry.name,
isDir: entry.isDir,
isShortcut: entry.isShortcut,
shortcutTo: entry.shortcutTo ?? null,
isSymlink: entry.isSymlink,
symlinkPath: entry.symlinkPath ?? null,
isPublic: entry.isPublic ?? null,
immutable: entry.immutable,
metadata: entry.metadata ?? null,
modified: entry.modified,
created: entry.created ?? null,
accessed: entry.accessed ?? null,
size: entry.size ?? null,
layout: entry.layout ?? null,
subdomains,
workers: entry.workers ?? [],
hasWebsite: entry.hasWebsite ?? subdomains.length > 0,
suggestedApps: entry.suggestedApps ?? [],
};
}
/**
* Sanitize the `fsEntry` a write response carries, leaving the rest of the
* envelope (session id, presigned upload targets) untouched — those fields
* are the point of the response. Applied at every `res.json` on the write
* paths, which previously serialized the raw database row.
*
* The return type is the sanitized counterpart, so a caller cannot keep
* treating the result as though it still held a full `FSEntry`.
*/
#withClientFsEntry<T extends { fsEntry?: FSEntry }>(
response: T,
): Omit<T, 'fsEntry'> & { fsEntry?: ClientFSEntry } {
const { fsEntry, ...rest } = response;
return {
...rest,
...(fsEntry ? { fsEntry: this.#toClientEntry(fsEntry) } : {}),
};
}
/**
* Drop the storage internals from a presigned-upload envelope. The client
* uploads to the presigned URLs, which already encode bucket and key, so
* naming the physical location of a user's bytes buys the caller nothing.
*/
#withoutStorageInternals<
T extends { bucket: string; bucketRegion: string; objectKey: string },
>(response: T): Omit<T, 'bucket' | 'bucketRegion' | 'objectKey'> {
const { bucket, bucketRegion, objectKey, ...rest } = response;
return rest;
}
/** Same, for the responses whose `fsEntry` is always present. */
#withRequiredClientFsEntry<T extends { fsEntry: FSEntry }>(
response: T,
): Omit<T, 'fsEntry'> & { fsEntry: ClientFSEntry } {
return {
...response,
fsEntry: this.#toClientEntry(response.fsEntry),
};
}
/**
* `GET /fs/readdir` — same contract as the POST form, with parameters in
* the query string instead of the body. A read as a GET is cacheable and
* can authenticate via `?auth_token=`, which lets callers fetch a listing
* without a JSON body. Every value arrives as a string, so parameter
* parsing goes through the same coercion helpers the POST path uses.
*/
@Get('/readdir', { subdomain: 'api', requireVerified: true })
async readdirEntriesViaGet(req: Request, res: Response) {
return this.readdirEntries(req, res);
}
@Post('/readdir', { subdomain: 'api', requireVerified: true })
async readdirEntries(req: Request, res: Response) {
const actor = this.#requireActor(req);
const body = this.#toObjectRecord(req.body);
// GET carries its parameters in the query string; POST in the body.
const body = this.#toObjectRecord(
req.method === 'GET' ? req.query : req.body,
);
// Presence of `cursor` (null/empty means "first page") or
// `includeTotal` opts into the paginated `{items, cursor?, total?}`
// envelope. Legacy limit/offset requests keep the bare-array response.
const includeTotal = this.#toBoolean(body.includeTotal) === true;
const paginated =
Object.prototype.hasOwnProperty.call(body, 'cursor') ||
includeTotal;
// Undocumented: `recursive` lists descendants (prefix scan) up to
// `depth` levels below the target. Always paginated; sorts by path.
const recursive = this.#toBoolean(body.recursive) === true;
if (this.#isRootPathRef(body)) {
if (recursive) {
throw new HttpError(
400,
'recursive listing is not supported at the root',
{ legacyCode: 'bad_request' },
);
}
const { listRootEntries } =
await import('../../services/fs/rootListing.js');
const rootChildren = await listRootEntries(
@@ -873,54 +1021,155 @@ export class FSController extends PuterController {
child.suggestedApps = rootSuggestions[index] ?? [];
}
}
res.json(rootChildren);
const rootItems = await this.#toReaddirEntries(rootChildren);
if (paginated) {
res.json({
items: rootItems,
...(includeTotal ? { total: rootItems.length } : {}),
});
return;
}
res.json(rootItems);
return;
}
const parent = await this.#resolveEntryForRequest(body);
if (!parent.isDir) {
throw new HttpError(400, 'Target is not a directory', {
legacyCode: 'bad_request',
legacyCode: 'dest_is_not_a_directory',
});
}
await this.#assertAccess(actor, parent.path, 'list');
// Use the legacy access assertion so this endpoint stays behaviorally
// identical to the `/readdir` route the SDK moved off of — same error
// codes and the same app-actor 404 masking on denial.
await assertLegacyAccess(
this.services.acl,
this.services.fs,
actor,
parent.path,
'list',
);
const limit = this.#toNumberOrUndefined(body.limit);
const offset = this.#toNumberOrUndefined(body.offset);
const sortByRaw =
typeof body.sort_by === 'string'
? body.sort_by.toLowerCase()
typeof (body.sortBy ?? body.sort_by) === 'string'
? String(body.sortBy ?? body.sort_by).toLowerCase()
: undefined;
const sortBy =
(['name', 'modified', 'type', 'size'] as const).find(
(v) => v === sortByRaw,
) ?? null;
const sortOrderRaw =
typeof body.sort_order === 'string'
? body.sort_order.toLowerCase()
typeof (body.sortOrder ?? body.sort_order) === 'string'
? String(body.sortOrder ?? body.sort_order).toLowerCase()
: undefined;
const sortOrder =
(['asc', 'desc'] as const).find((v) => v === sortOrderRaw) ?? null;
if (recursive) {
const requestedDepth = this.#toNumberOrUndefined(body.depth);
const maxDepth = Math.min(
MAX_READDIR_DEPTH,
Math.max(1, Math.floor(requestedDepth ?? MAX_READDIR_DEPTH)),
);
const page = await this.services.fs.listDirectoryTreePage(
parent.userId,
parent.path,
{
limit,
cursor:
typeof body.cursor === 'string'
? body.cursor
: undefined,
maxDepth,
},
);
await this.#attachSuggestedApps(page.entries);
const total = includeTotal
? await this.services.fs.countDirectoryTree(
parent.userId,
parent.path,
maxDepth,
)
: undefined;
res.json({
items: await this.#toReaddirEntries(page.entries),
...(page.cursor ? { cursor: page.cursor } : {}),
...(total !== undefined ? { total } : {}),
});
return;
}
if (paginated) {
const page = await this.services.fs.listDirectoryPage(parent.uuid, {
limit,
cursor:
typeof body.cursor === 'string' ? body.cursor : undefined,
sortBy,
sortOrder,
});
await this.#attachSuggestedApps(page.entries);
const total = includeTotal
? await this.services.fs.countDirectory(parent.uuid)
: undefined;
res.json({
items: await this.#toReaddirEntries(page.entries),
...(page.cursor ? { cursor: page.cursor } : {}),
...(total !== undefined ? { total } : {}),
});
return;
}
const children = await this.services.fs.listDirectory(parent.uuid, {
limit,
offset,
sortBy,
sortOrder,
});
await this.#attachSuggestedApps(children);
res.json(await this.#toReaddirEntries(children));
}
/**
* Shape readdir entries in the v2 (camelCase) response shape, enriched with
* the three fields the SDK cannot reconstruct on its own so it can rebuild
* the v1 shape: `type` (MIME), a signed `thumbnail`, and `associatedApp`.
*/
async #toReaddirEntries(entries: FSEntry[]): Promise<ClientReaddirEntry[]> {
const appsById = await loadLegacyAssociatedApps(
this.stores.app,
entries,
);
return Promise.all(
entries.map(async (entry) => ({
...this.#toClientEntry(entry),
// Fields the client cannot derive on its own.
type: fsEntryMimeType(entry),
thumbnail: await signEntryThumbnail(
this.clients.event,
entry.uuid,
entry.thumbnail,
),
associatedApp:
entry.associatedAppId !== null
? (appsById.get(entry.associatedAppId) ?? null)
: null,
})),
);
}
async #attachSuggestedApps(entries: FSEntry[]): Promise<void> {
const suggestions =
await this.services.suggestedApps.getSuggestedAppsForEntries(
children,
entries,
);
for (let index = 0; index < children.length; index++) {
const child = children[index];
for (let index = 0; index < entries.length; index++) {
const child = entries[index];
if (child) {
child.suggestedApps = suggestions[index] ?? [];
}
}
res.json(children);
}
@Post('/search', { subdomain: 'api', requireVerified: true })
@@ -1068,7 +1317,7 @@ export class FSController extends PuterController {
) ?? false,
});
this.#emitGuiItemAdded(entry);
res.json(entry);
res.json(this.#toClientEntry(entry));
}
@Post('/touch', { subdomain: 'api', requireVerified: true })
@@ -1104,7 +1353,7 @@ export class FSController extends PuterController {
createMissingParents:
this.#toBoolean(body.create_missing_parents) ?? false,
});
res.json(entry);
res.json(this.#toClientEntry(entry));
}
@Post('/rename', { subdomain: 'api', requireVerified: true })
@@ -1122,7 +1371,7 @@ export class FSController extends PuterController {
const renamed = await this.services.fs.rename(entry, newName);
this.#emitGuiItemUpdated(renamed);
res.json(renamed);
res.json(this.#toClientEntry(renamed));
}
@Post('/delete', { subdomain: 'api', requireVerified: true })
@@ -1168,7 +1417,7 @@ export class FSController extends PuterController {
this.#toBoolean(body.dedupe_name ?? body.change_name) ?? false,
});
this.#emitGuiItemMoved(source, moved);
res.json(moved);
res.json(this.#toClientEntry(moved));
}
@Post('/copy', { subdomain: 'api', requireVerified: true })
@@ -1196,7 +1445,7 @@ export class FSController extends PuterController {
this.#toBoolean(body.dedupe_name ?? body.change_name) ?? true,
});
this.#emitGuiItemAdded(copy);
res.json(copy);
res.json(this.#toClientEntry(copy));
}
@Post('/mkshortcut', { subdomain: 'api', requireVerified: true })
@@ -1225,7 +1474,7 @@ export class FSController extends PuterController {
dedupeName: this.#toBoolean(body.dedupe_name) ?? true,
});
this.#emitGuiItemAdded(shortcut);
res.json(shortcut);
res.json(this.#toClientEntry(shortcut));
}
// -- Read-side helpers -----------------------------------------------
@@ -1280,12 +1529,12 @@ export class FSController extends PuterController {
}
/**
* Authorize creation of a new entry at `targetPath`. The standard rule
* is write on the parent, but we also accept write on the target itself
* — this lets an app create its own `/<user>/AppData/<app_uid>` folder
* (parent `AppData` is off-limits, but the target is the app's own
* subtree per ACLService's short-circuit) and lets recipients of a
* direct share on a not-yet-existent path materialize it.
* Authorize creation of a new entry at `targetPath`. The standard rule is
* write on the parent, but we also accept write on the target itself — this
* lets an app create its own `/<user>/AppData/<app_uid>` folder (parent
* `AppData` is off-limits, but the target is the app's own subtree per
* ACLService's short-circuit) and lets recipients of a direct share on a
* not-yet-existent path materialize it.
*/
async #assertCanCreate(actor: Actor, targetPath: string) {
const parent = pathPosix.dirname(targetPath);
@@ -1763,13 +2012,13 @@ export class FSController extends PuterController {
/**
* Bind `associatedAppId` onto the write input only when the actor is
* entitled to reference that app. `associatedAppId` is client-supplied
* and never trusted for authz, but it's echoed back in legacy FS
* responses — so an attacker could plant another tenant's private app id
* to confirm the row exists (an enumeration oracle) and harvest its
* metadata. Allow binding to public apps (their existence isn't secret)
* or to apps the actor owns; drop the association otherwise so the file
* simply carries no associated app.
* entitled to reference that app. `associatedAppId` is client-supplied and
* never trusted for authz, but it's echoed back in legacy FS responses — so
* an attacker could plant another tenant's private app id to confirm the
* row exists (an enumeration oracle) and harvest its metadata. Allow
* binding to public apps (their existence isn't secret) or to apps the
* actor owns; drop the association otherwise so the file simply carries no
* associated app.
*
* `#resolveWriteFileMetadata` has already copied the raw client value onto
* `fileMetadata`, so a dropped association must be actively stripped — not
@@ -1810,8 +2059,7 @@ export class FSController extends PuterController {
// the ActorUser type. Access via the escape hatch until a proper
// storage-quota mechanism is in place.
const actorUser = req.actor?.user as
| Record<string, unknown>
| undefined;
Record<string, unknown> | undefined;
const candidates = [
this.#toStorageCapacityCandidate(actorUser?.free_storage),
@@ -18,12 +18,15 @@
*/
import type { Request, Response } from 'express';
import { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { signFile } from '../../util/fileSigning.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { LegacyFSController } from './LegacyFSController.js';
@@ -688,6 +691,59 @@ describe('LegacyFSController.readdir', () => {
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('pages children with cursors and reports totals', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
for (const name of ['p1', 'p2', 'p3']) {
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { path: `/${username}/Documents/${name}` },
actor,
}),
makeRes().res,
),
);
}
const readdir = async (body: Record<string, unknown>) => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdir(makeReq({ body, actor }), res),
);
return captured.body;
};
const seen: string[] = [];
let cursor: string | null | undefined = null;
let total: number | undefined;
do {
const page = (await readdir({
path: `/${username}/Documents`,
limit: 2,
cursor,
includeTotal: true,
})) as {
items: Array<{ name: string }>;
cursor?: string;
total?: number;
};
seen.push(...page.items.map((e) => e.name));
total = page.total;
cursor = page.cursor;
} while (cursor);
expect(seen).toEqual(['p1', 'p2', 'p3']);
expect(total).toBe(3);
// Legacy limit-only requests keep the bare array response.
const bare = await readdir({
path: `/${username}/Documents`,
limit: 2,
});
expect(Array.isArray(bare)).toBe(true);
expect((bare as unknown[]).length).toBe(2);
});
});
// ── copy ────────────────────────────────────────────────────────────
@@ -720,13 +776,9 @@ describe('LegacyFSController.copy', () => {
copied: { path: string; name: string };
}>;
expect(body).toHaveLength(1);
expect(body[0].copied.path).toBe(
`/${username}/Pictures/src-folder`,
);
expect(body[0].copied.path).toBe(`/${username}/Pictures/src-folder`);
// The original still exists; the copy lives under Pictures.
expect(
await server.stores.fsEntry.getEntryByPath(src),
).not.toBeNull();
expect(await server.stores.fsEntry.getEntryByPath(src)).not.toBeNull();
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Pictures/src-folder`,
@@ -761,9 +813,7 @@ describe('LegacyFSController.copy', () => {
);
const body = captured.body as Array<{ copied: { path: string } }>;
expect(body[0].copied.path).toBe(
`/${username}/Pictures/renamed-copy`,
);
expect(body[0].copied.path).toBe(`/${username}/Pictures/renamed-copy`);
});
it('copies an empty file (no backing S3 object) without erroring', async () => {
@@ -876,9 +926,7 @@ describe('LegacyFSController.copy', () => {
expect(Buffer.concat(chunks).byteLength).toBe(0);
expect(download.contentLength).toBe(0);
// The entry must still exist — the ghost handler must NOT have run.
expect(
await server.stores.fsEntry.getEntryByPath(src),
).not.toBeNull();
expect(await server.stores.fsEntry.getEntryByPath(src)).not.toBeNull();
});
});
@@ -991,10 +1039,7 @@ describe('LegacyFSController.search', () => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.search(
makeReq({ body: { query: needle }, actor }),
res,
),
controller.search(makeReq({ body: { query: needle }, actor }), res),
);
const results = captured.body as Array<{ name: string }>;
expect(Array.isArray(results)).toBe(true);
@@ -1046,9 +1091,7 @@ describe('LegacyFSController.search', () => {
).toBe(true);
}
expect(
results.some(
(r) => r.path === `/${username}/Documents/${needle}`,
),
results.some((r) => r.path === `/${username}/Documents/${needle}`),
).toBe(false);
});
});
@@ -1128,10 +1171,7 @@ describe('LegacyFSController.sign', () => {
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.sign(
makeReq({ body: { items: [] }, actor }),
res,
),
controller.sign(makeReq({ body: { items: [] }, actor }), res),
),
).rejects.toMatchObject({ statusCode: 400 });
});
@@ -1286,6 +1326,113 @@ describe('LegacyFSController.writeFile', () => {
});
});
// ── writeFile (write IDOR: authorized subject must equal write target) ─
//
// Holding a write signature + write ACL on a single file must not let the
// caller redirect the upload to an attacker-named sibling via the `name`
// field. The write has to land on the signed file itself.
describe('LegacyFSController.writeFile (write IDOR)', () => {
const signingCfgOf = () => {
const cfg = (
controller as unknown as {
config: {
api_base_url?: string;
url_signature_secret?: string;
};
}
).config;
cfg.api_base_url = cfg.api_base_url ?? 'http://api.test.local';
cfg.url_signature_secret =
cfg.url_signature_secret ?? 'test-signing-secret';
return {
secret: cfg.url_signature_secret,
apiBaseUrl: cfg.api_base_url,
};
};
const multipartWriteReq = (opts: {
actor: Actor;
uid: string;
expires: string;
signature: string;
name?: string;
content: string;
}): Request => {
const boundary = '----lfsWriteIdorBoundary';
const payload =
`--${boundary}\r\n` +
'Content-Disposition: form-data; name="file"; ' +
'filename="upload.bin"\r\n' +
'Content-Type: application/octet-stream\r\n\r\n' +
`${opts.content}\r\n` +
`--${boundary}--\r\n`;
const req = Readable.from([Buffer.from(payload)]) as unknown as Request;
Object.assign(req, {
headers: {
'content-type': `multipart/form-data; boundary=${boundary}`,
},
query: {
uid: opts.uid,
expires: opts.expires,
signature: opts.signature,
operation: 'write',
},
body: opts.name !== undefined ? { name: opts.name } : {},
actor: opts.actor,
});
return req;
};
it('writes to the signed file, not an attacker-named sibling', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const dir = `/${victim.actor.user!.username}/Documents`;
const target = `${dir}/secret.txt`;
const sibling = `${dir}/passwords.txt`;
// Victim owns a single file; attacker gets write on just that file.
await withActor(victim.actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor: victim.actor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
expect(entry).toBeTruthy();
await server.services.permission.grantUserUserPermission(
victim.actor,
attacker.actor.user!.username!,
`fs:${entry!.uuid}:write`,
{},
);
const writeUrl = new URL(signFile(entry!, signingCfgOf()).write_url!);
const { res, captured } = makeRes();
await withActor(attacker.actor, () =>
controller.writeFile(
multipartWriteReq({
actor: attacker.actor,
uid: writeUrl.searchParams.get('uid')!,
expires: writeUrl.searchParams.get('expires')!,
signature: writeUrl.searchParams.get('signature')!,
name: 'passwords.txt',
content: 'attacker-bytes',
}),
res,
),
);
// The write landed on the signed file …
expect((captured.body as { path?: string }).path).toBe(target);
// … and never created the attacker-named sibling.
const siblingEntry =
await server.stores.fsEntry.getEntryByPath(sibling);
expect(siblingEntry).toBeFalsy();
});
});
// ── file (validation paths) ─────────────────────────────────────────
describe('LegacyFSController.file', () => {
@@ -1433,10 +1580,7 @@ describe('LegacyFSController.requestAppRootDir', () => {
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.requestAppRootDir(
makeReq({ body: {}, actor }),
res,
),
controller.requestAppRootDir(makeReq({ body: {}, actor }), res),
),
).rejects.toMatchObject({ statusCode: 400 });
});
@@ -1612,10 +1756,7 @@ describe('LegacyFSController.down', () => {
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.down(
makeReq({ query: { path: '/' }, actor }),
res,
),
controller.down(makeReq({ query: { path: '/' }, actor }), res),
),
).rejects.toMatchObject({ statusCode: 400 });
});
@@ -1672,9 +1813,9 @@ describe('LegacyFSController.mkdir additional branches', () => {
}),
actor: undefined,
} as unknown as Request;
await expect(
controller.mkdir(req, res),
).rejects.toMatchObject({ statusCode: 401 });
await expect(controller.mkdir(req, res)).rejects.toMatchObject({
statusCode: 401,
});
});
});
@@ -1694,17 +1835,12 @@ describe('LegacyFSController.delete additional branches', () => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.delete(
makeReq({ body: { path: target }, actor }),
res,
),
controller.delete(makeReq({ body: { path: target }, actor }), res),
);
const body = captured.body as { ok: boolean; uid: string };
expect(body.ok).toBe(true);
expect(typeof body.uid).toBe('string');
expect(
await server.stores.fsEntry.getEntryByPath(target),
).toBeNull();
expect(await server.stores.fsEntry.getEntryByPath(target)).toBeNull();
});
it('forwards descendants_only into fs.remove', async () => {
@@ -1835,9 +1971,9 @@ describe('LegacyFSController.df actor gate', () => {
...makeReq({ body: {}, actor }),
actor: undefined,
} as unknown as Request;
await expect(
controller.df(req, makeRes().res),
).rejects.toMatchObject({ statusCode: 401 });
await expect(controller.df(req, makeRes().res)).rejects.toMatchObject({
statusCode: 401,
});
});
});
@@ -1854,7 +1990,8 @@ describe('LegacyFSController.down file streaming', () => {
ended: false,
};
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { Writable } = require('node:stream') as typeof import('node:stream');
const { Writable } =
require('node:stream') as typeof import('node:stream');
const writable = new Writable({
write(chunk: Buffer, _enc, cb) {
captured.bodyChunks.push(chunk);
@@ -1900,18 +2037,19 @@ describe('LegacyFSController.down file streaming', () => {
const { res, captured } = makeStreamingRes();
await withActor(actor, () =>
controller.down(
makeReq({ query: { path: target }, actor }),
res,
),
controller.down(makeReq({ query: { path: target }, actor }), res),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(captured.statusCode).toBe(200);
// /down forces octet-stream regardless of the entry's true type.
expect(captured.headers['Content-Type']).toBe('application/octet-stream');
expect(captured.headers['Content-Type']).toBe(
'application/octet-stream',
);
expect(captured.headers['Content-Disposition']).toMatch(/^attachment;/);
expect(captured.headers['Content-Length']).toBe(String(body.byteLength));
expect(captured.headers['Content-Length']).toBe(
String(body.byteLength),
);
// The piped bytes match the file contents.
await new Promise<void>((resolve) => setImmediate(resolve));
if (captured.bodyChunks.length > 0) {
@@ -1958,7 +2096,8 @@ describe('LegacyFSController.read file streaming', () => {
bodyChunks: [] as Buffer[],
};
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { Writable } = require('node:stream') as typeof import('node:stream');
const { Writable } =
require('node:stream') as typeof import('node:stream');
const writable = new Writable({
write(chunk: Buffer, _enc, cb) {
captured.bodyChunks.push(chunk);
@@ -2001,18 +2140,19 @@ describe('LegacyFSController.read file streaming', () => {
const { res, captured } = makeStreamingRes();
await withActor(actor, () =>
controller.read(
makeReq({ query: { file: target }, actor }),
res,
),
controller.read(makeReq({ query: { file: target }, actor }), res),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(captured.statusCode).toBe(200);
// The v1 contract is octet-stream regardless of real mime; /fs/read
// (v2) is the type-aware variant. This is documented in the controller.
expect(captured.headers['Content-Type']).toBe('application/octet-stream');
expect(captured.headers['Content-Length']).toBe(String(body.byteLength));
expect(captured.headers['Content-Type']).toBe(
'application/octet-stream',
);
expect(captured.headers['Content-Length']).toBe(
String(body.byteLength),
);
});
it('honors options.realMime by forwarding the real content-type from mime-types', async () => {
@@ -2036,7 +2176,10 @@ describe('LegacyFSController.read file streaming', () => {
controller.read(
makeReq({ query: { file: target }, actor }),
res,
{ realMime: true } as never,
undefined,
{
realMime: true,
},
),
);
expect(captured.headers['Content-Type']).toMatch(/text\/html/);
@@ -2142,10 +2285,7 @@ describe('LegacyFSController.search fallback fields', () => {
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.search(
makeReq({ body: { text: needle }, actor }),
res,
),
controller.search(makeReq({ body: { text: needle }, actor }), res),
);
const results = captured.body as Array<{ name: string }>;
expect(results.some((r) => r.name === needle)).toBe(true);
@@ -2260,7 +2400,9 @@ describe('LegacyFSController.batch additional operations', () => {
),
);
expect(captured.statusCode).toBe(200);
const body = captured.body as { results: Array<Record<string, unknown>> };
const body = captured.body as {
results: Array<Record<string, unknown>>;
};
expect(body.results).toHaveLength(1);
});
@@ -2288,12 +2430,10 @@ describe('LegacyFSController.batch additional operations', () => {
),
);
expect(captured.statusCode).toBe(200);
expect(
await server.stores.fsEntry.getEntryByPath(target),
).toBeNull();
expect(await server.stores.fsEntry.getEntryByPath(target)).toBeNull();
});
it("runs a `shortcut` op pointing at an existing target uid", async () => {
it('runs a `shortcut` op pointing at an existing target uid', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/batch-sc-target`;
@@ -2441,10 +2581,7 @@ describe('LegacyFSController.suggestApps', () => {
),
);
const spy = vi.spyOn(
server.services.suggestedApps,
'getSuggestedApps',
);
const spy = vi.spyOn(server.services.suggestedApps, 'getSuggestedApps');
try {
const { res } = makeRes();
await withActor(appActor, () =>
@@ -2476,10 +2613,7 @@ describe('LegacyFSController.suggestApps', () => {
),
);
const spy = vi.spyOn(
server.services.suggestedApps,
'getSuggestedApps',
);
const spy = vi.spyOn(server.services.suggestedApps, 'getSuggestedApps');
try {
const { res } = makeRes();
await withActor(actor, () =>
@@ -2532,10 +2666,7 @@ describe('LegacyFSController.readdirSubdomains', () => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdirSubdomains(
makeReq({ body: {}, actor }),
res,
),
controller.readdirSubdomains(makeReq({ body: {}, actor }), res),
);
const rows = captured.body as Array<{ subdomain: string }>;
expect(rows.some((r) => r.subdomain === sd)).toBe(true);
@@ -2613,3 +2744,170 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
expect(typeof body.thumbnail).toBe('string');
});
});
// ── GET /get-launch-apps ────────────────────────────────────────────
//
// `recent` is a launch-metadata producer: it returns `index_url` for
// apps the user has opened before. The taskbar happens to launch these
// by name (so AppDriver's hosted-backing guard applies), but the field
// is served regardless, so the guard is applied here too.
describe('LegacyFSController GET /get-launch-apps', () => {
const hostedUrl = (sub: string) => `https://${sub}.site.puter.localhost/`;
// The route is an inline lambda in `registerRoutes`, so pull it off a
// freshly-registered router rather than calling a named method.
const getLaunchAppsHandler = () => {
const router = new PuterRouter();
(
controller as unknown as {
registerRoutes: (r: PuterRouter) => void;
}
).registerRoutes(router);
const route = router.routes.find(
(r) => r.method === 'get' && r.path === '/get-launch-apps',
);
if (!route) throw new Error('No GET /get-launch-apps route');
return route.handler;
};
const recordOpen = async (userId: number, appUid: string) => {
await server.clients.db.write(
'INSERT INTO `app_opens` (`app_uid`, `user_id`, `ts`) VALUES (?, ?, ?)',
[appUid, userId, Math.floor(Date.now() / 1000)],
);
};
const fetchRecent = async (
actor: Actor,
): Promise<Array<Record<string, unknown>>> => {
const { res, captured } = makeRes();
await withActor(actor, async () => {
await getLaunchAppsHandler()(makeReq({ actor }), res, () => {
throw new Error('handler called next() unexpectedly');
});
});
return (captured.body as { recent: Array<Record<string, unknown>> })
.recent;
};
const makeHostedApp = async (userId: number, sub: string) => {
const name = `launch-${Math.random().toString(36).slice(2, 10)}`;
const app = await server.stores.app.create(
{ name, title: 'Launchable', index_url: hostedUrl(sub) },
{ ownerUserId: userId },
);
return app as { uid: string };
};
it('returns index_url while the hosted subdomain is owned', async () => {
const { actor, userId } = await makeUser();
const sub = `live-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.subdomain.create({ userId, subdomain: sub });
const app = await makeHostedApp(userId, sub);
await recordOpen(userId, app.uid);
const entry = (await fetchRecent(actor)).find(
(a) => a.uuid === app.uid,
);
expect(entry).toBeDefined();
expect(String(entry?.index_url)).toContain(sub);
expect(entry?.privateAccess).toBeUndefined();
});
it('nulls index_url and denies launch once the subdomain is deleted', async () => {
const { actor, userId } = await makeUser();
const sub = `gone-${Math.random().toString(36).slice(2, 10)}`;
const row = await server.stores.subdomain.create({
userId,
subdomain: sub,
});
const app = await makeHostedApp(userId, sub);
await recordOpen(userId, app.uid);
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId },
);
const entry = (await fetchRecent(actor)).find(
(a) => a.uuid === app.uid,
);
expect(entry).toBeDefined();
expect(entry?.index_url).toBeNull();
expect(entry?.privateAccess).toMatchObject({
hasAccess: false,
reason: 'hosted_backing_unavailable',
});
});
it('nulls index_url once the subdomain is reclaimed by another user', async () => {
const owner = await makeUser();
const attacker = await makeUser();
const sub = `reclaim-${Math.random().toString(36).slice(2, 10)}`;
const row = await server.stores.subdomain.create({
userId: owner.userId,
subdomain: sub,
});
const app = await makeHostedApp(owner.userId, sub);
await recordOpen(owner.userId, app.uid);
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId: owner.userId },
);
await server.stores.subdomain.create({
userId: attacker.userId,
subdomain: sub,
});
const entry = (await fetchRecent(owner.actor)).find(
(a) => a.uuid === app.uid,
);
expect(entry?.index_url).toBeNull();
expect(entry?.privateAccess).toMatchObject({
hasAccess: false,
reason: 'hosted_backing_unavailable',
});
});
it('fails closed when the subdomain lookup errors', async () => {
const { actor, userId } = await makeUser();
const sub = `flaky-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.subdomain.create({ userId, subdomain: sub });
const app = await makeHostedApp(userId, sub);
await recordOpen(userId, app.uid);
const spy = vi
.spyOn(server.stores.subdomain, 'getBySubdomain')
.mockRejectedValue(new Error('db down'));
try {
const entry = (await fetchRecent(actor)).find(
(a) => a.uuid === app.uid,
);
expect(entry?.index_url).toBeNull();
} finally {
spy.mockRestore();
}
});
it('leaves non-hosted index_urls untouched', async () => {
const { actor, userId } = await makeUser();
const name = `ext-${Math.random().toString(36).slice(2, 10)}`;
const app = (await server.stores.app.create(
{
name,
title: 'External',
index_url: 'https://dev-owned-domain.example/',
},
{ ownerUserId: userId },
)) as { uid: string };
await recordOpen(userId, app.uid);
const entry = (await fetchRecent(actor)).find(
(a) => a.uuid === app.uid,
);
expect(entry?.index_url).toBe('https://dev-owned-domain.example/');
expect(entry?.privateAccess).toBeUndefined();
});
});
+168 -74
View File
@@ -3,32 +3,42 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import Busboy from 'busboy';
import type { Request, RequestHandler, Response } from 'express';
import type { NextFunction, Request, RequestHandler, Response } from 'express';
import { contentType as contentTypeFromMime } from 'mime-types';
import { posix as pathPosix } from 'node:path';
import type { Actor } from '../../core/actor.js';
import { effectiveActorApp, isAccessTokenActor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import {
assertNotSuspended,
assertVerifiedAccount,
} from '../../core/http/middleware/gates.js';
import { RouteOptions } from '../../core/http/index.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { ACLService } from '../../services/acl/ACLService.js';
import type { SignedFile } from '../../util/fileSigning.js';
import { verifySignature } from '../../util/fileSigning.js';
import {
buildHostedBackingDenial,
hostedIndexUrlBackingIsUnavailable,
} from '../../util/hostedAppBacking.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
@@ -44,7 +54,6 @@ import {
signingConfigFromAppConfig,
toLegacyEntry,
} from './legacyFsHelpers.js';
import { RouteOptions } from '../../core/http/index.js';
type RouterCache = Map<string, RequestHandler | null>;
@@ -185,6 +194,18 @@ export class LegacyFSController extends PuterController {
}
).getByUid(uid);
if (app) {
// Don't hand out an index_url whose puter-hosted
// backing is gone or reclaimed. The taskbar launches
// recents by name (so AppDriver's guard applies), but
// this list is a launch-metadata producer like any
// other — a future consumer reading index_url
// straight off it shouldn't inherit a stale origin.
const backingGone =
await hostedIndexUrlBackingIsUnavailable({
app,
subdomainStore: this.stores.subdomain,
config: this.config,
}).catch(() => true);
apps.push({
uuid: app.uid,
name: app.name,
@@ -192,7 +213,10 @@ export class LegacyFSController extends PuterController {
icon: app.icon ?? null,
godmode: Boolean(app.godmode),
maximize_on_start: Boolean(app.maximize_on_start),
index_url: app.index_url,
index_url: backingGone ? null : app.index_url,
...(backingGone
? { privateAccess: buildHostedBackingDenial() }
: {}),
// An app with no owner isn't owned by a Puter user —
// it's an "external" (origin-bootstrapped) app.
external:
@@ -312,6 +336,13 @@ export class LegacyFSController extends PuterController {
const actor = this.#requireActor(req);
const body = asRecord(req.body);
// Presence of `cursor` (null means "first page") or `includeTotal`
// opts into the paginated `{items, cursor?, total?}` envelope.
// Requests without pagination params keep the bare-array response.
const paginated =
Object.prototype.hasOwnProperty.call(body, 'cursor') ||
body.includeTotal === true;
if (this.#isRootPathRef(body)) {
const { listRootEntries } =
await import('../../services/fs/rootListing.js');
@@ -341,6 +372,15 @@ export class LegacyFSController extends PuterController {
}),
),
);
if (paginated) {
res.json({
items: shaped,
...(body.includeTotal === true
? { total: shaped.length }
: {}),
});
return;
}
res.json(shaped);
return;
}
@@ -359,10 +399,37 @@ export class LegacyFSController extends PuterController {
'list',
);
const children = await this.services.fs.listDirectory(parent.uuid, {
sortBy: this.#parseSortBy(body),
sortOrder: this.#parseSortOrder(body),
});
const sortBy = this.#parseSortBy(body);
const sortOrder = this.#parseSortOrder(body);
const limit =
typeof body.limit === 'number' || typeof body.limit === 'string'
? Number(body.limit)
: undefined;
const offset =
typeof body.offset === 'number' || typeof body.offset === 'string'
? Number(body.offset)
: undefined;
let children;
let cursor: string | undefined;
if (paginated) {
const page = await this.services.fs.listDirectoryPage(parent.uuid, {
limit,
cursor:
typeof body.cursor === 'string' ? body.cursor : undefined,
sortBy,
sortOrder,
});
children = page.entries;
cursor = page.cursor;
} else {
children = await this.services.fs.listDirectory(parent.uuid, {
limit: Number.isFinite(limit) ? limit : undefined,
offset: Number.isFinite(offset) ? offset : undefined,
sortBy,
sortOrder,
});
}
const suggestions =
await this.services.suggestedApps.getSuggestedAppsForEntries(
@@ -385,6 +452,19 @@ export class LegacyFSController extends PuterController {
toLegacyEntry(this.clients.event, c, { appsById }),
),
);
if (paginated) {
const total =
body.includeTotal === true
? await this.services.fs.countDirectory(parent.uuid)
: undefined;
res.json({
items: shaped,
...(cursor ? { cursor } : {}),
...(total !== undefined ? { total } : {}),
});
return;
}
res.json(shaped);
};
@@ -826,7 +906,12 @@ export class LegacyFSController extends PuterController {
res.json(shaped);
};
read = async (req: Request, res: Response, options = {}): Promise<void> => {
read = async (
req: Request,
res: Response,
_next?: NextFunction,
options: { realMime?: boolean } = {},
) => {
const actor = this.#requireActor(req);
const query = asRecord(req.query);
@@ -867,7 +952,10 @@ export class LegacyFSController extends PuterController {
// GUI) expect the raw-Blob shape. Use `/fs/read` for type-aware
// streaming.
if (options.realMime) {
res.setHeader('Content-Type', contentTypeFromMime(entry.name));
res.setHeader(
'Content-Type',
contentTypeFromMime(entry.name) as string,
);
} else {
res.setHeader('Content-Type', 'application/octet-stream');
}
@@ -934,20 +1022,25 @@ export class LegacyFSController extends PuterController {
});
}
req.actor = actor;
// This endpoint authenticates the token by hand and never runs the
// route gate chain, so the suspension and pending-verification checks
// that guard every other authenticated FS route have to run here.
assertNotSuspended(actor!.user);
assertVerifiedAccount(actor!.user);
req.actor = actor!;
Context.set('actor', actor);
// Forward back to regular read after setting actor
return this.read(req, res, { realMime: true });
return this.read(req, res, undefined, { realMime: true });
};
// -- Signed-URL + meta routes ----------------------------------------
/**
* POST /sign
* Body: `{ items: [{ uid?, path?, action }], app_uid? }`. Returns
* `{ signatures: [...], token? }`. Apps may only sign files under their
* own AppData subtree.
* POST /sign Body: `{ items: [{ uid?, path?, action }], app_uid? }`.
* Returns `{ signatures: [...], token? }`. Apps may only sign files under
* their own AppData subtree.
*/
sign = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
@@ -1072,10 +1165,10 @@ export class LegacyFSController extends PuterController {
};
/**
* POST /writeFile?uid=<uid>&operation=<op>
* Signature-authenticated multipart upload. `operation` dispatches to one
* of write/copy/move/mkdir/delete/rename/trash. Signature must be valid
* for `write` action on `uid`.
* POST /writeFile?uid=<uid>&operation=<op> Signature-authenticated
* multipart upload. `operation` dispatches to one of
* write/copy/move/mkdir/delete/rename/trash. Signature must be valid for
* `write` action on `uid`.
*/
writeFile = async (req: Request, res: Response): Promise<void> => {
const query = asRecord(req.query);
@@ -1126,35 +1219,35 @@ export class LegacyFSController extends PuterController {
);
}
// ACL re-check: require an authenticated actor with write
// permission. A valid write signature alone is not sufficient —
// the caller must also pass ACL, preventing exploitation of
// leaked write URLs by read-only share recipients.
const callerActor = this.#requireActor(req);
if (operation === 'write') {
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetEntry.path,
'write',
);
}
if (operation === 'write') {
const body = asRecord(req.body);
const parentEntry = targetEntry.isDir
? targetEntry
: await this.#resolveParentOfEntry(targetEntry);
const name =
typeof body.name === 'string'
? body.name
: targetEntry.isDir
? `upload-${Date.now()}`
: targetEntry.name;
const targetPath =
parentEntry.path === '/'
? `/${name}`
: `${parentEntry.path}/${name}`;
let targetPath: string;
if (targetEntry.isDir) {
const name =
typeof body.name === 'string'
? body.name
: `upload-${Date.now()}`;
targetPath =
targetEntry.path === '/'
? `/${name}`
: `${targetEntry.path}/${name}`;
await assertCanCreate(
this.services.acl,
this.services.fs,
callerActor,
targetPath,
);
} else {
targetPath = targetEntry.path;
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetPath,
'write',
);
}
// Parse multipart and pipe the first `file` part into fsService.write.
const uploadResult = await this.#multipartWrite(
@@ -1294,10 +1387,9 @@ export class LegacyFSController extends PuterController {
};
/**
* GET /file?uid=<uid>&signature=...&expires=...
* Signature-authenticated file read. Directories return a signed listing
* of children; files stream bytes (with Range support when `download`
* isn't requested).
* GET /file?uid=<uid>&signature=...&expires=... Signature-authenticated
* file read. Directories return a signed listing of children; files stream
* bytes (with Range support when `download` isn't requested).
*/
file = async (req: Request, res: Response): Promise<void> => {
const query = asRecord(req.query);
@@ -1392,9 +1484,9 @@ export class LegacyFSController extends PuterController {
};
/**
* POST /open_item — resolve an entry, grant the default suggested app
* write access to it, and return a signed URL + user-app token so the
* launched app can read/write the file via its app-under-user token.
* POST /open_item — resolve an entry, grant the default suggested app write
* access to it, and return a signed URL + user-app token so the launched
* app can read/write the file via its app-under-user token.
*
* Matches v1 semantics: permission is always granted as `write` — the
* underlying user's permission check still caps the effective access
@@ -1468,8 +1560,8 @@ export class LegacyFSController extends PuterController {
};
/**
* POST /auth/request-app-root-dir — an app-under-user requests stat on
* its own app root directory. The app must own itself.
* POST /auth/request-app-root-dir — an app-under-user requests stat on its
* own app root directory. The app must own itself.
*/
requestAppRootDir = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
@@ -1654,14 +1746,6 @@ export class LegacyFSController extends PuterController {
}
}
async #resolveParentOfEntry(entry: { path: string; userId: number }) {
const parentPath = pathPosix.dirname(entry.path);
const parent = await resolveV1Selector(this.stores.fsEntry, {
path: parentPath,
});
return parent;
}
async #multipartWrite(
req: Request,
userId: number,
@@ -2180,12 +2264,21 @@ export class LegacyFSController extends PuterController {
#serializeBatchError(err: unknown): Record<string, unknown> {
if (err instanceof HttpError) {
return {
const payload: Record<string, unknown> = {
error: true,
status: err.statusCode,
message: err.message,
code: err.legacyCode ?? err.code,
};
// Same as the terminal errorHandler: extra fields (e.g.
// `entry_name` on item_with_same_name_exists) ride along
// without clobbering the canonical slots.
if (err.fields) {
for (const [k, v] of Object.entries(err.fields)) {
if (!(k in payload)) payload[k] = v;
}
}
return payload;
}
if (err instanceof Error) {
return { error: true, status: 500, message: err.message };
@@ -2233,7 +2326,7 @@ export class LegacyFSController extends PuterController {
#parseSortBy(
body: Record<string, unknown>,
): 'name' | 'modified' | 'type' | 'size' | null {
const raw = getString(body, 'sort_by');
const raw = getString(body, 'sortBy') || getString(body, 'sort_by');
if (!raw) return null;
const normalized = raw.toLowerCase();
return (
@@ -2244,7 +2337,8 @@ export class LegacyFSController extends PuterController {
}
#parseSortOrder(body: Record<string, unknown>): 'asc' | 'desc' | null {
const raw = getString(body, 'sort_order');
const raw =
getString(body, 'sortOrder') || getString(body, 'sort_order');
if (!raw) return null;
const normalized = raw.toLowerCase();
return (['asc', 'desc'] as const).find((v) => v === normalized) ?? null;
+100 -79
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { posix as pathPosix } from 'node:path';
@@ -43,9 +44,8 @@ import type { IConfig } from '../../types.js';
/**
* Shared helpers used by the legacy FS route shims (LegacyFSController).
*
* Legacy clients speak snake_case and expect specific response shapes —
* these helpers encapsulate that translation so the route handlers stay
* terse.
* Legacy clients speak snake_case and expect specific response shapes — these
* helpers encapsulate that translation so the route handlers stay terse.
*/
// -- Body parsing -----------------------------------------------------
@@ -220,15 +220,15 @@ export async function assertAccess(
}
/**
* Authorize creation of a new entry at `targetPath`. The standard rule is
* write on the parent, but we also allow it when the actor has explicit
* write on the target itself — this covers an app creating its own
* `/<user>/AppData/<app_uid>` folder (parent `AppData` is off-limits, but
* the target is the app's own subtree per ACLService's short-circuit) and
* shares granted directly on a not-yet-created path.
* Authorize creation of a new entry at `targetPath`. The standard rule is write
* on the parent, but we also allow it when the actor has explicit write on the
* target itself — this covers an app creating its own
* `/<user>/AppData/<app_uid>` folder (parent `AppData` is off-limits, but the
* target is the app's own subtree per ACLService's short-circuit) and shares
* granted directly on a not-yet-created path.
*
* On failure, delegates to `assertAccess` on the parent so the error
* shape stays identical to the previous parent-only check.
* On failure, delegates to `assertAccess` on the parent so the error shape
* stays identical to the previous parent-only check.
*/
export async function assertCanCreate(
aclService: ACLService,
@@ -272,20 +272,21 @@ const toIntBool = (v: unknown): number => (v ? 1 : 0);
/**
* Convert an AppStore-normalized app row into the v1 `associated_app` shape
* embedded in legacy FS entries. Booleans round-trip back to integers (0/1)
* because the v1 wire contract emits them that way and existing clients key
* off it. Other columns pass through as-is — `metadata` is already parsed.
* because the v1 wire contract emits them that way and existing clients key off
* it. Other columns pass through as-is — `metadata` is already parsed.
*
* Two redactions, because `associatedAppId` can name an app the actor does
* not own (the column is client-writable display metadata, never trusted for
* authz — see FSController) and may point cross-tenant:
* - Owner identifiers (`owner_user_id`, `app_owner`) are never emitted —
* they're internal user references with no client-side display use.
* - For private or protected apps, the direct hosting URL and launch
* capability flags (`index_url`, `godmode`, `maximize_on_start`,
* `background`, `metadata`) are dropped. This mirrors AppDriver, which
* withholds `index_url` from callers without read entitlement. Display
* fields (name, icon, title) still pass through so the GUI can label the
* file's associated app.
* Two redactions, because `associatedAppId` can name an app the actor does not
* own (the column is client-writable display metadata, never trusted for authz
* — see FSController) and may point cross-tenant:
*
* - Owner identifiers (`owner_user_id`, `app_owner`) are never emitted — they're
* internal user references with no client-side display use.
* - For private or protected apps, the direct hosting URL and launch capability
* flags (`index_url`, `godmode`, `maximize_on_start`, `background`,
* `metadata`) are dropped. This mirrors AppDriver, which withholds
* `index_url` from callers without read entitlement. Display fields (name,
* icon, title) still pass through so the GUI can label the file's associated
* app.
*/
function mapAppForLegacyAssociatedApp(
app: Record<string, unknown>,
@@ -319,13 +320,15 @@ function mapAppForLegacyAssociatedApp(
/**
* Batch-load `associated_app` payloads for a set of entries. Dedupes app ids
* across the input, hands them to `AppStore.getByIds` (one pipelined Redis
* MGET + a single `id IN (…)` query for any cache misses), and returns a map
* keyed by app id holding the v1-shaped embed. Callers pass the result to
* `toLegacyEntry` via `opts.appsById` so each entry hydrates without a
* second round-trip.
* MGET
*
* Empty input short-circuits — readdir on a directory of plain files makes
* zero extra calls.
* - A single `id IN (…)` query for any cache misses), and returns a map keyed by
* app id holding the v1-shaped embed. Callers pass the result to
* `toLegacyEntry` via `opts.appsById` so each entry hydrates without a second
* round-trip.
*
* Empty input short-circuits — readdir on a directory of plain files makes zero
* extra calls.
*/
export async function loadLegacyAssociatedApps(
appStore: AppRowLookup,
@@ -348,12 +351,55 @@ export async function loadLegacyAssociatedApps(
}
/**
* Produce the snake_case entry shape legacy clients expect. If `thumbnail`
* is set, asks the thumbnail extension (via `thumbnail.read` event) to swap
* an S3 URL for a signed one. Pass `fsEntryStore`/`userStore` to hydrate
* `is_empty` (directories) and `owner` — both are required fields per
* the legacy stat contract but need extra DB lookups. Pass `appsById`
* (built via `loadLegacyAssociatedApps`) to populate `associated_app`.
* The v1 `type` field: a MIME content-type (e.g. "image/png; charset=utf-8")
* for files, or "folder" for directories. The GUI's icon lookup keys off
* `type.startsWith('image/')`, so a bare extension breaks icon selection.
*/
export function fsEntryMimeType(entry: {
isDir: boolean;
name: string;
}): string | null {
return entry.isDir ? 'folder' : contentTypeFromMime(entry.name) || null;
}
/**
* Swap an S3 thumbnail key for a signed URL via the thumbnail extension
* (`thumbnail.read` event). Returns the input unchanged when there's no
* thumbnail or no event client; returns null if signing yields nothing.
*/
export async function signEntryThumbnail(
eventClient: EventClient | undefined,
uuid: string,
thumbnail: string | null,
): Promise<string | null> {
if (
typeof thumbnail !== 'string' ||
thumbnail.length === 0 ||
!eventClient
) {
return thumbnail ?? null;
}
const thumbnailEntry = { uuid, thumbnail };
try {
// emitAndWait — listener mutates `thumbnail` on the payload; plain
// `emit` is fire-and-forget and would drop the rewrite.
await eventClient.emitAndWait('thumbnail.read', thumbnailEntry, {});
} catch {
// ignore — non-critical.
}
return typeof thumbnailEntry.thumbnail === 'string' &&
thumbnailEntry.thumbnail.length > 0
? thumbnailEntry.thumbnail
: null;
}
/**
* Produce the snake_case entry shape legacy clients expect. If `thumbnail` is
* set, asks the thumbnail extension (via `thumbnail.read` event) to swap an S3
* URL for a signed one. Pass `fsEntryStore`/`userStore` to hydrate `is_empty`
* (directories) and `owner` — both are required fields per the legacy stat
* contract but need extra DB lookups. Pass `appsById` (built via
* `loadLegacyAssociatedApps`) to populate `associated_app`.
*/
export async function toLegacyEntry(
eventClient: EventClient | undefined,
@@ -367,13 +413,7 @@ export async function toLegacyEntry(
} = {},
): Promise<Record<string, unknown>> {
const dirname = pathPosix.dirname(entry.path);
// v1 contract: `type` is a MIME content-type (e.g. "image/png; charset=utf-8")
// for files, or "folder" for directories. The GUI's icon lookup does
// `type.startsWith('image/')` etc., so a bare extension breaks every
// banner that falls through the name-extension ladder in item_icon.js.
const mimeType = entry.isDir
? 'folder'
: contentTypeFromMime(entry.name) || null;
const mimeType = fsEntryMimeType(entry);
const pathComponents = entry.path.split('/');
const appdata_app =
@@ -444,28 +484,11 @@ export async function toLegacyEntry(
}
// Let the thumbnail extension swap an s3:// key for a signed URL.
if (
typeof response.thumbnail === 'string' &&
(response.thumbnail as string).length > 0 &&
eventClient
) {
const thumbnailEntry = {
uuid: entry.uuid,
thumbnail: response.thumbnail as string,
};
try {
// emitAndWait — listener mutates `thumbnail` on the payload;
// plain `emit` is fire-and-forget and would drop the rewrite.
await eventClient.emitAndWait('thumbnail.read', thumbnailEntry, {});
} catch {
// ignore — non-critical.
}
response.thumbnail =
typeof thumbnailEntry.thumbnail === 'string' &&
thumbnailEntry.thumbnail.length > 0
? thumbnailEntry.thumbnail
: null;
}
response.thumbnail = await signEntryThumbnail(
eventClient,
entry.uuid,
response.thumbnail as string | null,
);
return response;
}
@@ -475,8 +498,8 @@ export { normalizeAbsolutePath };
// -- Signing ---------------------------------------------------------
/**
* Pull the signing config off the app config. Throws if either value is
* missing — these are required for signed URL routes to function.
* Pull the signing config off the app config. Throws if either value is missing
* — these are required for signed URL routes to function.
*/
export function signingConfigFromAppConfig(config: IConfig): SigningConfig {
const secret = config.url_signature_secret;
@@ -498,9 +521,7 @@ export function signingConfigFromAppConfig(config: IConfig): SigningConfig {
return { secret, apiBaseUrl };
}
/**
* Convenience wrapper: turn an FSEntry into a signed-file response object.
*/
/** Convenience wrapper: turn an FSEntry into a signed-file response object. */
export function signEntry(
entry: {
uuid: string;
+112 -10
View File
@@ -3,22 +3,27 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import type { FSEntry, FSEntryWriteInput } from '../../stores/fs/FSEntry.js';
import type { Readable } from 'node:stream';
import type {
FSEntry,
FSEntrySubdomain,
FSEntryWriteInput,
} from '../../stores/fs/FSEntry.js';
export type UploadMode = 'single' | 'multipart';
@@ -129,3 +134,100 @@ export interface WriteResponse {
requestedThumbnail?: string | null;
contentHashSha256?: string | null;
}
/**
* An `FSEntry` as it is safe to hand to a client. Built by an allowlist, so the
* `fsentries` primary key (`id`) and its `parentId`/`associatedAppId`
* references, the storage columns (`bucket`, `bucketRegion`), the owning
* `userId`, and the `publicToken`/`fileRequestToken` capability tokens never
* reach the wire. Entries are addressed by `uuid`.
*
* `shortcutTo` is the one numeric row reference that stays: the v1 contract has
* always exposed it as `shortcut_to` and the desktop resolves shortcuts through
* it, so dropping it would break them.
*
* The `?: never` members below are guards, not fields: they make a raw
* `FSEntry` fail to typecheck wherever a `ClientFSEntry` is expected. Without
* them this type is just a structural subset of `FSEntry`, so an unsanitized
* row would be silently assignable and the distinction would buy nothing.
*
* Checked by `tsc -p tsconfig.json` (the strict config). Note that
* `tsconfig.build.json` sets `noCheck: true` — it only transpiles, so it will
* not catch a violation here.
*/
export interface ClientFSEntry {
uuid: string;
uid: string;
parentUid: string | null;
path: string;
name: string;
isDir: boolean;
isShortcut: boolean;
shortcutTo: number | null;
isSymlink: boolean;
symlinkPath: string | null;
isPublic: boolean | null;
immutable: boolean;
metadata: string | null;
modified: number;
created: number | null;
accessed: number | null;
size: number | null;
layout: string | null;
subdomains: FSEntrySubdomain[];
workers: FSEntrySubdomain[];
hasWebsite: boolean;
suggestedApps: unknown[];
id?: never;
userId?: never;
parentId?: never;
associatedAppId?: never;
bucket?: never;
bucketRegion?: never;
publicToken?: never;
fileRequestToken?: never;
}
/**
* Wire counterparts of the write responses. The bare `…Response` types describe
* what the service produces internally (a real `FSEntry`); these describe what
* the controller sends after sanitizing. Keeping them distinct is what makes
* "did this response get sanitized?" a question the compiler answers.
*/
/**
* The presigned-upload envelope minus the storage internals. A client uploads
* to the presigned `url` / `multipartPartUrls`, which already carry everything
* S3 needs, so `bucket`, `bucketRegion`, and `objectKey` are ours to keep —
* they name where a user's bytes physically live.
*/
export type ClientSignedWriteResponse = Omit<
SignedWriteResponse,
'fsEntry' | 'bucket' | 'bucketRegion' | 'objectKey'
> & { fsEntry?: ClientFSEntry };
export type ClientSignMultipartPartsResponse = Omit<
SignMultipartPartsResponse,
'bucket' | 'bucketRegion' | 'objectKey'
>;
export type ClientCompleteWriteResponse = Omit<
CompleteWriteResponse,
'fsEntry'
> & { fsEntry: ClientFSEntry };
export type ClientWriteResponse = Omit<WriteResponse, 'fsEntry'> & {
fsEntry: ClientFSEntry;
};
/**
* A directory-listing entry: a sanitized entry plus the three fields a client
* cannot derive on its own — the MIME `type`, a _signed_ `thumbnail` URL (the
* stored value is an S3 key, which is useless to a client), and the resolved
* `associatedApp`.
*/
export type ClientReaddirEntry = ClientFSEntry & {
type: string | null;
thumbnail: string | null;
associatedApp: Record<string, unknown> | null;
};
@@ -277,6 +277,43 @@ describe('HomepageController GET /app/:name', () => {
expect(html).toMatch(/<!DOCTYPE html>/i);
expect(html).toContain('Cool App');
});
it('does not leak a private app index_url or owner id to an anonymous visitor', async () => {
const { userId } = await makeUser();
const name = `priv-${Math.random().toString(36).slice(2, 10)}`;
const secretUrl = `https://secret-${name}.example.com/`;
// `is_private` is a READ_ONLY_COLUMN, so it can't be set through the
// store's create/update allow-list — insert the private row directly.
await server.clients.db.write(
`INSERT INTO \`apps\` (\`uid\`, \`name\`, \`title\`, \`description\`, \`index_url\`, \`owner_user_id\`, \`is_private\`)
VALUES (?, ?, ?, ?, ?, ?, ?)`,
[
`app-${uuidv4()}`,
name,
'Private App',
'a private app row',
secretUrl,
userId,
1,
],
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/app/:name',
makeReq({ params: { name }, path: `/app/${name}` }),
res,
);
expect(captured.statusCode).toBe(200);
const html = String(captured.body);
// Public meta (title) is fine to render...
expect(html).toContain('Private App');
// ...but the private hosting URL and owner id must be redacted.
expect(html).not.toContain(secretUrl);
expect(html).not.toContain('owner_user_id');
});
});
// ── /show/* ─────────────────────────────────────────────────────────
@@ -101,6 +101,18 @@ export class HomepageController extends PuterController {
? safeJsonParse(app.metadata)
: (app.metadata as Record<string, unknown> | null)) ??
{};
// Never bake the raw store row into the shell — it exposes
// `index_url`, `owner_user_id`, and admin flags. Run it
// through the authoritative serializer, which redacts to a
// safe field subset and drops `index_url` for a private app
// this actor (possibly anonymous) isn't entitled to.
const actor =
(req as express.Request & { actor?: unknown }).actor ??
null;
const clientApp = await this.drivers.apps.toClientView(
app,
actor,
);
await sendShell(req, res, {
title: String(app.title ?? name),
description: String(app.description ?? ''),
@@ -110,7 +122,7 @@ export class HomepageController extends PuterController {
typeof metadata.social_image === 'string'
? metadata.social_image
: undefined,
app: app as Record<string, unknown>,
app: clientApp as Record<string, unknown>,
});
return;
}
@@ -52,6 +52,12 @@ const TEST_ORIGIN = 'http://test.local';
let server: PuterServer;
let router: PuterRouter;
// Stand-in for the abuse extension's signup veto. EventClient has no off(),
// so one shared listener is installed in beforeAll and tests swap the
// override in and out (same pattern as AuthController.test.ts).
type SignupValidateOverride = (data: Record<string, unknown>) => void;
let signupValidateOverride: SignupValidateOverride | null = null;
beforeAll(async () => {
server = await setupTestServer({
origin: TEST_ORIGIN,
@@ -75,6 +81,14 @@ beforeAll(async () => {
} as never);
router = new PuterRouter();
server.controllers.oidc.registerRoutes(router);
server.clients.event.on(
'puter.signup.validate',
(_k: unknown, data: unknown) => {
if (signupValidateOverride) {
signupValidateOverride(data as Record<string, unknown>);
}
},
);
});
afterAll(async () => {
@@ -83,6 +97,7 @@ afterAll(async () => {
afterEach(() => {
vi.restoreAllMocks();
signupValidateOverride = null;
});
interface CapturedResponse {
@@ -92,6 +107,7 @@ interface CapturedResponse {
redirectUrl?: string;
headers: Record<string, string>;
cookies: Array<{ name: string; value: string; opts?: unknown }>;
clearedCookies: Array<{ name: string; opts?: unknown }>;
contentType?: string;
}
@@ -100,6 +116,7 @@ const makeReq = (init: {
query?: Record<string, unknown>;
params?: Record<string, unknown>;
headers?: Record<string, string>;
cookies?: Record<string, string>;
method?: string;
}): Request => {
return {
@@ -107,6 +124,7 @@ const makeReq = (init: {
query: init.query ?? {},
params: init.params ?? {},
headers: init.headers ?? {},
cookies: init.cookies ?? {},
method: init.method ?? 'GET',
} as unknown as Request;
};
@@ -117,6 +135,7 @@ const makeRes = () => {
body: undefined,
headers: {},
cookies: [],
clearedCookies: [],
};
const res = {
json: vi.fn((value: unknown) => {
@@ -155,6 +174,10 @@ const makeRes = () => {
captured.cookies.push({ name, value, opts });
return res;
}),
clearCookie: vi.fn((name: string, opts?: unknown) => {
captured.clearedCookies.push({ name, opts });
return res;
}),
type: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
@@ -593,6 +616,7 @@ describe('OIDCController login callback', () => {
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
expect(captured.redirectUrl).toContain('message=account_suspended');
// No session cookie issued for suspended accounts.
expect(captured.cookies).toHaveLength(0);
});
@@ -674,9 +698,7 @@ describe('OIDCController login callback', () => {
expect(captured.redirectUrl).toContain('embedded_in_popup=true');
expect(captured.redirectUrl).toContain('msg_id=msg-99');
expect(captured.redirectUrl).toContain('auth_error=1');
// 'This account is suspended.' is not in ALLOWED_ERRORS, so the
// builder falls back to 'unauthorized'.
expect(captured.redirectUrl).toContain('message=unauthorized');
expect(captured.redirectUrl).toContain('message=account_suspended');
expect(captured.redirectUrl).toContain(
`opener_origin=${encodeURIComponent('http://opener.test')}`,
);
@@ -767,6 +789,291 @@ describe('OIDCController login callback', () => {
});
});
// -- Browser binding / login-CSRF --------------------------------------
describe('OIDCController signup veto (abuse harness)', () => {
const vetoWithTrail = (email: string, trailId: string) => {
signupValidateOverride = (data) => {
if (data.email !== email) return;
data.allow = false;
data.trail_id = trailId;
};
};
const stubIdp = (sub: string, email: string) => {
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub,
email,
email_verified: true,
} as never);
};
it('createUserFromOIDC returns signup_blocked and the trail id as requestCode', async () => {
const email = `veto-${Math.random().toString(36).slice(2, 8)}@test.local`;
vetoWithTrail(email, 'trail-svc-1');
const created = await runWithContext(
{ req: makeReq({}) },
() =>
oidc().createUserFromOIDC('custom', {
sub: `sub-${Math.random().toString(36).slice(2, 8)}`,
email,
email_verified: true,
}),
);
expect(created.success).toBe(false);
expect(created.code).toBe('signup_blocked');
expect(created.requestCode).toBe('trail-svc-1');
});
it('login callback redirects with signup_blocked + request_code when first sign-in is vetoed', async () => {
const email = `veto-${Math.random().toString(36).slice(2, 8)}@test.local`;
vetoWithTrail(email, 'trail-login-1');
stubIdp(`sub-${Math.random().toString(36).slice(2, 8)}`, email);
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
});
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
expect(captured.redirectUrl).toContain('action=login');
expect(captured.redirectUrl).toContain('message=signup_blocked');
expect(captured.redirectUrl).toContain('request_code=trail-login-1');
expect(captured.cookies).toHaveLength(0);
});
it('signup callback redirects with signup_blocked + request_code when vetoed', async () => {
const email = `veto-${Math.random().toString(36).slice(2, 8)}@test.local`;
vetoWithTrail(email, 'trail-signup-1');
stubIdp(`sub-${Math.random().toString(36).slice(2, 8)}`, email);
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
});
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/signup',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
expect(captured.redirectUrl).toContain('action=signup');
expect(captured.redirectUrl).toContain('message=signup_blocked');
expect(captured.redirectUrl).toContain('request_code=trail-signup-1');
expect(captured.cookies).toHaveLength(0);
});
it('a veto with no trail id still redirects with signup_blocked and no request_code', async () => {
const email = `veto-${Math.random().toString(36).slice(2, 8)}@test.local`;
signupValidateOverride = (data) => {
if (data.email !== email) return;
data.allow = false;
};
stubIdp(`sub-${Math.random().toString(36).slice(2, 8)}`, email);
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
});
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/signup',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.redirectUrl).toContain('message=signup_blocked');
expect(captured.redirectUrl).not.toContain('request_code');
});
});
describe('OIDCController browser binding', () => {
const NONCE_COOKIE = 'puter_oidc_nonce';
const stubIdP = (sub: string, email: string) => {
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub,
email,
email_verified: true,
} as never);
};
it('/start sets an HttpOnly nonce cookie matching the nonce embedded in state', async () => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/:provider/start',
makeReq({ params: { provider: 'custom' } }),
res,
);
const nonceCookie = captured.cookies.find(
(c) => c.name === NONCE_COOKIE,
);
expect(nonceCookie).toBeTruthy();
expect(nonceCookie?.value).toBeTruthy();
expect((nonceCookie?.opts as { httpOnly?: boolean })?.httpOnly).toBe(
true,
);
// The cookie value must equal the nonce baked into the signed state.
const state = new URL(captured.redirectUrl ?? '').searchParams.get(
'state',
);
const decoded = oidc().verifyState(state!);
expect(decoded?.nonce).toBe(nonceCookie?.value);
});
it('completes login when the nonce cookie matches the state nonce', async () => {
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
const email = `bind-${Math.random().toString(36).slice(2, 8)}@test.local`;
const nonce = 'browser-nonce-match';
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
nonce,
});
stubIdP(sub, email);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({
query: { code: 'c', state },
cookies: { [NONCE_COOKIE]: nonce },
}),
res,
);
// Session cookie issued; single-use nonce cookie cleared.
expect(captured.cookies).toHaveLength(1);
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
expect(
captured.clearedCookies.some((c) => c.name === NONCE_COOKIE),
).toBe(true);
});
it('rejects login (no session cookie) when the nonce cookie is absent — the login-CSRF case', async () => {
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
nonce: 'attacker-flow-nonce',
});
// If enforcement were missing, this would resolve a user and set a
// session cookie for the victim's browser. It must not get that far.
const exchangeSpy = vi.spyOn(oidc(), 'exchangeCodeForTokens');
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
// Victim's browser has no nonce cookie for the attacker's flow.
makeReq({ query: { code: 'c', state }, cookies: {} }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
expect(captured.cookies).toHaveLength(0);
// We bail before ever exchanging the code.
expect(exchangeSpy).not.toHaveBeenCalled();
});
it('rejects login when the nonce cookie does not match the state nonce', async () => {
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
nonce: 'expected-nonce',
});
const exchangeSpy = vi.spyOn(oidc(), 'exchangeCodeForTokens');
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({
query: { code: 'c', state },
cookies: { [NONCE_COOKIE]: 'a-different-nonce' },
}),
res,
);
expect(captured.redirectUrl).toContain('auth_error=1');
expect(captured.cookies).toHaveLength(0);
expect(exchangeSpy).not.toHaveBeenCalled();
});
it('rejects the revalidate callback (400) when the nonce cookie is missing', async () => {
const state = oidc().signState({
provider: 'custom',
flow: 'revalidate',
user_uuid: uuidv4(),
nonce: 'reval-nonce',
});
const exchangeSpy = vi.spyOn(oidc(), 'exchangeCodeForTokens');
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/revalidate',
makeReq({ query: { code: 'c', state }, cookies: {} }),
res,
);
expect(captured.statusCode).toBe(400);
expect(exchangeSpy).not.toHaveBeenCalled();
});
it('lets legacy nonce-less state through (deploy grace) without touching the nonce cookie', async () => {
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
const email = `legacy-${Math.random().toString(36).slice(2, 8)}@test.local`;
// No `nonce` field — mimics a state signed before this shipped.
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
});
stubIdP(sub, email);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'c', state }, cookies: {} }),
res,
);
// Proceeds as before; no nonce cookie is cleared for legacy states.
expect(captured.cookies).toHaveLength(1);
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
expect(
captured.clearedCookies.some((c) => c.name === NONCE_COOKIE),
).toBe(false);
});
});
// ── /auth/oidc/callback/signup ──────────────────────────────────────
describe('OIDCController signup callback', () => {
+100 -21
View File
@@ -17,6 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import crypto from 'node:crypto';
import type { Request, Response } from 'express';
import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
@@ -26,45 +27,63 @@ import { sessionCookieFlags } from '../../util/cookieFlags.js';
const REVALIDATION_COOKIE_NAME = 'puter_revalidation';
const REVALIDATION_EXPIRY_SEC = 300;
// Companion cookie that binds an OIDC flow to the browser that started it.
// Expiry mirrors STATE_EXPIRY_SEC in OIDCService — the state and its
// browser-binding cookie must expire together.
const OIDC_NONCE_COOKIE_NAME = 'puter_oidc_nonce';
const OIDC_NONCE_EXPIRY_SEC = 600;
const OIDC_ERROR_REDIRECT_MAP: Record<string, Record<string, string>> = {
login: { account_not_found: 'signup', other: 'login' },
signup: { account_already_exists: 'login', other: 'signup' },
};
const ALLOWED_ERRORS = ['account_suspended', 'unauthorized'] as const;
// The `message` query param is clamped to these codes — the GUI maps them to
// display text. Free-text errors (which may describe internals or, for vetoed
// signups, the block reason) never reach the redirect URL.
const ALLOWED_ERRORS = [
'account_suspended',
'unauthorized',
'signup_blocked',
] as const;
function buildErrorRedirectUrl(
origin: string,
sourceFlow: string,
errorCondition: string,
message: (typeof ALLOWED_ERRORS)[number],
message: string,
stateDecoded?: Record<string, unknown>,
requestCode?: string,
): string {
const targetFlow =
OIDC_ERROR_REDIRECT_MAP[sourceFlow]?.[errorCondition] ?? sourceFlow;
const base = origin.replace(/\/$/, '') || '/';
const clamped = (ALLOWED_ERRORS as readonly string[]).includes(message)
? message
: 'unauthorized';
let params: URLSearchParams;
if (stateDecoded?.embedded_in_popup && stateDecoded?.msg_id != null) {
const params = new URLSearchParams({
params = new URLSearchParams({
embedded_in_popup: 'true',
msg_id: String(stateDecoded.msg_id),
auth_error: '1',
message: ALLOWED_ERRORS.includes(message)
? message
: 'unauthorized',
message: clamped,
action: targetFlow,
});
if (stateDecoded?.opener_origin) {
params.set('opener_origin', String(stateDecoded.opener_origin));
}
return `${base}/?${params.toString()}`;
} else {
params = new URLSearchParams({
action: targetFlow,
auth_error: '1',
message: clamped,
});
}
if (requestCode) {
params.set('request_code', requestCode);
}
const params = new URLSearchParams({
action: targetFlow,
auth_error: '1',
message: ALLOWED_ERRORS.includes(message) ? message : 'unauthorized',
});
return `${base}/?${params.toString()}`;
}
@@ -73,6 +92,14 @@ function appendQueryParam(url: string, key: string, value: string): string {
return `${url}${sep}${encodeURIComponent(key)}=${encodeURIComponent(value)}`;
}
/** Length-safe constant-time string compare (never throws on mismatch). */
function constantTimeEqual(a: string, b: string): boolean {
const ba = Buffer.from(a);
const bb = Buffer.from(b);
if (ba.length !== bb.length) return false;
return crypto.timingSafeEqual(ba, bb);
}
/**
* True iff `target` parses as a URL whose origin equals `origin`. Used to
* clamp OIDC redirect targets — `startsWith` would accept
@@ -211,6 +238,17 @@ export class OIDCController extends PuterController {
statePayload.flow = 'revalidate';
}
// Bind this flow to the initiating browser: a single-use
// nonce lives both in the signed `state` and in an HttpOnly
// companion cookie. The callback requires them to match, so a
// `state` captured from an attacker's own flow can't be
// replayed in a victim's browser (login-CSRF / session
// fixation).
const browserNonce = crypto
.randomBytes(32)
.toString('base64url');
statePayload.nonce = browserNonce;
const state = this.services.oidc.signState(statePayload);
const url = await this.services.oidc.getAuthorizationUrl(
provider,
@@ -224,6 +262,15 @@ export class OIDCController extends PuterController {
{ legacyCode: 'internal_error' },
);
res.cookie(OIDC_NONCE_COOKIE_NAME, browserNonce, {
// Same flags as the session cookie: SameSite=None;Secure
// on HTTPS so the cookie survives Apple's cross-site
// form_post callback; Lax on plain-HTTP self-host.
...sessionCookieFlags(this.config),
httpOnly: true,
maxAge: OIDC_NONCE_EXPIRY_SEC * 1000,
path: '/',
});
res.redirect(302, url);
},
);
@@ -237,7 +284,7 @@ export class OIDCController extends PuterController {
const loginCb = async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
const result = await this.#processCallback(req, 'login');
const result = await this.#processCallback(req, res, 'login');
if ('error' in result) {
console.warn(`OIDC login callback error: ${result.error}`);
return res.redirect(
@@ -268,8 +315,9 @@ export class OIDCController extends PuterController {
origin,
'login',
'other',
resolved.error,
resolved.code ?? 'unauthorized',
stateDecoded,
resolved.requestCode,
),
);
}
@@ -285,7 +333,7 @@ export class OIDCController extends PuterController {
origin,
'login',
'other',
'This account is suspended.',
'account_suspended',
stateDecoded,
),
);
@@ -300,7 +348,7 @@ export class OIDCController extends PuterController {
const signupCb = async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
const result = await this.#processCallback(req, 'signup');
const result = await this.#processCallback(req, res, 'signup');
if ('error' in result) {
return res.redirect(
302,
@@ -321,14 +369,18 @@ export class OIDCController extends PuterController {
(stateDecoded.referrer as string) ?? null,
);
if ('error' in resolved) {
console.warn(
`OIDC signup user resolution error: ${resolved.error}`,
);
return res.redirect(
302,
buildErrorRedirectUrl(
origin,
'signup',
'other',
'unauthorized',
resolved.code ?? 'unauthorized',
stateDecoded,
resolved.requestCode,
),
);
}
@@ -365,7 +417,7 @@ export class OIDCController extends PuterController {
req: Request,
res: Response,
): Promise<void> => {
const result = await this.#processCallback(req, 'revalidate');
const result = await this.#processCallback(req, res, 'revalidate');
if ('error' in result) {
res.status(400).send(result.error);
return;
@@ -464,7 +516,7 @@ if (window.opener) {
userinfo: { sub: string; email?: unknown; [k: string]: unknown },
referrer?: string | null,
): Promise<
| { error: string }
| { error: string; code?: string; requestCode?: string }
| {
user: import('../../stores/user/UserStore.js').UserRow;
origin: 'linked-sub' | 'linked-email' | 'created';
@@ -510,13 +562,18 @@ if (window.opener) {
referrer,
);
if (!outcome.success || !outcome.user) {
return { error: outcome.error ?? 'Account creation failed.' };
return {
error: outcome.error ?? 'Account creation failed.',
code: outcome.code,
requestCode: outcome.requestCode,
};
}
return { user: outcome.user, origin: 'created' };
}
async #processCallback(
req: Request,
res: Response,
flow: string,
): Promise<
| { error: string }
@@ -540,6 +597,28 @@ if (window.opener) {
if (!stateDecoded || !stateDecoded.provider)
return { error: 'Invalid or expired state.' };
// Enforce the browser binding set at /start. Every state minted by
// the current /start carries a nonce, so this covers all live flows.
// States signed before this shipped have no nonce and pass through
// until they expire (STATE_EXPIRY, 10 min) so in-flight logins don't
// break on deploy — a caller can't forge a nonce-less state because
// /start always adds one and the state is server-signed.
const expectedNonce =
typeof stateDecoded.nonce === 'string' ? stateDecoded.nonce : '';
if (expectedNonce) {
const cookieNonce = req.cookies?.[OIDC_NONCE_COOKIE_NAME];
// Single-use: drop the cookie regardless of the outcome.
res.clearCookie(OIDC_NONCE_COOKIE_NAME, { path: '/' });
if (
typeof cookieNonce !== 'string' ||
!constantTimeEqual(cookieNonce, expectedNonce)
) {
return {
error: 'This sign-in could not be verified for your browser. Please start again.',
};
}
}
const provider = String(stateDecoded.provider);
const callbackUrl = this.services.oidc.getCallbackUrl(flow);
if (!callbackUrl) return { error: 'Invalid flow.' };
@@ -44,7 +44,9 @@ import {
} from 'vitest';
import type { Actor } from '../../core/actor.js';
import type { RouteOptions } from '../../core/http/index.js';
import type { ChatCompletionDriver } from '../../drivers/ai-chat/ChatCompletionDriver.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../../drivers/util/aiLimits.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { PuterAIController } from './PuterAIController.js';
@@ -188,11 +190,16 @@ describe('PuterAIController.registerRoutes', () => {
]),
);
// The four upstream-proxy routes reject third-party apps (they must
// call puter-chat-completion directly), but admit the user's own
// full-access personal access token: `requireUserActor` keeps apps out
// and `allowFullAccessToken` opens the gate to a full-access PAT. Each
// proxy route carries both flags.
// The four upstream-proxy routes accept exactly one credential
// shape — a full-access API token minted from the dashboard:
// `requireUserActor` keeps apps out, `allowFullAccessToken` admits
// the PAT, and `noUserSession` rejects the account session ("root")
// token. Each route also carries `requireVerified` (fresh accounts
// must confirm their email before using the AI wire surface) and
// the shared per-tier AI rate-limit / concurrency policy — these
// routes bypass the `/drivers/call` dispatch (where the
// driver-declared limits are enforced), so without the route gates
// they'd be unthrottled.
const userOnlyPaths = [
'/puterai/openai/v1/chat/completions',
'/puterai/openai/v1/completions',
@@ -205,6 +212,18 @@ describe('PuterAIController.registerRoutes', () => {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
noUserSession: true,
requireVerified: true,
rateLimit: {
...AI_RATE_LIMIT.default,
scope: 'driver:puter-chat-completion:complete',
key: expect.any(Function),
},
concurrent: {
...AI_CONCURRENT.default,
scope: 'driver:puter-chat-completion:complete',
key: expect.any(Function),
},
});
}
const modelsRoute = calls.find(
@@ -215,6 +234,46 @@ describe('PuterAIController.registerRoutes', () => {
requireAuth: false,
});
});
it('keys the proxy-route AI limits by user uuid so they share the /drivers/call buckets', () => {
const calls: Array<{ path: string; opts: RouteOptions }> = [];
const router = {
post: vi.fn((path: string, opts: RouteOptions) => {
calls.push({ path, opts });
return router;
}),
get: vi.fn(() => router),
};
controller.registerRoutes(router as never);
const route = calls.find(
(c) => c.path === '/puterai/openai/v1/chat/completions',
);
const rateLimit = route?.opts.rateLimit as {
key: (req: Request) => string;
scope: string;
};
const concurrent = route?.opts.concurrent as {
key: (req: Request) => string;
scope: string;
};
// The dispatch buckets requests as `driver:<iface>:<method>:<uid>`
// where uid is the actor's user uuid; scope + key must compose to the
// identical string or wire traffic mints a second per-user budget.
const req = makeReq({ actor: makeUserActor() });
expect(rateLimit.key(req)).toBe('u-7');
expect(concurrent.key(req)).toBe('u-7');
expect(rateLimit.scope).toBe('driver:puter-chat-completion:complete');
expect(concurrent.scope).toBe('driver:puter-chat-completion:complete');
// No-actor fallback still yields a usable (fingerprint) key rather
// than throwing or bucketing everyone together under undefined.
const anonKey = rateLimit.key(makeReq({}));
expect(typeof anonKey).toBe('string');
expect(anonKey.length).toBeGreaterThan(0);
});
});
// ── /openai/v1/chat/completions ─────────────────────────────────────
@@ -22,6 +22,7 @@ import crypto from 'node:crypto';
import { Readable } from 'node:stream';
import { HttpError } from '../../core/http/HttpError.js';
import { RouteOptions } from '../../core/http/index.js';
import { computeNetworkFingerprint } from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { ChatCompletionDriver } from '../../drivers/ai-chat/ChatCompletionDriver.js';
import type {
@@ -29,6 +30,7 @@ import type {
ICompleteArguments,
} from '../../drivers/ai-chat/types.js';
import { isDriverStreamResult } from '../../drivers/meta.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../../drivers/util/aiLimits.js';
import { PuterController } from '../types.js';
const GEMINI_DOWNLOAD_BASE =
@@ -44,20 +46,49 @@ const GEMINI_DOWNLOAD_BASE =
* ChatCompletionDriver, and translate the result (or NDJSON stream) back
* into the vendor's response / SSE shape.
*
* All routes live on `subdomain: 'api'` and reject app-under-user actors —
* only user actors may proxy.
* All routes live on `subdomain: 'api'` and require a full-access API
* token minted from the dashboard (user-scoped worker tokens also pass —
* workers are never treated as root tokens). Apps, scoped tokens, and
* account session ("root") tokens are rejected.
*/
export class PuterAIController extends PuterController {
registerRoutes(router: PuterRouter): void {
/**
* The wire routes call the chat driver directly instead of going
* through the `/drivers/call` dispatch, so the shared per-tier AI
* rate-limit / concurrency policy must be declared as route gates
* here. `scope` + `key` reproduce the dispatch's bucket key
* (`driver:<iface>:<method>:<uid>`) exactly, so wire traffic and
* `/drivers/call` traffic draw from one per-user budget rather than
* each surface minting its own.
*/
const aiPolicyScope = 'driver:puter-chat-completion:complete';
const aiPolicyKey = (req: Request): string =>
req.actor?.user?.uuid || computeNetworkFingerprint(req);
const apiAuthOpts = {
subdomain: 'api',
// The wire routes want a delegated credential — a full-access
// API token minted from the dashboard (or a user-scoped worker
// token, which is never treated as a root token).
// `requireUserActor` keeps apps out, `allowFullAccessToken`
// admits the PAT, and `noUserSession` rejects the account's
// session ("root") token — a copied session credential
// shouldn't double as an AI API key. `requireVerified` keeps
// fresh/unconfirmed accounts out, same as FS writes.
requireUserActor: true,
// These are the user's own programmatic AI surface (OpenAI/Anthropic
// wire). `requireUserActor` here exists to keep third-party apps
// out, not to block the user's own credential — so admit full-access
// personal access tokens (CLI/MCP/scripts). Apps and scoped tokens
// stay blocked.
allowFullAccessToken: true,
noUserSession: true,
requireVerified: true,
rateLimit: {
...AI_RATE_LIMIT.default!,
scope: aiPolicyScope,
key: aiPolicyKey,
},
concurrent: {
...AI_CONCURRENT.default!,
scope: aiPolicyScope,
key: aiPolicyKey,
},
} as RouteOptions;
const publicOpts = { subdomain: 'api', requireAuth: false } as const;
@@ -37,17 +37,33 @@ export class SystemController extends PuterController {
) {
// -- Healthcheck ---------------------------------------------
// Delegates to ServerHealthService for the real check-based
// status. Returns `{ ok: true }` when all registered checks pass,
// or `{ ok: false, failed: [...] }` + 503 when any fail or the
// status. Returns `{ ok: true }` + 200 when all registered checks
// pass, or `{ ok: false, failed: [...] }` + 503 when any fail or the
// server is draining.
router.get('/healthcheck', { subdomain: '*' }, async (_req, res) => {
//
// `?ignore=a,b` disregards the named checks for this request only.
// `?marked-degraded=a,b` demotes the named checks to a non-fatal
// `degraded` list: `ok` stays true but the response is 207 so the
// caller can tell the node is running in a degraded state.
const parseNames = (value) =>
typeof value === 'string'
? value
.split(',')
.map((name) => name.trim())
.filter(Boolean)
: [];
router.get('/healthcheck', { subdomain: '*' }, async (req, res) => {
const health = this.services.health;
if (!health || typeof health.getStatus !== 'function') {
// Fallback for boot ordering / missing service.
return res.send('ok');
}
const status = await health.getStatus();
const status = await health.getStatus({
ignore: parseNames(req.query.ignore),
degrade: parseNames(req.query['marked-degraded']),
});
if (!status.ok) return res.status(503).json(status);
if (status.degraded?.length) return res.status(207).json(status);
return res.json(status);
});
@@ -141,28 +157,27 @@ export class SystemController extends PuterController {
});
});
// -- GET /lsmod ----------------------------------------------
// Enumerates driver interfaces and their implementors.
// -- GET|POST /lsmod -----------------------------------------
// Enumerates driver interfaces and their implementors. POST is
// also routed because puter.js `drivers.list()` sends POST.
router.get(
'/lsmod',
{ subdomain: 'api', requireAuth: true },
(_req, res) => {
const interfaces = {};
for (const [key, driver] of Object.entries(this.drivers)) {
const ifaceName = driver?.driverInterface;
if (!ifaceName) continue;
const driverName = driver.driverName ?? key;
if (!interfaces[ifaceName]) {
interfaces[ifaceName] = { implementors: {} };
}
interfaces[ifaceName].implementors[driverName] = {
isDefault: Boolean(driver.isDefault),
};
const lsmod = (_req, res) => {
const interfaces = {};
for (const [key, driver] of Object.entries(this.drivers)) {
const ifaceName = driver?.driverInterface;
if (!ifaceName) continue;
const driverName = driver.driverName ?? key;
if (!interfaces[ifaceName]) {
interfaces[ifaceName] = { implementors: {} };
}
res.json({ interfaces });
},
);
interfaces[ifaceName].implementors[driverName] = {
isDefault: Boolean(driver.isDefault),
};
}
res.json({ interfaces });
};
router.get('/lsmod', { subdomain: 'api', requireAuth: true }, lsmod);
router.post('/lsmod', { subdomain: 'api', requireAuth: true }, lsmod);
}
onServerStart() {}
@@ -18,9 +18,10 @@
*/
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { kv } from '../../util/kvSingleton.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -79,10 +80,11 @@ interface CapturedResponse {
const makeReq = (init: {
body?: unknown;
actor?: Actor;
query?: Record<string, unknown>;
}): Request => {
return {
body: init.body ?? {},
query: {},
query: init.query ?? {},
headers: {},
actor: init.actor,
} as unknown as Request;
@@ -139,6 +141,191 @@ describe('SystemController GET /healthcheck', () => {
expect(captured.body).toMatchObject({ ok: true });
expect(captured.statusCode).toBe(200);
});
it('parses ?ignore and ?marked-degraded into trimmed name lists', async () => {
const spy = vi
.spyOn(server.services.health, 'getStatus')
.mockResolvedValue({ ok: true });
try {
const { res } = makeRes();
await callRoute(
'get',
'/healthcheck',
makeReq({
query: {
ignore: 'database-liveness, thumbnailer',
'marked-degraded': ' socket-initialized ',
},
}),
res,
);
expect(spy).toHaveBeenCalledWith({
ignore: ['database-liveness', 'thumbnailer'],
degrade: ['socket-initialized'],
});
} finally {
spy.mockRestore();
}
});
it('returns ok:true + 200 when the only failures are ignored', async () => {
const spy = vi
.spyOn(server.services.health, 'getStatus')
.mockImplementation(async ({ ignore = [] } = {}) => {
const failed = ['database-liveness'].filter(
(name) => !ignore.includes(name),
);
return failed.length === 0
? { ok: true }
: { ok: false, failed };
});
try {
const { res, captured } = makeRes();
await callRoute(
'get',
'/healthcheck',
makeReq({ query: { ignore: 'database-liveness' } }),
res,
);
expect(captured.body).toEqual({ ok: true });
expect(captured.statusCode).toBe(200);
} finally {
spy.mockRestore();
}
});
it('returns ok:true + 207 when the only failures are marked degraded', async () => {
const spy = vi
.spyOn(server.services.health, 'getStatus')
.mockResolvedValue({ ok: true, degraded: ['database-liveness'] });
try {
const { res, captured } = makeRes();
await callRoute(
'get',
'/healthcheck',
makeReq({ query: { 'marked-degraded': 'database-liveness' } }),
res,
);
expect(captured.body).toEqual({
ok: true,
degraded: ['database-liveness'],
});
expect(captured.statusCode).toBe(207);
} finally {
spy.mockRestore();
}
});
it('still 503s when a non-ignored failure remains', async () => {
const spy = vi
.spyOn(server.services.health, 'getStatus')
.mockImplementation(async ({ ignore = [] } = {}) => {
const failed = ['database-liveness', 'socket-initialized'].filter(
(name) => !ignore.includes(name),
);
return failed.length === 0
? { ok: true }
: { ok: false, failed };
});
try {
const { res, captured } = makeRes();
await callRoute(
'get',
'/healthcheck',
makeReq({ query: { ignore: 'database-liveness' } }),
res,
);
expect(captured.statusCode).toBe(503);
expect(captured.body).toEqual({
ok: false,
failed: ['socket-initialized'],
});
} finally {
spy.mockRestore();
}
});
});
// ── ServerHealthService.getStatus ignore / degrade filtering ────────
//
// Exercises the real service by seeding the in-process status cache
// (the kv.js singleton) it reads from, so the actual per-request
// classification runs — not a stubbed getStatus.
describe('ServerHealthService.getStatus ignore/degrade filtering', () => {
const STATUS_CACHE_KEY = 'server-health:status';
const seedStatus = (status: unknown) => {
kv.set(STATUS_CACHE_KEY, status, { EX: 5 });
};
afterEach(() => {
kv.del(STATUS_CACHE_KEY);
});
it('collapses to ok:true when every failure is ignored', async () => {
seedStatus({ ok: false, failed: ['database-liveness', 'thumbnailer'] });
const status = await server.services.health.getStatus({
ignore: ['database-liveness', 'thumbnailer'],
});
expect(status).toEqual({ ok: true });
});
it('keeps the non-ignored failures', async () => {
seedStatus({ ok: false, failed: ['database-liveness', 'thumbnailer'] });
const status = await server.services.health.getStatus({
ignore: ['database-liveness'],
});
expect(status).toEqual({ ok: false, failed: ['thumbnailer'] });
});
it('is a no-op for a healthy status', async () => {
seedStatus({ ok: true });
const status = await server.services.health.getStatus({
ignore: ['database-liveness'],
});
expect(status).toEqual({ ok: true });
});
it('ignores unknown names without affecting real failures', async () => {
seedStatus({ ok: false, failed: ['database-liveness'] });
const status = await server.services.health.getStatus({
ignore: ['not-a-check'],
});
expect(status).toEqual({ ok: false, failed: ['database-liveness'] });
});
it('demotes marked failures to degraded and stays ok:true', async () => {
seedStatus({ ok: false, failed: ['database-liveness'] });
const status = await server.services.health.getStatus({
degrade: ['database-liveness'],
});
expect(status).toEqual({ ok: true, degraded: ['database-liveness'] });
});
it('reports degraded alongside remaining hard failures (ok:false)', async () => {
seedStatus({
ok: false,
failed: ['database-liveness', 'socket-initialized'],
});
const status = await server.services.health.getStatus({
degrade: ['database-liveness'],
});
expect(status).toEqual({
ok: false,
failed: ['socket-initialized'],
degraded: ['database-liveness'],
});
});
it('lets ignore take precedence over degrade for the same name', async () => {
seedStatus({ ok: false, failed: ['database-liveness'] });
const status = await server.services.health.getStatus({
ignore: ['database-liveness'],
degrade: ['database-liveness'],
});
expect(status).toEqual({ ok: true });
});
});
// ── /version ────────────────────────────────────────────────────────
@@ -335,6 +335,63 @@ describe('WebDAVController', () => {
});
});
describe('suspension gate', () => {
// WebDAV must also enforce the suspension gate every other authenticated
// route gets from requireAuthGate. Same single-use() dispatch means that
// middleware is never wired in, so #dispatch calls assertNotSuspended
// itself. Without it, a suspended account could read/write/delete its
// whole filesystem over the `dav` subdomain.
it('rejects a suspended session actor with 403', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'PROPFIND',
actor: {
user: {
id: 1,
uuid: 'suspended-uuid',
username: 'suspended',
suspended: true,
},
},
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
it('rejects a suspended user on the Basic-auth path with 403', async () => {
const username = `webdav-suspended-${Math.random()
.toString(36)
.slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: await bcryptHash('correct-horse', 4),
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await server.stores.user.update(created.id, {
suspended: 1,
});
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'PROPFIND',
headers: {
authorization: basicAuth(username, 'correct-horse'),
},
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
});
describe('unsupported methods', () => {
it('returns 405 for unknown HTTP methods', async () => {
const { res, captured } = makeRes();
@@ -23,7 +23,10 @@ import { posix as pathPosix } from 'node:path';
import { EventMap } from '../../clients/event/types.js';
import type { Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { assertVerifiedAccount } from '../../core/http/middleware/gates.js';
import {
assertNotSuspended,
assertVerifiedAccount,
} from '../../core/http/middleware/gates.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
@@ -90,14 +93,15 @@ export class WebDAVController extends PuterController {
const actor = await this.#resolveActor(req, res);
if (!actor) return; // 401 already sent
// Apply the same pending-verification gate every other authenticated
// route gets from `requireVerifiedAccount`. WebDAV dispatches every
// method off a single `router.use` with no route options, so that
// middleware is never inserted into its chain — without this call an
// account still pending email / phone / card verification could read,
// write, and delete its entire filesystem over the `dav` subdomain,
// bypassing the gate. Throws a 403 HttpError, surfaced by the catch in
// registerRoutes.
// Apply the same suspension + pending-verification gates every other
// authenticated route gets from `requireAuthGate` / `requireVerifiedAccount`.
// WebDAV dispatches every method off a single `router.use` with no route
// options, so that middleware is never inserted into its chain — without
// these calls a suspended account (or one still pending email / phone /
// card verification) could read, write, and delete its entire filesystem
// over the `dav` subdomain, bypassing the gates. Both throw a 403
// HttpError, surfaced by the catch in registerRoutes.
assertNotSuspended(actor.user);
assertVerifiedAccount(actor.user);
// Expand `~`/`~/...` against the authenticated actor's username.
+4 -2
View File
@@ -54,9 +54,11 @@ export interface Actor {
* Session reference when authenticated via a session token (user actors)
* or an app-under-user token that carries a session. Absent for system,
* raw-app, and pure access-token actors. Used for session introspection
* and targeted logout.
* and targeted logout. `kind` mirrors the session row's kind (e.g.
* 'web', 'app', 'worker') so callers can gate on how the credential
* was minted without an extra session lookup.
*/
session?: { uid: string } | null;
session?: { uid: string; kind?: string | null } | null;
}
/** UUID of the baked-in system user (see 0025 seed migration). */
+2 -1
View File
@@ -53,7 +53,8 @@ export type LegacyErrorCodes =
| 'password_required'
| 'password_mismatch'
| 'field_not_allowed_for_create'
| 'account_is_not_verified';
| 'account_is_not_verified'
| 'app_or_api_token_required';
/**
* Copyright (C) 2024-present Puter Technologies Inc.
+10
View File
@@ -46,6 +46,16 @@ export {
subdomainGate,
} from './middleware/gates';
export { createNotFoundHandler } from './middleware/notFoundHandler';
export {
createStepUpGate,
signStepUpToken,
STEP_UP_COOKIE_NAME,
STEP_UP_PURPOSE,
STEP_UP_SCOPE,
STEP_UP_TTL_SECONDS,
stepUpCookieOptions,
verifyStepUpSession,
} from './middleware/stepUpSession';
export { PuterRouter } from './PuterRouter';
export {
PREFIX_METADATA_KEY,
@@ -232,3 +232,50 @@ describe('createErrorHandler — when the response has already started streaming
expect(onError).toHaveBeenCalledTimes(1);
});
});
// ── Database load-shed translation ──────────────────────────────────
describe('createErrorHandler — dbBatchFailed load-shed errors', () => {
const makeDbBatchError = (reason: string) => {
const err = new Error('Database operation failed') as Error & {
code: string;
reason: string;
};
err.code = 'dbBatchFailed';
err.reason = reason;
return err;
};
it('maps to 503 + Retry-After instead of a generic 500', () => {
const onUnhandled = vi.fn();
const handler = createErrorHandler({ onUnhandled });
const { out } = runHandler(handler, makeDbBatchError('breakerOpen'));
expect(out.statusCode).toBe(503);
expect(out.headers['Retry-After']).toBe(5);
expect(out.body).toMatchObject({
code: 'db_unavailable',
error: 'Service temporarily unavailable',
});
// Translated errors are expected degradation, not unhandled bugs.
expect(onUnhandled).not.toHaveBeenCalled();
});
it('translates every load-shed reason the batcher emits', () => {
const handler = createErrorHandler({ onUnhandled: () => {} });
for (const reason of ['breakerOpen', 'queueOverflow', 'connAcquire']) {
const { out } = runHandler(handler, makeDbBatchError(reason));
expect(out.statusCode).toBe(503);
}
});
it('leaves unrelated coded errors on the generic 500 path', () => {
const onUnhandled = vi.fn();
const handler = createErrorHandler({ onUnhandled });
const err = new Error('boom') as Error & { code: string };
err.code = 'somethingElse';
const { out } = runHandler(handler, err);
expect(out.statusCode).toBe(500);
expect(onUnhandled).toHaveBeenCalledTimes(1);
});
});
@@ -85,6 +85,17 @@ export const createErrorHandler = (
err = translated;
}
// Database-batcher load-shed (circuit open, queue overflow, or no
// connection available): the persistence layer is temporarily
// degraded, not a programming bug. Surface as 503 so clients back
// off and retry instead of treating it as a hard failure.
if ((err as { code?: string } | null)?.code === 'dbBatchFailed') {
res.setHeader('Retry-After', 5);
err = new HttpError(503, 'Service temporarily unavailable', {
code: 'db_unavailable',
});
}
if (isHttpError(err)) {
opts.onError?.(err, req);
if (err.statusCode === 402 || err.statusCode === 413) {
@@ -55,13 +55,15 @@ function readDeviceFingerprint(req: {
* Stamp request-scoped fingerprints onto `req` so any downstream gate, handler,
* or service (via the ALS `Context.get('req')`) can read them without recomputing:
*
* - `req.networkFingerprint` — always set; a coarse IP+headers hash (the same
* value the rate limiter keys on). Server-derived, so it can't be forged
* away, but it's coarse (shared behind NAT/VPN, rotates with UA).
* - `req.networkFingerprint` — always set; a coarse IP+headers hash (the
* anchor of the rate limiter's default key). Server-derived, so it can't be
* forged away, but it's coarse (shared behind NAT/VPN, rotates with UA).
* - `req.deviceFingerprint` — set only when the client supplied a well-shaped
* device fingerprint (ThumbmarkJS hash) in the body or the
* `x-puter-device-fingerprint` header; `undefined` otherwise. Client-supplied
* and spoofable, but stable per real device across IP rotation.
* and spoofable, but stable per real device across IP rotation. The rate
* limiter's 'fingerprint' strategy appends it to the network hash so each
* device behind a shared network gets its own bucket.
*
* Install AFTER the body parsers (so the body fingerprint is readable) and
* before `requestContext` (so the snapshot into ALS already carries them).
@@ -24,6 +24,8 @@ import {
DEFAULT_ADMIN_USERNAMES,
adminOnlyGate,
allowedAppIdsGate,
assertNotUserSession,
noUserSessionGate,
requireAuthGate,
requireVerifiedAccount,
requireNonAccessTokenGate,
@@ -331,6 +333,103 @@ describe('requireNonAccessTokenGate', () => {
});
});
// ── noUserSessionGate ───────────────────────────────────────────────
describe('noUserSessionGate', () => {
it('rejects a bare user-session ("root" token) actor with 403', () => {
const got = runGate(noUserSessionGate(), {
actor: { user: { uuid: 'u-1' } },
});
expectHttpError(got, 403, 'app_or_api_token_required');
});
it('passes through for app-under-user actors (app/worker tokens)', () => {
const got = runGate(noUserSessionGate(), {
actor: { user: { uuid: 'u-1' }, app: { uid: 'app-1' } },
});
expect(got).toBeUndefined();
});
it('passes through for user-scoped worker sessions (kind="worker")', () => {
const got = runGate(noUserSessionGate(), {
actor: {
user: { uuid: 'u-1' },
session: { uid: 'sess-1', kind: 'worker' },
},
});
expect(got).toBeUndefined();
});
it('passes through for access-token actors (which access tokens are OK is decided by the other gates)', () => {
const got = runGate(noUserSessionGate(), {
actor: {
user: { uuid: 'u-1' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1' } },
fullAccess: true,
},
},
});
expect(got).toBeUndefined();
});
it("falls back to 401 if there's no actor at all (defensive)", () => {
const got = runGate(noUserSessionGate(), {});
expectHttpError(got, 401, 'token_missing');
});
});
describe('assertNotUserSession', () => {
it('is a no-op for a missing actor (anonymous is the auth gate’s job)', () => {
expect(() => assertNotUserSession(undefined)).not.toThrow();
expect(() => assertNotUserSession(null)).not.toThrow();
});
it('always admits a user-scoped worker session — workers are never root tokens', () => {
// Workers deployed with no app binding hold a session-TYPE token
// whose row is kind='worker' (see AuthService.createWorkerSessionToken)
// — a bare user actor plus that session ref. The gate is an
// annoyance for sign-up-and-scrape abuse; a deployed worker is
// already a delegated, revocable credential.
expect(() =>
assertNotUserSession({
app: null,
accessToken: null,
session: { uid: 'sess-1', kind: 'worker' },
}),
).not.toThrow();
});
it('still rejects web sessions', () => {
expect(() =>
assertNotUserSession({
app: null,
accessToken: null,
session: { uid: 'sess-1', kind: 'web' },
}),
).toThrow();
});
it('throws 403 app_or_api_token_required for a bare session actor', () => {
try {
assertNotUserSession({ app: null, accessToken: null });
expect.unreachable('expected assertNotUserSession to throw');
} catch (err) {
expect(isHttpError(err)).toBe(true);
expect((err as HttpError).statusCode).toBe(403);
expect((err as HttpError).legacyCode).toBe(
'app_or_api_token_required',
);
// The user asked for a helpful message: it must point at the
// credentials that DO work and where to get one.
expect((err as HttpError).message).toMatch(/app or worker token/i);
expect((err as HttpError).message).toMatch(/API token/);
expect((err as HttpError).message).toMatch(/dashboard/i);
}
});
});
// ── adminOnlyGate ───────────────────────────────────────────────────
describe('adminOnlyGate', () => {
@@ -407,6 +506,113 @@ describe('adminOnlyGate', () => {
}),
).toBeUndefined();
});
// -- Root-token requirement --
//
// Admin endpoints require a root token (an actor with no app anywhere
// in its token chain), so a third-party app an admin authorized can't
// reach them on the admin's behalf.
it('admits an admin acting via a session (root token)', () => {
const got = runGate(adminOnlyGate(), {
actor: { user: { uuid: 'u-1', username: 'admin' } },
});
expect(got).toBeUndefined();
});
it("admits an admin's full-access PAT (still a root token — no app)", () => {
const got = runGate(adminOnlyGate(), {
actor: {
user: { uuid: 'u-1', username: 'admin' },
accessToken: {
uid: 'tok-1',
issuer: { user: { uuid: 'u-1', username: 'admin' } },
fullAccess: true,
},
},
});
expect(got).toBeUndefined();
});
it('rejects an admin acting through an app with 403 (not a root token)', () => {
const got = runGate(adminOnlyGate(), {
actor: {
user: { uuid: 'u-1', username: 'admin' },
app: { uid: 'app-1' },
},
});
expectHttpError(got, 403, 'forbidden');
});
it('rejects an admin access token issued through an app (app in the token chain)', () => {
// Access-token actors carry their app on `accessToken.issuer.app`,
// not top-level `actor.app` — the root-token check must walk the
// chain, not just the top level.
const got = runGate(adminOnlyGate(), {
actor: {
user: { uuid: 'u-1', username: 'admin' },
accessToken: {
uid: 'tok-1',
issuer: {
user: { uuid: 'u-1', username: 'admin' },
app: { uid: 'app-1' },
},
},
},
});
expectHttpError(got, 403, 'forbidden');
});
it('rejects an app-issued access token even when appGated', () => {
// The appGated deferral only applies to direct app-under-user
// actors: `allowedAppIdsGate` reads top-level `actor.app` and would
// pass a chain-only app straight through, so it must not be
// deferred to.
const got = runGate(adminOnlyGate([], { appGated: true }), {
actor: {
user: { uuid: 'u-1', username: 'admin' },
accessToken: {
uid: 'tok-1',
issuer: {
user: { uuid: 'u-1', username: 'admin' },
app: { uid: 'app-1' },
},
},
},
});
expectHttpError(got, 403, 'forbidden');
});
it('admits an admin acting through an app when appGated (allowedAppIdsGate then decides)', () => {
// On an appId-gated route the root-token check is deferred to
// `allowedAppIdsGate`; this gate must let the app actor through.
const got = runGate(adminOnlyGate([], { appGated: true }), {
actor: {
user: { uuid: 'u-1', username: 'admin' },
app: { uid: 'app-1' },
},
});
expect(got).toBeUndefined();
});
it('still admits a root token when appGated', () => {
const got = runGate(adminOnlyGate([], { appGated: true }), {
actor: { user: { uuid: 'u-1', username: 'admin' } },
});
expect(got).toBeUndefined();
});
it('applies the username check before the root-token check', () => {
// A non-admin acting through an app is rejected for being non-admin,
// regardless of the app scope.
const got = runGate(adminOnlyGate(), {
actor: {
user: { uuid: 'u-1', username: 'random-user' },
app: { uid: 'app-1' },
},
});
expectHttpError(got, 403, 'forbidden');
});
});
// ── requireVerifiedGate ─────────────────────────────────────────────
+98 -10
View File
@@ -18,6 +18,8 @@
*/
import type { Request, RequestHandler } from 'express';
import { effectiveActorApp } from '../../actor';
import type { Actor } from '../../actor';
import { HttpError } from '../HttpError';
import { assertVerifiedEmail } from '../verifiedEmail';
@@ -93,12 +95,10 @@ export const requireAuthGate = (): RequestHandler => {
next(rejectAuth(req));
return;
}
if (req.actor.user.suspended) {
next(
new HttpError(403, 'Account suspended', {
legacyCode: 'forbidden',
}),
);
try {
assertNotSuspended(req.actor.user);
} catch (err) {
next(err);
return;
}
next();
@@ -149,6 +149,59 @@ export const requireUserActorGate = (
};
};
/**
* Reject bare user-session actors — the "root" credential a browser session
* (or `/login`) holds, with no app and no access token in play. Use on API
* surfaces that must only be driven by a delegated credential: an app or
* worker token, or an API token minted from the dashboard. The point is that
* a leaked-or-copied session token (full account control) shouldn't double
* as an AI/API credential; users are pushed to mint a revocable token
* instead.
*
* This gate only rejects the bare-session shape. Which delegated credentials
* are acceptable is decided by the gates it composes with (`requireUserActor`
* + `allowFullAccessToken` to also keep apps out, `requireNonAccessTokenGate`
* for scoped tokens, etc.).
*/
export const assertNotUserSession = (
actor: Pick<Actor, 'app' | 'accessToken' | 'session'> | null | undefined,
): void => {
if (!actor) return; // anonymous requests are the auth gate's problem
if (actor.app || actor.accessToken) return;
// User-scoped workers (deployed with no app binding) authenticate with
// a session-TYPE token whose session row is `kind='worker'` — a managed,
// revocable deployment credential, not a browser sign-in. Workers are
// never treated as root tokens: this gate is an annoyance for
// sign-up-and-scrape abuse, and someone who deploys a worker to reach
// an API has already left that path.
if (actor.session?.kind === 'worker') return;
throw new HttpError(
403,
'This API cannot be called with an account session token. ' +
'Use an app or worker token, or create an API token from the ' +
'dashboard (Account → API Token).',
{ legacyCode: 'app_or_api_token_required' },
);
};
/** Route-option form of {@link assertNotUserSession} (`noUserSession: true`). */
export const noUserSessionGate = (): RequestHandler => {
return (req, _res, next) => {
const actor = req.actor;
if (!actor) {
next(rejectAuth(req));
return;
}
try {
assertNotUserSession(actor);
} catch (err) {
next(err);
return;
}
next();
};
};
export const requireNonAccessTokenGate = (): RequestHandler => {
return (req, _res, next) => {
const actor = req.actor;
@@ -184,13 +237,24 @@ export const DEFAULT_ADMIN_USERNAMES = ['admin', 'system'] as const;
* the supplied extras. Extras are *additional* allowed users on top of the
* built-in pair, not a replacement for it.
*
* Implies `requireAuth`. Does *not* imply `requireUserActor` — admin
* endpoints are callable via an admin's access token or app-under-user
* actor; combine with `requireUserActor` explicitly if a route must be
* restricted to browser sessions.
* Also requires a *root token* — an actor with no app anywhere in its token
* chain (see `effectiveActorApp`) — so a third-party app an admin has
* authorized can't reach admin endpoints on the admin's behalf. The one
* exception is `appGated`: on a route that is also appId-gated
* (`allowedAppIds`), a direct app-under-user actor is deferred to
* `allowedAppIdsGate`, so the net effect there is "a root token OR a token
* scoped to an allowed app". Access tokens issued through an app are
* rejected even then — `allowedAppIdsGate` only sees top-level `actor.app`
* and would otherwise wave them through.
*
* Implies `requireAuth`. Does *not* imply `requireUserActor` — a root token
* still includes an admin's full-access personal access token, not only
* browser sessions; combine with `requireUserActor` explicitly if a route
* must be restricted to browser sessions.
*/
export const adminOnlyGate = (
extras: readonly string[] = [],
opts: { appGated?: boolean } = {},
): RequestHandler => {
// Match the case-insensitivity guarantee of the username column
// (MySQL: ascii_general_ci; SQLite: idx_user_username_nocase). Comparing
@@ -209,6 +273,20 @@ export const adminOnlyGate = (
);
return;
}
// Root-token requirement: reject actors carrying an app anywhere in
// their token chain — app-under-user, or an access token issued
// through an app. A direct app-under-user actor is deferred to
// `allowedAppIdsGate` when the route is appId-gated; chain-only apps
// are rejected even then, since that gate can't see them.
const chainApp = req.actor ? effectiveActorApp(req.actor) : null;
if (chainApp && !(opts.appGated && req.actor?.app?.uid)) {
next(
new HttpError(403, 'Only admins may request this resource', {
legacyCode: 'forbidden',
}),
);
return;
}
next();
};
};
@@ -312,6 +390,16 @@ export const assertVerifiedAccount = (
}
};
export const assertNotSuspended = (
user: { suspended?: unknown } | undefined,
): void => {
if (user?.suspended) {
throw new HttpError(403, 'Account suspended', {
legacyCode: 'forbidden',
});
}
};
/**
* Reject unless the actor is acting through one of the named apps.
* App-under-user actors are permitted iff `actor.app.uid` is in the allowList;
@@ -0,0 +1,150 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { RequestHandler } from 'express';
import { Readable } from 'node:stream';
import type { puterClients } from '../../../clients';
import type { puterServices } from '../../../services';
import type { puterStores } from '../../../stores';
import type { IConfig, LayerInstances } from '../../../types';
interface Layers {
clients: LayerInstances<typeof puterClients>;
stores: LayerInstances<typeof puterStores>;
services: LayerInstances<typeof puterServices>;
}
// Local analogue of the production `<name>.puter.work` worker domain. Requests
// to `<name>.workers.puter.localhost` are dispatched into a Miniflare instance
// by `LocalWorkerService`, which mirrors the real Cloudflare dispatch path.
const WORKER_HOST_SUFFIX = 'workers.puter.localhost';
// Minimal WHATWG-Response shape we consume from Miniflare's `dispatchFetch`.
// It isn't the Node global `Response`, so we type it structurally rather than
// importing Miniflare's classes into the HTTP layer.
interface FetchResponse {
status: number;
headers: { forEach(cb: (value: string, key: string) => void): void };
body: ReadableStream<Uint8Array> | null;
}
function normalizeHost(value: string | undefined | null): string | null {
if (typeof value !== 'string') return null;
const trimmed = value.trim().toLowerCase().replace(/^\./, '');
if (!trimmed) return null;
return trimmed.split(':')[0] || null;
}
// `<name>.workers.puter.localhost` → `name`. Returns null for the bare zone or
// any host outside it. Flip this if your dev DNS puts the name elsewhere.
function workerNameFromHost(host: string): string | null {
if (host === WORKER_HOST_SUFFIX) return null;
if (!host.endsWith(`.${WORKER_HOST_SUFFIX}`)) return null;
const prefix = host.slice(0, host.length - WORKER_HOST_SUFFIX.length - 1);
return prefix.split('.')[0] || null;
}
// Express (Node) request → WHATWG Request the Worker's `fetch(request)` sees.
// Must run BEFORE any body-parsing middleware so `req` is still an unconsumed
// stream; otherwise the Worker gets an empty body on POST/PUT.
function toFetchRequest(req: Parameters<RequestHandler>[0]): Request {
const url = `http://${req.headers.host ?? WORKER_HOST_SUFFIX}${req.originalUrl}`;
const headers = new Headers();
for (const [key, value] of Object.entries(req.headers)) {
if (Array.isArray(value)) {
for (const v of value) headers.append(key, v);
} else if (value != null) {
headers.set(key, value);
}
}
const method = (req.method ?? 'GET').toUpperCase();
const hasBody = method !== 'GET' && method !== 'HEAD';
return new Request(url, {
method,
headers,
// `duplex: 'half'` is required by undici whenever a stream body is set.
body: hasBody ? (Readable.toWeb(req) as ReadableStream) : undefined,
...(hasBody ? { duplex: 'half' } : {}),
} as RequestInit);
}
// WHATWG Response from the Worker → Express response.
function sendFetchResponse(
res: Parameters<RequestHandler>[1],
response: FetchResponse,
): void {
res.status(response.status);
response.headers.forEach((value, key) => {
// Node manages framing headers itself; forwarding them corrupts the
// response (double content-length, stale transfer-encoding).
const lower = key.toLowerCase();
if (lower === 'content-length' || lower === 'transfer-encoding') return;
res.setHeader(key, value);
});
if (!response.body) {
res.end();
return;
}
const nodeStream = Readable.fromWeb(response.body as never);
nodeStream.on('error', () => res.destroy());
nodeStream.pipe(res);
}
/**
* Serves local Workers on `*.workers.puter.localhost` by dispatching into
* Miniflare via `LocalWorkerService`. No-op unless `config.workers.localServer`
* is set — production keeps hitting real Cloudflare through `WorkerDriver`.
*
* Mount this BEFORE the body-parsing middleware in `server.ts` so the Worker
* receives the raw request stream.
*/
export const createLocalWorkerProxyMiddleware = (
config: IConfig,
layers: Layers,
): RequestHandler => {
if (!config.workers?.localServer) {
return (_req, _res, next) => next();
}
const localWorkerService = layers.services.localworkerservice;
return async (req, res, next) => {
const host = normalizeHost(req.hostname);
if (!host) return next();
const workerName = workerNameFromHost(host);
if (!workerName) return next();
try {
const fetchRequest = toFetchRequest(req);
const response = (await localWorkerService.cfCallLocal(
workerName,
fetchRequest,
)) as unknown as FetchResponse;
sendFetchResponse(res, response);
} catch (err) {
next(err);
}
};
};
@@ -22,25 +22,102 @@ import { describe, expect, it, vi } from 'vitest';
import { isHttpError } from '../HttpError';
import { createNotFoundHandler } from './notFoundHandler';
const makeReq = (over: Partial<Request> = {}): Request =>
({
method: 'GET',
hostname: 'puter.example',
path: '/no-such-page',
...over,
}) as Request;
const makeRes = () => ({
status: vi.fn(),
json: vi.fn(),
redirect: vi.fn(),
});
const expect404 = (next: ReturnType<typeof vi.fn>) => {
expect(next).toHaveBeenCalledTimes(1);
const err = next.mock.calls[0][0];
expect(isHttpError(err)).toBe(true);
expect(err.statusCode).toBe(404);
expect(err.legacyCode).toBe('not_found');
};
describe('createNotFoundHandler', () => {
it("forwards an HttpError(404, 'not_found') to next() — does not write the response itself", () => {
// The handler must NOT call res.json/status — that's the error
// handler's job, so every failure goes through the same serializer.
const handler = createNotFoundHandler();
const next = vi.fn();
const res = {
status: vi.fn(),
json: vi.fn(),
} as unknown as Response;
handler({} as Request, res, next);
const res = makeRes();
handler(makeReq(), res as unknown as Response, next);
expect(next).toHaveBeenCalledTimes(1);
const err = next.mock.calls[0][0];
expect(isHttpError(err)).toBe(true);
expect(err.statusCode).toBe(404);
expect(err.legacyCode).toBe('not_found');
expect404(next);
// Never wrote a response directly.
expect((res.status as ReturnType<typeof vi.fn>)).not.toHaveBeenCalled();
expect((res.json as ReturnType<typeof vi.fn>)).not.toHaveBeenCalled();
expect(res.status).not.toHaveBeenCalled();
expect(res.json).not.toHaveBeenCalled();
expect(res.redirect).not.toHaveBeenCalled();
});
describe('with guiDomain set', () => {
const handler = createNotFoundHandler({ guiDomain: 'puter.example' });
it('redirects an unmatched GET on the GUI domain to /', () => {
const next = vi.fn();
const res = makeRes();
handler(makeReq(), res as unknown as Response, next);
expect(res.redirect).toHaveBeenCalledWith('/');
expect(next).not.toHaveBeenCalled();
});
it('redirects HEAD like GET', () => {
const next = vi.fn();
const res = makeRes();
handler(
makeReq({ method: 'HEAD' }),
res as unknown as Response,
next,
);
expect(res.redirect).toHaveBeenCalledWith('/');
expect(next).not.toHaveBeenCalled();
});
it('still 404s non-GET methods on the GUI domain', () => {
const next = vi.fn();
const res = makeRes();
handler(
makeReq({ method: 'POST' }),
res as unknown as Response,
next,
);
expect404(next);
expect(res.redirect).not.toHaveBeenCalled();
});
it('still 404s on subdomains (api., etc.)', () => {
const next = vi.fn();
const res = makeRes();
handler(
makeReq({ hostname: 'api.puter.example' }),
res as unknown as Response,
next,
);
expect404(next);
expect(res.redirect).not.toHaveBeenCalled();
});
it('never redirects / to itself', () => {
const next = vi.fn();
const res = makeRes();
handler(makeReq({ path: '/' }), res as unknown as Response, next);
expect404(next);
expect(res.redirect).not.toHaveBeenCalled();
});
});
});
@@ -20,6 +20,16 @@
import type { RequestHandler } from 'express';
import { HttpError } from '../HttpError';
export interface NotFoundHandlerOptions {
/**
* The bare GUI domain (`config.domain`). When set, unmatched GET/HEAD
* requests whose host is exactly this domain redirect to `/` instead of
* 404ing, so a typo'd or stale URL lands back on the desktop. Subdomains
* (api., etc.) and custom domains are unaffected and still 404.
*/
guiDomain?: string;
}
/**
* Catch-all 404 middleware. Install last (just before the error handler);
* any request that didn't match a route lands here.
@@ -28,8 +38,22 @@ import { HttpError } from '../HttpError';
* the same error-handler pipeline serializes the body — keeps the wire shape
* consistent with every other failure (`{ error: '...', code: 'not_found' }`).
*/
export const createNotFoundHandler = (): RequestHandler => {
return (_req, _res, next): void => {
export const createNotFoundHandler = (
opts: NotFoundHandlerOptions = {},
): RequestHandler => {
const guiDomain = opts.guiDomain?.trim().toLowerCase() || null;
return (req, res, next): void => {
if (
guiDomain &&
(req.method === 'GET' || req.method === 'HEAD') &&
req.hostname?.toLowerCase() === guiDomain &&
// '/' always matches the shell route; the guard just makes a
// misconfigured deployment 404 instead of redirect-looping.
req.path !== '/'
) {
res.redirect('/');
return;
}
next(new HttpError(404, 'Not Found', { legacyCode: 'not_found' }));
};
};
@@ -331,6 +331,27 @@ describe('buildPrivateHostRedirect', () => {
expect(url).toBe('http://beans.app.puter.localhost:4100/');
});
it('collapses a scheme-relative path so it cannot escape the host (open redirect)', () => {
for (const evil of [
'//evil.com/',
'///evil.com/',
'/\\evil.com/',
'/\\/evil.com/',
]) {
const url = buildPrivateHostRedirect(
reqOf({
hostname: 'beans.site.puter.localhost',
originalUrl: evil,
}),
{ name: 'beans' },
cfg,
);
expect(url).toBe(
'http://beans.app.puter.localhost:4100/evil.com/',
);
}
});
it('returns null when no private hosting domain is configured', () => {
const noPrivate = {
...cfg,
@@ -400,6 +421,26 @@ describe('buildPublicHostRedirect', () => {
expect(url).toBe('http://beans.site.puter.localhost:4100/');
});
it('collapses a scheme-relative path so it cannot escape the host (open redirect)', () => {
for (const evil of [
'//evil.com/',
'///evil.com/',
'/\\evil.com/',
'/\\/evil.com/',
]) {
const url = buildPublicHostRedirect(
reqOf({
hostname: 'beans.app.puter.localhost',
originalUrl: evil,
}),
cfg,
);
expect(url).toBe(
'http://beans.site.puter.localhost:4100/evil.com/',
);
}
});
it('returns null when no public hosting domain is configured', () => {
const noPublic = {
...cfg,
@@ -584,6 +584,17 @@ export async function resolvePublicHostedIdentity(opts: {
// -- Redirect helpers ------------------------------------------------
/**
* Turn a request path into a safe reference for `new URL(path, base)`.
* Collapses any run of leading slashes/backslashes into a single `/` so a
* scheme-relative path (`//evil.com`, `/\evil.com`, which the URL parser
* folds to `//`) can't override the base authority and turn the redirect
* into an open redirect. Guarantees exactly one leading slash.
*/
function normalizeRedirectPath(originalUrl: string | undefined): string {
return '/' + (originalUrl || '/').replace(/^[/\\]+/, '');
}
/** Build the URL to redirect a private-app request to its private host. */
export function buildPrivateHostRedirect(
req: Request,
@@ -605,9 +616,7 @@ export function buildPrivateHostRedirect(
try {
const protocol = config.protocol || req.protocol || 'https';
const base = `${protocol}://${subdomain}.${privateDomain}`;
const reqPath = (req.originalUrl || '/').startsWith('/')
? req.originalUrl || '/'
: `/${req.originalUrl}`;
const reqPath = normalizeRedirectPath(req.originalUrl);
return new URL(reqPath, base).toString();
} catch {
return null;
@@ -638,9 +647,7 @@ export function buildPublicHostRedirect(
try {
const protocol = config.protocol || req.protocol || 'https';
const base = `${protocol}://${subdomain}.${publicDomain}`;
const reqPath = (req.originalUrl || '/').startsWith('/')
? req.originalUrl || '/'
: `/${req.originalUrl}`;
const reqPath = normalizeRedirectPath(req.originalUrl);
return new URL(reqPath, base).toString();
} catch {
return null;
+48 -11
View File
@@ -18,6 +18,7 @@
*/
import crypto from 'node:crypto';
import { withSpan } from '../../../util/span.js';
import { HttpError } from '../HttpError.js';
/**
@@ -256,8 +257,32 @@ async function acquireKvConcurrent(kv, key, limit) {
// limiting — so route / driver code never has to reason about which
// backend is wired for which mode.
/**
* Wrap a backend pair so every rate / acquire call runs inside a span
* tagged with the backend name. Applied at registration, so all gates
* (route middleware, driver helpers, imperative checks) are covered.
*/
function instrumentBackendPair(name, pair) {
const attrs = { 'rate_limit.backend': name };
return {
rate: (key, limit, windowMs) =>
withSpan('rate_limit.check', attrs, () =>
pair.rate(key, limit, windowMs),
),
acquire: (key, limit) =>
withSpan('rate_limit.acquire', attrs, () =>
pair.acquire(key, limit),
),
};
}
const memoryBackendPair = instrumentBackendPair('memory', {
rate: checkMemory,
acquire: acquireMemoryConcurrent,
});
const backends = {
memory: { rate: checkMemory, acquire: acquireMemoryConcurrent },
memory: memoryBackendPair,
};
let defaultBackendName = 'memory';
@@ -295,19 +320,19 @@ export function configureRateLimit({
} = {}) {
// Reset (test reconfigure clears stale wiring).
for (const name of Object.keys(backends)) delete backends[name];
backends.memory = { rate: checkMemory, acquire: acquireMemoryConcurrent };
backends.memory = memoryBackendPair;
if (redis) {
backends.redis = {
backends.redis = instrumentBackendPair('redis', {
rate: (key, limit, windowMs) =>
checkRedis(redis, key, limit, windowMs),
acquire: (key, limit) => acquireRedisConcurrent(redis, key, limit),
};
});
}
if (kv) {
backends.kv = {
backends.kv = instrumentBackendPair('kv', {
rate: (key, limit, windowMs) => checkKv(kv, key, limit, windowMs),
acquire: (key, limit) => acquireKvConcurrent(kv, key, limit),
};
});
}
meteringService = metering ?? null;
@@ -351,9 +376,10 @@ function resolveBackend(name) {
* Build a rate-limit key from the request.
*
* Strategies:
* 'fingerprint' — IP + User-Agent hash (default). Good for
* unauthenticated endpoints where the same IP may
* serve many users (offices, VPNs).
* 'fingerprint' — network hash (IP + headers), refined by the client's
* device fingerprint when one was supplied (default).
* Good for unauthenticated endpoints where the same
* IP may serve many users (offices, VPNs).
* 'ip' — bare IP. Simpler but coarser.
* 'user' — actor UUID. Use for authenticated endpoints where
* you want per-account limits regardless of IP.
@@ -395,7 +421,7 @@ function ip(req) {
/**
* A coarse network fingerprint for a request: a short hash of the (proxy-aware)
* IP plus the headers a client can't trivially vary per-request without also
* changing how the request looks. Used as the default rate-limit key here, and
* changing how the request looks. Anchors the default rate-limit key here, and
* exported so the global fingerprint middleware can stamp the identical value
* on `req.networkFingerprint` (one key space shared by both).
*/
@@ -413,8 +439,19 @@ export function computeNetworkFingerprint(req) {
.slice(0, 16);
}
/**
* The device fingerprint (validated and stamped by the fingerprint middleware)
* refines the bucket so devices behind one NAT don't crowd each other's limit.
* It stays anchored to the network hash because the value is client-supplied:
* alone it could be spoofed to drain another device's bucket, and rotating it
* to mint fresh buckets is caught by the same stacked 'ip' backstop that
* catches User-Agent rotation.
*/
function fingerprint(req) {
return computeNetworkFingerprint(req);
const network = req.networkFingerprint ?? computeNetworkFingerprint(req);
return req.deviceFingerprint
? `${network}:${req.deviceFingerprint}`
: network;
}
// -- Route middleware ------------------------------------------------
@@ -67,6 +67,7 @@ describe('rateLimitGate — memory backend (default)', () => {
actor: init.actor,
route: init.route,
socket: init.socket ?? { remoteAddress: '1.2.3.4' },
deviceFingerprint: init.deviceFingerprint,
});
it('admits up to `limit` hits and rejects the next one with 429', async () => {
@@ -140,6 +141,62 @@ describe('rateLimitGate — memory backend (default)', () => {
expect(isHttpError(re)).toBe(true);
});
it("'fingerprint' (default) gives each device behind one network its own bucket", async () => {
// Same IP and identical headers (a NAT'd office of look-alike
// machines) but distinct device fingerprints (stamped by the
// fingerprint middleware) → independent buckets, so one device
// hitting its limit doesn't block the whole network.
const opts = { limit: 1, window: 60_000, scope: 'mem-fp-device' };
const shared = {
ip: '5.6.7.8',
headers: { 'user-agent': 'shared-browser' },
};
expect(
await runGate(
opts,
makeReq({ ...shared, deviceFingerprint: 'device-alice-01' }),
),
).toBeUndefined();
expect(
await runGate(
opts,
makeReq({ ...shared, deviceFingerprint: 'device-bob-02' }),
),
).toBeUndefined();
// Same device again → its own bucket is full.
const re = await runGate(
opts,
makeReq({ ...shared, deviceFingerprint: 'device-alice-01' }),
);
expect(isHttpError(re)).toBe(true);
});
it('device fingerprint stays anchored to the network — spoofing a value from another IP cannot drain its bucket', async () => {
const opts = { limit: 1, window: 60_000, scope: 'mem-fp-anchor' };
const fp = 'device-victim-01';
// Victim exhausts their bucket from their own network.
expect(
await runGate(
opts,
makeReq({ ip: '10.0.0.1', deviceFingerprint: fp }),
),
).toBeUndefined();
const re = await runGate(
opts,
makeReq({ ip: '10.0.0.1', deviceFingerprint: fp }),
);
expect(isHttpError(re)).toBe(true);
// Attacker replays the same device fingerprint from elsewhere:
// different network hash → different bucket, and the victim's
// bucket is untouched.
expect(
await runGate(
opts,
makeReq({ ip: '99.99.99.99', deviceFingerprint: fp }),
),
).toBeUndefined();
});
// Stacked gates, mirroring the materializer's handling of a
// `rateLimit: [...]` array (each entry is its own gate; a rejection
// short-circuits the chain). This is the credential-endpoint pattern:
@@ -0,0 +1,263 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { describe, expect, it, vi } from 'vitest';
import { TokenService } from '../../../services/auth/TokenService.js';
import {
createStepUpGate,
signStepUpToken,
STEP_UP_COOKIE_NAME,
verifyStepUpSession,
} from './stepUpSession.js';
const V2_SECRET = 'test-v2-secret';
const USER_UUID = 'a1111111-1111-1111-1111-111111111111';
function tokenService(): TokenService {
const config = {
jwt_secret: 'test-v1-secret',
jwt_secret_v2: V2_SECRET,
allow_v1_tokens: true,
} as ConstructorParameters<typeof TokenService>[0];
const svc = new TokenService(
config,
{} as ConstructorParameters<typeof TokenService>[1],
{} as ConstructorParameters<typeof TokenService>[2],
{} as ConstructorParameters<typeof TokenService>[3],
);
svc.onServerStart();
return svc;
}
function reqWith(
cookie: string | undefined,
actorUuid: string | undefined,
extra: Partial<Request> = {},
): Request {
return {
cookies: cookie ? { [STEP_UP_COOKIE_NAME]: cookie } : {},
actor: actorUuid ? { user: { uuid: actorUuid } } : undefined,
...extra,
} as unknown as Request;
}
describe('verifyStepUpSession', () => {
it('accepts a token bound to the acting user', () => {
const ts = tokenService();
const token = signStepUpToken(ts, { uuid: USER_UUID });
expect(
verifyStepUpSession(reqWith(token, USER_UUID), { tokenService: ts }),
).toBe(true);
});
it('rejects a token bound to a different user (cookie alone is useless)', () => {
const ts = tokenService();
const token = signStepUpToken(ts, { uuid: USER_UUID });
expect(
verifyStepUpSession(
reqWith(token, 'b2222222-2222-2222-2222-222222222222'),
{ tokenService: ts },
),
).toBe(false);
});
it('rejects when there is no actor (no live session)', () => {
const ts = tokenService();
const token = signStepUpToken(ts, { uuid: USER_UUID });
expect(
verifyStepUpSession(reqWith(token, undefined), {
tokenService: ts,
}),
).toBe(false);
});
it('rejects when the cookie is missing', () => {
const ts = tokenService();
expect(
verifyStepUpSession(reqWith(undefined, USER_UUID), {
tokenService: ts,
}),
).toBe(false);
});
it('rejects an expired token', () => {
const ts = tokenService();
// Sign with a lifetime past the verifier's 30s clock tolerance.
const expired = ts.sign(
'step-up',
{ user_uuid: USER_UUID, purpose: 'elevation' },
{ expiresIn: -60 },
);
expect(
verifyStepUpSession(reqWith(expired, USER_UUID), {
tokenService: ts,
}),
).toBe(false);
});
it('rejects a token minted under a different scope/purpose (no cross-use)', () => {
const ts = tokenService();
// A well-formed session-style token must not satisfy the gate.
const authToken = ts.sign('auth', {
type: 'session',
version: '2',
user_uid: USER_UUID,
});
expect(
verifyStepUpSession(reqWith(authToken, USER_UUID), {
tokenService: ts,
}),
).toBe(false);
});
});
describe('createStepUpGate', () => {
it('passes a session with a valid elevation cookie', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const token = signStepUpToken(ts, { uuid: USER_UUID });
const next = vi.fn();
gate(reqWith(token, USER_UUID), {} as Response, next);
expect(next).toHaveBeenCalledWith();
});
it('rejects a session without an elevation cookie, hinting the factor', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const next = vi.fn();
const req = reqWith(undefined, USER_UUID, {
actor: { user: { uuid: USER_UUID, otp_enabled: true } },
} as never);
gate(req, {} as Response, next);
const err = next.mock.calls[0][0];
expect(err.statusCode).toBe(403);
expect(err.legacyCode).toBe('elevation_required');
expect(err.fields.factor).toBe('otp');
});
it('hints the password factor when 2FA is off', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const next = vi.fn();
gate(reqWith(undefined, USER_UUID), {} as Response, next);
expect(next.mock.calls[0][0].fields.factor).toBe('password');
});
// A stolen session can mint a full-access token via
// /auth/create-access-token without re-proving identity, so exempting one
// here would be a way around the gate rather than an exception to it.
it('does NOT exempt full-access personal access tokens', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const next = vi.fn();
const req = reqWith(undefined, USER_UUID, {
actor: {
user: { uuid: USER_UUID },
accessToken: { fullAccess: true },
},
} as never);
gate(req, {} as Response, next);
expect(next.mock.calls[0][0]?.statusCode).toBe(403);
});
// App-gated routes (adminOnly + allowedAppIds): an admin acting through an
// allowlisted app can't elevate — apps have no password/TOTP and are blocked
// from /auth/elevate. The exemption keys off the token carrying an allowed
// app id, not the route flag.
it('exempts a token that carries an allowlisted app id', () => {
const ts = tokenService();
const gate = createStepUpGate({
tokenService: ts,
allowedAppUids: ['app-xyz'],
});
const next = vi.fn();
const req = reqWith(undefined, USER_UUID, {
actor: { user: { uuid: USER_UUID }, app: { uid: 'app-xyz' } },
} as never);
gate(req, {} as Response, next);
expect(next).toHaveBeenCalledWith();
});
it('still requires step-up for an app id NOT in the allowlist', () => {
const ts = tokenService();
const gate = createStepUpGate({
tokenService: ts,
allowedAppUids: ['other-app'],
});
const next = vi.fn();
const req = reqWith(undefined, USER_UUID, {
actor: { user: { uuid: USER_UUID }, app: { uid: 'app-xyz' } },
} as never);
gate(req, {} as Response, next);
expect(next.mock.calls[0][0]?.statusCode).toBe(403);
});
it('still requires step-up when the route has no allowedAppIds', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const next = vi.fn();
const req = reqWith(undefined, USER_UUID, {
actor: { user: { uuid: USER_UUID }, app: { uid: 'app-xyz' } },
} as never);
gate(req, {} as Response, next);
expect(next.mock.calls[0][0]?.statusCode).toBe(403);
});
it('still requires step-up on the human/root-token path (no app id in token)', () => {
const ts = tokenService();
const gate = createStepUpGate({
tokenService: ts,
allowedAppUids: ['app-xyz'],
});
const next = vi.fn();
gate(reqWith(undefined, USER_UUID), {} as Response, next);
expect(next.mock.calls[0][0]?.statusCode).toBe(403);
});
it('accepts the elevation via the x-puter-elevation header (API clients)', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const token = signStepUpToken(ts, { uuid: USER_UUID });
const next = vi.fn();
const req = {
cookies: {},
headers: { 'x-puter-elevation': token },
actor: { user: { uuid: USER_UUID } },
} as never;
gate(req, {} as Response, next);
expect(next).toHaveBeenCalledWith();
});
it('rejects a header elevation bound to a different user', () => {
const ts = tokenService();
const gate = createStepUpGate({ tokenService: ts });
const token = signStepUpToken(ts, {
uuid: 'b2222222-2222-2222-2222-222222222222',
});
const next = vi.fn();
const req = {
cookies: {},
headers: { 'x-puter-elevation': token },
actor: { user: { uuid: USER_UUID } },
} as never;
gate(req, {} as Response, next);
expect(next.mock.calls[0][0]?.statusCode).toBe(403);
});
});
@@ -0,0 +1,191 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler } from 'express';
import type { IConfig } from '../../../types';
import type { UserRow } from '../../../stores/user/UserStore';
import type { TokenService } from '../../../services/auth/TokenService';
import { sessionCookieFlags } from '../../../util/cookieFlags';
import { HttpError } from '../HttpError';
// Make sure the `Express.Request.actor` augmentation is in scope.
import '../expressAugmentation';
/**
* Step-up ("elevation") sessions — a second factor layered on top of an
* ordinary session for privileged endpoints (`adminOnly` routes).
*
* An ordinary session cookie proves only that someone holds the credential; for
* privileged endpoints that isn't enough, since a leaked session would inherit
* the privilege. Elevation makes the caller re-prove identity — a fresh TOTP
* code when 2FA is enabled, otherwise the account password — via
* `POST /auth/elevate`, which mints the cookie below.
*
* Both halves are required and neither is sufficient: gates demand a live
* session actor AND this proof, and the proof is bound to that actor's
* `user_uuid`. So a stolen session can't elevate itself, and a stolen elevation
* proof is inert without the session.
*
* The proof travels as an httpOnly cookie for browsers, or as the
* `x-puter-elevation` header for API clients (which have no cookie jar). The
* two are equivalent — both are the same signed token, and obtaining either
* requires the password/TOTP. Nothing is exempt from the requirement: there is
* deliberately no carve-out for any credential kind, because any credential a
* stolen session can obtain *without* re-proving identity would be a way around
* this control rather than an exception to it.
*
* The token has its own scope and `purpose` claim and carries no auth `type`
* claim, so `AuthService.authenticate` rejects it — it can never be spent as a
* main auth token even though every scope shares `jwt_secret_v2`.
*/
export const STEP_UP_COOKIE_NAME = 'puter_elevated';
export const STEP_UP_HEADER_NAME = 'x-puter-elevation';
export const STEP_UP_SCOPE = 'step-up';
export const STEP_UP_PURPOSE = 'elevation';
export const STEP_UP_TTL_SECONDS = 7 * 24 * 60 * 60;
interface StepUpPayload {
user_uuid: string;
purpose: string;
}
/** Sign an elevation token bound to the user's uuid. */
export function signStepUpToken(
tokenService: TokenService,
user: Pick<UserRow, 'uuid'>,
): string {
return tokenService.sign(
STEP_UP_SCOPE,
{ user_uuid: user.uuid, purpose: STEP_UP_PURPOSE },
{ expiresIn: STEP_UP_TTL_SECONDS },
);
}
/**
* Cookie flags for the elevation cookie. `domain` and `maxAge` are what
* `sessionCookieFlags` doesn't set: the domain keeps the cookie readable across
* the site's subdomains (privileged endpoints aren't all on one origin), and
* `maxAge` gives the elevation its lifetime.
*/
export function stepUpCookieOptions(config: IConfig): {
httpOnly: true;
sameSite: 'none' | 'lax';
secure: boolean;
maxAge: number;
domain?: string;
} {
return {
...sessionCookieFlags(config),
httpOnly: true,
maxAge: STEP_UP_TTL_SECONDS * 1000,
...(config.domain ? { domain: config.domain } : {}),
};
}
/**
* True iff a valid elevation proof (cookie or `x-puter-elevation` header) is
* present AND bound to the acting user. Never throws — a missing/expired/
* mismatched proof returns false so callers can prompt for the second factor
* instead of erroring.
*/
export function verifyStepUpSession(
req: Request,
deps: { tokenService: TokenService },
): boolean {
const header = req.headers?.[STEP_UP_HEADER_NAME];
const token =
req.cookies?.[STEP_UP_COOKIE_NAME] ??
(typeof header === 'string' ? header : undefined);
const actorUuid = req.actor?.user?.uuid;
if (!token || !actorUuid) return false;
try {
const payload = deps.tokenService.verify<StepUpPayload>(
STEP_UP_SCOPE,
token,
);
return (
payload?.purpose === STEP_UP_PURPOSE &&
payload.user_uuid === actorUuid
);
} catch {
return false;
}
}
/**
* Require an elevated session. Runs after the privilege gate it supplements
* (`adminOnlyGate`), so it only adds the re-authentication requirement.
*
* Narrow by design — the only exemption is the app-gated path:
*
* - Not env-conditional, so the flow exercised locally is the one that ships.
* - No carve-out for full-access tokens. That looks safe (a deliberately
* minted, header-borne credential) but isn't: `/auth/create-access-token`
* needs only a session, so a stolen session can mint a full-access token
* without ever re-proving identity and walk straight around this gate.
* - No carve-out based on how the credential arrived (cookie vs bearer). The
* holder of a token chooses which header to put it in, so that distinction
* is attacker-controlled and worthless as a gate.
* - `allowedAppUids`: the exemption is keyed off the *token*, not the route.
* An actor whose token carries one of these allowlisted app ids (an admin
* acting through an allowlisted app) is exempt — that actor can't elevate at
* all (apps have no password/TOTP and are blocked from `/auth/elevate`), so
* step-up is unsatisfiable for it. A token WITHOUT an allowlisted app id — a
* root/human session — still requires step-up, exactly as it would on a
* route with no `allowedAppIds`. So this is not a session or token-kind
* carve-out: reaching the exempt path needs an admin's OAuth grant to a
* specific allowlisted app.
*
* The invariant for the human path: reaching a privileged endpoint requires
* proving the password or a TOTP code within the elevation's lifetime.
*
* A caller without a valid elevation proof is rejected with
* `elevation_required`; `factor` tells the client which credential to collect.
*/
export function createStepUpGate(deps: {
tokenService: TokenService;
allowedAppUids?: readonly string[];
}): RequestHandler {
return (req, _res, next) => {
// Exempt only an actor whose token carries one of the route's
// allowlisted app ids: an admin acting through an allowlisted app can't
// elevate, so step-up is unsatisfiable for it. Any other actor — most
// importantly a root/human session with no app id in its token — falls
// through and must present the elevation proof.
const appUid = req.actor?.app?.uid;
if (appUid && deps.allowedAppUids?.includes(appUid)) {
next();
return;
}
if (verifyStepUpSession(req, deps)) {
next();
return;
}
next(
new HttpError(403, 'Re-authentication required', {
legacyCode: 'elevation_required',
fields: {
code: 'elevation_required',
factor: req.actor?.user?.otp_enabled ? 'otp' : 'password',
},
}),
);
};
}
+31 -6
View File
@@ -116,14 +116,33 @@ export interface RouteOptions {
/** Allows access-tokens */
allowAccessToken?: boolean;
/**
* Reject bare user-session actors ("root" tokens — no app, no access
* token) with 403 `app_or_api_token_required`. Implies `requireAuth`.
* Use on API surfaces that must only be driven by a delegated
* credential (app/worker token or a dashboard-minted API token), so a
* copied session token can't double as an API credential. Compose with
* `requireUserActor` + `allowFullAccessToken` to further narrow to
* API tokens only (apps stay out too). Worker sessions
* (`session.kind === 'worker'`) always pass — a deployed worker is a
* delegated, revocable credential, never a root token.
*/
noUserSession?: boolean;
/**
* Reject unless the actor's username is `admin`, `system`, or one of the
* extras in this array. `true` means just `admin`/`system`; an array adds
* to that pair (does not replace it). Implies `requireAuth`.
*
* Does NOT imply `requireUserActor` — admin endpoints accept an admin's
* access-token or app-under-user actor. Combine with `requireUserActor`
* to restrict to browser sessions.
* Also requires a *root token* (an actor with no app anywhere in its
* token chain), so an admin acting through a third-party app can't reach
* the route. Pair with `allowedAppIds` to make an admin route reachable
* by specific apps: the combination admits a root token OR a token
* scoped to an allowed app.
*
* Does NOT imply `requireUserActor` — a root token still includes an
* admin's full-access personal access token, not only browser sessions.
* Combine with `requireUserActor` to restrict to browser sessions.
*/
adminOnly?: boolean | string[];
@@ -210,8 +229,10 @@ export interface RouteOptions {
* request identity.
*
* `key` controls how requests are bucketed:
* - `'fingerprint'` (default) — IP + User-Agent hash. Safe for
* shared IPs (offices, VPNs).
* - `'fingerprint'` (default) — network hash (IP + headers),
* refined by the client's device fingerprint when one was
* supplied. Safe for shared IPs (offices, VPNs): each device
* gets its own bucket instead of the whole network sharing one.
* - `'ip'` — bare IP address.
* - `'user'` — actor's user ID. Use for authenticated routes
* where you want per-account limits.
@@ -265,6 +286,8 @@ export interface RouteOptions {
// Reserved — wire as the corresponding features/services land:
// bodyFiles?: string[]; // multer-style multipart fields
// responseTimeout?: number;
realMime?: boolean; // for legacy FS controller, see `LegacyFSController#serveFile`
}
/**
@@ -322,7 +345,9 @@ export type AuthRequired<O extends RouteOptions> = O extends {
? true
: O extends { allowedAppIds: readonly string[] | string[] }
? true
: false;
: O extends { noUserSession: true }
? true
: false;
/** Express `Request` with `actor` narrowed based on the route's options. */
export type TypedRequest<O extends RouteOptions> = Omit<Request, 'actor'> & {
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import crypto from 'node:crypto';
@@ -72,13 +73,13 @@ type ProviderAttempt = {
};
/**
* Capture what an upstream provider gave us so the classifier downstream
* can decide a user-facing status code instead of always returning 500.
* Capture what an upstream provider gave us so the classifier downstream can
* decide a user-facing status code instead of always returning 500.
*
* OpenAI-SDK-based providers throw `APIError` with `.status` and a
* structured `.error` body — pull both. For arbitrary errors we fall
* back to the message and a status sniff so providers that throw plain
* `Error("... 503 ...")` strings still classify correctly.
* OpenAI-SDK-based providers throw `APIError` with `.status` and a structured
* `.error` body — pull both. For arbitrary errors we fall back to the message
* and a status sniff so providers that throw plain `Error("... 503 ...")`
* strings still classify correctly.
*/
const toAttempt = (
modelId: string,
@@ -126,11 +127,12 @@ const isUpstream5xx = (a: ProviderAttempt) =>
* Map an exhausted fallback chain to a single user-facing HttpError.
*
* Per-class rules (see also alarm gate in server.ts):
* - all rate-limited → 429 `upstream_rate_limited` (paged: forced alert)
* - all auth failures → 500 `upstream_auth_failed` (paged: our config)
* - all upstream 5xx → 400 `upstream_provider_unavailable` (no page)
* - all upstream 4xx (other) → 400 `upstream_bad_request` (no page)
* - mixed → 400 `upstream_failed` (no page)
*
* - All rate-limited → 429 `upstream_rate_limited` (paged: forced alert)
* - All auth failures → 500 `upstream_auth_failed` (paged: our config)
* - All upstream 5xx → 400 `upstream_provider_unavailable` (no page)
* - All upstream 4xx (other) → 400 `upstream_bad_request` (no page)
* - Mixed → 400 `upstream_failed` (no page)
*/
const classifyAttempts = (attempts: ProviderAttempt[]): HttpError => {
const fields = { attempts };
@@ -196,10 +198,10 @@ const classifyAttempts = (attempts: ProviderAttempt[]): HttpError => {
/**
* Driver implementing the `puter-chat-completion` interface.
*
* Manages multiple upstream providers (Claude, OpenAI, …) and handles
* model resolution, provider routing, fallback on failure, and message
* normalisation. Each provider is a plain `IChatProvider` — the driver
* instantiates them from config on boot.
* Manages multiple upstream providers (Claude, OpenAI, …) and handles model
* resolution, provider routing, fallback on failure, and message normalisation.
* Each provider is a plain `IChatProvider` — the driver instantiates them from
* config on boot.
*
* Providers handle their own metering internally.
*/
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import Anthropic from '@anthropic-ai/sdk';
@@ -271,6 +272,7 @@ export class ClaudeProvider implements IChatProvider {
'claude-sonnet-5',
'claude-opus-4-7',
'claude-opus-4-8',
'claude-opus-5',
].includes(modelUsed.id);
const resolvedTemperature = omitsTemperature
? undefined
@@ -280,6 +282,7 @@ export class ClaudeProvider implements IChatProvider {
const supportsEffort = [
'claude-fable-5',
'claude-sonnet-5',
'claude-opus-5',
'claude-opus-4-8',
'claude-opus-4-7',
'claude-opus-4-6',
@@ -682,6 +685,7 @@ export class ClaudeProvider implements IChatProvider {
// the stream.
if (
modelId === 'claude-fable-5' ||
modelId === 'claude-opus-5' ||
modelId === 'claude-opus-4-8' ||
modelId === 'claude-opus-4-7'
) {
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import type { IChatModel } from '../../types.js';
@@ -79,6 +80,36 @@ export const CLAUDE_MODELS: IChatModel[] = [
context: 1000000,
max_tokens: 64000,
},
{
puterId: 'anthropic:anthropic/claude-opus-5',
id: 'claude-opus-5',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2026-05',
release_date: '2026-07-24',
aliases: [
'claude-opus',
'claude-opus-latest',
'claude-opus-5-latest',
'claude-opus-5',
'anthropic/claude-opus-5',
],
name: 'Claude Opus 5',
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 500,
ephemeral_5m_input_tokens: 500 * 1.25,
ephemeral_1h_input_tokens: 500 * 2,
cache_read_input_tokens: 500 * 0.1,
output_tokens: 2500,
},
context: 1000000,
max_tokens: 128000,
},
{
puterId: 'anthropic:anthropic/claude-opus-4-8',
id: 'claude-opus-4-8',
@@ -88,8 +119,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
knowledge: '2026-01',
release_date: '2026-05-28',
aliases: [
'claude-opus',
'claude-opus-latest',
'claude-opus-4-8-latest',
'claude-opus-4.8',
'claude-opus-4-8',
@@ -338,127 +367,4 @@ export const CLAUDE_MODELS: IChatModel[] = [
context: 200000,
max_tokens: 32000,
},
{
puterId: 'anthropic:anthropic/claude-sonnet-4',
id: 'claude-sonnet-4-20250514',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2025-03-31',
release_date: '2025-05-22',
aliases: [
'claude-sonnet-4',
'claude-sonnet-4-latest',
'anthropic/claude-sonnet-4',
],
name: 'Claude Sonnet 4',
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 300,
ephemeral_5m_input_tokens: 300 * 1.25,
ephemeral_1h_input_tokens: 300 * 2,
cache_read_input_tokens: 300 * 0.1,
output_tokens: 1500,
},
context: 200000,
max_tokens: 64000,
},
{
puterId: 'anthropic:anthropic/claude-3-7-sonnet',
id: 'claude-3-7-sonnet-20250219',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-10-31',
release_date: '2025-02-19',
aliases: ['claude-3-7-sonnet-latest', 'anthropic/claude-3-7-sonnet'],
succeeded_by: 'claude-sonnet-4-20250514',
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 300,
ephemeral_5m_input_tokens: 300 * 1.25,
ephemeral_1h_input_tokens: 300 * 2,
cache_read_input_tokens: 300 * 0.1,
output_tokens: 1500,
},
context: 200000,
max_tokens: 8192,
},
{
puterId: 'anthropic:anthropic/claude-3-5-sonnet',
id: 'claude-3-5-sonnet-20241022',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-04-30',
release_date: '2024-10-22',
name: 'Claude 3.5 Sonnet',
aliases: ['claude-3-5-sonnet-latest', 'anthropic/claude-3-5-sonnet'],
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 300,
ephemeral_5m_input_tokens: 300 * 1.25,
ephemeral_1h_input_tokens: 300 * 2,
cache_read_input_tokens: 300 * 0.1,
output_tokens: 1500,
},
context: 200000,
max_tokens: 8192,
},
{
puterId: 'anthropic:anthropic/claude-3-5-sonnet-20240620',
id: 'claude-3-5-sonnet-20240620',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2024-04-30',
release_date: '2024-06-20',
succeeded_by: 'claude-3-5-sonnet-20241022',
aliases: ['anthropic/claude-3-5-sonnet-20240620'],
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 300,
ephemeral_5m_input_tokens: 300 * 1.25,
ephemeral_1h_input_tokens: 300 * 2,
cache_read_input_tokens: 300 * 0.1,
output_tokens: 1500,
},
context: 200000,
max_tokens: 8192,
},
{
puterId: 'anthropic:anthropic/claude-3-haiku',
id: 'claude-3-haiku-20240307',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2023-08-31',
release_date: '2024-03-13',
aliases: ['anthropic/claude-3-haiku'],
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 25,
ephemeral_5m_input_tokens: 25 * 1.25,
ephemeral_1h_input_tokens: 25 * 2,
cache_read_input_tokens: 25 * 0.1,
output_tokens: 125,
},
context: 200000,
max_tokens: 4096,
},
];
@@ -202,7 +202,25 @@ describe('MoonshotProvider.complete request shape', () => {
usage: { prompt_tokens: 1, completion_tokens: 1 },
};
it('forwards model + messages and locks max_tokens=1000', async () => {
it('forwards model + messages and passes max_tokens through unchanged', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'kimi-k2.6',
messages: [{ role: 'user', content: 'hello' }],
max_tokens: 2048,
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.model).toBe('kimi-k2.6');
expect(args.messages).toEqual([{ role: 'user', content: 'hello' }]);
expect(args.max_tokens).toBe(2048);
});
it('forwards max_tokens as undefined when not supplied', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
@@ -214,11 +232,7 @@ describe('MoonshotProvider.complete request shape', () => {
);
const [args] = createMock.mock.calls[0]!;
expect(args.model).toBe('kimi-k2.6');
expect(args.messages).toEqual([{ role: 'user', content: 'hello' }]);
// max_tokens is hardcoded by the provider — the call to Moonshot
// should always cap at 1000 tokens of completion.
expect(args.max_tokens).toBe(1000);
expect(args.max_tokens).toBeUndefined();
});
it('omits the `tools` key entirely when no tools are supplied', async () => {
@@ -21,12 +21,12 @@ import { OpenAI } from 'openai';
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
import { Context } from '../../../../core/context.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
import type {
IChatCompleteResult,
IChatProvider,
ICompleteArguments,
IChatCompleteResult,
} from '../../types.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
import { inlineHttpImageUrls } from './imageHandling.js';
import { MOONSHOT_MODELS } from './models.js';
@@ -68,6 +68,7 @@ export class MoonshotProvider implements IChatProvider {
stream,
model,
tools,
max_tokens,
}: ICompleteArguments): Promise<IChatCompleteResult> {
const actor = Context.get('actor');
const availableModels = this.models();
@@ -89,7 +90,7 @@ export class MoonshotProvider implements IChatProvider {
messages,
model: modelUsed.id,
...(tools ? { tools } : {}),
max_tokens: 1000,
max_tokens,
stream,
...(stream
? {
@@ -21,6 +21,27 @@ import type { IChatModel } from '../../types.js';
export const MOONSHOT_MODELS: IChatModel[] = [
// -- Flagship ----------------------------------------------------
{
puterId: 'moonshotai:moonshotai/kimi-k3',
id: 'kimi-k3',
name: 'Kimi K3',
aliases: ['moonshotai/kimi-k3', 'moonshot/kimi-k3'],
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 300, // $3.00 per 1M
completion_tokens: 1500, // $15.00 per 1M
cached_tokens: 30, // $0.30 per 1M
},
context: 1_048_576,
max_tokens: 1_048_576,
tool_call: true,
},
// -- Kimi K2.6 --------------------------------------------------
{
puterId: 'moonshotai:moonshotai/kimi-k2.6',
id: 'kimi-k2.6',
@@ -23,6 +23,74 @@ import type { IChatModel } from '../../types.js';
// Hardcoded from https://models.dev/api.json
export const OPEN_AI_MODELS: IChatModel[] = [
{
puterId: 'openai:openai/gpt-5.6-sol',
id: 'gpt-5.6-sol',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2026-02-16',
aliases: [
'gpt-5.6',
'gpt-5.6-sol',
'openai/gpt-5.6',
'openai/gpt-5.6-sol',
],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 500,
cached_tokens: 50,
completion_tokens: 3000,
},
context: 1_050_000,
max_tokens: 128_000,
responses_api_only: true,
},
{
puterId: 'openai:openai/gpt-5.6-terra',
id: 'gpt-5.6-terra',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2026-02-16',
aliases: ['gpt-5.6-terra', 'openai/gpt-5.6-terra'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 250,
cached_tokens: 25,
completion_tokens: 1500,
},
context: 1_050_000,
max_tokens: 128_000,
responses_api_only: true,
},
{
puterId: 'openai:openai/gpt-5.6-luna',
id: 'gpt-5.6-luna',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2026-02-16',
aliases: ['gpt-5.6-luna', 'openai/gpt-5.6-luna'],
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 100,
cached_tokens: 10,
completion_tokens: 600,
},
context: 1_050_000,
max_tokens: 128_000,
responses_api_only: true,
},
{
puterId: 'openai:openai/gpt-5.5',
id: 'gpt-5.5-2026-04-23',
@@ -21,6 +21,27 @@ import type { IChatModel } from '../../types.js';
// Hardcoded from https://models.dev/api.json
export const XAI_MODELS: IChatModel[] = [
{
puterId: 'x-ai:x-ai/grok-4.5',
id: 'grok-4.5',
modalities: { input: ['text', 'image'], output: ['text'] },
open_weights: false,
tool_call: true,
release_date: '2026-07-08',
name: 'Grok 4.5',
aliases: ['x-ai/grok-4.5', 'grok-4.5-latest'],
context: 500_000,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs: {
tokens: 1_000_000,
prompt_tokens: 200,
completion_tokens: 600,
cached_tokens: 50,
},
max_tokens: 30_000,
},
{
puterId: 'x-ai:x-ai/grok-beta',
id: 'grok-beta',
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import crypto from 'node:crypto';
@@ -37,13 +38,13 @@ import type { IGenerateParams, IImageModel, IImageProvider } from './types.js';
/**
* Driver implementing the `puter-image-generation` interface.
*
* Manages multiple upstream providers and routes `generate()` calls
* based on the requested model. Mirrors ChatCompletionDriver's pattern:
* providers are instantiated from config on boot, a model map is built
* from each provider's declared models, and calls are dispatched.
* Manages multiple upstream providers and routes `generate()` calls based on
* the requested model. Mirrors ChatCompletionDriver's pattern: providers are
* instantiated from config on boot, a model map is built from each provider's
* declared models, and calls are dispatched.
*
* Output is a URL string (web URL or data URI) — no streaming, no
* TypedValue wrapper.
* Output is a URL string (web URL or data URI) — no streaming, no TypedValue
* wrapper.
*/
export class ImageGenerationDriver extends PuterDriver {
readonly driverInterface = 'puter-image-generation';
@@ -374,6 +374,23 @@ it('meters one usage line per detected page at the per-page rate from costs.ts',
// ── Mistral OCR ─────────────────────────────────────────────────────
describe('OCRDriver.recognize (mistral)', () => {
it('throws 402 when the actor does not have enough credits', async () => {
hasCreditsSpy.mockResolvedValueOnce(false);
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.recognize({
source: dataUrl(Buffer.from('img'), 'image/png'),
provider: 'mistral',
}),
),
).rejects.toMatchObject({ statusCode: 402 });
// The paid Mistral call must not happen when credits are short.
expect(mistralOcrProcessMock).not.toHaveBeenCalled();
});
it('packages an image as an image_url chunk with a base64 data URL', async () => {
const { actor } = await makeUser();
mistralOcrProcessMock.mockResolvedValueOnce({
+25 -12
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import {
@@ -33,9 +34,9 @@ import { loadFileInput, type LoadedFile } from '../util/fileInput.js';
import { OCR_COSTS } from './costs.js';
/**
* Driver implementing `puter-ocr` — document OCR. Two providers:
* • `aws-textract` — AWS Textract (region-aware clients; direct S3 source when available)
* • `mistral` — Mistral OCR (URL/data-URL based)
* Driver implementing `puter-ocr` — document OCR. Two providers: •
* `aws-textract` — AWS Textract (region-aware clients; direct S3 source when
* available) • `mistral` — Mistral OCR (URL/data-URL based)
*/
interface RecognizeArgs {
source?: unknown;
@@ -324,6 +325,18 @@ export class OCRDriver extends PuterDriver {
args: RecognizeArgs,
actor: Actor,
) {
// Gate on credits before the paid upstream call, mirroring the
// Textract branch. Page count isn't known until Mistral responds, so
// pre-flight one page's cost and meter the real total afterward.
const hasCredits = await this.services.metering.hasEnoughCredits(
actor,
OCR_COSTS['mistral-ocr:ocr:page'],
);
if (!hasCredits)
throw new HttpError(402, 'Insufficient credits', {
legacyCode: 'insufficient_funds',
});
const model = args.model ?? 'mistral-ocr-latest';
const chunk = this.#mistralBuildChunk(loaded);
const payload: Record<string, unknown> = { model, document: chunk };
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { Readable } from 'node:stream';
@@ -27,8 +28,8 @@ import { loadFileInput } from '../util/fileInput.js';
import { VOICE_CHANGER_COSTS } from './costs.js';
/**
* Driver implementing `puter-speech2speech` — voice changer. Currently a
* single provider (ElevenLabs).
* Driver implementing `puter-speech2speech` — voice changer. Currently a single
* provider (ElevenLabs).
*/
const DEFAULT_MODEL = 'eleven_multilingual_sts_v2';
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import OpenAI, { toFile } from 'openai';
@@ -26,8 +27,8 @@ import { loadFileInput } from '../util/fileInput.js';
import { SPEECH_TO_TEXT_COSTS } from './costs.js';
/**
* Driver implementing `puter-speech2txt`. Wraps OpenAI's audio API
* (Whisper + GPT-4o transcribe models) for transcription and translation.
* Driver implementing `puter-speech2txt`. Wraps OpenAI's audio API (Whisper +
* GPT-4o transcribe models) for transcription and translation.
*
* `file` may be a path, uid/uuid ref, or data URL.
*/
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { Context } from '../../core/context.js';
@@ -26,10 +27,11 @@ import { loadFileInput } from '../util/fileInput.js';
/**
* Driver implementing `puter-speech2txt` for the xAI (Grok) STT API.
*
* Uses the xAI /v1/stt REST endpoint which accepts multipart/form-data
* with an audio file and returns a JSON transcript with word-level timestamps.
* Uses the xAI /v1/stt REST endpoint which accepts multipart/form-data with an
* audio file and returns a JSON transcript with word-level timestamps.
*
* Pricing: $0.10/hr REST = 10 cents/hr = 10 * 1_000_000 / 3600 ≈ 2778 microcents/second
* Pricing: $0.10/hr REST = 10 cents/hr = 10 * 1_000_000 / 3600 ≈ 2778
* microcents/second
*/
const API_BASE = 'https://api.x.ai/v1';
+18 -25
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { Context } from '../../core/context.js';
@@ -37,10 +38,9 @@ import type {
/**
* Driver implementing the `puter-tts` interface.
*
* Manages multiple upstream TTS providers (OpenAI, ElevenLabs, AWS Polly)
* and handles provider routing, voice/engine aggregation, and speech
* synthesis. Each provider is an `ITTSProvider` instantiated from config
* on boot.
* Manages multiple upstream TTS providers (OpenAI, ElevenLabs, AWS Polly) and
* handles provider routing, voice/engine aggregation, and speech synthesis.
* Each provider is an `ITTSProvider` instantiated from config on boot.
*/
// puter-js still routes TTS via legacy per-provider driver names rather
// than passing `{ provider }` in args, so alias the unified driver under
@@ -87,9 +87,7 @@ export class TTSDriver extends PuterDriver {
// -- Interface methods -------------------------------------------
/**
* List all available voices across all configured providers.
*/
/** List all available voices across all configured providers. */
async list_voices(args?: Record<string, unknown>): Promise<ITTSVoice[]> {
const provider =
(args?.provider as string | undefined) ?? this.#providerFromAlias();
@@ -108,9 +106,7 @@ export class TTSDriver extends PuterDriver {
return allVoices;
}
/**
* List all available engines/models across all configured providers.
*/
/** List all available engines/models across all configured providers. */
async list_engines(args?: Record<string, unknown>): Promise<ITTSEngine[]> {
const provider =
(args?.provider as string | undefined) ?? this.#providerFromAlias();
@@ -129,9 +125,7 @@ export class TTSDriver extends PuterDriver {
return allEngines;
}
/**
* List provider names that are currently configured.
*/
/** List provider names that are currently configured. */
async list(): Promise<string[]> {
return Object.keys(this.#providers);
}
@@ -157,9 +151,8 @@ export class TTSDriver extends PuterDriver {
}
/**
* Synthesize speech from text. Routes to the appropriate provider
* based on the `provider` argument, or falls back to the first
* available provider.
* Synthesize speech from text. Routes to the appropriate provider based on
* the `provider` argument, or falls back to the first available provider.
*/
async synthesize(
args: ISynthesizeArgs,
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { posix as pathPosix } from 'node:path';
+187 -150
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { Context } from '../../core/context.js';
@@ -29,6 +30,17 @@ import {
isRawBase64ImageString,
normalizeRawBase64ImageString,
} from '../../util/appIcon.js';
import {
buildHostedBackingDenial,
extractPuterHostedSubdomain,
hostedIndexUrlBackingIsUnavailable,
} from '../../util/hostedAppBacking.js';
import {
decodeCursor,
encodeCursor,
normalizeLimit,
normalizeOffset,
} from '../../util/pagination.js';
import { resolvePrivateLaunchAccess } from '../../util/privateLaunchAccess.js';
import {
validateArrayOfStrings,
@@ -83,15 +95,16 @@ const hasIndexUrlUniquenessExemption = (candidates) => {
/**
* Driver exposing the `puter-apps` interface.
*
* Wraps AppStore with input validation + permission checks.
* Methods follow the `crud-q` shape client SDKs expect:
* create, read, select, update, upsert, delete
* Wraps AppStore with input validation + permission checks. Methods follow the
* `crud-q` shape client SDKs expect: create, read, select, update, upsert,
* delete
*
* Permission model:
* - Owner (apps.owner_user_id === actor.user.id) has full access
* - App actor matching app_owner has full access
* - `system:es:write-all-owners` grants blanket write
* - `app:uid#<app_uid>:access` grants protected-app access
*
* - Owner (apps.owner_user_id === actor.user.id) has full access
* - App actor matching app_owner has full access
* - `system:es:write-all-owners` grants blanket write
* - `app:uid#<app_uid>:access` grants protected-app access
*/
export class AppDriver extends PuterDriver {
driverInterface = 'puter-apps';
@@ -241,17 +254,46 @@ export class AppDriver extends PuterDriver {
return this.#toClient(app, actor, { ...params, stats });
}
async select({ predicate, params = {} } = {}) {
async select(args = {}) {
const { predicate, params = {} } = args;
const actor = this.#requireActor();
this.#requireUserOrAppActor(actor);
const limit = normalizeLimit(args.limit, { cap: 5000 }) ?? 500;
const offset = normalizeOffset(args.offset);
const hasCursor = Object.prototype.hasOwnProperty.call(args, 'cursor');
const payload = decodeCursor(args.cursor);
if (payload && offset !== undefined) {
throw new HttpError(400, 'cursor and offset cannot be combined', {
legacyCode: 'bad_request',
});
}
const includeTotal = args.includeTotal === true;
const paginated = hasCursor || offset !== undefined || includeTotal;
const filters = {};
// predicate: ['user-can-edit'] → scope to owner
if (Array.isArray(predicate) && predicate[0] === 'user-can-edit') {
const ownerScoped =
Array.isArray(predicate) && predicate[0] === 'user-can-edit';
if (ownerScoped) {
filters.ownerUserId = actor.user.id;
}
const apps = await this.appStore.list(filters);
let apps = await this.appStore.list({
...filters,
limit: paginated ? limit + 1 : limit,
offset,
afterId: payload?.id !== undefined ? Number(payload.id) : undefined,
});
let cursor;
if (paginated && apps.length > limit) {
apps = apps.slice(0, limit);
// The cursor tracks the last fetched row, not the last visible
// one, so rows hidden by the permission filter below aren't
// re-scanned on the next page.
cursor = encodeCursor({ id: Number(apps[apps.length - 1].id) });
}
// Resolve protected-app visibility:
// 1. Cheap local short-circuits (non-protected, self-app, owner).
@@ -302,7 +344,7 @@ export class AppDriver extends PuterDriver {
),
]);
return Promise.all(
const items = await Promise.all(
visible.map((app) =>
this.#toClient(app, actor, {
...params,
@@ -311,6 +353,22 @@ export class AppDriver extends PuterDriver {
}),
),
);
if (!paginated) return items;
let total;
if (includeTotal) {
total = ownerScoped
? await this.appStore.count({ ownerUserId: actor.user.id })
: await this.appStore.count({
visibleToUserId: actor.user?.id ?? null,
});
}
return {
items,
...(cursor ? { cursor } : {}),
...(total !== undefined ? { total } : {}),
};
}
async update({ uid, id, object } = {}) {
@@ -597,13 +655,13 @@ export class AppDriver extends PuterDriver {
}
/**
* App iframes run with `allow-same-origin allow-scripts`, so an
* index_url loading from the GUI host would execute third-party code
* same-origin with the desktop — a full sandbox escape. The API host
* is reserved for the same reason, and the builtin sentinel host is
* rewritten by the GUI to `<gui origin>/builtin/…` (see
* BUILTIN_APPS_HOST above). Host comparison (rather than full origin)
* deliberately also catches scheme/port variants of these hosts.
* App iframes run with `allow-same-origin allow-scripts`, so an index_url
* loading from the GUI host would execute third-party code same-origin with
* the desktop — a full sandbox escape. The API host is reserved for the
* same reason, and the builtin sentinel host is rewritten by the GUI to
* `<gui origin>/builtin/…` (see BUILTIN_APPS_HOST above). Host comparison
* (rather than full origin) deliberately also catches scheme/port variants
* of these hosts.
*/
#assertIndexUrlHostAllowed(indexUrl) {
let hostname;
@@ -673,15 +731,15 @@ export class AppDriver extends PuterDriver {
}
/**
* uid lookup with canonical-uid alias fallback. When two app rows
* have been merged (see {@link #maybeJoinOwnedHostedIndexUrlApp}),
* the source uid is recorded as an alias to the canonical uid. A
* direct uid miss therefore re-queries with the canonical uid so
* any client still holding the old uid keeps resolving to the
* joined row. Mirrors v1 AppES's `#read` alias plumbing.
* Uid lookup with canonical-uid alias fallback. When two app rows have been
* merged (see {@link #maybeJoinOwnedHostedIndexUrlApp}), the source uid is
* recorded as an alias to the canonical uid. A direct uid miss therefore
* re-queries with the canonical uid so any client still holding the old uid
* keeps resolving to the joined row. Mirrors v1 AppES's `#read` alias
* plumbing.
*
* The alias query is fired in parallel with the direct lookup so
* the common (no-alias) case pays only one round-trip.
* The alias query is fired in parallel with the direct lookup so the common
* (no-alias) case pays only one round-trip.
*/
async #getByUidWithAlias(uid) {
const aliasPromise = this.#readCanonicalAppUidAlias(uid);
@@ -748,23 +806,24 @@ export class AppDriver extends PuterDriver {
* Resolve the canonical app row that backs `app.index_url`.
*
* Returns `{ origin, expectedUid, canonicalApp }`:
* - `origin` — the parsed origin string from `index_url`.
* - `expectedUid` — the canonical app uid for that origin (oldest
* `apps.index_url` match, or a deterministic UUIDv5 fallback for
* unknown origins).
* - `canonicalApp` — the actual `apps` row at `expectedUid`, or
* `null` when the uid is a UUIDv5 fallback with no DB row.
*
* - `origin` — the parsed origin string from `index_url`.
* - `expectedUid` — the canonical app uid for that origin (oldest
* `apps.index_url` match, or a deterministic UUIDv5 fallback for unknown
* origins).
* - `canonicalApp` — the actual `apps` row at `expectedUid`, or `null` when
* the uid is a UUIDv5 fallback with no DB row.
*
* Used in `#toClient` for two things:
* 1. `created_from_origin` derivation (only set when
* `expectedUid === app.uid`, mirroring v1 AppES).
* 2. The canonical-private gate — when `expectedUid !== app.uid`
* and `canonicalApp.is_private`, the row is squatting on
* someone else's private hosted URL. We must run the
* privateAccess gate against the *canonical* row, not the
* possibly-public squatter row, otherwise pre-existing data
* from before the `subdomain_not_owned` check leaks the
* victim's index_url.
*
* 1. `created_from_origin` derivation (only set when `expectedUid ===
* app.uid`, mirroring v1 AppES).
* 2. The canonical-private gate — when `expectedUid !== app.uid` and
* `canonicalApp.is_private`, the row is squatting on someone else's
* private hosted URL. We must run the privateAccess gate against the
* _canonical_ row, not the possibly-public squatter row, otherwise
* pre-existing data from before the `subdomain_not_owned` check leaks
* the victim's index_url.
*
* Returns `null` when there's no `index_url` or it doesn't parse.
*/
@@ -794,6 +853,35 @@ export class AppDriver extends PuterDriver {
}
}
/**
* Launch-safety check for puter-hosted `index_url`s. See
* `util/hostedAppBacking.ts` — the check lives there because every producer
* of launchable app metadata needs it, not just this driver.
*/
async #hostedIndexUrlBackingIsUnavailable(app) {
return hostedIndexUrlBackingIsUnavailable({
app,
subdomainStore: this.stores.subdomain,
config: this.config,
});
}
/**
* Public wrapper over the authoritative client serializer. Callers outside
* the driver (e.g. the homepage shell baking app metadata into the rendered
* GUI) must never embed a raw store row — it carries `owner_user_id`, admin
* flags, and the private `index_url`. This projects the safe field subset
* and applies the private-app `index_url` gate for `actor`, which may be
* null for an anonymous request.
*
* @param {Record<string, unknown>} app - Raw app store row
* @param {any} [actor] - Request actor, or null when anonymous
* @param {Record<string, unknown>} [params]
*/
async toClientView(app, actor = null, params = {}) {
return this.#toClient(app, actor, params);
}
async #toClient(app, actor, params = {}) {
if (!app) return null;
@@ -801,12 +889,14 @@ export class AppDriver extends PuterDriver {
// batched query and threads them through `params.filetypes` to
// avoid the N+1 in this hot loop. Single-app callers (`read`,
// `create`, `update`) fall back to the per-app query.
const [filetypes, canonicalForIndexUrl] = await Promise.all([
params.filetypes !== undefined
? Promise.resolve(params.filetypes)
: this.appStore.getFiletypeAssociations(app.id),
this.#resolveCanonicalForIndexUrl(app),
]);
const [filetypes, canonicalForIndexUrl, hostedBackingUnavailable] =
await Promise.all([
params.filetypes !== undefined
? Promise.resolve(params.filetypes)
: this.appStore.getFiletypeAssociations(app.id),
this.#resolveCanonicalForIndexUrl(app),
this.#hostedIndexUrlBackingIsUnavailable(app),
]);
const createdFromOrigin =
canonicalForIndexUrl && canonicalForIndexUrl.expectedUid === app.uid
@@ -903,82 +993,24 @@ export class AppDriver extends PuterDriver {
}
}
// Hosted-subdomain launch guard (independent of the private-app
// gate): deny launch when the app's puter-hosted backing is gone or
// has been reclaimed by another user, so the GUI never appends the
// launch token to an origin the app owner no longer controls. Only
// set when not already denied so a private app's existing decision
// is preserved.
if (
hostedBackingUnavailable &&
result.privateAccess?.hasAccess !== false
) {
result.privateAccess = buildHostedBackingDenial();
}
return result;
}
// -- Puter-hosted index_url merge logic ---------------------------
//
// Ported from v1 AppES (`#maybeJoinOwnedHostedIndexUrlAppOnCreate`,
// `#ensure_puter_site_subdomain_is_owned`, `#ensureIndexUrlNotAlreadyInUse`,
// and the `app:canonicalUidAlias:*` kvstore pair). When a user creates
// or repoints an app at a puter-hosted subdomain (`*.puter.site`,
// `*.puter.app`, …) — or at a custom host claimed by an
// `app_origin_aliases` group — we want exactly one app row to back
// that URL:
// • If a no-owner row exists (origin-bootstrap stub auto-created
// when an unknown origin first hit Puter) → claim it for the
// user, merge the new fields into it.
// • If the same user already has an origin-bootstrap row at that
// URL → merge into it; otherwise reject as a duplicate.
// • If a different user owns the row → reject with
// `app_index_url_already_in_use`.
// Source uid (when called from `update`) is recorded in a kvstore
// alias so `#resolve` can redirect old uids to the canonical row.
#normalizeConfiguredHostedDomain(domainValue) {
if (typeof domainValue !== 'string') return null;
const normalizedDomain = domainValue
.trim()
.toLowerCase()
.replace(/^\./, '');
if (!normalizedDomain) return null;
return normalizedDomain.split(':')[0] || null;
}
#getPuterHostedDomains() {
const domains = new Set();
const config = this.config ?? {};
for (const configuredDomain of [
config.static_hosting_domain,
config.static_hosting_domain_alt,
config.private_app_hosting_domain,
config.private_app_hosting_domain_alt,
]) {
const normalized =
this.#normalizeConfiguredHostedDomain(configuredDomain);
if (normalized) domains.add(normalized);
}
return [...domains];
}
#extractPuterHostedSubdomain(indexUrl) {
if (typeof indexUrl !== 'string' || !indexUrl) return null;
let hostname;
try {
hostname = new URL(indexUrl).hostname.toLowerCase();
} catch {
return null;
}
// Sort longest-first so `foo.puter.app` matches `puter.app` (not
// a shorter `app` if it ever appeared in the configured list).
const hostedDomains = this.#getPuterHostedDomains().sort(
(a, b) => b.length - a.length,
);
for (const hostedDomain of hostedDomains) {
const suffix = `.${hostedDomain}`;
if (hostname.endsWith(suffix)) {
const subdomain = hostname.slice(
0,
hostname.length - suffix.length,
);
return subdomain || null;
}
}
return null;
return extractPuterHostedSubdomain(indexUrl, this.config);
}
#isPuterHostedIndexUrl(indexUrl) {
@@ -1012,8 +1044,8 @@ export class AppDriver extends PuterDriver {
}
/**
* Return the alias group containing this index_url's host, or null when
* the host isn't claimed by any group.
* Return the alias group containing this index_url's host, or null when the
* host isn't claimed by any group.
*/
#findOriginAliasGroupForIndexUrl(indexUrl) {
if (typeof indexUrl !== 'string' || !indexUrl) return null;
@@ -1030,11 +1062,10 @@ export class AppDriver extends PuterDriver {
}
/**
* Generate the set of equivalent index_url strings that should
* collide with a given input. We only collapse trailing-slash and
* `/index.html` variants — the underlying `apps.index_url` column
* is matched by exact string, so anything not in this list won't
* be deduped. Mirrors v1 AppES exactly.
* Generate the set of equivalent index_url strings that should collide with
* a given input. We only collapse trailing-slash and `/index.html` variants
* — the underlying `apps.index_url` column is matched by exact string, so
* anything not in this list won't be deduped. Mirrors v1 AppES exactly.
*/
#buildEquivalentIndexUrlCandidates(indexUrl) {
if (typeof indexUrl !== 'string' || !indexUrl.trim()) {
@@ -1117,7 +1148,15 @@ export class AppDriver extends PuterDriver {
const subdomain = this.#extractPuterHostedSubdomain(indexUrl);
if (!subdomain) return;
const row = await this.stores.subdomain.getBySubdomain(subdomain);
let row = await this.stores.subdomain.getBySubdomain(subdomain);
if (!row) {
// Deploys create the subdomain and immediately point the app
// at it, so a replica or peer-cache miss here would wrongly
// refuse the owner. Confirm against the primary before failing.
row = await this.stores.subdomain.getBySubdomain(subdomain, {
primary: true,
});
}
if (!row || row.user_id !== user.id) {
throw new HttpError(400, 'Subdomain not owned by user', {
legacyCode: 'subdomain_not_owned',
@@ -1127,12 +1166,11 @@ export class AppDriver extends PuterDriver {
}
/**
* Origin-bootstrap detection: rows auto-created when an unknown
* origin first needed an app row (no human-supplied metadata).
* Marker is `name === uid && title === uid` and a description
* starting with "App created from origin ". Only these rows are
* eligible for same-owner merging — refusing to merge arbitrary
* same-owner apps prevents accidental data loss.
* Origin-bootstrap detection: rows auto-created when an unknown origin
* first needed an app row (no human-supplied metadata). Marker is `name ===
* uid && title === uid` and a description starting with "App created from
* origin ". Only these rows are eligible for same-owner merging — refusing
* to merge arbitrary same-owner apps prevents accidental data loss.
*/
#isOriginBootstrapApp(app) {
if (!app || typeof app !== 'object') return false;
@@ -1220,16 +1258,15 @@ export class AppDriver extends PuterDriver {
}
/**
* Merge an incoming create/update into an existing app row that
* already owns the same puter-hosted or alias-group index_url.
* Returns the joined
* (client-shaped) app on success, or `null` when no merge applied.
* Throws `app_index_url_already_in_use` when a conflict exists but
* cannot be merged (different owner, or same-owner non-bootstrap).
* Merge an incoming create/update into an existing app row that already
* owns the same puter-hosted or alias-group index_url. Returns the joined
* (client-shaped) app on success, or `null` when no merge applied. Throws
* `app_index_url_already_in_use` when a conflict exists but cannot be
* merged (different owner, or same-owner non-bootstrap).
*
* `sourceAppUid` is set when called from update — when present and
* different from the conflict row's uid, the source row is deleted
* and an alias is recorded so old-uid clients keep resolving.
* different from the conflict row's uid, the source row is deleted and an
* alias is recorded so old-uid clients keep resolving.
*/
async #maybeJoinOwnedHostedIndexUrlApp({
object,
+357
View File
@@ -918,6 +918,125 @@ describe('AppDriver.select additional branches', () => {
});
});
// -- select pagination --
describe('AppDriver.select pagination', () => {
const makeApps = async (count: number) => {
const { actor } = await makeUser();
const names: string[] = [];
for (let i = 0; i < count; i++) {
const name = uniqueName(`pg${i}`);
names.push(name);
await withActor(actor, () =>
driver.create({
object: { name, title: 't', index_url: uniqueIndexUrl() },
}),
);
}
return { actor, names };
};
it('keeps the bare array response for plain limit requests', async () => {
const { actor } = await makeApps(2);
const result = await withActor(actor, () =>
driver.select({ predicate: ['user-can-edit'], limit: 1 }),
);
expect(Array.isArray(result)).toBe(true);
expect((result as unknown[]).length).toBe(1);
});
it('pages through owned apps with cursors', async () => {
const { actor, names } = await makeApps(5);
const seen: string[] = [];
let cursor: string | null | undefined = null;
do {
const page = (await withActor(actor, () =>
driver.select({
predicate: ['user-can-edit'],
limit: 2,
cursor,
}),
)) as { items: Array<{ name: string }>; cursor?: string };
seen.push(...page.items.map((r) => r.name));
cursor = page.cursor;
} while (cursor);
expect(seen).toEqual(names);
});
it('supports offset paging', async () => {
const { actor, names } = await makeApps(3);
const page = (await withActor(actor, () =>
driver.select({
predicate: ['user-can-edit'],
limit: 10,
offset: 1,
}),
)) as { items: Array<{ name: string }> };
expect(page.items.map((r) => r.name)).toEqual(names.slice(1));
});
it('rejects cursor combined with offset', async () => {
const { actor } = await makeApps(2);
const first = (await withActor(actor, () =>
driver.select({
predicate: ['user-can-edit'],
limit: 1,
cursor: null,
}),
)) as { cursor?: string };
expect(first.cursor).toBeDefined();
await expect(
withActor(actor, () =>
driver.select({ offset: 1, cursor: first.cursor }),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('reports an exact total for owner-scoped selects', async () => {
const { actor, names } = await makeApps(3);
const page = (await withActor(actor, () =>
driver.select({
predicate: ['user-can-edit'],
limit: 1,
includeTotal: true,
}),
)) as { items: unknown[]; total?: number };
expect(page.items.length).toBe(1);
expect(page.total).toBe(names.length);
});
it("hides other users' protected apps from paginated catalog listings", async () => {
const a = await makeApps(3);
const [visible1, hidden, visible2] = a.names;
const created = await withActor(a.actor, () =>
driver.read({ id: { name: hidden } }),
);
const row = await server.stores.app.getByUid(
(created as Record<string, unknown>).uid as string,
);
await server.clients.db.write(
'UPDATE `apps` SET `protected` = 1 WHERE `id` = ?',
[row!.id],
);
await server.stores.app.invalidateByUid(row!.uid as string);
const b = await makeUser();
const seen: string[] = [];
let cursor: string | null | undefined = null;
do {
const page = (await withActor(b.actor, () =>
driver.select({ limit: 50, cursor }),
)) as { items: Array<{ name: string }>; cursor?: string };
seen.push(...page.items.map((r) => r.name));
cursor = page.cursor;
} while (cursor);
expect(seen).toContain(visible1);
expect(seen).toContain(visible2);
expect(seen).not.toContain(hidden);
});
});
// ── upsert ──────────────────────────────────────────────────────────
describe('AppDriver.upsert additional branches', () => {
@@ -1081,3 +1200,241 @@ describe('AppDriver alias-group index_url merge', () => {
expect(first.uid).not.toBe(second.uid);
});
});
// ── hosted-subdomain ownership check ────────────────────────────────
//
// `#ensurePuterSiteSubdomainIsOwned` gates puter-hosted index_urls on a
// subdomain row the caller owns. Deploy flows create that row and point
// the app at it in back-to-back requests, so the check must tolerate a
// replica/cache miss by confirming against the primary before refusing.
describe('AppDriver hosted-subdomain ownership check', () => {
const hostedUrl = (sub: string) => `https://${sub}.site.puter.localhost/`;
it('accepts a hosted index_url when the subdomain row has not reached the replica yet', async () => {
const { actor, userId } = await makeUser();
const sub = uniqueName('deploy');
await server.stores.subdomain.create({ userId, subdomain: sub });
// Simulate a peer node with a lagging replica: no cache entry for
// the row, and replica reads (`read`) that don't see it yet while
// primary reads (`pread`) do. Sqlite's `pread` delegates to
// `this.read`, so pin it to the original before stubbing `read`.
await server.clients.redis.del(`subdomains:name:${sub}`);
const db = server.clients.db as unknown as {
read: (q: string, p?: unknown[]) => Promise<unknown[]>;
pread: (q: string, p?: unknown[]) => Promise<unknown[]>;
};
const originalRead = db.read.bind(server.clients.db);
const hadOwnPread = Object.prototype.hasOwnProperty.call(db, 'pread');
db.pread = async (query: string, params?: unknown[]) =>
originalRead(query, params);
db.read = async (query: string, params?: unknown[]) =>
query.includes('FROM `subdomains`') && (params ?? []).includes(sub)
? []
: originalRead(query, params);
try {
const result = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'Deployed App',
index_url: hostedUrl(sub),
},
}),
);
expect(result.uid).toEqual(expect.any(String));
// The primary hit must also heal the stale cache: a normal
// lookup now resolves from cache even though the replica
// still misses.
const healed =
await server.stores.subdomain.getBySubdomain(sub);
expect(healed?.subdomain).toBe(sub);
} finally {
delete (db as { read?: unknown }).read;
if (!hadOwnPread) delete (db as { pread?: unknown }).pread;
}
});
it('create merges into an owner-stamped bootstrap stub for an owned subdomain', async () => {
// The get-user-app-token bootstrap path stamps the subdomain owner
// on the stub at mint time — the owner's later create must still
// absorb the stub (claimOwnership is skipped, merge proceeds).
const { actor, userId } = await makeUser();
const sub = uniqueName('ownedstub');
await server.stores.subdomain.create({ userId, subdomain: sub });
const stubUid = `app-${uuidv4()}`;
await server.stores.app.createFromOrigin(
stubUid,
`https://${sub}.site.puter.localhost`,
{ ownerUserId: userId },
);
const name = uniqueName('owned-create');
const result = await withActor(actor, () =>
driver.create({
object: {
name,
title: 'Owned stub',
index_url: hostedUrl(sub),
},
}),
);
expect(result.uid).toBe(stubUid);
expect(result.name).toBe(name);
const stored = await server.stores.app.getByUid(stubUid);
expect(stored?.owner_user_id).toBe(userId);
});
it('rejects a hosted index_url whose subdomain does not exist anywhere', async () => {
const { actor } = await makeUser();
await expect(
withActor(actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'x',
index_url: hostedUrl(uniqueName('ghost')),
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it("rejects a hosted index_url pointing at another user's subdomain", async () => {
const owner = await makeUser();
const intruder = await makeUser();
const sub = uniqueName('theirs');
await server.stores.subdomain.create({
userId: owner.userId,
subdomain: sub,
});
await expect(
withActor(intruder.actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'x',
index_url: hostedUrl(sub),
},
}),
),
).rejects.toMatchObject({ statusCode: 400 });
});
// -- Launch guard: the backing subdomain can disappear AFTER the app
// was created (deleted by its owner, then reclaimable by anyone). The
// read path must refuse to launch so the GUI never appends the launch
// token to a now-reclaimable origin.
it('denies launch when the hosted subdomain is later deleted', async () => {
const { actor, userId } = await makeUser();
const sub = uniqueName('gone');
const row = await server.stores.subdomain.create({
userId,
subdomain: sub,
});
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'Backed App',
index_url: hostedUrl(sub),
},
}),
);
// While the subdomain is still owned, the app launches normally.
const before = await withActor(actor, () =>
driver.read({ uid: created.uid }),
);
expect(String(before.index_url)).toContain(sub);
expect(
(before.privateAccess as { hasAccess?: boolean } | undefined)
?.hasAccess,
).not.toBe(false);
// Delete the subdomain but keep the app pointing at it.
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId },
);
const after = await withActor(actor, () =>
driver.read({ uid: created.uid }),
);
const access = after.privateAccess as {
hasAccess?: boolean;
reason?: string;
};
expect(access?.hasAccess).toBe(false);
expect(access?.reason).toBe('hosted_backing_unavailable');
});
it('denies launch when the hosted subdomain was reclaimed by another user', async () => {
const owner = await makeUser();
const attacker = await makeUser();
const sub = uniqueName('reclaim');
const row = await server.stores.subdomain.create({
userId: owner.userId,
subdomain: sub,
});
const created = await withActor(owner.actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'Backed App',
index_url: hostedUrl(sub),
},
}),
);
// Owner deletes the subdomain; the attacker re-registers the name.
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId: owner.userId },
);
await server.stores.subdomain.create({
userId: attacker.userId,
subdomain: sub,
});
const after = await withActor(owner.actor, () =>
driver.read({ uid: created.uid }),
);
expect(
(after.privateAccess as { hasAccess?: boolean }).hasAccess,
).toBe(false);
});
it('keeps launching while the hosted subdomain is still owned', async () => {
const { actor, userId } = await makeUser();
const sub = uniqueName('live');
await server.stores.subdomain.create({ userId, subdomain: sub });
const created = await withActor(actor, () =>
driver.create({
object: {
name: uniqueName('app'),
title: 'Backed App',
index_url: hostedUrl(sub),
},
}),
);
const result = await withActor(actor, () =>
driver.read({ uid: created.uid }),
);
expect(String(result.index_url)).toContain(sub);
expect(
(result.privateAccess as { hasAccess?: boolean } | undefined)
?.hasAccess,
).not.toBe(false);
});
});
@@ -0,0 +1,94 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { puterDrivers } from './index.js';
import { RESERVED_DRIVER_METHODS, resolveCallableMethods } from './meta.js';
// Guard on the exact set of methods each driver exposes over `/drivers/call`.
// The RPC surface is derived structurally (see `resolveCallableMethods`): a
// method is callable iff it is a novel public method on the concrete driver
// class. This test pins that surface so that ADDING a plain public method to a
// driver — which would silently make it a remote endpoint — fails CI until the
// expected list here is updated. It is the "fail loud" backstop for the
// otherwise fail-open structural gate.
//
// PuterDriver's constructor signature is (config, clients, stores, services);
// field initializers don't read them, so empty mocks are fine (same trick as
// driverPolicies.test.ts).
const fake = () => [{}, {}, {}, {}] as [any, any, any, any];
// Expected callable surface, keyed by the registry key in `puterDrivers`.
// Keep alphabetical within each list for easy diffing.
const EXPECTED: Record<string, string[]> = {
kvStore: [
'add', 'batchPut', 'decr', 'del', 'expire', 'expireAt', 'flush',
'get', 'incr', 'list', 'remove', 'set', 'update',
],
aiChat: ['complete', 'list', 'models'],
aiImage: ['generate', 'list', 'models'],
aiTts: ['list', 'list_engines', 'list_voices', 'synthesize'],
aiVideo: ['generate', 'list', 'models'],
aiSpeech2Speech: ['convert'],
aiSpeech2Txt: ['list_models', 'transcribe', 'translate'],
aiSpeech2TxtXai: ['list_models', 'transcribe', 'translate'],
aiOcr: ['recognize'],
// AppDriver is the legacy `.js` driver: its non-RPC helpers are plain
// public methods (not `#`-private), so `isNameAvailable` (an AppController
// helper) and `toClientView` (a safe-field projection used by the homepage
// shell) sit on the callable surface. Both are (and always were, under the
// old reflection dispatch) remotely callable — pinned here rather than
// silently exposed. See the follow-up note: lock these down by making them
// `#`-private with dedicated call sites if the exposure is unwanted.
apps: [
'create', 'delete', 'isNameAvailable', 'read', 'select',
'toClientView', 'update', 'upsert',
],
subdomains: ['create', 'delete', 'read', 'select', 'update', 'upsert'],
notifications: ['create', 'mark_acknowledged', 'mark_shown', 'read', 'select'],
workers: ['create', 'destroy', 'getFilePaths', 'getLoggingUrl'],
};
describe('driver callable-method surface', () => {
for (const [key, DriverClass] of Object.entries(puterDrivers)) {
const instance = new (DriverClass as new (
...a: [any, any, any, any]
) => object)(...fake());
const callable = resolveCallableMethods(instance);
it(`${key}: exposes exactly its declared RPC methods`, () => {
expect([...callable].sort()).toEqual(EXPECTED[key]);
});
it(`${key}: never exposes lifecycle/framework methods`, () => {
for (const reserved of RESERVED_DRIVER_METHODS) {
expect(callable.has(reserved)).toBe(false);
}
for (const framework of [
'constructor',
'toString',
'valueOf',
'hasOwnProperty',
'isPrototypeOf',
]) {
expect(callable.has(framework)).toBe(false);
}
});
}
});
+8
View File
@@ -22,6 +22,7 @@ import {
DRIVER_DEFAULT_KEY,
DRIVER_INTERFACE_KEY,
DRIVER_NAME_KEY,
DRIVER_NO_USER_SESSION_KEY,
DRIVER_RATE_LIMIT_KEY,
validateDriverConcurrent,
validateDriverRateLimit,
@@ -80,6 +81,12 @@ export interface DriverOptions {
* unbounded — matching today's behaviour where nothing is gated.
*/
concurrent?: DriverConcurrentConfig;
/**
* When true, `/drivers/call` rejects bare account-session ("root")
* tokens for this driver — callers need an app/worker token or a
* dashboard-minted API token. See `DriverMeta.noUserSession`.
*/
noUserSession?: boolean;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -130,5 +137,6 @@ export function Driver(interfaceName: string, opts: DriverOptions = {}) {
proto[DRIVER_DEFAULT_KEY] = opts.default ?? false;
if (rateLimit) proto[DRIVER_RATE_LIMIT_KEY] = rateLimit;
if (concurrent) proto[DRIVER_CONCURRENT_KEY] = concurrent;
if (opts.noUserSession) proto[DRIVER_NO_USER_SESSION_KEY] = true;
};
}
@@ -150,6 +150,24 @@ describe.each([
it('points at the shared AI_CONCURRENT constant', () => {
expect(m.concurrent).toBe(AI_CONCURRENT);
});
it('accepts bare account-session ("root") tokens', () => {
// Privileged ("godmode") apps run on the user's own session token
// rather than an app token, so the AI drivers can't distinguish
// them from a browser session and have to admit both.
expect(m.noUserSession).toBe(false);
});
});
describe('non-AI drivers — session tokens stay allowed', () => {
it.each([
['KVStoreDriver', () => new KVStoreDriver(...fake())],
['AppDriver', () => new AppDriver(...fake())],
['SubdomainDriver', () => new SubdomainDriver(...fake())],
['NotificationDriver', () => new NotificationDriver(...fake())],
])('%s does not set noUserSession', (_name, build) => {
expect(meta(build()).noUserSession).toBe(false);
});
});
// ── Iface coordination cross-check ──────────────────────────────────
+6
View File
@@ -223,6 +223,9 @@ export class KVStoreDriver extends PuterDriver {
limit?: number;
cursor?: string | Record<string, unknown>;
pattern?: string;
offset?: number;
includeTotal?: boolean;
fetchUntilFull?: boolean;
optConfig?: { appUuid?: string };
}): Promise<unknown> {
const opts = this.#opts(args.optConfig?.appUuid);
@@ -232,6 +235,9 @@ export class KVStoreDriver extends PuterDriver {
limit: args.limit,
cursor: args.cursor,
pattern: args.pattern,
offset: args.offset,
includeTotal: args.includeTotal,
fetchUntilFull: args.fetchUntilFull,
},
opts,
);
+120 -43
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import type { Readable } from 'node:stream';
@@ -57,6 +58,7 @@ export const DRIVER_DEFAULT_KEY = '__driverDefault' as const;
export const DRIVER_ALIASES_KEY = '__driverAliases' as const;
export const DRIVER_RATE_LIMIT_KEY = '__driverRateLimit' as const;
export const DRIVER_CONCURRENT_KEY = '__driverConcurrent' as const;
export const DRIVER_NO_USER_SESSION_KEY = '__driverNoUserSession' as const;
// -- Driver rate-limit config ----------------------------------------
//
@@ -77,24 +79,24 @@ export interface DriverRateLimitSpec {
/** Window length, in milliseconds. */
window: number;
/**
* Per-subscription overrides for `limit`. Keyed by
* `SubscriptionPolicy.id` (`user_free`, `temp_free`, `unlimited`,
* etc.). Falls back to `limit` when the actor's subscription isn't
* in the map. Same mechanic as `DriverConcurrentSpec.bySubscription`.
* Per-subscription overrides for `limit`. Keyed by `SubscriptionPolicy.id`
* (`user_free`, `temp_free`, `unlimited`, etc.). Falls back to `limit` when
* the actor's subscription isn't in the map. Same mechanic as
* `DriverConcurrentSpec.bySubscription`.
*/
bySubscription?: Record<string, number>;
/**
* Storage backend to count against. Omit to use the server-wide
* default configured by `config.rate_limit.backend`.
* Storage backend to count against. Omit to use the server-wide default
* configured by `config.rate_limit.backend`.
*/
backend?: RateLimitBackend;
}
export interface DriverRateLimitConfig {
/**
* Applied to any method not listed in `methods`. Lets a driver opt
* the whole interface into tighter limits than the global driver
* default without enumerating every method.
* Applied to any method not listed in `methods`. Lets a driver opt the
* whole interface into tighter limits than the global driver default
* without enumerating every method.
*/
default?: DriverRateLimitSpec;
/** Per-method overrides. Keys are driver method names. */
@@ -102,10 +104,9 @@ export interface DriverRateLimitConfig {
}
/**
* Validate a `rateLimit` block declared by a driver. Throws on bad
* shape so registration fails loudly at boot rather than silently
* misconfiguring production traffic. Returns the value unchanged on
* success for chaining.
* Validate a `rateLimit` block declared by a driver. Throws on bad shape so
* registration fails loudly at boot rather than silently misconfiguring
* production traffic. Returns the value unchanged on success for chaining.
*/
export function validateDriverRateLimit(
value: unknown,
@@ -182,9 +183,9 @@ function validateBySubscription(value: unknown, label: string): void {
}
/**
* Resolve the spec that applies to a given method on a driver. Per-method
* entry wins over `default`; returns `undefined` if neither is set so the
* caller can apply its own fallback.
* Resolve the spec that applies to a given method on a driver. Per-method entry
* wins over `default`; returns `undefined` if neither is set so the caller can
* apply its own fallback.
*/
export function resolveDriverMethodRateLimit(
cfg: DriverRateLimitConfig | undefined,
@@ -204,15 +205,15 @@ export interface DriverConcurrentSpec {
/** Maximum simultaneous in-flight requests. */
limit: number;
/**
* Per-subscription overrides keyed by `SubscriptionPolicy.id`
* (`user_free`, `temp_free`, `unlimited`, etc.). Falls back to
* `limit` when the actor's subscription isn't in the map.
* Per-subscription overrides keyed by `SubscriptionPolicy.id` (`user_free`,
* `temp_free`, `unlimited`, etc.). Falls back to `limit` when the actor's
* subscription isn't in the map.
*/
bySubscription?: Record<string, number>;
/**
* Storage backend. Memory is per-process (use only on single-node
* deployments); redis coordinates across nodes; kv is rarely the
* right choice for concurrency but supported for parity.
* deployments); redis coordinates across nodes; kv is rarely the right
* choice for concurrency but supported for parity.
*/
backend?: RateLimitBackend;
}
@@ -225,8 +226,8 @@ export interface DriverConcurrentConfig {
}
/**
* Validate a `concurrent` block. Mirrors `validateDriverRateLimit` —
* throws with a labelled path so a malformed entry surfaces at boot.
* Validate a `concurrent` block. Mirrors `validateDriverRateLimit` — throws
* with a labelled path so a malformed entry surfaces at boot.
*/
export function validateDriverConcurrent(
value: unknown,
@@ -283,10 +284,10 @@ function validateConcurrentSpec(value: unknown, label: string): void {
}
/**
* Resolve the concurrent spec for a given method on a driver. Same
* precedence as `resolveDriverMethodRateLimit`: per-method wins over
* `default`; `undefined` means no concurrency cap is declared, and the
* caller should leave the method unbounded.
* Resolve the concurrent spec for a given method on a driver. Same precedence
* as `resolveDriverMethodRateLimit`: per-method wins over `default`;
* `undefined` means no concurrency cap is declared, and the caller should leave
* the method unbounded.
*/
export function resolveDriverMethodConcurrent(
cfg: DriverConcurrentConfig | undefined,
@@ -319,22 +320,30 @@ export interface DriverMeta {
/**
* Rate-limit policy for this driver. Per-method specs override the
* `default` spec; both are optional. `DriverController` consults this
* before invoking the method and falls back to the global driver
* default (600/min) if nothing is declared.
* before invoking the method and falls back to the global driver default
* (600/min) if nothing is declared.
*/
rateLimit?: DriverRateLimitConfig;
/**
* Concurrent in-flight policy for this driver. When set, the
* controller acquires a slot before invoking the method and releases
* in `finally`. Absent → no concurrency cap (current behaviour).
* Concurrent in-flight policy for this driver. When set, the controller
* acquires a slot before invoking the method and releases in `finally`.
* Absent → no concurrency cap (current behaviour).
*/
concurrent?: DriverConcurrentConfig;
/**
* When true, `/drivers/call` rejects bare account-session ("root") tokens
* for this driver: callers must present an app/worker token or a
* dashboard-minted API token. Per-driver counterpart of the `noUserSession`
* route option — the dispatch route is shared, so the flag lives on the
* driver.
*/
noUserSession?: boolean;
}
/**
* Extract driver metadata from a driver instance. Checks decorator-set
* prototype metadata first, then falls back to instance properties.
* Returns `null` if the driver doesn't declare an interface.
* prototype metadata first, then falls back to instance properties. Returns
* `null` if the driver doesn't declare an interface.
*/
export function resolveDriverMeta(
driver: WithLifecycle & Record<string, unknown>,
@@ -384,6 +393,11 @@ export function resolveDriverMeta(
);
}
const noUserSession =
(proto[DRIVER_NO_USER_SESSION_KEY] as boolean | undefined) ??
(driver.noUserSession as boolean | undefined) ??
false;
if (!interfaceName || !driverName) return null;
return {
@@ -393,5 +407,68 @@ export function resolveDriverMeta(
aliases,
rateLimit,
concurrent,
noUserSession,
};
}
/**
* Framework/lifecycle method names that must never be reachable via
* `/drivers/call`. These live on `PuterDriver` (see `drivers/types.ts`) and are
* the machinery the dispatch surface must exclude. For class-based drivers a
* concrete `override` of one still carries the same name and is caught here;
* for plain-object drivers (registered by extensions — see `server.ts`, `typeof
* DriverClass === 'object'`) there is no base prototype to distinguish them, so
* this denylist is the _only_ thing keeping a hook off the RPC surface. Any
* lifecycle hook added to `PuterDriver` must be added here too — the per-driver
* guard test (`callableMethods.test.ts`) fails loudly if a base method starts
* leaking into every driver's surface.
*/
export const RESERVED_DRIVER_METHODS: ReadonlySet<string> = new Set([
'onServerStart',
'onServerPrepareShutdown',
'onServerShutdown',
'getReportedCosts',
]);
/**
* Compute the set of method names a driver exposes over `/drivers/call`.
*
* The RPC surface is defined structurally rather than by a hand-maintained
* per-method allow-list. Walking from the instance up to (but not including)
* `Object.prototype`, a name is callable iff it resolves to a function and is
* neither `constructor` nor a `RESERVED_DRIVER_METHODS` entry. This covers both
* driver shapes the server accepts (`server.ts`): class instances (RPC methods
* on the concrete prototype, config on the instance) and plain objects
* (everything own, used verbatim by extensions). It excludes all
* `Object.prototype` members (`toString`, `valueOf`, `__proto__`, …), the
* `constructor`, and the lifecycle hooks.
*
* `#`-private helpers need no handling: they are not real property keys, so
* `getOwnPropertyNames` never lists them and `driver['#x']` is `undefined`.
* Only _plain_ public methods can appear here.
*
* Getters are excluded (we read the descriptor's `.value`, never access the
* property), so evaluating this set never runs driver code. Intended to be
* called once per driver at registration and cached — not on the hot path.
*/
export function resolveCallableMethods(driver: object): Set<string> {
const callable = new Set<string>();
const seen = new Set<string>();
for (
let o: object | null = driver;
o && o !== Object.prototype;
o = Object.getPrototypeOf(o) as object | null
) {
for (const name of Object.getOwnPropertyNames(o)) {
// First (lowest) definition wins — a subclass override shadows
// the base, and we decide against the resolved descriptor.
if (seen.has(name)) continue;
seen.add(name);
if (name === 'constructor') continue;
if (RESERVED_DRIVER_METHODS.has(name)) continue;
const desc = Object.getOwnPropertyDescriptor(o, name);
if (desc && typeof desc.value === 'function') callable.add(name);
}
}
return callable;
}
@@ -332,6 +332,103 @@ describe('SubdomainDriver.read / select', () => {
});
});
// -- select pagination --
describe('SubdomainDriver.select pagination', () => {
const makeSubs = async (count: number) => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const subs: string[] = [];
for (let i = 0; i < count; i++) {
const sub = uniqueSubdomain(`page${i}`);
subs.push(sub);
await withActor(actor, () =>
driver.create({
object: { subdomain: sub, root_dir: `/${username}/Public` },
}),
);
}
return { actor, subs };
};
it('keeps the bare array response for plain limit requests', async () => {
const { actor } = await makeSubs(2);
const result = await withActor(actor, () =>
driver.select({ limit: 10 }),
);
expect(Array.isArray(result)).toBe(true);
});
it('pages through subdomains with cursors', async () => {
const { actor, subs } = await makeSubs(3);
const seen: string[] = [];
let cursor: string | null | undefined = null;
do {
const page = (await withActor(actor, () =>
driver.select({ limit: 2, cursor }),
)) as { items: Array<{ subdomain: string }>; cursor?: string };
seen.push(...page.items.map((r) => r.subdomain));
cursor = page.cursor;
} while (cursor);
expect(seen.sort()).toEqual([...subs].sort());
});
it('supports offset paging', async () => {
const { actor, subs } = await makeSubs(3);
const page = (await withActor(actor, () =>
driver.select({ limit: 10, offset: 1 }),
)) as { items: Array<{ subdomain: string }> };
expect(page.items.length).toBe(subs.length - 1);
});
it('rejects cursor combined with offset', async () => {
const { actor } = await makeSubs(2);
const first = (await withActor(actor, () =>
driver.select({ limit: 1, cursor: null }),
)) as { cursor?: string };
expect(first.cursor).toBeDefined();
await expect(
withActor(actor, () =>
driver.select({ offset: 1, cursor: first.cursor }),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('excludes worker-backed subdomains from listings and totals', async () => {
const { actor, subs } = await makeSubs(2);
await server.stores.subdomain.create({
userId: actor.user!.id as number,
subdomain: `workers.puter.wk-${Date.now()}`,
rootDirId: null,
associatedAppId: null,
appOwner: null,
});
const bare = (await withActor(actor, () =>
driver.select({}),
)) as Array<{ subdomain: string }>;
expect(bare.map((r) => r.subdomain).sort()).toEqual([...subs].sort());
const page = (await withActor(actor, () =>
driver.select({ limit: 10, cursor: null, includeTotal: true }),
)) as { items: Array<{ subdomain: string }>; total?: number };
expect(page.items.map((r) => r.subdomain).sort()).toEqual(
[...subs].sort(),
);
expect(page.total).toBe(subs.length);
});
it('reports total scoped to the actor with includeTotal', async () => {
const { actor, subs } = await makeSubs(3);
await makeSubs(2); // another user's rows must not count
const page = (await withActor(actor, () =>
driver.select({ limit: 1, includeTotal: true }),
)) as { items: unknown[]; total?: number };
expect(page.items.length).toBe(1);
expect(page.total).toBe(subs.length);
});
});
// ── update ──────────────────────────────────────────────────────────
describe('SubdomainDriver.update', () => {
@@ -32,6 +32,13 @@ import type { DriverRateLimitConfig } from '../meta.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import type { UserRow } from '../../stores/user/UserStore.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import { WORKER_SUBDOMAIN_PREFIX } from '../../stores/subdomain/SubdomainStore.js';
import {
decodeCursor,
encodeCursor,
normalizeLimit,
normalizeOffset,
} from '../../util/pagination.js';
const SUBDOMAIN_MAX_LEN = 64;
const SUBDOMAIN_REGEX = /^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$/;
@@ -171,12 +178,24 @@ export class SubdomainDriver extends PuterDriver {
return shaped ?? null;
}
async select(args: Record<string, unknown>): Promise<unknown[]> {
async select(args: Record<string, unknown>): Promise<unknown> {
const actor = this.#requireActor();
this.#requireUser(actor);
const predicate = args.predicate as unknown[] | string | undefined;
const limit = Math.min(Number(args.limit ?? 5000), 5000);
const limit = normalizeLimit(args.limit, { cap: 5000 }) ?? 5000;
const offset = normalizeOffset(args.offset);
const hasCursor = Object.prototype.hasOwnProperty.call(args, 'cursor');
const payload = decodeCursor(
args.cursor as string | null | undefined,
) as { id?: number } | undefined;
if (payload && offset !== undefined) {
throw new HttpError(400, 'cursor and offset cannot be combined', {
legacyCode: 'bad_request',
});
}
const includeTotal = args.includeTotal === true;
const paginated = hasCursor || offset !== undefined || includeTotal;
// Match v1: when the actor has `read-all-subdomains` (admin /
// privileged accounts), widen to every subdomain. Without this
@@ -186,19 +205,55 @@ export class SubdomainDriver extends PuterDriver {
predicate !== 'user-can-edit' &&
(await this.#hasPermission(actor, 'read-all-subdomains'));
let rows = widenToAll
? await this.stores.subdomain.listAll({ limit })
: await this.stores.subdomain.listByUserId(actor.user.id, {
limit,
});
// App actors only see subdomains they own; read-all bypasses scoping.
const appOwner = !widenToAll && actor.app ? actor.app.id : undefined;
// Worker deployments live in the same table but aren't sites —
// they're listed through the workers driver instead.
const listOpts = {
limit: paginated ? limit + 1 : limit,
offset,
afterId: payload?.id !== undefined ? Number(payload.id) : undefined,
appOwner,
excludePrefix: WORKER_SUBDOMAIN_PREFIX,
};
// App-limited: app actors only see subdomains they own. Skip when
// we widened above — read-all is meant to bypass scoping.
if (!widenToAll && actor.app) {
rows = rows.filter((r) => r.app_owner === actor.app!.id);
let rows = (
widenToAll
? await this.stores.subdomain.listAll(listOpts)
: await this.stores.subdomain.listByUserId(
actor.user.id,
listOpts,
)
) as Array<Record<string, unknown>>;
let cursor: string | undefined;
if (paginated && rows.length > limit) {
rows = rows.slice(0, limit);
const last = rows[rows.length - 1]!;
cursor = encodeCursor({ id: Number(last.id) });
}
return this.#hydrateRows(rows as Array<Record<string, unknown>>);
const items = await this.#hydrateRows(rows);
if (!paginated) return items;
let total: number | undefined;
if (includeTotal) {
total = widenToAll
? await this.stores.subdomain.count({
excludePrefix: WORKER_SUBDOMAIN_PREFIX,
})
: await this.stores.subdomain.count({
userId: actor.user.id,
appOwner,
excludePrefix: WORKER_SUBDOMAIN_PREFIX,
});
}
return {
items,
...(cursor ? { cursor } : {}),
...(total !== undefined ? { total } : {}),
};
}
async update(args: Record<string, unknown>): Promise<unknown> {
@@ -244,9 +299,11 @@ export class SubdomainDriver extends PuterDriver {
if (object.domain !== undefined)
patch.domain = object.domain != null ? String(object.domain) : null;
const updated = await this.stores.subdomain.update(row.uuid, patch, {
userId: row.user_id,
});
const updated = await this.stores.subdomain.update(
String(row.uuid),
patch,
{ userId: row.user_id as number },
);
const [shaped] = await this.#hydrateRows(
updated ? [updated as Record<string, unknown>] : [],
);
@@ -351,8 +408,8 @@ export class SubdomainDriver extends PuterDriver {
}
await this.#checkWriteAccess(row, actor);
await this.stores.subdomain.deleteByUuid(row.uuid, {
userId: row.user_id,
await this.stores.subdomain.deleteByUuid(String(row.uuid), {
userId: row.user_id as number,
});
try {
+7
View File
@@ -102,6 +102,13 @@ export const PuterDriver = class PuterDriver implements WithCostsReporting {
* See `DriverConcurrentConfig` in `./meta` for the shape.
*/
declare readonly concurrent?: DriverConcurrentConfig;
/**
* When true, `/drivers/call` rejects bare account-session ("root")
* tokens for this driver — callers need an app/worker token or a
* dashboard-minted API token. Set by `@Driver({ noUserSession: true })`
* or declared imperatively. See `DriverMeta.noUserSession` in `./meta`.
*/
declare readonly noUserSession?: boolean;
constructor(
protected config: IConfig,
@@ -0,0 +1,37 @@
// Focused unit coverage for hot-reload subscription idempotency. The main
// WorkerDriver.test.ts boots a full server where workers aren't Cloudflare-
// configured (so listeners never register); here we drive onServerStart
// directly with a local-server config and a spy event client.
//
// Regression guard for the /drivers/call exposure fix: onServerStart was
// remotely invokable, and each invocation used to stack another set of
// fs.* listeners — so a single caller could multiply every user's
// worker-source save into N edge redeploys. Dispatch is now gated, but the
// subscription must also be structurally idempotent regardless.
import { describe, expect, it, vi } from 'vitest';
import { WorkerDriver } from './WorkerDriver.js';
const build = () => {
const on = vi.fn();
const clients = { event: { on } } as any;
const config = { workers: { localServer: true } } as any;
const driver = new WorkerDriver(config, clients, {} as any, {} as any);
return { driver, on };
};
describe('WorkerDriver hot-reload subscription', () => {
it('registers each fs listener exactly once across repeated onServerStart calls', () => {
const { driver, on } = build();
driver.onServerStart();
driver.onServerStart();
driver.onServerStart();
expect(on).toHaveBeenCalledTimes(3);
expect(on.mock.calls.map((c) => c[0]).sort()).toEqual([
'fs.move.node',
'fs.remove.node',
'fs.write.file',
]);
});
});
@@ -212,6 +212,95 @@ describe('WorkerDriver', () => {
});
});
describe('getFilePaths pagination', () => {
let nextUserId = 90_000;
const seedWorkers = async (count: number) => {
const userId = nextUserId++;
const workerActor = makeActor({
user: {
uuid: `wk-user-${userId}`,
id: userId,
username: `wk-user-${userId}`,
email: `wk-${userId}@test.com`,
email_confirmed: true,
},
app: undefined,
});
const names: string[] = [];
for (let i = 0; i < count; i++) {
const name = `wk${userId}n${i}`;
names.push(name);
await server.stores.subdomain.create({
userId,
subdomain: `workers.puter.${name}`,
rootDirId: null,
associatedAppId: null,
appOwner: null,
});
}
return { workerActor, names };
};
it('keeps the bare array response when no pagination params are given', async () => {
const { workerActor, names } = await seedWorkers(3);
const res = (await inCtx(
() => target.getFilePaths({}),
workerActor,
)) as Array<{ name: string }>;
expect(Array.isArray(res)).toBe(true);
expect(res.map((r) => r.name)).toEqual(names);
});
it('returns the envelope and pages with cursors when limit is given', async () => {
const { workerActor, names } = await seedWorkers(5);
const seen: string[] = [];
let cursor: string | null | undefined = null;
do {
const page = (await inCtx(
() => target.getFilePaths({ limit: 2, cursor }),
workerActor,
)) as { items: Array<{ name: string }>; cursor?: string };
seen.push(...page.items.map((r) => r.name));
cursor = page.cursor;
} while (cursor);
expect(seen).toEqual(names);
});
it('supports offset paging and totals', async () => {
const { workerActor, names } = await seedWorkers(4);
const page = (await inCtx(
() =>
target.getFilePaths({
limit: 10,
offset: 1,
includeTotal: true,
}),
workerActor,
)) as { items: Array<{ name: string }>; total?: number };
expect(page.items.map((r) => r.name)).toEqual(names.slice(1));
expect(page.total).toBe(names.length);
});
it('rejects cursor combined with offset', async () => {
const { workerActor } = await seedWorkers(2);
const first = (await inCtx(
() => target.getFilePaths({ limit: 1, cursor: null }),
workerActor,
)) as { cursor?: string };
expect(first.cursor).toBeDefined();
await expect(
inCtx(
() =>
target.getFilePaths({
offset: 1,
cursor: first.cursor,
}),
workerActor,
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
describe('getLoggingUrl', () => {
it('returns null when loggingUrl is not configured', async () => {
const res = await inCtx(() => target.getLoggingUrl());
+119 -21
View File
@@ -17,7 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readFileSync } from 'node:fs';
import { existsSync, readFileSync } from 'node:fs';
import path from 'node:path';
import type { EventMetadata } from '../../clients/event/types.js';
import type { Actor } from '../../core/actor.js';
@@ -25,14 +25,24 @@ import { Context } from '../../core/context.js';
import { HttpError, type LegacyErrorCodes } from '../../core/http/HttpError.js';
import { assertVerifiedEmail } from '../../core/http/verifiedEmail.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import {
WORKER_SUBDOMAIN_PREFIX,
type SubdomainRow,
} from '../../stores/subdomain/SubdomainStore.js';
import { PuterDriver } from '../types.js';
import { loadFileInput } from '../util/fileInput.js';
import {
decodeCursor,
encodeCursor,
normalizeLimit,
normalizeOffset,
} from '../../util/pagination.js';
const CF_BASE_URL = 'https://api.cloudflare.com/client/v4/accounts';
const WORKER_NAME_REGEX = /^[a-zA-Z0-9_-]+$/;
const MAX_WORKERS_PER_USER = 100;
const MAX_SOURCE_SIZE = 10 * 1024 * 1024; // 10 MB
const WORKER_SUBDOMAIN_PREFIX = 'workers.puter.';
let USE_LOCAL_WORKERD = false;
// -- Preamble --------------------------------------------------------
//
@@ -45,10 +55,17 @@ let preambleError = false;
let preambleLineCount = 0;
let preambleVersion: string | null = null;
try {
const preamblePath = path.join(
__dirname,
'../../../../../src/worker/dist/workerPreamble.js',
);
// Five levels up from `dist/src/backend/drivers/workers` (compiled
// runtime); four when running from `src/backend/drivers/workers`
// directly (vitest transforms the TS sources in place).
const preamblePath = [
path.join(
__dirname,
'../../../../../src/worker/dist/workerPreamble.js',
),
path.join(__dirname, '../../../../src/worker/dist/workerPreamble.js'),
].find((candidate) => existsSync(candidate));
if (!preamblePath) throw new Error('workerPreamble.js not found');
console.log('reading: ' + preamblePath);
preamble = readFileSync(preamblePath, 'utf-8');
preambleLineCount = preamble.split('\n').length - 1;
@@ -66,6 +83,16 @@ try {
preambleError = true;
}
/**
* The puter.js/router preamble prepended to every worker's source before
* deploy. Exposed so the local-workerd path (`LocalWorkerService`) can build
* the same `preamble + sourceCode` script when it lazily re-deploys a worker
* into Miniflare after a server restart.
*/
export function getWorkerPreamble(): string {
return preamble;
}
/**
* Driver exposing the `workers` interface — Cloudflare Workers
* deployment, lifecycle, and file-path queries.
@@ -83,6 +110,7 @@ export class WorkerDriver extends PuterDriver {
readonly isDefault = true;
#cfBaseUrl = '';
#hotReloadSubscribed = false;
static currentPreambleVersion(): string | null {
return preambleVersion;
@@ -100,6 +128,8 @@ export class WorkerDriver extends PuterDriver {
'[workers] preamble not build but workers configured to be enabled. Halting start',
);
}
} else if (cfg.localServer) {
USE_LOCAL_WORKERD = true;
}
this.#subscribeHotReload();
}
@@ -296,11 +326,31 @@ export class WorkerDriver extends PuterDriver {
return cfResult;
}
async getFilePaths(args: Record<string, unknown>): Promise<unknown[]> {
async getFilePaths(args: Record<string, unknown>): Promise<unknown> {
const actor = this.#requireActor();
const workerName = args.workerName as string | undefined;
let rows: Array<Record<string, unknown>>;
const limit = normalizeLimit(args.limit, { cap: 5000 });
const offset = normalizeOffset(args.offset);
const hasCursor = Object.prototype.hasOwnProperty.call(args, 'cursor');
const payload = decodeCursor(
args.cursor as string | null | undefined,
) as { id?: number } | undefined;
if (payload && offset !== undefined) {
throw new HttpError(400, 'cursor and offset cannot be combined', {
legacyCode: 'bad_request',
});
}
const includeTotal = args.includeTotal === true;
const paginated =
hasCursor ||
limit !== undefined ||
offset !== undefined ||
includeTotal;
const pageSize = limit ?? 500;
let rows: SubdomainRow[];
let cursor: string | undefined;
if (typeof workerName === 'string' && workerName.length > 0) {
const sub = await this.stores.subdomain.getBySubdomain(
`${WORKER_SUBDOMAIN_PREFIX}${workerName}`,
@@ -310,8 +360,26 @@ export class WorkerDriver extends PuterDriver {
rows = await this.stores.subdomain.listByUserIdAndPrefix(
actor.user.id,
WORKER_SUBDOMAIN_PREFIX,
actor.app ? { appId: actor.app.id } : {},
{
...(actor.app ? { appId: actor.app.id } : {}),
...(paginated
? {
limit: pageSize + 1,
offset,
afterId:
payload?.id !== undefined
? Number(payload.id)
: undefined,
}
: {}),
},
);
if (paginated && rows.length > pageSize) {
rows = rows.slice(0, pageSize);
cursor = encodeCursor({
id: Number(rows[rows.length - 1]!.id),
});
}
}
const rootDirIds = rows
@@ -330,7 +398,7 @@ export class WorkerDriver extends PuterDriver {
});
}
return rows.map((r) => {
const items = rows.map((r) => {
const name =
String(r.subdomain ?? '')
.split('.')
@@ -352,6 +420,23 @@ export class WorkerDriver extends PuterDriver {
: null,
};
});
if (!paginated) return items;
let total: number | undefined;
if (includeTotal) {
total = await this.stores.subdomain.countByUserIdAndPrefix(
actor.user.id,
WORKER_SUBDOMAIN_PREFIX,
actor.app ? { appId: actor.app.id } : {},
);
}
return {
items,
...(cursor ? { cursor } : {}),
...(total !== undefined ? { total } : {}),
};
}
async getLoggingUrl(): Promise<string | null> {
@@ -365,6 +450,13 @@ export class WorkerDriver extends PuterDriver {
authorization: string,
code: string,
): Promise<Record<string, unknown>> {
if (USE_LOCAL_WORKERD) {
return this.services.localworkerservice.cfDeployLocal(
workerName,
authorization,
code,
);
}
const cfg = this.#workerConfig();
const metadata = JSON.stringify({
body_part: 'swCode',
@@ -430,6 +522,9 @@ export class WorkerDriver extends PuterDriver {
}
async #cfDelete(workerName: string): Promise<Record<string, unknown>> {
if (USE_LOCAL_WORKERD) {
return this.services.localworkerservice.cfDeleteLocal(workerName);
}
const cfg = this.#workerConfig();
const res = await fetch(`${this.#cfBaseUrl}/scripts/${workerName}/`, {
method: 'DELETE',
@@ -455,7 +550,7 @@ export class WorkerDriver extends PuterDriver {
#requireCfConfig(): void {
const cfg = this.#workerConfig();
if (!cfg.XAUTHKEY || !cfg.ACCOUNTID) {
if ((!cfg.XAUTHKEY || !cfg.ACCOUNTID) && !cfg.localServer) {
throw new HttpError(503, 'Cloudflare Workers not configured', {
legacyCode: 'response_timeout',
});
@@ -472,7 +567,7 @@ export class WorkerDriver extends PuterDriver {
}
#checkWorkerWriteAccess(
row: Record<string, unknown>,
row: SubdomainRow,
actor: Actor & { user: { id: number } },
errorStatus: number,
errorMessage: string,
@@ -527,7 +622,12 @@ export class WorkerDriver extends PuterDriver {
// while worker subdomains are keyed to the numeric fsentries.id.
#subscribeHotReload(): void {
if (!this.#cfBaseUrl) return; // CF not configured — skip
if (!this.#cfBaseUrl && !USE_LOCAL_WORKERD) return;
// Idempotent: re-entry (e.g. a second onServerStart) must not stack
// duplicate listeners — each duplicate would multiply redeploys on
// every user's worker-source save.
if (this.#hotReloadSubscribed) return;
this.#hotReloadSubscribed = true;
this.clients.event.on(
'fs.write.file',
@@ -715,14 +815,12 @@ export class WorkerDriver extends PuterDriver {
);
}
async #listWorkerRowsForEntry(
entry: FSEntry,
): Promise<Array<Record<string, unknown>>> {
async #listWorkerRowsForEntry(entry: FSEntry): Promise<SubdomainRow[]> {
const workerSubs = await this.stores.subdomain.listByUserIdAndPrefix(
entry.userId,
WORKER_SUBDOMAIN_PREFIX,
);
return workerSubs.filter((r: Record<string, unknown>) => {
return workerSubs.filter((r) => {
return (
String(r.root_dir_id) === String(entry.id) ||
String(r.root_dir_id) === String(entry.uuid) ||
@@ -734,17 +832,17 @@ export class WorkerDriver extends PuterDriver {
async #listWorkerRowsUnderPath(
userId: number,
parentPath: string,
): Promise<Array<Record<string, unknown>>> {
): Promise<SubdomainRow[]> {
const workerSubs = await this.stores.subdomain.listByUserIdAndPrefix(
userId,
WORKER_SUBDOMAIN_PREFIX,
);
const rootDirIds = workerSubs
.map((r: Record<string, unknown>) => r.root_dir_id)
.map((r) => r.root_dir_id)
.filter((id): id is number => typeof id === 'number');
const entriesById =
await this.stores.fsEntry.getEntriesByIds(rootDirIds);
return workerSubs.filter((row: Record<string, unknown>) => {
return workerSubs.filter((row) => {
const rootDirId = row.root_dir_id;
if (typeof rootDirId !== 'number') return false;
const entry = entriesById.get(rootDirId);
@@ -761,7 +859,7 @@ export class WorkerDriver extends PuterDriver {
}
async #deleteWorkerForSourceRow(
row: Record<string, unknown>,
row: SubdomainRow,
userId: number,
): Promise<void> {
const workerFullName = String(row.subdomain ?? '');
+19
View File
@@ -19,6 +19,7 @@
import { existsSync, readFileSync } from 'node:fs';
import path from 'node:path';
import { isSpanContextValid, trace } from '@opentelemetry/api';
import { puterClients } from './clients';
import { puterControllers } from './controllers';
import { puterDrivers } from './drivers';
@@ -26,6 +27,7 @@ import { PuterServer } from './server';
import { puterServices } from './services';
import { puterStores } from './stores';
import type { IConfig } from './types';
import { installJsonConsole } from './util/jsonConsole.js';
// Config resolution order:
// 1. `process.env.PUTER_CONFIG_PATH` — absolute path to a config file. Used
@@ -167,6 +169,23 @@ const loadConfig = (): IConfig => {
// if called directly, start the server
if (require.main === module) {
const config = loadConfig();
// Structured logging: when `log_format: "json"`, replace the global console
// so each call emits one JSON line (level, timestamp, msg, and the active
// trace id) — one event per call, so a line-oriented log collector can't
// split stack traces across events. Installed here rather than in the OTel
// preload so it applies even when telemetry is disabled; the trace id is
// simply absent when no span is active.
if (config.log_format === 'json') {
installJsonConsole({
getTraceContext: () => {
const ctx = trace.getActiveSpan()?.spanContext();
if (!ctx || !isSpanContextValid(ctx)) return undefined;
return { traceId: ctx.traceId, spanId: ctx.spanId };
},
});
}
const server = new PuterServer(
config,
puterClients,
+1
View File
@@ -79,6 +79,7 @@
"@types/busboy": "^1.5.4",
"@types/jsonwebtoken": "^9.0.10",
"@types/node": "^24.0.0",
"@types/nodemailer": "^8.0.1",
"@types/pg": "^8.6.1",
"@types/validator": "^13.15.10",
"chai": "^4.3.7",
+201 -107
View File
@@ -39,6 +39,7 @@ import { isHttpError } from './core/http/HttpError';
import {
adminOnlyGate,
allowedAppIdsGate,
noUserSessionGate,
requireAuthGate,
requireVerifiedAccount,
requireNonAccessTokenGate,
@@ -46,6 +47,7 @@ import {
requireVerifiedGate,
subdomainGate,
} from './core/http/middleware/gates';
import { createStepUpGate } from './core/http/middleware/stepUpSession';
import { createNotFoundHandler } from './core/http/middleware/notFoundHandler';
import {
requireAntiCsrf,
@@ -62,6 +64,7 @@ import {
createUserSubdomainRedirect,
createNativeAppStatic,
} from './core/http/middleware/hostRedirects';
import { createLocalWorkerProxyMiddleware } from './core/http/middleware/localWorkerProxy';
import { createPuterSiteMiddleware } from './core/http/middleware/puterSite';
import { PuterRouter } from './core/http/PuterRouter';
import { createRouteLifecycleMiddleware } from './core/http/routeLifecycle';
@@ -454,6 +457,18 @@ export class PuterServer {
res.sendStatus(200);
});
// -- Local Worker proxy (*.workers.puter.localhost) ----------
// Dev-only Miniflare dispatch, gated on `config.workers.localServer`.
// Mounted BEFORE body parsing so the Worker receives the raw request
// stream; no-op in production (real Cloudflare via WorkerDriver).
this.#app.use(
createLocalWorkerProxyMiddleware(this.#config, {
clients: this.clients,
stores: this.stores,
services: this.services,
}),
);
// -- Body parsing (JSON + text-as-json shim) -----------------
const captureRawBody: NonNullable<
Parameters<typeof express.json>[0]
@@ -696,7 +711,9 @@ export class PuterServer {
* without `next(err)` ceremony.
*/
#installTerminalMiddleware() {
this.#app.use(createNotFoundHandler());
this.#app.use(
createNotFoundHandler({ guiDomain: this.#config.domain }),
);
this.#app.use(
createErrorHandler({
onError: (err, req) => {
@@ -835,7 +852,8 @@ export class PuterServer {
opts.requireUserActor ||
opts.adminOnly ||
opts.allowedAppIds ||
opts.requireVerified,
opts.requireVerified ||
opts.noUserSession,
);
if (needsAuth) {
mwChain.push(requireAuthGate());
@@ -865,10 +883,12 @@ export class PuterServer {
// carry, so it works for either actor shape.
//
// `adminOnly` also does NOT imply `requireUserActor`: admin endpoints
// should be callable from scripts/automation using an admin's access
// token, not only from browser sessions. `adminOnlyGate` gates on
// `actor.user.username`, which is populated for access-token and
// app-under-user actors alike.
// stay callable from scripts/automation using an admin's full-access
// token, not only from browser sessions — both are root tokens.
// Beyond the username check, `adminOnlyGate` requires a root token
// (rejecting an admin acting through a third-party app) unless the
// route is also appId-gated, in which case `allowedAppIdsGate` governs
// which apps may pass.
if (opts.requireUserActor) {
mwChain.push(
requireUserActorGate({
@@ -877,9 +897,34 @@ export class PuterServer {
);
}
// Bare user-session ("root" token) rejection. Runs after
// `requireUserActor` so that on routes combining both, an app is
// rejected with the user-actor message and only a bare session gets
// the "use an app or API token" message.
if (opts.noUserSession) {
mwChain.push(noUserSessionGate());
}
if (opts.adminOnly) {
const extras = Array.isArray(opts.adminOnly) ? opts.adminOnly : [];
mwChain.push(adminOnlyGate(extras));
mwChain.push(
adminOnlyGate(extras, {
appGated: Boolean(opts.allowedAppIds),
}),
);
// An admin username on a leaked session isn't enough — also require
// a recent re-authentication. Exempt only a token that carries one
// of the route's allowlisted app ids: an admin acting through an
// allowlisted app can't elevate (apps have no password/TOTP; see
// createStepUpGate). A root/human session — no app id in the token —
// still requires step-up, and `allowedAppIdsGate` still enforces the
// allowlist for the app path.
mwChain.push(
createStepUpGate({
tokenService: this.services.token,
allowedAppUids: opts.allowedAppIds,
}),
);
}
if (opts.allowedAppIds) {
@@ -1069,7 +1114,31 @@ export class PuterServer {
const entryPath = `${extDir}/${entry.name}`;
if (entry.isFile()) {
if (/\.(js|mjs|cjs)$/.test(entry.name)) {
const name = entry.name;
let shouldImport: boolean;
if (this.#config.import_ts_extensions) {
// Extensions ship as compiled .js at runtime, but
// transform-capable runtimes (the test harness)
// import the .ts sources directly. Skip tests and
// declarations, and skip built .js siblings of a
// .ts source so a previously-built tree doesn't
// double-register.
if (name.endsWith('.ts')) {
shouldImport =
!name.endsWith('.test.ts') &&
!name.endsWith('.d.ts');
} else {
shouldImport =
/\.(js|mjs|cjs)$/.test(name) &&
!/\.test\.(js|mjs|cjs)$/.test(name) &&
!existsSync(
entryPath.replace(/\.(js|mjs|cjs)$/, '.ts'),
);
}
} else {
shouldImport = /\.(js|mjs|cjs)$/.test(name);
}
if (shouldImport) {
console.log(`Importing extension file ${entryPath}`);
await import(pathToFileURL(entryPath).href);
}
@@ -1132,72 +1201,82 @@ export class PuterServer {
}
if (!noHttpServer) {
this.#server = httpServer.listen(this.#config.port, async () => {
const cfg = this.#config;
const liveUrl =
cfg.origin ??
`${cfg.protocol ?? 'http'}://${cfg.domain ?? 'localhost'}:${this.#config.port}`;
console.log(
'\n************************************************************',
);
console.log(`* Puter is now live at: ${liveUrl}`);
console.log(
'************************************************************\n',
);
await this.#fireOnServerStart();
console.log('PuterServer has fully booted.');
// CLI: `--server` (optionally `--puter-backend=<gui-origin>`)
// runs the AuthMe flow against a remote Puter (default
// puter.com), then opens the local GUI already logged in and
// pointed at that backend. Restores the v1 WebServerService
// `--server` behavior; works in any env. When set, it takes
// over browser launch so we don't also open a plain tab.
const { values: cliArgs } = parseArgs({
args: process.argv.slice(2),
options: {
server: { type: 'boolean' },
'puter-backend': { type: 'string' },
},
strict: false,
// Await 'listening' (and full boot below) so callers can rely on
// the server being reachable once start() resolves — test
// harnesses connect real clients immediately after.
this.#server = httpServer.listen(this.#config.port);
await new Promise<void>((resolve, reject) => {
const onError = (err: Error) => reject(err);
httpServer.once('error', onError);
httpServer.once('listening', () => {
// Detach so post-boot 'error' events aren't swallowed
// by a no-op reject on this settled promise.
httpServer.removeListener('error', onError);
resolve();
});
if (cliArgs.server) {
try {
// tools/auth_gui.js is not compiled into dist/, so
// resolve it from the package root (cwd, per the
// `start` script) rather than relative to this module.
const authGuiUrl = pathToFileURL(
path.resolve(process.cwd(), 'tools/auth_gui.js'),
).href;
const authGui = (await import(authGuiUrl)).default;
await authGui(
cliArgs['puter-backend'] as string | undefined,
);
} catch (e) {
console.log(
'[server] could not start AuthMe browser flow:',
(e as Error).message,
);
}
} else if (
this.#config.env === 'dev' &&
!cfg.no_browser_launch
) {
// Auto-launch the browser on dev boot (matches v1
// WebServerService). Opt out via `no_browser_launch: true`.
try {
const openModule = await import('open');
await openModule.default(liveUrl);
} catch (e) {
console.log(
'[server] could not auto-open browser:',
(e as Error).message,
);
}
}
});
const cfg = this.#config;
const liveUrl =
cfg.origin ??
`${cfg.protocol ?? 'http'}://${cfg.domain ?? 'localhost'}:${this.#config.port}`;
console.log(
'\n************************************************************',
);
console.log(`* Puter is now live at: ${liveUrl}`);
console.log(
'************************************************************\n',
);
await this.#fireOnServerStart();
console.log('PuterServer has fully booted.');
// CLI: `--server` (optionally `--puter-backend=<gui-origin>`)
// runs the AuthMe flow against a remote Puter (default
// puter.com), then opens the local GUI already logged in and
// pointed at that backend. Restores the v1 WebServerService
// `--server` behavior; works in any env. When set, it takes
// over browser launch so we don't also open a plain tab.
const { values: cliArgs } = parseArgs({
args: process.argv.slice(2),
options: {
server: { type: 'boolean' },
'puter-backend': { type: 'string' },
},
strict: false,
});
if (cliArgs.server) {
try {
// tools/auth_gui.js is not compiled into dist/, so
// resolve it from the package root (cwd, per the
// `start` script) rather than relative to this module.
const authGuiUrl = pathToFileURL(
path.resolve(process.cwd(), 'tools/auth_gui.js'),
).href;
const authGui = (await import(authGuiUrl)).default;
await authGui(
cliArgs['puter-backend'] as string | undefined,
);
} catch (e) {
console.log(
'[server] could not start AuthMe browser flow:',
(e as Error).message,
);
}
} else if (this.#config.env === 'dev' && !cfg.no_browser_launch) {
// Auto-launch the browser on dev boot (matches v1
// WebServerService). Opt out via `no_browser_launch: true`.
try {
const openModule = await import('open');
await openModule.default(liveUrl);
} catch (e) {
console.log(
'[server] could not auto-open browser:',
(e as Error).message,
);
}
}
} else {
this.#server = {
close: (cb: (error?: Error) => void | undefined) => {
@@ -1236,47 +1315,51 @@ export class PuterServer {
}
}
#prepareShutdownHooksRan = false;
/**
* Run every layer's `onServerPrepareShutdown` exactly once, whichever
* of `prepareShutdown()` / `shutdown()` gets there first.
*/
async #runPrepareShutdownHooks() {
if (this.#prepareShutdownHooksRan) return;
this.#prepareShutdownHooksRan = true;
for (const client of Object.values(this.clients) as WithLifecycle[]) {
if (client.onServerPrepareShutdown) {
await client.onServerPrepareShutdown();
}
}
for (const store of Object.values(this.stores) as WithLifecycle[]) {
if (store.onServerPrepareShutdown) {
await store.onServerPrepareShutdown();
}
}
for (const service of Object.values(this.services) as WithLifecycle[]) {
if (service.onServerPrepareShutdown) {
await service.onServerPrepareShutdown();
}
}
for (const controller of Object.values(
this.controllers,
) as WithLifecycle[]) {
if (controller.onServerPrepareShutdown) {
await controller.onServerPrepareShutdown();
}
}
for (const driver of Object.values(this.drivers) as WithLifecycle[]) {
if (driver.onServerPrepareShutdown) {
await driver.onServerPrepareShutdown();
}
}
}
async prepareShutdown() {
if (this.#server) {
this.#server.close(async () => {
console.log(
'PuterServer has stopped accepting new connections',
);
for (const client of Object.values(
this.clients,
) as WithLifecycle[]) {
if (client.onServerPrepareShutdown) {
await client.onServerPrepareShutdown();
}
}
for (const store of Object.values(
this.stores,
) as WithLifecycle[]) {
if (store.onServerPrepareShutdown) {
await store.onServerPrepareShutdown();
}
}
for (const service of Object.values(
this.services,
) as WithLifecycle[]) {
if (service.onServerPrepareShutdown) {
await service.onServerPrepareShutdown();
}
}
for (const controller of Object.values(
this.controllers,
) as WithLifecycle[]) {
if (controller.onServerPrepareShutdown) {
await controller.onServerPrepareShutdown();
}
}
for (const driver of Object.values(
this.drivers,
) as WithLifecycle[]) {
if (driver.onServerPrepareShutdown) {
await driver.onServerPrepareShutdown();
}
}
await this.#runPrepareShutdownHooks();
});
}
}
@@ -1284,7 +1367,18 @@ export class PuterServer {
async shutdown() {
if (this.#server) {
console.log('PuterServer is shutting down');
// Prepare hooks come first: SocketService's hook closes
// socket.io, disconnecting upgraded websocket connections that
// `closeAllConnections()` does not cover — without this,
// `close()` waits forever on any connected socket.io client.
await this.#runPrepareShutdownHooks();
// Stop accepting new connections, then sever live ones; the
// close callback fires once the listener is fully released.
const closed = new Promise<void>((resolve) => {
this.#server!.close(() => resolve());
});
this.#server.closeAllConnections();
await closed;
for (const client of Object.values(
this.clients,
) as WithLifecycle[]) {
@@ -32,6 +32,8 @@ const RECOMMENDED_APP_NAMES = [
'app-center',
'dev-center',
'butler',
'calculator',
'calendar',
'code',
'music-player',
'contacts',
@@ -0,0 +1,358 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
// ── Test harness ────────────────────────────────────────────────────
//
// `SuggestedAppsService` produces launch metadata that the GUI's
// default-open path (`open_item` → `launch_app({ app_obj })`) consumes
// *without* re-reading the app through AppDriver. That makes it a
// launch-metadata producer in its own right, so it carries the same
// hosted-backing guard — these tests pin that.
let server: PuterServer;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const uniqueName = (prefix: string) =>
`${prefix}-${Math.random().toString(36).slice(2, 10)}`;
const hostedUrl = (sub: string) => `https://${sub}.site.puter.localhost/`;
const makeUser = async (): Promise<{ userId: number }> => {
const username = `sa-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
return { userId: created.id };
};
/**
* Creates an app row already approved for opening items and associated
* with `ext` — that pairing is what puts a third-party app into the
* suggested list. `approved_for_opening_items` is in the store's
* read-only column set (it's an admin decision, not a dev-settable
* field), so set it with a direct write rather than through `create`.
*/
const makeOpenerApp = async ({
userId,
indexUrl,
ext,
}: {
userId: number;
indexUrl: string;
ext: string;
}): Promise<string> => {
const name = uniqueName('opener');
const app = await server.stores.app.create(
{ name, title: 'Opener', index_url: indexUrl },
{ ownerUserId: userId },
);
const appId = (app as { id: number }).id;
await server.clients.db.write(
'UPDATE `apps` SET `approved_for_opening_items` = 1 WHERE `id` = ?',
[appId],
);
// Stored verbatim and matched exactly against the bare, lowercased
// extension `SuggestedAppsService` derives from the filename.
await server.stores.app.setFiletypeAssociations(appId, [ext]);
return name;
};
/**
* Points a built-in opener app at `indexUrl`, creating the row if this
* environment didn't seed it. Some built-in names ship in the default-apps
* migration and some don't, so neither create nor update is safe alone.
*/
const pointBuiltinAt = async (
name: string,
userId: number,
indexUrl: string,
): Promise<void> => {
const existing = await server.stores.app.getByName(name);
if (existing) {
await server.stores.app.update((existing as { id: number }).id, {
index_url: indexUrl,
});
await server.clients.db.write(
'UPDATE `apps` SET `owner_user_id` = ? WHERE `id` = ?',
[userId, (existing as { id: number }).id],
);
return;
}
await server.stores.app.create(
{ name, title: name, index_url: indexUrl },
{ ownerUserId: userId },
);
};
const suggestFor = async (ext: string): Promise<Array<{ name?: unknown }>> =>
(await server.services.suggestedApps.getSuggestedApps({
name: `file.${ext}`,
path: `/x/file.${ext}`,
})) as Array<{ name?: unknown }>;
describe('SuggestedAppsService hosted-backing guard', () => {
it('suggests an opener app while its hosted subdomain is owned', async () => {
const { userId } = await makeUser();
const ext = uniqueName('ext1').replace(/-/g, '');
const sub = uniqueName('live');
await server.stores.subdomain.create({ userId, subdomain: sub });
const name = await makeOpenerApp({
userId,
indexUrl: hostedUrl(sub),
ext,
});
const suggested = await suggestFor(ext);
const entry = suggested.find((a) => a.name === name) as
| { index_url?: unknown }
| undefined;
expect(entry).toBeDefined();
expect(String(entry?.index_url)).toContain(sub);
});
it('drops an opener app whose hosted subdomain was deleted', async () => {
const { userId } = await makeUser();
const ext = uniqueName('ext2').replace(/-/g, '');
const sub = uniqueName('gone');
const row = await server.stores.subdomain.create({
userId,
subdomain: sub,
});
const name = await makeOpenerApp({
userId,
indexUrl: hostedUrl(sub),
ext,
});
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId },
);
const suggested = await suggestFor(ext);
expect(suggested.find((a) => a.name === name)).toBeUndefined();
});
it('drops an opener app whose hosted subdomain was reclaimed by another user', async () => {
const owner = await makeUser();
const attacker = await makeUser();
const ext = uniqueName('ext3').replace(/-/g, '');
const sub = uniqueName('reclaim');
const row = await server.stores.subdomain.create({
userId: owner.userId,
subdomain: sub,
});
const name = await makeOpenerApp({
userId: owner.userId,
indexUrl: hostedUrl(sub),
ext,
});
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId: owner.userId },
);
await server.stores.subdomain.create({
userId: attacker.userId,
subdomain: sub,
});
const suggested = await suggestFor(ext);
expect(suggested.find((a) => a.name === name)).toBeUndefined();
});
it('leaves apps on non-hosted index_urls alone', async () => {
const { userId } = await makeUser();
const ext = uniqueName('ext4').replace(/-/g, '');
const name = await makeOpenerApp({
userId,
indexUrl: 'https://someone-elses-domain.example/',
ext,
});
const suggested = await suggestFor(ext);
expect(suggested.find((a) => a.name === name)).toBeDefined();
});
it('fails closed when the subdomain lookup errors', async () => {
const { userId } = await makeUser();
const ext = uniqueName('ext5').replace(/-/g, '');
const sub = uniqueName('flaky');
await server.stores.subdomain.create({ userId, subdomain: sub });
const name = await makeOpenerApp({
userId,
indexUrl: hostedUrl(sub),
ext,
});
// A store failure must not be read as "backing is fine" — an
// unverifiable app is withheld rather than handed to the launcher.
const spy = vi
.spyOn(server.stores.subdomain, 'getBySubdomain')
.mockRejectedValue(new Error('db down'));
try {
const suggested = await suggestFor(ext);
expect(suggested.find((a) => a.name === name)).toBeUndefined();
} finally {
spy.mockRestore();
}
});
it('applies the guard on the batched multi-entry path too', async () => {
// `readdir` fans out through `getSuggestedAppsForEntries`, a
// separate entry point from `getSuggestedApps`. Both must gate.
const { userId } = await makeUser();
const liveExt = uniqueName('ext6').replace(/-/g, '');
const deadExt = uniqueName('ext7').replace(/-/g, '');
const liveSub = uniqueName('live');
await server.stores.subdomain.create({ userId, subdomain: liveSub });
const liveName = await makeOpenerApp({
userId,
indexUrl: hostedUrl(liveSub),
ext: liveExt,
});
const deadSub = uniqueName('dead');
const deadRow = await server.stores.subdomain.create({
userId,
subdomain: deadSub,
});
const deadName = await makeOpenerApp({
userId,
indexUrl: hostedUrl(deadSub),
ext: deadExt,
});
await server.stores.subdomain.deleteByUuid(
String((deadRow as { uuid: string }).uuid),
{ userId },
);
const [liveResult, deadResult] = (await server.services.suggestedApps.getSuggestedAppsForEntries(
[{ name: `a.${liveExt}` }, { name: `b.${deadExt}` }],
)) as Array<Array<{ name?: unknown }>>;
expect(liveResult.find((a) => a.name === liveName)).toBeDefined();
expect(deadResult.find((a) => a.name === deadName)).toBeUndefined();
});
// Built-ins enter the list by stable name rather than through
// `app_filetype_association`, which is a separate loop. In prod their
// index_urls aren't puter-hosted, but nothing structurally prevents it
// — so the guard has to cover that branch too. Split across two
// extensions because the per-extension cache would otherwise serve the
// first call's result to the second.
it('keeps a built-in-name opener whose hosted backing is live', async () => {
const { userId } = await makeUser();
const sub = uniqueName('builtinlive');
await server.stores.subdomain.create({ userId, subdomain: sub });
// 'markus' is the first built-in opener mapped to `.md`.
await pointBuiltinAt('markus', userId, hostedUrl(sub));
expect((await suggestFor('md')).map((a) => a.name)).toContain('markus');
});
it('drops a built-in-name opener whose hosted backing is gone', async () => {
const { userId } = await makeUser();
const sub = uniqueName('builtindead');
const row = await server.stores.subdomain.create({
userId,
subdomain: sub,
});
// 'pdf' is the sole built-in opener mapped to `.pdf`.
await pointBuiltinAt('pdf', userId, hostedUrl(sub));
await server.stores.subdomain.deleteByUuid(
String((row as { uuid: string }).uuid),
{ userId },
);
expect((await suggestFor('pdf')).map((a) => a.name)).not.toContain(
'pdf',
);
});
it('does not cache a failed resolve, so the next call retries', async () => {
// The per-extension cache holds the in-flight promise. A resolve
// that throws must drop its entry — otherwise one transient app
// store failure would withhold an app's suggestions for the full
// 5-minute TTL.
const { userId } = await makeUser();
const ext = uniqueName('ext9').replace(/-/g, '');
const name = await makeOpenerApp({
userId,
indexUrl: 'https://dev-owned-domain.example/',
ext,
});
const spy = vi
.spyOn(server.stores.app, 'getAppsByFiletype')
.mockRejectedValue(new Error('db down'));
await expect(suggestFor(ext)).rejects.toThrow('db down');
spy.mockRestore();
const suggested = await suggestFor(ext);
expect(suggested.find((a) => a.name === name)).toBeDefined();
});
it('does not hit the subdomain store for non-hosted index_urls', async () => {
// Built-ins and apps on a developer's own domain aren't on a
// hosting domain, so the guard short-circuits on the URL alone.
// This keeps the check off the readdir hot path — a regression
// here would add a DB round-trip per suggested app, per extension.
const { userId } = await makeUser();
const ext = uniqueName('ext8').replace(/-/g, '');
const name = await makeOpenerApp({
userId,
indexUrl: 'https://dev-owned-domain.example/',
ext,
});
const spy = vi.spyOn(server.stores.subdomain, 'getBySubdomain');
try {
const suggested = await suggestFor(ext);
expect(suggested.find((a) => a.name === name)).toBeDefined();
expect(spy).not.toHaveBeenCalled();
} finally {
spy.mockRestore();
}
});
});
@@ -3,22 +3,24 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { posix as pathPosix } from 'node:path';
import { getAppIconUrl } from '../../util/appIcon.js';
import { hostedIndexUrlBackingIsUnavailable } from '../../util/hostedAppBacking.js';
import { PuterService } from '../types.js';
// -- Extension → suggested app names mapping -------------------------
@@ -176,13 +178,13 @@ function extractExtension(entry: { name?: string; path?: string }): string {
}
/**
* Given a file entry (path, name, or extension), returns an ordered list
* of apps that can open it. Built-in apps come from the hardcoded map
* above; third-party apps come from the `app_filetype_association` table.
* Given a file entry (path, name, or extension), returns an ordered list of
* apps that can open it. Built-in apps come from the hardcoded map above;
* third-party apps come from the `app_filetype_association` table.
*
* Lookups cache per-extension (plus a separate per-app-name cache for the
* small set of built-in opener apps), so a `readdir` with N children of
* the same type pays the DB cost once.
* Lookups cache per-extension (plus a separate per-app-name cache for the small
* set of built-in opener apps), so a `readdir` with N children of the same type
* pays the DB cost once.
*/
export class SuggestedAppsService extends PuterService {
// Keyed by the normalized extension (lowercase, no leading dot). The
@@ -198,9 +200,9 @@ export class SuggestedAppsService extends PuterService {
}
/**
* Resolve suggestions for many entries in one pass. Entries that share
* an extension are deduped to a single underlying lookup; results are
* returned positionally so callers can `entries[i].suggestedApps = out[i]`.
* Resolve suggestions for many entries in one pass. Entries that share an
* extension are deduped to a single underlying lookup; results are returned
* positionally so callers can `entries[i].suggestedApps = out[i]`.
*/
async getSuggestedAppsForEntries(
entries: Array<{ name?: string; path?: string }>,
@@ -248,7 +250,7 @@ export class SuggestedAppsService extends PuterService {
const builtinNames = suggestionsForExtension(ext);
const seen = new Set<number>();
const results: Array<Record<string, unknown>> = [];
const candidates: Array<Record<string, unknown>> = [];
const apiBaseUrl = this.config.api_base_url as string | undefined;
@@ -260,7 +262,7 @@ export class SuggestedAppsService extends PuterService {
for (const app of builtinApps) {
if (app && !seen.has(app.id)) {
seen.add(app.id);
results.push(toAppSummary(app, apiBaseUrl));
candidates.push(app);
}
}
@@ -270,12 +272,41 @@ export class SuggestedAppsService extends PuterService {
if (seen.has(app.id)) continue;
if (app.approved_for_opening_items) {
seen.add(app.id);
results.push(toAppSummary(app, apiBaseUrl));
candidates.push(app);
}
}
}
return results;
// Drop apps whose puter-hosted backing is gone or has been reclaimed
// by another user. These summaries feed the GUI's default-open path,
// which launches straight from `app_obj` without re-reading the app
// through AppDriver — so its hosted-backing guard never runs and the
// launcher would append `puter.auth.token` to a subdomain the app
// owner no longer controls. `/open_item` also mints a user-app token
// (and grants `fs:<uuid>:write`) for `suggested[0]`, so an
// unlaunchable app must not reach the head of this list either.
//
// Built-ins never hit the DB here: their index_urls aren't on a
// hosting domain, so the check short-circuits on the URL alone.
const availability = await Promise.all(
candidates.map((app) =>
hostedIndexUrlBackingIsUnavailable({
app,
subdomainStore: this.stores.subdomain,
config: this.config,
}).catch(() => {
// A subdomain lookup failure must not silently widen the
// guard into "suggest nothing" — but it must not open it
// either. Treat the backing as unavailable: the app is
// unlaunchable for this window, not deleted.
return true;
}),
),
);
return candidates
.filter((_app, index) => !availability[index])
.map((app) => toAppSummary(app, apiBaseUrl));
}
}
@@ -1348,6 +1348,91 @@ describe('AuthService (integration)', () => {
});
});
describe('subdomainOwnerIdFromOrigin', () => {
// Test servers inherit the four hosting domains (production:
// puter.site / puter.host / puter.app / puter.dev) from
// config.default.json.
it.each([
'site.puter.localhost',
'host.puter.localhost',
'app.puter.localhost',
'dev.puter.localhost',
])(
'returns the subdomain owner for an origin under %s',
async (hostingDomain) => {
const user = await makeUser();
const subdomain = `own-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.subdomain.create({
userId: user.id,
subdomain,
});
await expect(
authService.subdomainOwnerIdFromOrigin(
`https://${subdomain}.${hostingDomain}`,
),
).resolves.toBe(user.id);
},
);
it('matches a hosted origin that carries an explicit port', async () => {
const user = await makeUser();
const subdomain = `own-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.subdomain.create({
userId: user.id,
subdomain,
});
await expect(
authService.subdomainOwnerIdFromOrigin(
`http://${subdomain}.site.puter.localhost:4100`,
),
).resolves.toBe(user.id);
});
it('returns null for an origin outside the hosting domains', async () => {
await expect(
authService.subdomainOwnerIdFromOrigin(
`https://external-${uuidv4()}.example.com`,
),
).resolves.toBeNull();
});
it('returns null for a subdomain of the main domain', async () => {
// `<sub>.puter.localhost` sits under the main `domain`, not a
// hosting domain — no owner resolves even when a subdomain row
// with the same name exists.
const user = await makeUser();
const subdomain = `own-${Math.random().toString(36).slice(2, 10)}`;
await server.stores.subdomain.create({
userId: user.id,
subdomain,
});
await expect(
authService.subdomainOwnerIdFromOrigin(
`https://${subdomain}.puter.localhost`,
),
).resolves.toBeNull();
});
it('returns null for an unregistered subdomain and for the apex host', async () => {
await expect(
authService.subdomainOwnerIdFromOrigin(
`https://ghost-${uuidv4().slice(0, 8)}.site.puter.localhost`,
),
).resolves.toBeNull();
await expect(
authService.subdomainOwnerIdFromOrigin(
'https://site.puter.localhost',
),
).resolves.toBeNull();
});
it('returns null for an unparseable origin', async () => {
await expect(
authService.subdomainOwnerIdFromOrigin('not-a-url'),
).resolves.toBeNull();
});
});
describe('app origin blocklist enforcement', () => {
// The blocklist service caches with a TTL, so seed the row then
// invalidate the in-memory snapshot to force a reload for the test.
Loaded 100 of 358 files, more files were not shown because too many files have changed in this diff. Show more