Compare commits

..
124 Commits
Author SHA1 Message Date
Daniel Salazar 5918e3f4e2 fix: kv max number value (#3629) 2026-08-23 06:31:08 -07:00
Nariman Jelveh 2c9e818e5f Docs/remove discord reddit links (#3628)
* docs: remove Discord and Reddit links

Drops both links from the docs README (header row and Support list) and
from the footer generated on every built docs page.

* docs: remove Mastodon link
2026-08-22 22:14:56 -07:00
Nariman Jelveh 8826a9fc51 Remove Discord and Reddit links from README (#3627)
Removed links to Discord and Reddit from the README.
2026-08-22 13:07:33 -07:00
Daniel Salazar 8307e8c9f0 chore: bump up deps (#3626) 2026-08-21 21:03:54 -07:00
Daniel Salazar 732786a029 fix: clean up email validation (#3622) 2026-08-21 20:27:05 -07:00
f0cd251626 🛠️ PUT-1521: Cleanup puter js permissions api + backend routes (#3607)
* refactor(puter-js): collapse puter.perms request* to resource + access

Fifteen request methods differed only by a folder name or an access level, so
every new resource meant another method. Replace them with requestFolder,
requestApps, requestSubdomains and requestAppRootDir, each taking the access
level as an argument.

The old names stay as @deprecated aliases: puter.js ships unpinned from
js.puter.com/v2, so removing them would break live apps. They remain in the
generated declarations because stripInternal has no effect on declarations
emitted from JavaScript, and hand-omitting them would break TypeScript callers
the runtime still serves.

Also drops the user-to-user and user-to-group grant wrappers (groups.js and the
grantUser/grantGroup half of grants.js) plus the req_ shim, none of which were
documented or called. The app, origin and dev-app grants stay: the dashboard
uses puter.perms.revokeApp() to clear grants on app uninstall.

* docs(perms): document the collapsed puter.perms surface

Replace the twelve one-method-per-task pages with requestFolder, requestApps
and requestSubdomains, and rewrite the Perms overview around the seven public
methods. The deprecated aliases keep working but are no longer documented.

Boy Scout: drops the long-dead commented-out grantUser/revokeOrigin sidebar
block for pages that were never published.

* refactor(perms): drop unused user-to-user and user-to-group permission routes

Filesystem access is shared through /share, which records the grant so the
owner can see and revoke it. The older direct-grant paths were left behind with
no caller anywhere - not the GUI, not a doc, not an app: grant-user-user
(already a 501 stub), revoke-user-user, grant/revoke-user-group, and the five
/group/* CRUD routes.

Removing them orphans PermissionService.grant/revokeUserGroupPermission and its
group-members cache bump, the three PermissionStore group writers, and six
GroupStore methods, so those go too.

What stays, and why:
- grant/revokeUserUserPermission - ACLService and ShareService power fs.share
  through them.
- The group permission read path (#scanUserGroup, readUserGroupPerms) - a
  migration seeds the admin group unrestricted driver access, so it is
  load-bearing.
- GroupStore getByUid/addUsers/removeUsers - signup, save_account, OIDC and the
  self-hosted default user assign group membership.

No schema change: user_to_user_permissions, user_to_group_permissions and their
audit tables are untouched. Group rows now come only from migrations, so tests
that need one seed it with SQL the way a migration does.

* refactor(perms): reduce GroupStore to membership writes

With the /group/* routes gone, `getByUid` had no production caller left — the
routes were the only thing that read a group back. Removing it takes the row
decoder and the GroupRow type with it, since they exist only to shape its
result.

What remains is `addUsers`/`removeUsers`: signup, save_account, OIDC and the
self-hosted admin bootstrap all assign group membership. Permissions attached to
a group are read through PermissionStore, which joins the junction table itself
and never needed the store.

Tests that wanted a group id now select it, which is all `getByUid` was doing
for them.

* chore(perms): drop the three by-hand groups no code reads

freeai, experimental and dangerous exist in prod but in no migration — they
were added by hand when hardcoded permissions were keyed by group name. That
map is now a flat per-user floor (`default_user_permissions`), so a group
nothing looks up grants nothing.

Guarded rather than unconditional, because both tables the delete can reach
cascade: dropping a group that still carries permissions or members would
silently revoke them from every member. Only a group with neither goes. One that
survives has dependents and needs a deliberate decision — query
user_to_group_permissions by group_id to see what it holds.

system, admin, user and temp are untouched: config names two of them and code
names the others.

Matches on `extra.name`, not `metadata.name` — `metadata` carries the display
title and colour, and `critical: true` is set on all of these including freeai,
so it does not discriminate.

* feat(puter-js): collapse puter.perms onto request(resource, details) + check()

One method per task meant a new method, doc page and sidebar entry for every
resource. `request` now takes the resource and a payload whose accepted fields
depend on it, and `check` answers the same question without prompting.

    request('folder', { name: 'Documents', access: 'write' })  -> path
    request('apps', { access: 'read' })                        -> boolean
    request('email')                                           -> address
    check('folder', { name: 'Documents', access: 'write' })     -> boolean

Returns stay per-resource: a folder gives its path, email the address, the rest
a boolean, and anything denied is falsy so one `if` covers both.

An array asks for several at once. Everything already held is settled first, so
the prompt covers only what is missing and does not appear when the whole set is
held - the user answers once for the lot. `check` answers per entry, in order,
so a caller can tell which parts are missing rather than only that some are.

Each resource declares four things in one registry entry: how to ask for it
alone, whether it is held, the strings a batch pools into a prompt, and the
value once held. The strings themselves are defined once in
lib/permissionStrings.js, so a request and its check cannot name them
differently. `check` is built on /auth/check-permissions, already live and
already used by UI.js, and it throws rather than answering false when the check
cannot run: a caller that cannot tell "denied" from "never ran" would prompt
someone who had already granted it.

Backward compatibility: all 22 older methods stay callable and typed, marked
@deprecated with the call that replaces them. A lone string still routes to the
raw-permission path - no resource name contains a `:` and every permission
string does, so the two forms cannot collide. The grant/revoke app methods are
untouched; the consent dialog and the dashboard's uninstall path use them.

Also drops three copies of the access-level assertion onto one shared
validator, and gives `appRootDir` a non-prompting server probe, since
`app-root-dir:` only resolves while a grant is being written and a permission
check on it always answers false.

* docs(perms): document request() and check() as the perms surface

Five per-method pages became one `request()` page carrying the resource table,
the batch form and the raw-string escape hatch, plus a `check()` page. The
overview is rewritten around the two methods.

requestAppData's page is re-homed as /Perms/appData rather than deleted - its
scope table, private-entry guidance and lifetime notes are not signature
documentation and have nowhere else to live. Inbound links from KV/set.md and
Objects/app.md follow it.

Playground examples move to the new call form. They are not wired into
examples.js, but an example demonstrating a deprecated method is worse than one
nobody loads.

* fix(perms): keep /auth/revoke-user-user as a deprecated route

Dropping this route with the rest of the unused user-to-user plumbing went too
far. The grant side is retired and stays retired - puter.fs.share() is the only
way in - but access those grants left behind has to remain withdrawable, and a
caller reaching the endpoint over HTTP directly had no replacement. Revoking can
only ever narrow what someone can reach, so keeping it carries no risk.

revokeUserUserPermission never left the permission service; it is load-bearing
for puter.fs.share(). This only re-wires the handler to it, with the gates it
always had.

Nothing in this repo calls the route, which makes it exactly what a later
cleanup reads as dead, so a test pins the registration and its gate alongside
the restored 400 and grant/revoke round-trip cases.

The 501 stub at grant-user-user and the never-called /group/* routes stay
deleted, as does puter.perms.revokeUser - puter.fs.unshare() replaces it and
falls back to live grants when no share row exists.

* fix(perms): let a write grant satisfy a read check on apps and subdomains

`apps-of-user:<uuid>:write` covers managing the user's apps, which includes
reading them, but nothing said so to the permission system. Prefix implication
only widens the other way — an `apps-of-user:<uuid>` grant covers both modes —
so a scan for `:read` missed a `:write` grant, and `puter.perms.check('apps')`
reported an app holding write as holding nothing. A batched request would then
prompt again for access already granted.

Adds the read-from-write exploder for both namespaces, mirroring
`fs-access-levels`. The widening runs one way only, and does not cross into
another user's namespace; both are covered by tests.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): answer the app-root-dir check without provisioning it

`/auth/request-app-root-dir` conflates two questions: may the caller claim its
root directory, and where is it. The second provisions `AppData/<uid>` on first
ask, so a caller that only wanted the first — `puter.perms.check('appRootDir')`
— created a directory by asking about it.

Adds `check: true`, which runs the same actor guard and stops at the answer.
A caller that may not claim it still gets the 403, so the flag can't widen
anything. Existing callers are unaffected: without it the route behaves exactly
as before.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): put request() and check() on one path

`request` dispatched to the old per-task methods while `check` asked the
permission tables, so the two answered different questions about the same
access. Concretely, before this: `request('folder', { access: 'write' })`,
`'apps'`, `'subdomains'`, `'appData'` and `'permission'` prompted every time,
whether or not the access was held — which the docs said they wouldn't;
`check('folder')` reported false for a folder the app could read through an ACL
grant that no `fs:` string names, so a batch prompted for it needlessly; a
batch entry for `'appRootDir'` skipped the post-grant retry the single call
does, resolving `undefined` after a grant that had in fact succeeded; and an
N-entry batch made N permission reads plus 2N `whoami` calls.

Both now run the same pipeline — resolve the permission strings, read what is
held once, prompt for the remainder, resolve each entry — with per-resource
hooks for the parts only that resource can answer. So a batch costs one
permission read and one `whoami`, a check reports exactly what a request would
skip the prompt for, and `'folder'` uses the same stat-or-permission reading in
both.

Also:

- A resource is looked up as an own property, so `request('constructor')` is
  the permission string it always was rather than a TypeError.
- A permission read that fails no longer decides anything: `request` falls
  through to the prompt it would have raised anyway, `check` throws. Before,
  `check('appRootDir')` folded a failed check into "not granted", which is what
  the documentation says must not happen.
- Drops `requestFolder`, `requestApps`, `requestSubdomains` and
  `requestAppRootDir`. They were added in this branch and immediately deprecated
  — never shipped, and `request()` no longer needs to route through them. The 22
  methods that did ship keep their exact behaviour, prompting without consulting
  what is held, which the suite now asserts alongside the new behaviour.
- Documents `'appRootDir'`, which was a supported resource in every overload and
  in `PermsResource` but named in none of the docs.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(perms): act on review of the three preceding commits

- `src/puter-js/test/perms.test.js` still called `requestApps`,
  `requestFolder`, `requestSubdomains` and `requestAppRootDir`, which the
  previous commit removed. Four cases in the interactive browser harness threw.
  Pointed at `request(...)` instead.

- `request('appRootDir', …)` made two round trips where the shipped method
  makes one: a read-only probe, then the call that names the directory. A
  request is going to claim it either way, so the claim is now the check, and
  the entry it returns carries through to the result. `check` keeps the
  read-only mode, which is the reason that mode exists. Matters because the
  route sits on the FS_SIGN bucket, shared with signed-URL minting.

- `requestPermission` is one of the shipped methods, and the previous commit's
  message was wrong to say all 22 keep their exact behaviour: it forwards to
  `request`, so it now settles a permission the caller already holds instead of
  prompting for it. The value can differ, not just the prompt count — a user who
  would have clicked Deny on a re-prompt used to get `false`. It is the more
  honest answer (the app does hold the access, and denying a re-prompt never
  took it away), but it is a change, and the suite assertion had been switched
  to an unheld permission, which hid it. Asserted both ways instead, in the unit
  tests and the API suite.

- An entry that names no permission no longer rides a grant given for the other
  entries in the same call. Unreachable today — every resource either names one
  or reports itself held — but nothing pinned it.

- Reverted three type-union reformats in `LegacyFSController.ts` that a
  formatter had folded into the app-root-dir commit. That file was not
  prettier-clean to begin with; reformatting it is somebody else's change.

- Docs and types: `Perms.md`'s `appRootDir` row now matches `request.md`'s,
  `check.md` says that a `true` is per entry and a batch still prompts if any
  one entry is missing, and `types.js` no longer names `requestFolder` /
  `requestAppData` in prose that ships in the generated declarations.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-20 23:48:08 -07:00
ZHOU7821 943adae622 docs: use absolute link to router page in workers create docs (#3615) 2026-08-21 13:30:30 +07:00
Daniel Salazar 65572dcf23 fix: nicer sms card fallback (#3621) 2026-08-20 23:12:09 -07:00
ZHOU7821andZHOU7821 b3923f86da docs: fix typo in workers description (#3614)
Co-authored-by: ZHOU7821 <[email protected]>
2026-08-21 12:26:20 +07:00
Nariman Jelveh 729114bd09 Prioritize app-center in recommendations
Reordered the recommended app list so app-center appears earlier and dev-center is kept at the end. This keeps the default app suggestions aligned with the expected homepage/app-center placement without changing behavior beyond ordering.
2026-08-20 19:16:03 -07:00
Nariman Jelveh da1c85d3c5 Add Meetings to recommended app defaults
Include `meetings` in `RECOMMENDED_APP_NAMES` so it appears in the default recommended apps set alongside other core productivity tools.
2026-08-20 17:16:39 -07:00
Nariman Jelveh 66410d7f1f Revert "Let a browser send the device fingerprint header it is offered"
This reverts commit fef1337b2b.
2026-08-20 17:10:55 -07:00
Nariman Jelveh 76bd05f228 Dashboard: let the trial be a detail line, not an alarm
The Usage card gives each of its three columns a meter and two lines. The
plan column had a boxed, tinted, bordered paragraph instead — and at the
~230px a third of that card comes to, thirty words of it wrapped into a
five-line ribbon standing twice as tall as the storage and resources meters
beside it. The box is the shape we use for the dunning banner. A trial is a
date you should know about, not a fault, and it was reading as the loudest
thing on the card.

So it takes the slot its neighbours use for "n% of 100 GB": a quiet hint-
coloured line under the badge. All three columns now measure the same.

The copy stopped repeating itself while it was there. The badge said "ends
Sep 20" and the sentence beneath it said "ends on September 20, 2026" — two
elements, one fact, and neither of them answering the question someone
actually has, which is how much room is left to decide. The badge now
carries "31 days left" and the line under it carries the consequence:
"First charge Sep 20 unless you cancel." Billing keeps the full sentence and
the opt-out, and Manage is directly below.
2026-08-20 16:59:07 -07:00
Nariman Jelveh fef1337b2b Let a browser send the device fingerprint header it is offered
`fingerprint.ts` reads the device fingerprint from a request body or, for
authenticated requests with no body to carry it, from
`x-puter-device-fingerprint`. That header is documented there as "the
header the GUI may send the device fingerprint on for non-signup
requests", and the middleware has always honoured it.

A browser could never actually send it. The CORS allowlist in
`#installCors` does not include it, so the preflight comes back without
it in `Access-Control-Allow-Headers`, the browser abandons the request,
and `fetch` rejects before anything reaches the server. Nothing in tree
sends the header today, which is why this went unnoticed: the first
client to try it sees every call fail with a network error rather than a
readable status.

Adds the header to the allowlist, and a preflight test asserting the
browser is allowed to ask for it.
2026-08-20 12:40:48 -07:00
404oops 58794f4f76 feat: add MetaProvider for Muse Spark models and integrate with ChatCompletionDriver (#3616)
- Implemented MetaProvider to interface with Meta's Muse Spark models.
- Integrated MetaProvider into ChatCompletionDriver, allowing for dynamic model selection.
- Added unit and integration tests for MetaProvider to ensure functionality and reliability.
- Updated documentation to include Meta as a supported vendor for AI models.
2026-08-20 15:09:56 -04:00
Nariman Jelveh 9809e428b5 Update shared label in email template
Adjust the share-notification email copy to reference the current "Shared" files section instead of "Shared with me," in both HTML and plain-text variants. This keeps onboarding messaging aligned with the UI.
2026-08-19 20:05:56 -07:00
Nariman Jelveh eaddd1d44e redesign: share notification emails (#3612)
* redesign: share notification emails

The share digests were an unstyled fragment: a 520px div with a couple
of paragraphs, no preview line, no small-screen or dark-mode handling,
and an html-only body that left every text-only client reading a machine
down-conversion of the markup.

Both digests (`file_shared_with_you` and `file_shared_invite`) now render
through one layout: a 600px table column that goes full width under
600px, every color written inline so a client that drops `<style>` still
gets the design, with the stylesheet carrying only what inline CSS can't
express — the small-screen and dark-mode overrides. The shares are a
tinted panel of hairline-separated rows rather than loose text, the call
to action is a cell-padded button Outlook can draw, and the reason for
receiving plus the unsubscribe sit below the card where they belong. No
images at all, so nothing depends on an asset a client may refuse.

Templates may now carry a `text` part, and both digests do: it goes out
beside the HTML as multipart/alternative, which is what plain-text
clients, screen readers and spam scoring prefer.

Copy: a preview line that adds to the subject instead of repeating it, a
heading before the list, and an honest unsubscribe label — the link
silences notification mail account-wide, not just shares. The invite now
says what to do with the address and that an existing account can claim
the shares by confirming it.

Also drops `share_by_username` and `share_by_email`, unreferenced since
the digests replaced them.

* fix: widen the template map to the interface before compiling

`Object.entries` over the literal map yields a union with a distinct
type per template, and only the share digests carry a `text` part —
reading it off the union is an error the build tsconfig hides
(`noCheck: true`) and `npm run typecheck` catches.
2026-08-19 18:49:11 -07:00
Reynaldi Chernando 53b792c52c connect to worker's kv with cli (#3613) 2026-08-20 08:36:46 +07:00
Daniel Salazar 2dd7073034 fix: vscode webdav (#3610) 2026-08-19 14:04:56 -07:00
Daniel Salazar a552ce0b87 fix: better metrics for cache hit rates (#3609)
* fix: better metrics for cache hit rates

* chore: type + docs
2026-08-19 11:38:14 -07:00
Neal Shah c04827fd3f add dynamic worker create event (#3608) 2026-08-19 14:20:15 -04:00
Daniel Salazar 9debdffc5d fix: email send frequency + ai ordering (#3604) 2026-08-18 18:13:55 -07:00
Nariman JelvehandJuan Castro 57b4cbab66 feat(dashboard): rewrite Files-tab sharing as a from-scratch modal (#3598)
* feat(dashboard): rewrite Files-tab sharing as a from-scratch modal

The Files tab's Share… action opened UIWindowShare, a desktop UIWindow,
which doesn't fit the dashboard. Replace it with UIShareModal, built on
the same overlay pattern as UIItemPropertiesModal: a centered card on
desktop, a bottom sheet on mobile, styled with the dashboard's design
tokens (dark mode included).

Wiring follows the onShowProperties precedent: generate_file_context_menu
gains an optional onShare hook that TabFiles supplies; every other caller
falls back to the desktop share window unchanged.

Feature parity with UIWindowShare — grant by email/username with an
access level, list who has access (owner, direct, and inherited-from-
ancestor grants), change a grant's mode, revoke — plus:

- revoke confirmation inlined into the row instead of a stacked UIAlert
- real form semantics: Enter submits, button disabled until input,
  spinner while in flight, API errors keep the typed name for correction
- dialog a11y: focus moves in on open and back on close, Tab is trapped,
  Escape closes (first Escape cancels an open revoke confirmation),
  status messages announce via aria-live
- backdrop close keyed on where the press started, so a text-selection
  drag out of the input can't dismiss the dialog
- name-derived avatar colors and a loading state for the first fetch

Also fixes a double-encoding bug (shared by UIWindowShare) where mode
labels ran through html_encode twice, rendering "Can edit &amp; share".

* fix(dashboard): keep keyboard focus inside the share modal across re-renders

Changing a grant's mode, revoking, or Escape-canceling the inline
confirmation re-renders the access list (or disables the focused
control), which dropped focus onto <body> - outside the dialog's Tab
trap and invisible to screen readers. Focus is now restored explicitly
after each of those actions: to the same holder's control when it
survives, else to the dialog itself (tabindex=-1), and the Tab trap
steps back into the cycle when focus rests on the dialog container.
A failed grant likewise returns focus to the recipient input.

* fix(dashboard): give share modal controls 44px touch targets

On touch devices the close (32x32) and revoke (30x30) buttons and the
26px-tall per-row mode select were below the 44px minimum. On coarse
pointers the icon buttons now hit-test at 44x44 through a centered
pseudo-element (visuals unchanged, and the row spacing absorbs the
overhang without overlapping neighbors), and the row select gets
taller padding. Fine-pointer rendering is untouched.

* fix(dashboard): clear the iOS home indicator in the share bottom sheet

The app opts into viewport-fit=cover, so on notched phones the
bottom-docked sheet extends to the physical screen edge and its last
row sat under the home indicator. The sheet body's bottom padding now
adds env(safe-area-inset-bottom), matching how the codebase pads other
bottom-docked surfaces.

* fix(share): say 'Updated access' when changing a grant's mode

Changing someone's access level reported 'Shared with X', the same
message as a fresh grant, which reads as if a new share happened.
Both share dialogs now confirm mode changes with a dedicated
'Updated access for X' message. The UIWindowShare call also drops the
html_encode() around the recipient - i18n() already encodes
interpolations, so the wrapper double-encoded.

* fix(dashboard): align the share dialog's accessible name with the properties modal

The dialog's aria-label read 'name - Share' with an em dash while the
sibling properties modal uses plain 'name Properties'; screen readers
announce the dash as noise. Use the same name-then-noun pattern.

* fix(dashboard): ellipsize the share input's placeholder when it overflows

At phone widths the 'Add people by email or username' placeholder was
clipped mid-letter; text-overflow: ellipsis truncates it cleanly.

* fix(dashboard): raise the share selects' chevron contrast in light mode

The hardcoded slate-400 stroke measured ~2.6:1 against light surfaces,
below the 3:1 minimum for non-text indicators. Light mode now uses
slate-500 (~4.7:1); dark mode keeps slate-400, which already clears
5:1 there.

* fix(dashboard): stop Files-tab shortcuts from firing behind the share modal

The document-level keydown.tabfiles handler kept running while the
share (or item-properties) modal was open. With focus on any of the
modal's buttons or selects, Enter and Space were preventDefault-ed
before they could activate the control, arrows could not drive the
mode selects, and letter typeahead was hijacked into row typesearch —
while Enter opened the selected row behind the overlay, Delete moved
it to Trash, and Cmd+A/C/X/V acted on the hidden list. Yield the
keyboard to the modal for as long as one is up; its own handlers
already cover Escape and Tab.

* fix(dashboard): keep the on-screen keyboard down when the share sheet opens

Autofocusing the recipient input popped the keyboard over the bottom
sheet the moment it opened, hiding the access list before the user had
chosen what to do — the same reason the revoke flow already focuses
the dialog instead of the input. On touch-primary devices give the
dialog container initial focus (which also anchors the Tab trap);
desktop keeps the input autofocus.

* fix(dashboard): give the share sheet's confirm and submit buttons 44px touch height

The inline revoke confirmation's Cancel/Remove pair rendered ~29px
tall on touch — small targets 6px apart where one of the two is
destructive — and the submit button was fixed at 38px. Grow both to
44px under pointer: coarse, matching the standard the modal's icon
buttons already meet. The touch block moves below the confirm-button
base rules it now overrides, since a media query adds no specificity
and source order decides.

* fix(dashboard): name the person in the share rows' accessible labels

Every grant row's mode select announced as bare 'Access level' and
every revoke button as 'Remove access', so a screen reader user
tabbing the list could not tell whose grant a control changes. Carry
the holder in the aria-label ('Access level for alice' / 'Remove
access for alice'); the visible UI and the revoke tooltip stay as
they were. i18n() encodes the interpolated string as a whole, quotes
included, so the labels stay attribute-safe for any holder name.

* fix(dashboard): keep the share status region in the accessibility tree

* fix(dashboard): give the share modal's add-row controls 44px touch height

* refactor(gui): extract the share dialogs' pure logic into tested modules

* feat(dashboard): adopt the shared mode helpers and handle pending invitations

---------

Co-authored-by: Juan Castro <[email protected]>
2026-08-18 17:34:40 -07:00
Juan Fernando Castro 5e1645d40e 🔧 PUT-1524: Sanity check file sharing api (#3603)
* fix: sharing answers "who can reach this" from the grants, not the index

Two ways a share listing could name access that was no longer there.

A recipient's listing paired the flat permission read with the list of
permissions it asked for by array index. That read drops misses and
dedupes its keys, so the two are not positional: one entry's live grant
vouched for another entry that had none, and listShared() kept
publishing a withdrawn item's name, size and signed thumbnail URL to
someone who could no longer open it. Read the permission off the value
instead.

getShares() had the same gap from the owner's side, with no liveness
check at all — a grant withdrawn through /auth/revoke-user-user or an
ACL mode change left the index row behind, and the owner was told
someone could reach a file they could not. Checked against the grants
now, one batched read per distinct holder. Pending invites are not
subject to it: they have no grant yet, which is the point of them.

Also covers the access-token actor, which reaches the same reach bound
as an app through a different arm of the ACL check. No behavior change
there — it was correct and untested.

* test: pin that a revoke crosses regions

A revoke reaches a peer region as a replicated SQL delete plus three
invalidation events, one per cache the region owns: the u2u row cache,
the flat view and the scan generation. The delete alone changes nothing
there, and each cache has a different consequence if its event is lost
— 20s for the generation, 5 minutes for the row cache, and forever for
the flat view, whose grant-path entries carry no expiry.

Nothing covered the whole path end to end. The store tests prove each
event is emitted and applied; this proves the result, which is that the
recipient stops being able to read. It asserts the read still succeeds
after the SQL delete alone, so the test also records why replication is
not sufficient on its own.

* test: pin that a leaked uuid buys no access

A masked share path hides which folder an item sits in; it was never the
thing deciding who may open it. Nothing checked that at the route level,
so the guarantee rested on unit tests of the resolver alone.

Reads one shared file through its masked path, then tries the sibling
four ways: the shared uuid with the sibling's name, the sibling's own
uuid, a `..` back out of the root, and the owner's real path.

Worth knowing about this one: it's mutation-checked. Removing the head !== root.name guard in sharePathMask.ts fails it with reachable: /testuser/55dd54c0…/share-http-1df8933e.txt. I verified that specifically because two tests I wrote earlier in this chunk passed with their guards broken — both were vacuous, and I deleted them rather than commit false assurance.

* docs: say what listShared's total actually counts

`total` counts the shares recorded for you; items are filtered after the
page is read, so a withdrawn grant leaves the count higher than anything
paging will yield. The page description already explained the short-page
behaviour, but the field read as an exact count and the example printed
it as one.

The test pins the gap it describes: two shares, one withdrawn outside
the index, one item listed and a total of two.

* fix: keep an undelivered broadcast event instead of dropping it

The outbound queue was cleared before the send, so a peer that timed out
took its events with it. Most were survivable — a lost cache invalidation
heals when the entry's TTL lapses. A revoke's flat-perm invalidation is
not: grant-path entries carry no expiry, so a peer went on serving a
withdrawn grant until something else wrote that key.

Failed sends now go back on the queue, which the existing flush timer
retries. Anything queued since wins over the retry, and the queue is
bounded at 10,000 with the oldest dropped first, so a peer that stays
down cannot grow it without limit. Each retry is signed at send time, so
it is not rejected against the replay window.

* fix: stop plus-addressing from deciding who a share reaches
2026-08-18 15:51:23 -07:00
b795b219a8 ✨ PUT-1497: Share file link sharing and notifications (#3595)
* refactor(share): move share notifications into their own service

* feat(share): invite an address with no account, and email it

* feat(share): surface pending invites in the SDK and share dialog

* fix(share): unreachable revoke confirmation, and double-encoded labels

* feat(share): budget share announcements, group them, and let people block senders

Sharing had one defence against noise: a 15-minute quiet window per (sender,
recipient) pair, which dropped the second share rather than folding it in.
Twenty senders each under their own window could still bury someone, and there
was no way to make one of them stop.

Announcements are now budgeted on two axes through the existing sliding-window
limiter: 1 per 15 minutes and 20 per day from one sender, and 10 per hour /
50 per day to one recipient from anyone. Over budget the share still succeeds
and the recipient's notification is still brought up to date — only the
interruption is dropped. Invite email to an address with no account is budgeted
the same way, keyed on a hash of the canonical address.

Notifications now fold across senders: a new share rewrites the notification
the recipient hasn't dismissed, so "alice and bob shared 5 items with you"
replaces a stack of five. The record is written even when suppressed, so the
count is right whenever they next look.

Blocking is a new `user_block` table with enforcement in ShareService: a blocked
sender's share is refused with `recipient_not_accepting_shares`, spends no
quota, and writes no row, and their unclaimed invite is dropped when the address
is confirmed. Existing access is untouched — that is what revoke is for.
Managed from a Blocked people card in the dashboard's Security tab.

Also publishes the sharing limits, including the ones already on this branch
that were never documented.

* fix(share): name the item in share email, instead of 'an item'

* fix(share): make the invite lifecycle canonical, authorized, and race-safe

* refactor(email): drop EmailClient.isConfigured; callers read config.email

* feat(share): batch share email into a per-recipient digest, durably

* docs(share): document the share error codes; steady the disk migration tests

* fix(share): log why a digest wasn't sent, and recover orphaned ones

* feat(share): email recipients about shares by default, with a way to decline

Share email was off unless a deployment opted in, which meant an account
holder was told about a share in the app only. It is now on unless
`share_email_notifications` is set to false.

The reason it defaulted off was that nobody could decline. So this also
honors `user.unsubscribed` — the account-wide opt-out the /unsubscribe page
already writes and app feedback already respects, which share email ignored —
and the digest carries that link. Sharing and the in-app notification are
unaffected by it; only the mail stops.

The link is composed in the template around an interpolated uuid rather than
passed pre-built: Handlebars escapes interpolated values, so a whole URL came
out as `user_uuid&#x3D;…`, which browsers decode but link scanners and older
mail clients need not.

* fix(share): count every shared file in the digest, not just the first

* feat(share): let a recipient refuse shares from everyone

Blocking answered "not from this person" but had no answer to "not from
anyone", so the only way to stop a stream of unwanted shares was to name
each sender after they had already reached you.

Stored as a key in the user row's existing `metadata` blob rather than a
column: the share path already holds the recipient's row by the time it
asks, so reading it costs nothing, and a one-bit preference doesn't earn
a migration per dialect. `updateMetadata` merges and refreshes the cached
row, so the switch bites on the very next share.

Refusing everyone reports the same code as refusing one person — which of
the two it is is the recipient's business, not the sender's. Enforced at
both moments the per-sender block is: when the share is issued, and when
a pending invite is claimed. The per-sender list is untouched while the
blanket switch is on, so turning it off restores what it hid.

`GET /share/blocks` now carries `all`; `POST`/`DELETE` take `{ all: true }`
beside the existing `{ username }`. Managed from the same Blocked people
card in the dashboard's Security tab.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(share): keep the digest sweep off a window that still has a timer

The sweep treated an entry as orphaned the moment its window closed, which
is also the moment the node that armed it fires. Claiming an entry is only
exclusive among flushers that can see each other's deletes, so the two
could each claim a share of the same digest and both send. It now waits
out a grace period first, which costs a genuinely stranded digest that
much delay and nothing else.

Both digest listings were capped at 200 with no word when they hit it — a
truncated flush sends a digest that undercounts and reads as complete.
The cap is named and logged.

Also: `#emailHolder` still described share email as off by default, which
it stopped being; the config doc said the batch window defaults to 60s
when it is 90; and the two tests that need several calls inside one window
were racing a 50ms window across four sequential round trips, so they
failed under full-suite load rather than on the behaviour they cover.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(share): stop double-encoding the recipient in two dialog messages

`i18n()` encodes what it returns, replacements included, so encoding the
recipient first showed the entities to anyone whose address or username
contains one. Same pattern already fixed two lines above.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix(notification): widen the mysql shown/acknowledged columns

Both arrived from the v1 schema as `tinyint(1)`, where they were flags.
The backend rework changed the writes to a unix second; sqlite (`INTEGER`)
and postgres (`bigint`) took it and mysql did not, so on mysql every
`markShown` and `markAcknowledged` has failed with
ER_WARN_DATA_OUT_OF_RANGE and left the column NULL. Dismissing a
notification never stuck — the unacknowledged count never moved and one
already delivered came back on every reconnect.

No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a
legacy `1` keeps meaning "yes" once widened. Guarded on the current type,
because changing a column type copies the table and this directory
replays on every boot.

Not reachable from the test suite — it runs against sqlite and postgres,
both of which already have the right type. Verified by hand against mysql:
`/notif/mark-read` and `/notif/mark-ack` now persist, and a dismissed
share notification is no longer the one a later share folds into.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-18 09:36:33 -07:00
Reynaldi Chernando 401b6066b3 Use gpt 5.6 luna as default for docs (#3602) 2026-08-18 09:36:23 -07:00
Daniel Salazar ec3c33fea6 fix: misc hardening fixes (#3600) 2026-08-18 00:06:58 -07:00
Anany Singh 95d797b047 feat(workers): expose app_uid in puter.workers.get and puter.workers.list (#3596) 2026-08-18 00:05:17 -07:00
Daniel Salazar bd06e88185 feat: bring back referrals PUT-1463 (#3599) 2026-08-17 23:49:09 -07:00
Nariman Jelveh cabedc30b0 Require login on app landing pages
Prevent first-time visitors from being treated as temporary users when they open an app landing page such as `/app/<name>` or `/desktop/app/<name>`. These routes now always go through the real login/signup flow.
2026-08-17 20:11:29 -07:00
Daniel Salazar f81b76c1d1 fix: webdav connectivity fixes for macos (#3597) 2026-08-17 19:02:55 -07:00
404oops 9604cc3f23 feat(ai-chat): overhaul provider model catalogs to match current APIs (#3586) 2026-08-17 14:06:44 -04:00
2c852bf6b3 ✨ PUT-1412 File sharing backend api (#3553)
* refactor(permissions): drop hardcoded group permission map for a flat default

* test(drivers): assert credential-gate intent instead of a 403 proxy

* fix(permissions): report whether a revoke removed anything and persist the linked grant row before the flat view

* feat(permissions): replicate permission invalidations across regions

* feat(share): extend the share table into an index of active shares

* feat(share): query and maintain active shares in ShareStore

* feat(users): add a batched lookup by email

* fix(cache): apply cache updates broadcast from peer regions

* fix(permissions): scope a revoke to the issuer that granted it

* feat(share): add ShareService with a per-day share limit

A share is two writes that belong together: the permission grant, which
authorizes access, and a share row, which makes it listable and ties it to an
fsentry so it dies with the file. Nothing else grants fs:* to a user.

Authorization reuses canManagePermission — an owner satisfies it through the
is-owner implicator, a delegate through an explicit manage:fs:<uid> grant. An
owner may clear any issuer's share of their node; anyone else only the ones
they issued, or their own access. Self-revoke skips the manage gate but still
requires `see`, so it cannot be used to probe for files.

The per-day limit counts shares created rather than live rows, so revoking and
re-sharing cannot recycle a slot, and changing an existing share's mode is not
new reach and does not spend budget. Tunable via share_daily_limit.

* feat(share): expose sharing over HTTP

POST /share, POST /share/revoke, GET /share/shared-with-me, GET /share/shares.
The controller was registered but entirely commented out.

Recipients × items fan out concurrently — every pair is a distinct
(holder, entry) key, so none of them contend — bounded by
runWithConcurrencyLimitSettled, which returns results index-aligned with the
input for the per-pair outcome list. Responses carry usernames only, never
internal ids, and the 404-not-403 rule is preserved so a failed call cannot
confirm a file the caller could not otherwise see. Notifications are fired off
the response path; a share must not fail over its own notification.

Per-request caps on recipients and items bound one call's fan-out; the daily
limit bounds the total.

* feat(share): keep recipients consistent when a shared item changes

* fix(fs): stop listing issuer homes at the filesystem root

* fix(acl): serialize concurrent mode changes on one node and pin app containment on shared paths

* fix(fs): expire signed URLs over entries the signer doesn't own

signFile defaults to a ~317k-year TTL and verifySignature checks only uid,
expires and signature — never the ACL. A recipient who ever signed a shared
file therefore held a permanent, revocation-proof URL to its bytes: revoking
the share did nothing to it.

signEntry now takes the acting user and drops to NON_OWNER_SIGNATURE_TTL_SECONDS
(1 hour) when the signer is not the entry's owner. Owners keep the permanent
default, so no existing client changes behavior.

The signature-authenticated directory listing bounds its children
unconditionally: that route has no session actor, and a signature proves
possession rather than ownership, so a recipient holding a short-lived
directory signature could otherwise mint permanent URLs for every child.

A bounded window is not revocation — the durable fix is a per-entry signature
epoch folded into the HMAC and bumped on any permission change.

* refactor(permissions): drop the unused permission-issuer lookup

listUserPermissionIssuers and its store method listUserPermissionIssuerIds
existed to synthesize the filesystem root from the home directories of everyone
who had granted the caller a permission. That listing is gone — it advertised
folders readdir then refused to open — and the share index answers "who shared
with me" directly, so nothing wants them back.

One removed test only asserted that the call returned an array; the other
covered readLinkedUserUserPerms round-tripping and is kept, rewritten without
the issuer lookup.

* fix(share): return the created share, not just an acknowledgement

* feat(puter.js): add file sharing to puter.fs

share(), unshare(), listShared() and getShares() on puter.fs, following the
existing FS operation shape: positional and options-object forms through
defineOperation, JSDoc overloads as the published signature, relative paths
resolved against the app's root directory.

A bare recipient string is read as an email when it contains @ and as a
username otherwise. Sharing an item with someone who already has it replaces
their access rather than stacking a second grant, so raising read to write is
one more call.

Adds a sharing suite to the API runner, which passes unchanged on node,
browser and workerd. Documents all four methods with runnable examples, and
corrects the FS overview callout that told readers one user cannot read
another's files — true before this, not after.

* feat(gui): add a Shared folder for items others shared with you

A sidebar entry listing everything other users have shared with you, backed by
puter.fs.listShared().

The path is the sentinel `puter://shared` rather than /<user>/Shared: this is a
query, not a directory, and a path-shaped value could collide with a folder
someone actually creates. refresh_item_container and update_window_path both
branch on it to skip the stat there is no fsentry for, and the listing swaps
readdir for listShared.

Entries render at their real paths under their owners' directories — the item
container already preferred an explicit fsentry.path over joining onto the
container, so nothing else had to change. Each carries who shared it and at
what level, which the context menu reads next.

* feat(gui): share items from the context menu

A sharing dialog shaped like its neighbours — options object, HTML-string
template, jQuery wiring, delegating to UIWindow() — with a recipient field, a
read/edit/share dropdown, and the current access list with revoke buttons.

Reached from a new "Share…" context menu entry, which is hidden on items shared
*with* you: re-sharing needs manage, so the dialog would only surface an error.

Those items get "Remove from Shared" in place of Delete. Delete moves an item
to *your* trash, which for someone else's file means moving their data out of
their tree — FSService refuses it, and the user saw a bare 403. Removing your
own access is what the action was reaching for, so that is what it now does.

* fix(share): withdraw what a removed recipient re-shared

* feat(share): report access inherited from a parent folder

* fix(gui): load the puter.js bundle the server configured

* refactor(gui): extract the action icon set into a helper

* feat(gui): surface Shared in the file browser

* feat(gui): manage access from the share dialog

* test(share): cover access inherited from a parent folder

* fix(share): keep downstream access from surviving a delegate who leaves

* fix(gui): name the real owner in the share dialog

* fix(gui): page through every shared item instead of the first 50

* feat(share): return item metadata with a share

* fix(share): invalidate a holder's cache when the entry is deleted

* fix(gui): treat items inside a shared folder as someone else's

* feat(permissions): let manage inherit down the filesystem tree

Access already reached descendants through the ancestor chain while authority did not, so someone trusted to manage a shared folder could re-share the folder but nothing inside it, and could not see who had access to a file within it.

A manage-inherits-from-ancestor implicator resolves it in the permission layer, beside is-owner, so every caller agrees rather than just ShareService. It consults only the immediate parent — resolving that re-enters one level up, making a chain of depth d cost d checks rather than d².

That makes two cascade gaps reachable, both fixed here. A revoke now walks the subtree, since a grant on a descendant can rest on authority held at the folder. And it stops at a delegate whose authority survives another issuer, because what they granted was never theirs to lose.

Also pins that manage is not transitive: granting it needs manage:manage:fs:<uid>, which only the owner holds, so delegation is one level deep by construction.

* fix(gui): offer sharing inside a folder you manage

The menus encoded "manage does not inherit" and would now hide an action that works. The Shared listing records each root's mode; the menus resolve a child's by longest matching ancestor, loading on demand so a deep link or restored window works too.

* fix(share): make the daily share limit hold under concurrency

* test(share): cover concurrency, measure cost, and name cases for what they verify

* fix(gui): import the ownership helpers the item menu calls

The single-item context menu handler calls is_owned_by_me and
shared_mode_for, but the imports were only ever added to
generate_file_context_menu.js — so every right-click on an item threw a
ReferenceError before the menu could build, and the non-owner Delete
gating never ran.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): authorize before resolving the recipient

share() looked up the recipient in parallel with the entry, before the
manage check — and the two failures carried different error codes. Any
verified user with a real entry uid could probe arbitrary emails and
usernames for account existence, at no quota cost. Resolve the entry,
authorize, and only then resolve the recipient: an unauthorized caller
now sees the identical safe 404 whether or not the recipient exists.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): broadcast permission row-cache invalidations to peer regions

Every publishCacheKeys call for the u2u, u2a, and access-token row
caches omitted broadcast, so a revoke only cleared the mutating
region's Redis. A peer region applied the replicated generation bump,
re-scanned, read the deleted row from its own still-warm 5-minute row
cache, and re-warmed the flat view from it — revoked access outlived
the revoke by the row-cache TTL instead of the intended 60-second
bound. CacheReplicationService already consumes these events; the
emits were just never sent.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(fs): refuse to rename an entry owned by another user

remove and move both refuse to act on an entry the caller does not
own, even when the ACL allows the write — rename had no such guard, so
a write-mode share recipient could rename the owner's file, or the
shared folder itself, rewriting the owner's whole subtree's paths.
rename now takes the acting user and applies the same policy.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): decide a flat delete from the primary, not a lagging replica

revokeUserUserPermission deletes the SQL grant, then only drops the
flat KV entry once no issuer still grants the permission. That
remaining-check read through the row cache the delete had just
invalidated, straight to a replica — under any lag the deleted row
reappeared, the flat delete was skipped, and the stale rows were
re-cached for another five minutes. Grant-path flat entries carry no
TTL, so the holder kept working access with zero SQL rows behind it,
invisible to every listing. The check now reads the primary and
re-warms the cache with what it actually saw.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(permissions): keep a failed remote flat-invalidation from crashing the process

The outer.permission.flatInvalidated applier was fire-and-forget with
no catch, and it awaits a KV delete — one transient KV error while
applying a peer region's revoke became an unhandled rejection, which
is process-fatal under default Node. Its sibling appliers were already
guarded; this one now logs and moves on, leaving the entry to the next
invalidation or its TTL, same as a lost event.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): revoke every requested item and recipient, not just the first

revokeShare destructured only the first recipient and first item while
the parsers accept arrays up to the request caps — unshare({items:
[a, b, c]}) returned success having revoked only a, leaving access the
caller believes is gone. Revoke now fans out over every (recipient,
item) pair exactly like POST /share, reports per-pair outcomes, and
sums the revoked count; the response stays backward compatible.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): only a confirmed email designates a recipient

Recipient resolution by email accepted unconfirmed accounts, so
pre-registering someone else's address (unconfirmed) was enough to
receive shares meant for them once no confirmed account held it.
An email now only resolves to an account that has confirmed it;
username shares are unaffected.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): accept tilde-rooted paths like the FS routes do

The SDK resolves relative paths to ~/..., but the share routes never
expanded the tilde — a ~-prefixed string was read as a uid and every
relative-path call 404'd. Item parsing now treats ~ as path-shaped and
expands it to the actor's home with the same helper the legacy FS
routes use, on share, revoke, and the shares listing.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): walk a directory revoke by parent linkage, not path prefix

listByFsentrySubtree matched descendants with fsentry_id = ? OR path
LIKE ?, which has two problems: fsentries.path is lazily backfilled
and NULL on old rows, so those descendants' shares silently survived a
directory revoke, and the OR'd predicates forced a scan of every
active share. A recursive CTE over parent_id — the same shape the
lineage resolver already uses — covers every descendant and runs on
idx_parentId_name.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(gui): give each item its own share dialog

single_instance keyed the dialog on the app id alone, so opening
Share… on a second file focused the first file's dialog — typing a
recipient there granted access to the wrong file, with only the title
hinting at it. The dialog is now instanced per path: same item
refocuses, different item opens fresh. Also stops pre-encoding the
title, which UIWindow encodes again.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(permissions): let manage answer a write check

A manage grant let its holder re-share a folder but not work in it: the
ACL mode family stops at write, and the fs exploder had no rule for the
narrowest mode, so `manage:fs:<uid>` never satisfied `fs:<uid>:write`.

Fold manage into the candidate list for every non-manage mode, in both
the access-token branch and the scan branch, and give `write` an (empty)
exploder rule so the manage arm is emitted for it too.

* fix(fs): authorize restructuring by write on the parent

* fix(fs): let a share recipient work inside a shared folder

rename, remove and move refused outright when the entry belonged to
someone else, so a recipient with write could neither delete nor rename
anything inside a folder shared with them. The GUI compounded it by
hiding Delete for any item it did not own.

Authorize the three by ACL write on the entry's parent. For an owner
that is the same answer; for a recipient it grants the inside of a
shared folder and withholds the folder itself, whose parent is the
owner's private tree.

Deleting sends the item to its owner's trash rather than the deleter's,
so it leaves the recipient's view without leaving the owner's account
and without changing hands. A move may not otherwise carry someone
else's entry out of their tree.

* fix(fs): give a new entry to the owner of the folder it lands in

A file a share recipient added to a shared folder was recorded as
theirs while living in the owner's tree, so a subtree could hold rows
belonging to several people — and the storage it consumed was checked
against the writer while being counted against the owner.

Take the owner from the parent row at every insert, charge the
allowance to that owner, and hand a moved entry over to the tree it
moves into. An entry now always belongs to whoever owns the directory
holding it.

* feat(fs): address shared entries as ~/share/<uid>

A recipient could read the owner's whole path off any shared entry —
where they keep the file and what sits beside it, neither of which the
share is about.

Give shares their own namespace. `~/share/<entry-uid>/rel/path` resolves
to the real path on the way in, and outgoing paths are rewritten to it
on the way out. Entries the actor owns pass through untouched, so no
existing client contract moves.

* revert(fs): mask only the directory bar, not the addressing

fe535d598 made `~/share/<uid>` the actual address for every shared
entry. That reached far past the intent: item names became uuids, the
Shared views rendered uuids instead of filenames, and navigation
addressed entries through a namespace nothing else understood.

Put real paths back everywhere — responses, the share listing, and
request handling — and do the masking where it was wanted, in the
window's directory bar. A recipient sees `Shared › Contents › sub`
while every crumb keeps the real path it navigates to.

The share listing now carries the entry's name, content type, owner and
a signed thumbnail. A share row has no fsentry behind it for a client
to stat, and the stored thumbnail is an `s3://bucket/key` URI that no
client can render and none should see.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* fix: path obfuscation, webdav + small ui stuff

* test(share): assert the masked share path by its exact shape

The substring check tripped on the scratch files' own names, which start
with `sharing-`; the exact-equality assertion on `/<owner>/<uid>/<name>`
already proves nothing above the share leaks.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): rename policy, shared-view guards, webdav share parent, quota/bucket invariants

- Rename: a directly-shared FILE renames with write on it; a shared
  folder root stays fixed (its name is the owner's tree structure).
  GUI can_rename mirrors the backend, guards all editor entry points.
- Up from a share root goes to the Shared view on both surfaces; the
  Shared view gets a single crumb, a disabled Up button, and refuses
  drops, New/Paste, uploads and ctrl+V everywhere (it is a query, not
  a directory).
- WebDAV: PROPFIND on /owner/uuid answers as a virtual collection
  holding the share root (ACL-gated; 404 for strangers).
- Storage allowance override no longer crosses user boundaries: a
  recipient's plan cannot raise the owner's cap.
- Overwrites stay in the bucket the entry already lives in instead of
  repointing to the handling server's bucket and stranding the old
  object.
- manage mode documented as implying write (matches enforcement);
  share dialog label now "Can edit & share". Documented that fs socket
  events are owner-only.
- Sidebar: saved orders gain the Shared entry once the user has shares.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(share): close review findings, and retire /auth/grant-user-user

Security
- Recipient writes no longer echo the owner's real path back. The pending-write
  event and the upload-progress meta both go to the *acting* user, so a write
  into a shared folder handed out the layout above the share root.
- `maskerFor` failed open: the first caller fixed the actor, and one that ran
  before the request knew who was acting pinned `undefined` — after which every
  path published unmasked. It now adopts the first real actor and rebuilds for
  a different one.
- `/auth/grant-user-user` returns 501. It wrote user-to-user grants straight to
  the permission tables with no share row, so nothing could list or cascade a
  revoke over them — and the new `manage` mode meant a delegate could reach it
  for the owner's files. Filesystem access goes through `/share`; nothing else
  is meant to pass between two users. Undocumented (its docs page was never
  written) and no callers. Revoking is untouched, so grants made before this
  can still be withdrawn.
- `#assertCanManage` asks about the permission actually being granted rather
  than a fixed `read`, so sharing at `manage` is refused at the share layer
  instead of by `grantUserUserPermission` two levels down.
- `listSharedWithMe` checks the grants, not just the index: withdrawing access
  any other way left the row publishing name, size and a signed thumbnail URL.
- Moving an item into another tree now retires its shares. Grants are keyed on
  uuid, so they followed it and left the new owner with recipients they never
  agreed to.

Correctness
- `acl.check` gets the real path again, not the masked one. ACL matches on the
  path string, and a mask hides the `AppData/<appUid>` shape it needs.
- "Leave this share" works for a grant that predates the index; the fallback
  scoped the delete to the caller as issuer, which can never match.
- `move`/`copy` reject a name with a slash or a `.`/`..` segment, as `rename`
  already did. It matters more here: a move into the owner's Trash skips the
  destination write check.
- Descendant walks scope by path, not by owner, so rows predating the
  one-owner-per-subtree invariant aren't orphaned by their parent's deletion.

Performance
- Index `user_to_user_permissions(permission)`. Retiring a node's grants is a
  prefix match, but the primary key is (issuer, holder, permission), so it was
  a full scan.
- Retirement is coalesced and chunked. `remove()` emits one event per
  descendant, so deleting a directory fired one unindexed lookup per file, all
  at once and unawaited.
- `listReaching` is a plain `fsentry_id IN (...)` on `idx_share_fsentry`. It
  runs behind every file write, and the join-plus-OR it replaced was
  unindexable.

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: Daniel Salazar <[email protected]>
2026-08-17 09:08:23 -04:00
Ayush tiwari fc861da89f feat: add Gemini 3.7 Flash (#3587) 2026-08-16 23:51:37 -07:00
Parman Mohammadalizadeh facb844747 fix: derive express subdomain offset from the configured root domain (#3575)
Express derives `req.subdomains` by dropping a fixed number of labels
from the right of the hostname, defaulting to 2. A deployment whose
`domain` has more than two labels therefore reads its own root domain as
an active subdomain, so the user-site redirect sends the root origin to
the static hosting domain, which sends it back. Self-hosting docs
recommend exactly that shape (`puter.example.com`).

Set the offset from the label count of `config.domain`. Two-label
domains keep the express default, so existing deployments are unchanged.

Fixes #3561
2026-08-16 23:48:20 -07:00
Yashwarhade8010 3b791bb334 docs: update web search examples to gpt-5.6-luna (#3589) 2026-08-16 23:45:02 -07:00
Daniel Salazar a64a444a17 feat: batch delete for flush + metering fixes (#3593) 2026-08-16 15:26:33 -07:00
Daniel Salazar d3c9828207 fix: metering gui maths (#3592) 2026-08-16 14:57:29 -07:00
Daniel Salazar 87180b1081 fix: metering (#3591) 2026-08-16 14:19:05 -07:00
Daniel Salazar d22b656df8 fix: metering issues with addons (#3588) 2026-08-15 22:06:51 -07:00
Nariman Jelveh 19cd5f4476 feat: add BytePlus ModelArk providers (chat, image, video) (#3498)
* feat: add BytePlus ModelArk chat provider

Adds BytePlus ModelArk as a provider for the puter-chat-completion
driver, following the MiniMax/ZAI providers as reference per
doc/contributing-apis.md.

- OpenAI-compatible endpoint at ark.ap-southeast.bytepluses.com/api/v3
  (apiBaseUrl config selects the region)
- Static catalog of 16 chat models (Seed 2.x/1.x incl. vision, GLM,
  DeepSeek, GPT-OSS) with limits and per-token pricing from the
  official docs
- Passes Ark's thinking/response_format/stop params through custom;
  normalizes reasoning_content to reasoning
- Bare deepseek-v4-* names stay with the first-party DeepSeek
  provider; BytePlus only claims prefixed aliases
- Offline unit tests (mocked SDK against a real test server) plus an
  env-gated integration test

* feat: add BytePlus image and video providers

Extends the BytePlus ModelArk integration to the puter-image-generation
and puter-video-generation drivers, reusing the same services.byteplus
API key and regional apiBaseUrl as the chat provider.

Image (Seedream/SeedEdit via OpenAI-compatible /images/generations):
- dola-seedream-5-0-pro (pixel-tier pricing + billed input images from
  the 2nd on), seedream-5-0-lite, 4-5, 4-0, and seededit-3-0-i2i
- quality tiers 1K/1.5K/2K; aspect ratios resolve to Ark's documented
  pixel sizes; explicit WxH passes through with Ark's bounds enforced

Video (Seedance via Ark's async /contents/generations/tasks + polling):
- Seedance 2.0 / 2.0 Fast / 2.0 Mini / 1.5 Pro / 1.0 Pro / 1.0 Pro Fast
  (2.5 is priced but its API isn't live yet, so it's excluded)
- per-video-token billing from usage.completion_tokens, with per-second
  estimates feeding the credit cap; audio vs silent rates for 1.5 Pro
- first/last frame and reference-image inputs; generate_audio param
  added to IGenerateVideoParams

Pricing and capabilities hardcoded from the official docs (ModelArk
pages 1544106, 1330310, 1520757, 1521309, 1541523). Offline unit tests
mock the SDK / global fetch; integration tests are env-gated on
PUTER_TEST_AI_BYTEPLUS_API_KEY.

* fix: correct BytePlus catalogs and validation against the live API

Verified the three BytePlus providers against ModelArk with a real key;
these are the mismatches that surfaced.

- Drop seededit-3-0-i2i-250628. Ark reports it as Shutdown and every
  request 404s. Its now-unreachable image-to-image branches in the
  provider go with it.
- seedream-4-5 and the 5.0 series enforce a 3,686,400 pixel minimum, so
  they only accept the 2K tier. Mark them 2k-only and snap an
  unsupported tier up to the nearest allowed one, which also keeps the
  aspect-ratio table from mapping to a sub-minimum size.
- glm-4-7 has a 204,800 token context, not 256K.
- Guard the actor in the image provider like the video provider does.
- Round a sub-minimum video duration up to the shortest supported clip
  instead of reporting it as insufficient funds.
- Gate video resolution on the model's own dimensions; the dims table is
  shared across a family and accepts more than any one model does.

* Tighten BytePlus AI provider handling

Extract shared reasoning-content normalization for OpenAI-style chat providers, and harden BytePlus image/video behavior. This updates image tier and size validation, normalizes aspect ratios and input image refs, prevents mismatched BytePlus key/base URL fallback config, makes video resolution matching case-insensitive, and rejects excess reference images instead of silently truncating them. Tests were expanded to cover the new BytePlus request and validation paths.
2026-08-15 21:08:17 -07:00
404oops fc56075115 Merge pull request #3584 from rupesh0001-tech/rupesh
feat(ai-chat): add grok-4.6 model to xai provider
2026-08-15 21:22:01 +02:00
404oops e1998e39ab Increase max_tokens limit to 500,000 2026-08-15 21:20:45 +02:00
Daniel Salazar fb7968a1c7 fix: metering hardening; handle burst of unfinished ai requests (#3585) 2026-08-15 10:52:29 -07:00
rupesh0001-tech c88eeb3868 feat(ai-chat): add grok-4.6 model to xai provider
- Add grok-4.6 model definition with pricing and context specs
- Add aliases x-ai/grok-4.6 and grok-4.6-latest
- Add unit tests for alias resolution and metering calculation

Closes #3562
2026-08-15 20:32:22 +05:30
Reynaldi ChernandoandCopilot Autofix powered by AI 9e25ce1401 [PUT-1478] Add KV commands to CLI (#3579)
* Add KV commands to CLI

* docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-08-15 11:43:30 +07:00
Daniel Salazar f15d835eeb fix: restrict openai and anthropic compatible endpoints to be subscription (#3583) 2026-08-14 18:55:15 -07:00
Daniel SalazarandClaude Fable 5 27c4506e02 feat: usage rendered as credits on a 50c base, free allowance doubled (#3569)
* feat: usage rendered as credits on a 50c base, free allowance doubled

Usage is now displayed in credits: a configurable creditsPerDollar rate
(default 2,000, making the free tier's 50c allowance an even
1,000-credit base) ships with the metering usage response, and the
dashboard's usage cards and per-API table show raw credit numbers
through one shared formatter. The dashboard budget math is also fixed —
capacity is spend plus server-netted remainder, so held purchased
credit can never render as negative usage. The registered-user free
allowance doubles to 50c, tier display names move to Basic/Plus/Pro,
and dead referral-promise strings leave the English dictionary.

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: the credits display rate exists only in deployment config

No default in code and no rate on any wire the plan surfaces read: a
deployment that doesn't configure creditsPerDollar renders usage in
dollars, exactly as before credits existed. The usage endpoint sends
the rate only when configured, and nothing in code or comments states
what any deployment's rate is.

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: metering reports pre-scaled credits, never raw amounts

The usage, per-app usage, and cost-catalogue endpoints multiply every
monetary field by the config multiplier (renamed creditMultiplier)
before responding, and flag the unit; the multiplier itself never
ships. Counts, units, and byte figures pass through untouched. With no
multiplier configured the endpoints report raw amounts and clients
render dollars, as before credits existed.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: absent addon fields stay absent when scaling to credits

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 16:29:01 -07:00
Neal Shah 6a7c09f54e blacklist dynamic workers (#3582) 2026-08-14 18:35:34 -04:00
Reynaldi Chernando 6254714484 deprecate gemini 2 flash and lite (#3581)
* deprecate gemini 2 flash and lite

* fix test
2026-08-14 17:27:48 -04:00
Neal Shah efa2cb4441 add more extensibility to workers driver (#3565) 2026-08-14 17:24:36 -04:00
Nariman Jelveh 4bc969b99e Update RecommendedAppsService.ts 2026-08-14 14:05:44 -07:00
Reynaldi Chernando 65f6d38432 update gpt 5.6 luna and terra pricing (#3580) 2026-08-14 16:38:13 -04:00
Daniel SalazarandClaude Fable 5 d641f6f7a1 tests: count only this test's dynamo reads in the block-window assertion (#3576)
The spy sits on the test server's shared dynamo client, so a raw call
count also picks up background work and the async tail of earlier tests
in the file — which made the block-window test flakily report a third
read. Each test runs in its own random namespace, so filtering the
spy's calls to that namespace removes the cross-talk without loosening
the assertion.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 11:05:25 -07:00
Daniel SalazarandClaude Fable 5 00828b9998 fix: concurrency limiter heals legacy INCR keys instead of failing open (#3574)
The zset rework of the redis concurrent backend (#3529) kept the same
key names the old INCR counter used. Against a leftover string key every
zset command in the acquire MULTI fails WRONGTYPE while the trailing
EXPIRE still succeeds — so steady traffic refreshes the stale key
forever, and the failed zcard result turned into NaN, which admitted
every caller unbounded. Release then errored WRONGTYPE on each request
(the '[concurrent] release failed' log storm).

Surface per-command MULTI errors instead of coercing them to NaN, and on
WRONGTYPE delete the legacy key and re-acquire against a clean zset.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 01:40:05 -07:00
Daniel SalazarandClaude Fable 5 126ec09556 docs: publish rate limits and quotas, and make limit changes move the docs (#3568)
Developer-facing reference for every rate limit, concurrency cap,
quota, and error shape — an advanced page, since puter.js already turns
the common failures into prompts. AGENTS.md and CONTRIBUTING.md now
carry the rule that a PR moving any of these numbers updates the page
in the same PR: an undisclosed limit is one developers discover as a
service failure.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 00:41:01 -07:00
jelveh 25a5799088 Update RecommendedAppsService.ts 2026-08-14 00:38:05 -07:00
Daniel SalazarandClaude Fable 5 ea2f9967a6 fix: over-quota batch uploads surface the storage prompt instead of failing quietly (#3566)
* fix: over-quota batch uploads surface the storage prompt instead of failing quietly

Partial batch failures now carry each item's code/status, and when every
failed item failed the same way the shared code/status is hoisted onto
the rejection itself — so an upload that exceeds the storage quota
rejects with storage_limit_reached/413 and the SDK's upload handler
shows the free-up-space prompt. A partial failure is also no longer
misread as the signed-batch endpoint being unavailable.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: out-of-storage copies prompt to upgrade the same way uploads do

Any fs operation the server refuses with 413 storage_limit_reached now
surfaces the upgrade prompt — the check that lived inline in upload's
error handler moves to a shared helper wired into the operation
scaffold's reject path, so copy/move/mkdir/rename get it too. The
desktop's copy/paste suppresses its generic alert for that code, since
the SDK dialog already explains the refusal and carries the fix.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-14 00:26:07 -07:00
Daniel Salazar ba9b885a5a Revert "feat: dashboard app search continues into the App Center (#3567)" (#3572)
This reverts commit 6cc241ac31.
2026-08-13 21:00:02 -07:00
Daniel SalazarandClaude Fable 5 6cc241ac31 feat: dashboard app search continues into the App Center (#3567)
A search that matches none of the user's own apps now puts the same
query to the App Center catalogue and renders the hits under the empty
state — debounced, filtered to apps the user doesn't already have, and
launchable in place.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-13 20:48:03 -07:00
Nariman Jelveh 7f8e24f06d fix: keep "Add an app" in the grid while searching
Typing in the Apps tab's search dropped the tail tile — deliberately, on the
grounds that a query asks which apps you HAVE and a tile matching nothing has
no business in the answer. That reads the search too narrowly: looking for an
app you don't have is the same gesture as looking for one you do, and the
moment the search comes up short is exactly when the way to GET an app is
worth having on screen.

So the tile is pushed unconditionally now, and a query that matches nothing no
longer replaces the grid with a bare "No apps match your search" — the line
moves above a grid holding just that tile. Keeping it a real grid, rather than
parking a tile inside the empty state, is also what keeps it reachable: arrows
and Enter only walk tiles inside .myapps-page, and the tail tile carries
tabindex -1 for the pager's roving focus.

The no-apps-at-all state is untouched: with nothing installed and nothing
typed, the empty state still stands in for the whole grid.
2026-08-13 18:53:59 -07:00
Nariman Jelveh 9b6627b1ea fix: an app's background helper must stay private to it
`launchApp(name, args, { background: true })` hid the child's window and
nothing else, so the instance was hidden but not private: it kept its
taskbar item, lit the running dot on its dashboard tile, and — because
the dashboard enforces one instance per app — WAS what clicking that tile
opened. Minimize the parent, click the child's tile, and Puter handed the
user the window the parent was mid-conversation with; the app, which only
ever meant that instance for its caller, said so on screen. Showing it
also dropped the never-shown marker, so from then on the helper outlived
its launcher and owned the tile.

Hidden now means unreachable. `user_facing_windows` is the one rule: of
an app's windows, the ones that are the user's. Everything the user
drives filters through it — the tile's reopen and its running dot and
Quit item, the taskbar item's click, window list and show/hide/close-all,
the file row's switcher and open-file dot, and the dashboard's
Back/Forward — so a helper is never focused, restored, listed, or counted
as running. With none of the user's own left, the tile simply launches a
fresh instance and the helper keeps serving whoever launched it. Paths
that act on the APP rather than on a window it happens to have open —
uninstall, and a launcher's close taking its helpers down with it —
deliberately keep the unfiltered list.

The taskbar item is now what a hidden launch EARNS by becoming visible,
rather than something it advertises from the start; makeWindowVisible
adds it (and fires the dots event) before showing, since focusWindow
marks an item active and needs it to exist. Windows stamp data-in_taskbar
when they take their place in data-open-windows, so the close path only
decrements a count it actually joined — otherwise a helper closing beside
the user's own window would zero the count and strip the item off a
window still open. For the same reason a helper's close no longer pops
the dashboard URL: it never claimed an entry (the push only happens for a
window created visible), and consuming one would minimize the user's
window of the same app.

is_unseen_background_window gains the other half of its own definition:
launched hidden AND launched BY another app. An app that is always
windowless (`background` on the app record) starts hidden too, but when
the user opens it there is nobody it is serving — it is the instance
their tile has to find, or every click would start another one they
cannot see.
2026-08-13 18:38:13 -07:00
Nariman Jelveh 7576e0b03a fix: route dashboard file menus by input modality, not touch capability
A mouse right-click on a file row in the dashboard Files tab opened the
mobile context-menu sheet (centered over the row, nowhere near the
cursor) on any touch-capable laptop or desktop, because phone/tablet
detection used navigator.maxTouchPoints > 0. The same check applied the
.touch-device class, degrading mouse selection and drag on those
machines.

Route by how the menu was invoked (the row's tracked pointer type) plus
isTouchPrimaryDevice() — coarse pointer and no hover — which is false on
touch-capable laptops but true on iPads whose UA claims macOS. Also let
sidebar-folder and background menus accept taphold on such iPads, where
they were previously unreachable (taphold dismissed, and iOS never fires
native contextmenu).
2026-08-13 17:46:58 -07:00
Nariman Jelveh d980a9f590 fix: don't invert full-color icons in the dashboard context menu 2026-08-13 17:33:14 -07:00
Nariman Jelveh d18ea1adb5 fix: a background app must not outlive the app that launched it
An app launched with `background: true` gets a real window from the moment it
starts, just hidden. Nothing ever took it down: when the app that launched it
closed, the child kept running with no way to reach it and no reason to be
there. On the dashboard the only sign was a running dot on a tile the user had
never lit up, and clicking that tile did nothing at all.

Three things were wrong, and all three had to go:

A window nobody has seen now dies with its launcher. UIWindow's close path
closes hidden children the closing app launched, keyed on a marker stamped at
creation. makeWindowVisible drops that marker the first time the window becomes
visible — showing itself with `puter.ui.showWindow()`, or the user showing it —
because from then on the window is the user's, and keeps running.

The dashboard tile is a real handle again. focusExistingAppWindow only routed
MINIMIZED windows through showWindow(); a hidden one fell through to
focusWindow(), which leaves it invisible while handing it the keyboard. With no
taskbar in dashboard mode the tile is the only handle on a background app, so
that click had nowhere else to go. It now asks whether the window is on screen
at all. The Files tab's row-click had the same one-line defect.

And a background instance can't take a tile from the user's own session: an
on-screen window wins, then a window the user has seen, then a hidden one.

Removing the child then hit a crash of its own: ExecService's `remove` handler
dereferenced the launcher's iframe to say goodbye, and the launcher was already
gone. Throwing there aborts jQuery's remove() itself, so the window stayed in
the DOM — running dot and all. That one also hit anyone closing a background app
from the taskbar after its launcher had closed.

The predicates behind all of this live in one helper, window_visibility.js, with
showWindow() reading the same `hidden, not minimized` rule it spelled out inline
before.
2026-08-13 14:36:08 -07:00
Nariman Jelveh 08f075d774 Update RecommendedAppsService.ts 2026-08-13 11:45:28 -07:00
Nariman Jelveh 664de0016b fix: a hidden window must not hold the keyboard
Launching an app to serve another one took focus away from the app the user
was working in, and they had to click it to get it back. Typing a guest name
into Calendar's event editor — which launches Contacts to read the address
book — dropped the user out of the field mid-word.

Four things conspired, all of them variations on "a window nobody can see is
treated as the window in front":

`window-active` was hardcoded into every window's markup, including windows
created hidden. focusWindow() — the thing that normally grants that class,
and strips it from everyone else — is already skipped for a hidden window, so
two windows ended up claiming it. It now follows the same rule as the
window_stack push a few lines above: visible windows only.

Both "focus the window once its IPC attaches" sites read that class to decide,
so a hidden child window pulled focus a beat after launching. Neither focuses
a window with data-is_visible="0" any more.

makeWindowInvisible left `window-active` on the window it had just hidden.
Since focusWindow() disables pointer events on every other app's iframe, an
app calling puter.ui.hideWindow() on itself left whoever launched it both
unfocused AND unclickable until the next click. It now hands activation back
to the top of the window stack, the way closing a window does, and drops
itself out of the activation order until it is shown again.

showWindow only ever un-minimized: for a window hidden by hideWindow() (which
keeps its geometry and writes no data-orig-*) every branch left it hidden and
stamped NaN geometry on it from the missing attributes. It now routes such a
window to makeWindowVisible. That is what makes the taskbar item a real
handle on a background app — clicking the item calls showWindow on the group.
2026-08-13 09:59:37 -07:00
Nariman Jelveh e273431f14 feat: let an app launch another app in the background
`puter.ui.launchApp(name, args)` had no way to say "I need this app's API,
not its window". That matters because we create and show an app's window
before the app's own code runs, so an app launched purely to serve another
one cannot avoid appearing on screen: the best it can do is call
`puter.ui.hideWindow()` once it boots, which reads as a window flashing
open and shut. In dashboard mode it was worse than a flash — the child
maximized into the tab and minimized its parent behind it, so asking a
service app a question took the user's app away from them.

So `launchApp` now accepts `background: true`, and the window starts
hidden. The app is otherwise entirely normal: it keeps its taskbar item,
so a user can see that it is running, show it, or close it, and it can
show itself with `puter.ui.showWindow()` whenever it has something to say.
Only a literal `true` counts, since the flag arrives over IPC from another
app.

The decision now lives in one predicate, `starts_hidden(app_info, options)`,
which folds this together with the existing app-level `background` flag and
is used everywhere the old flag was read — including the dashboard's
minimize-the-parent branch. `show_in_taskbar` deliberately still keys on the
app-level flag alone: an app that is always windowless has nothing to put in
the taskbar, while a background *launch* should stay visible there.

Existing callers are unaffected: with `background` unset, both paths
evaluate exactly as they did.
2026-08-13 09:47:35 -07:00
Daniel Salazar 8ed8feed4b fix: don't let an aborted upload take down the process (#3557) 2026-08-13 01:02:17 -07:00
Daniel Salazar 22f5bf5429 fix: duplicate emails (#3556) 2026-08-12 22:00:48 -07:00
Nariman Jelveh 198184f986 fix: drop the feedback dialog's rate limit entirely
Reopening still failed intermittently: any call the backoff refused looks
exactly like the user closing the dialog, so an app's "Send feedback"
button just did nothing, with nothing to see anywhere.

Rate limiting this dialog is not worth that. The other modals an app can
open with no user gesture — requestPermission, alert, prompt — have no
rate limit either, so one dialog at a time is the only rule left, and a
refusal now says so in the console.
2026-08-12 17:46:05 -07:00
Nariman Jelveh 026ae46530 fix: let an app reopen the feedback dialog after the user closes it
The abuse guard counted every dismissal that sent nothing as a strike
against the app, so closing the dialog blocked the app's next
showFeedbackDialog() for 10s, then 60s, then for the rest of the page's
life. The key is the app uuid, so relaunching the app did not clear it,
and a blocked call replies sent:false — the app just sees nothing happen.

Key the guard off how fast the app comes back instead. A reopen within a
second of the app's last dialog activity is machine-paced and gets the
same escalating backoff; attempts made while backed off count as activity
too, so a loop cannot wait out a tier and start from zero strikes. Any
human-paced reopen clears the app's record, so the tiers are only reached
by an app that reopens at machine speed several times running.

One dialog at a time, and a successful send clearing the record, are
unchanged.
2026-08-12 17:35:54 -07:00
Nariman Jelveh 0615c3aa3c feat: add "Show hidden" to the dashboard's folder context menu
The dashboard's files tab filtered out dot-files unconditionally, so
there was no way to reach them the way the desktop Explorer allows.
Add the same "Show hidden" toggle next to Refresh, driven by the
shared show_hidden_files preference so the two stay in sync, and dim
revealed rows with item-revealed like the Explorer does.

The socket-driven item.added path applies the same rule, so a hidden
file created elsewhere can't appear in a view that filtered its
siblings out.
2026-08-12 17:26:04 -07:00
Nariman Jelveh 748461aabb fix: show the new folder's row in the dashboard before mkdir answers (#3555)
* fix: show the new folder's row in the dashboard before mkdir answers

Both New Folder paths in the dashboard Files tab awaited mkdir before
drawing anything, so on a slow connection the button appeared to do
nothing for seconds. The toolbar path then did a full renderDirectory on
top of that -- a second round-trip, and with the default eventual
consistency the new folder could miss the listing entirely, leaving the
rename editor unopened.

Both now go through one createFolderInstant(), which renders the row
first: a locally predicted name (mirroring the backend's " (N)" dedupe
convention), a temporary uid, and the name editor already open. mkdir is
still asked for the plain "New Folder" path with rename: true, so the
server keeps owning deduplication; the prediction is only what we draw in
the meantime, and the row corrects itself if the two disagree. On failure
the row is withdrawn and the error is shown rather than swallowed.

Renaming is the next thing the user does and it needs the real uid, so
rename() awaits the create promise parked on the row. Everything else
that acts on the item -- open, menus, drag -- sits out while the row is
pending. mkdir's item.added comes back to the originating client (the
event carries no original_client_socket_id) and cannot match a temporary
uid, so _creatingItem still suppresses it, now as a counter: as a flag,
one create finishing uncovered another still in flight.

Two things found on the way:

- directory_depth_limit_exceeded had no translation key, so that alert
  (including the pre-existing desktop create_folder path) rendered the
  raw slug.
- The empty-directory notice is now restorable without refetching, since
  a withdrawn create has to put it back. It carries its own class because
  the loading overlay shares the same container.

Verified end to end against a dev backend with mkdir slowed to 3s:
toolbar and context-menu paths, rename and Escape before the response,
two overlapping creates, and a rejected mkdir.

* fix: don't draw the instant new-folder row while a listing is rebuilding

renderDirectory() clears .files before its readdir resolves, so a row
drawn during a load survives the clear and is left behind in whatever
directory the load lands on. Clicking the new-folder button while a
folder is still opening produced a row for `<old dir>/New Folder` sitting
in the new directory's listing, with the rename editor open on it —
clicking it navigated elsewhere and renaming it renamed a folder the user
wasn't looking at.

Drop the click while a rebuild is in flight, the same way renderDirectory
already drops navigation clicks that arrive while it renders.

* fix: stop drops onto a not-yet-created folder row from mangling files

A row drawn ahead of its mkdir carries a predicted path, but it was still
registered as a live drop target. Dropping a file on it called
move_items() with a destination that does not exist yet, and move treats
a non-existent destination as a rename target — so the dragged file was
silently renamed to "New Folder" instead of moving into the folder.
Reproduced against a slow mkdir: victim.txt became a 5-byte file named
"New Folder".

Sit the row out of the jQuery UI droppable (drop and the spring-load
hover) until the real fsentry lands, matching the guards already on
opening it, its menus and dragging it. The native-file dragster drop gets
the same guard: it uploads into the same predicted path.
2026-08-12 17:16:44 -07:00
Daniel Salazar 78cfe0c0f6 metrics: add metering buffer store coutns (#3554) 2026-08-12 15:55:03 -07:00
Nariman Jelveh 81f9f00fc0 fix: stop app feedback dialog clicks from focusing the iframe beneath
Clicking inside the feedback overlay (e.g. the textarea) reached
initgui's global mousedown -> focusWindow path: mouseover_window is
computed geometrically and is blind to the overlay, so focusWindow
focused the app window's iframe underneath — stealing keyboard focus
from the textarea and forwarding the click into the app.

Suppress it the established way (same fix as the dashboard app-group
panel): set window.mouseover_window = undefined in the overlay's own
mousedown handler, which runs before the document-level one; undefined
is the only value initgui's guard skips.
2026-08-12 15:00:32 -07:00
Nariman Jelveh 556380cf16 fix: keep app feedback send button blue on hover
The generic .app-feedback-btn:hover:not(:disabled) rule out-ranked the
.app-feedback-btn-primary:hover override (the :disabled argument inside
:not() counts toward specificity), so hovering Send repainted it with
the light --afb-hover background under white text. Strengthen the
primary hover selector so the blue gradient wins.
2026-08-12 13:42:11 -07:00
Daniel Salazar 908ec23eda fix: sanitize user data (#3552) 2026-08-12 12:08:54 -07:00
Juan Fernando Castro d5ae5a0049 🔧 PUR-1072: Flatten driver permissions to hardcoded values (#3545)
* refactor(permissions): drop hardcoded group permission map for a flat default

* test(drivers): assert credential-gate intent instead of a 403 proxy
2026-08-12 12:08:38 -07:00
Daniel Salazar 419d0aaa89 doc: update bounty doc 2 (#3551) 2026-08-12 09:56:34 -07:00
Daniel Salazar c3a0c28705 docs: update bug bountu (#3550) 2026-08-12 09:55:13 -07:00
Nariman Jelveh 7ceb2090b7 feat: app user feedback system (#3546)
* feat: app user feedback system

Add puter.ui.showFeedbackDialog(), letting users send feedback to an
app's developer. In the app environment the Puter desktop renders the
dialog; on a third-party website a puter.com popup hosts it. The message
is stored in a new app_feedback table and emailed to the app owner's
confirmed email — it never passes through the app's own code.

Feedback is strictly opt-in per app via a new apps.feedback_enabled
column (a real column, not an app-metadata key, so Dev Center's
whole-blob metadata saves can't silently erase it), settable through the
existing puter.apps.update path (feedbackEnabled).

Backend follows the layered stack: AppFeedbackStore (durable count
queries) -> AppFeedbackService (opt-in check, message normalization,
abuse caps, best-effort owner email) -> AppFeedbackController
(POST /app-feedback, GET /app-feedback/target). New app-user-feedback
email template uses the escaping-safe nl2br triple-stash.

Defensive by design:
- requireUserActor blocks app tokens, so feedback can't be submitted
  programmatically; guiOriginOnly keeps cross-origin pages out.
- App identity comes only from the validated IPC sender (desktop) or the
  browser-attested opener origin (popup), never from message contents.
- The send-feedback popup action is in NON_AUTH_POPUP_ACTIONS, so it
  never delivers a token to the opener.
- Layered limits: route rate limits, plus DB-count caps that fail closed
  when the limiter backend is down, plus a per-app daily owner-email cap.
- Owner email is fully best-effort: an unconfigured transport,
  unconfirmed/unsubscribed/suspended owner, or send failure never fails
  the request or blocks storage.
- The dialog and SDK method are resolve-only and always settle, so a
  caller is never left hanging.

Migrations for sqlite/mysql/postgres, puter.js types, docs, backend
tests (sqlite + postgres), and a Playwright e2e spec are included.

* feat: add feedback control to the dashboard app-drawer

Surface the feedback dialog directly from the app window's chrome in
dashboard mode: apps that opt in (apps.feedbackEnabled) get a "Send
Feedback" button in the dashboard app-drawer, next to minimize/close.
It opens the same UIWindowAppFeedback dialog, targeting this app by uid.

The control is only rendered when the app opted in — feedback_enabled is
threaded from the launched app's metadata into the window options — and
the dialog still re-checks opt-in server-side, so a stale flag can't send
anywhere. Reuses the existing .dashboard-app-drawer-btn styling and the
app_feedback_title i18n string, so no new CSS or strings.

Adds e2e coverage: the control appears and opens the dialog for an
opted-in app, and is absent for an app that hasn't opted in.

* feat: enable app feedback by default in Dev Center

New apps created in Dev Center now have feedbackEnabled set on creation,
so users can send the developer feedback without any extra setup. A "User
Feedback" toggle in the app's settings lets developers turn it off (and
back on); it's wired into the save payload, the dirty-state tracking, and
the reset-to-original path like the neighboring toggles.

The Save update omits feedbackEnabled unless the toggle is present, and
the backend leaves an omitted field untouched, so the default survives
the create-then-save flow Dev Center runs. Add an SDK apps-suite guard
covering that round-trip (create-on -> unrelated update keeps it -> can
be turned off).

* fix: feedback modal polish + share sender email

Address four issues with the app feedback UI:

- Dashboard app-drawer: the extra "feedback" control pushed the close
  button past the drawer's derived width and clipped it. A `has-feedback`
  modifier widens the surface by one button + gap so all three controls
  fit. The control's glyph is now a message bubble with text lines, which
  reads more clearly at 14px than the previous bare speech bubble.

- The feedback dialog is no longer a UIWindow. It's a from-scratch
  overlay modal in the spirit of the dashboard modals (uninstall,
  add-app): a fixed scrim + centered card with self-contained,
  theme-aware color tokens (light default + dark override), a bottom-sheet
  layout on narrow screens, backdrop/Escape close, and an entrance
  transition. This renders consistently across the three contexts it's
  opened from (desktop app-IPC, dashboard drawer, standalone popup), so
  the callers no longer pass UIWindow-specific window_options.

- Feedback now shares the sender's email (not just their username) with
  the developer so they can respond: the owner email sets Reply-To to the
  sender and shows the address in the body — but only when the sender's
  email is verified (an unverified address could be anyone's, so it's
  never used as a reply target). EmailClient.send gains an optional
  replyTo. The dialog note now says the email will be shared.

Tests: e2e updated for the new modal (7 pass); backend feedback suite
covers the verified/unverified sender-email split (sqlite + postgres);
EmailClient + GUI unit suites pass; type-check clean.

* fix: resolve 'app-'-prefixed app names in feedback target lookup

APP_NAME_REGEX allows names beginning with "app-" (e.g. the seeded
app-center), but resolveTargetApp's startsWith('app-') heuristic sent
those to a uid-only lookup with no name fallback, so feedback for such
apps 403'd even when enabled. Use AppStore.resolveApp (uid, then name)
like the rest of the codebase.

* fix: make feedback daily caps fail closed under concurrent submissions

The per-user and per-user-per-app caps were check-then-insert and the
per-app email cap was count-then-send, so parallel requests (or multiple
nodes, or the route limiter failing open) could all read a stale
under-cap count and push past every limit — the exact scenario the
DB-backed caps exist to stop.

Now the user caps recount after the insert (own row included) and roll
the row back with 429 if a burst breached them, and the email cap claims
its slot (email_sent=1) before sending, recounts, and releases the slot
if over cap or if the send fails.

* docs: disclose the Dev Center's feedback-on-by-default in SDK docs/types

The Dev Center deliberately creates apps with feedbackEnabled (see
32c950de5), but the SDK docs, apps.d.ts, and AppFeedbackService's class
doc all described feedback as strictly opt-in / default-false with no
qualification — so a Dev Center developer reading them would wrongly
conclude feedback is off for their app. State the Dev Center behavior
alongside the API default, and correct the update-path docs: an omitted
feedbackEnabled leaves the current value unchanged rather than
defaulting to false.

* fix: don't mint a user-app token as a side effect of the feedback popup

Every embedded_in_popup boot ran the user-app token exchange, and the
exchange is a write: /auth/get-user-app-token bootstraps an app row for
the opener origin, grants flag:app-is-authenticated (what makes the
site count as connected to the account), and creates its AppData dir.
So merely opening — or immediately cancelling — a send-feedback popup
recorded a user<->site relationship the read-only feedback flow never
needs: the server resolves the feedback target from the attested origin
without any of it.

Gate the exchange behind runsUserAppTokenExchange(action) in all three
popup paths that mint (main postAuthActions exchange, temp-user signup
success, manual signup fallback). request-permission keeps the exchange
since grants are written against the app row it bootstraps.

* fix: refuse feedback when the deployment cannot deliver it

With no email transport configured (the common self-hosted default),
submissions were stored in app_feedback — a table with no read path
beyond the abuse-cap COUNTs — the owner email was silently skipped, and
the sender was still shown 'Feedback sent. Thank you!'. The developer
never learns the feedback exists while the user believes it was
delivered.

Gate acceptsFeedback on clients.email.isConfigured so the pre-flight
reports enabled:false (the dialog shows its 'not accepting feedback'
pane) and submit returns 403 instead of swallowing messages. Owner-level
store-without-email cases (unconfirmed owner email, per-app email cap
overflow) keep their existing deliberate semantics.

* fix: settle showFeedbackDialog instead of hanging on older host GUIs

In the app environment showFeedbackDialog awaited an IPC reply with no
capability check. A host GUI that predates this feature (self-hosted
Puter running the live js.puter.com SDK) has no handler for the message
and never replies, so the promise documented as 'never rejects' also
never resolved.

The GUI now advertises the IPC dialogs it can answer via a
puter.gui_features param on the app iframe URL, and the SDK resolves
false when 'feedback-dialog' isn't listed. A reply timeout could not
substitute: legitimate replies only arrive when the user closes the
dialog, so any timeout would false-negative while the user is typing.
Older SDKs ignore the extra param.

* fix: guard app-triggered feedback dialog against desktop-lockout loops

The showFeedbackDialog IPC handler had no re-entry or abuse guard, and
the dialog it opens is a full-viewport overlay above the taskbar and
every window — so 'while (true) await puter.ui.showFeedbackDialog()'
kept the desktop permanently covered (for signed-out users, the same
loop spams the full-page signup window instead). Every dismissal just
settled the promise and let the app immediately reopen it.

Allow one dialog at a time, and back off reopens per app after each
dismissal that sent nothing: 10s, then 60s, then blocked until page
reload. A successful send resets the backoff, and user-initiated paths
(dashboard drawer) are unaffected since they don't go through IPC.

* fix: carry feedback_enabled in suggested/recommended app summaries

launch_app uses options.app_obj verbatim when provided, and the
suggested-apps launch paths (open_item.js, UIWindowSearch.js,
UIDesktop.js) pass summaries from toAppSummary — which omitted
feedback_enabled. So an opted-in editor launched by opening a .txt file
showed no Send Feedback control in the dashboard drawer, while the same
app launched from the Apps tab (full puter.apps.get object) did.

* fix: gate feedback dialog Cancel/X on an in-flight submit

Escape and backdrop clicks were already ignored while the POST was
pending, but the X, Close, and Cancel buttons weren't — clicking one
mid-send settled the promise false and tore down the overlay while the
submission still landed server-side: the developer got the email, the
app was told sent=false, and a user who resubmitted 'the failed one'
sent a duplicate and burned a daily-cap slot.

Apply the same !sending gate to the buttons and disable them visually
while the send is in flight.

* fix: reject feedback origins longer than the source_origin column

readTargetParam accepted values up to 3000 chars but the raw origin is
stored verbatim into source_origin VARCHAR(2048) (MySQL/Postgres), so a
2049-3000 char origin passed every validation and then blew up the
INSERT with an HTTP 500 on Postgres/strict MySQL — or was silently
truncated on non-strict MySQL, corrupting the abuse-forensics value the
column exists for. Cap the param at the column size.

* fix: make the feedback dialog's privacy note match what is shared

The note unconditionally said 'Your email address will be shared with
the developer so they can respond', but AppFeedbackService shares the
username always and the email only when it exists and is verified — an
unverified or temp-user sender was promised a reply path that never
materializes, and nobody was told about the username.

Show 'username and email' when the signed-in user's email is verified,
and 'username' otherwise.

* fix: stop HTML-escaping email subject lines

Subjects were compiled with default Handlebars escaping, so the
app-user-feedback subject rendered a title like "Bob's App & Games" as
"Bob&#x27;s App &amp; Games" — literal entities in the recipient's mail
client. Subjects are plain-text headers, not HTML; compile them with
noEscape. Header safety is unaffected: the transport encodes newlines
and free-form values collapse whitespace upstream.

* docs: state exactly what showFeedbackDialog shares with the developer

The doc claimed the dialog 'tells the user their username will be
shared', while the dialog's note talked only about the email address and
the implementation shares the username always plus the email (as
Reply-To) only when verified. Describe the actual disclosure: username
always, email when verified.

* docs: document the COOP false-resolve limitation of showFeedbackDialog

Under COOP the popup's opener link is severed, so the SDK deliberately
resolves false while the popup stays open and the user can still submit
(a feedback submission has no server read-back the way a permission
grant does). The documented contract ('resolves to true if the user
submitted feedback') was silently wrong on cross-origin-isolated pages —
state the limitation in the doc and the SDK jsdoc: false means 'not
confirmed', not 'not sent'.

* feat: point the feedback email footer at the Dev Center

The footer told developers to turn feedback off via a puter.apps.update
one-liner, but the toggle lives in the Dev Center app settings — and
Dev Center is where apps get feedback enabled by default in the first
place. Reword it to 'manage it in the Dev Center' with a link built
from config.origin (like app_link) so it holds on self-hosted
deployments.

* test: cover the feedback service and store layers directly

The service owns every feedback business rule — target resolution,
eligibility, message normalization, the durable caps, and the owner-email
preconditions — but was only reachable through the controller's tests. Give
it and the store their own suites so a regression names the layer it broke.

Service coverage adds the branches the route tests could not reach: a
blocked origin resolving to null rather than surfacing a 403, owners who are
suspended or unsubscribed, length measured after normalization, the 24h cap
window boundary, subject-header injection via the app title, and the email
links being rooted at config.origin.

The two describes already labelled `AppFeedbackService ...` move out of the
controller test, which keeps only the caller-facing promise that a failed
send still returns success.

* refactor: prep the feedback cap error once instead of via a factory

Both throw sites are in one call and only one can ever run, so a plain
const reads the same and drops a function that existed only to defer a
constructor.

* Require verified users for app feedback API

Add `requireVerified: true` to both app feedback routes (`GET /target` and `POST /`) in `AppFeedbackController`. This tightens access control so only verified user accounts can fetch feedback targets or submit app feedback.
2026-08-12 09:49:24 -07:00
jelveh ea63c3093e feat(dashboard): add Publish as website and Properties to folder menu
The Files tab's folder-background context menu (empty listing area,
sidebar folders, breadcrumb segments) now offers Publish as website and
Properties for the folder the menu was opened on, matching the desktop's
folder menus. Both are omitted at the filesystem root and in Trash, and
Publish is omitted inside trashed folders.

The properties modal can now resolve an entry by path when the caller
has no uid, which the folder-background menu doesn't.

The publish gate (account creation + email confirmation) was duplicated
in three places, so it moves to a publish_as_website helper that all
call sites now share.
2026-08-12 08:50:03 -07:00
Daniel Salazar e8236a100c chore: bump versions (#3548) 2026-08-12 01:18:17 -07:00
Daniel Salazar 20b3b88e39 metering: big fixes to metering + jsdoc types (#3547)
Changes are:
- global egress metering
- remove file egress cost
- introduce file op cost for the per request cost s3 has
- enforce fs read/download etc to through 402 when out of usage; allow for subdomains
- enforce kv metering when out of usage through 402; allow for workers
- jsdoc as source of truth for puter.js types
- kv driver caching for get and batchget operations with decreased costs
2026-08-12 01:06:04 -07:00
Chen, Ting-An cbae39a08f fix(i18n): add Traditional Chinese app management strings (#3541)
Translate the app discovery, app request, desktop shortcut, and app folder controls that currently fall back to English in the Traditional Chinese interface. Preserve the positional placeholders used by accessibility labels.
2026-08-11 21:18:37 -07:00
Daniel Salazar afad3e37bf fix: server health tolerance for primary read (#3542) 2026-08-11 01:19:09 -07:00
Daniel Salazar a6b2161b2c fix: misc sec fixes (#3540) 2026-08-10 22:24:44 -07:00
Neal Shah 9e864e9574 PUT-1432 (#3539) 2026-08-10 23:33:20 -04:00
Daniel Salazar 79d4201f12 fix: rate limits, AI routing, and a type-check gate (#3529)
- declare rate + concurrency limits on every route and driver that lacked one
- add acquireConcurrent for websocket connections and the DAV mount
- bucket AI models by identity key only; keep resold duplicates of any vendor
- skip recently-failed provider routes; cap the fallback chain at 3 attempts
- let full-access access tokens bind a worker to an app their own user owns
- cache resolved subscriptions so tiered limits don't add a round trip
2026-08-10 19:09:47 -07:00
Daniel Salazar 0c2d7dfa34 feat: add user created timestamp on whoami (#3538) 2026-08-10 19:08:50 -07:00
Neal Shah 2a6d2fea32 allow max-tokenless models on openrouter (#3536) 2026-08-10 18:40:10 -04:00
Juan Fernando Castro 1dec5e90c4 docs: add requestAppData() to Perms sidebar (#3534) 2026-08-10 11:38:09 -07:00
Neal Shah 566298d89c make context length on togetherAI more lenient (#3533) 2026-08-10 13:14:33 -04:00
Daniel Salazar 6bb5eacf61 fix: metering kv spamc (#3532) 2026-08-10 08:36:56 -07:00
Reynaldi Chernando 390a83c860 Fix puter.js docs and types drift 10 aug (#3531) 2026-08-10 12:47:18 +07:00
Nariman Jelveh 8f23bf7b75 fix(gui): hover the add-an-app options with the accent wash, not grey
The rows already answer hover with a --select-ring border; filling them
with the grey --dashboard-hover under that accent ring read as two
systems disagreeing. The weak select tint keeps the hover in one voice
and stays under the icon chip's stronger tint in both themes.
2026-08-09 18:23:07 -07:00
Nariman Jelveh aa85dddd38 Refresh Add App AI description copy
Update the English `add_app_ai_desc` translation string in the GUI to a more direct, enthusiastic message: “Describe the app you want and AI will build it!”.
2026-08-09 18:21:31 -07:00
Nariman Jelveh 8284f35b93 fix(gui): honor godmode file_paths launches now that /apps returns booleans
The godmode gate in ExecService.launchApp checked `godmode === 1`, but
GET /apps/:name serializes the flag as a boolean, so the check always
failed and file_paths launches (e.g. Dev Center opening a worker's
source in the editor, #2218) silently dropped the file. Accept both
shapes, matching the existing guard in launch_app.js.

Also fix the two issues hiding behind that gate:
- puter.apps.get returns `uid`, not `uuid`, so the /sign call for the
  target app sent app_uid undefined — no write grant and no user-app
  token for the launched app. Use `uid` with a `uuid` fallback.
- The closeApp IPC handler had the same `godmode === 1` comparison,
  preventing godmode apps from closing other apps' windows.
2026-08-09 18:06:58 -07:00
Nariman Jelveh ccec3ce39b fix(gui): open add-an-app choices through the deep-link intro, not a + tile morph
60686643b made App Center and the AI builder morph out of the + tile the
user clicked, which answers where the window came from but not the
question these two options exist to answer: where the APP went. The grid
never showed it arriving, so the user still ended a launch with nothing
to find, no tile for minimize to fly back to, and no lesson learned.

Choosing either option now plays exactly what an /app/<name> landing
plays (beginDeepLinkLaunch): when the app already has a tile the intro
travels to it, plays the click flourish, and the window grows out of the
app's own slot; when it doesn't, the tile is INSTALLED first — the slot
opens, the progress stroke draws, the icon springs in — and only then
does the window morph out of the tile that just arrived. Same beats,
same exposure decay, same interruptibility, same duplicate-launch claim.

The app info is prefetched in parallel and handed to launch_app, the
same contract as initgui's landing path, so the intro (which may need to
draw the arriving tile from it) never costs a second round-trip.

This also reverts 60686643b's anchor plumbing (UIWindow's
dashboard_tile_el option and the dashboard_tile_in_view export): the
intro installs a real tile for the by-app-name morph to find, so no
stand-in anchor is needed.
2026-08-09 18:02:39 -07:00
Nariman Jelveh 60686643b3 fix(gui): launch from the add-an-app modal with the tile morph, not a fade
Choosing App Center or the AI builder opened the window with the plain
fade while every other launch in the Apps tab grows out of the icon that
was clicked — the one launch that most needs to say where it came from
read as the app appearing from nowhere.

The morph resolved its anchor by app name, which cannot work here: the
whole point of these two options is installing an app you do not have,
and an app with no tile has no name to find. So the launcher may name its
anchor element (dashboard_tile_el), and TabApps resolves it once before
launching so the click flourish and the window's half can never disagree
about which tile the launch came from: the app's own tile when it already
has one on screen — minimize flies the window back there, so that is
where opening should come from — and otherwise the + tile itself.

dashboard_tile_in_view's visibility test is now a named helper it shares
with the new path, so an anchor on an offscreen pager page is still
rejected and still falls through to the fade.
2026-08-09 17:49:50 -07:00
Nariman Jelveh 61bfcbdc83 feat(gui): add an add-an-app tile at the end of the Apps grid
The Apps grid showed what you have and offered no way to get more. A
dashed plus tile now rides at the tail of every page of it: after every
app, on the last page, and nowhere else. It wears the same tile skeleton
as an app so hover, focus, and arrow keys treat it identically, but
carries no app name or group id — which is what keeps it out of the
saved order, the running dots, and the deep-link tile lookups. It cannot
be picked up, is never a drop target (so no app can land past it), has
no context menu, and sits still while the apps jiggle in reorder mode.
Search results leave it out: a query asks which apps you have.

Clicking it asks how you would like to get an app: find one in the App
Center, have one built by the AI builder, or request one that does not
exist yet. The request is a form in the same card rather than the stock
Contact Us window — a titled pane one step deeper, whose Back arrow
returns to the options with the draft intact. It posts the same
/contactUs request that window did; a failure is said inside the form,
where the request is still there to retry, rather than in an alert
window the next click on the dashboard would bury.
2026-08-09 17:37:09 -07:00
Nariman Jelveh c8985b621d Add copy-link support for app tiles
Extract app tile URL generation into a shared helper and use it for a new dashboard context menu action that copies the tile's destination. This keeps copied links aligned with the existing open-in-new-tab behavior and adds regression coverage for app, external, encoded, and folder tiles.
2026-08-09 16:54:18 -07:00
Nariman Jelveh 42299a64ed Update recommended apps list
Add 'blockarena' and remove 'pretty-tiles', 'galaxy-troops', and 'blend-fruits' from the recommended apps list.
2026-08-09 11:20:40 -07:00
jelveh 344db5cb0b Remove vault from recommended apps
Drop `vault` from the recommended apps list so it no longer appears in app recommendations.
2026-08-08 11:54:56 -07:00
jelveh 66824f607d feat(gui): install an app onto the grid when its landing is what installs it
A /app/<name> landing for an app the dashboard didn't list opened with
a plain fade and left the app NOWHERE: the grid had been fetched before
the launch granted the permission that installedApps reports, so Back
found no tile to minimize into and the user landed on a dashboard
without the app they were just inside. The round trip that teaches
"windows are inflated tiles" broke on exactly the landing where a new
user is forming that model.

Now the landing SAYS what it does. The app joins the grid at the tail
(where new apps land) with a tile drawn from the landing's own app-info
prefetch -- only a confirmed-real app ever materializes, with its real
icon -- held invisible until the intro has revealed the grid and
travelled to the tile's page, and then it is INSTALLED, in the grammar
every app store taught: the slot opens (the folder well's own surface)
with the icon dim inside it, a progress stroke draws clockwise around
the slot, and on completion the icon springs to full color and size as
the label names it. Only then does the flourish play and the window
grow out of the tile. The arrival is the one beat that never decays:
per-app news delivered at most once per app, not a repeated lesson.
Minimize now has a target, and the app is simply there afterward.

The regression surface is guarded on every edge: the parked-invisible
state is re-applied by renderApps itself so mid-intro re-renders can't
strip it, and settleDeepLinkLaunch -- which the landing's finally
always reaches -- force-reveals it, so no path leaves an invisible
tile. A _pendingInstalls overlay (the mirror of _removedLocal) keeps a
refresh fetched before the grant lands from evicting the tile, and
retires itself once installedApps confirms the app or the user
uninstalls it again. The order is never saved on the splice, so the
default-position append can't freeze a custom order the user never
made. Failed or absent app info adds nothing; installed-app landings
are untouched; animations-off and reduced-motion users skip the
choreography but still get the tile, silently. The progress ring is a
transient element (the tile's ::before is the well, its ::after the
running dot) drawn as an SVG dashoffset sweep -- no Houdini dependency
-- and the app-info prefetch now asks for the 128px icons every other
tile already uses.
2026-08-08 11:25:48 -07:00
jelveh 66dedf3082 feat(gui): let the app drawer's intro flash retire once learned
The control drawer on headless dashboard apps opened itself on EVERY
window open, forever -- 2.6 seconds of tray over the app's own top
edge to teach a lesson (the controls live in this tongue) that is
learned in a few exposures. Same doctrine as the dashboard's
deep-link intro: pedagogy that knows when to step aside.

A per-account kv counter records deliveries, once per window: a flash
played while the user could see it, or the user opening the drawer
themselves (hover, tap, focus) -- the stronger proof, since the
drawer is a headless app's only chrome and must never decay out of a
user's awareness. After three, new windows keep the bare tongue and
the flash stops; hover, tap, and focus expansion never decay.

The attach is synchronous and the stored count is not: the first
drawer of a session races the read against a short cap and defaults
to teaching -- a hung read must not hold the intro hostage, and every
failure mode (unparseable value, failed read, timed-out read, failed
write) errs toward teaching once more, never toward never teaching.
The resolved count is cached and kept current in memory, so later
windows decide synchronously and a session's own not-yet-written
increments still count. kv.incr keeps two devices from losing an
increment, and the pre-read caps the key once the lesson is learned.

If the user beats a still-deciding flash to the drawer, the flash is
skipped rather than played over their hand -- its auto-collapse would
shut a drawer they are actively using. A flash in a hidden tab still
defers to first reveal (it rides requestAnimationFrame, as before)
and an unseen one is never counted.
2026-08-08 10:25:26 -07:00
Daniel Salazar 95da802b13 fix: metering hardening (#3527) 2026-08-08 06:10:01 -07:00
Daniel Salazar 0491d5fea7 fix: mcp better uploads (#3528) 2026-08-08 06:09:49 -07:00
Juan Fernando CastroandDaniel Salazar d202be10a9 feat: let apps use another app's data with user consent (#3516)
* feat(perms): add cross-app app-data permission vocabulary

* feat(perms): sweep app grants by permission prefix

* feat(perms): resolve and withdraw cross-app data grants

* feat(kv): support an authorized namespace override and per-key privacy

* feat(kv): gate cross-app KV access behind app-data grants

* feat(fs): allow cross-app AppData access and require a scope to delete

* feat(auth): accept permission lists and gate app-data grants

* feat(perms): add requestAppData to the puter.js SDK

* feat(gui): carry permission lists through the IPC and popup transports

* feat(gui): describe cross-app data requests in the consent dialog

* docs: document requestAppData and per-entry KV privacy

* perf(perms): sweep cross-app grants only for origin-bootstrapped apps

* fix(gui): stop double-encoding cross-app consent text

* fix(perms): close three gaps in cross-app grant enforcement

* fix(kv): meter and batch the per-entry privacy probe

* fix(perms): resolve app identifiers and scopes more strictly in the SDK

* test(perms): cover the cross-app consent flow end to end

* fix: small missing token resolution for app

also adds the same exclusion for the batchPut api, small change

* fix: make resolved actor optional

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-08-08 04:04:07 -07:00
jelveh 2c17476c07 fix(gui): folder pages, live drag-out, and one merge pulse in My Apps
Three fixes to the My Apps folders:

A folder big enough to need it now paginates like the grid outside
(iOS folders page too) instead of growing a scrollbar: pages of
cols x 3 tiles in a scroll-snap scroller with dots below, wheel and
swipe to flip, and a drag held at the card's edge flips folder pages
so reordering works across them. The card's height stays constant
across pages, and a deep-link landing on an app that lives on a later
folder page travels there before the launch morph.

Dragging an app out of the open folder no longer drops it blind:
after a beat held outside the card, the app leaves the folder, the
card closes behind it, and the SAME drag carries on over the grid --
placeholder shuffle, edge page flips, even a drop into another
folder -- so the app lands where the user watched themselves put it.
A quick drop during the beat still lands beside the folder, and
swinging back inside within the beat cancels cleanly.

The folder well's fill played twice on nearly every merge: the rest
countdown anchored at first contact with the target, so the hand
decelerating INTO the tile always read as movement when the dwell
elapsed, and the deliberate refill-from-empty replayed the fill. The
countdown now re-anchors on the move events themselves -- rest is
only visible at its edges -- so the fill that completes is the one
started by the last movement, played exactly once.
2026-08-08 01:28:09 -07:00
Nariman Jelveh 06d94534e8 feat(gui): iOS-style app folders in the My Apps grid (#3525)
* feat(gui): iOS-style app folders in the My Apps grid

Drag one app onto another and let it settle: a well opens under the
target and the drop makes a folder of the two. Dropping onto an existing
folder joins it. A folder opens by growing out of its own icon into a
card over a blurred grid, where its apps can be launched, rearranged,
renamed, or carried back out.

Hovering a tile mid-drag means two things — "push over, I'm passing
through" and "swallow me" — and the tile is barely bigger than its icon,
so pixels can't separate them; motion does. The shuffle is held while
the folder offer stands and fires when the icon leaves the tile, or at
the drop, so a quick drop onto a neighbour still reorders exactly as it
did. The offer itself re-arms rather than cancelling on movement: the
last events of a drag are the ones carrying the icon onto the target and
nothing is dispatched while it rests, so a cancel-on-movement dwell
could never fire at all.

Folders are stored in their own kv key; the grid's ORDER stays entirely
in the existing saved app order, with a folder occupying its first
member's slot and its members contiguous. Every saved order therefore
stays valid with no migration, and an app whose installedApps page
failed to load keeps both its folder and its position. Folders never
nest, one that drops below two apps dissolves, and a corrupt kv value
degrades to "no folders" rather than a broken tab.

Elsewhere:
- Search looks THROUGH folders — a match the user then has to hunt for
  inside one is not an answer.
- Minimize morphs into the FOLDER when an app lives in one, and a
  /app/<name> landing opens the folder so the launch grows out of the
  icon where the app actually is.
- The uninstall FLIP keyed surviving tiles by app name, which a folder
  tile doesn't have; it now keys by identity.

New pure model in appGroups.js with tests; verified end to end in the
running dashboard (create, join, open, rename, reorder, eject, ungroup,
launch-from-folder) alongside plain reorder, search, and uninstall.

* fix(gui): keep the folder name field from inheriting input[type=text] sizing

style.css styles every input[type=text] with `width: 100%` and grows it to
`padding: 7px; border: 2px` on focus. `input.myapps-group-name` matches at the
same specificity, so it only wins the properties it actually declares — width
was never one of them, and the focus rule declared neither padding nor border
width. The name field therefore spanned the entire folder card (so its hover
and focus chip read as a full-width bar rather than the name) and grew 8px
taller the moment it was clicked, shoving the folder's app grid down.

Spell the three out, in both the resting and the focus rule — the same trap
.myapps-search already documents next door.

* fix(gui): size folder icon ghosts to the icon they stand on

Border-box only reaches a folder's icon through `.dashboard * { box-sizing }`,
and every ghost cloned from one is appended to <body>, outside that rule: the
drag ghost, the click-time launch flourish, and the open/minimize morph ghosts
all fall back to content-box, where .myapps-group-icon's 5px padding is added
to the 56px slot. Each ghost rendered 66px square and 5px off, so it visibly
popped at exactly the moment it was supposed to sit flush on the real icon.

State box-sizing on the rule itself so a clone carries it wherever it lands.

* fix(gui): stop a closing folder from swallowing the next click

_closeGroup drops the open class and leaves the overlay in place for
GROUP_PANEL_CLOSE_MS so the card can recede into its tile. The scrim is
`position: fixed; inset: 0` and still hit-testable for that whole quarter
second, so a click on the grid during it landed on the outgoing overlay — whose
handler only re-runs _closeGroup, now a no-op. Shutting a folder and reaching
straight for an app did nothing.

Take the outgoing overlay out of hit-testing; it has no interactive job left.

* fix(gui): keep a folder name typed right up to the moment it closes

The name box commits on blur, and every exit that goes through a pointer blurs
it while the folder is still open — so clicking outside, or launching an app
from inside, keeps what was typed. Escape does not: _closeGroup clears
_openGroupId first and only then moves focus to the tile below (or removes the
card outright), so the blur arrives with no open folder to rename and
_renameGroup drops it. A brand-new folder opens with its name selected for
exactly this edit, so "type Games, press Escape" — the obvious way to dismiss
a dialog — was the path most likely to lose it.

Commit the pending name on the way out, before the folder id is gone.

* fix(gui): close an open folder when the Apps tab is re-entered

The dashboard hides an inactive section and calls onActivate on the way back
in; there is no deactivate hook, so a folder left open survives the round trip
and greets the user still open over a grid they walked away from. Worse,
onActivate's focusSearch then lands the caret in the search box behind the
folder's scrim, and typing filters the grid the card is covering — a modal with
the keyboard pointed outside it.

Shut the folder as the tab comes back: returning to the tab is returning to
the grid.

* fix(gui): move focus into a folder when it opens

Opening a folder called .focus({ preventScroll: true }) on a jQuery
object. jQuery's .focus() shorthand reads a lone non-function argument
as event DATA and binds a handler with it, so it never moved focus:
the folder opened modal over the grid with focus still on the tile
behind its scrim, where Tab walked away through the inert grid instead
of cycling inside the dialog — and the object it bound as a handler
threw a TypeError on that tile's every subsequent focus.

Focus the DOM node instead, as every other focus call in this file
already does.

* fix(gui): stop renaming a folder from swallowing the click that commits it

The folder's name box commits on blur, and blur fires on the PRESS —
before the click that press belongs to. Committing re-rendered, and the
re-render replaced every tile in the open folder, so by the time the
click was dispatched the tile under the pointer was detached and the
delegated handler never saw it. Typing a name and then tapping an app
in the folder — the path a brand-new folder puts the user on — renamed
the folder and did nothing else; the app only opened on a second click.

Rebuild the folder's contents only when they actually differ from what
is on screen. A rename doesn't change them, so nothing is detached, and
a background refresh no longer throws away hover/focus either. Tiles
that survive get their drag resting-rects cleared, since the card can
have moved under them since the rects were taken.

* fix(gui): keep Enter in a folder's name box from leaving the folder

Committing the name with Enter blurred the box, which left focus on
<body> — outside a dialog that is marked aria-modal and that traps Tab
on its own subtree. The next Tab therefore walked off through the inert
grid the folder is covering, exactly what the trap exists to prevent.

Step out onto the folder's first app instead; the same blur still
commits the name. The keystroke is stopped at the box because the
grid's document-level key handler reads Enter on a focused tile as
"launch it", and would otherwise have taken the focus move as its cue
to open an app the user never asked for.

* fix(gui): stop an open folder clipping its own uninstall badges

The folder's grid scrolls, so it clips anything outside its padding box
— and reorder mode's uninstall badge deliberately overhangs the top-left
corner of every tile. The top row's badges therefore rendered as flat
tabs rather than circles, on the one surface where they are a touch
user's only way to uninstall an app they have filed away.

Give the scroller 9px of top padding for the overhang to sit in and take
it straight back off as margin, so the card and everything in it stays
exactly where it was.

* fix(gui): hold page edge-flips while a folder merge is being offered

A tile in the pager's last column sits inside the 60px edge-flip zone, so
resting a dragged icon on it — the folder-making gesture — armed the edge
dwell alongside the merge dwell, and the page flipped out from under the
very folder the user was watching form (the merge target scrolls away
mid-offer, and the drop then resolves against its stale resting rect).
Worst on phones, where 72px tiles overlap the zone across the whole last
column.

A live merge offer now holds the edge flip: entering the zone arms no
dwell while an offer stands, and an offer that arrives during a running
dwell is re-checked at the flip (a resting pointer fires no event that
could clear the timer). Carrying the icon off the tile withdraws the offer
and the hold with it, so deliberate flips — resting in the bare edge
gutter — behave as before.

* fix(gui): make Escape cancel a folder rename instead of saving it

Escape pressed mid-edit fell through to the folder's close handler, and
closing commits whatever the name box holds — so the one key every inline
rename uses for "never mind" stored the abandoned half-typed name. Escape
in the box now puts the stored name back and steps out to the folder's
tiles, exactly the cancel Finder/Explorer taught; with nothing left to
cancel (name untouched) it falls through and closes the folder as before,
so a second press still exits.

* fix(gui): refill the folder well when the merge countdown restarts

The merge dwell pins its stillness anchor at first contact with the tile,
and a hand decelerating INTO a target routinely covers more than the
7px allowance between that moment and the first tick — so the countdown
quietly restarts. The well's fill, tuned to the same 460ms, had already
completed by then and just sat there half-open: a drop it seemed to
promise a folder for actually reordered. Restart the fill with the
countdown, so what the well shows is always the countdown that is
actually running.

* fix(gui): keep keyboard focus inside an open folder across its edits

The folder card is a modal dialog, but three of its flows stranded focus
on <body>, where Tab walks the inert grid behind the scrim:

- Remove from Folder rebuilds the card's grid after the context menu has
  dropped focus, so nothing inside the dialog holds it. The rebuild now
  hands focus back to the same app's tile (or the first) whenever it finds
  focus on <body> — never stealing from an uninstall modal, which holds it
  legitimately.
- The same eject dissolving the folder re-renders the grid right after
  _closeGroup's focus hand-back, replacing the very tile it chose. The
  ejected app's own tile — where the user is looking — takes focus instead.
- A click on the card's empty space focused nothing at all. The card now
  carries tabindex=-1 so such clicks land on it (no visible ring; a focus
  ring around the whole card would misread plumbing as selection), and the
  Tab trap wraps from there instead of stepping off through the scrim.

* fix(gui): size the folder name box to the name it holds

The box sat at the browser's default ~20ch regardless of content: a short
name floated in a hover pill far wider than the word, and a 40-character
one clipped while the card had room to spare. field-sizing: content hugs
the text between a 90px floor and the card's width, iOS-style; engines
without the property keep the default box exactly as before.
2026-08-08 00:28:03 -07:00
Neal Shah aea828ff12 add worker create event and new query methods (#3524) 2026-08-07 17:51:37 -04:00
Daniel Salazar c24f0fe08b fix: mcp fs copy, move, rename (#3517)
* fix: mcp fs copy, move, rename

* feat: more mcp tools for kv
2026-08-07 17:14:40 -04:00
jelveh 085ac116c7 fix(gui): keep app landings through signup, and unstack the session picker
Signing up from a direct /app/<name> landing redirected to '/' on
success, silently dropping the app the URL asked for — the deep link's
launch (and its dashboard intro) never happened. Signup now mirrors
login's pathname-preserving redirect, but only for app-landing routes
(/app/<name>, /desktop/app/<name>): every other route keeps the
historical '/', notably /action/signup, where returning to the same
path would just show the signup form again. The query string stays
dropped, matching login's credential-leak hygiene. This covers all
three signup entry points reachable from a landing: the login cover's
"Sign up", the session picker's "Create Account", and the
must_login_or_signup fallback.

The session picker (UIWindowSessionList) also left itself on top of
the cover windows its two links open, hiding their username fields:
"Create Account" tried to close the picker via the LOGIN window's
c2a selector (which matches nothing in the picker), and "Log Into
Another Account" never closed it at all. Both now close the picker —
in the reload flows only: the no-reload (popup) flows keep it open,
where it doubles as the fallback UI when the login/signup window is
abandoned mid-flow.

Picking an account was already correct (location.reload() keeps the
landing URL); with these fixes all three picker paths, plain login,
and signup all return to the app landing, where the boot replays the
launch and its intro.
2026-08-07 11:51:44 -07:00
jelveh 5538a5d24b feat(gui): let the deep-link intro step aside for users who know it
The intro exists to teach the dashboard's spatial model; once learned it
would only tax every bookmarked landing. Two mechanisms remove that tax:

- Interruptible: any real user input (pointer/key/wheel, isTrusted only)
  during the intro skips the remaining choreography and launches at
  once — it wakes the in-flight beat sleep, so the skip is immediate.
  Input never cancels the launch itself and is never swallowed;
  whatever it was doing (typing a search, clicking a tile) still
  happens.

- Exposure decay: after 3 delivered — or deliberately skipped — intros
  the beats collapse and the sequence plays in one breath, exactly like
  a warm tile click. Counted per account in kv (kv.incr, atomic across
  devices; capped at the threshold so the key stops changing) rather
  than per device: the lesson lives in the user's head and the account
  follows them, while localStorage would also bleed between accounts on
  a shared browser. Only real exposures count: a delivered flourish or
  an active skip with the grid on screen — hidden tabs, timeouts, and
  no-tile landings teach nothing and don't count. The animated page
  flip is exempt from decay: it isn't a repeated lesson, it's live
  wayfinding to where this app lives, and its settle is needed anyway
  to put the tile in view for the morph. Every failure mode (slow or
  failed kv read) errs toward teaching once more, never toward never
  teaching.

Also bounds the intro's wait on the app-list load by the same deadline
as the tile wait — fetch has no timeout of its own, so a stalled
installedApps request used to hold the deep-link launch hostage
indefinitely.
2026-08-07 11:23:47 -07:00
jelveh 14c14fb5c5 feat(gui): animate the pager flip when a deep-linked app lives on a later page
The deep-link intro used to flip to the tile's page instantly behind the
grid's load-fade, so landings on off-page apps woke up on page N with
nothing but the pager dots hinting a move happened. Now the grid always
reveals on its first page, holds the grid beat, and then visibly travels
to the tile's page before the tile's flourish — the journey shows where
the app lives, which is also where minimize will put it back.

Smooth scrollTo has no reliable completion event across engines, so the
travel uses a settle allowance (DEEP_LINK_INTRO_FLIP_SETTLE_MS, same
pattern as the drag code's DRAG_FLIP_SETTLE_MS): the scroll's ~450ms
plus a rest so the landing reads before the tile pops. First-page apps
skip the flip entirely and are unaffected; the hidden-page bail-out is
re-checked after the flip settles.
2026-08-07 11:12:03 -07:00
jelveh 41fc8d890b feat(gui): play the tile click→morph→open intro on /app/<name> landings
A direct landing on /app/<name> now plays the same sequence a real
Apps-tab tile click does — grid appears, a beat, the tile's icon ghost
pops out of its slot, a beat, and the window morphs out of the tile —
so the landing shows the user what is being opened and where minimize
puts it back.

TabApps.beginDeepLinkLaunch waits (3s cap) for the tile to be genuinely
visible (list loaded, render done, pager flipped to the tile's page,
load-fade revealed, icon painted), paces the beats, and claims the app
in _launchingApps so a click mid-intro can't spawn a duplicate. The
launch's app-info fetch is prefetched in parallel so the intro never
delays the app's own round-trip. No tile, animations off, reduced
motion, or a background tab (hidden pages throttle timers and defer
rendering) all skip straight to the immediate plain-fade launch.

Also fixes the tab title sticking as the app's name after closing a
deep-linked app: the landing's replaceState('/') committed the
dashboard's history entry while the page still carried the server's
app-name title, and Chrome shows an entry's stored title when close/
Back traverses onto it — document.title is now reset before the
replaceState so the entry is stamped with the dashboard's own title.
2026-08-07 10:34:21 -07:00
Reynaldi Chernando 45bdc46df0 Remove deprecated models 7 aug (#3522) 2026-08-07 23:20:50 +07:00
jelveh 82ca390bf4 fix(gui): create-in-folder via sidebar right-click renders in target dir
Right-clicking a sidebar/breadcrumb folder in the dashboard Files tab and
choosing New > Folder (or any file type) created the item in the right-
clicked folder correctly, but the UI inserted the new row and started the
inline rename in whatever directory was currently open — so the item
appeared to be created in the wrong place until a refresh.

Now, when the creation target isn't the directory on screen, navigate to
the target and run the select + rename flow there; same-directory
creation keeps the incremental insert.
2026-08-07 08:49:13 -07:00
Neal Shah edfd67a83c fix PUT-1444 (#3515) 2026-08-06 17:11:04 -04:00
404oops ff16ec8900 Add Neuralwatt support (#3509) 2026-08-06 17:01:33 -04:00
Daniel Salazar c7edfc0c74 perf: improve metering perf + add extra compat for monthly charges (#3513)
closes PUT-1445 and PUT-1446
2026-08-06 10:26:47 -07:00
velzie 09f30d693c Peer: allow anontoken in place of authentication (#3501) 2026-08-06 02:03:53 -04:00
641 changed files with 76063 additions and 17021 deletions

No files matched your search

+32
View File
@@ -13,6 +13,9 @@ on:
- 'tools/**'
- 'package.json'
- 'package-lock.json'
# The type-check job below reads these.
- 'tsconfig.json'
- 'tsconfig.build.json'
- '.github/workflows/backend-tests.yaml'
permissions:
@@ -20,6 +23,35 @@ permissions:
pull-requests: write
jobs:
# `tsconfig.build.json` builds with `noCheck: true`, so nothing else in CI
# runs the type checker and a missing export compiles to `undefined`,
# surfacing only when the call is finally reached at runtime. This diffs
# against tools/typecheck-baseline.json and fails only on *new* errors.
#
# The consuming repo runs an equivalent gate, but only once someone bumps the
# submodule pointer — too late to keep the error off this repo's default
# branch. Hence a gate here, on this repo's own pull requests.
#
# Its own job rather than a step in `test`: that one runs a base/PR matrix for
# coverage comparison, and the check only needs the PR ref, once.
typecheck:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Type check (new errors only)
run: npm run typecheck
test:
runs-on: ubuntu-latest
strategy:
+27
View File
@@ -23,6 +23,33 @@ permissions:
pull-requests: write
jobs:
# The JSDoc in src/puter-js/src is the source of truth for the SDK's public
# types. The declarations shipped to npm are generated from it at build time
# and never committed, so what needs guarding is the JSDoc itself: this
# generates the declarations and type-checks the published surface without
# skipLibCheck, which is how broken re-exports used to go unnoticed.
#
# Its own job rather than a step in `test`: that one builds bundles and
# installs a browser, and this needs neither.
types:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Check the puter.js JSDoc produces declarations that type-check
run: npm run check:puterjs:types
test:
runs-on: ubuntu-latest
+9 -9
View File
@@ -52,6 +52,7 @@ When in doubt, return less. Auth-, permission-, or data-export-related changes d
- **Test new behavior.** Every new function, endpoint, driver method, or logic branch gets a test; every bug fix gets a regression test that fails before the fix. If something is genuinely hard to test, skip it but say so in the PR.
- **Boy Scout Rule, proportional to the change.** Fix the obvious typo or dead import in files you're already touching; don't ride a refactor along with a bug fix.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't defend in review is not.
- **A limit change is not done until the docs change with it.** Every rate limit, concurrency cap, quota, and allowance in the code is published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) — an undisclosed limit is one developers discover as a service failure. If a PR moves any of these numbers, the same PR updates that page. The numbers live in `src/backend/controllers/fs/limits.ts` (filesystem), `src/backend/drivers/util/aiLimits.ts` (all AI drivers), per-driver `rateLimit`/`concurrent` configs, and `src/backend/data/subPolicies/` (free-tier allowances).
---
@@ -98,24 +99,23 @@ Follow the same layered structure inside an extension — unless it only needs a
[src/puter-js/](src/puter-js/) is the public SDK. It ships live from `https://js.puter.com/v2/` with no version pinning — every existing app picks up changes immediately. Treat every observable behavior (signatures, response fields, error codes) as something a production app depends on.
Layout: SDK modules in [src/puter-js/src/modules/](src/puter-js/src/modules/) (one file or directory per area — `FileSystem/`, `KV.js`, `ai/`, …), shared plumbing in [src/puter-js/src/lib/](src/puter-js/src/lib/), hand-maintained type declarations in [src/puter-js/types/](src/puter-js/types/), API tests in [src/puter-js/tests/api/](src/puter-js/tests/api/), UI e2e tests in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/), developer docs in [src/docs/](src/docs/).
Layout: SDK modules in [src/puter-js/src/modules/](src/puter-js/src/modules/) (one file or directory per area — `FileSystem/`, `kv/`, `ai/`, …), shared plumbing in [src/puter-js/src/lib/](src/puter-js/src/lib/), generated (gitignored) type declarations in `src/puter-js/types/`, API tests in [src/puter-js/tests/api/](src/puter-js/tests/api/), UI e2e tests in [src/puter-js/tests/e2e/](src/puter-js/tests/e2e/), developer docs in [src/docs/](src/docs/).
Language & typing: puter.js source is plain JavaScript — never TypeScript files — typed via JSDoc. Reference the hand-maintained declarations in [src/puter-js/types/](src/puter-js/types/) with `import(...)` types rather than re-declaring shapes:
Language & typing: puter.js source is plain JavaScript — never TypeScript files — typed via JSDoc. **The JSDoc is the source of truth for the SDK's types.** `src/puter-js/types/` is `tsc --emitDeclarationOnly` output: the SDK build (`npm run build` in `src/puter-js`) generates it, the npm tarball ships it so TypeScript consumers get declarations, and git ignores it — it is never committed and never edited by hand. `npm run check:puterjs:types` generates it in CI and type-checks the result without `skipLibCheck`. The one hand-written declaration file is [src/puter-js/index.d.ts](src/puter-js/index.d.ts), which decides what is public and re-exports the generated names.
Declare a shape where it belongs, and reference it with an `import(...)` type from elsewhere. A shape more than one file needs lives in the module's `types.js`; a shape shared across modules lives in [src/puter-js/src/lib/types.js](src/puter-js/src/lib/types.js); a shape with one consumer can stay next to it:
```js
/** @typedef {import('../../types/modules/ai').ChatOptions} ChatOptions */
/** @type {ChatOptions} */
const options = { model: 'gpt-5-nano' };
/** @typedef {import('./types.js').KVOptConfig} KVOptConfig */
```
When a shape only exists locally, declare it as an inline object literal — not the `@typedef {Object}` + `@property` list form — and use `unknown` over `*`:
Use `@typedef {Object}` + `@property` for any shape whose fields need documenting — it is the only JSDoc form that carries a doc comment per field into the generated declaration. Keep the inline object-literal form for small internal shapes with nothing to say about each field, and prefer `unknown` over `*`:
```js
/** @typedef {{ key: string, value: unknown }} KVEntry */
```
Public (exposed) methods must carry JSDoc types — parameters and return value, matching the `.d.ts` declarations exactly. Unexposed/private helpers are typed at the contributor's discretion: annotate where it helps the next reader, and either way keep them clean.
Public (exposed) methods must carry JSDoc types — parameters and return value — with one `@overload` block per accepted call form, since those overloads *are* the published signature. Unexposed/private helpers are typed at the contributor's discretion: annotate where it helps the next reader, and either way keep them clean. Members tagged `@internal` are stripped from the generated declarations, so use that tag rather than `@private` to keep something off the public surface.
Typing in JS files is encouraged: annotate with JSDoc `@type`/`@param`/`@returns` using the TypeScript type system, and define shared shapes with `@typedef`. API types must not be `unknown` or untyped `...args` — spell out the real parameter and return shapes; the only exception is values passed through transparently to an upstream layer that owns their type. For example:
@@ -133,7 +133,7 @@ Every SDK change carries all five of the following — a puter.js PR missing one
1. **Backward compatibility.** Mandatory unless a maintainer explicitly signs off on a break. Existing call signatures keep working (including both positional and options-object forms where a method supports them); new parameters are optional with defaults that preserve old behavior; never rename or repurpose existing params, response fields, or error codes. New parameter names are `camelCase` (existing `snake_case` stays for compatibility). Say in the PR how existing callers are unaffected.
2. **Tests.** Add or extend a suite in [tests/api/suites/](src/puter-js/tests/api/suites/) (`<area>.suite.ts`; register new suites in `suites/index.ts` — no globbing). One suite runs unchanged on node, browser, and workerd via `npm run test:puterjs`; never write a per-platform test. The runners execute the **built** bundle — run `npm run build:workerLib` after SDK changes or the suite silently tests stale code. For `puter.ui.*` methods rendered by the desktop, use the Playwright e2e harness instead — see [src/puter-js/TESTING.md](src/puter-js/TESTING.md).
3. **Docs.** New or changed APIs update [src/docs/src/](src/docs/src/): the method page (`<Area>/<method>.md`, with frontmatter and a runnable example) and the area overview when the surface changes. Docs are the contract users code against — signatures, defaults, and return shapes must match the implementation exactly.
4. **Types.** Update [src/puter-js/types/modules/](src/puter-js/types/modules/)`<module>.d.ts` and re-export new types through `index.d.ts` / `types/puter.d.ts`. Declarations must match runtime behavior exactly — a wrong type is worse than a missing one.
4. **Types.** Type the change in JSDoc on the implementation, then run `npm run check:puterjs:types` to confirm it still produces declarations that type-check. Name any new type in [src/puter-js/index.d.ts](src/puter-js/index.d.ts) if consumers should be able to import it. Declarations must match runtime behavior exactly — a wrong type is worse than a missing one.
5. **Error handling.** Reject/throw `{ message, code }` objects with stable `snake_case` codes, matching the existing modules (see `KV.js`). Validate cheap preconditions client-side before making the network call; pass backend errors through unchanged rather than swallowing or re-wrapping them. Error codes are API surface — changing one is a breaking change.
[doc/contributing-apis.md](doc/contributing-apis.md) walks the full lifecycle of adding an API across backend + SDK.
+39 -35
View File
@@ -6,33 +6,33 @@ We at **Puter** are committed to maintaining a secure experience for our users a
The following are in scope for this program:
* **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
* **`puter.com`**
* **`api.puter.com`**
- **The Puter open-source project** (available at [github.com/HeyPuter/puter](https://github.com/HeyPuter/puter))
- **`puter.com`**
- **`api.puter.com`**
Out-of-scope:
* Third-party services, applications, or libraries not maintained by Puter.
* Social engineering attacks (e.g., phishing against staff).
* Denial of Service (DoS), spam, or volumetric attacks.
* Physical security issues.
- Third-party services, applications, or libraries not maintained by Puter.
- Social engineering attacks (e.g., phishing against staff).
- Denial of Service (DoS), spam, or volumetric attacks.
- Physical security issues.
## Known Non-Issues (Please Check Before Submitting)
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
* **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we *do* evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
* **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
* **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
* **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
* **App metadata or app user-count "leaks".** This information is currently public by design.
* **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
* **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
* **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
* **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
* **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
* **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
* **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
- **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we _do_ evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
- **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
- **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
- **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
- **App metadata or app user-count "leaks".** This information is currently public by design.
- **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
- **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
- **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
- **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
- **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
- **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
- **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
@@ -40,7 +40,7 @@ If you believe you have a **genuinely new** exploit chain that defeats one of th
To participate, you must:
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see *Proof of Reproduction* below).
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see _Proof of Reproduction_ below).
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
@@ -51,26 +51,30 @@ Reports that do not meet these guidelines may not be eligible for a reward.
## Proof of Reproduction
Reports must demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
Reports **MUST** demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
* Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
* The **observed** result versus the **expected** result.
* For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
* For server-side bugs: a runnable proof-of-concept.
- Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
- The **observed** result versus the **expected** result.
- For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
- For server-side bugs: a runnable proof-of-concept and screenshot or video showing the attack process.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and are generally not eligible**. If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and not eligible for bounty**.
If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Please submit **one issue per report**. Bundled "audit packs" of many speculative findings will be declined; send each confirmed issue separately.
Please submit **one issue per report**.
Bundled "audit packs" of many speculative findings will not be elligible for bounty; send each confirmed issue separately.
Additionally please **DO NOT** submit too many issues at once; these will be treated as a bundled report, and will not be elligible for bounty.
## Reporting Process
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
Include:
* A description of the vulnerability
* Steps to reproduce
* Potential impact
* Suggested remediation (if available)
- A description of the vulnerability
- Steps to reproduce
- Potential impact
- Suggested remediation (if available)
- Paypal id where we can send the bounty (if eligible)
We aim to acknowledge receipt within **72 hours** and provide a resolution timeline.
@@ -78,10 +82,10 @@ We aim to acknowledge receipt within **72 hours** and provide a resolution timel
We offer monetary rewards based on the severity of the vulnerability, as determined by our internal assessment (using CVSS as a guide).
* **Critical: \$1,000 – \$2,000**
* **High: \$500 – \$1,000**
* **Medium: \$200 – \$500**
* **Low: \$50 – \$100**
- **Critical: \$1,000 – \$2,000**
- **High: \$500 – \$1,000**
- **Medium: \$200 – \$500**
- **Low: \$50 – \$100**
Non-security issues, suggestions, and best practices feedback are always welcome, but may not qualify for a reward.
If multiple researchers report the same issue, the bounty will be awarded to the first eligible report we receive.
+2
View File
@@ -32,6 +32,8 @@ Don't commit code you couldn't have written, debugged, or defended yourself. Rea
If you add or change a public API (an endpoint, driver method, or puter-js method), follow [doc/contributing-apis.md](doc/contributing-apis.md) — backward compatibility, [developer docs](src/docs/), types, and tests all move in the same PR.
The same rule applies to limits: rate limits, concurrency caps, quotas, and allowances are published in [src/docs/src/rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md), and a PR that changes one of those numbers updates that page in the same PR. A limit nobody published is a limit developers only discover as a service failure.
## 6. Boy Scout Rule — leave it 1% better
![Boy Scout Rule](https://imgs.search.brave.com/DMmIWl5-NuZVtrR9kXBb06AKF8kturkgSW9UMb2-6m4/rs:fit:860:0:0:0/g:ce/aHR0cHM6Ly9sYXdz/b2Zzb2Z0d2FyZWVu/Z2luZWVyaW5nLmNv/bS9pbWFnZXMvbGF3/cy9ib3ktc2NvdXQt/cnVsZS5wbmc)
-7
View File
@@ -12,10 +12,6 @@
·
<a href="https://developer.puter.com" target="_blank">Developers</a>
·
<a href="https://discord.com/invite/PQcx7Teh8u">Discord</a>
·
<a href="https://reddit.com/r/puter">Reddit</a>
·
<a href="https://twitter.com/HeyPuter">X</a>
</p>
@@ -82,10 +78,7 @@ Puter is available as a hosted service at [**puter.com**](https://puter.com).
Connect with the maintainers and community through these channels:
- Bug report or feature request? Please [open an issue](https://github.com/HeyPuter/puter/issues/new/choose).
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
- Mastodon: [mastodon.social/@puter](https://mastodon.social/@puter)
- Security issues or abuse reports? [[email protected]](mailto:[email protected])
- Email maintainers at [[email protected]](mailto:[email protected])
+59 -2
View File
@@ -251,6 +251,28 @@
}
},
// ── Sharing ─────────────────────────────────────────────────────────
// Email a recipient who already has an account about a new share. On
// unless set to false; they opt out via the unsubscribe link, or by
// blocking a sender. An invite to an address with no account always goes.
// "share_email_notifications": false,
//
// New shares one account may create per UTC day. Default 200.
// "share_daily_limit": 200,
//
// How often a share may interrupt its recipient — the notification pushed
// to their screen and the email with it. Over budget the share still
// succeeds and their notification is still brought up to date; only the
// interruption is dropped. The recipient bounds are what stop many senders
// burying one person between them. A non-positive value removes a bound.
// "share_notify_limits": {
// "pairWindowSeconds": 900,
// "pairDaily": 20,
// "recipientHourly": 10,
// "recipientDaily": 50,
// "emailBatchSeconds": 90
// },
// ── Alarms / alerting ───────────────────────────────────────────────
// Where system alarms go. Severity is the routing decision — each
// transport takes everything at or above its own `minSeverity`:
@@ -324,6 +346,10 @@
"apiURL": ""
},
"gemini": { "apiKey": "" },
// Meta Model API (Muse Spark). `muse-spark-1.2-contributor`
// trades a ~12x discount for Meta training on its prompts and
// completions, so it is only served to callers that name it.
"meta": { "apiKey": "" },
"groq": { "apiKey": "" },
"deepseek": { "apiKey": "" },
"mistral": { "apiKey": "" },
@@ -341,6 +367,16 @@
"apiKey": "",
"apiBaseUrl": "https://llm.onerouter.pro/v1"
},
// BytePlus ModelArk. One key powers chat (Seed/GLM/DeepSeek), image
// generation (Seedream) and video generation (Seedance); `apiBaseUrl`
// selects the region; see
// https://docs.byteplus.com/en/docs/ModelArk/1330310 for options.
"byteplus": {
"apiKey": "",
"apiBaseUrl": "https://ark.ap-southeast.bytepluses.com/api/v3"
},
"zai": { "apiKey": "" },
"alibaba": { "apiKey": "" },
"together-ai": { "apiKey": "" },
// Local Ollama. `enabled: false` skips the auto-probe at startup
// (otherwise Puter logs ECONNREFUSED on every boot when no Ollama
@@ -437,6 +473,27 @@
],
// ── Metering ────────────────────────────────────────────────────────
// When true, all metering checks pass — no per-actor limits enforced.
"unlimitedMetering": false
// When true, every account resolves to an unlimited policy: usage is still
// recorded, but nothing is ever refused for lack of budget. This is the
// setting for a deployment with no way to buy more — without it, accounts
// are held to the free monthly allowance and start getting 402s from the AI
// surfaces, file transfers and KV once they pass it.
"unlimitedMetering": false,
// Whether an account that has spent its whole allowance is refused the
// operations that spend it — file transfers, KV calls. Recording is
// unaffected either way. `workers` extends the same refusal to
// worker-driven calls, which are exempt by default because a deployed
// worker has nowhere to surface a payment prompt. `subscriptions` covers
// the separate question of surfaces reserved for paid plans (the
// vendor-compatible AI endpoints): turn it off on a deployment with no
// paid plans, or every account is refused them.
// "meteringEnforcement": { "enabled": true, "workers": false, "subscriptions": true },
// Fleet-wide spend rate, in micro-cents per minute, past which metering
// raises the `metering:excessiveGlobalUsageRate` alarm. Omit it (the
// default) to leave the check off: the only useful value is a multiple of
// what this deployment's normal traffic costs, so it has to be measured
// rather than guessed, and a stale number here alarms on healthy growth.
// "maxGlobalUsagePerMinute": 200000000
}
+19 -5
View File
@@ -6,9 +6,9 @@ transports by **severity**.
```ts
this.clients.alarm.create(
`driver_rate_limit_hit:${iface}:${method}`, // de-dupe key
`Driver rate limit hit on ${iface}:${method}`, // what a human reads
{ iface, method, userUuid }, // context fields
`metering_write_failed:${userUuid}`, // de-dupe key
'Metering write failed', // what a human reads
{ userUuid, appId, error }, // context fields
'info', // severity
);
```
@@ -37,8 +37,22 @@ pass one explicitly unless you really mean "page someone".
- Did the server fail to do its job in a way nobody expected? → `critical`
- Is a background job, rate, or dependency degraded? → `warning`
- Is this a user doing something notable (hitting a limit, tripping an abuse
heuristic, overspending)? → `info`
- Is this a user doing something notable (tripping an abuse heuristic,
overspending)? → `info`
- Did one of *our own* limits reject a caller — a rate limit, a concurrency
cap, a quota? → don't alarm at all. The limit doing its job is not an
event; the 429 is the whole signal, and alarming on it only produces noise
proportional to traffic. An *upstream provider* rate-limiting us is the
opposite case and still alarms (`upstream_rate_limited`, `info`) — that one
is not something we chose. The exception is a free model: nothing is billed,
nothing is actionable, and the throttling is the price of the model, so the
AI chat driver marks those `noAlarm` and only paid models still record.
`noAlarm` is the code-level mute: an `HttpError` carrying it skips the
terminal gate entirely, no matter its status or code. It is for failures the
call site *already knows* are expected and traffic-proportional — reach for it
there, not as a way to quiet an alarm you haven't diagnosed
(`severityOverrides` below is the knob for that).
An extension whose signals are all one tier can default its own local
`raiseAlarm` helper to that tier instead of repeating it at every call site —
+8 -2
View File
@@ -53,7 +53,10 @@ Both are supported ways to define an API. **Prefer a controller when you need fi
#### Middleware and gates
- **Controller routes** (and extension routes — same options) take [`RouteOptions`](../src/backend/core/http/types.ts): auth gates (`requireAuth`, `requireUserActor`, `noUserSession`, `adminOnly`, `allowedAppIds`, and the access-token controls), `subdomain` routing, per-route `rateLimit`, body parsers, and arbitrary extra `middleware`. The auth flavors are subtle and default-deny — read the JSDoc on each field before picking.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), and `noUserSession`. The `/drivers/call` surface enforces them.
- **Driver methods** get their policies from the `@Driver` options: per-method `rateLimit` (limit/window/backend), `concurrent` in-flight caps (optionally `bySubscription`), `requireSubscription`, and `noUserSession`. The `/drivers/call` surface enforces them.
- **A surface only paying accounts should reach** declares `requireSubscription` — the route option on a controller endpoint, the per-method `@Driver({ requireSubscription })` block on a driver (`/drivers/call` is one shared route, so a driver's requirement can't live in route options). `true` accepts any plan that isn't free, so a plan registered by an extension counts without core naming it; an array of policy ids (`['business', 'pro']`) accepts only those; `false` is "no requirement", the same as leaving it out. An empty array is a boot error rather than a silent no-op — it reads as subscribers-only while admitting everyone. Off unless asked for, and answered from the metering service's cached per-actor subscription, so it costs a map lookup. Distinct from `requireCredits`: this asks which plan the account is on, not whether it has budget left. Deployments with no paid plans (self-hosted installs) turn the whole thing off with `meteringEnforcement.subscriptions: false`.
- **An account must have verified a specific factor** — `requireVerified` (email, and only under `strict_email_verification_required`), `requirePhoneVerified`, `requireCardVerified`. These are opt-in and require the factor to have actually been verified, unlike the default-on gate that turns away accounts still carrying a pending verification the abuse harness asked for.
- **An endpoint that spends metered resources** on the caller's behalf — moving file content, making object-store requests, anything else the account is billed for — also declares `requireCredits: true`, which turns an account with nothing left of its budget away with a 402 before the handler runs. Endpoints that only describe or delete things deliberately don't: an account that has run out still has to be able to see what it has, clear it, and reach its billing pages. Drivers have no route options to declare this on, so they call `assertActorHasCredits` themselves ([src/backend/services/metering/enforcement.ts](../src/backend/services/metering/enforcement.ts)) — see `KVStoreDriver`, which does it once for every method.
### 3. puter.js
@@ -62,7 +65,10 @@ Both are supported ways to define an API. **Prefer a controller when you need fi
### 4. Types
- Add or extend the declaration in [src/puter-js/types/modules/](../src/puter-js/types/modules/)`<module>.d.ts` and re-export new types through `index.d.ts`. Declarations must match the runtime exactly — optionality, defaults, and return types included.
- Type the method where you wrote it, in JSDoc: `@param`/`@returns` on the implementation, one `@overload` block per accepted call form, and `@typedef {Object}` + `@property` for any new shape. The JSDoc is the source of truth — declarations are generated from it, so there is nothing to keep in sync by hand.
- Put a shape more than one file needs in the module's `types.js` (e.g. [src/puter-js/src/modules/kv/types.js](../src/puter-js/src/modules/kv/types.js)); anything shared across modules goes in [src/puter-js/src/lib/types.js](../src/puter-js/src/lib/types.js). A shape with one consumer can stay next to it.
- Run `npm run check:puterjs:types` — it generates the declarations and type-checks the published surface without `skipLibCheck`. **Never edit anything under `src/puter-js/types/`**: it is gitignored build output, produced by the SDK build and shipped in the npm tarball, and the next build overwrites it.
- Name the new type in [src/puter-js/index.d.ts](../src/puter-js/index.d.ts) if consumers should be able to import it. That file is the one hand-written declaration in the package: it decides what is public and re-exports nothing else.
### 5. Docs
+48
View File
@@ -122,6 +122,10 @@ cat > puter/config/config.json <<EOF
"ollama": { "enabled": false }
},
"meteringEnforcement": {
"subscriptions": false
},
"trust_proxy": 1
}
EOF
@@ -136,6 +140,7 @@ Why these knobs:
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
- `meteringEnforcement.subscriptions: false` — a few surfaces (the OpenAI/Anthropic-compatible AI endpoints) are declared paid-plan-only. A self-hosted install has no paid plans, so every account sits on the free policy and those routes would answer `402 subscription_required` for everyone. Turning the plan gates off opens them; nothing else about metering changes.
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
- `s3.s3Config.forcePathStyle: true` — RustFS / MinIO / fauxqs need path-style URLs (`<endpoint>/<bucket>`). Real AWS S3 wants virtual-hosted (`<bucket>.<endpoint>`) — drop this flag (or set `false`) when you swap to real S3.
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. Caddy routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
@@ -295,6 +300,18 @@ Used for password resets, email confirmation, and notifications. Without it thos
To require email confirmation before login, also set `"strict_email_verification_required": true`.
To read the mail yourself instead of sending it, point the transport at a local SMTP sink — [MailHog](https://github.com/mailhog/MailHog) catches everything and serves it at `http://localhost:8025`. Publish both ports; a container that only exposes them is unreachable from the host:
```sh
docker run -d -p 1025:1025 -p 8025:8025 mailhog/mailhog
```
```json
"email": { "from": "\"Puter\" <[email protected]>", "host": "127.0.0.1", "port": 1025, "secure": false, "ignoreTLS": true }
```
Share email is on once a transport exists: a recipient who already has an account is emailed as well as notified in-app, and they opt out with the unsubscribe link the mail carries. Set `"share_email_notifications": false` to keep those in the app only — an address with no account is still emailed, since there is nothing else to reach them with.
### Sign in with Google (or another OIDC provider)
```json
@@ -370,6 +387,37 @@ Default is 100 MB per user.
Set `is_storage_limited: false` for unlimited (bounded by host disk).
### Usage metering and budgets
Puter meters what an account costs to serve — bytes sent back, object-store
requests, KV capacity, AI tokens — against a monthly budget, and refuses the
operations that spend it once that budget is gone. The refusal is a `402` with
code `insufficient_funds`; reads that only describe things, and every kind of
deletion, stay available so an account can always see what it has and clear it.
On a self-hosted install this is almost certainly not what you want. There is
nowhere to buy more, so accounts are held to the free monthly allowance
(US$0.25 of measured cost — roughly 2 GiB of downloads) and start being refused
after that. Turn it off:
```json
"unlimitedMetering": true
```
Every account then resolves to an unlimited policy. Usage is still recorded, so
the dashboard still shows what is being consumed; nothing is ever refused for
lack of budget.
To keep the budgets but stop them blocking anything — recording only:
```json
"meteringEnforcement": { "enabled": false }
```
Calls driven by a deployed worker are exempt from enforcement by default,
because a worker has no prompt to show and nobody watching it fail. Set
`"meteringEnforcement": { "workers": true }` to include them.
### Captcha on signup / login
Built-in proof-of-work captcha — no external service needed.
+37 -4
View File
@@ -2,6 +2,14 @@ import { Context } from '@heyputer/backend/src/core';
import type { Actor } from '@heyputer/backend/src/core/actor';
import { HttpError } from '@heyputer/backend/src/core/http';
import { PuterDriver } from '@heyputer/backend/src/drivers/types';
import type {
DriverConcurrentConfig,
DriverRateLimitConfig,
} from '@heyputer/backend/src/drivers/meta';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '@heyputer/backend/src/services/metering/consts';
import { extension } from '@heyputer/backend/src/extensions';
// App-telemetry lets an app owner enumerate the users who have
@@ -50,17 +58,42 @@ const parseIntParam = (
* Driver exposing the `app-telemetry` interface.
*
* The `/drivers/call` permission gate checks
* `service:app-telemetry:ii:app-telemetry`, which every actor already holds
* via the blanket `service` grant (hardcoded-permissions.js +
* `service:app-telemetry:ii:app-telemetry`, which every actor already holds via
* the blanket `service` grant (hardcoded-permissions.js +
* `default_implicit_user_app_permissions`). The real authorization — "is the
* caller the app owner?" — is enforced inside `get_users` below, exactly as
* v1 did.
* caller the app owner?" — is enforced inside `get_users` below, exactly as v1
* did.
*/
export class AppTelemetryDriver extends PuterDriver {
readonly driverInterface = 'app-telemetry';
readonly driverName = 'app-telemetry';
readonly isDefault = true;
// Declaring nothing here would leave both methods on the generic
// 600/minute driver default, which does not fit a paginated scan that
// can ask for MAX_LIMIT rows at MAX_OFFSET. This is a dashboard read —
// nobody calls it in a loop.
readonly rateLimit: DriverRateLimitConfig = {
default: {
limit: 60,
window: 60_000,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 30,
[DEFAULT_TEMP_SUBSCRIPTION]: 10,
},
},
};
readonly concurrent: DriverConcurrentConfig = {
default: {
limit: 5,
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 2,
[DEFAULT_TEMP_SUBSCRIPTION]: 2,
},
},
};
/** Users who have authenticated into the given app (owner-only). */
async get_users({
app_uuid,
+11 -1
View File
@@ -81,6 +81,16 @@ export const handleInstalledApps = async (
extension.get(
'/installedApps',
{ subdomain: 'api', requireUserActor: true, allowFullAccessToken: true },
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: {
scope: 'installed-apps',
limit: 120,
window: 60_000,
key: 'user',
},
},
handleInstalledApps,
);
+136 -8
View File
@@ -3,8 +3,13 @@ import { HttpError } from '@heyputer/backend/src/core/http';
import {
controllersContainers,
driversContainers,
servicesContainers,
} from '@heyputer/backend/src/exports';
import { extension } from '@heyputer/backend/src/extensions';
import {
creditMultiplierFrom,
toCredits,
} from '@heyputer/backend/src/services/metering/utils';
import type { Request, Response } from 'express';
const services = extension.import('service');
@@ -18,7 +23,7 @@ function collectAllCosts(): Record<string, unknown>[] {
const all: Record<string, unknown>[] = [];
const collect = (
source: Record<string, unknown>,
kind: 'driver' | 'controller',
kind: 'driver' | 'controller' | 'service',
) => {
for (const [name, instance] of Object.entries(source)) {
const fn = (
@@ -43,9 +48,48 @@ function collectAllCosts(): Record<string, unknown>[] {
};
collect(driversContainers as Record<string, unknown>, 'driver');
collect(controllersContainers as Record<string, unknown>, 'controller');
// Services report the costs that aren't tied to one endpoint — egress,
// which is metered for every response there is.
collect(servicesContainers as Record<string, unknown>, 'service');
return all;
}
// -- Credits scaling ---------------------------------------------------
//
// When the deployment configures a credit multiplier, every monetary amount
// is scaled to display credits BEFORE it leaves the server — raw metered
// amounts are never reported. Counts, units, and byte figures pass through
// untouched. A response that was scaled says so (`unit: 'credits'`); without
// a configured multiplier amounts pass through raw and clients render them
// as dollars.
/** The per-usage-type records of a UsageByType map, costs scaled. */
const scaleUsageByType = (
usage: Record<string, unknown>,
multiplier: number,
): Record<string, unknown> => {
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(usage)) {
// `allowanceUsed` is the month's allowance-charged spend — monetary,
// scalar, scaled like the total.
if (key === 'total' || key === 'allowanceUsed') {
out[key] = toCredits(Number(value) || 0, multiplier);
} else if (
value &&
typeof value === 'object' &&
typeof (value as { cost?: unknown }).cost === 'number'
) {
out[key] = {
...(value as Record<string, unknown>),
cost: toCredits((value as { cost: number }).cost, multiplier),
};
} else {
out[key] = value;
}
}
return out;
};
export const handleMeteringUsage = async (
_req: Request,
res: Response,
@@ -57,7 +101,44 @@ export const handleMeteringUsage = async (
services.metering.getActorCurrentMonthUsageDetails(actor),
services.metering.getAllowedUsage(actor),
]);
res.json({ ...actorUsage, allowanceInfo });
const multiplier = creditMultiplierFrom(extension.config);
if (!multiplier) {
res.json({ ...actorUsage, allowanceInfo });
return;
}
res.json({
usage: scaleUsageByType(
actorUsage.usage as unknown as Record<string, unknown>,
multiplier,
),
appTotals: Object.fromEntries(
Object.entries(actorUsage.appTotals).map(([appId, totals]) => [
appId,
{ ...totals, total: toCredits(totals.total, multiplier) },
]),
),
allowanceInfo: {
...allowanceInfo,
remaining: toCredits(allowanceInfo.remaining, multiplier),
monthUsageAllowance: toCredits(
allowanceInfo.monthUsageAllowance,
multiplier,
),
// Monetary addon fields scale; absent ones stay absent rather
// than materializing as NaN/null. purchasedStorage is bytes.
addons: Object.fromEntries(
Object.entries(allowanceInfo.addons ?? {}).map(([k, v]) => [
k,
(k === 'purchasedCredits' ||
k === 'consumedPurchaseCredits') &&
typeof v === 'number'
? toCredits(v, multiplier)
: v,
]),
),
unit: 'credits',
},
});
};
export const handleMeteringUsageForApp = async (
@@ -88,7 +169,18 @@ export const handleMeteringUsageForApp = async (
actor,
appId,
);
res.json(appUsage);
const multiplier = creditMultiplierFrom(extension.config);
if (!multiplier) {
res.json(appUsage);
return;
}
res.json({
...scaleUsageByType(
appUsage as unknown as Record<string, unknown>,
multiplier,
),
unit: 'credits',
});
};
export const handleMeteringGlobalUsage = async (
@@ -105,31 +197,67 @@ export const handleMeteringAllCosts = async (
res: Response,
): Promise<void> => {
if (!cachedAllCosts) {
cachedAllCosts = collectAllCosts();
const raw = collectAllCosts();
const multiplier = creditMultiplierFrom(extension.config);
// Price the catalogue in the same unit the usage endpoints report:
// per-unit credits when a multiplier is configured, raw otherwise.
cachedAllCosts = multiplier
? raw.map(({ ucentsPerUnit, costs_currency: _cc, ...rest }) =>
typeof ucentsPerUnit === 'number'
? {
...rest,
creditsPerUnit: toCredits(
ucentsPerUnit,
multiplier,
),
}
: rest,
)
: raw;
}
res.json({ costs: cachedAllCosts });
};
/** Dashboard reads over the per-actor KV aggregates. */
const USAGE_READ_LIMIT = {
scope: 'metering-usage',
limit: 120,
window: 60_000,
key: 'user' as const,
};
extension.get(
'/metering/usage',
{ subdomain: 'api', requireAuth: true },
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringUsage,
);
extension.get(
'/metering/usage/:appIdOrName',
{ subdomain: 'api', requireAuth: true },
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringUsageForApp,
);
extension.get(
'/metering/globalUsage',
{ subdomain: 'api', adminOnly: true },
{
subdomain: 'api',
adminOnly: true,
// Sums across every shard of the global aggregate. Admin-gated, so
// this is loop protection — but one accidental poll is an
// expensive minute.
rateLimit: {
scope: 'metering-global-usage',
limit: 10,
window: 60_000,
key: 'user',
},
},
handleMeteringGlobalUsage,
);
extension.get(
'/metering/allCosts',
{ subdomain: 'api', requireAuth: true },
{ subdomain: 'api', requireAuth: true, rateLimit: USAGE_READ_LIMIT },
handleMeteringAllCosts,
);
+84 -1
View File
@@ -109,6 +109,9 @@ describe('whoami extension — handleWhoami', () => {
expect(body.oidc_only).toBe(false);
// is_user_token is present (true) for user actors.
expect(body.is_user_token).toBe(true);
// Account creation time, in unix seconds.
expect(typeof body.created_ts).toBe('number');
expect(body.created_ts).toBeGreaterThan(0);
// `directories` is only sent to user actors — confirm it’s present.
expect(body.directories).toBeDefined();
// taskbar_items is only sent to user actors.
@@ -132,7 +135,7 @@ describe('whoami extension — handleWhoami', () => {
expect(flags.payment_bypass).toBeUndefined();
});
it('strips desktop_bg_* and human_readable_age fields for app actors', async () => {
it('strips desktop_bg_*, created_ts and human_readable_age fields for app actors', async () => {
const user = await seedUser();
const { res, captured } = makeRes();
@@ -154,6 +157,8 @@ describe('whoami extension — handleWhoami', () => {
expect(body.desktop_bg_color).toBeUndefined();
expect(body.desktop_bg_fit).toBeUndefined();
expect(body.human_readable_age).toBeUndefined();
// Account age, in either form, is not exposed to apps.
expect(body.created_ts).toBeUndefined();
// Directories are user-only.
expect(body.directories).toBeUndefined();
});
@@ -201,6 +206,84 @@ describe('whoami extension — handleWhoami', () => {
expect(JSON.stringify(body)).not.toContain('bootstrap-secret');
});
it('never exposes phone, card fingerprint or signup identity', async () => {
const user = await seedUser();
await server.stores.user.update(user.id as number, {
phone: '+15551234567',
card_fingerprint: 'fp_ABC123',
requires_phone_verification: false,
requires_card_verification: false,
});
// Guard against a vacuous assertion below: the columns really do hold
// the values we then expect never to see on the wire.
const stored = await server.stores.user.getById(user.id as number, {
cached: false,
force: true,
});
expect(stored?.phone).toBe('+15551234567');
expect(stored?.card_fingerprint).toBe('fp_ABC123');
for (const actor of [
{ user: { uuid: user.uuid, id: user.id as number } },
{
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
},
]) {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
expect(body.phone).toBeUndefined();
expect(body.card_fingerprint).toBeUndefined();
expect(body.password).toBeUndefined();
expect(body.otp_secret).toBeUndefined();
expect(body.signup_ip).toBeUndefined();
// Not just absent as a key — the values must not appear anywhere
// in the payload (nested under metadata, taskbar items, …).
const serialized = JSON.stringify(body);
expect(serialized).not.toContain('5551234567');
expect(serialized).not.toContain('fp_ABC123');
// The verification flags, which the GUI acts on, still ship.
expect(body).toHaveProperty('requires_phone_verification');
expect(body).toHaveProperty('requires_card_verification');
}
});
it('scrubs sensitive keys added to metadata, and leaves the cached row intact', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
tmp_password: 'bootstrap-secret',
billing: { card_fingerprint: 'fp_NESTED', tier: 'pro' },
});
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleWhoami(makeReq(), res),
);
const body = captured.body as Record<string, unknown>;
const metadata = body.metadata as Record<string, unknown>;
expect(metadata.tmp_password).toBeUndefined();
// Nested sensitive keys are removed too; siblings survive.
expect(metadata.billing).toEqual({ tier: 'pro' });
expect(JSON.stringify(body)).not.toContain('fp_NESTED');
// The scrub works on a copy — server-side state is untouched.
const fresh = await server.stores.user.getById(user.id as number, {
cached: false,
force: true,
});
expect(fresh?.metadata?.tmp_password).toBe('bootstrap-secret');
expect(
(fresh?.metadata?.billing as Record<string, unknown>)
?.card_fingerprint,
).toBe('fp_NESTED');
});
it('marks the user as oidc_only when password is null', async () => {
const slug = Math.random().toString(36).slice(2, 8);
const oidcUser = await server.stores.user.create({
+113 -14
View File
@@ -14,6 +14,15 @@ const timeago = (() => {
return new TimeAgo('en-US');
})();
// User timestamps come off the DB as SQL datetime strings; the wire format
// for all of them is unix seconds. Unparseable values are dropped rather
// than sent as NaN.
const toUnixSeconds = (value: unknown): number | undefined => {
if (!value) return undefined;
const ms = new Date(value as string | number | Date).getTime();
return Number.isNaN(ms) ? undefined : Math.round(ms / 1000);
};
// Allowlist of `config.feature_flags` keys safe to surface via /whoami.
// Anything not listed here stays server-side, so internal flags
// (payment_bypass, staff_only_*, etc.) cannot leak by accident. Add a
@@ -24,6 +33,68 @@ const CLIENT_VISIBLE_FEATURE_FLAGS: ReadonlySet<string> = new Set([
'prompt_user_when_navigation_away_from_puter',
]);
// Keys that must never leave the server, whoever put them on the response.
// `details` is an explicit pick, but the `whoami.details` event hands
// listeners the full UserRow next to the object they may write to, and
// `metadata` is a free-form blob — so any of these can arrive on the
// response without an edit to the pick above. The scrub runs last, over the
// whole payload, and is the one place that decides what "sensitive" means.
//
// Credentials and single-use tokens, the payment/phone identifiers used for
// verification (`card_fingerprint` is the Stripe fingerprint, stable per
// card number), the network identity recorded at signup, and internal
// anti-abuse bookkeeping. `requires_phone_verification` /
// `requires_card_verification` stay: they are the flags the GUI acts on, and
// they carry no identifier.
const SENSITIVE_KEYS: ReadonlySet<string> = new Set([
'password',
'tmp_password',
'pass_recovery_token',
'email_confirm_code',
'email_confirm_token',
'change_email_confirm_token',
'otp_secret',
'otp_recovery_codes',
'card_fingerprint',
'phone',
'clean_email',
'signup_ip',
'signup_ip_forwarded',
'signup_user_agent',
'signup_origin',
'signup_server',
'audit_metadata',
]);
// Depth-limited, cycle-safe walk deleting every SENSITIVE_KEYS entry it finds
// at any level (`metadata` and `taskbar_items` are both nested structures).
const scrubSensitive = (
value: unknown,
seen: Set<object> = new Set(),
depth = 0,
): void => {
if (depth > 8 || value === null || typeof value !== 'object') return;
if (seen.has(value as object)) return;
seen.add(value as object);
if (Array.isArray(value)) {
for (const entry of value) scrubSensitive(entry, seen, depth + 1);
return;
}
for (const key of Object.keys(value as Record<string, unknown>)) {
if (SENSITIVE_KEYS.has(key)) {
delete (value as Record<string, unknown>)[key];
continue;
}
scrubSensitive(
(value as Record<string, unknown>)[key],
seen,
depth + 1,
);
}
};
export const handleWhoami = async (
req: Request,
res: Response,
@@ -65,8 +136,12 @@ export const handleWhoami = async (
}
}
const metadata = user.metadata ? { ...user.metadata } : user.metadata;
if (metadata) delete metadata.tmp_password;
// Deep-copied (it is decoded JSON) so the scrub below edits the response
// and not the cached UserRow. Sensitive keys inside it — tmp_password and
// anything else on the denylist — are removed by scrubSensitive.
const metadata = user.metadata
? structuredClone(user.metadata)
: user.metadata;
const details: Record<string, unknown> = {
username: user.username,
@@ -75,7 +150,9 @@ export const handleWhoami = async (
unconfirmed_email: user.email,
email_confirmed: user.email_confirmed || user.username === 'admin',
requires_email_confirmation: user.requires_email_confirmation,
phone: user.phone,
// The phone number itself is deliberately absent: nothing on the
// client reads it, and it is PII that would otherwise be handed to
// every app actor. Only the verification flag ships.
requires_phone_verification: user.requires_phone_verification,
requires_card_verification: user.requires_card_verification,
desktop_bg_url: user.desktop_bg_url,
@@ -83,6 +160,10 @@ export const handleWhoami = async (
desktop_bg_fit: user.desktop_bg_fit,
is_temp: user.password === null && user.email === null,
is_user_token: true,
// Present only once the account has actually asked for a code (see the
// referral extension) — null until then, and never minted from here:
// this endpoint is polled, and a mint is a write.
referral_code: user.referral_code,
oidc_only: oidcOnly,
taskbar_items: isUser
? await getTaskbarItems(
@@ -98,6 +179,7 @@ export const handleWhoami = async (
: undefined,
otp: !!user.otp_enabled,
feature_flags,
created_ts: toUnixSeconds(user.timestamp),
human_readable_age: user.timestamp
? timeago.format(new Date(user.timestamp as string))
: null,
@@ -141,14 +223,9 @@ export const handleWhoami = async (
}
// Last activity
if (user.last_activity_ts) {
try {
details.last_activity_ts = Math.round(
new Date(user.last_activity_ts as string).getTime() / 1000,
);
} catch {
/* ignore parse error */
}
const lastActivityTs = toUnixSeconds(user.last_activity_ts);
if (lastActivityTs !== undefined) {
details.last_activity_ts = lastActivityTs;
}
// Strip sensitive fields for app actors
@@ -164,8 +241,11 @@ export const handleWhoami = async (
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.created_ts;
delete details.is_user_token;
delete details.metadata;
// An app has no business reading the code its user earns credit with.
delete details.referral_code;
}
if (actor.app) {
@@ -183,19 +263,38 @@ export const handleWhoami = async (
}
const subscription = details.subscription as
| { offering?: Record<string, unknown> }
| undefined;
{ offering?: Record<string, unknown> } | undefined;
if (subscription?.offering) {
delete subscription.offering.group;
delete subscription.offering.benefits;
delete subscription.offering.price_id;
}
// Last word on what ships, after every listener has had its say.
scrubSensitive(details);
res.json(details);
};
extension.get(
'/whoami',
{ subdomain: 'api', requireAuth: true, allowUnconfirmed: true },
{
subdomain: 'api',
requireAuth: true,
allowUnconfirmed: true,
// The GUI polls this, and each call fans out to every `whoami`
// event listener — so it costs more than the response suggests.
//
// It is also the call everything else leans on to find out who it is
// talking to, so it rides along with unrelated work rather than
// arriving at its own pace: the ceiling has to clear whatever the
// busiest session is doing, not what a person clicks.
rateLimit: {
scope: 'whoami',
limit: 1_800,
window: 60_000,
key: 'user',
},
},
handleWhoami,
);
+9
View File
@@ -131,6 +131,11 @@ S3_SECRET_KEY=$S3_SECRET_KEY
S3_BUCKET=puter-local
EOF
# `meteringEnforcement.subscriptions` is off here because a self-hosted
# install has no paid plans to be on: every account resolves to the free
# policy, so leaving the plan gates armed would 402 the surfaces that
# declare one (the OpenAI/Anthropic-compatible AI endpoints) for everybody.
# The code default is on — this is a property of the install, not of Puter.
log "writing puter/config/config.json"
cat > puter/config/config.json <<EOF
{
@@ -191,6 +196,10 @@ EOF
"ollama": { "enabled": false }
},
"meteringEnforcement": {
"subscriptions": false
},
"trust_proxy": $PUTER_TRUST_PROXY
}
EOF
+2106 -3713
View File
File diff suppressed because it is too large. Load diff
+17 -25
View File
@@ -13,14 +13,9 @@
"@babel/core": "^7.29.7",
"@eslint/js": "^9.35.0",
"@playwright/test": "^1.56.1",
"@stylistic/eslint-plugin": "^5.3.1",
"@types/better-sqlite3": "^7.6.13",
"@types/express": "^5.0.0",
"@types/mime-types": "^3.0.1",
"@typescript-eslint/eslint-plugin": "^8.46.1",
"@typescript-eslint/parser": "^8.46.1",
"@vitest/coverage-v8": "^4.0.14",
"@vitest/ui": "^4.0.14",
"babel-loader": "^10.1.1",
"babel-plugin-istanbul": "^8.0.0",
"chalk": "^4.1.0",
@@ -30,25 +25,19 @@
"eslint": "^9.35.0",
"eslint-config-prettier": "^10.1.8",
"eslint-plugin-prettier": "^5.5.5",
"eslint-rule-composer": "^0.3.0",
"globals": "^15.15.0",
"html-webpack-plugin": "^5.6.0",
"husky": "^9.1.7",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-reports": "^3.2.0",
"license-check-and-add": "^4.0.5",
"nodemon": "^3.1.0",
"prettier": "^3.8.3",
"prettier-plugin-jsdoc": "^1.8.1",
"simple-git": "^3.32.3",
"typescript": "^5.4.5",
"uglify-js": "^3.17.4",
"vite-plugin-static-copy": "^3.3.0",
"vitest": "^4.1.5",
"webpack": "^5.88.2",
"webpack-cli": "^5.1.1",
"yaml": "^2.8.1"
"webpack-cli": "^5.1.1"
},
"scripts": {
"test:backend": "npm run setupExtensions && npm run build:workerLib && vitest run --config src/backend/vitest.config.ts ",
@@ -67,12 +56,13 @@
"check-translations": "node tools/check-translations.js",
"prepare": "husky",
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
"check:puterjs:types": "node tools/checkPuterjsTypes.mjs",
"typecheck": "node tools/typecheck.mjs",
"typecheck:update": "node tools/typecheck.mjs --update",
"setupExtensions": "node ./tools/extensionSetup.mjs"
},
"workspaces": [
"src/*",
"tools/*",
"experiments/js-parse-and-output"
"src/*"
],
"nodemonConfig": {
"ext": "js, json, mjs, jsx, svg, css",
@@ -82,11 +72,8 @@
]
},
"dependencies": {
"@ai-sdk/openai": "^3.0.25",
"@aws-sdk/client-s3": "^3.1020.0",
"@aws-sdk/s3-request-presigner": "^3.1028.0",
"@heyputer/putility": "^1.0.2",
"ai": "^6.0.73",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.6",
@@ -98,12 +85,17 @@
},
"allowScripts": {
"@google/genai": false,
"bcrypt@5.1.1": true,
"better-sqlite3@12.11.1": true,
"browser-tabs-lock@1.3.0": true,
"classic-level@3.0.0": true,
"[email protected]": true,
"[email protected]": true,
"[email protected]": true
"bcrypt": true,
"better-sqlite3": true,
"browser-tabs-lock": true,
"classic-level": true,
"esbuild": true,
"fsevents": true,
"protobufjs": true,
"sharp": true,
"workerd": true
},
"overrides": {
"undici": "^7.29.0"
}
}
@@ -371,6 +371,24 @@ describe('AlarmClient alarm registry', () => {
expect(client.get('flap')?.occurrences).toHaveLength(2);
});
it('caps retained occurrences while still counting every repeat', () => {
const client = makeClient();
const seen = capture(client);
for (let i = 0; i < 50; i++) {
client.create('hot', `occurrence ${i}`, { i });
}
const alarm = client.get('hot')!;
expect(alarm.count).toBe(50);
expect(alarm.occurrences).toHaveLength(20);
expect(alarm.timestamps).toHaveLength(20);
// The window kept is the most recent one, not the oldest.
expect(alarm.occurrences[19].message).toBe('occurrence 49');
// Trimming history must not rewind what the transports are told.
expect(seen[49]).toMatchObject({ repeatCount: 50, isRepeat: true });
});
it('names anonymous handlers by their registration order', () => {
const client = makeClient();
client.addAlertHandler(async () => {
+27 -8
View File
@@ -60,6 +60,15 @@ interface RegisteredHandler {
/** Severity used when neither the call site nor config picks one. */
const FALLBACK_SEVERITY: PagerSeverity = 'critical';
/**
* How many recent occurrences an alarm keeps. An alarm is never cleared unless
* something calls `clear`, and a hot one repeats for as long as the fault lasts
* — so retaining every occurrence means retaining every message and field set
* it was ever raised with, request bodies and actors included, for the life of
* the process. The last few are what a human reads; the rest is only a count,
* and `count` keeps that.
*/
const OCCURRENCE_HISTORY_LIMIT = 20;
/** Keeps `info` alarms out of the paging system unless config says otherwise. */
const DEFAULT_PAGERDUTY_MIN_SEVERITY: PagerSeverity = 'warning';
/** Slack's ceiling once a pager exists: chat gets what doesn't page. */
@@ -321,6 +330,7 @@ export class AlarmClient extends PuterClient {
started: Date.now(),
// `recordOccurrence` below stamps the first occurrence; seeding one
// here too would report every alarm as one occurrence ahead.
count: 0,
timestamps: [],
occurrences: [],
};
@@ -381,16 +391,25 @@ export class AlarmClient extends PuterClient {
message: string,
fields: AlarmFields,
): void {
const now = Date.now();
alarm.message = message;
alarm.fields = { ...alarm.fields, ...fields };
alarm.timestamps.push(Date.now());
alarm.count++;
if (fields.error) alarm.error = fields.error;
alarm.occurrences.push({
message,
fields,
timestamp: Date.now(),
});
alarm.timestamps.push(now);
alarm.occurrences.push({ message, fields, timestamp: now });
if (alarm.timestamps.length > OCCURRENCE_HISTORY_LIMIT) {
alarm.timestamps.splice(
0,
alarm.timestamps.length - OCCURRENCE_HISTORY_LIMIT,
);
alarm.occurrences.splice(
0,
alarm.occurrences.length - OCCURRENCE_HISTORY_LIMIT,
);
}
}
private applyKnownErrors(alarm: Alarm): void {
@@ -438,7 +457,7 @@ export class AlarmClient extends PuterClient {
this.applyKnownErrors(alarm);
console.warn(
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.timestamps.length})`,
`[alarm] REPEAT ${displayId(alarm)} :: ${alarm.message} (${alarm.count})`,
);
if (alarm.noAlert) return;
@@ -476,7 +495,7 @@ export class AlarmClient extends PuterClient {
alarm.severity = resolved;
const fieldsClean = cleanFields(alarm.fields);
const repeatCount = alarm.timestamps.length;
const repeatCount = alarm.count;
const id = alarm.id || 'something-bad';
+10
View File
@@ -48,7 +48,17 @@ export interface Alarm extends AlarmOptions {
fields: AlarmFields;
error?: Error;
started: number;
/**
* Every occurrence ever counted, including those aged out of the two lists
* below.
*/
count: number;
/**
* Timestamps of the most recent occurrences only — see
* `OCCURRENCE_HISTORY_LIMIT`.
*/
timestamps: number[];
/** The most recent occurrences only — see `OCCURRENCE_HISTORY_LIMIT`. */
occurrences: AlarmOccurrence[];
severity?: PagerSeverity;
noAlert?: boolean;
+11 -4
View File
@@ -22,6 +22,7 @@ import {
POOL_ACQUIRE_TIMEOUT,
isNeverSentError,
isRetriableError,
isRolledBackError,
} from './retriableErrors.js';
const DEFAULT_MAX_QUEUE_SIZE = 1000;
@@ -361,8 +362,14 @@ export class SQLBatcher {
// Run one fallback item, retrying transient failures with backoff.
// A read-only batcher may retry anything transient; a batcher that
// carries writes only retries failures where the statement provably
// never reached the server — a write that died mid-flight may have
// committed, and re-running it would double-apply.
// did not apply — either it never reached the server, or the server
// rolled it back itself. A write that died mid-flight may have
// committed, and re-running that one would double-apply.
//
// Lock contention lands in the second group and is worth retrying rather
// than surfacing: an item is a single statement, so a deadlock victim has
// been fully undone, and the caller sees an unhandled 500 for what the
// database is telling us to just run again.
async #runFallbackItem(b) {
let attempt = 0;
while (true) {
@@ -379,8 +386,8 @@ export class SQLBatcher {
};
} catch (error) {
const canRetry = this.readOnly
? isRetriableError(error)
: isNeverSentError(error);
? isRetriableError(error) || isRolledBackError(error)
: isNeverSentError(error) || isRolledBackError(error);
if (!canRetry || attempt >= ITEM_RETRY_ATTEMPTS) {
return { ok: false, error };
}
@@ -208,6 +208,36 @@ describe('SQLBatcher', () => {
expect(fallbackAttempts).toBe(2);
});
it('retries a deadlocked write instead of surfacing it', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw makeError('ER_LOCK_DEADLOCK');
fallbackAttempts++;
if (fallbackAttempts === 1) throw makeError('ER_LOCK_DEADLOCK');
return [[{ ok: 1 }], undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
// The victim statement was rolled back by the server, so re-running it
// can't double-apply — the caller should never see the deadlock.
const result = await batcher.query('UPDATE notification SET x', []);
expect(result[0]).toEqual([{ ok: 1 }]);
expect(fallbackAttempts).toBe(2);
});
it('gives up on a write that deadlocks past the retry budget', async () => {
const conn = makeConnection(() => {
throw makeError('ER_LOCK_DEADLOCK');
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxTimeInQueue: 5 });
await expect(batcher.query('UPDATE hot SET x', [])).rejects.toMatchObject(
{ code: 'ER_LOCK_DEADLOCK' },
);
});
it('never retries deterministic row-level errors and does not escalate the breaker', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
@@ -18,7 +18,7 @@
*/
import Database from 'better-sqlite3';
import { existsSync, mkdtempSync, rmSync } from 'node:fs';
import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
@@ -27,7 +27,15 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 60;
const CURRENT_SCHEMA_VERSION = 66;
/**
* These suites migrate real files on disk. Idle they finish in well under a
* second, but the whole migration chain runs against the filesystem — enough
* that a loaded runner blows the 5s default and reports a timeout where there
* is no fault.
*/
const DISK_MIGRATION_TIMEOUT_MS = 30_000;
const SYSTEM_USER_UUID = '5d4adce0-a381-4982-9c02-6e2540026238';
const sqliteConfig = (
@@ -52,7 +60,7 @@ const userVersionOf = async (client: SqliteDatabaseClient): Promise<number> => {
return row.user_version as number;
};
describe('SqliteDatabaseClient — boot and migrations', () => {
describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATION_TIMEOUT_MS }, () => {
let client: SqliteDatabaseClient;
beforeEach(async () => {
@@ -83,6 +91,105 @@ describe('SqliteDatabaseClient — boot and migrations', () => {
expect(rows).toEqual([{ username: 'system' }]);
});
// 0070 sweeps three by-hand groups no code reads. Both tables it can reach
// cascade on delete, so the guard is the point: a group that still carries
// permissions or members has to survive, or the sweep silently revokes
// them. Re-running the statement is safe — it is a plain conditional DELETE.
describe('0070 drop-orphaned-default-groups', () => {
const MIGRATION = readFileSync(
new URL(
'./migrations/sqlite/0070_drop-orphaned-default-groups.sql',
import.meta.url,
),
'utf8',
);
const seedGroup = async (name: string): Promise<number> => {
const [system] = await client.read(
'SELECT `id` FROM `user` WHERE `uuid` = ?',
[SYSTEM_USER_UUID],
);
const uid = `grp-${name}-${Math.random().toString(36).slice(2, 10)}`;
await client.write(
'INSERT INTO `group` (`uid`, `owner_user_id`, `extra`, `metadata`) ' +
'VALUES (?, ?, ?, ?)',
[uid, system.id, JSON.stringify({ name }), '{}'],
);
const [row] = await client.read(
'SELECT `id` FROM `group` WHERE `uid` = ?',
[uid],
);
return Number(row.id);
};
const stillThere = async (id: number): Promise<boolean> => {
const rows = await client.read(
'SELECT `id` FROM `group` WHERE `id` = ?',
[id],
);
return rows.length > 0;
};
it('drops an orphaned freeai/experimental/dangerous group', async () => {
const ids = await Promise.all(
['freeai', 'experimental', 'dangerous'].map(seedGroup),
);
await client.write(MIGRATION);
for (const id of ids) expect(await stillThere(id)).toBe(false);
});
it('spares one that still carries a permission row', async () => {
const id = await seedGroup('freeai');
const [system] = await client.read(
'SELECT `id` FROM `user` WHERE `uuid` = ?',
[SYSTEM_USER_UUID],
);
await client.write(
'INSERT INTO `user_to_group_permissions` ' +
'(`user_id`, `group_id`, `permission`, `extra`) VALUES (?, ?, ?, ?)',
[system.id, id, 'service:some-paid-thing', '{}'],
);
await client.write(MIGRATION);
expect(await stillThere(id)).toBe(true);
// And the grant it carries is still intact, not cascaded away.
const perms = await client.read(
'SELECT `permission` FROM `user_to_group_permissions` WHERE `group_id` = ?',
[id],
);
expect(perms).toEqual([{ permission: 'service:some-paid-thing' }]);
});
it('spares one that still has members', async () => {
const id = await seedGroup('experimental');
const [system] = await client.read(
'SELECT `id` FROM `user` WHERE `uuid` = ?',
[SYSTEM_USER_UUID],
);
await client.write(
'INSERT INTO `jct_user_group` (`user_id`, `group_id`) VALUES (?, ?)',
[system.id, id],
);
await client.write(MIGRATION);
expect(await stillThere(id)).toBe(true);
});
it('leaves the groups the platform actually depends on alone', async () => {
// system, admin, user and temp are named by config or code.
await client.write(MIGRATION);
const rows = await client.read(
"SELECT json_extract(`extra`, '$.name') AS name FROM `group`",
);
const names = rows.map((r) => String(r.name));
for (const name of ['system', 'admin', 'user']) {
expect(names).toContain(name);
}
});
});
it('leaves an already-migrated database untouched on a second boot', async () => {
const dir = mkdtempSync(join(tmpdir(), 'puter-sqlite-'));
const path = join(dir, 'nested', 'puter.sqlite');
@@ -137,7 +244,10 @@ describe('SqliteDatabaseClient — boot and migrations', () => {
});
});
describe('SqliteDatabaseClient — legacy version inference', () => {
describe(
'SqliteDatabaseClient — legacy version inference',
{ timeout: DISK_MIGRATION_TIMEOUT_MS },
() => {
let dir: string;
beforeEach(() => {
@@ -94,6 +94,12 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[57, ['0062_blocked-app-origins.sql']],
[58, ['0063_add_suspended_reason.sql']],
[59, ['0064_abuse-moderation-events.sql']],
[60, ['0065_app-feedback.sql']],
[61, ['0066_owned-email-unique.sql']],
[62, ['0067_share_entries.sql']],
[63, ['0068_referral-code-unique.sql']],
[64, ['0069_user-block.sql']],
[65, ['0070_drop-orphaned-default-groups.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,48 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- User-to-developer feedback for apps that opt in. Mirrors SQLite migration
-- 0065. Opt-in is the new `apps.feedback_enabled` column (developer-writable
-- through the regular `puter.apps.update` path). Each row of `app_feedback`
-- is one message a signed-in user submitted through the GUI feedback dialog;
-- a copy is emailed to the app owner unless the per-app daily email cap
-- suppressed it (`email_sent` records which). `app_uid` is denormalized
-- alongside `app_id` so rows stay attributable after an app is deleted.
-- `created_at` is unix seconds.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- always runs first) and the table uses `CREATE TABLE IF NOT EXISTS`, so the
-- directory can replay safely.
CALL _puter_add_col('apps', 'feedback_enabled', '`feedback_enabled` tinyint(1) DEFAULT ''0''');
CREATE TABLE IF NOT EXISTS `app_feedback` (
`id` INT NOT NULL AUTO_INCREMENT,
`uid` CHAR(36) NOT NULL,
`app_id` INT NOT NULL,
`app_uid` CHAR(40) NOT NULL,
`user_id` INT NOT NULL,
`message` TEXT NOT NULL,
`source_env` VARCHAR(16) DEFAULT NULL,
`source_origin` VARCHAR(2048) DEFAULT NULL,
`email_sent` TINYINT(1) NOT NULL DEFAULT 0,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uq_app_feedback_uid` (`uid`),
KEY `idx_app_feedback_app_created` (`app_id`, `created_at`),
KEY `idx_app_feedback_user_created` (`user_id`, `created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,128 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce "at most one account owns an email address". Mirrors SQLite
-- migration 0066.
--
-- `user.email` is deliberately not UNIQUE: several rows may legitimately hold
-- the same address while unconfirmed (admin-provisioned placeholders, signups
-- that were never confirmed, temp accounts on their way to becoming real). What
-- must never happen is two rows both *owning* an address — owning meaning the
-- row is confirmed, or holds a password and so can drive password recovery for
-- that inbox.
--
-- Signup, save-account, change-email, OIDC and admin provisioning each check for
-- an owner before writing, but a check and an insert are not one operation: two
-- requests can both read "free" and both write. This index is what actually
-- holds the invariant; the application checks just produce a nicer error most of
-- the time.
--
-- SQLite expresses that with a partial index. MySQL has none, so the predicate
-- lives in a generated column that evaluates to NULL for every row that does not
-- own its address — and NULLs do not collide in an InnoDB unique index, which is
-- exactly the "unlimited unconfirmed placeholders" behaviour we need.
--
-- The column is VIRTUAL, not STORED, on purpose: adding a stored generated
-- column rebuilds the table, while a virtual one is a metadata-only change and
-- the index that follows builds INPLACE. On a `user` table of any size with
-- read replicas attached, that is the difference between a routine change and an
-- outage. The ALGORITHM/LOCK clauses are spelled out so a server that cannot
-- honour them refuses the statement instead of quietly copying the table.
--
-- Matching is on the canonical address so provider aliases
-- (`[email protected]` vs `[email protected]`) collide. `clean_email` is
-- written on every modern write path; the COALESCE covers rows old enough to
-- predate the column. Run the `clean_email` backfill before this migration or
-- alias collisions among those rows go unnoticed.
--
-- Idempotent: both steps are guarded on INFORMATION_SCHEMA so the directory
-- replays safely.
--
-- If the index creation fails with ER_DUP_ENTRY, the DB already contains
-- duplicate owners. Collapse them first (admin → One-off Jobs → Collapse
-- Duplicate Emails) — there is no safe automatic merge of two accounts.
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
DELIMITER //
CREATE PROCEDURE _puter_add_owned_email()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND COLUMN_NAME = 'owned_email'
) THEN
ALTER TABLE `user`
ADD COLUMN `owned_email` VARCHAR(256)
CHARACTER SET latin1 COLLATE latin1_swedish_ci
GENERATED ALWAYS AS (
CASE
WHEN `email` IS NOT NULL
AND (`email_confirmed` = 1 OR `password` IS NOT NULL)
THEN COALESCE(`clean_email`, LOWER(`email`))
ELSE NULL
END
) VIRTUAL,
ALGORITHM=INSTANT;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_owned_email'
) THEN
ALTER TABLE `user`
ADD UNIQUE KEY `idx_user_owned_email` (`owned_email`),
ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_owned_email();
DROP PROCEDURE IF EXISTS _puter_add_owned_email;
-- One Puter account per external identity. OIDCStore.link already assumes this
-- constraint exists — it catches the unique violation to tell "re-linking the
-- same account" apart from "this sub belongs to someone else" — but the table
-- never actually had it, so two concurrent first-time logins could each create
-- an account and each link the same sub. Subsequent logins then resolved to
-- whichever row came back first.
--
-- Dedupe `user_oidc_providers` before applying this: keep the lowest `id` per
-- (provider, provider_sub) and point it at the account the collapse job kept.
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
DELIMITER //
CREATE PROCEDURE _puter_add_oidc_sub_unique()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_oidc_providers'
AND INDEX_NAME = 'idx_user_oidc_provider_sub'
) THEN
ALTER TABLE `user_oidc_providers`
ADD UNIQUE KEY `idx_user_oidc_provider_sub` (`provider`, `provider_sub`);
END IF;
END//
DELIMITER ;
CALL _puter_add_oidc_sub_unique();
DROP PROCEDURE IF EXISTS _puter_add_oidc_sub_unique;
@@ -0,0 +1,106 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Grow `share` from a pending-email-invite table into the index of active
-- shares. See sqlite/0067_share_entries.sql for the column rationale.
--
-- Idempotent: columns go through _puter_add_col (from mig_1), indexes and
-- foreign keys through the guarded procedure below. There is no per-file
-- applied-state tracking, so every statement must tolerate a re-run.
CALL _puter_add_col('share', 'holder_user_id', '`holder_user_id` int unsigned DEFAULT NULL');
CALL _puter_add_col('share', 'fsentry_id', '`fsentry_id` int unsigned DEFAULT NULL');
CALL _puter_add_col('share', 'mode', '`mode` varchar(20) DEFAULT NULL');
CALL _puter_add_col('share', 'applied_at', '`applied_at` timestamp NULL DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
DELIMITER //
CREATE PROCEDURE _puter_add_share_index_constraints()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_holder` (`holder_user_id`, `id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_fsentry'
) THEN
ALTER TABLE `share` ADD INDEX `idx_share_fsentry` (`fsentry_id`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND INDEX_NAME = 'idx_share_holder_entry_issuer'
) THEN
ALTER TABLE `share` ADD UNIQUE INDEX `idx_share_holder_entry_issuer`
(`holder_user_id`, `fsentry_id`, `issuer_user_id`);
END IF;
-- Retiring a deleted node's grants looks them up by permission text
-- (`fs:<uuid>` and `fs:<uuid>:%`); the subject lives in that text rather
-- than a column, so no foreign key can cascade it. The primary key is
-- (issuer, holder, permission), which never puts `permission` first, so
-- without this both the equality and the left-anchored LIKE degrade to a
-- full scan on the delete path. The column is ascii varchar(255), well
-- inside InnoDB's key limit, so it is indexed whole.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_to_user_permissions'
AND INDEX_NAME = 'idx_user_to_user_permissions_permission'
) THEN
ALTER TABLE `user_to_user_permissions`
ADD INDEX `idx_user_to_user_permissions_permission` (`permission`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_holder_user_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_holder_user_fk`
FOREIGN KEY (`holder_user_id`) REFERENCES `user` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
-- The cascade that retires a share with its file.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'share'
AND CONSTRAINT_NAME = 'share_fsentry_fk'
) THEN
ALTER TABLE `share` ADD CONSTRAINT `share_fsentry_fk`
FOREIGN KEY (`fsentry_id`) REFERENCES `fsentries` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END//
DELIMITER ;
CALL _puter_add_share_index_constraints();
DROP PROCEDURE IF EXISTS _puter_add_share_index_constraints;
@@ -0,0 +1,35 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- One user refusing contact from another. See sqlite/0068_user-block.sql for
-- the rationale. `created_at` is unix seconds.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` with the indexes declared inline,
-- so the directory can replay safely.
CREATE TABLE IF NOT EXISTS `user_block` (
`id` INT NOT NULL AUTO_INCREMENT,
`blocker_user_id` INT UNSIGNED NOT NULL,
`blocked_user_id` INT UNSIGNED NOT NULL,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_user_block_pair` (`blocker_user_id`, `blocked_user_id`),
CONSTRAINT `fk_user_block_blocker` FOREIGN KEY (`blocker_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_user_block_blocked` FOREIGN KEY (`blocked_user_id`)
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,61 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Widen `notification.shown` and `notification.acknowledged` to hold the unix
-- second the store has been writing to them since the backend rework.
--
-- Both arrived from the v1 schema as `tinyint(1)`, where they were flags. The
-- rework changed the writes to timestamps and sqlite (`INTEGER`) and postgres
-- (`bigint`) took them; only mysql was left too narrow, so every `markShown`
-- and `markAcknowledged` there fails with ER_WARN_DATA_OUT_OF_RANGE and the
-- column stays NULL. Dismissing a notification therefore never sticks, and one
-- already delivered is re-sent on every reconnect.
--
-- No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a legacy
-- `1` keeps meaning "yes" once widened.
--
-- Guarded on the current type rather than run unconditionally. Changing a
-- column type copies the table, and this one grows without bound — a migration
-- directory that replays on every boot must not rebuild it every time.
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
DELIMITER //
CREATE PROCEDURE _puter_widen_notification_stamps()
BEGIN
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND COLUMN_NAME = 'shown'
AND DATA_TYPE = 'tinyint'
) THEN
ALTER TABLE `notification` MODIFY `shown` BIGINT DEFAULT NULL;
END IF;
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'notification'
AND COLUMN_NAME = 'acknowledged'
AND DATA_TYPE = 'tinyint'
) THEN
ALTER TABLE `notification` MODIFY `acknowledged` BIGINT DEFAULT NULL;
END IF;
END //
DELIMITER ;
CALL _puter_widen_notification_stamps();
DROP PROCEDURE IF EXISTS _puter_widen_notification_stamps;
@@ -0,0 +1,22 @@
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
-- `dangerous`. No migration creates them — they were added by hand back when
-- hardcoded permissions were keyed by group name. That map is now a flat
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
-- so a group nothing looks up grants nothing, and the rows are dead weight.
--
-- Guarded, and deliberately so: `user_to_group_permissions.group_id` and
-- `jct_user_group.group_id` both cascade on delete, so removing a group that
-- still carries permissions or members would silently revoke them from every
-- member. Only a group with neither is dropped. A group that survives this
-- migration has dependents and needs a deliberate decision, not a sweep --
-- query `user_to_group_permissions` by `group_id` to see what it holds.
--
-- The joins rather than `NOT IN` subqueries: MySQL will not read from the table
-- it is deleting from inside a subquery on the same statement.
DELETE g FROM `group` g
LEFT JOIN `user_to_group_permissions` p ON p.`group_id` = g.`id`
LEFT JOIN `jct_user_group` j ON j.`group_id` = g.`id`
WHERE JSON_UNQUOTE(JSON_EXTRACT(g.`extra`, '$.name'))
IN ('freeai', 'experimental', 'dangerous')
AND p.`group_id` IS NULL
AND j.`group_id` IS NULL;
@@ -0,0 +1,40 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce "at most one account owns an email address". Mirrors SQLite
-- migration 0066 and MySQL migration 21; see those for the full rationale.
--
-- In short: `user.email` is deliberately not UNIQUE because several rows may
-- hold the same address while unconfirmed. What must not happen is two rows
-- both owning it — confirmed, or holding a password and so able to drive
-- password recovery for that inbox. The application checks for an owner before
-- every write, but a check and a write are not one operation.
--
-- If this fails, the DB already contains duplicate owners; collapse them first
-- (admin → One-off Jobs → Collapse Duplicate Emails).
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_owned_email
ON "user" (COALESCE(clean_email, LOWER(email)))
WHERE email IS NOT NULL
AND (email_confirmed = TRUE OR password IS NOT NULL);
-- One Puter account per external identity. OIDCStore.link already assumes this
-- constraint exists — it catches the unique violation to tell "re-linking the
-- same account" apart from "this sub belongs to someone else" — but the table
-- never actually had it.
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_provider_sub
ON user_oidc_providers (provider, provider_sub);
@@ -0,0 +1,42 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Grow share from a pending-email-invite table into the index of active
-- shares. See sqlite/0067_share_entries.sql for the column rationale.
ALTER TABLE share
ADD COLUMN IF NOT EXISTS holder_user_id integer
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
ADD COLUMN IF NOT EXISTS fsentry_id integer
REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
ADD COLUMN IF NOT EXISTS mode varchar(20),
ADD COLUMN IF NOT EXISTS applied_at timestamp;
CREATE INDEX IF NOT EXISTS idx_share_holder
ON share (holder_user_id, id);
CREATE INDEX IF NOT EXISTS idx_share_fsentry
ON share (fsentry_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_share_holder_entry_issuer
ON share (holder_user_id, fsentry_id, issuer_user_id);
-- Retiring a deleted node's grants looks them up by permission text, and the
-- primary key is (issuer, holder, permission) — `permission` is never the
-- leading column, so the delete path would otherwise scan the table. The
-- text_pattern_ops class is what makes the left-anchored LIKE a range scan
-- under a non-C collation; plain equality still uses it.
CREATE INDEX IF NOT EXISTS idx_user_to_user_permissions_permission
ON user_to_user_permissions (permission text_pattern_ops);
@@ -0,0 +1,33 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- One user refusing contact from another. See sqlite/0068_user-block.sql for
-- the rationale. `created_at` is unix seconds.
--
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS user_block (
id BIGSERIAL PRIMARY KEY,
blocker_user_id integer NOT NULL
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
blocked_user_id integer NOT NULL
REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
created_at BIGINT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_block_pair
ON user_block (blocker_user_id, blocked_user_id);
@@ -0,0 +1,20 @@
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
-- `dangerous`. No migration creates them — they were added by hand back when
-- hardcoded permissions were keyed by group name. That map is now a flat
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
-- so a group nothing looks up grants nothing, and the rows are dead weight.
--
-- Guarded, and deliberately so: user_to_group_permissions.group_id and
-- jct_user_group.group_id both cascade on delete, so removing a group that
-- still carries permissions or members would silently revoke them from every
-- member. Only a group with neither is dropped. A group that survives this
-- migration has dependents and needs a deliberate decision, not a sweep --
-- query user_to_group_permissions by group_id to see what it holds.
DELETE FROM "group" g
WHERE g.extra ->> 'name' IN ('freeai', 'experimental', 'dangerous')
AND NOT EXISTS (
SELECT 1 FROM user_to_group_permissions p WHERE p.group_id = g.id
)
AND NOT EXISTS (
SELECT 1 FROM jct_user_group j WHERE j.group_id = g.id
);
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- User-to-developer feedback for apps that opt in. Mirrors SQLite migration
-- 0065 / MySQL mysql_mig_20. Opt-in is the new `apps.feedback_enabled` column
-- (developer-writable through the regular `puter.apps.update` path). Each row
-- of `app_feedback` is one message a signed-in user submitted through the GUI
-- feedback dialog; a copy is emailed to the app owner unless the per-app
-- daily email cap suppressed it (`email_sent` records which). `app_uid` is
-- denormalized alongside `app_id` so rows stay attributable after an app is
-- deleted. `created_at` is unix seconds.
--
-- Idempotent via IF NOT EXISTS.
ALTER TABLE apps ADD COLUMN IF NOT EXISTS feedback_enabled boolean NOT NULL DEFAULT FALSE;
CREATE TABLE IF NOT EXISTS app_feedback (
id BIGSERIAL PRIMARY KEY,
uid CHAR(36) NOT NULL UNIQUE,
app_id BIGINT NOT NULL,
app_uid VARCHAR(40) NOT NULL,
user_id BIGINT NOT NULL,
message TEXT NOT NULL,
source_env VARCHAR(16) DEFAULT NULL,
source_origin VARCHAR(2048) DEFAULT NULL,
email_sent BOOLEAN NOT NULL DEFAULT FALSE,
created_at BIGINT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_app_feedback_app_created
ON app_feedback (app_id, created_at);
CREATE INDEX IF NOT EXISTS idx_app_feedback_user_created
ON app_feedback (user_id, created_at);
@@ -0,0 +1,51 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- User-to-developer feedback for apps that opt in. Opt-in is the new
-- `apps.feedback_enabled` column (developer-writable through the regular
-- `puter.apps.update` path; a dedicated column rather than a `metadata` key
-- because Dev Center saves replace the whole metadata blob and would erase
-- it). Each row of `app_feedback` is one message a signed-in user submitted
-- through the GUI feedback dialog; a copy is emailed to the app owner unless
-- the per-app daily email cap suppressed it (`email_sent` records which).
-- `app_uid` is denormalized alongside `app_id` so rows stay attributable
-- after an app is deleted (abuse forensics). `source_env` is 'app' (desktop
-- dialog) or 'web' (puter.com popup opened from an external site);
-- `source_origin` is the popup opener's browser-attested origin, null for
-- desktop submissions. `created_at` is unix seconds. The (user_id,
-- created_at) and (app_id, created_at) indexes serve the sliding-window
-- rate-limit counts in AppFeedbackStore.
ALTER TABLE apps ADD COLUMN "feedback_enabled" tinyint(1) DEFAULT '0';
CREATE TABLE IF NOT EXISTS `app_feedback` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"uid" TEXT NOT NULL UNIQUE,
"app_id" INTEGER NOT NULL,
"app_uid" TEXT NOT NULL,
"user_id" INTEGER NOT NULL,
"message" TEXT NOT NULL,
"source_env" TEXT DEFAULT NULL, -- 'app' | 'web'
"source_origin" TEXT DEFAULT NULL, -- attested opener origin (web popups)
"email_sent" INTEGER NOT NULL DEFAULT 0,
"created_at" INTEGER NOT NULL -- unix seconds
);
CREATE INDEX IF NOT EXISTS idx_app_feedback_app_created
ON `app_feedback` (`app_id`, `created_at`);
CREATE INDEX IF NOT EXISTS idx_app_feedback_user_created
ON `app_feedback` (`user_id`, `created_at`);
@@ -0,0 +1,53 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce "at most one account owns an email address".
--
-- `user.email` is deliberately not UNIQUE: several rows may legitimately hold
-- the same address while unconfirmed (admin-provisioned placeholders, signups
-- that were never confirmed, temp accounts on their way to becoming real). What
-- must never happen is two rows both *owning* an address — owning meaning the
-- row is confirmed, or holds a password and so can drive password recovery for
-- that inbox.
--
-- Signup, save-account, change-email, OIDC and admin provisioning each check for
-- an owner before writing, but a check and an insert are not one operation: two
-- requests can both read "free" and both write. This index is what actually
-- holds the invariant; the application checks just produce a nicer error most of
-- the time.
--
-- Matching is on the canonical address so provider aliases
-- (`[email protected]` vs `[email protected]`) collide. `clean_email` is
-- written on every modern write path; the COALESCE covers rows old enough to
-- predate the column.
--
-- If this CREATE fails, the DB already contains duplicate owners. Collapse them
-- first (admin → One-off Jobs → Collapse Duplicate Emails) — there is no safe
-- automatic merge of two accounts.
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_owned_email
ON user(COALESCE(clean_email, lower(email)))
WHERE email IS NOT NULL
AND (email_confirmed = 1 OR password IS NOT NULL);
-- One Puter account per external identity. OIDCStore.link already assumes this
-- constraint exists — it catches the unique violation to tell "re-linking the
-- same account" apart from "this sub belongs to someone else" — but the table
-- never actually had it, so two concurrent first-time logins could each create
-- an account and each link the same sub. Subsequent logins then resolved to
-- whichever row came back first.
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_provider_sub
ON user_oidc_providers(provider, provider_sub);
@@ -0,0 +1,60 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Grow `share` from a pending-email-invite table into the index of active
-- shares. Permissions stay the source of truth for access; this is what makes
-- shares listable and gives them a lifecycle.
--
-- - `holder_user_id` : NULL while an invite awaits signup.
-- - `fsentry_id` : the shared node. ON DELETE CASCADE retires the share
-- with the file, which permissions alone don't do.
-- - `mode` : see|list|read|write|manage. Unconstrained on purpose —
-- ACLService owns the mode set and adding one shouldn't
-- need a three-dialect migration.
-- - `applied_at` : set when an invite is claimed. Claiming updates the
-- row rather than deleting it, so the share stays
-- queryable.
ALTER TABLE `share` ADD COLUMN `holder_user_id` INTEGER DEFAULT NULL
REFERENCES `user` (`id`) ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TABLE `share` ADD COLUMN `fsentry_id` INTEGER DEFAULT NULL
REFERENCES `fsentries` (`id`) ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TABLE `share` ADD COLUMN `mode` TEXT DEFAULT NULL;
ALTER TABLE `share` ADD COLUMN `applied_at` TIMESTAMP DEFAULT NULL;
-- "Shared with me", keyset-paginated: ORDER BY ends in `id` as the tiebreaker.
CREATE INDEX IF NOT EXISTS `idx_share_holder`
ON `share` (`holder_user_id`, `id`);
-- Who has access to one node — also how an owner sees a manage-delegate's
-- re-grants, which the issuer/holder permission tables can't answer.
CREATE INDEX IF NOT EXISTS `idx_share_fsentry` ON `share` (`fsentry_id`);
-- One row per (holder, node, issuer). Pending invites have a NULL
-- holder_user_id and so aren't covered here; dedup for those belongs with the
-- invite flow.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_share_holder_entry_issuer`
ON `share` (`holder_user_id`, `fsentry_id`, `issuer_user_id`);
-- Retiring a deleted node's grants looks them up by permission text
-- (`fs:<uuid>` and `fs:<uuid>:%`), and the subject only lives in that text —
-- no column, so no foreign key can cascade it. The table's primary key is
-- (issuer, holder, permission), which never puts `permission` first, so
-- without this the equality and the left-anchored LIKE both degrade to a full
-- scan on the delete path. With it, each is a range scan.
CREATE INDEX IF NOT EXISTS `idx_user_to_user_permissions_permission`
ON `user_to_user_permissions` (`permission`);
@@ -0,0 +1,32 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- One referral code, one account. The mysql and postgres baselines have
-- declared `user.referral_code` UNIQUE since the column was introduced; the
-- sqlite baseline only declared the column, so the invariant held on hosted
-- deployments and not on self-hosted ones.
--
-- Codes are minted by picking at random and writing, so this index is not a
-- nicety: it is what makes "did anyone already take this code" a decision the
-- database makes once, instead of a read the next mint can race. A caller that
-- loses the race sees a unique violation and picks again.
--
-- NULL is distinct from NULL in a sqlite unique index, so accounts that have
-- never asked for a code (the majority) are unaffected.
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_referral_code
ON user(referral_code)
WHERE referral_code IS NOT NULL;
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- One user refusing contact from another. Sharing is what reads it today —
-- a blocked sender's share is refused outright, and their pending invite is
-- dropped when the blocker confirms the address it was aimed at — but the
-- table is deliberately not share-specific: the same answer serves anything
-- else one person can push at another.
--
-- Rows are read on the share path by exact (blocker, blocked) pair, which the
-- unique index answers as a point lookup; the same index's leading column
-- serves "who have I blocked". `created_at` is unix seconds, matching
-- `app_feedback`.
CREATE TABLE IF NOT EXISTS `user_block` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"blocker_user_id" INTEGER NOT NULL
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
"blocked_user_id" INTEGER NOT NULL
REFERENCES `user` ("id") ON DELETE CASCADE ON UPDATE CASCADE,
"created_at" INTEGER NOT NULL -- unix seconds
);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_user_block_pair`
ON `user_block` (`blocker_user_id`, `blocked_user_id`);
@@ -0,0 +1,33 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Drops three groups that no code has ever read: `freeai`, `experimental` and
-- `dangerous`. No migration creates them — they were added by hand back when
-- hardcoded permissions were keyed by group name. That map is now a flat
-- per-user floor (`default_user_permissions` in data/hardcoded-permissions.js),
-- so a group nothing looks up grants nothing, and the rows are dead weight.
--
-- Guarded, and deliberately so: `user_to_group_permissions.group_id` and
-- `jct_user_group.group_id` both cascade on delete, so removing a group that
-- still carries permissions or members would silently revoke them from every
-- member. Only a group with neither is dropped. A group that survives this
-- migration has dependents and needs a deliberate decision, not a sweep --
-- query `user_to_group_permissions` by `group_id` to see what it holds.
DELETE FROM `group`
WHERE json_extract(`extra`, '$.name') IN ('freeai', 'experimental', 'dangerous')
AND `id` NOT IN (SELECT `group_id` FROM `user_to_group_permissions`)
AND `id` NOT IN (SELECT `group_id` FROM `jct_user_group`);
@@ -54,6 +54,26 @@ const NEVER_SENT_ERROR_CODES = new Set([
POOL_ACQUIRE_TIMEOUT,
]);
/**
* Failures the server itself rolled back before returning. The statement did
* reach the database, so this is not `NEVER_SENT`, but InnoDB guarantees it
* left no effect — which makes a retry just as safe for writes.
*
* Only true for a _single_ statement: under autocommit each statement is its
* own transaction, so "the transaction was rolled back" means "this statement
* was rolled back". Retrying a multi-statement string on one of these would
* re-run the statements that already committed ahead of the failure.
*/
const ROLLED_BACK_ERROR_CODES = new Set([
// Deadlock — InnoDB picked this transaction as the victim and undid it.
// MySQL's own message for it is "try restarting transaction".
'ER_LOCK_DEADLOCK',
// Lock wait timeout. Rolls back the statement rather than the transaction
// unless innodb_rollback_on_timeout is set — the same thing when the
// transaction is one statement.
'ER_LOCK_WAIT_TIMEOUT',
]);
const errorCode = (error: unknown): string | undefined =>
(error as { code?: string } | null)?.code;
@@ -74,3 +94,13 @@ export const isNeverSentError = (error: unknown): boolean => {
const code = errorCode(error);
return Boolean(code && NEVER_SENT_ERROR_CODES.has(code));
};
/**
* Lock-contention failures the server rolled back on its own. Safe to retry a
* single statement on, writes included — see `ROLLED_BACK_ERROR_CODES` for the
* one-statement precondition.
*/
export const isRolledBackError = (error: unknown): boolean => {
const code = errorCode(error);
return Boolean(code && ROLLED_BACK_ERROR_CODES.has(code));
};
+39 -9
View File
@@ -3,18 +3,19 @@
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
import { createServer, type Server } from 'node:http';
@@ -302,6 +303,35 @@ describe('DDBClient — batch writes', () => {
const stored = await client.query(TABLE, { pk: 'bulk' });
expect(stored.Items).toHaveLength(60);
});
it('reports no consumed capacity for an empty delete batch', async () => {
await expect(client.batchDel([])).resolves.toEqual({
ConsumedCapacity: [],
});
});
it('batch-deletes across chunks, leaving unrelated items alone', async () => {
const items = Array.from({ length: 30 }, (_, index) => ({
table: TABLE,
item: { pk: 'bulk-del', sk: `item-${index}`, index },
}));
await client.batchPut(items);
// Delete all but the last item — 29 keys still exercises chunking
// paths shared with batchPut while proving deletion is targeted.
const result = await client.batchDel(
items.slice(0, -1).map(({ table, item }) => ({
table,
key: { pk: item.pk, sk: item.sk },
})),
);
expect(result.ConsumedCapacity).toHaveLength(1);
expect(result.ConsumedCapacity[0].TableName).toBe(TABLE);
const stored = await client.query(TABLE, { pk: 'bulk-del' });
expect(stored.Items?.map((item) => item.sk)).toEqual(['item-29']);
});
});
describe('DDBClient — expired item sweep', () => {
+137 -31
View File
@@ -45,6 +45,11 @@ import { HttpError } from '../../core/http';
import type { IConfig, IDynamoConfig } from '../../types';
import { Span } from '../../util/span.js';
import { PuterClient } from '../types';
import {
clampStoredNumber,
isRepairableMarshallError,
repairForMarshall,
} from './marshallRepair.js';
const LOCAL_DYNAMO_MEMORY_PREFIX = ':memory:';
const localDynaliteEndpointPromises = new Map<string, Promise<string>>();
@@ -114,6 +119,49 @@ const sleep = async (ms: number) => {
await new Promise((resolve) => setTimeout(resolve, ms));
};
// One caller writing an out-of-range number usually keeps doing it, so the
// warning is per-target and throttled rather than one line per write.
const REPAIR_WARNING_INTERVAL_MS = 60_000;
const lastRepairWarningAt = new Map<string, number>();
const warnRepaired = (target: string, message: string): void => {
const now = Date.now();
const lastWarnedAt = lastRepairWarningAt.get(target) ?? 0;
if (now - lastWarnedAt < REPAIR_WARNING_INTERVAL_MS) return;
lastRepairWarningAt.set(target, now);
console.warn(
`[ddb] clamped an out-of-range value in ${target}: ${message}`,
);
};
/**
* Send a write, and if the payload was rejected while being encoded for a value
* the store cannot represent, clamp it and send it once more.
*
* Nothing is inspected on the way in: the payload of a successful write is
* never walked, so this costs one `try` on the hot path. The repair only runs
* on the failure, and only retries when it actually changed something —
* otherwise the original error stands.
*/
const sendRepairingValues = async <TPayload, TResult>(
payload: TPayload,
send: (payload: TPayload) => Promise<TResult>,
describe: () => string,
): Promise<TResult> => {
try {
return await send(payload);
} catch (error) {
if (!isRepairableMarshallError(error)) throw error;
const repaired = repairForMarshall(payload);
if (!repaired.changed) throw error;
warnRepaired(describe(), (error as Error).message);
return send(repaired.value as TPayload);
}
};
export class DDBClient extends PuterClient {
#documentClient: DynamoDBDocumentClient | null = null;
#localInitPromise: Promise<void> | null = null;
@@ -171,14 +219,19 @@ export class DDBClient extends PuterClient {
@Span('ddb.put', (table: string) => ({ 'db.table': table }))
async put<T extends Record<string, unknown>>(table: string, item: T) {
const command = new PutCommand({
TableName: table,
Item: item,
ReturnConsumedCapacity: 'TOTAL',
});
const client = await this.#getDocumentClient();
return client.send(command);
return sendRepairingValues(
item,
(itemToWrite) =>
client.send(
new PutCommand({
TableName: table,
Item: itemToWrite,
ReturnConsumedCapacity: 'TOTAL',
}),
),
() => `a put to ${table}`,
);
}
@Span('ddb.batchGet', (params: unknown[]) => ({
@@ -222,6 +275,37 @@ export class DDBClient extends PuterClient {
'db.batch_size': params.length,
}))
async batchPut(params: { table: string; item: Record<string, unknown> }[]) {
return this.#batchWrite(
params.map(({ table, item }) => ({
table,
request: { PutRequest: { Item: item } },
})),
);
}
@Span('ddb.batchDel', (params: unknown[]) => ({
'db.batch_size': params.length,
}))
async batchDel(params: { table: string; key: Record<string, unknown> }[]) {
return this.#batchWrite(
params.map(({ table, key }) => ({
table,
request: { DeleteRequest: { Key: key } },
})),
);
}
// Shared BatchWriteItem plumbing for batchPut/batchDel: 25-item chunks,
// UnprocessedItems retried with capped exponential backoff, consumed
// capacity accumulated per table across every request.
async #batchWrite(
params: {
table: string;
request: NonNullable<
BatchWriteCommandInput['RequestItems']
>[string][number];
}[],
) {
const consumedCapacityByTable = new Map<string, number>();
if (params.length === 0) {
return { ConsumedCapacity: [] };
@@ -258,11 +342,7 @@ export class DDBClient extends PuterClient {
let requestItems = chunk.reduce(
(acc, curr) => {
const tableRequests = acc[curr.table] ?? [];
tableRequests.push({
PutRequest: {
Item: curr.item,
},
});
tableRequests.push(curr.request);
acc[curr.table] = tableRequests;
return acc;
},
@@ -278,11 +358,17 @@ export class DDBClient extends PuterClient {
break;
}
const response = await client.send(
new BatchWriteCommand({
RequestItems: requestItems,
ReturnConsumedCapacity: 'TOTAL',
}),
const response = await sendRepairingValues(
requestItems,
(itemsToWrite) =>
client.send(
new BatchWriteCommand({
RequestItems: itemsToWrite,
ReturnConsumedCapacity: 'TOTAL',
}),
),
() =>
`a batch write to ${Object.keys(requestItems).join(', ')}`,
);
accumulateConsumedCapacity(
response.ConsumedCapacity as
@@ -328,11 +414,18 @@ export class DDBClient extends PuterClient {
}
@Span('ddb.del', (table: string) => ({ 'db.table': table }))
async del<T extends Record<string, unknown>>(table: string, key: T) {
async del<T extends Record<string, unknown>>(
table: string,
key: T,
opts?: { returnOld?: boolean },
) {
const command = new DeleteCommand({
TableName: table,
Key: key,
ReturnConsumedCapacity: 'TOTAL',
// ALL_OLD makes the delete an atomic claim: exactly one caller
// gets the attributes back.
...(opts?.returnOld ? { ReturnValues: 'ALL_OLD' as const } : {}),
});
const client = await this.#getDocumentClient();
@@ -423,20 +516,27 @@ export class DDBClient extends PuterClient {
!!expressionValues && Object.keys(expressionValues).length > 0;
const hasNames =
!!expressionNames && Object.keys(expressionNames).length > 0;
const command = new UpdateCommand({
TableName: table,
Key: key,
UpdateExpression: expression,
...(hasValues
? { ExpressionAttributeValues: expressionValues }
: {}),
...(hasNames ? { ExpressionAttributeNames: expressionNames } : {}),
ReturnValues: 'ALL_NEW',
ReturnConsumedCapacity: 'TOTAL',
});
const client = await this.#getDocumentClient();
return client.send(command);
return sendRepairingValues(
expressionValues,
(valuesToWrite) =>
client.send(
new UpdateCommand({
TableName: table,
Key: key,
UpdateExpression: expression,
...(hasValues
? { ExpressionAttributeValues: valuesToWrite }
: {}),
...(hasNames
? { ExpressionAttributeNames: expressionNames }
: {}),
ReturnValues: 'ALL_NEW',
ReturnConsumedCapacity: 'TOTAL',
}),
),
() => `an update to ${table}`,
);
}
async createTableIfNotExists(
@@ -538,6 +638,9 @@ export class DDBClient extends PuterClient {
marshallOptions: {
removeUndefinedValues: true,
},
unmarshallOptions: {
wrapNumbers: clampStoredNumber,
},
});
}
@@ -565,6 +668,9 @@ export class DDBClient extends PuterClient {
marshallOptions: {
removeUndefinedValues: true,
},
unmarshallOptions: {
wrapNumbers: clampStoredNumber,
},
});
}
@@ -0,0 +1,126 @@
import { describe, expect, it } from 'vitest';
import {
clampStoredNumber,
isRepairableMarshallError,
MAX_STORED_NUMBER,
MIN_STORED_NUMBER,
repairForMarshall,
} from './marshallRepair.ts';
describe('isRepairableMarshallError', () => {
it('recognizes the out-of-range and special-value failures', () => {
for (const message of [
'Number 1.6515584833071455e+55 is greater than Number.MAX_SAFE_INTEGER. Use NumberValue from @aws-sdk/lib-dynamodb.',
'Number -1e+55 is lesser than Number.MIN_SAFE_INTEGER. Use NumberValue from @aws-sdk/lib-dynamodb.',
'Special numeric value NaN is not allowed',
'Special numeric value Infinity is not allowed',
]) {
expect(isRepairableMarshallError(new Error(message))).toBe(true);
}
});
it('leaves every other failure alone', () => {
for (const error of [
new Error('Unsupported type passed: [object Symbol].'),
new Error('ValidationException'),
new Error(''),
undefined,
'not an error',
]) {
expect(isRepairableMarshallError(error)).toBe(false);
}
});
});
describe('repairForMarshall', () => {
it('clamps a number past the safe range in both directions', () => {
expect(repairForMarshall(1.6515584833071455e55)).toEqual({
value: MAX_STORED_NUMBER,
changed: true,
});
expect(repairForMarshall(-1e55)).toEqual({
value: MIN_STORED_NUMBER,
changed: true,
});
});
it('clamps infinities and nulls NaN', () => {
expect(repairForMarshall(Infinity).value).toBe(MAX_STORED_NUMBER);
expect(repairForMarshall(-Infinity).value).toBe(MIN_STORED_NUMBER);
expect(repairForMarshall(NaN)).toEqual({ value: null, changed: true });
});
it('repairs numbers nested in objects and arrays', () => {
const repaired = repairForMarshall({
profile: { netWorth: 1e55, level: 29 },
scores: [1, [2, -1e55]],
});
expect(repaired.changed).toBe(true);
expect(repaired.value).toEqual({
profile: { netWorth: MAX_STORED_NUMBER, level: 29 },
scores: [1, [2, MIN_STORED_NUMBER]],
});
});
it('returns the original payload untouched when nothing is out of range', () => {
const payload = {
a: MAX_STORED_NUMBER,
b: [1, 2, { c: 'three' }],
d: null,
};
const repaired = repairForMarshall(payload);
expect(repaired.changed).toBe(false);
expect(repaired.value).toBe(payload);
});
it('does not mutate the payload it repairs', () => {
const nested = { netWorth: 1e55 };
const payload = { nested };
repairForMarshall(payload);
expect(nested.netWorth).toBe(1e55);
});
it('keeps a `__proto__` key as data', () => {
const payload = JSON.parse('{"__proto__":{"big":1e55}}');
const repaired = repairForMarshall(payload) as {
value: Record<string, unknown>;
changed: boolean;
};
expect(repaired.changed).toBe(true);
expect(Object.getPrototypeOf(repaired.value)).toBe(Object.prototype);
expect(
Object.getOwnPropertyDescriptor(repaired.value, '__proto__')?.value,
).toEqual({ big: MAX_STORED_NUMBER });
});
it('leaves values it does not own alone', () => {
const date = new Date(0);
expect(repairForMarshall(date).value).toBe(date);
expect(repairForMarshall('str')).toEqual({
value: 'str',
changed: false,
});
});
});
describe('clampStoredNumber', () => {
it('decodes a number inside the range unchanged', () => {
expect(clampStoredNumber('42')).toBe(42);
expect(clampStoredNumber('-0.5')).toBe(-0.5);
expect(clampStoredNumber(String(MAX_STORED_NUMBER))).toBe(
MAX_STORED_NUMBER,
);
});
it('clamps a stored number past the range instead of decoding a BigInt', () => {
expect(clampStoredNumber('18014398509481982')).toBe(MAX_STORED_NUMBER);
expect(clampStoredNumber('-18014398509481982')).toBe(MIN_STORED_NUMBER);
expect(clampStoredNumber('1e400')).toBe(MAX_STORED_NUMBER);
});
});
@@ -0,0 +1,129 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* A stored number keeps its exact value only within the IEEE-754 safe integer
* range; a magnitude past this cannot round-trip, and the document client
* refuses to marshall one rather than write a value that reads back wrong.
*/
export const MAX_STORED_NUMBER = Number.MAX_SAFE_INTEGER;
export const MIN_STORED_NUMBER = Number.MIN_SAFE_INTEGER;
/**
* Marshalling failures a retry can fix by clamping: an out-of-range number, or
* a special numeric value (`NaN`, `±Infinity`) that has no representation at
* all. Matched on the message because they are plain `Error`s thrown while
* encoding the payload — no error name or code separates them from anything
* else, and they never reach the wire, so there is no response to inspect.
*/
const REPAIRABLE_MARSHALL_ERRORS = [
/is greater than Number\.MAX_SAFE_INTEGER/,
/is lesser than Number\.MIN_SAFE_INTEGER/,
/^Special numeric value /,
];
export const isRepairableMarshallError = (error: unknown): boolean => {
const message = (error as Error | undefined)?.message;
if (typeof message !== 'string') return false;
return REPAIRABLE_MARSHALL_ERRORS.some((pattern) => pattern.test(message));
};
/**
* The same bound, applied to what comes back. A stored number past the safe
* range — left by an older write, or reached by a counter incremented up to it
* — is otherwise decoded as a `BigInt`, which no JSON response can carry. This
* hands back the same kind of value a write accepts, and does less work per
* number than the default decoding it replaces.
*/
export const clampStoredNumber = (stored: string): number => {
const num = Number(stored);
if (num > MAX_STORED_NUMBER) return MAX_STORED_NUMBER;
if (num < MIN_STORED_NUMBER) return MIN_STORED_NUMBER;
return num;
};
export interface MarshallRepair {
value: unknown;
/** False when nothing in the payload was out of range — nothing to retry. */
changed: boolean;
}
const unchanged = (value: unknown): MarshallRepair => ({
value,
changed: false,
});
// Matches the shapes the marshaller walks into as a map: object literals and
// null-prototype objects, but not class instances (Date, Buffer, Set, …),
// which it encodes by their own rules.
const isPlainObject = (value: unknown): value is Record<string, unknown> =>
typeof value === 'object' &&
value !== null &&
(value.constructor === Object || value.constructor === undefined);
const repairNumber = (num: number): MarshallRepair => {
// No representation for these; JSON already turns them into null, so a
// caller that got one this far did not send it as JSON.
if (Number.isNaN(num)) return { value: null, changed: true };
if (num > MAX_STORED_NUMBER)
return { value: MAX_STORED_NUMBER, changed: true };
if (num < MIN_STORED_NUMBER)
return { value: MIN_STORED_NUMBER, changed: true };
return unchanged(num);
};
/**
* Clamp every number a payload holds into the range the store accepts,
* returning the original object untouched when nothing needed it — callers hold
* on to the values they pass (caches, retries), so a repair copies rather than
* mutating in place.
*
* Only run after a marshalling failure. Walking a payload this way costs as
* much as marshalling it, which is exactly why writes are not checked up
* front.
*/
export const repairForMarshall = (value: unknown): MarshallRepair => {
if (typeof value === 'number') return repairNumber(value);
if (Array.isArray(value)) {
let changed = false;
const repairedEntries = value.map((entry) => {
const repaired = repairForMarshall(entry);
changed = changed || repaired.changed;
return repaired.value;
});
return changed ? { value: repairedEntries, changed } : unchanged(value);
}
if (isPlainObject(value)) {
let changed = false;
const repairedEntries = Object.entries(value).map(([key, entry]) => {
const repaired = repairForMarshall(entry);
changed = changed || repaired.changed;
return [key, repaired.value] as const;
});
// `fromEntries` defines each property rather than assigning it, so a
// `__proto__` key in the payload stays data here.
return changed
? { value: Object.fromEntries(repairedEntries), changed }
: unchanged(value);
}
return unchanged(value);
};
+168 -5
View File
@@ -54,14 +54,22 @@ const sentMessage = (info: unknown) =>
};
describe('EmailClient — transport lifecycle', () => {
it('reports as configured once a transport is wired up', () => {
it('stops sending once the transport is shut down', async () => {
const client = startClient();
expect(client.isConfigured).toBe(true);
expect(
await client.sendRaw({ to: '[email protected]', subject: 'up', text: 'x' }),
).not.toBeNull();
client.onServerShutdown();
expect(client.isConfigured).toBe(false);
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
expect(
await client.sendRaw({ to: '[email protected]', subject: 'down', text: 'x' }),
).toBeNull();
warn.mockRestore();
});
it('warns and stays unconfigured when no transport is set', () => {
it('warns at boot when no transport is set', () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
const client = new EmailClient({
port: 0,
@@ -69,7 +77,6 @@ describe('EmailClient — transport lifecycle', () => {
} as unknown as IConfig);
client.onServerStart();
expect(client.isConfigured).toBe(false);
expect(warn).toHaveBeenCalledWith(
expect.stringContaining('no email transport configured'),
);
@@ -179,6 +186,29 @@ describe('EmailClient — template rendering', () => {
expect(captured.html).toContain('<strong>424242</strong>');
});
it('does not HTML-escape values in the plain-text subject header', async () => {
const client = startClient();
const captured: { subject?: string } = {};
vi.spyOn(client, 'sendRaw').mockImplementation(async (options) => {
captured.subject = options.subject;
return null;
});
await client.send('[email protected]', 'app-user-feedback', {
owner_username: 'dev',
sender_username: 'user',
sender_email: null,
app_title: "Bob's App & Games",
app_name: 'bobs-app',
app_link: 'https://puter.example/app/bobs-app',
message: 'hi',
});
// A subject is not HTML — entities would render literally in the
// recipient's mail client.
expect(captured.subject).toBe("New user feedback for Bob's App & Games");
});
it('escapes html and converts newlines in nl2br values', async () => {
const client = startClient();
let html = '';
@@ -215,6 +245,139 @@ describe('EmailClient — template rendering', () => {
});
});
describe('EmailClient — share notification templates', () => {
/** Captures both parts of a send without touching the transport. */
const renderShare = async (
template: 'file_shared_with_you' | 'file_shared_invite',
values: Record<string, unknown>,
) => {
const client = startClient();
const captured: { html: string; text: string } = { html: '', text: '' };
vi.spyOn(client, 'sendRaw').mockImplementation(async (options) => {
captured.html = options.html ?? '';
captured.text = options.text ?? '';
return null;
});
await client.send('[email protected]', template, values);
return captured;
};
const HOLDER = {
recipient: 'alice',
subject_line: 'bob shared notes.md with you',
shares: [
{ sender: 'bob', what: 'notes.md' },
{ sender: 'carol', what: '3 items — a.txt, b.txt, +1 more' },
],
link: 'https://puter.test',
unsubscribe_uuid: null,
};
it('sends a plain-text alternative beside the html', async () => {
const { html, text } = await renderShare(
'file_shared_with_you',
HOLDER,
);
expect(html).toContain('<!DOCTYPE html>');
// Every sender reaches both parts, so a text-only client loses nothing.
for (const part of [html, text]) {
expect(part).toContain('bob');
expect(part).toContain('notes.md');
expect(part).toContain('carol');
expect(part).toContain('+1 more');
}
expect(text).not.toContain('<');
});
it('escapes item names in the html and leaves them raw in the text', async () => {
const { html, text } = await renderShare('file_shared_with_you', {
...HOLDER,
shares: [{ sender: 'bob', what: 'r&d "notes".md' }],
});
expect(html).toContain('r&amp;d &quot;notes&quot;.md');
expect(text).toContain('r&d "notes".md');
});
it('renders as a responsive single column with no remote assets', async () => {
const { html } = await renderShare('file_shared_with_you', HOLDER);
expect(html).toContain(
'<meta name="viewport" content="width=device-width, initial-scale=1" />',
);
expect(html).toContain('@media only screen and (max-width: 600px)');
expect(html).toContain('@media (prefers-color-scheme: dark)');
expect(html).toContain('max-width: 600px');
// Images are the one thing a client can refuse to load, so the design
// does without them — the layout can't depend on a blocked asset.
expect(html).not.toContain('<img');
// The colors have to survive a client that drops the stylesheet.
expect(html).toContain('style="background-color: #ffffff;');
});
it('opens with a preview line that is hidden in the body', async () => {
const { html } = await renderShare('file_shared_with_you', HOLDER);
const preheader = html.indexOf('Waiting for you under Shared with me.');
expect(preheader).toBeGreaterThan(-1);
expect(html.slice(0, preheader)).toContain('mso-hide: all');
expect(preheader).toBeLessThan(html.indexOf('Hi alice,'));
});
it('points the call to action at the app and nowhere else', async () => {
const { html, text } = await renderShare(
'file_shared_with_you',
HOLDER,
);
expect(html).toContain('href="https://puter.test"');
expect(html).toContain('>Open Puter</a>');
expect(text).toContain('Open Puter: https://puter.test');
});
it('separates senders without a rule above the first', async () => {
const { html } = await renderShare('file_shared_with_you', HOLDER);
// One rule between two senders, none leading the list.
expect(html.split('border-top: 1px solid').length - 1).toBe(1);
});
it('offers the unsubscribe link only to a recipient who has an account', async () => {
const withAccount = await renderShare('file_shared_with_you', {
...HOLDER,
unsubscribe_uuid: 'a-uuid',
});
expect(withAccount.html).toContain(
'href="https://puter.test/unsubscribe?user_uuid=a-uuid"',
);
expect(withAccount.text).toContain(
'https://puter.test/unsubscribe?user_uuid=a-uuid',
);
const anonymous = await renderShare('file_shared_with_you', HOLDER);
expect(anonymous.html).not.toContain('/unsubscribe');
expect(anonymous.text).not.toContain('/unsubscribe');
});
it('tells an invited address what to do with it', async () => {
const { html, text } = await renderShare('file_shared_invite', {
email: '[email protected]',
subject_line: 'bob shared notes.md with you on Puter',
shares: [{ sender: 'bob', what: 'notes.md' }],
link: 'https://puter.test',
});
for (const part of [html, text]) {
expect(part).toContain('[email protected]');
expect(part).toContain('Create your free account');
// An invite has no account to unsubscribe from.
expect(part).not.toContain('/unsubscribe');
}
expect(html).toContain('href="https://puter.test"');
});
});
describe('EmailClient.clean', () => {
const clean = (email: string) => startClient().clean(email);
+38 -10
View File
@@ -22,7 +22,11 @@ import handlebars, { template } from 'handlebars';
import nodemailer from 'nodemailer';
import type { IConfig } from '../../types';
import { PuterClient } from '../types';
import { EMAIL_TEMPLATES, type EmailTemplateName } from './templates';
import {
EMAIL_TEMPLATES,
type EmailTemplate,
type EmailTemplateName,
} from './templates';
/** Attachment shape passed through to the underlying transport. */
export interface EmailAttachment {
@@ -64,6 +68,7 @@ export type EmailValidator = (email: string) => Promise<boolean> | boolean;
interface CompiledTemplate {
subject: ReturnType<typeof template>;
html: ReturnType<typeof template>;
text?: ReturnType<typeof template>;
}
// -- Clean-email rules ------------------------------------------------
@@ -152,11 +157,16 @@ export class EmailClient extends PuterClient {
// -- Public API: sending ------------------------------------------
/** Render a template and send it to `to`. */
/**
* Render a template and send it to `to`. `options.replyTo` sets the
* Reply-To header (e.g. so a recipient can respond to the originator of the
* message rather than the no-reply From address).
*/
async send<T extends EmailTemplateName>(
to: string,
template: T,
values: Record<string, unknown> = {},
options: { replyTo?: string } = {},
): Promise<void> {
const compiled = this.compiledTemplates[template];
if (!compiled) {
@@ -168,6 +178,8 @@ export class EmailClient extends PuterClient {
to,
subject: compiled.subject(values),
html: compiled.html(values),
...(compiled.text ? { text: compiled.text(values) } : {}),
...(options.replyTo ? { replyTo: options.replyTo } : {}),
});
}
@@ -177,7 +189,8 @@ export class EmailClient extends PuterClient {
*
* Returns the transport's send result, or `null` when no transport is
* configured (the send is a no-op in that case — callers that must not
* silently drop mail should check `isConfigured` first).
* silently drop mail should check `config.email` before composing, or the
* `null` return after).
*/
async sendRaw(options: SendMailOptions) {
if (!this.transport) {
@@ -193,11 +206,6 @@ export class EmailClient extends PuterClient {
});
}
/** Whether an SMTP transport is configured (sends are no-ops otherwise). */
get isConfigured(): boolean {
return this.transport !== null;
}
// -- Public API: clean / validate ---------------------------------
/**
@@ -283,10 +291,30 @@ export class EmailClient extends PuterClient {
}
private compileTemplates(): void {
for (const [name, template] of Object.entries(EMAIL_TEMPLATES)) {
// Widened to the interface: the literal map keeps a distinct type per
// template, and only some of them carry a `text` part.
const templates: Record<EmailTemplateName, EmailTemplate> =
EMAIL_TEMPLATES;
for (const [name, template] of Object.entries(templates)) {
this.compiledTemplates[name as EmailTemplateName] = {
subject: handlebars.compile(template.subject),
// Subjects are plain-text headers: HTML-escaping would put
// literal entities in front of the recipient (&amp; etc.).
// Header safety is handled elsewhere — the transport encodes
// newlines, and free-form values (e.g. app_title) collapse
// whitespace upstream.
subject: handlebars.compile(template.subject, {
noEscape: true,
}),
html: handlebars.compile(dedent(template.html)),
// Same reasoning as the subject: a text part is not HTML, so
// escaping would show entities to the reader.
...(template.text
? {
text: handlebars.compile(dedent(template.text), {
noEscape: true,
}),
}
: {}),
};
}
}
+272 -24
View File
@@ -29,8 +29,184 @@
export interface EmailTemplate {
subject: string;
html: string;
/**
* Optional plain-text alternative. Where a template has one it goes out
* beside the HTML as multipart/alternative, which is what plain-text
* clients, screen readers and spam scoring all prefer to a machine
* down-conversion of the markup.
*/
text?: string;
}
// -- Shared layout ----------------------------------------------------
/**
* Colors, in one place so the two share templates can't drift apart. Every one
* of them is also written inline further down: clients that drop `<style>`
* (Gmail reading a non-Gmail account, Outlook.com) would otherwise render an
* unstyled page, so the stylesheet only carries what inline CSS can't express —
* the small-screen and dark-mode overrides.
*
* `ACCENT` is the GUI's primary blue darkened until white text on it clears
* 4.5:1, since a button label is small text however bold it is.
*/
const INK = '#101828';
const INK_SOFT = '#5b6472';
const INK_FAINT = '#667085';
const PAGE_BG = '#f2f4f7';
const CARD_BG = '#ffffff';
const CARD_BORDER = '#e4e7ec';
const RULE = '#e6e9ee';
const PANEL_BG = '#f8f9fb';
const PANEL_BORDER = '#eceff3';
const ACCENT = '#0a76cc';
const FONT =
"-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,'Helvetica Neue',Arial,sans-serif";
/** Pads the preview line so body text can't leak into it. */
const PREHEADER_FILL = '&#847;&zwnj;&nbsp;'.repeat(60);
/**
* The document both share notifications render into: one 600px column that goes
* full width on a phone, laid out with tables and `bgcolor` because Outlook
* renders through Word and ignores anything more modern.
*
* `preheader` is the line the inbox shows beside the subject — it should add to
* the subject, not repeat it. `content` is the card's rows; `footer` sits below
* the card, where the reason-for-receiving and unsubscribe belong.
*/
const shareEmailLayout = (parts: {
preheader: string;
content: string;
footer: string;
}): string => `
<!DOCTYPE html>
<html lang="en" dir="ltr" xmlns:o="urn:schemas-microsoft-com:office:office">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="x-apple-disable-message-reformatting" />
<meta name="color-scheme" content="light dark" />
<meta name="supported-color-schemes" content="light dark" />
<title>{{subject_line}}</title>
<!--[if mso]>
<noscript><xml><o:OfficeDocumentSettings><o:PixelsPerInch>96</o:PixelsPerInch></o:OfficeDocumentSettings></xml></noscript>
<![endif]-->
<style>
body { margin: 0; padding: 0; width: 100% !important; -webkit-text-size-adjust: 100%; -ms-text-size-adjust: 100%; }
table { border-spacing: 0; mso-table-lspace: 0pt; mso-table-rspace: 0pt; }
td { mso-line-height-rule: exactly; }
a[x-apple-data-detectors] { color: inherit !important; text-decoration: none !important; font-size: inherit !important; font-family: inherit !important; font-weight: inherit !important; line-height: inherit !important; }
@media only screen and (max-width: 600px) {
.col { width: 100% !important; }
.pad { padding-left: 24px !important; padding-right: 24px !important; }
.pad-y { padding-top: 28px !important; padding-bottom: 28px !important; }
.h1 { font-size: 22px !important; line-height: 30px !important; }
.btn { width: 100% !important; }
}
@media (prefers-color-scheme: dark) {
.page { background-color: #0e1116 !important; }
.card { background-color: #171b22 !important; border-color: #272d38 !important; }
.ink { color: #e9ebef !important; }
.ink-soft { color: #9aa3b2 !important; }
.ink-soft a { color: #9aa3b2 !important; }
.rule { border-color: #272d38 !important; }
.panel { background-color: #1d222a !important; border-color: #2b323d !important; }
.mark { color: #4da3f2 !important; }
}
</style>
</head>
<body class="page" style="margin: 0; padding: 0; width: 100%; background-color: ${PAGE_BG};">
<div style="display: none; font-size: 1px; line-height: 1px; max-height: 0; max-width: 0; opacity: 0; overflow: hidden; mso-hide: all;">${parts.preheader}${PREHEADER_FILL}</div>
<div role="article" aria-roledescription="email" lang="en">
<table role="presentation" class="page" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%; background-color: ${PAGE_BG};">
<tr>
<td align="center" style="padding: 32px 12px;">
<!--[if mso]><table role="presentation" width="600" cellpadding="0" cellspacing="0" border="0"><tr><td><![endif]-->
<table role="presentation" class="col" width="600" cellpadding="0" cellspacing="0" border="0" style="width: 100%; max-width: 600px; text-align: left;">
<tr>
<td class="pad mark" style="padding: 0 4px 14px; font-family: ${FONT}; font-size: 15px; line-height: 20px; font-weight: 700; letter-spacing: 0.01em; color: ${ACCENT};">Puter</td>
</tr>
<tr>
<td class="card" style="background-color: ${CARD_BG}; border: 1px solid ${CARD_BORDER}; border-radius: 14px;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">
<tr>
<td class="pad pad-y" style="padding: 36px 36px 34px;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">${parts.content}
</table>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td class="pad ink-soft" style="padding: 20px 4px 0; font-family: ${FONT}; font-size: 13px; line-height: 20px; color: ${INK_FAINT};">${parts.footer}
</td>
</tr>
</table>
<!--[if mso]></td></tr></table><![endif]-->
</td>
</tr>
</table>
</div>
</body>
</html>
`;
/** The greeting line above the heading. */
const greetingRow = (text: string): string => `
<tr>
<td class="ink-soft" style="padding: 0 0 6px; font-family: ${FONT}; font-size: 15px; line-height: 22px; color: ${INK_SOFT};">${text}</td>
</tr>`;
const headingRow = (text: string): string => `
<tr>
<td class="ink h1" style="padding: 0 0 18px; font-family: ${FONT}; font-size: 24px; line-height: 32px; font-weight: 700; letter-spacing: -0.01em; color: ${INK};">${text}</td>
</tr>`;
/**
* One row per sender, hairline-separated. The wording comes pre-composed from
* the digest (`digestLines`), so the list stays a list however many senders and
* items fold into it.
*/
const SHARE_LIST_ROW = `
<tr>
<td class="panel" style="padding: 6px 18px; background-color: ${PANEL_BG}; border: 1px solid ${PANEL_BORDER}; border-radius: 10px;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">
{{#each shares}}
<tr>
<td class="ink rule" style="padding: 12px 0;{{#unless @first}} border-top: 1px solid ${RULE};{{/unless}} font-family: ${FONT}; font-size: 16px; line-height: 24px; color: ${INK};"><strong style="font-weight: 600;">{{this.sender}}</strong> shared {{this.what}}</td>
</tr>
{{/each}}
</table>
</td>
</tr>`;
/** A body paragraph. `padding` lets a caller tune the rhythm around it. */
const textRow = (text: string, padding = '18px 0 0'): string => `
<tr>
<td class="ink-soft" style="padding: ${padding}; font-family: ${FONT}; font-size: 15px; line-height: 23px; color: ${INK_SOFT};">${text}</td>
</tr>`;
/**
* The call to action. Padding sits on the cell and the color on `bgcolor` so
* Outlook still draws a real button (square-cornered, which is fine); the table
* goes full width under 600px so the tap target spans the card.
*/
const buttonRow = (label: string): string => `
<tr>
<td style="padding: 26px 0 0;">
<table role="presentation" class="btn" cellpadding="0" cellspacing="0" border="0" style="border-collapse: separate;">
<tr>
<td align="center" bgcolor="${ACCENT}" style="background-color: ${ACCENT}; border-radius: 10px; padding: 14px 26px;">
<a href="{{link}}" style="display: inline-block; font-family: ${FONT}; font-size: 16px; line-height: 20px; font-weight: 600; color: #ffffff; text-decoration: none;">${label}</a>
</td>
</tr>
</table>
</td>
</tr>`;
export const EMAIL_TEMPLATES = {
'approved-for-listing': {
subject: '🎉 Your app has been approved for listing!',
@@ -75,6 +251,25 @@ Please update <a href="https://puter.com/app/{{app_name}}">{{app_title}}</a>.
<blockquote>{{nl2br message}}</blockquote>
<p>Best,<br />
The Puter Team
</p>
`,
},
'app-user-feedback': {
subject: 'New user feedback for {{app_title}}',
html: `
<p>Hi{{#if owner_username}} {{owner_username}}{{/if}},</p>
<p>
<strong>{{sender_username}}</strong>{{#if sender_email}} (<a href="mailto:{{sender_email}}">{{sender_email}}</a>){{/if}} sent feedback about <a href="{{app_link}}">{{app_title}}</a>:
</p>
<blockquote>{{{nl2br message}}}</blockquote>
{{#if sender_email}}<p>Just reply to this email to respond to them directly.</p>{{/if}}
<p>
You're receiving this because feedback is enabled for
<a href="{{app_link}}">{{app_title}}</a> — manage it in the
<a href="{{dev_center_link}}">Dev Center</a> under your app's settings.
</p>
<p>Best,<br />
The Puter Team
</p>
`,
},
@@ -164,32 +359,85 @@ immediately</p>
<p>Puter</p>
`,
},
share_by_username: {
subject: 'Puter share from {{susername}}',
html: `
<p>Hi there {{rusername}},</p>
<p>You've received a share from {{susername}} on Puter.</p>
<p>Go to puter.com to check it out.</p>
{{#if message}}
<p>The following message was included:</p>
<blockquote>{{message}}</blockquote>
{{/if}}
<p>Sincerely,</p>
<p>Puter</p>
/**
* A digest: shares to one recipient are held briefly and merged, so
* `shares` may carry several senders. The subject is composed by the
* service (see `digestSubject`), which owns the grouped wording.
*/
file_shared_with_you: {
subject: '{{subject_line}}',
html: shareEmailLayout({
preheader: 'Waiting for you under Shared with me.',
content:
greetingRow('Hi{{#if recipient}} {{recipient}}{{/if}},') +
headingRow('Shared with you') +
SHARE_LIST_ROW +
buttonRow('Open Puter') +
textRow(
'Shared items live under <span style="font-weight: 600;">Shared</span> in your files.',
'24px 0 0',
),
footer: `You're receiving this because someone shared with your Puter account.{{#if unsubscribe_uuid}}
<br /><a href="{{link}}/unsubscribe?user_uuid={{unsubscribe_uuid}}" style="color: inherit; text-decoration: underline;">Unsubscribe from notification emails</a>{{/if}}`,
}),
text: `
Hi{{#if recipient}} {{recipient}}{{/if}},
Shared with you on Puter:
{{#each shares}}
- {{this.sender}} shared {{this.what}}
{{/each}}
Open Puter: {{link}}
Shared items live under "Shared" in your files.
--
You're receiving this because someone shared with your Puter account.
{{#if unsubscribe_uuid}}Unsubscribe from notification emails:
{{link}}/unsubscribe?user_uuid={{unsubscribe_uuid}}{{/if}}
`,
},
share_by_email: {
subject: 'share by email',
html: `
<p>Hi there,</p>
<p>You've received a share from {{sender_name}} on Puter:</p>
<p><a href="{{link}}">{{link}}</a></p>
{{#if message}}
<p>The following message was included:</p>
<blockquote>{{message}}</blockquote>
{{/if}}
<p>Sincerely,</p>
<p>Puter</p>
// The only way to reach someone with no account. Same digest shape.
file_shared_invite: {
subject: '{{subject_line}}',
html: shareEmailLayout({
preheader: 'Claim it with a free Puter account.',
content:
greetingRow('Hi there,') +
headingRow('Shared with you on Puter') +
SHARE_LIST_ROW +
textRow(
'There\'s no Puter account for <span style="font-weight: 600;">{{email}}</span> yet. Create one with this address, confirm it, and everything above will be waiting for you. It\'s free and takes about a minute.',
) +
buttonRow('Create your free account') +
textRow(
'Already on Puter? Add {{email}} to your account and confirm it to get the same access.',
'24px 0 0',
),
footer: `You're receiving this because someone shared with {{email}}. Nothing is shared until the address is confirmed, so you can ignore this email and nothing happens.`,
}),
text: `
Hi there,
Shared with you on Puter:
{{#each shares}}
- {{this.sender}} shared {{this.what}}
{{/each}}
There's no Puter account for {{email}} yet. Create one with this
address, confirm it, and everything above will be waiting for you.
It's free and takes about a minute.
Create your free account: {{link}}
Already on Puter? Add {{email}} to your account and confirm it to get
the same access.
--
You're receiving this because someone shared with {{email}}. Nothing is
shared until the address is confirmed, so you can ignore this email
and nothing happens.
`,
},
} satisfies Record<string, EmailTemplate>;
@@ -214,6 +214,38 @@ describe('EventClient', () => {
});
});
describe('hasListeners', () => {
it('is false for a key nobody subscribed to', () => {
expect(target.hasListeners(key)).toBe(false);
});
it('is true for an exact subscriber', () => {
target.on(key, vi.fn());
expect(target.hasListeners(key)).toBe(true);
});
it('is true when only a wildcard prefix matches', () => {
target.on(`${key}.*`, vi.fn());
expect(target.hasListeners(`${key}.child.deep`)).toBe(true);
});
it('is true for an extension-registered listener', () => {
extensionStore.events[key] = [vi.fn()];
try {
expect(target.hasListeners(key)).toBe(true);
} finally {
delete extensionStore.events[key];
}
});
it('goes back to false once the last listener is removed', () => {
const listener = vi.fn();
target.on(key, listener);
target.off(key, listener);
expect(target.hasListeners(key)).toBe(false);
});
});
describe('extension-registered listeners', () => {
afterEach(() => {
delete extensionStore.events[key];
+21
View File
@@ -129,6 +129,27 @@ export class EventClient extends PuterClient {
});
}
/**
* Whether anything would run if `key` were emitted, wildcards included.
*
* For emitters whose work is only worth doing when someone is listening —
* gathering the payload costs a round trip, say. Emitting into the void is
* otherwise perfectly cheap and doesn't need this.
*/
hasListeners<T extends keyof EventMap>(key: T): boolean {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey = (
i === parts.length - 1
? key
: `${parts.slice(0, i + 1).join('.')}.*`
) as ListenKey;
if (this.#eventListeners[matchKey]?.length) return true;
if (extensionStore.events[matchKey]?.length) return true;
}
return false;
}
/**
* Subscribe to an event by exact key OR a wildcard prefix.
*
+137 -8
View File
@@ -20,6 +20,7 @@ import type {
Response as ExpressResponse,
} from 'express';
import { Actor } from '../../core';
import type { UsageInput } from '../../services/metering/types';
import { FSEntry } from '../../stores/fs/FSEntry';
// GUI write events spread an entry plus per-event metadata into `response`.
@@ -31,6 +32,24 @@ type GuiEvent<R = Record<string, unknown>> = {
response: R;
};
/**
* Extension-augmentable half of {@link EventMap}. Extensions that emit their own
* events declare the payload here by declaration merging, so both the emitter
* and every listener are typed against the same shape:
*
* declare module '@heyputer/backend/clients/event/types' {
* interface IExtensionEventMap {
* 'my.thing.happened': { thingId: string };
* }
* }
*
* Deliberately member-less and index-signature-free: an index signature here
* would widen `keyof EventMap` to `string` and silently disable key checking on
* every `emit` in the tree.
*/
// eslint-disable-next-line @typescript-eslint/no-empty-object-type
export interface IExtensionEventMap {}
export type EventMap = {
// ---- Server lifecycle ----
serverStart: Record<string, never>;
@@ -141,6 +160,19 @@ export type EventMap = {
'puter.signup.validate': {
allow: boolean;
email?: string;
/**
* The address in the same canonical form the `email.validate` hook was
* given (`cleanEmail`), so a handler can correlate the two hooks on one
* key. Aliases (`[email protected]`, `[email protected]`) differ from
* `email` here.
*/
clean_email?: string;
/**
* True for a temp (frictionless, no-password) signup. Those carry a
* synthetic `<username>@gmail.com` and never reach email validation, so
* a handler must not draw conclusions from `email`.
*/
is_temp?: boolean;
ip?: string | null;
source?: 'oidc';
req?: unknown;
@@ -289,7 +321,17 @@ export type EventMap = {
sourceObjectKey: string;
copyObjectKey: string;
};
'fs.move.node': { node: FSEntry; fromPath: string; toPath: string };
/**
* `fromUserId` is the owner the node had before the move. A move into
* someone else's tree re-owns the row, and `node` already carries the new
* owner — listeners that care about the hand-over need the old one.
*/
'fs.move.node': {
node: FSEntry;
fromPath: string;
toPath: string;
fromUserId?: number;
};
'fs.remove.node': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.write.file': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.storage.upload-progress': {
@@ -305,13 +347,81 @@ export type EventMap = {
};
'storage.quota.bonus': { userId: number; extra: number };
// ---- Metering ----
// Recurring charges are pure mechanism here: the metering service knows
// when to ask (once per user per month, the first time that month's usage
// record is touched) and how to record the answer, and an extension
// decides what the account owes. Listeners push onto `charges`; the
// service applies them as ordinary usage once every listener has run.
// Nothing listening → no charges and no claim is ever taken.
//
// Emitted after the claim is settled, so a listener that calls back into
// metering can't re-trigger it.
'metering.monthly.charges': {
/**
* User-scoped, with no app on it: the account is what recurs, and the
* app that happened to trigger the month's first call has nothing to do
* with what is owed. Price against what the user owns.
*/
actor: Actor;
/** The month being charged for, `YYYY-MM` in UTC. */
month: string;
/**
* Push what the user owes here. Every listener's charges are merged
* into one amount map and recorded as a single increment.
*/
charges: UsageInput[];
};
// ---- Workers ----
// Only a genuinely new worker. Redeploying an existing name updates its
// row instead, and never reaches here — so a listener that prices this
// is pricing workers that came into existence, not deploys.
'worker.create': { actor: Actor; workerName: string };
// The same announcement for worker code that lives inside a hosted site
// and is deployed on demand rather than by an explicit call. Such a worker
// has no row of ours to key on, so what stands in for "genuinely new" is
// the absence of any prior record that it ran — which is why it is
// announced separately, and identified by where its source lives rather
// than by a worker name.
'worker.dynamic.create': {
actor: Actor;
subdomain: string;
worker: string;
};
// ---- Outer / GUI broadcast ----
'outer.cacheUpdate': {
cacheKey: string[];
data?: unknown;
ttlSeconds?: number;
};
/**
* Permission cache generations were bumped, so peer regions must bump their
* own — the counter is per-cluster, so a local bump says nothing to them.
* Carries the actors, not the values: the numbers only have to change.
*/
'outer.permission.generationBumped': { actorUids: string[] };
/**
* Flat permission entries were deleted. Grant-path flat entries carry no
* expiry, so without this a revoke never lands in a peer region whose KV
* table isn't replicated. Revoke-only: a grant that fails to replicate just
* denies there, which is the safe direction.
*/
'outer.permission.flatInvalidated': {
entries: Array<{ holderUserId: number; permission: string }>;
};
'outer.fs.write-hash': { hash: string; uuid: string };
/**
* Cache keys the KV read cache must stop serving, because the entries
* behind them were just written somewhere else.
*
* `outer.*` rather than `outer.pubsub.*` on purpose: the cache lives in the
* Redis a cluster shares, so one node applying the invalidation covers the
* whole cluster — fanning it out to siblings would just repeat the write.
*/
'outer.kv.cacheInvalidated': { cacheKeys: string[] };
'outer.gui.item.added': GuiEvent;
'outer.gui.item.updated': GuiEvent;
'outer.gui.item.moved': GuiEvent;
@@ -325,8 +435,10 @@ export type EventMap = {
}>;
// ---- Subdomains ----
'subdomain.delete': { subdomain: string };
'subdomain.update': { subdomain: string };
// `uid` is the row uuid — for `delete` it is the only surviving handle a
// listener can use to find state keyed to the row after it is gone.
'subdomain.delete': { subdomain: string; uid?: string };
'subdomain.update': { subdomain: string; uid?: string };
'site.htmlServed': {
subdomain: string;
entry: unknown;
@@ -402,7 +514,23 @@ export type EventMap = {
[K in `route.${string}`]: RouteLifecycleEvent;
} & {
[K in `pubsub.login.${string}`]: { authtoken: string };
};
} & {
/**
* A user's subscription now resolves to a different policy. Carried on the
* `outer.pubsub.*` channel so it reaches sibling nodes and peer clusters,
* not just the one that handled the change — every node caches the resolved
* policy, so a purchase is only live once they have all dropped theirs.
*/
'outer.pubsub.metering.subscription-changed': { userUuid: string };
/**
* A user's purchased credit balance changed. Separate from a policy change
* because it moves the other half of the same budget, and carried on the
* `outer.pubsub.*` channel for the same reason: every node caches whether
* an account has budget left, so a top-up only lifts enforcement once they
* have all dropped that answer.
*/
'outer.pubsub.metering.credits-changed': { userUuid: string };
} & IExtensionEventMap;
/**
* Phase of a request/method lifecycle. `reject` is emitted when a `before`
@@ -482,10 +610,11 @@ export type EventKey = keyof EventMap & string;
// Generates a wildcard for every non-final dot-separated prefix of K.
export type WildcardPrefixes<K extends string> =
K extends `${infer Head}.${infer Tail}`
? | `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
?
| `${Head}.*`
| (Tail extends `${string}.${string}`
? `${Head}.${WildcardPrefixes<Tail>}`
: never)
: never;
export type ListenKey = EventKey | WildcardPrefixes<EventKey>;
@@ -30,7 +30,7 @@ describe('PreludeClient', () => {
expect(makeClient().isConfigured()).toBe(false);
});
describe('isCountrySupported (€0.07 cap)', () => {
describe('isCountrySupported (€0.20 cap)', () => {
const client = makeClient('sk_test');
it('allows revenue markets up to the cap (incl. the priciest)', () => {
@@ -40,6 +40,17 @@ describe('PreludeClient', () => {
expect(client.isCountrySupported('us')).toBe(true); // case-insensitive
});
it('allows the markets the old €0.07 cap excluded', () => {
// Raising the cap to €0.20 brought ~89 countries into the phone
// gate. These pin the new range so a future change to the default
// has to say so out loud.
expect(client.isCountrySupported('UA')).toBe(true); // €0.0940
expect(client.isCountrySupported('PH')).toBe(true); // €0.1237
expect(client.isCountrySupported('EG')).toBe(true); // €0.1561
expect(client.isCountrySupported('NG')).toBe(true); // €0.1980
expect(client.isCountrySupported('WS')).toBe(true); // €0.2000, at the cap
});
it('rejects countries above the cap, with no SMS, or unknown', () => {
expect(client.isCountrySupported('PK')).toBe(false); // €0.3548
expect(client.isCountrySupported('ID')).toBe(false); // €0.2430
@@ -63,9 +74,7 @@ describe('PreludeClient', () => {
});
it('createVerification POSTs the phone target + ip signal with bearer auth', async () => {
fetchMock.mockResolvedValue(
okJson({ id: 'vrf_1', status: 'success' }),
);
fetchMock.mockResolvedValue(okJson({ id: 'vrf_1', status: 'success' }));
const client = makeClient('sk_test');
const res = await client.createVerification('+14155550123', {
@@ -81,7 +90,11 @@ describe('PreludeClient', () => {
target: { type: 'phone_number', value: '+14155550123' },
// Defaults to RCS (cheaper); Prelude falls back to SMS. locale is
// hardcoded to en-US so the message text is always English.
options: { code_size: 6, preferred_channel: 'rcs', locale: 'en-US' },
options: {
code_size: 6,
preferred_channel: 'rcs',
locale: 'en-US',
},
signals: { ip: '203.0.113.7' },
});
});
@@ -196,9 +209,9 @@ describe('PreludeClient', () => {
it('throws (does not call fetch) when not configured', async () => {
const client = makeClient();
await expect(
client.createVerification('+14155550123'),
).rejects.toThrow(/not configured/i);
await expect(client.createVerification('+14155550123')).rejects.toThrow(
/not configured/i,
);
expect(fetchMock).not.toHaveBeenCalled();
});
+8 -5
View File
@@ -50,12 +50,15 @@ export type PreludeChannel =
export type PreludeDeliveryChannel = PreludeChannel | 'silent' | 'voice';
/**
* Default per-SMS cost ceiling (EUR). Countries whose Prelude SMS rate exceeds
* this — or that have no SMS channel — are not offered phone verification. The
* cap covers every realistic revenue market (priciest are Germany €0.0598 and
* Saudi Arabia €0.0638) while excluding the expensive, high-fraud long tail.
* Override per-deployment with `config.prelude.maxSmsCostEur`.
* this — or that have no SMS channel — are not offered phone verification.
* Raised from €0.07 to €0.20: the old cap sat only marginally above the
* priciest revenue markets (Germany €0.0598, Saudi Arabia €0.0638), so any rate
* drift silently withdrew the phone gate from a real market — and the abuse
* harness now routes far more legitimate signups into the SMS band, where an
* unavailable gate is a dead end rather than a mild inconvenience. Override
* per-deployment with `config.prelude.maxSmsCostEur`.
*/
const DEFAULT_MAX_SMS_COST_EUR = 0.07;
const DEFAULT_MAX_SMS_COST_EUR = 0.2;
/** Status returned by Prelude when creating/retrying a verification. */
export type PreludeCreateStatus =
+14
View File
@@ -21,6 +21,7 @@ import {
AbortMultipartUploadCommand,
CompleteMultipartUploadCommand,
CreateMultipartUploadCommand,
HeadBucketCommand,
PutObjectCommand,
S3Client as AwsS3Client,
type S3ClientConfig,
@@ -244,6 +245,19 @@ export class S3Client extends PuterClient {
return client;
}
/**
* Cheapest round-trip that proves the object store answers for a bucket: no
* object data, no listing, just a HEAD. Throws on any failure (missing
* bucket, bad credentials, unreachable endpoint) so callers can treat it as
* a liveness probe. Reuses the pooled per-region client.
*/
async headBucket(
bucket = this.config.s3_bucket || LEGACY_STORAGE_BUCKET,
region?: string,
): Promise<void> {
await this.get(region).send(new HeadBucketCommand({ Bucket: bucket }));
}
// ------------------------------------------------------------------
// Legacy storage migration
// ------------------------------------------------------------------
+56 -5
View File
@@ -35,6 +35,39 @@ import DEFAULT_APP_ICON from './default-app-icon.js';
* are just thin shape adapters that translate REST conventions into driver
* calls.
*/
/**
* Desktop boot reads the app list and individual app records repeatedly, so the
* ceiling is set well above normal boot traffic and exists to catch a runaway
* client rather than to pace one.
*
* "Repeatedly" is the operative word: an app record is read on launch, on
* permission checks, and again by anything resolving an app by name, so these
* accumulate against whatever else a session is doing rather than arriving on
* their own. Sized for a session working hard, not for a person clicking.
*/
const APP_READ_LIMIT = {
scope: 'app-read',
limit: 1_800,
window: 60_000,
key: 'user',
};
/**
* Unauthenticated icon serving; no actor to key on, so the bucket is the
* address — and an address is a NAT, a campus or a carrier gateway that can
* hold hundreds of desktops. Each desktop boot pulls the taskbar's icons at
* once, so a single burst from one network is already thousands of requests.
* Responses are publicly cacheable and usually a redirect, so the ceiling is
* not protecting bandwidth; it is there so a client looping on a broken icon
* can't spin unbounded.
*/
const APP_ICON_LIMIT = {
scope: 'app-icon',
limit: 12_000,
window: 60_000,
key: 'ip',
};
export class AppController extends PuterController {
get appStore() {
return this.stores.app;
@@ -100,7 +133,9 @@ export class AppController extends PuterController {
return d;
}
registerRoutes(router) {
registerRoutes(
/** @type {import('../../core/http/PuterRouter').PuterRouter} */ router,
) {
// GET /apps — list apps owned by the current user
router.get(
'/apps',
@@ -108,6 +143,7 @@ export class AppController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: APP_READ_LIMIT,
},
async (req, res) => {
const apps = await this.appDriver.select({
@@ -122,8 +158,16 @@ export class AppController extends PuterController {
'/apps/nameAvailable',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
requireAuth: true,
// Answers "does this name exist?" for any name, so it is a
// name-enumeration oracle however cheap it is to serve.
// Mirrors the `isNameAvailable` budget on AppDriver.
rateLimit: {
scope: 'app-name-available',
limit: 60,
window: 60_000,
key: 'user',
},
},
async (req, res) => {
const name = req.query?.name;
@@ -158,6 +202,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireAuth: true,
rateLimit: APP_READ_LIMIT,
},
async (req, res) => {
const actor = req.actor;
@@ -216,6 +261,7 @@ export class AppController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: APP_READ_LIMIT,
},
async (req, res) => {
const raw = req.params.name;
@@ -289,6 +335,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireAuth: true,
rateLimit: APP_READ_LIMIT,
},
async (req, res) => {
const appList = Array.isArray(req.body) ? req.body : [];
@@ -497,10 +544,14 @@ export class AppController extends PuterController {
// Icons are <img src> targets from the GUI (root) AND resolved via
// api_base_url in taskbar payloads. Register on both so either origin
// works without a cross-subdomain redirect.
router.get('/app-icon/:app_uid', { subdomain: ['api', ''] }, serveIcon);
router.get(
'/app-icon/:app_uid',
{ subdomain: ['api', ''], rateLimit: APP_ICON_LIMIT },
serveIcon,
);
router.get(
'/app-icon/:app_uid/:size',
{ subdomain: ['api', ''] },
{ subdomain: ['api', ''], rateLimit: APP_ICON_LIMIT },
serveIcon,
);
}
@@ -1018,3 +1018,25 @@ describe('AppController GET /app-icon remote icons', () => {
expect(captured.headers['content-type']).toContain('image/svg+xml');
});
});
// ── app-icon rate limit ─────────────────────────────────────────────
describe('AppController app-icon rate limit', () => {
it('sizes the icon bucket for a whole network rather than one desktop', () => {
// Icons are unauthenticated `<img src>` targets, so the only key is
// the address — which one NAT shares across every desktop behind it,
// and each boot pulls the taskbar's icons in a burst.
for (const path of ['/app-icon/:app_uid', '/app-icon/:app_uid/:size']) {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === path,
);
if (!route) throw new Error(`No GET ${path} route`);
expect(route.options.rateLimit).toEqual({
scope: 'app-icon',
limit: 12_000,
window: 60_000,
key: 'ip',
});
}
});
});
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -32,6 +32,19 @@ const ALLOWED_SORT_ORDER = ['asc', 'desc'];
* - User-level: desktop background, taskbar items (UserStore)
* - Folder-level: layout, sort_by/sort_order (fsentries table)
*/
/**
* Desktop preference writes — background, taskbar, layout, sort order. All four
* persist to the user row on every call, and the GUI fires them on direct user
* action, so a per-minute ceiling well above human speed is enough to catch a
* stuck client.
*/
const PREFERENCE_WRITE_LIMIT = {
scope: 'desktop-preference',
limit: 120,
window: 60_000,
key: 'user',
};
export class DesktopController extends PuterController {
constructor(config, clients, stores, services) {
super(config, clients, stores, services);
@@ -53,6 +66,7 @@ export class DesktopController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { url, color, fit } = req.body ?? {};
@@ -108,6 +122,7 @@ export class DesktopController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { items } = req.body ?? {};
@@ -134,6 +149,7 @@ export class DesktopController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { item_uid, item_path, layout } = req.body ?? {};
@@ -161,6 +177,7 @@ export class DesktopController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: PREFERENCE_WRITE_LIMIT,
},
async (req, res) => {
const { item_uid, item_path, sort_by, sort_order } =
@@ -124,9 +124,9 @@ const makeSyntheticDriver = () => ({
const buildController = (driver: ReturnType<typeof makeSyntheticDriver>) => {
// The controller reads `this.services?.permission` only when an actor
// is on the request; otherwise the services bag is unused. The
// alarm client is invoked on the rate-limit / concurrency rejection
// paths, so stub a no-op `create`.
// is on the request; otherwise the services bag is unused. The rejection
// paths no longer alarm, but other paths still reach the client, so keep
// a no-op `create` stubbed.
const clients = { alarm: { create: () => {} } };
return new DriverController(
{} as any,
@@ -409,7 +409,7 @@ describe('DriverController per-method rate limiting', () => {
configureRateLimit({ disabled: false } as never);
});
it('answers 429 and raises a de-duped info alarm once the per-method budget is spent', async () => {
it('answers 429 without alarming once the per-method budget is spent', async () => {
const { handler, alarms, iface } = build({
run: () => ({ ok: true }),
rateLimit: { default: { limit: 1, window: 60_000 } },
@@ -430,9 +430,8 @@ describe('DriverController per-method rate limiting', () => {
legacyCode: 'too_many_requests',
});
expect(alarms).toContainEqual({
id: 'driver_rate_limit_hit:rate-limited-iface:run',
severity: 'info',
});
// Spending your own budget is the limit working as designed, so it
// must not raise anything — the 429 is the whole signal.
expect(alarms).toEqual([]);
});
});
@@ -0,0 +1,164 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/* eslint-disable @typescript-eslint/no-explicit-any */
import { EventEmitter } from 'node:events';
import type { Request, RequestHandler, Response } from 'express';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { runWithContext } from '../../core/context.js';
import { isHttpError } from '../../core/http/HttpError.js';
import { configureRateLimit } from '../../core/http/middleware/rateLimit.js';
import { DriverController } from './DriverController.js';
// The subscription requirement a driver declares per method, exercised
// through `#handleCall` the same way the concurrent test does: a synthetic
// driver carrying only the field under test, plus a stub metering service
// that answers with a fixed plan.
const captureCallHandler = (controller: DriverController): RequestHandler => {
let handler: RequestHandler | undefined;
const fakeRouter = {
post: (path: string, _opts: unknown, h: RequestHandler) => {
if (path === '/call') handler = h;
return fakeRouter;
},
get: () => fakeRouter,
use: () => fakeRouter,
};
controller.registerRoutes(
fakeRouter as unknown as Parameters<
typeof controller.registerRoutes
>[0],
);
if (!handler) throw new Error('failed to capture POST /call handler');
return handler;
};
class StubRes extends EventEmitter {
statusCode = 200;
body: unknown = undefined;
status(code: number) {
this.statusCode = code;
return this;
}
json(body: unknown) {
this.body = body;
return this;
}
setHeader() {
return this;
}
type() {
return this;
}
send() {
return this;
}
}
const makeReq = (method: string, tag: string): Request =>
({
body: { interface: 'test-iface', method },
actor: { user: { uuid: `u-${tag}`, username: 'u' } },
headers: { 'user-agent': tag },
query: {},
ip: '127.0.0.1',
socket: { remoteAddress: '127.0.0.1' },
}) as unknown as Request;
const syntheticDriver = {
driverInterface: 'test-iface',
driverName: 'test-driver',
isDefault: true,
requireSubscription: {
methods: { paid: true, premium: ['business', 'pro'] },
},
onServerStart() {},
onServerPrepareShutdown() {},
onServerShutdown() {},
free: async () => 'free-ok',
paid: async () => 'paid-ok',
premium: async () => 'premium-ok',
};
const buildController = (plan: string | null) => {
const clients = { alarm: { create: () => {} } };
const services = plan
? {
metering: {
getActorSubscription: vi.fn().mockResolvedValue({ id: plan }),
},
}
: {};
return new DriverController(
{} as any,
clients as any,
{} as any,
services as any,
{ syntheticDriver } as any,
);
};
const call = (plan: string | null, method: string, tag: string) => {
const handler = captureCallHandler(buildController(plan));
const res = new StubRes();
return runWithContext({}, () =>
handler(makeReq(method, tag), res as unknown as Response, () => {}),
).then(() => res);
};
const expect402 = (promise: Promise<unknown>) =>
expect(promise).rejects.toSatisfy(
(e) =>
isHttpError(e) &&
(e as { statusCode: number }).statusCode === 402 &&
(e as { legacyCode?: string }).legacyCode ===
'subscription_required',
);
describe('DriverController — per-method subscription requirement', () => {
beforeEach(() => {
configureRateLimit();
});
it('leaves a method that declares nothing open to a free account', async () => {
const res = await call('user_free', 'free', 'open');
expect(res.body).toMatchObject({ success: true, result: 'free-ok' });
});
it('rejects a free account on a method that requires a subscription', async () => {
await expect402(call('user_free', 'paid', 'paid-free'));
});
it('admits a subscriber on that same method', async () => {
const res = await call('basic', 'paid', 'paid-sub');
expect(res.body).toMatchObject({ success: true, result: 'paid-ok' });
});
it('honours a per-method plan allowlist', async () => {
await expect402(call('basic', 'premium', 'premium-basic'));
const res = await call('pro', 'premium', 'premium-pro');
expect(res.body).toMatchObject({ success: true, result: 'premium-ok' });
});
it('enforces nothing on a deployment with no metering service', async () => {
const res = await call(null, 'paid', 'no-metering');
expect(res.body).toMatchObject({ success: true, result: 'paid-ok' });
});
});
@@ -158,11 +158,14 @@ interface MockRes {
sentBody: string | undefined;
contentType: string | undefined;
pipedFrom: Readable | undefined;
listeners: Record<string, Array<() => void>>;
status(code: number): MockRes;
json(body: unknown): MockRes;
setHeader(key: string, value: string): MockRes;
type(t: string): MockRes;
send(body: string): MockRes;
once(event: string, fn: () => void): MockRes;
emit(event: string): void;
}
const makeRes = (): MockRes => {
const res: MockRes = {
@@ -172,6 +175,19 @@ const makeRes = (): MockRes => {
sentBody: undefined,
contentType: undefined,
pipedFrom: undefined,
// `#handleCall` releases a driver's concurrency slot on `finish` /
// `close`, so the stub has to behave like an emitter for any driver
// that declares a `concurrent` policy.
listeners: {},
once(event: string, fn: () => void) {
(this.listeners[event] ??= []).push(fn);
return this;
},
emit(event: string) {
const fns = this.listeners[event] ?? [];
this.listeners[event] = [];
for (const fn of fns) fn();
},
status(code: number) {
this.statusCode = code;
return this;
@@ -196,6 +212,30 @@ const makeRes = (): MockRes => {
return res;
};
/**
* Run the call route and return the rejection, or null when it resolves.
* Lets a test assert on *which* gate rejected without depending on how far
* an admitted call gets afterwards.
*/
const callForError = async (
routes: Captured,
req: Request,
actor: Actor,
): Promise<{ statusCode?: number; legacyCode?: string } | null> => {
try {
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
);
return null;
} catch (e) {
return e as { statusCode?: number; legacyCode?: string };
}
};
const makeReq = (body: Record<string, unknown> = {}, actor?: Actor): Request =>
({
body,
@@ -322,8 +362,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
it('admits a bare session ("root" token) actor on the AI drivers', async () => {
// Privileged ("godmode") apps call AI with the user's own session
// token, so the AI drivers no longer set `noUserSession`: the
// session actor reaches the permission scan (403 forbidden — no
// perms granted) instead of the credential-shape rejection.
// session actor must get past the credential-shape gate.
const actor = await makeUserActor();
const req = makeReq(
{
@@ -333,18 +372,8 @@ describe('DriverController.#handleCall (via captured router)', () => {
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
const err = await callForError(routes, req, actor);
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
});
it('admits app and access-token actors on the AI drivers (they fail later on permission, not credential shape)', async () => {
@@ -369,29 +398,15 @@ describe('DriverController.#handleCall (via captured router)', () => {
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
// Fresh actors hold no service:* perms, so the call still
// 403s — but at the permission scan, proving the
// credential-shape gate let the delegated actor through.
legacyCode: 'forbidden',
});
const err = await callForError(routes, req, actor);
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
}
});
it('admits a user-scoped worker session on the AI drivers', async () => {
// Workers deployed without an app binding authenticate as a user
// actor whose session row is kind='worker' — they must keep their
// AI access (they fail later on permission here, not on
// credential shape).
// AI access rather than be rejected for their credential shape.
const base = await makeUserActor();
const actor: Actor = {
...base,
@@ -405,24 +420,13 @@ describe('DriverController.#handleCall (via captured router)', () => {
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
const err = await callForError(routes, req, actor);
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
});
it('still allows bare session actors on drivers without the noUserSession flag', async () => {
// puter-kvstore doesn't set the flag; the session actor reaches
// the permission scan (403 forbidden — no perms granted) instead
// of being rejected for its credential shape.
// puter-kvstore doesn't set the flag, so the session actor must get
// past the credential-shape gate.
const actor = await makeUserActor();
const req = makeReq(
{
@@ -432,24 +436,24 @@ describe('DriverController.#handleCall (via captured router)', () => {
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'forbidden',
});
const err = await callForError(routes, req, actor);
expect(err?.legacyCode).not.toBe('app_or_api_token_required');
});
it('rejects with 403 when the actor lacks the service permission', async () => {
// A fresh user has no service:* perms → permission.check returns
// false → 403.
const actor = await makeUserActor();
// A scoped access token holds only the permissions written to its
// own row, so it is the actor shape that can still be denied — the
// `driver`/`service` grants every user inherits are resolved from
// the issuer only for full-access tokens.
const base = await makeUserActor();
const actor: Actor = {
...base,
accessToken: {
uid: `tok-${uuidv4()}`,
issuer: base,
fullAccess: false,
},
};
const req = makeReq(
{
interface: 'puter-kvstore',
@@ -458,15 +462,8 @@ describe('DriverController.#handleCall (via captured router)', () => {
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({ statusCode: 403 });
const err = await callForError(routes, req, actor);
expect(err).toMatchObject({ statusCode: 403 });
});
it('returns the wrapped {success, result, service} envelope on a successful call', async () => {
@@ -36,7 +36,9 @@ import {
resolveDriverMeta,
resolveDriverMethodConcurrent,
resolveDriverMethodRateLimit,
resolveDriverMethodRequireSubscription,
} from '../../drivers/meta.js';
import { assertActorHasSubscription } from '../../services/metering/enforcement.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import { PermissionUtil } from '../../services/permission/permissionUtil.js';
import type { WithLifecycle } from '../../types';
@@ -45,6 +47,27 @@ import { PuterController } from '../types.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
/**
* Coarse envelope over the whole `/call` surface, so that spreading calls
* across many interfaces can't dodge every individual bucket. Per-driver limits
* are what actually shape traffic.
*
* "Coarse" is a constraint, not a description: for this to be an envelope it
* has to sit _above_ every per-driver budget, or it silently becomes the real
* limit for the widest ones and overrides the tier policy they declare.
* `driverPolicies.test.ts` asserts that ordering against every registered
* driver, so raising a driver's budget past this number fails there rather than
* in production. The headroom above the widest driver (notifications, at
* 3000/30s) is what leaves room for one caller to be busy on two interfaces at
* once.
*/
export const DRIVERS_CALL_LIMIT = {
scope: 'drivers-call',
limit: 8000,
window: 60_000,
key: 'user' as const,
};
// Every driver call is timed here already, for the lifecycle events below.
// Recording the same number as a histogram makes the per-interface latency
// distribution available downstream; which interfaces are worth keeping is a
@@ -180,12 +203,26 @@ export class DriverController extends PuterController {
registerRoutes(router: PuterRouter): void {
router.post(
'/call',
{ subdomain: 'api', requireAuth: true },
{
subdomain: 'api',
requireAuth: true,
rateLimit: DRIVERS_CALL_LIMIT,
},
this.#handleCall,
);
router.get(
'/list-interfaces',
{ subdomain: 'api', requireAuth: true },
{
subdomain: 'api',
requireAuth: true,
// Static introspection output, read once at boot.
rateLimit: {
scope: 'drivers-list-interfaces',
limit: 60,
window: 60_000,
key: 'user',
},
},
this.#handleListInterfaces,
);
}
@@ -262,8 +299,7 @@ export class DriverController extends PuterController {
if (req.actor) {
const permService = this.services.permission as unknown as
| PermissionService
| undefined;
PermissionService | undefined;
if (permService) {
// Build via PermissionUtil.join so any `:` in a driver or
// interface name is escaped — raw interpolation would let a
@@ -291,6 +327,25 @@ export class DriverController extends PuterController {
}
}
// Subscriber-only methods. Declared per-driver
// (`@Driver({ requireSubscription })`) because `/drivers/call` is a
// single route and a route option would apply to every driver at once.
// Checked before the rate limit — the same order the route chain uses
// — so a caller whose plan never included the method is told that
// rather than spending a bucket on it.
const subscriptionRequirement = resolveDriverMethodRequireSubscription(
driverMeta?.requireSubscription,
method,
);
if (subscriptionRequirement !== undefined) {
await assertActorHasSubscription(
this.services.metering,
req.actor,
subscriptionRequirement,
this.config,
);
}
// Per-method rate-limit and concurrent specs both live on the
// driver's resolved meta (set by `@Driver({ rateLimit, concurrent })`
// or imperative fields). Rate-limit is single-shot; concurrent
@@ -305,19 +360,8 @@ export class DriverController extends PuterController {
if (
!(await checkDriverRateLimit(req, ifaceName, method, rateLimitSpec))
) {
// De-dupe on (iface, method) so a hot loop across many users
// aggregates as occurrences on a single low-severity alarm
// instead of fanning out one per user.
this.clients.alarm.create(
`driver_rate_limit_hit:${ifaceName}:${method}`,
`Driver rate limit hit on ${ifaceName}:${method}`,
{
iface: ifaceName,
method,
userUuid: req.actor?.user?.uuid,
},
'info',
);
// Deliberately unalarmed: a caller spending its own budget is
// the limit working, not an incident. The 429 is the signal.
throw new HttpError(429, 'Too many requests.', {
legacyCode: 'too_many_requests',
});
@@ -340,16 +384,8 @@ export class DriverController extends PuterController {
concurrentSpec,
);
if (!handle.ok) {
this.clients.alarm.create(
`driver_concurrent_limit_hit:${ifaceName}:${method}`,
`Driver concurrency limit hit on ${ifaceName}:${method}`,
{
iface: ifaceName,
method,
userUuid: req.actor?.user?.uuid,
},
'info',
);
// Unalarmed for the same reason as the rate-limit rejection
// above: hitting a declared cap is the cap doing its job.
throw new HttpError(429, 'Too many concurrent requests.', {
legacyCode: 'too_many_requests',
});
@@ -0,0 +1,600 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, RequestHandler, Response } from 'express';
import {
afterAll,
afterEach,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { AppFeedbackService } from '../../services/feedback/AppFeedbackService.js';
import { setupTestServer } from '../../testUtil.js';
// Boots one real PuterServer (in-memory sqlite + mocked externals) and
// registers AppFeedbackController's decorated routes onto a fresh
// PuterRouter. Tests drive the captured handlers with stub req/res; the
// stores/services underneath are the live wired ones, so rows land in the
// real `app_feedback` table.
let server: PuterServer;
let router: PuterRouter;
beforeAll(async () => {
server = await setupTestServer();
router = new PuterRouter();
server.controllers.appFeedback.registerRoutes(router);
});
afterAll(async () => {
await server?.shutdown();
});
afterEach(() => {
vi.restoreAllMocks();
// Back to the self-hosted no-SMTP baseline the server booted with.
delete liveConfig().email;
});
const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const username = `fdbk-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const makeApp = async (
ownerUserId: number,
opts: { feedbackEnabled?: boolean; indexUrl?: string; name?: string } = {},
) => {
const name =
opts.name ?? `fdbk-app-${Math.random().toString(36).slice(2, 10)}`;
return await server.stores.app.create(
{
name,
title: `Feedback Test ${name}`,
index_url: opts.indexUrl ?? `https://${name}.example.com`,
...(opts.feedbackEnabled ? { feedback_enabled: 1 } : {}),
},
{ ownerUserId },
);
};
// Feedback is only offered when the deployment can deliver it (email
// transport configured); most tests want that baseline without asserting
// anything about the mail itself. The service reads `config.email` — the
// same live object every layer holds — so the helper writes it there and
// the global afterEach clears it.
const liveConfig = () =>
(server.clients.email as unknown as { config: Record<string, unknown> })
.config;
const mockEmailConfigured = () => {
liveConfig().email = { jsonTransport: true };
};
const confirmOwnerEmail = async (userId: number) => {
await server.clients.db.write(
'UPDATE `user` SET `email_confirmed` = ? WHERE `id` = ?',
[server.clients.db.booleanValue(true), userId],
);
const user = await server.stores.user.getById(userId);
if (user) await server.stores.user.invalidate(user);
};
const makeReq = (init: {
body?: unknown;
actor?: Actor;
query?: Record<string, unknown>;
}): Request => {
return {
body: init.body ?? {},
query: init.query ?? {},
headers: {},
actor: init.actor,
} as unknown as Request;
};
const makeRes = () => {
const captured: { statusCode: number; body: unknown } = {
statusCode: 200,
body: undefined,
};
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn((code: number) => {
captured.statusCode = code;
return res;
}),
};
return { res: res as unknown as Response, captured };
};
const findRoute = (method: string, path: string) => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route;
};
const callRoute = async (
method: string,
path: string,
req: Request,
res: Response,
) => {
const handler: RequestHandler = findRoute(method, path).handler;
await handler(req, res, () => {
throw new Error('handler called next() unexpectedly');
});
};
const submit = (actor: Actor, body: unknown) => {
const { res, captured } = makeRes();
return callRoute('post', '/', makeReq({ body, actor }), res).then(
() => captured,
);
};
// ── Route gates ─────────────────────────────────────────────────────
describe('AppFeedbackController route options', () => {
it('rejects app actors and cross-origin pages on submit', () => {
const { options } = findRoute('post', '/');
// requireUserActor is what makes feedback impossible to submit
// programmatically with an app token; guiOriginOnly keeps
// cross-origin browser pages out even with a leaked user token.
expect(options.requireUserActor).toBe(true);
expect(options.guiOriginOnly).toBe(true);
});
it('stacks a per-user budget with a per-IP backstop', () => {
const { options } = findRoute('post', '/');
const limits = options.rateLimit;
expect(Array.isArray(limits)).toBe(true);
const keys = (limits as Array<{ key?: unknown }>).map((l) => l.key);
expect(keys).toContain('user');
expect(keys).toContain('ip');
});
it('requires a user actor on the target pre-flight too', () => {
const { options } = findRoute('get', '/target');
expect(options.requireUserActor).toBe(true);
});
});
// ── GET /app-feedback/target ────────────────────────────────────────
describe('AppFeedbackController GET /target', () => {
it('throws 400 when neither or both of app/origin are given', async () => {
const { actor } = await makeUser();
for (const query of [
{},
{ app: 'x', origin: 'https://x.example.com' },
]) {
const { res } = makeRes();
await expect(
callRoute('get', '/target', makeReq({ query, actor }), res),
).rejects.toMatchObject({ statusCode: 400 });
}
});
it('reports enabled:false for an unknown app', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({ query: { app: 'no-such-app-xyz' }, actor }),
res,
);
expect(captured.body).toEqual({ enabled: false, app: null });
});
it('reports enabled:false for an app that has not opted in', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId);
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({ query: { app: app.name }, actor }),
res,
);
expect(captured.body).toMatchObject({ enabled: false });
});
it('reports enabled:true with canonical title/name for an opted-in app', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({ query: { app: app.uid }, actor }),
res,
);
expect(captured.body).toEqual({
enabled: true,
app: { name: app.name, title: app.title },
});
});
it('resolves an opted-in app whose name starts with "app-"', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const name = `app-fdbk-${Math.random().toString(36).slice(2, 10)}`;
const app = await makeApp(ownerId, { feedbackEnabled: true, name });
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({ query: { app: name }, actor }),
res,
);
expect(captured.body).toEqual({
enabled: true,
app: { name: app.name, title: app.title },
});
});
it('resolves an origin to the app whose index_url it matches', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const origin = new URL(app.index_url).origin;
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({ query: { origin }, actor }),
res,
);
expect(captured.body).toEqual({
enabled: true,
app: { name: app.name, title: app.title },
});
});
it('reports enabled:false when the email transport is unconfigured', async () => {
// No mockEmailConfigured(): this is the self-hosted no-SMTP default.
// Feedback that can never be delivered must not be solicited.
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({ query: { app: app.name }, actor }),
res,
);
expect(captured.body).toMatchObject({ enabled: false });
});
it('reports enabled:false for an origin with no registered app', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
await callRoute(
'get',
'/target',
makeReq({
query: { origin: 'https://nobody-registered.example.com' },
actor,
}),
res,
);
expect(captured.body).toEqual({ enabled: false, app: null });
});
});
// ── POST /app-feedback ──────────────────────────────────────────────
describe('AppFeedbackController POST /', () => {
it('throws 400 when message is missing or not a string', async () => {
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor } = await makeUser();
for (const message of [undefined, 12345, '']) {
await expect(
submit(actor, { app: app.name, message }),
).rejects.toMatchObject({ statusCode: 400 });
}
});
it('throws 400 when both app and origin are given', async () => {
const { actor } = await makeUser();
await expect(
submit(actor, {
app: 'x',
origin: 'https://x.example.com',
message: 'hi',
}),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 400 when the origin exceeds the stored column size', async () => {
// source_origin is VARCHAR(2048) on MySQL/Postgres; a longer origin
// must be rejected up front, not fail (or silently truncate) at the
// INSERT after passing every other validation.
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const origin = `${new URL(app.index_url).origin}/${'x'.repeat(2500)}`;
const { actor } = await makeUser();
await expect(
submit(actor, { origin, message: 'hi' }),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws 403 feedback_not_enabled when the app has not opted in', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId);
const { actor } = await makeUser();
await expect(
submit(actor, { app: app.name, message: 'hi there' }),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'feedback_not_enabled',
});
});
it('throws 403 for an unknown app and an unknown origin alike', async () => {
const { actor } = await makeUser();
await expect(
submit(actor, { app: 'no-such-app-xyz', message: 'hi' }),
).rejects.toMatchObject({ statusCode: 403 });
await expect(
submit(actor, {
origin: 'https://nobody-registered.example.com',
message: 'hi',
}),
).rejects.toMatchObject({ statusCode: 403 });
});
it('throws 403 feedback_not_enabled when the email transport is unconfigured', async () => {
// No mockEmailConfigured(): the opted-in app must still refuse — a
// stored row nothing can read, sold to the sender as delivered, is
// worse than an honest refusal.
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor } = await makeUser();
await expect(
submit(actor, { app: app.name, message: 'into the void' }),
).rejects.toMatchObject({
statusCode: 403,
legacyCode: 'feedback_not_enabled',
});
});
it('throws 400 when the message exceeds the length limit', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor } = await makeUser();
await expect(
submit(actor, {
app: app.name,
message: 'x'.repeat(
AppFeedbackService.MESSAGE_MAX_LENGTH + 1,
),
}),
).rejects.toMatchObject({ statusCode: 400 });
});
it('stores a normalized row and responds with an empty object', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor, userId } = await makeUser();
const captured = await submit(actor, {
app: app.uid,
message: ' Great\r\napp! ',
context: 'app',
});
expect(captured.body).toEqual({});
const rows = (await server.clients.db.read(
'SELECT * FROM `app_feedback` WHERE `user_id` = ?',
[userId],
)) as Array<Record<string, unknown>>;
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
app_uid: app.uid,
message: 'Great\napp!',
source_env: 'app',
source_origin: null,
});
// Engine-agnostic reads: pg returns BIGINT as string and BOOLEAN as
// boolean, sqlite returns numbers for both.
expect(Number(rows[0].app_id)).toBe(app.id);
expect(Boolean(rows[0].email_sent)).toBe(false);
expect(typeof rows[0].uid).toBe('string');
expect(Number.isFinite(Number(rows[0].created_at))).toBe(true);
});
it('records the attested origin on web submissions', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const origin = new URL(app.index_url).origin;
const { actor, userId } = await makeUser();
await submit(actor, { origin, message: 'from the web', context: 'web' });
const rows = (await server.clients.db.read(
'SELECT `source_env`, `source_origin` FROM `app_feedback` WHERE `user_id` = ?',
[userId],
)) as Array<Record<string, unknown>>;
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
source_env: 'web',
source_origin: origin,
});
});
it('enforces the per-user-per-app daily cap with 429', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor, userId } = await makeUser();
const since = Math.floor(Date.now() / 1000);
for (let i = 0; i < AppFeedbackService.PER_USER_APP_DAILY_LIMIT; i++) {
await server.stores.appFeedback.create({
appId: app.id,
appUid: app.uid,
userId,
message: `seed ${i}`,
});
}
expect(
await server.stores.appFeedback.countByUserAndAppSince(
userId,
app.id,
since - 60,
),
).toBe(AppFeedbackService.PER_USER_APP_DAILY_LIMIT);
await expect(
submit(actor, { app: app.name, message: 'one too many' }),
).rejects.toMatchObject({
statusCode: 429,
legacyCode: 'too_many_requests',
});
});
it('rolls back the stored row when a concurrent burst breaches the cap', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor, userId } = await makeUser();
for (let i = 0; i < AppFeedbackService.PER_USER_APP_DAILY_LIMIT; i++) {
await server.stores.appFeedback.create({
appId: app.id,
appUid: app.uid,
userId,
message: `seed ${i}`,
});
}
// Simulate the losing side of the check-then-insert race: the
// pre-insert check reads a stale under-cap count; the post-insert
// recount (real implementation) sees the truth.
vi.spyOn(
server.stores.appFeedback,
'countByUserAndAppSince',
).mockResolvedValueOnce(0);
await expect(
submit(actor, { app: app.name, message: 'raced past the cap' }),
).rejects.toMatchObject({
statusCode: 429,
legacyCode: 'too_many_requests',
});
const rows = (await server.clients.db.read(
'SELECT COUNT(*) AS n FROM `app_feedback` WHERE `user_id` = ?',
[userId],
)) as Array<{ n: unknown }>;
expect(Number(rows[0]?.n)).toBe(
AppFeedbackService.PER_USER_APP_DAILY_LIMIT,
);
});
it('enforces the per-user daily cap across apps with 429', async () => {
mockEmailConfigured();
const { userId: ownerId } = await makeUser();
const target = await makeApp(ownerId, { feedbackEnabled: true });
const other = await makeApp(ownerId, { feedbackEnabled: true });
const { actor, userId } = await makeUser();
for (let i = 0; i < AppFeedbackService.PER_USER_DAILY_LIMIT; i++) {
await server.stores.appFeedback.create({
appId: other.id,
appUid: other.uid,
userId,
message: `seed ${i}`,
});
}
await expect(
submit(actor, { app: target.name, message: 'over the limit' }),
).rejects.toMatchObject({ statusCode: 429 });
});
});
// ── Owner email delivery ────────────────────────────────────────────
// Which submissions get emailed, and what the mail contains, is service
// logic covered in AppFeedbackService.test.ts. What the controller owes the
// caller is that mail trouble never becomes the sender's problem.
describe('AppFeedbackController owner email', () => {
it('a failing email send never fails the request', async () => {
mockEmailConfigured();
vi.spyOn(server.clients.email, 'send').mockRejectedValue(
new Error('smtp down'),
);
const { userId: ownerId } = await makeUser();
await confirmOwnerEmail(ownerId);
const app = await makeApp(ownerId, { feedbackEnabled: true });
const { actor, userId } = await makeUser();
const captured = await submit(actor, {
app: app.name,
message: 'still stored',
});
expect(captured.body).toEqual({});
const rows = (await server.clients.db.read(
'SELECT `email_sent` FROM `app_feedback` WHERE `user_id` = ?',
[userId],
)) as Array<{ email_sent: unknown }>;
expect(rows).toHaveLength(1);
expect(Boolean(rows[0]?.email_sent)).toBe(false);
});
});
@@ -0,0 +1,169 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { AppFeedbackService } from '../../services/feedback/AppFeedbackService.js';
import { PuterController } from '../types.js';
/**
* Endpoints behind the "send feedback to this app's developer" dialog
* (`puter.ui.showFeedbackDialog()`). Only the GUI (desktop dialog or the
* puter.com popup) calls these; apps cannot — both routes reject app actors,
* which is what makes feedback impossible to submit programmatically on a
* user's behalf.
*
* The target may be named either by `app` (uid or name — the desktop knows
* which app asked) or by `origin` (external site — the popup passes its
* browser-attested opener origin). Exactly one must be provided.
*/
/** Sanity cap on the raw body field; the service enforces the real limit. */
const RAW_MESSAGE_CAP = 50_000;
// Upper bound on the `app`/`origin` target params. Must not exceed the
// `source_origin` column (VARCHAR(2048) on MySQL/Postgres): the raw origin is
// stored verbatim, and a longer value would fail the INSERT after passing
// every validation — or be silently truncated on non-strict MySQL.
const TARGET_PARAM_MAX_LENGTH = 2048;
const readTargetParam = (value: unknown): string | undefined => {
return typeof value === 'string' &&
value.length > 0 &&
value.length <= TARGET_PARAM_MAX_LENGTH
? value
: undefined;
};
@Controller('/app-feedback')
export class AppFeedbackController extends PuterController {
/**
* GET /app-feedback/target — pre-flight for the dialog: whether the target
* app accepts feedback, plus its canonical title/name for display. Reveals
* nothing that `puter.apps.get` doesn't already.
*/
@Get('/target', {
subdomain: 'api',
requireUserActor: true,
requireVerified: true,
rateLimit: {
scope: 'app-feedback-target',
limit: 60,
window: 60_000,
key: 'user',
},
})
async target(req: Request, res: Response): Promise<void> {
const app = readTargetParam(req.query.app);
const origin = readTargetParam(req.query.origin);
if (!app === !origin) {
throw new HttpError(
400,
'Exactly one of `app` and `origin` is required',
{ legacyCode: 'bad_request' },
);
}
const service = this.services.appFeedback as AppFeedbackService;
res.json(await service.getTarget({ app, origin }));
}
/**
* POST /app-feedback — store one feedback message and email the app's
* developer. Strict limits: the route limits below are the cheap first
* line; AppFeedbackService enforces durable per-user/per-app daily caps
* from the database (the route limiter fails open, the DB caps don't).
*/
@Post('/', {
subdomain: 'api',
requireUserActor: true,
requireVerified: true,
// Submissions only ever originate from our own GUI pages (desktop
// dialog / popup). Cross-origin browser pages get stopped here even
// if they somehow hold a user token; non-browser clients still pass
// and are handled by the caps.
guiOriginOnly: true,
rateLimit: [
{
scope: 'app-feedback-user',
limit: 5,
window: 30 * 60_000,
key: 'user',
},
// IP backstop so freshly minted accounts can't stack per-user
// budgets from one machine.
{
scope: 'app-feedback-ip',
limit: 30,
window: 24 * 60 * 60_000,
key: 'ip',
},
],
})
async submit(req: Request, res: Response): Promise<void> {
const body = req.body ?? {};
const app = readTargetParam(body.app);
const origin = readTargetParam(body.origin);
if (!app === !origin) {
throw new HttpError(
400,
'Exactly one of `app` and `origin` is required',
{ legacyCode: 'bad_request' },
);
}
const message = body.message;
if (typeof message !== 'string' || message.length === 0) {
throw new HttpError(400, '`message` is required', {
legacyCode: 'bad_request',
});
}
if (message.length > RAW_MESSAGE_CAP) {
throw new HttpError(
400,
`\`message\` is too long (max ${AppFeedbackService.MESSAGE_MAX_LENGTH} characters)`,
{ legacyCode: 'bad_request' },
);
}
const sourceEnv =
body.context === 'app' || body.context === 'web'
? body.context
: undefined;
const userId = req.actor?.user?.id;
if (!userId) {
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
}
const service = this.services.appFeedback as AppFeedbackService;
await service.submit({
userId,
app,
origin,
message,
sourceEnv,
sourceOrigin: origin ?? null,
});
res.json({});
}
}
+14 -11
View File
@@ -21,7 +21,7 @@ import type { Request, Response } from 'express';
import type { Readable } from 'node:stream';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import type { Actor } from '../../core/actor.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -927,7 +927,7 @@ describe('FSController.searchEntries', () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const appActor = makeActor({ ...userActor, app: { uid: appUid } });
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
// User-owned entry outside AppData — must NOT appear for the app.
@@ -978,7 +978,7 @@ describe('FSController.searchEntries', () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const appActor = makeActor({ ...userActor, app: { uid: appUid } });
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
// Only seed outside AppData — the app must not be able to find it.
@@ -1127,7 +1127,7 @@ describe('FSController.mkdirEntry', () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const appActor = makeActor({ ...userActor, app: { uid: appUid } });
const target = `/${username}/AppData/${appUid}`;
await withActor(userActor, () =>
@@ -1889,10 +1889,10 @@ describe('FSController.readdirEntries recursive', () => {
it('masks denials for app-under-user actors as a 404 (legacy parity)', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
const appActor = makeActor({
...userActor,
app: { uid: `app-readdir-${uuidv4()}` },
};
});
// The user's Documents is outside the app's AppData subtree, so the
// app can't list it. Legacy `/readdir` masks this as a 404
// subject_does_not_exist rather than leaking a 403.
@@ -2372,15 +2372,18 @@ describe('FSController.statEntry additional branches', () => {
// ── #getReportedCosts ───────────────────────────────────────────────
describe('FSController.getReportedCosts', () => {
it('mirrors every FS_COSTS entry as a per-byte line item', async () => {
const { FS_COSTS } = await import('./costs.js');
it('mirrors every storage-operation price as a per-operation line item', async () => {
const { STORAGE_OP_COSTS } =
await import('../../services/metering/costs.js');
const reported = controller.getReportedCosts();
expect(reported.length).toBe(Object.keys(FS_COSTS).length);
for (const [usageType, ucentsPerUnit] of Object.entries(FS_COSTS)) {
expect(reported.length).toBe(Object.keys(STORAGE_OP_COSTS).length);
for (const [usageType, ucentsPerUnit] of Object.entries(
STORAGE_OP_COSTS,
)) {
expect(reported).toContainEqual({
usageType,
ucentsPerUnit,
unit: 'byte',
unit: 'operation',
source: 'controller:fs',
});
}
+224 -77
View File
@@ -22,11 +22,18 @@ import type { Request, Response } from 'express';
import { posix as pathPosix } from 'node:path';
import { pipeline } from 'node:stream/promises';
import type { Actor } from '../../core/actor.js';
import { effectiveActorApp } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import { assertNormalized } from '../../services/fs/resolveNode.js';
import {
assertNormalized,
isOwnersTrash,
} from '../../services/fs/resolveNode.js';
import {
maskEntryPath,
maskPathForRequest,
resolveSharePath,
} from '../../services/fs/sharePathMask.js';
import type {
PreparedBatchWrite,
UploadedBatchWriteItem,
@@ -39,7 +46,19 @@ import {
} from '../../util/concurrency.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import { STORAGE_OP_COSTS } from '../../services/metering/costs.js';
import {
FS_MULTIPART_LIMIT,
FS_MUTATE_LIMIT,
FS_READ_CONCURRENT,
FS_READ_LIMIT,
FS_READDIR_LIMIT,
FS_SEARCH_CONCURRENT,
FS_SEARCH_LIMIT,
FS_STAT_LIMIT,
FS_WRITE_CONCURRENT,
FS_WRITE_LIMIT,
} from './limits.js';
import {
assertAccess as assertLegacyAccess,
fsEntryMimeType,
@@ -106,16 +125,26 @@ const DEFAULT_BATCH_WRITE_SIDE_EFFECT_CONCURRENCY = 8;
@Controller('/fs')
export class FSController extends PuterController {
// Object-store requests are reported here because the filesystem is what
// makes them. Bytes leaving the server are not: they are metered for every
// response, so `MeteringService` prices them.
override getReportedCosts() {
return Object.entries(FS_COSTS).map(([usageType, ucentsPerUnit]) => ({
usageType,
ucentsPerUnit,
unit: 'byte',
source: 'controller:fs',
}));
return Object.entries(STORAGE_OP_COSTS).map(
([usageType, ucentsPerUnit]) => ({
usageType,
ucentsPerUnit,
unit: 'operation',
source: 'controller:fs',
}),
);
}
@Post('/startWrite', { subdomain: 'api', requireVerified: true })
@Post('/startWrite', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_MULTIPART_LIMIT,
})
async startWrite(
req: Request<RouteParams, null, SignedWriteRequest>,
res: Response<ClientSignedWriteResponse>,
@@ -123,7 +152,7 @@ export class FSController extends PuterController {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
const requestBody = this.#withGuiMetadata(req.body, req.body);
requestBody.fileMetadata = this.#normalizeFileMetadataPath(
requestBody.fileMetadata = await this.#normalizeFileMetadataPath(
req,
requestBody.fileMetadata,
requestBody,
@@ -173,7 +202,12 @@ export class FSController extends PuterController {
);
}
@Post('/startBatchWrite', { subdomain: 'api', requireVerified: true })
@Post('/startBatchWrite', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_MULTIPART_LIMIT,
})
async startBatchWrites(
req: Request<RouteParams, null, SignedWriteRequest[]>,
res: Response<ClientSignedWriteResponse[]>,
@@ -189,7 +223,7 @@ export class FSController extends PuterController {
req.body,
);
normalizedRequestBody.fileMetadata =
this.#normalizeFileMetadataPath(
await this.#normalizeFileMetadataPath(
req,
normalizedRequestBody.fileMetadata,
normalizedRequestBody,
@@ -273,7 +307,12 @@ export class FSController extends PuterController {
);
}
@Post('/completeWrite', { subdomain: 'api', requireVerified: true })
@Post('/completeWrite', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_MULTIPART_LIMIT,
})
async completeWrite(
req: Request<RouteParams, null, CompleteWriteRequest>,
res: Response<ClientCompleteWriteResponse>,
@@ -304,7 +343,12 @@ export class FSController extends PuterController {
);
}
@Post('/completeBatchWrite', { subdomain: 'api', requireVerified: true })
@Post('/completeBatchWrite', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_MULTIPART_LIMIT,
})
async completeBatchWrites(
req: Request<RouteParams, null, CompleteWriteRequest[]>,
res: Response<ClientCompleteWriteResponse[]>,
@@ -347,7 +391,11 @@ export class FSController extends PuterController {
);
}
@Post('/abortWrite', { subdomain: 'api', requireVerified: true })
@Post('/abortWrite', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MULTIPART_LIMIT,
})
async abortWrite(
req: Request<RouteParams, null, AbortWriteRequest>,
res: Response<{ ok: true }>,
@@ -363,7 +411,12 @@ export class FSController extends PuterController {
res.json({ ok: true });
}
@Post('/signMultipartParts', { subdomain: 'api', requireVerified: true })
@Post('/signMultipartParts', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_MULTIPART_LIMIT,
})
async signMultipartParts(
req: Request<RouteParams, null, SignMultipartPartsRequest>,
res: Response<ClientSignMultipartPartsResponse>,
@@ -376,7 +429,13 @@ export class FSController extends PuterController {
res.json(this.#withoutStorageInternals(response));
}
@Post('/write', { subdomain: 'api', requireVerified: true })
@Post('/write', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_WRITE_LIMIT,
concurrent: FS_WRITE_CONCURRENT,
})
async write(
req: Request<RouteParams, null, WriteRequest>,
res: Response<ClientWriteResponse>,
@@ -384,7 +443,7 @@ export class FSController extends PuterController {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
const requestBody = this.#withGuiMetadata(req.body, req.body);
requestBody.fileMetadata = this.#normalizeFileMetadataPath(
requestBody.fileMetadata = await this.#normalizeFileMetadataPath(
req,
requestBody.fileMetadata,
requestBody,
@@ -398,7 +457,7 @@ export class FSController extends PuterController {
await this.#assertWriteAccess(req, requestBody.fileMetadata, {
pathAlreadyNormalized: true,
});
const normalizedPath = this.#normalizePath(
const normalizedPath = await this.#resolveClientPath(
requestBody.fileMetadata.path,
);
const uploadTracker = await this.#createUploadTracker(
@@ -422,7 +481,13 @@ export class FSController extends PuterController {
res.json(this.#withRequiredClientFsEntry(updatedResponse));
}
@Post('/batchWrite', { subdomain: 'api', requireVerified: true })
@Post('/batchWrite', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_WRITE_LIMIT,
concurrent: FS_WRITE_CONCURRENT,
})
async batchWrites(
req: Request<RouteParams, null, WriteRequest[]>,
res: Response<ClientWriteResponse[]>,
@@ -498,7 +563,7 @@ export class FSController extends PuterController {
...parsedManifest,
items: parsedManifest.items.map((item) => ({
...item,
fileMetadata: this.#normalizeFileMetadataPath(
fileMetadata: this.#expandFileMetadataPath(
req,
item.fileMetadata,
item,
@@ -529,7 +594,12 @@ export class FSController extends PuterController {
...item,
fileMetadata:
await this.#resolveAssociatedAppMetadata(
item.fileMetadata,
{
...item.fileMetadata,
path: await this.#unmaskPath(
item.fileMetadata.path,
),
},
item,
appUidLookupCache,
userId,
@@ -774,7 +844,7 @@ export class FSController extends PuterController {
req.body,
);
normalizedRequestBody.fileMetadata =
this.#normalizeFileMetadataPath(
await this.#normalizeFileMetadataPath(
req,
normalizedRequestBody.fileMetadata,
normalizedRequestBody,
@@ -889,7 +959,11 @@ export class FSController extends PuterController {
// -- Read-side routes ------------------------------------------------
@Post('/stat', { subdomain: 'api', requireVerified: true })
@Post('/stat', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_STAT_LIMIT,
})
async statEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -933,7 +1007,7 @@ export class FSController extends PuterController {
uuid: entry.uuid,
uid: entry.uid ?? entry.uuid,
parentUid: entry.parentUid ?? null,
path: entry.path,
path: maskEntryPath(entry),
name: entry.name,
isDir: entry.isDir,
isShortcut: entry.isShortcut,
@@ -1003,12 +1077,20 @@ export class FSController extends PuterController {
* without a JSON body. Every value arrives as a string, so parameter
* parsing goes through the same coercion helpers the POST path uses.
*/
@Get('/readdir', { subdomain: 'api', requireVerified: true })
@Get('/readdir', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_READDIR_LIMIT,
})
async readdirEntriesViaGet(req: Request, res: Response) {
return this.readdirEntries(req, res);
}
@Post('/readdir', { subdomain: 'api', requireVerified: true })
@Post('/readdir', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_READDIR_LIMIT,
})
async readdirEntries(req: Request, res: Response) {
const actor = this.#requireActor(req);
// GET carries its parameters in the query string; POST in the body.
@@ -1041,7 +1123,6 @@ export class FSController extends PuterController {
const rootChildren = await listRootEntries(
actor,
this.stores.fsEntry,
this.services.permission,
);
const rootSuggestions =
await this.services.suggestedApps.getSuggestedAppsForEntries(
@@ -1204,7 +1285,12 @@ export class FSController extends PuterController {
}
}
@Post('/search', { subdomain: 'api', requireVerified: true })
@Post('/search', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_SEARCH_LIMIT,
concurrent: FS_SEARCH_CONCURRENT,
})
async searchEntries(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1230,7 +1316,13 @@ export class FSController extends PuterController {
res.json(results);
}
@Get('/read', { subdomain: 'api', requireVerified: true })
@Get('/read', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_READ_LIMIT,
concurrent: FS_READ_CONCURRENT,
})
async readEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const query = this.#toObjectRecord(req.query);
@@ -1274,15 +1366,6 @@ export class FSController extends PuterController {
);
res.status(range ? 206 : 200);
const metering = this.services.metering as
| {
batchIncrementUsages?: (
actor: unknown,
entries: unknown[],
) => void;
}
| undefined;
try {
await pipeline(download.body, res);
} catch {
@@ -1290,28 +1373,15 @@ export class FSController extends PuterController {
// tore down both ends. Response is partially sent; nothing to do.
return;
}
// Meter egress only on successful completion.
if (metering?.batchIncrementUsages && download.contentLength) {
try {
const bytes = download.contentLength;
metering.batchIncrementUsages(actor, [
{
usageType: 'filesystem:egress:bytes',
usageAmount: bytes,
costOverride:
FS_COSTS['filesystem:egress:bytes'] * bytes,
},
]);
} catch {
// ignore — metering is non-critical.
}
}
}
// -- Mutation routes ------------------------------------------------
@Post('/mkdir', { subdomain: 'api', requireVerified: true })
@Post('/mkdir', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MUTATE_LIMIT,
})
async mkdirEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1327,7 +1397,7 @@ export class FSController extends PuterController {
// would compute a wrong parent for `~/...` inputs (e.g. dirname of
// `/~/Documents/foo` is `/~/Documents`, not `/<username>/Documents`).
const username = this.#getActorUsername(req);
const path = this.#normalizePath(rawPath, username);
const path = await this.#resolveClientPath(rawPath, username);
if (path === '/')
throw new HttpError(400, 'Cannot mkdir at root', {
legacyCode: 'bad_request',
@@ -1352,7 +1422,11 @@ export class FSController extends PuterController {
res.json(this.#toClientEntry(entry));
}
@Post('/touch', { subdomain: 'api', requireVerified: true })
@Post('/touch', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MUTATE_LIMIT,
})
async touchEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1364,7 +1438,7 @@ export class FSController extends PuterController {
});
const username = this.#getActorUsername(req);
const path = this.#normalizePath(rawPath, username);
const path = await this.#resolveClientPath(rawPath, username);
if (path === '/')
throw new HttpError(400, 'Cannot touch root', {
legacyCode: 'bad_request',
@@ -1388,7 +1462,11 @@ export class FSController extends PuterController {
res.json(this.#toClientEntry(entry));
}
@Post('/rename', { subdomain: 'api', requireVerified: true })
@Post('/rename', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MUTATE_LIMIT,
})
async renameEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const body = this.#toObjectRecord(req.body);
@@ -1401,12 +1479,17 @@ export class FSController extends PuterController {
const entry = await this.#resolveEntryForRequest(body);
await this.#assertAccess(actor, entry.path, 'write');
const renamed = await this.services.fs.rename(entry, newName);
const userId = this.#getActorUserId(req);
const renamed = await this.services.fs.rename(userId, entry, newName);
this.#emitGuiItemUpdated(renamed);
res.json(this.#toClientEntry(renamed));
}
@Post('/delete', { subdomain: 'api', requireVerified: true })
@Post('/delete', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MUTATE_LIMIT,
})
async deleteEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1424,7 +1507,11 @@ export class FSController extends PuterController {
res.json({ ok: true });
}
@Post('/move', { subdomain: 'api', requireVerified: true })
@Post('/move', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MUTATE_LIMIT,
})
async moveEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1437,7 +1524,9 @@ export class FSController extends PuterController {
await this.#resolveEntryForRequest(destinationRef);
await this.#assertAccess(actor, source.path, 'write');
await this.#assertAccess(actor, destinationParent.path, 'write');
if (!isOwnersTrash(source, destinationParent)) {
await this.#assertAccess(actor, destinationParent.path, 'write');
}
const moved = await this.services.fs.move(userId, {
source,
@@ -1452,7 +1541,12 @@ export class FSController extends PuterController {
res.json(this.#toClientEntry(moved));
}
@Post('/copy', { subdomain: 'api', requireVerified: true })
@Post('/copy', {
subdomain: 'api',
requireVerified: true,
requireCredits: true,
rateLimit: FS_MUTATE_LIMIT,
})
async copyEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1467,6 +1561,7 @@ export class FSController extends PuterController {
await this.#assertAccess(actor, source.path, 'read');
await this.#assertAccess(actor, destinationParent.path, 'write');
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
const copy = await this.services.fs.copy(userId, {
source,
destinationParent,
@@ -1475,12 +1570,19 @@ export class FSController extends PuterController {
overwrite: this.#toBoolean(body.overwrite) ?? false,
dedupeName:
this.#toBoolean(body.dedupe_name ?? body.change_name) ?? true,
...(storageAllowanceMax !== undefined
? { storageAllowanceMax }
: {}),
});
this.#emitGuiItemAdded(copy);
res.json(this.#toClientEntry(copy));
}
@Post('/mkshortcut', { subdomain: 'api', requireVerified: true })
@Post('/mkshortcut', {
subdomain: 'api',
requireVerified: true,
rateLimit: FS_MUTATE_LIMIT,
})
async mkshortcutEntry(req: Request, res: Response) {
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
@@ -1536,7 +1638,7 @@ export class FSController extends PuterController {
const ref = {
path:
rawPath !== undefined
? mod.expandTildePath(rawPath, username)
? await this.#resolveClientPath(rawPath, username)
: undefined,
uid:
typeof source.uid === 'string'
@@ -2105,6 +2207,18 @@ export class FSController extends PuterController {
return Math.max(...candidates);
}
/**
* `#normalizePath`, plus turning a masked share path back into the owner's
* real one. Every client-authored path goes through here.
*/
async #resolveClientPath(path: string, username?: string): Promise<string> {
return resolveSharePath(
this.stores.fsEntry,
Context.get('actor'),
this.#normalizePath(path, username),
);
}
#normalizePath(path: string, username?: string): string {
const trimmedPath = path.trim();
if (trimmedPath.length === 0) {
@@ -2135,7 +2249,7 @@ export class FSController extends PuterController {
return normalizedPath;
}
#normalizeFileMetadataPath(
#expandFileMetadataPath(
req: Request,
fileMetadata: FSEntryWriteInput | undefined,
fallbackSource?: unknown,
@@ -2150,13 +2264,38 @@ export class FSController extends PuterController {
});
}
const username = this.#getActorUsername(req);
return {
...resolvedFileMetadata,
path: this.#normalizePath(resolvedFileMetadata.path, username),
path: this.#normalizePath(
resolvedFileMetadata.path,
this.#getActorUsername(req),
),
};
}
/** As above, plus turning a masked share path into the owner's real one. */
async #normalizeFileMetadataPath(
req: Request,
fileMetadata: FSEntryWriteInput | undefined,
fallbackSource?: unknown,
): Promise<FSEntryWriteInput> {
const expanded = this.#expandFileMetadataPath(
req,
fileMetadata,
fallbackSource,
);
return { ...expanded, path: await this.#unmaskPath(expanded.path) };
}
/** The un-masking half, for callers that already expanded the path. */
async #unmaskPath(path: string): Promise<string> {
return resolveSharePath(
this.stores.fsEntry,
Context.get('actor'),
path,
);
}
#extractGuiMetadata(
input: unknown,
fallback: WriteGuiMetadata | undefined,
@@ -2235,7 +2374,7 @@ export class FSController extends PuterController {
}
const normalizedFileMetadata = options?.pathAlreadyNormalized
? fileMetadata
: this.#normalizeFileMetadataPath(req, fileMetadata);
: await this.#normalizeFileMetadataPath(req, fileMetadata);
if (!normalizedFileMetadata) {
throw new HttpError(400, 'Missing path', {
legacyCode: 'bad_request',
@@ -2398,14 +2537,19 @@ export class FSController extends PuterController {
requestBody: SignedWriteRequest,
response: SignedWriteResponse,
): Promise<void> {
const normalizedPath = this.#normalizePath(
const normalizedPath = await this.#resolveClientPath(
requestBody.fileMetadata.path,
);
// This goes to the *acting* user, not the owner, so a recipient writing
// into a shared folder would otherwise be handed the owner's real path
// — the one thing the masking exists to withhold. There is no row yet,
// so the mask is built from the path the caller already named.
const publishedPath = maskPathForRequest(normalizedPath);
const pendingResponse = {
id: response.objectKey,
uid: response.objectKey,
uuid: response.objectKey,
path: normalizedPath,
path: publishedPath,
name: pathPosix.basename(normalizedPath),
is_dir: false,
content_type: response.contentType,
@@ -2437,7 +2581,7 @@ export class FSController extends PuterController {
// search results to this path so app actors can't see entries outside
// their AppData via `/fs/search`.
#appDataScopeForActor(actor: Actor): string | undefined {
const app = effectiveActorApp(actor);
const app = actor.effectiveApp;
if (!app) return undefined;
const username = actor.user?.username;
if (typeof username !== 'string' || username.length === 0)
@@ -2688,7 +2832,10 @@ export class FSController extends PuterController {
user_id: userId,
userId: userId,
item_uid: itemUid,
item_path: itemPath,
// Progress is reported to the acting user, who is not the
// owner when the write lands in a shared folder — publish
// the path in the form they addressed it by.
item_path: maskPathForRequest(itemPath),
...this.#toEventGuiMetadata(guiMetadata),
},
},
@@ -21,7 +21,7 @@ import type { Request, RequestHandler, Response } from 'express';
import { Readable, Writable } from 'node:stream';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import type { Actor } from '../../core/actor.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
@@ -377,7 +377,7 @@ describe('LegacyFSController.readdirSubdomains', () => {
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`) VALUES (?, ?, ?)',
[uuidv4(), `sd-${Math.random().toString(36).slice(2, 8)}`, userId],
);
const appActor: Actor = { ...actor, app: { uid: `app-${uuidv4()}` } };
const appActor = makeActor({ ...actor, app: { uid: `app-${uuidv4()}` } });
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.readdirSubdomains(makeReq({ actor: appActor }), res),
@@ -21,7 +21,7 @@ import type { Request, Response } from 'express';
import { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
@@ -250,7 +250,7 @@ describe('LegacyFSController.mkdir', () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const appActor = makeActor({ ...userActor, app: { uid: appUid } });
const parent = `/${username}/AppData`;
await withActor(userActor, () =>
@@ -934,7 +934,7 @@ describe('LegacyFSController.copy', () => {
const username = actor.user!.username!;
const src = `/${username}/Documents/dup.txt`;
const existing = `/${username}/Pictures/dup.txt`;
for ( const p of [src, existing] ) {
for (const p of [src, existing]) {
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: p }, actor }),
@@ -964,9 +964,8 @@ describe('LegacyFSController.copy', () => {
fields: { entry_name: 'dup.txt' },
});
const replaced = (await server.stores.fsEntry.getEntryByPath(
existing,
))!;
const replaced =
(await server.stores.fsEntry.getEntryByPath(existing))!;
// With overwrite: the replaced entry rides along in the response
// (so the caller can drop its row) and item.removed tells every
@@ -1091,7 +1090,7 @@ describe('LegacyFSController.move', () => {
const username = actor.user!.username!;
const src = `/${username}/Documents/clash.txt`;
const existing = `/${username}/Pictures/clash.txt`;
for ( const p of [src, existing] ) {
for (const p of [src, existing]) {
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: p }, actor }),
@@ -1121,9 +1120,8 @@ describe('LegacyFSController.move', () => {
fields: { entry_name: 'clash.txt' },
});
const replaced = (await server.stores.fsEntry.getEntryByPath(
existing,
))!;
const replaced =
(await server.stores.fsEntry.getEntryByPath(existing))!;
// With overwrite: the replaced entry rides along in the response
// (so the caller can drop its row) and item.removed tells every
@@ -1168,6 +1166,115 @@ describe('LegacyFSController.move', () => {
};
expect(removedPayload.response?.uid).toBe(replaced.uuid);
});
it('lets a share recipient trash an item into the owner’s trash', async () => {
const owner = await makeUser();
const holder = await makeUser();
const ownerName = owner.actor.user!.username!;
const sharedPath = `/${ownerName}/Documents/Contents`;
await withActor(owner.actor, () =>
controller.mkdir(
makeReq({ body: { path: sharedPath }, actor: owner.actor }),
makeRes().res,
),
);
await withActor(owner.actor, () =>
controller.touch(
makeReq({
body: { path: `${sharedPath}/note.txt` },
actor: owner.actor,
}),
makeRes().res,
),
);
const shared = (await server.stores.fsEntry.getEntryByPath(sharedPath))!;
await server.services.acl.setUserUser(
owner.actor,
holder.actor,
{
path: shared.path,
resolveAncestors: () =>
server.services.fs.getAncestorChain(shared.path),
},
'write',
);
const { res, captured } = makeRes();
await withActor(holder.actor, () =>
controller.move(
makeReq({
body: {
source: `${sharedPath}/note.txt`,
destination: `/${ownerName}/Trash`,
},
actor: holder.actor,
}),
res,
),
);
// The move landed in the owner's trash, but the recipient is told so
// in masked form — the owner's real layout stays theirs.
const body = captured.body as { moved: { uid: string; path: string } };
expect(body.moved.path).toBe(
`/${ownerName}/${body.moved.uid}/note.txt`,
);
const moved = await server.stores.fsEntry.getEntryByUuid(
body.moved.uid,
);
expect(moved!.path).toBe(`/${ownerName}/Trash/note.txt`);
});
it('refuses a share recipient moving an item into their own trash', async () => {
const owner = await makeUser();
const holder = await makeUser();
const ownerName = owner.actor.user!.username!;
const holderName = holder.actor.user!.username!;
const sharedPath = `/${ownerName}/Documents/Shared`;
await withActor(owner.actor, () =>
controller.mkdir(
makeReq({ body: { path: sharedPath }, actor: owner.actor }),
makeRes().res,
),
);
await withActor(owner.actor, () =>
controller.touch(
makeReq({
body: { path: `${sharedPath}/theirs.txt` },
actor: owner.actor,
}),
makeRes().res,
),
);
const shared = (await server.stores.fsEntry.getEntryByPath(sharedPath))!;
await server.services.acl.setUserUser(
owner.actor,
holder.actor,
{
path: shared.path,
resolveAncestors: () =>
server.services.fs.getAncestorChain(shared.path),
},
'write',
);
await expect(
withActor(holder.actor, () =>
controller.move(
makeReq({
body: {
source: `${sharedPath}/theirs.txt`,
destination: `/${holderName}/Trash`,
},
actor: holder.actor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── search ──────────────────────────────────────────────────────────
@@ -1214,7 +1321,7 @@ describe('LegacyFSController.search', () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const appActor = makeActor({ ...userActor, app: { uid: appUid } });
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
await withActor(userActor, () =>
@@ -1419,10 +1526,10 @@ describe('LegacyFSController.sign', () => {
},
{ ownerUserId: userActor.user!.id! },
);
const attackerActor: Actor = {
const attackerActor = makeActor({
...userActor,
app: { uid: `attacker-${uuidv4()}` },
};
});
const { res } = makeRes();
await expect(
@@ -1456,7 +1563,7 @@ describe('LegacyFSController.sign', () => {
},
{ ownerUserId: userActor.user!.id! },
);
const appActor: Actor = { ...userActor, app: { uid: ownApp.uid } };
const appActor = makeActor({ ...userActor, app: { uid: ownApp.uid } });
const { res, captured } = makeRes();
await withActor(appActor, () =>
@@ -1588,8 +1695,11 @@ describe('LegacyFSController.writeFile (write IDOR)', () => {
),
);
// The write landed on the signed file …
expect((captured.body as { path?: string }).path).toBe(target);
// The write landed on the signed file — reported masked, since the
// attacker doesn't own it …
expect((captured.body as { path?: string }).path).toBe(
`/${victim.actor.user!.username}/${entry!.uuid}/secret.txt`,
);
// … and never created the attacker-named sibling.
const siblingEntry =
await server.stores.fsEntry.getEntryByPath(sibling);
@@ -1736,6 +1846,185 @@ describe('LegacyFSController.openItem (write_url stripping)', () => {
});
});
// ── openItem: the grant is a user-authority action ──────────────────
//
// /open_item writes a user→app ACL row. An app actor reaching it could hand
// itself write on any file it can read — read-only access to a folder would
// silently widen into persistent write on the files inside it, with no second
// consent prompt. The user has to be the one opening the item, and the row it
// leaves can never exceed what that user proved on the entry.
describe('LegacyFSController.openItem (grant scope)', () => {
const makeApp = async (ownerUserId: number) => {
const name = `opener-${uuidv4()}`;
return (await server.stores.app.create(
{
name,
title: 'Opener test app',
index_url: `https://${name}.test/`,
},
{ ownerUserId },
)) as { id: number; uid: string };
};
it('is registered behind the user-session gate', () => {
const router = new PuterRouter();
(
controller as unknown as {
registerRoutes: (r: PuterRouter) => void;
}
).registerRoutes(router);
const route = router.routes.find(
(r) => r.method === 'post' && r.path === '/open_item',
);
expect(route?.options).toMatchObject({
requireUserActor: true,
allowFullAccessToken: true,
});
});
it('rejects an app actor, leaving no grant behind', async () => {
const { actor: userActor, userId } = await makeUser();
const username = userActor.user!.username!;
const target = `/${username}/Documents/app-opened.txt`;
await withActor(userActor, () =>
controller.touch(
makeReq({ body: { path: target }, actor: userActor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
// The app holds the read the user approved, and is its own default
// opener — the shape that would let it widen that read into write.
const app = await makeApp(userId);
await withActor(userActor, () =>
server.services.permission.grantUserAppPermission(
userActor,
app.uid,
`fs:${entry!.uuid}:read`,
),
);
const appActor = makeActor({
...userActor,
app: { uid: app.uid, id: app.id },
});
const spy = vi
.spyOn(server.services.suggestedApps, 'getSuggestedApps')
.mockResolvedValue([{ uuid: app.uid }] as never);
try {
await expect(
withActor(appActor, () =>
controller.openItem(
makeReq({
body: { uid: entry!.uuid },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 403 });
} finally {
spy.mockRestore();
}
expect(
await server.stores.permission.hasUserAppPerm(
userId,
app.id,
`fs:${entry!.uuid}:write`,
),
).toBe(false);
});
it('grants the suggested app write when the caller has write', async () => {
const { actor, userId } = await makeUser();
const target = `/${actor.user!.username}/Documents/owned-open.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const app = await makeApp(userId);
const spy = vi
.spyOn(server.services.suggestedApps, 'getSuggestedApps')
.mockResolvedValue([{ uuid: app.uid }] as never);
try {
await withActor(actor, () =>
controller.openItem(
makeReq({ body: { path: target }, actor }),
makeRes().res,
),
);
} finally {
spy.mockRestore();
}
expect(
await server.stores.permission.hasUserAppPerm(
userId,
app.id,
`fs:${entry!.uuid}:write`,
),
).toBe(true);
});
it('grants only read when the caller is a read-only sharee', async () => {
const victim = await makeUser();
const sharee = await makeUser();
const target = `/${victim.actor.user!.username}/Documents/ro-open.txt`;
await withActor(victim.actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor: victim.actor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
await server.services.permission.grantUserUserPermission(
victim.actor,
sharee.actor.user!.username!,
`fs:${entry!.uuid}:read`,
{},
);
const app = await makeApp(sharee.userId);
const spy = vi
.spyOn(server.services.suggestedApps, 'getSuggestedApps')
.mockResolvedValue([{ uuid: app.uid }] as never);
try {
await withActor(sharee.actor, () =>
controller.openItem(
makeReq({
body: { uid: entry!.uuid },
actor: sharee.actor,
}),
makeRes().res,
),
);
} finally {
spy.mockRestore();
}
expect(
await server.stores.permission.hasUserAppPerm(
sharee.userId,
app.id,
`fs:${entry!.uuid}:write`,
),
).toBe(false);
expect(
await server.stores.permission.hasUserAppPerm(
sharee.userId,
app.id,
`fs:${entry!.uuid}:read`,
),
).toBe(true);
});
});
// ── requestAppRootDir ───────────────────────────────────────────────
describe('LegacyFSController.requestAppRootDir', () => {
@@ -1766,10 +2055,10 @@ describe('LegacyFSController.requestAppRootDir', () => {
it('rejects with 403 when the actor.app.uid differs from the requested app_uid', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
const appActor = {
const appActor = makeActor({
...actor,
app: { uid: 'app-mismatch' },
} as unknown as Actor;
});
await expect(
withActor(appActor, () =>
controller.requestAppRootDir(
@@ -1787,10 +2076,10 @@ describe('LegacyFSController.requestAppRootDir', () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const appUid = 'app-self';
const appActor = {
const appActor = makeActor({
...actor,
app: { uid: appUid },
} as unknown as Actor;
});
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.requestAppRootDir(
@@ -1805,6 +2094,50 @@ describe('LegacyFSController.requestAppRootDir', () => {
expect(body.path).toBe(`/${username}/AppData/${appUid}`);
expect(body.is_dir).toBe(true);
});
// `check: true` answers the question without acting on the answer, so that
// asking whether the access is held is not itself a filesystem write.
it('answers check:true without creating the directory', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const appUid = 'app-check-only';
const appActor = makeActor({ ...actor, app: { uid: appUid } });
const rootPath = `/${username}/AppData/${appUid}`;
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.requestAppRootDir(
makeReq({
body: { app_uid: appUid, access: 'read', check: true },
actor: appActor,
}),
res,
),
);
expect(captured.body).toEqual({ allowed: true });
expect(
await server.stores.fsEntry.getEntryByPath(rootPath),
).toBeFalsy();
});
// The guard runs first either way, so a caller that may not claim it is
// refused rather than told it is allowed.
it('still refuses check:true from a caller that is not the app', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.requestAppRootDir(
makeReq({
body: { app_uid: 'app-xyz', check: true },
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 403 });
});
});
// ── checkAppAcl ─────────────────────────────────────────────────────
@@ -2463,10 +2796,10 @@ describe('LegacyFSController.sign app sandbox + write downgrade', () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
// Build an app-under-user actor whose AppData root is the test app.
const appActor = {
const appActor = makeActor({
...actor,
app: { uid: 'sandbox-app' },
} as unknown as Actor;
});
// Create a file *outside* /Documents (anywhere outside AppData/<uid>).
const target = `/${username}/Documents/forbidden.txt`;
@@ -2730,10 +3063,10 @@ describe('LegacyFSController.suggestApps', () => {
it('refuses to look up entries an app actor cannot see', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
const appActor = makeActor({
...userActor,
app: { uid: `app-suggest-${uuidv4()}` },
};
});
const target = `/${username}/Documents/probe.txt`;
await withActor(userActor, () =>
controller.touch(
@@ -2801,10 +3134,10 @@ describe('LegacyFSController.suggestApps', () => {
describe('LegacyFSController.readdirSubdomains', () => {
it('returns an empty array for app-under-user actors', async () => {
const { actor: userActor, userId } = await makeUser();
const appActor: Actor = {
const appActor = makeActor({
...userActor,
app: { uid: `app-subd-${uuidv4()}` },
};
});
await server.clients.db.write(
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`) VALUES (?, ?, ?)',
[uuidv4(), `sd-${uuidv4().slice(0, 8)}`, userId],
@@ -2843,10 +3176,10 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
it('rejects an app actor probing entries outside AppData with 404', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
const appActor = makeActor({
...userActor,
app: { uid: `app-thumb-${uuidv4()}` },
};
});
const target = `/${username}/Documents/thumbme.txt`;
await withActor(userActor, () =>
controller.touch(
@@ -2915,40 +3248,48 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
// object — an fs object's key is its fsentry uuid — and have the server
// read or destroy it. Only inline image data is accepted.
it.each([
['an s3:// pointer', 's3://puter-local/00000000-0000-4000-8000-000000000000'],
[
'an s3:// pointer',
's3://puter-local/00000000-0000-4000-8000-000000000000',
],
['an https URL', 'https://cdn.example.com/x.png'],
['a bare object key', 'thumbnails/whatever'],
])('rejects %s instead of storing it verbatim', async (_label, thumbnail) => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/thumbme3-${uuidv4()}.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor,
}),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.updateFsentryThumbnail(
])(
'rejects %s instead of storing it verbatim',
async (_label, thumbnail) => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/thumbme3-${uuidv4()}.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({
body: { uid: entry!.uuid, thumbnail },
body: { path: target, set_modified_to_now: true },
actor,
}),
res,
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const after = await server.stores.fsEntry.getEntryByUuid(entry!.uuid);
expect(after?.thumbnail ?? null).toBeNull();
});
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.updateFsentryThumbnail(
makeReq({
body: { uid: entry!.uuid, thumbnail },
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
const after = await server.stores.fsEntry.getEntryByUuid(
entry!.uuid,
);
expect(after?.thumbnail ?? null).toBeNull();
},
);
});
// ── GET /get-launch-apps ────────────────────────────────────────────
+354 -159
View File
@@ -21,30 +21,56 @@ import Busboy from 'busboy';
import type { NextFunction, Request, RequestHandler, Response } from 'express';
import { contentType as contentTypeFromMime } from 'mime-types';
import { posix as pathPosix } from 'node:path';
import type { Actor } from '../../core/actor.js';
import { effectiveActorApp, isAccessTokenActor } from '../../core/actor.js';
import {
assertResolvedActor,
isAccessTokenActor,
makeActor,
} from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { RouteOptions } from '../../core/http/index.js';
import {
assertNotSuspended,
assertVerifiedAccount,
} from '../../core/http/middleware/gates.js';
import { RouteOptions } from '../../core/http/index.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { ACLService } from '../../services/acl/ACLService.js';
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
import type { SignedFile } from '../../util/fileSigning.js';
import { verifySignature } from '../../util/fileSigning.js';
import {
NON_OWNER_SIGNATURE_TTL_SECONDS,
verifySignature,
} from '../../util/fileSigning.js';
import { maskEntryPath } from '../../services/fs/sharePathMask.js';
import {
buildHostedBackingDenial,
hostedIndexUrlBackingIsUnavailable,
} from '../../util/hostedAppBacking.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import {
FS_BATCH_CONCURRENT,
FS_BATCH_LIMIT,
FS_DF_LIMIT,
FS_HELPER_LIMIT,
FS_MUTATE_LIMIT,
FS_POLL_LIMIT,
FS_READ_CONCURRENT,
FS_READ_LIMIT,
FS_READDIR_LIMIT,
FS_SEARCH_CONCURRENT,
FS_SEARCH_LIMIT,
FS_SIGN_LIMIT,
FS_SIGNED_CONCURRENT,
FS_SIGNED_READ_LIMIT,
FS_SIGNED_WRITE_LIMIT,
FS_STAT_LIMIT,
} from './limits.js';
import {
asRecord,
assertAccess,
assertCanCreate,
assertCanMoveInto,
getBoolean,
getString,
loadLegacyAssociatedApps,
@@ -95,6 +121,14 @@ export class LegacyFSController extends PuterController {
subdomain: 'api',
requireVerified: true,
} as RouteOptions;
// Operations that move file content or make object-store requests on
// the caller's behalf, and so are refused to an account with nothing
// left of its budget. The metadata routes above deliberately aren't:
// an account that has run out still has to be able to look at what it
// has and delete it. The signature-authorised routes aren't either —
// they carry no session to answer for, and `/sign` (which does) is
// where the URL that reaches them is minted.
const spends = { ...apiOptions, requireCredits: true } as RouteOptions;
// Signed-URL routes: the handler validates the URL signature itself,
// so no auth gate is applied (matches v1, which mounted these routers
// with no middleware).
@@ -103,40 +137,118 @@ export class LegacyFSController extends PuterController {
} as RouteOptions;
// Core filesystem_api routes — direct handlers over the FS service.
router.post('/stat', apiOptions, this.stat);
router.post('/readdir', apiOptions, this.readdir);
router.post('/mkdir', apiOptions, this.mkdir);
router.post('/copy', apiOptions, this.copy);
router.post('/move', apiOptions, this.move);
router.post('/delete', apiOptions, this.delete);
router.post('/rename', apiOptions, this.rename);
router.post('/touch', apiOptions, this.touch);
router.post('/search', apiOptions, this.search);
router.get('/read', apiOptions, this.read);
// Limits come from `./limits` and carry an explicit `scope`, so these
// draw from the same per-user budget as their v2 counterparts rather
// than handing a caller a second allowance for the same operation.
const mutate = { ...apiOptions, rateLimit: FS_MUTATE_LIMIT };
router.post(
'/stat',
{ ...apiOptions, rateLimit: FS_STAT_LIMIT },
this.stat,
);
router.post(
'/readdir',
{ ...apiOptions, rateLimit: FS_READDIR_LIMIT },
this.readdir,
);
router.post('/mkdir', mutate, this.mkdir);
router.post('/copy', { ...mutate, requireCredits: true }, this.copy);
router.post('/move', mutate, this.move);
router.post('/delete', mutate, this.delete);
router.post('/rename', mutate, this.rename);
router.post('/touch', mutate, this.touch);
router.post(
'/search',
{
...apiOptions,
rateLimit: FS_SEARCH_LIMIT,
concurrent: FS_SEARCH_CONCURRENT,
},
this.search,
);
router.get(
'/read',
{
...spends,
rateLimit: FS_READ_LIMIT,
concurrent: FS_READ_CONCURRENT,
},
this.read,
);
router.get(
'/token-read',
{
subdomain: 'api',
requireVerified: false,
allowAccessToken: true,
// An access token may or may not carry a user, so this shares
// the network-keyed budget the other signed routes use.
rateLimit: FS_SIGNED_READ_LIMIT,
},
this.tokenRead,
);
router.post('/batch', apiOptions, this.batch);
router.post(
'/batch',
{
...spends,
rateLimit: FS_BATCH_LIMIT,
concurrent: FS_BATCH_CONCURRENT,
},
this.batch,
);
// Signed-URL + meta routes.
router.post('/sign', apiOptions, this.sign);
router.post('/writeFile', signedOptions, this.writeFile);
router.get('/file', signedOptions, this.file);
router.all('/df', apiOptions, this.df);
router.post('/open_item', apiOptions, this.openItem);
router.post(
'/sign',
// Gated even though it moves nothing itself: the URL it returns
// outlives the request and is served by a route with no session to
// check, so this is the last point at which the account is known.
{ ...spends, rateLimit: FS_SIGN_LIMIT },
this.sign,
);
router.post(
'/writeFile',
{
...signedOptions,
rateLimit: FS_SIGNED_WRITE_LIMIT,
concurrent: FS_SIGNED_CONCURRENT,
},
this.writeFile,
);
router.get(
'/file',
{
...signedOptions,
rateLimit: FS_SIGNED_READ_LIMIT,
concurrent: FS_SIGNED_CONCURRENT,
},
this.file,
);
router.all('/df', { ...apiOptions, rateLimit: FS_DF_LIMIT }, this.df);
router.post(
'/open_item',
{
...apiOptions,
// Opening an item grants an app access to a file on the user's
// behalf, so only the user's own credential may drive it — an
// app calling it for itself would be widening its own ACL.
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: FS_HELPER_LIMIT,
},
this.openItem,
);
router.post(
'/auth/request-app-root-dir',
apiOptions,
{ ...apiOptions, rateLimit: FS_SIGN_LIMIT },
this.requestAppRootDir,
);
router.post('/auth/check-app-acl', apiOptions, this.checkAppAcl);
router.post(
'/auth/check-app-acl',
{ ...apiOptions, rateLimit: FS_SIGN_LIMIT },
this.checkAppAcl,
);
// `/down` — session-auth'd file download. Unlike `/file` (signed URL)
// this accepts a path on the user's behalf and streams as attachment.
@@ -153,7 +265,10 @@ export class LegacyFSController extends PuterController {
// (CSRF can't forge a header-credentialed request).
allowFullAccessToken: true,
requireVerified: true,
requireCredits: true,
antiCsrf: true,
rateLimit: FS_READ_LIMIT,
concurrent: FS_READ_CONCURRENT,
},
this.down,
);
@@ -164,92 +279,102 @@ export class LegacyFSController extends PuterController {
});
});
router.get('/get-launch-apps', apiOptions, async (req, res) => {
const recommendedSvc = this.services.recommendedApps as unknown as
{ getRecommendedApps?: () => Promise<unknown[]> } | undefined;
const recommended = recommendedSvc?.getRecommendedApps
? await recommendedSvc.getRecommendedApps()
: [];
router.get(
'/get-launch-apps',
{ ...apiOptions, rateLimit: FS_HELPER_LIMIT },
async (req, res) => {
const recommendedSvc = this.services
.recommendedApps as unknown as
| { getRecommendedApps?: () => Promise<unknown[]> }
| undefined;
const recommended = recommendedSvc?.getRecommendedApps
? await recommendedSvc.getRecommendedApps()
: [];
let recent: unknown[] = [];
const userId = req.actor?.user?.id;
if (userId) {
const recentUids =
(await (
let recent: unknown[] = [];
const userId = req.actor?.user?.id;
if (userId) {
const recentUids =
(await (
this.stores.app as unknown as {
getRecentAppOpens?: (
id: number,
opts?: { limit?: number },
) => Promise<string[]>;
}
).getRecentAppOpens?.(userId, { limit: 10 })) ?? [];
// One batched read for the rows, then the backing checks
// concurrently. Serially awaiting a lookup per uid put ~2
// round trips of latency on every desktop boot.
const appsByUid = await (
this.stores.app as unknown as {
getRecentAppOpens?: (
id: number,
opts?: { limit?: number },
) => Promise<string[]>;
getByUids: (
uids: string[],
) => Promise<Map<string, Record<string, unknown>>>;
}
).getRecentAppOpens?.(userId, { limit: 10 })) ?? [];
// One batched read for the rows, then the backing checks
// concurrently. Serially awaiting a lookup per uid put ~2
// round trips of latency on every desktop boot.
const appsByUid = await (
this.stores.app as unknown as {
getByUids: (
uids: string[],
) => Promise<Map<string, Record<string, unknown>>>;
}
).getByUids(recentUids);
).getByUids(recentUids);
// `recentUids` is ordered most-recent-first; preserve it.
const orderedApps = recentUids
.map((uid) => appsByUid.get(uid))
.filter((app): app is Record<string, unknown> =>
Boolean(app),
// `recentUids` is ordered most-recent-first; preserve it.
const orderedApps = recentUids
.map((uid) => appsByUid.get(uid))
.filter((app): app is Record<string, unknown> =>
Boolean(app),
);
// Don't hand out an index_url whose puter-hosted backing is
// gone or reclaimed. The taskbar launches recents by name (so
// AppDriver's guard applies), but this list is a
// launch-metadata producer like any other — a future consumer
// reading index_url straight off it shouldn't inherit a stale
// origin.
const backingGoneFlags = await Promise.all(
orderedApps.map((app) =>
hostedIndexUrlBackingIsUnavailable({
app,
subdomainStore: this.stores.subdomain,
config: this.config,
}).catch(() => true),
),
);
// Don't hand out an index_url whose puter-hosted backing is
// gone or reclaimed. The taskbar launches recents by name (so
// AppDriver's guard applies), but this list is a
// launch-metadata producer like any other — a future consumer
// reading index_url straight off it shouldn't inherit a stale
// origin.
const backingGoneFlags = await Promise.all(
orderedApps.map((app) =>
hostedIndexUrlBackingIsUnavailable({
app,
subdomainStore: this.stores.subdomain,
config: this.config,
}).catch(() => true),
),
);
recent = orderedApps.map((app, index) => {
const backingGone = backingGoneFlags[index];
return {
uuid: app.uid,
name: app.name,
title: app.title,
icon: app.icon ?? null,
godmode: Boolean(app.godmode),
maximize_on_start: Boolean(app.maximize_on_start),
index_url: backingGone ? null : app.index_url,
...(backingGone
? { privateAccess: buildHostedBackingDenial() }
: {}),
// An app with no owner isn't owned by a Puter user —
// it's an "external" (origin-bootstrapped) app.
external:
app.owner_user_id == null ||
app.owner_user_id === '',
};
});
}
recent = orderedApps.map((app, index) => {
const backingGone = backingGoneFlags[index];
return {
uuid: app.uid,
name: app.name,
title: app.title,
icon: app.icon ?? null,
godmode: Boolean(app.godmode),
maximize_on_start: Boolean(app.maximize_on_start),
index_url: backingGone ? null : app.index_url,
...(backingGone
? { privateAccess: buildHostedBackingDenial() }
: {}),
// An app with no owner isn't owned by a Puter user —
// it's an "external" (origin-bootstrapped) app.
external:
app.owner_user_id == null ||
app.owner_user_id === '',
};
});
}
res.json({ recommended, recent });
},
);
res.json({ recommended, recent });
});
router.post('/suggest_apps', apiOptions, this.suggestApps);
router.post(
'/suggest_apps',
{ ...apiOptions, rateLimit: FS_HELPER_LIMIT },
this.suggestApps,
);
// puter-js polls this to decide whether to purge its in-memory FS
// cache. SocketService bumps a per-user Redis key on every
// `outer.gui.item.*` mutation — read it back here.
router.get(
'/cache/last-change-timestamp',
apiOptions,
{ ...apiOptions, rateLimit: FS_POLL_LIMIT },
async (req, res) => {
const userId = req.actor?.user?.id;
if (!userId) {
@@ -270,10 +395,14 @@ export class LegacyFSController extends PuterController {
},
);
router.post('/readdir-subdomains', apiOptions, this.readdirSubdomains);
router.post(
'/readdir-subdomains',
{ ...apiOptions, rateLimit: FS_HELPER_LIMIT },
this.readdirSubdomains,
);
router.post(
'/update-fsentry-thumbnail',
apiOptions,
{ ...apiOptions, rateLimit: FS_HELPER_LIMIT },
this.updateFsentryThumbnail,
);
@@ -360,7 +489,6 @@ export class LegacyFSController extends PuterController {
const rootChildren = await listRootEntries(
actor,
this.stores.fsEntry,
this.services.permission,
);
const rootSuggestions =
await this.services.suggestedApps.getSuggestedAppsForEntries(
@@ -666,12 +794,12 @@ export class LegacyFSController extends PuterController {
source.path,
'write',
);
await assertAccess(
await assertCanMoveInto(
this.services.acl,
this.services.fs,
actor,
destinationParent.path,
'write',
source,
destinationParent,
);
// The v1 wire contract reports the entry an overwrite replaced
@@ -820,7 +948,8 @@ export class LegacyFSController extends PuterController {
'write',
);
const renamed = await this.services.fs.rename(entry, newName);
const userId = this.#getActorUserId(req);
const renamed = await this.services.fs.rename(userId, entry, newName);
await this.#emitGuiEvent('outer.gui.item.updated', renamed);
res.json(await toLegacyEntry(this.clients.event, renamed));
};
@@ -908,7 +1037,7 @@ export class LegacyFSController extends PuterController {
const actor = this.#requireActor(req);
// Subdomain enumeration is a user-level concern; app actors would
// otherwise see root_dir uids pointing outside their AppData scope.
if (effectiveActorApp(actor)) {
if (actor.effectiveApp) {
res.json([]);
return;
}
@@ -985,7 +1114,7 @@ export class LegacyFSController extends PuterController {
// App-under-user actors only see entries within their AppData root;
// user actors are unscoped. Mirrors the ACL short-circuit in
// ACLService.check.
const app = effectiveActorApp(actor);
const app = actor.effectiveApp;
const username = actor.user?.username;
const pathScope =
app && typeof username === 'string' && username.length > 0
@@ -1070,32 +1199,6 @@ export class LegacyFSController extends PuterController {
);
res.status(range ? 206 : 200);
// Best-effort egress metering.
const metering = this.services.metering as
| {
batchIncrementUsages?: (
actor: unknown,
entries: unknown[],
) => void;
}
| undefined;
if (metering?.batchIncrementUsages && download.contentLength) {
download.body.once('end', () => {
try {
const bytes = download.contentLength!;
metering.batchIncrementUsages!(actor, [
{
usageType: 'filesystem:egress:bytes',
usageAmount: bytes,
costOverride:
FS_COSTS['filesystem:egress:bytes'] * bytes,
},
]);
} catch {
// ignore — non-critical.
}
});
}
download.body.on('error', (err) => {
res.destroy(err);
});
@@ -1125,9 +1228,20 @@ export class LegacyFSController extends PuterController {
assertNotSuspended(actor!.user);
assertVerifiedAccount(actor!.user);
req.actor = actor!;
req.actor = assertResolvedActor(actor!);
Context.set('actor', actor);
// And the budget gate the other read routes declare with
// `requireCredits`. The global auth probe only looks for `auth_token`,
// so `?token=` leaves `req.actor` unset for the whole gate chain and
// the declarative form would wave every request through — this streams
// file content like `/read` does, so it is refused on the same terms.
await assertActorHasCredits(
this.services.metering,
req.actor,
this.config,
);
// Forward back to regular read after setting actor
return this.read(req, res, undefined, { realMime: true });
};
@@ -1222,6 +1336,11 @@ export class LegacyFSController extends PuterController {
);
let finalAction: 'read' | 'write' = 'read';
if (action === 'write') {
// The real path, not the masked one: ACL reads the path
// string itself for its short-circuits (own home, an app's
// AppData under another user), and a mask hides the
// `AppData/<appUid>` shape those match on. Nothing here
// publishes the descriptor, so there is nothing to hide.
const writeOk = await this.services.acl.check(
actor,
{
@@ -1245,12 +1364,20 @@ export class LegacyFSController extends PuterController {
);
}
const signed = signEntry(entry, signingCfg);
const signed = signEntry(entry, signingCfg, {
actorUserId: actor.user?.id,
});
if (finalAction !== 'write') {
const { write_url: _, ...rest } = signed;
result.signatures.push({ ...rest, path: entry.path });
result.signatures.push({
...rest,
path: maskEntryPath(entry),
});
} else {
result.signatures.push({ ...signed, path: entry.path });
result.signatures.push({
...signed,
path: maskEntryPath(entry),
});
}
} catch {
// Silently skip unresolvable items.
@@ -1356,8 +1483,10 @@ export class LegacyFSController extends PuterController {
'outer.gui.item.added',
uploadResult.fsEntry,
);
const signed = signEntry(uploadResult.fsEntry, signingCfg);
res.json({ ...signed, path: uploadResult.fsEntry.path });
const signed = signEntry(uploadResult.fsEntry, signingCfg, {
actorUserId: callerActor.user?.id,
});
res.json({ ...signed, path: maskEntryPath(uploadResult.fsEntry) });
return;
}
@@ -1385,7 +1514,12 @@ export class LegacyFSController extends PuterController {
dedupeName: true,
});
await this.#emitGuiEvent('outer.gui.item.added', entry);
res.json({ ...signEntry(entry, signingCfg), path: entry.path });
res.json({
...signEntry(entry, signingCfg, {
actorUserId: callerActor.user?.id,
}),
path: maskEntryPath(entry),
});
return;
}
if (operation === 'rename') {
@@ -1402,9 +1536,18 @@ export class LegacyFSController extends PuterController {
targetEntry.path,
'write',
);
const renamed = await this.services.fs.rename(targetEntry, newName);
const renamed = await this.services.fs.rename(
userId,
targetEntry,
newName,
);
await this.#emitGuiEvent('outer.gui.item.updated', renamed);
res.json({ ...signEntry(renamed, signingCfg), path: renamed.path });
res.json({
...signEntry(renamed, signingCfg, {
actorUserId: callerActor.user?.id,
}),
path: maskEntryPath(renamed),
});
return;
}
if (operation === 'delete' || operation === 'trash') {
@@ -1472,7 +1615,12 @@ export class LegacyFSController extends PuterController {
? { old_path: targetEntry.path }
: undefined,
);
res.json({ ...signEntry(result, signingCfg), path: result.path });
res.json({
...signEntry(result, signingCfg, {
actorUserId: callerActor.user?.id,
}),
path: maskEntryPath(result),
});
return;
}
@@ -1514,14 +1662,38 @@ export class LegacyFSController extends PuterController {
});
}
// Name who this response's bytes are billed to. A signature authorises
// access to a file; it says nothing about who is asking, so an
// unidentified caller is billed to the account whose file it is —
// otherwise a signed URL is a way to serve content for free. A caller
// who did identify themselves pays for what they fetch, as on a hosted
// site.
if (owner?.uuid) {
req.egressActor = req.actor ?? {
user: {
uuid: owner.uuid,
id: owner.id,
username: owner.username,
suspended: !!(owner as { suspended?: unknown }).suspended,
},
};
}
// Directory: return a signed listing of direct children.
// The caller only proved read access, so strip write_url from
// each child to prevent privilege escalation via /writeFile.
//
// Child paths are the owner's real ones — there is no actor here to
// mask for. Accepted: a signature is minted and handed out by the
// owner, so the layout it reveals is theirs to reveal, and the child
// signatures expire; the paths are the only part that outlives them.
if (entry.isDir) {
const children = await this.services.fs.listDirectory(entry.uuid);
const signedChildren = children.map((child) => {
const { write_url: _, ...rest } = signEntry(child, signingCfg);
return { ...rest, path: child.path };
const { write_url: _, ...rest } = signEntry(child, signingCfg, {
ttlSeconds: NON_OWNER_SIGNATURE_TTL_SECONDS,
});
return { ...rest, path: maskEntryPath(child) };
});
res.json(signedChildren);
return;
@@ -1581,16 +1753,25 @@ export class LegacyFSController extends PuterController {
};
/**
* POST /open_item — resolve an entry, grant the default suggested app write
* POST /open_item — resolve an entry, grant the default suggested app
* access to it, and return a signed URL + user-app token so the launched
* app can read/write the file via its app-under-user token.
*
* Matches v1 semantics: permission is always granted as `write` — the
* underlying user's permission check still caps the effective access
* (grantUserAppPermission doesn't escalate user privileges).
* This is a user-authority action: it writes a user→app ACL row. Two things
* keep an app from opening an item to widen its own access — the caller
* must be the user themselves (the route gate, re-checked here because
* extensions can reach handlers directly), and the grant is capped at the
* access the caller actually proved on the entry.
*/
openItem = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
if ((actor as { app?: unknown }).app) {
throw new HttpError(
403,
'This endpoint is only available to user sessions',
{ legacyCode: 'forbidden' },
);
}
const body = asRecord(req.body);
const entry = await resolveV1Selector(this.stores.fsEntry, body);
@@ -1602,10 +1783,11 @@ export class LegacyFSController extends PuterController {
'read',
);
// Downgrade the envelope when the caller only proved read.
// `/writeFile`'s ACL re-check would still block the write, but
// Downgrade the envelope and the grant when the caller only proved
// read. `/writeFile`'s ACL re-check would still block the write, but
// returning `write_url` to a read-only caller is the same
// privilege-leak shape that `/sign` and `/readdir` strip.
// Real path — see the note on the other `acl.check` above.
const writeOk = await this.services.acl.check(
actor,
{
@@ -1619,7 +1801,7 @@ export class LegacyFSController extends PuterController {
const suggested =
(await this.services.suggestedApps?.getSuggestedApps({
name: entry.name,
path: entry.path,
path: maskEntryPath(entry),
})) ?? [];
let token: string | null = null;
@@ -1631,7 +1813,7 @@ export class LegacyFSController extends PuterController {
await this.services.permission.grantUserAppPermission(
actor,
defaultAppUid,
`fs:${entry.uuid}:write`,
`fs:${entry.uuid}:${writeOk ? 'write' : 'read'}`,
{},
{ reason: 'open_item' },
);
@@ -1642,12 +1824,14 @@ export class LegacyFSController extends PuterController {
}
const signingCfg = signingConfigFromAppConfig(this.config);
const signed = signEntry(entry, signingCfg);
const signed = signEntry(entry, signingCfg, {
actorUserId: actor.user?.id,
});
const signature = writeOk
? { ...signed, path: entry.path }
? { ...signed, path: maskEntryPath(entry) }
: (() => {
const { write_url: _, ...rest } = signed;
return { ...rest, path: entry.path };
return { ...rest, path: maskEntryPath(entry) };
})();
res.json({
signature,
@@ -1659,6 +1843,12 @@ export class LegacyFSController extends PuterController {
/**
* POST /auth/request-app-root-dir — an app-under-user requests stat on its
* own app root directory. The app must own itself.
*
* `check: true` answers whether the caller may claim it and stops there,
* for callers asking the question rather than acting on the answer —
* otherwise `puter.perms.check('appRootDir', …)` would provision a
* directory just by being asked. Answering is the whole response: a caller
* that may not claim it gets the 403 below either way.
*/
requestAppRootDir = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
@@ -1685,6 +1875,11 @@ export class LegacyFSController extends PuterController {
legacyCode: 'unauthorized',
});
if (getBoolean(body, 'check')) {
res.json({ allowed: true });
return;
}
const rootPath = `/${username}/AppData/${appUid}`;
// Auto-create the AppData/<uid> tree on first call.
const entry = await this.services.fs.mkdir(userId, {
@@ -1727,10 +1922,10 @@ export class LegacyFSController extends PuterController {
});
// Build an actor-under-user shape for the check.
const actorForApp = {
user: (req.actor as { user?: unknown }).user,
const actorForApp = makeActor({
user: req.actor!.user,
app: { uid: (app as { uid: string }).uid },
} as unknown as Actor;
});
const descriptor = {
path: subject.path,
resolveAncestors: () =>
@@ -2161,12 +2356,12 @@ export class LegacyFSController extends PuterController {
source.path,
'write',
);
await assertAccess(
await assertCanMoveInto(
this.services.acl,
this.services.fs,
actor,
destinationParent.path,
'write',
source,
destinationParent,
);
const moved = await this.services.fs.move(userId, {
source,
@@ -24,11 +24,13 @@ import {
getBoolean,
getString,
loadLegacyAssociatedApps,
signEntry,
signEntryThumbnail,
signingConfigFromAppConfig,
toLegacyEntry,
} from './legacyFsHelpers.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import { NON_OWNER_SIGNATURE_TTL_SECONDS } from '../../util/fileSigning.js';
const entryWithApp = (associatedAppId: number): FSEntry =>
({ associatedAppId }) as unknown as FSEntry;
@@ -444,3 +446,48 @@ describe('loadLegacyAssociatedApps short-circuit', () => {
expect(seen).toEqual([[4, 5]]);
});
});
describe('signEntry', () => {
const cfg = { secret: 'test-secret', apiBaseUrl: 'https://api.test' };
const entry = {
uuid: 'e-1',
name: 'x.txt',
isDir: false,
size: 1,
accessed: 0,
modified: 0,
created: 0,
userId: 7,
};
// signFile ceils its own clock read, so allow a second of slack.
const inSeconds = (expires: number) =>
expires - Math.ceil(Date.now() / 1000);
it('leaves an owner’s signature effectively permanent', () => {
const signed = signEntry(entry, cfg, { actorUserId: 7 });
expect(inSeconds(signed.expires)).toBeGreaterThan(
NON_OWNER_SIGNATURE_TTL_SECONDS * 10,
);
});
it('expires a signature over someone else’s entry', () => {
const signed = signEntry(entry, cfg, { actorUserId: 8 });
const ttl = inSeconds(signed.expires);
expect(ttl).toBeLessThanOrEqual(NON_OWNER_SIGNATURE_TTL_SECONDS);
expect(ttl).toBeGreaterThan(0);
});
it('keeps the old default when no actor is supplied', () => {
expect(inSeconds(signEntry(entry, cfg).expires)).toBeGreaterThan(
NON_OWNER_SIGNATURE_TTL_SECONDS * 10,
);
});
it('honours an explicit ttl over the ownership check', () => {
const signed = signEntry(entry, cfg, {
actorUserId: 7,
ttlSeconds: 30,
});
expect(inSeconds(signed.expires)).toBeLessThanOrEqual(30);
});
});
+66 -6
View File
@@ -30,10 +30,16 @@ import { HttpError } from '../../core/http/HttpError.js';
import {
resolveNode,
normalizeAbsolutePath,
isOwnersTrash,
joinChildPath,
expandTildePath,
} from '../../services/fs/resolveNode.js';
import {
maskEntryPath,
resolveSharePath,
} from '../../services/fs/sharePathMask.js';
import {
NON_OWNER_SIGNATURE_TTL_SECONDS,
signFile,
type SigningConfig,
type SignedFile,
@@ -93,6 +99,22 @@ export function getBoolean(
// username, read from the ALS-backed Context. Legacy clients send tilde-
// rooted paths (e.g. `~/AppData/<app-uid>/...`); FSController does the same
// expansion via its own `#normalizePath` helper.
/**
* Everything a client-authored path needs before it names a row: `~` expanded,
* and a masked share path turned back into the owner's real one.
*/
export async function expandClientPath(
fsEntryStore: FSEntryStore,
raw: string,
username?: string,
): Promise<string> {
return resolveSharePath(
fsEntryStore,
Context.get('actor'),
expandTildePath(raw, username),
);
}
export async function resolveV1Selector(
fsEntryStore: FSEntryStore,
raw: unknown,
@@ -106,7 +128,7 @@ export async function resolveV1Selector(
if (typeof raw === 'string') {
const isPath = raw.startsWith('/') || raw.startsWith('~');
const ref = isPath
? { path: expandTildePath(raw, username) }
? { path: await expandClientPath(fsEntryStore, raw, username) }
: { uid: raw };
const entry = await resolveNode(fsEntryStore, ref, { required: true });
if (!entry)
@@ -138,7 +160,7 @@ export async function resolveV1Selector(
const ref = {
path:
rawPath !== undefined
? expandTildePath(rawPath, username)
? await expandClientPath(fsEntryStore, rawPath, username)
: undefined,
uid:
typeof record.uid === 'string'
@@ -260,6 +282,24 @@ export async function assertCanCreate(
await assertAccess(aclService, fsService, actor, parentForCheck, 'write');
}
/** `write` on the destination parent, unless it is the entry's own Trash. */
export async function assertCanMoveInto(
aclService: ACLService,
fsService: FSService,
actor: Actor,
source: FSEntry,
destinationParent: FSEntry,
): Promise<void> {
if (isOwnersTrash(source, destinationParent)) return;
await assertAccess(
aclService,
fsService,
actor,
destinationParent.path,
'write',
);
}
// -- Response shaping ------------------------------------------------
type AppRowLookup = {
@@ -411,7 +451,10 @@ export async function toLegacyEntry(
appsById?: Map<number, Record<string, unknown>>;
} = {},
): Promise<Record<string, unknown>> {
const dirname = pathPosix.dirname(entry.path);
// Someone else's entry is published under its masked path; the owner's
// real one, and everything above the share, stays server-side.
const publishedPath = maskEntryPath(entry);
const dirname = pathPosix.dirname(publishedPath);
const mimeType = fsEntryMimeType(entry);
const pathComponents = entry.path.split('/');
@@ -424,7 +467,7 @@ export async function toLegacyEntry(
uuid: entry.uuid,
parent_id: entry.parentUid,
parent_uid: entry.parentUid,
path: entry.path,
path: publishedPath,
dirname,
dirpath: dirname,
name: entry.name,
@@ -520,7 +563,13 @@ export function signingConfigFromAppConfig(config: IConfig): SigningConfig {
return { secret, apiBaseUrl };
}
/** Convenience wrapper: turn an FSEntry into a signed-file response object. */
/**
* Convenience wrapper: turn an FSEntry into a signed-file response object.
*
* Pass `actorUserId` so a signature over someone else's entry — a shared file —
* expires rather than outliving the share (see
* NON_OWNER_SIGNATURE_TTL_SECONDS).
*/
export function signEntry(
entry: {
uuid: string;
@@ -530,8 +579,19 @@ export function signEntry(
accessed: number | null;
modified: number;
created: number | null;
userId?: number;
},
config: SigningConfig,
opts: { actorUserId?: number; ttlSeconds?: number } = {},
): SignedFile {
return signFile(entry as Parameters<typeof signFile>[0], config);
const isForeign =
typeof opts.actorUserId === 'number' &&
typeof entry.userId === 'number' &&
entry.userId !== opts.actorUserId;
const ttlSeconds =
opts.ttlSeconds ??
(isForeign ? NON_OWNER_SIGNATURE_TTL_SECONDS : undefined);
return signFile(entry as Parameters<typeof signFile>[0], config, {
...(ttlSeconds === undefined ? {} : { ttlSeconds }),
});
}
+163
View File
@@ -0,0 +1,163 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import * as limits from './limits.js';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
import type { RouteOptions, RouteRateLimit } from '../../core/http/types';
type Spec = RouteRateLimit | NonNullable<RouteOptions['concurrent']>;
const all = Object.entries(limits) as Array<[string, Spec | RouteRateLimit[]]>;
const flat: Array<[string, Spec]> = all.flatMap(([name, value]) =>
Array.isArray(value)
? value.map((v, i): [string, Spec] => [`${name}[${i}]`, v])
: [[name, value] as [string, Spec]],
);
const windows = flat.filter(([, s]) => 'window' in s) as Array<
[string, RouteRateLimit]
>;
const concurrents = flat.filter(([, s]) => !('window' in s));
describe('filesystem limit specs', () => {
// The whole point of this module is that the legacy and v2 controllers
// import the same specs. That only ties the counters together if every
// spec carries an explicit scope — without one the gate falls back to
// the route path, and the two controllers get separate budgets.
it.each(flat)('%s pins an explicit scope', (_name, spec) => {
expect(spec.scope).toBeTruthy();
});
it.each(windows)('%s has a positive limit and window', (_name, spec) => {
expect(spec.limit).toBeGreaterThan(0);
expect(spec.window).toBeGreaterThan(0);
});
// Base is the paid value; the free tiers are carved out beneath it.
// A free tier above the base would mean paying made you worse off.
it.each(windows)('%s never lets a free tier exceed paid', (_name, spec) => {
for (const n of Object.values(spec.bySubscription ?? {})) {
expect(n).toBeLessThanOrEqual(spec.limit);
}
});
it.each(windows)(
'%s caps temp at or below registered-free',
(_name, spec) => {
const free = spec.bySubscription?.[DEFAULT_FREE_SUBSCRIPTION];
const temp = spec.bySubscription?.[DEFAULT_TEMP_SUBSCRIPTION];
if (free === undefined || temp === undefined) return;
expect(temp).toBeLessThanOrEqual(free);
},
);
// A single in-flight slot turns incidental client parallelism into a
// spurious 429; paid tiers keep room to actually parallelise.
it.each(concurrents)(
'%s keeps concurrency at 5+ paid and 2+ for every tier',
(_name, spec) => {
expect(spec.limit).toBeGreaterThanOrEqual(5);
for (const n of Object.values(spec.bySubscription ?? {})) {
expect(n).toBeGreaterThanOrEqual(2);
}
},
);
it('gives search the tightest window of the read paths', () => {
expect(limits.FS_SEARCH_LIMIT.limit).toBeLessThan(
limits.FS_STAT_LIMIT.limit,
);
expect(limits.FS_SEARCH_LIMIT.limit).toBeLessThan(
limits.FS_READ_LIMIT.limit,
);
});
// The desktop deletes/moves one item per request with no batching and no
// pacing, so a single "empty trash" or "select all, delete" has to fit
// inside the minute window on every tier. The hour window is the abuse
// ceiling that a minute window this wide can no longer be.
it('gives mutations a bulk-sized minute window plus an hourly backstop', () => {
const [minute, hourly] = limits.FS_MUTATE_LIMIT;
expect(limits.FS_MUTATE_LIMIT).toHaveLength(2);
expect(minute.window).toBe(60_000);
expect(hourly.window).toBe(60 * 60_000);
const tiers = (spec: RouteRateLimit) => [
spec.limit,
spec.bySubscription![DEFAULT_FREE_SUBSCRIPTION],
spec.bySubscription![DEFAULT_TEMP_SUBSCRIPTION],
];
// A few hundred items clears in one pass, anonymous included.
for (const n of tiers(minute)) expect(n).toBeGreaterThanOrEqual(500);
// Anonymous stays meaningfully tighter than paid on both windows.
expect(tiers(minute)[2]).toBeLessThanOrEqual(minute.limit / 2);
expect(tiers(hourly)[2]).toBeLessThanOrEqual(hourly.limit / 2);
// The hour window has to bind rather than decorate: it allows a
// handful of bulk passes an hour, not sixty minutes' worth.
for (const [perMinute, perHour] of tiers(minute).map(
(n, i): [number, number] => [n, tiers(hourly)[i]],
)) {
expect(perHour).toBeGreaterThan(perMinute);
expect(perHour).toBeLessThanOrEqual(perMinute * 10);
}
});
// The DAV gate is consumed imperatively, before the request has an actor:
// it keys on the network fingerprint and reads only `limit` / `window`. A
// `key: 'user'` or a `bySubscription` map here would describe tiering that
// never happens.
it('shapes the DAV specs the way the DAV gate consumes them', () => {
expect(limits.DAV_LIMIT.key).toBe('fingerprint');
expect(limits.DAV_CONCURRENT.key).toBe('fingerprint');
expect(limits.DAV_LIMIT.bySubscription).toBeUndefined();
expect(limits.DAV_CONCURRENT.bySubscription).toBeUndefined();
});
// With no session to key on, the alternative is the bare address — and an
// address is a household, an office or a carrier gateway, so keying there
// makes one bucket serve everyone behind it and tighten as more real users
// arrive. The fingerprint separates clients within a network while still
// being something one client can't vary per request.
it('keys the signed-URL routes on the network fingerprint', () => {
expect(limits.FS_SIGNED_READ_LIMIT.key).toBe('fingerprint');
expect(limits.FS_SIGNED_WRITE_LIMIT.key).toBe('fingerprint');
expect(limits.FS_SIGNED_CONCURRENT.key).toBe('fingerprint');
});
// These serve page subresources — a gallery, an app's own assets — so the
// in-flight cap has to sit above what a browser opens to one origin at
// once, or a normal page load is what trips it.
it('leaves the signed-URL in-flight cap above a browser`s own parallelism', () => {
expect(limits.FS_SIGNED_CONCURRENT.limit).toBeGreaterThan(30);
});
it('uses distinct scopes so counters cannot collide', () => {
const scopes = flat
.filter(([, s]) => 'window' in s)
.map(([, s]) => s.scope);
expect(new Set(scopes).size).toBe(scopes.length);
});
});
+245
View File
@@ -0,0 +1,245 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
import type { RouteOptions, RouteRateLimit } from '../../core/http/types';
// -- Shared filesystem limits ----------------------------------------
//
// The v2 controller and the legacy controller expose the same operations
// on separate route tables. Both import the specs below so a caller can't
// get two budgets for one operation by switching endpoints — the `scope`
// is what ties the counters together, and it only works if both sides
// pass the same one.
//
// The base `limit` is what subscribed tiers see; `bySubscription` carves
// out the free tiers beneath it. Any plan id not enumerated (a Stripe
// plan, the dev-only `unlimited`) falls through to the base, so new plans
// are generous by default rather than accidentally throttled.
//
// Storage quota already bounds total bytes, and egress and object-store
// requests are metered. These bound request *count*, which neither does:
// metadata reads cost database time while costing the caller almost
// nothing, and metering settles seconds behind the traffic, so a limit is
// what actually stops a runaway loop in the moment.
/** Per-user sliding window. Free tiers are carved out of the paid base. */
const userWindow = (
scope: string,
paid: number,
free: number,
temp: number,
window = 60_000,
): RouteRateLimit => ({
scope,
limit: paid,
window,
key: 'user',
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: free,
[DEFAULT_TEMP_SUBSCRIPTION]: temp,
},
});
/**
* In-flight cap. Nothing drops below 2 — a single slot turns any incidental
* parallelism in a client (two tabs, a prefetch alongside a user action) into a
* spurious 429 — and subscribed tiers keep enough headroom to actually
* parallelise.
*/
const userConcurrent = (
scope: string,
paid: number,
free: number,
temp: number,
): NonNullable<RouteOptions['concurrent']> => ({
scope,
limit: paid,
key: 'user',
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: free,
[DEFAULT_TEMP_SUBSCRIPTION]: temp,
},
});
/**
* Per-network window, for the signature-authenticated routes with no session.
*
* `fingerprint` rather than `ip`: without an actor the only alternative is the
* bare address, and an address is an aggregate — a household, an office, a
* mobile carrier's gateway. Keying on it means one bucket for everyone behind
* it, which is a shared cap that gets tighter the more real users are present.
* The fingerprint folds in the headers a client can't vary per request without
* also looking like a different client, so separate browsers behind one NAT get
* separate buckets while a single client still can't mint fresh ones per call.
*/
const networkWindow = (
scope: string,
limit: number,
window = 60_000,
): RouteRateLimit => ({ scope, limit, window, key: 'fingerprint' });
// -- Metadata reads --------------------------------------------------
/** Chattiest call in the GUI; high enough to only ever catch a loop. */
export const FS_STAT_LIMIT = userWindow('fs:stat', 1200, 600, 300);
/**
* Desktop boot fans out hard, so the minute budget is generous — the short
* second window is what actually catches a runaway loop before it has spent the
* whole minute's allowance.
*/
export const FS_READDIR_LIMIT: RouteRateLimit[] = [
userWindow('fs:readdir', 600, 300, 120),
userWindow('fs:readdir-burst', 120, 60, 30, 10_000),
];
/**
* Unindexed scan across the user's tree, and close to unmetered — a result set
* is a few hundred bytes of egress against an arbitrary amount of database
* work, so this is the cheapest way to occupy a connection. Tightest limit in
* the file.
*/
export const FS_SEARCH_LIMIT = userWindow('fs:search', 60, 30, 10);
export const FS_SEARCH_CONCURRENT = userConcurrent('fs:search', 5, 2, 2);
/** Aggregation over the whole tree; the GUI needs it rarely. */
export const FS_DF_LIMIT = userWindow('fs:df', 60, 30, 15);
// -- Content transfer ------------------------------------------------
/**
* Egress is billed, so cost is already bounded — the cap here is against
* connection exhaustion, which is why the concurrency slot matters more than
* the window.
*/
export const FS_READ_LIMIT = userWindow('fs:read', 600, 300, 120);
export const FS_READ_CONCURRENT = userConcurrent('fs:read', 10, 5, 3);
export const FS_WRITE_LIMIT = userWindow('fs:write', 300, 120, 30);
export const FS_WRITE_CONCURRENT = userConcurrent('fs:write', 15, 6, 3);
/**
* Multipart handshake — several calls per upload, so a multiple of the write
* budget rather than a peer of it. Signing is the half that costs us
* object-store calls.
*
* The multiple is what matters: a per-minute ceiling here divides down into a
* much smaller number of files, and selecting a folder's worth of them is one
* gesture. `FS_WRITE_LIMIT` and storage quota already bound what actually gets
* written, so this only has to stay out of the way of a real upload.
*/
export const FS_MULTIPART_LIMIT = userWindow('fs:multipart', 2400, 1200, 600);
// -- Metadata mutations ----------------------------------------------
/**
* Mkdir / touch / rename / delete / move / copy / mkshortcut.
*
* The desktop issues one call per item and does not pace them: emptying the
* trash, deleting a multi-selection, or dragging a folder's worth of files
* fires the whole set back to back. A minute budget in the low hundreds turns
* an ordinary "select all, delete" into a partial failure, so the minute window
* is sized to clear a bulk pass over a few hundred items on every tier — an
* anonymous session gets a real desktop, so it needs a real bulk allowance too,
* just a smaller one.
*
* The hour window is where the abuse ceiling actually lives. It allows a few of
* those bulk passes per hour, which is more than a person driving a file
* manager will ever need and well under what a script grinding metadata writes
* would want.
*/
export const FS_MUTATE_LIMIT: RouteRateLimit[] = [
userWindow('fs:mutate', 1200, 900, 600),
userWindow('fs:mutate-sustained', 6000, 3000, 1800, 60 * 60_000),
];
/** Mints a URL that outlives the request, so worth its own budget. */
export const FS_SIGN_LIMIT = userWindow('fs:sign', 300, 150, 60);
/** Low-frequency GUI helpers. */
export const FS_HELPER_LIMIT = userWindow('fs:helper', 120, 60, 30);
/**
* Puter.js polls this on a timer to decide whether to purge its FS cache. Same
* ceiling for free and paid — it is a single cache read.
*/
export const FS_POLL_LIMIT = userWindow('fs:poll', 240, 240, 120);
// -- Legacy multipart upload -----------------------------------------
/**
* `/batch` buffers every file fully into memory before any quota or storage
* check runs, up to BATCH_MAX_FILES × BATCH_MAX_FILE_SIZE. The concurrency slot
* is doing the real work here; the window is secondary, and sized to say so —
* one upload is one call, so a per-minute ceiling in the tens is a cap on how
* many files someone may upload rather than a bound on cost. What actually
* bounds the memory this route can tie up is how many run at once.
*/
export const FS_BATCH_LIMIT = userWindow('fs:batch', 600, 300, 300);
export const FS_BATCH_CONCURRENT = userConcurrent('fs:batch', 5, 2, 2);
// -- Signed-URL routes (no session to key on) ------------------------
//
// The URL's own signature is what authorizes these; the limits below only
// bound what an unsigned flood can cost. Sized for what the routes are
// actually used for, which is content the browser fetches as a subresource:
// a gallery, a document's images, an app loading its own assets. A page
// opening a few dozen of those at once is ordinary, and every rejection here
// is a broken image rather than a slow one — so the ceiling clears a burst of
// real page loads and only catches something looping.
export const FS_SIGNED_READ_LIMIT = networkWindow('fs:signed-read', 3_000);
export const FS_SIGNED_WRITE_LIMIT = networkWindow('fs:signed-write', 600);
export const FS_SIGNED_CONCURRENT: NonNullable<RouteOptions['concurrent']> = {
scope: 'fs:signed',
// Above what a browser will open to one origin at once, so the in-flight
// cap never decides the outcome for a single client — it is there for a
// client that opens connections without closing them.
limit: 60,
key: 'fingerprint',
};
// -- WebDAV ----------------------------------------------------------
/**
* One `router.use` fronts the whole DAV surface, so a single gate there covers
* every verb. Desktop DAV clients are bursty — a lower ceiling shows up as
* spurious failures in Finder / Explorer.
*
* Unlike everything above, the DAV gate runs before the request is
* authenticated: there is no actor yet, so there is no subscription to resolve
* a tier against and one ceiling applies to every caller. These are shaped to
* say that — keyed on the network fingerprint the gate buckets on, with no
* `bySubscription` map to imply a tiering that never happens.
*/
export const DAV_LIMIT: RouteRateLimit = {
scope: 'dav',
limit: 600,
window: 60_000,
key: 'fingerprint',
};
export const DAV_CONCURRENT: NonNullable<RouteOptions['concurrent']> = {
scope: 'dav',
limit: 10,
key: 'fingerprint',
};
@@ -19,6 +19,10 @@
import { HttpError } from '../../core/http/HttpError.js';
import { PuterController } from '../types.js';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
/**
* Site hosting endpoints. Listing and create/update are not exposed as
@@ -46,6 +50,18 @@ export class HostingController extends PuterController {
requireUserActor: true,
allowFullAccessToken: true,
requireVerified: true,
// Destructive, and pairs with the `subdomains:create`
// budget on the driver side.
rateLimit: {
scope: 'delete-site',
limit: 60,
window: 60_000,
key: 'user',
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 30,
[DEFAULT_TEMP_SUBSCRIPTION]: 10,
},
},
},
async (req, res) => {
const { site_uuid } = req.body ?? {};
+2
View File
@@ -18,6 +18,7 @@
*/
import { AppController } from './apps/AppController.js';
import { AppFeedbackController } from './feedback/AppFeedbackController.js';
import { AuthController } from './auth/AuthController.js';
import { BroadcastController } from './broadcast/BroadcastController.js';
import { DesktopController } from './desktop/DesktopController.js';
@@ -43,6 +44,7 @@ export const puterControllers = {
staticPages: StaticPagesController,
auth: AuthController,
apps: AppController,
appFeedback: AppFeedbackController,
desktop: DesktopController,
hosting: HostingController,
system: SystemController,
@@ -41,6 +41,14 @@ export class NotificationController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
// Fires per notification interaction, so the ceiling stays
// generous — it is here to catch a loop, not to pace a user.
rateLimit: {
scope: 'notification-mark',
limit: 300,
window: 60_000,
key: 'user',
},
})
async markAck(req: Request, res: Response): Promise<void> {
const uid = req.body?.uid;
@@ -56,8 +64,7 @@ export class NotificationController extends PuterController {
});
const notifService = this.services.notification as unknown as
| NotificationService
| undefined;
NotificationService | undefined;
if (notifService?.markAcknowledged) {
await notifService.markAcknowledged(uid, userId);
} else {
@@ -84,6 +91,14 @@ export class NotificationController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
// Fires per notification interaction, so the ceiling stays
// generous — it is here to catch a loop, not to pace a user.
rateLimit: {
scope: 'notification-mark',
limit: 300,
window: 60_000,
key: 'user',
},
})
async markRead(req: Request, res: Response): Promise<void> {
const uid = req.body?.uid;
@@ -99,8 +114,7 @@ export class NotificationController extends PuterController {
});
const notifService = this.services.notification as unknown as
| NotificationService
| undefined;
NotificationService | undefined;
if (notifService?.markShown) {
await notifService.markShown(uid, userId);
} else {
@@ -1746,3 +1746,35 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
expect((await redeem(proof)).body).toMatchObject({ oidc_login: false });
});
});
// ── rate-limit scopes ───────────────────────────────────────────────
describe('OIDCController rate limits', () => {
const rateLimitOf = (method: string, path: string) => {
const route = router.routes.find(
(r) => r.method === method && r.path === path,
);
if (!route) throw new Error(`No ${method.toUpperCase()} ${path} route`);
return route.options.rateLimit as { scope: string; limit: number };
};
it('keeps the revalidate landing page off the identity-provider bucket', () => {
// `/auth/revalidate-done` serves a constant HTML page; the start and
// callback routes exchange codes with an identity provider. Sharing
// a scope made the cheap page inherit the flow routes' tight ceiling,
// which one shared address can exhaust on its own.
const done = rateLimitOf('get', '/auth/revalidate-done');
const start = rateLimitOf('get', '/auth/oidc/:provider/start');
expect(done.scope).not.toBe(start.scope);
expect(done.scope).toBe('oidc-revalidate-done');
expect(done.limit).toBeGreaterThan(start.limit);
});
it('sizes the public provider list for a shared address, not a fleet', () => {
// Unauthenticated and keyed on IP, so the bucket covers every client
// behind one NAT or campus, not one browser. It is still login
// surface, so it stays bounded well below the ceilings given to
// static reads like icons or version info.
expect(rateLimitOf('get', '/auth/oidc/providers').limit).toBe(1_200);
});
});
+53 -4
View File
@@ -228,10 +228,26 @@ export class OIDCController extends PuterController {
// -- GET /auth/oidc/providers --------------------------------
// Public — list enabled provider IDs for the frontend.
//
// Every render of a login form reads this, and with no actor the
// bucket is the address: one corporate egress, campus or carrier
// gateway stands for every person behind it. Size it for that — a
// large shared address is thousands of people, and a shift or class
// change bunches their sign-ins into the same minute — but no
// further. This is login surface, so the ceiling should still bound
// someone enumerating which identity providers a deployment accepts.
router.get(
'/auth/oidc/providers',
{ subdomain: 'api' },
{
subdomain: 'api',
rateLimit: {
scope: 'oidc-providers',
limit: 1_200,
window: 60_000,
key: 'ip',
},
},
async (_req: Request, res: Response) => {
const providers =
await this.services.oidc.getEnabledProviderIds();
@@ -584,10 +600,26 @@ export class OIDCController extends PuterController {
// -- GET /auth/revalidate-done -------------------------------
// Landing page after revalidation; posts to opener for popup flow.
//
// Deliberately not on the `oidc-general` bucket the start/callback
// routes share: those exchange codes with an identity provider, this
// one is a constant HTML page that touches nothing. Sharing a bucket
// meant everyone reachable through one address — an office, a school,
// a carrier gateway — competed for the same 30 popup closes a minute.
// The replacement is sized for the humans behind one such address
// re-validating at once, not for a fleet: it is still auth surface,
// and one page view per revalidation is a low-volume event.
router.get(
'/auth/revalidate-done',
{ subdomain: '' },
{
subdomain: '',
rateLimit: {
scope: 'oidc-revalidate-done',
limit: 600,
window: 60_000,
},
},
(_req: Request, res: Response) => {
const origin = this.config.origin ?? '';
res.set('Content-Type', 'text/html; charset=utf-8');
@@ -629,6 +661,7 @@ if (window.opener) {
provider: string,
userinfo: { sub: string; email?: unknown; [k: string]: unknown },
referrer?: string | null,
attempt = 0,
): Promise<
| { error: string; code?: string; requestCode?: string }
| {
@@ -647,8 +680,12 @@ if (window.opener) {
const claimedEmail =
typeof userinfo.email === 'string' ? userinfo.email : null;
if (claimedEmail) {
const byEmail =
await this.services.oidc.findUserByEmail(claimedEmail);
// On the retry after a lost race, read the primary — the winning
// row may be younger than the replica snapshot.
const byEmail = await this.services.oidc.findUserByEmail(
claimedEmail,
{ force: attempt > 0 },
);
if (byEmail) {
if (!byEmail.email_confirmed) {
return {
@@ -675,6 +712,18 @@ if (window.opener) {
userinfo as { sub: string; email?: string },
referrer,
);
// A concurrent callback (a second tab, a provider retry) created the
// account between step 2 and the insert. Nothing went wrong for the
// user — start over and we'll find the winner at step 1 or 2. One retry
// only: a second miss means something other than a race is going on.
if (outcome.raced && attempt === 0) {
return this.#resolveOrCreateOIDCUser(
provider,
userinfo,
referrer,
attempt + 1,
);
}
if (!outcome.success || !outcome.user) {
return {
error: outcome.error ?? 'Account creation failed.',
@@ -438,6 +438,7 @@ describe('PeerController TURN', () => {
id: created.id,
username: created.username,
},
effectiveApp: null,
});
} finally {
meterSpy.mockRestore();
+51 -6
View File
@@ -19,11 +19,15 @@
import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
import type { Request, Response } from 'express';
import type { Actor } from '../../core/actor.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterController } from '../types.js';
import { PEER_COSTS } from './costs.js';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
/**
* Constant-time secret comparison for the internal-auth header. HMAC both sides
@@ -103,17 +107,58 @@ export class PeerController extends PuterController {
registerRoutes(router: PuterRouter): void {
router.get(
'/peer/signaller-info',
{ subdomain: 'api' },
{
subdomain: 'api',
// Public config read — the signaller URL and the fallback
// ICE list, both deploy constants. Unauthenticated, so the
// key is the address, and one address covers every client
// on that network; a peer session starts with this call, so
// the bucket has to hold a whole network's sessions. Nothing
// here is secret or expensive, so the ceiling only bounds a
// client stuck re-reading it.
rateLimit: {
scope: 'peer-signaller-info',
limit: 3_000,
window: 60_000,
key: 'ip',
},
},
this.#signallerInfo,
);
router.post(
'/peer/generate-turn',
{ subdomain: 'api', requireAuth: true },
{
subdomain: 'api',
requireAuth: true,
// Every call reaches the upstream TURN API and mints
// credentials against a paid allocation, so this is a
// spend limit as much as an abuse limit.
rateLimit: {
scope: 'peer-generate-turn',
limit: 30,
window: 60_000,
key: 'user',
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 10,
[DEFAULT_TEMP_SUBSCRIPTION]: 5,
},
},
},
this.#generateTurn,
);
router.post(
'/turn/ingest-usage',
{ subdomain: 'api' },
{
subdomain: 'api',
// Shared-secret authenticated, so this only bounds how
// fast someone can guess the secret.
rateLimit: {
scope: 'turn-ingest',
limit: 60,
window: 60_000,
key: 'ip',
},
},
this.#ingestUsage,
);
}
@@ -217,13 +262,13 @@ export class PeerController extends PuterController {
if (!user) continue;
const costInMicrocents =
egressBytes * PEER_COSTS['turn:egress-bytes'];
const actor = {
const actor = makeActor({
user: {
uuid: user.uuid,
id: user.id,
username: user.username,
},
};
});
await this.services.metering.incrementUsage(
actor,
'turn:egress-bytes',
@@ -0,0 +1,157 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
/**
* The vendor-compatible wire routes are subscriber-only, over real HTTP.
*
* The gate's decision is unit-tested in
* `services/metering/enforcement.test.ts`; what this covers is that the route
* option reaches the middleware chain on these four routes, that it sits behind
* the credential gates (a session token is still turned away for being the
* wrong credential, not for the plan), and that the model catalogue stays open
* — clients fetch it before they have any reason to have authenticated at all.
*/
describe('AI wire routes require a subscription', () => {
let env: PuterTestEnv;
const WIRE_ROUTES = [
'/puterai/openai/v1/chat/completions',
'/puterai/openai/v1/completions',
'/puterai/openai/v1/responses',
'/puterai/anthropic/v1/messages',
];
/**
* The plan the test user is on. A single resolver reads this, since
* resolvers are registered for the life of the service and can't be taken
* back off — flipping a variable is what keeps one test's plan out of the
* next one's.
*/
let plan: string | null = null;
let userUuid = '';
beforeAll(async () => {
env = await setupPuterTestEnv();
const row = await env.server.stores.user.getByUsername(
env.users.user.username,
);
userUuid = row!.uuid;
// Paid plans are registered by an extension in production, so a
// resolver alone resolves to nothing here — the policy has to exist
// for the id to be found.
env.server.services.metering.registerPolicy({
id: 'business',
monthUsageAllowance: 45 * 1_000_000 * 100,
monthlyStorageAllowance: 1024 ** 3,
});
env.server.services.metering.registerSubscriptionResolver((actor) =>
actor.user?.uuid === userUuid ? plan : null,
);
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const call = (route: string, token: string) =>
fetch(new URL(route, env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
Origin: env.apiOrigin,
},
body: JSON.stringify({
model: 'fake',
provider: 'fake-chat',
prompt: 'hi',
messages: [{ role: 'user', content: 'hi' }],
max_tokens: 16,
}),
});
/** Put the test user on a plan for the duration of one block. */
const withSubscription = async <T>(
id: string | null,
fn: () => Promise<T>,
): Promise<T> => {
const metering = env.server.services.metering;
plan = id;
metering.invalidateActorSubscription(userUuid);
try {
return await fn();
} finally {
plan = null;
metering.invalidateActorSubscription(userUuid);
}
};
it('turns a free account away with 402 subscription_required', async () => {
for (const route of WIRE_ROUTES) {
const res = await call(route, env.users.user.apiToken);
expect(res.status, route).toBe(402);
expect(await res.json(), route).toMatchObject({
code: 'subscription_required',
});
}
});
it('lets a subscriber past the gate', async () => {
await withSubscription('business', async () => {
for (const route of WIRE_ROUTES) {
const res = await call(route, env.users.user.apiToken);
// Whether the fake provider answers is the controller's
// business; all this asserts is that the plan gate is not
// what stopped the request.
expect(res.status, route).not.toBe(402);
}
});
});
it('answers a session token on credential shape, not plan', async () => {
// Credential gates run first, so a caller with the wrong token shape
// never learns anything about the account's plan.
const res = await call(WIRE_ROUTES[0]!, env.users.user.token);
expect(res.status).toBe(403);
expect(await res.json()).toMatchObject({
code: 'app_or_api_token_required',
});
});
it('leaves the model catalogue open', async () => {
const res = await fetch(new URL('/puterai/chat/models', env.apiOrigin));
expect(res.status).toBe(200);
});
it('leaves the video proxy outside the plan gate', async () => {
// Unsigned, so this is turned away on the signature — the point is
// that a free account gets that answer rather than a 402. The proxy
// delivers a video the account already paid to generate.
const res = await fetch(
new URL(
'/puterai/video/proxy?url=https://x.test/v.mp4',
env.apiOrigin,
),
);
expect(res.status).not.toBe(402);
});
});
@@ -195,7 +195,9 @@ describe('PuterAIController.registerRoutes', () => {
// `requireUserActor` keeps apps out, `allowFullAccessToken` admits
// the PAT, and `noUserSession` rejects the account session ("root")
// token. Each route also carries `requireVerified` (fresh accounts
// must confirm their email before using the AI wire surface) and
// must confirm their email before using the AI wire surface),
// `requireSubscription` (the vendor-compatible surface is a paid
// feature; free accounts reach the same models via /drivers/call) and
// the shared per-tier AI rate-limit / concurrency policy — these
// routes bypass the `/drivers/call` dispatch (where the
// driver-declared limits are enforced), so without the route gates
@@ -214,6 +216,7 @@ describe('PuterAIController.registerRoutes', () => {
allowFullAccessToken: true,
noUserSession: true,
requireVerified: true,
requireSubscription: true,
rateLimit: {
...AI_RATE_LIMIT.default,
scope: 'driver:puter-chat-completion:complete',
@@ -229,12 +232,53 @@ describe('PuterAIController.registerRoutes', () => {
const modelsRoute = calls.find(
(c) => c.path === '/puterai/chat/models',
);
// Unauthenticated, so the limit keys on IP rather than an actor —
// which makes the bucket an aggregate over every client behind that
// address, hence a ceiling sized for a network rather than a browser.
expect(modelsRoute?.opts).toEqual({
subdomain: 'api',
requireAuth: false,
rateLimit: {
scope: 'puterai-models',
limit: 3_000,
window: 60_000,
key: 'ip',
},
});
});
it('leaves the catalogue and the video proxy outside the plan gate', () => {
const calls: Array<{ path: string; opts: RouteOptions }> = [];
const router = {
post: vi.fn(() => router),
get: vi.fn((path: string, opts: RouteOptions) => {
calls.push({ path, opts });
return router;
}),
};
controller.registerRoutes(router as never);
// The catalogue is fetched before a client has any reason to
// authenticate, and the proxy streams a video the account already
// paid to generate to whoever holds the signed link — neither has a
// plan to read, let alone one to charge for.
const openPaths = [
'/puterai/chat/models',
'/puterai/chat/models/details',
'/puterai/image/models',
'/puterai/image/models/details',
'/puterai/video/models',
'/puterai/video/models/details',
'/puterai/video/proxy',
];
for (const path of openPaths) {
const route = calls.find((c) => c.path === path);
expect(route, path).toBeDefined();
expect(route?.opts.requireSubscription, path).toBeUndefined();
}
});
it('keys the proxy-route AI limits by user uuid so they share the /drivers/call buckets', () => {
const calls: Array<{ path: string; opts: RouteOptions }> = [];
const router = {
@@ -49,7 +49,11 @@ const GEMINI_DOWNLOAD_BASE =
* All routes live on `subdomain: 'api'` and require a full-access API token
* minted from the dashboard (user-scoped worker tokens also pass — workers are
* never treated as root tokens). Apps, scoped tokens, and account session
* ("root") tokens are rejected.
* ("root") tokens are rejected. The vendor-compatible routes additionally
* require the account to be on a paid plan. The model listings stay open —
* clients fetch the catalogue before they have any reason to authenticate — and
* so does the video proxy, whose signed URLs are handed out by a generation the
* account already paid for.
*/
export class PuterAIController extends PuterController {
registerRoutes(router: PuterRouter): void {
@@ -79,6 +83,13 @@ export class PuterAIController extends PuterController {
allowFullAccessToken: true,
noUserSession: true,
requireVerified: true,
// The vendor-compatible wire surface is a paid feature: it is
// what makes Puter a drop-in for a metered vendor API, and an
// account on a free plan reaches the same models through
// `/drivers/call` (and puter.js) without it. Free accounts get
// 402 `subscription_required` here rather than a rate limit,
// so the answer is "upgrade", not "slow down".
requireSubscription: true,
rateLimit: {
...AI_RATE_LIMIT.default!,
scope: aiPolicyScope,
@@ -90,7 +101,23 @@ export class PuterAIController extends PuterController {
key: aiPolicyKey,
},
} as RouteOptions;
const publicOpts = { subdomain: 'api', requireAuth: false } as const;
// Model listings are unauthenticated, so the only key available is
// the address — which is an aggregate, not a user: a NAT, a school,
// a mobile carrier gateway or a server-side renderer all arrive as
// one address, and the SDK and GUI both fetch the catalogue on
// startup. The ceiling therefore has to cover a whole network's page
// loads. What it still protects is the serialisation cost of the
// catalogue under a client stuck in a fetch loop.
const publicOpts = {
subdomain: 'api',
requireAuth: false,
rateLimit: {
scope: 'puterai-models',
limit: 3_000,
window: 60_000,
key: 'ip',
},
} as RouteOptions;
// Every route below carries the `/puterai` prefix for wire
// compatibility with puter-js and existing API tests.
@@ -150,10 +177,28 @@ export class PuterAIController extends PuterController {
// -- Video URL proxy -----------------------------------------
// Reverse-proxies AI-generated video URLs that can't be given
// directly to the client (auth-gated provider downloads). The
// URL itself is HMAC-signed, so no additional auth gate.
// URL itself is HMAC-signed, so no additional auth gate — and no
// plan gate either: it delivers a video the account already paid to
// generate, to whoever holds the link.
router.get(
'/puterai/video/proxy',
{ subdomain: 'api' },
{
subdomain: 'api',
// HMAC-signed but unauthenticated, and it streams provider
// bandwidth through us — so the in-flight cap matters as
// much as the window.
rateLimit: {
scope: 'puterai-video-proxy',
limit: 60,
window: 60_000,
key: 'ip',
},
concurrent: {
scope: 'puterai-video-proxy',
limit: 5,
key: 'ip',
},
},
this.#videoProxy,
);
}
@@ -542,8 +587,7 @@ export class PuterAIController extends PuterController {
text: extractTextContent(
(
messageResult.message as
| Record<string, unknown>
| undefined
Record<string, unknown> | undefined
)?.content,
),
index: 0,
@@ -0,0 +1,554 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
/**
* Route-level coverage for the sharing endpoints. The service unit tests drive
* the semantics; this suite exists to catch a route that was never registered,
* a gate that rejects a legitimate request, and anything the response shape
* leaks.
*/
describe('share endpoints over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const post = (path: string, token: string, body: unknown) =>
fetch(new URL(path, env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
const get = (path: string, token: string, params: Record<string, string>) => {
const url = new URL(path, env.apiOrigin);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
return fetch(url, { headers: { authorization: `Bearer ${token}` } });
};
/** A file in the owner's home. Written directly — these tests are about
* the share routes, not the upload path. */
const makeFile = async (owner: { username: string }) => {
const uid = crypto.randomUUID();
const name = `share-http-${uid.slice(0, 8)}.txt`;
const path = `/${owner.username}/${name}`;
const user = await env.server.stores.user.getByUsername(owner.username);
await env.server.clients.db.write(
'INSERT INTO `fsentries` (`uuid`, `name`, `path`, `user_id`, `is_dir`, `modified`) VALUES (?, ?, ?, ?, 0, ?)',
[uid, name, path, user!.id, Math.floor(Date.now() / 1000)],
);
return { uid, path, name };
};
// Masking hides where an item sits, not who may open it.
it('will not let a masked path reach an unshared sibling', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const shared = await makeFile(owner);
const secret = await makeFile(owner);
await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: shared.uid }],
mode: 'read',
});
const read = (path: string) =>
fetch(
`${env.apiOrigin}/read?${new URLSearchParams({ file: path })}`,
{
headers: {
authorization: `Bearer ${recipient.token}`,
origin: env.apiOrigin,
},
},
);
const masked = `/${owner.username}/${shared.uid}/${shared.name}`;
expect((await read(masked)).status).toBe(200);
for (const attempt of [
// The shared item's root, renamed to the sibling.
`/${owner.username}/${shared.uid}/${secret.name}`,
// The sibling's own uuid, as if it had leaked.
`/${owner.username}/${secret.uid}/${secret.name}`,
// Back out of the root the uuid vouched for.
`/${owner.username}/${shared.uid}/${shared.name}/../${secret.name}`,
// The owner's real path, named outright.
secret.path,
]) {
const res = await read(attempt);
expect(res.status, `reachable: ${attempt}`).not.toBe(200);
}
});
it('shares an item, lists it for the recipient, then revokes it', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const file = await makeFile(owner);
const shareRes = await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(shareRes.status).toBe(200);
const shareBody = (await shareRes.json()) as {
status: string;
results: Array<{ status: string; mode?: string; name?: string }>;
};
expect(shareBody.status).toBe('success');
expect(shareBody.results[0].mode).toBe('read');
expect(shareBody.results[0].name).toBe(file.name);
const listRes = await get(
'/share/shared-with-me',
recipient.token,
{ includeTotal: 'true' },
);
expect(listRes.status).toBe(200);
const listed = (await listRes.json()) as {
items: Array<Record<string, unknown>>;
total?: number;
};
const row = listed.items.find((i) => i.uid_entry === file.uid);
expect(row).toBeDefined();
expect(row?.issuer).toBe(owner.username);
expect(row?.mode).toBe('read');
expect(typeof listed.total).toBe('number');
// Nothing internal rides along in the response.
for (const key of ['issuer_user_id', 'holder_user_id', 'fsentry_id']) {
expect(row).not.toHaveProperty(key);
}
const revokeRes = await post('/share/revoke', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
});
expect(revokeRes.status).toBe(200);
expect(await revokeRes.json()).toMatchObject({ revoked: 1 });
const afterRes = await get('/share/shared-with-me', recipient.token, {});
const after = (await afterRes.json()) as {
items: Array<Record<string, unknown>>;
};
expect(after.items.find((i) => i.uid_entry === file.uid)).toBeUndefined();
});
it('revokes every item in the request, not just the first', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const fileA = await makeFile(owner);
const fileB = await makeFile(owner);
for (const file of [fileA, fileB]) {
const res = await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(res.status).toBe(200);
}
// A truncated revoke is a silent security failure: the caller is told
// "success" while items after the first keep their grants.
const revokeRes = await post('/share/revoke', owner.token, {
recipients: [recipient.username],
items: [{ uid: fileA.uid }, { uid: fileB.uid }],
});
expect(revokeRes.status).toBe(200);
expect(await revokeRes.json()).toMatchObject({
status: 'success',
revoked: 2,
});
const afterRes = await get('/share/shared-with-me', recipient.token, {});
const after = (await afterRes.json()) as {
items: Array<Record<string, unknown>>;
};
for (const file of [fileA, fileB]) {
expect(
after.items.find((i) => i.uid_entry === file.uid),
).toBeUndefined();
}
});
it('accepts tilde-rooted paths the way the FS routes do', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const file = await makeFile(owner);
const tildePath = `~/${file.path.split('/').pop()}`;
const shareRes = await post('/share', owner.token, {
recipients: [recipient.username],
items: [tildePath],
mode: 'read',
});
expect(shareRes.status).toBe(200);
expect(await shareRes.json()).toMatchObject({ status: 'success' });
const listRes = await get('/share/shares', owner.token, {
path: tildePath,
});
expect(listRes.status).toBe(200);
const listed = (await listRes.json()) as {
items: Array<{ holder: string }>;
};
expect(
listed.items.some((i) => i.holder === recipient.username),
).toBe(true);
const revokeRes = await post('/share/revoke', owner.token, {
recipients: [recipient.username],
items: [tildePath],
});
expect(revokeRes.status).toBe(200);
expect(await revokeRes.json()).toMatchObject({ revoked: 1 });
});
it('tells the recipient over the wire, once for the whole batch', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const [a, b] = [await makeFile(owner), await makeFile(owner)];
// These accounts are shared between tests, so both things that would
// otherwise decide this outcome are cleared first: the budgets that
// silence a repeat interruption, and any notification still open for
// this recipient, which a new share folds into instead of creating one.
// What's left under test is the batching.
const [issuer, holder] = await Promise.all([
env.server.stores.user.getByUsername(owner.username),
env.server.stores.user.getByUsername(recipient.username),
]);
await env.server.clients.redis.del(
`rate:share:notify:pair:${issuer!.id}:${holder!.id}`,
`rate:share:notify:pair-day:${issuer!.id}:${holder!.id}`,
`rate:share:notify:to:${holder!.id}`,
`rate:share:notify:to-day:${holder!.id}`,
);
for (const row of await env.server.stores.notification.listByUserId(
holder!.id,
{ filter: 'unacknowledged' },
)) {
await env.server.stores.notification.markAcknowledged(
row.uid,
holder!.id,
);
}
const seen: Array<{ userIds: number[]; payload: Record<string, unknown> }> = [];
const notification = env.server.services.notification;
const original = notification.notify.bind(notification);
notification.notify = async (userIds, payload) => {
seen.push({ userIds, payload });
return original(userIds, payload);
};
try {
const res = await post('/share', owner.token, {
items: [{ path: a.path }, { path: b.path }],
recipients: [{ username: recipient.username }],
mode: 'read',
});
expect(res.status).toBe(200);
// Delivery is off the response path, so it may land just after.
await vi.waitFor(() => expect(seen.length).toBeGreaterThan(0), {
timeout: 5000,
});
} finally {
notification.notify = original;
}
// Two items, one recipient — one notification carrying the count.
expect(seen).toHaveLength(1);
expect(seen[0].payload).toMatchObject({
source: 'sharing',
fields: { count: 2 },
});
});
it('reports per-pair outcomes when only some recipients resolve', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const res = await post('/share', owner.token, {
recipients: [env.users.other.username, 'nosuchuser-zzz'],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(res.status).toBe(200);
const body = (await res.json()) as {
status: string;
results: Array<{ status: string; recipient: string }>;
};
expect(body.status).toBe('mixed');
expect(body.results).toHaveLength(2);
expect(
body.results.find((r) => r.recipient === 'nosuchuser-zzz')?.status,
).toBe('error');
});
it('lists who can reach an item for its owner', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const file = await makeFile(owner);
await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'write',
});
const res = await get('/share/shares', owner.token, { uid: file.uid });
expect(res.status).toBe(200);
const body = (await res.json()) as {
items: Array<{ holder: string; mode: string }>;
};
expect(body.items).toHaveLength(1);
expect(body.items[0].holder).toBe(recipient.username);
expect(body.items[0].mode).toBe('write');
});
it('hides an item from a stranger asking who can reach it', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const res = await get('/share/shares', env.users.other.token, {
uid: file.uid,
});
expect(res.status).toBe(404);
});
it('runs a duplicated pair once, and answers for both positions', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const invitee = `dup-${crypto.randomUUID().slice(0, 8)}@puter.local`;
// The same pair twice raced itself into two pending rows; it must
// execute once, with the one outcome reported in both positions.
const res = await post('/share', owner.token, {
recipients: [invitee],
items: [{ uid: file.uid }, { uid: file.uid }],
mode: 'read',
});
expect(res.status).toBe(200);
const body = (await res.json()) as {
status: string;
results: Array<{ status: string; uid?: string }>;
};
expect(body.results).toHaveLength(2);
expect(body.results[0].status).toBe('pending');
expect(body.results[1]).toEqual(body.results[0]);
const rows = await env.server.stores.share.listPendingByEmail(invitee);
expect(rows).toHaveLength(1);
});
it('rejects an unauthenticated share', async () => {
const res = await fetch(new URL('/share', env.apiOrigin), {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ recipients: ['x'], items: ['y'] }),
});
expect(res.status).toBeGreaterThanOrEqual(400);
expect(res.status).toBeLessThan(500);
});
it('caps how many recipients one request can reach', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const many = Array.from({ length: 64 }, (_, i) => `user-${i}`);
const res = await post('/share', owner.token, {
recipients: many,
items: [{ uid: file.uid }],
mode: 'read',
});
expect(res.status).toBe(400);
expect(await res.json()).toMatchObject({
code: 'too_many_recipients',
});
});
it('caps how many items one request can carry', async () => {
const owner = env.users.user;
const many = Array.from({ length: 128 }, () => ({
uid: crypto.randomUUID(),
}));
const res = await post('/share', owner.token, {
recipients: [env.users.other.username],
items: many,
mode: 'read',
});
expect(res.status).toBe(400);
expect(await res.json()).toMatchObject({ code: 'too_many_items' });
});
it('rejects a request with no recipients or no items', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
expect(
(await post('/share', owner.token, { items: [{ uid: file.uid }] }))
.status,
).toBe(400);
expect(
(
await post('/share', owner.token, {
recipients: [env.users.other.username],
})
).status,
).toBe(400);
});
it('blocks a sender, refuses their share, then unblocks them', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const file = await makeFile(owner);
const blocked = await post('/share/blocks', recipient.token, {
username: owner.username,
});
expect(blocked.status).toBe(200);
expect(await blocked.json()).toMatchObject({
username: owner.username,
blocked: true,
created: true,
});
const refused = await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'read',
});
// A per-pair outcome, so the envelope reports it rather than the status.
expect(refused.status).toBe(200);
expect(await refused.json()).toMatchObject({
status: 'aborted',
results: [{ status: 'error', code: 'recipient_not_accepting_shares' }],
});
const listed = await get('/share/blocks', recipient.token, {});
expect(listed.status).toBe(200);
const body = (await listed.json()) as {
items: Array<Record<string, unknown>>;
};
const row = body.items.find((i) => i.username === owner.username);
expect(row).toBeDefined();
expect(typeof row?.created_at).toBe('number');
// Nothing internal rides along.
for (const key of ['blocker_user_id', 'blocked_user_id', 'id']) {
expect(row).not.toHaveProperty(key);
}
const unblocked = await fetch(new URL('/share/blocks', env.apiOrigin), {
method: 'DELETE',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${recipient.token}`,
},
body: JSON.stringify({ username: owner.username }),
});
expect(unblocked.status).toBe(200);
expect(await unblocked.json()).toMatchObject({ unblocked: true });
const shared = await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(await shared.json()).toMatchObject({ status: 'success' });
});
it('requires a username to block', async () => {
const res = await post('/share/blocks', env.users.other.token, {});
expect(res.status).toBe(400);
expect(await res.json()).toMatchObject({ code: 'bad_request' });
});
it('refuses every sender while the blanket switch is on', async () => {
const owner = env.users.user;
const recipient = env.users.other;
const file = await makeFile(owner);
const del = (body: unknown) =>
fetch(new URL('/share/blocks', env.apiOrigin), {
method: 'DELETE',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${recipient.token}`,
},
body: JSON.stringify(body),
});
const on = await post('/share/blocks', recipient.token, { all: true });
expect(on.status).toBe(200);
expect(await on.json()).toMatchObject({ all: true, blocked: true });
const listed = await get('/share/blocks', recipient.token, {});
expect(await listed.json()).toMatchObject({ all: true, items: [] });
const refused = await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(await refused.json()).toMatchObject({
status: 'aborted',
results: [
{ status: 'error', code: 'recipient_not_accepting_shares' },
],
});
const off = await del({ all: true });
expect(off.status).toBe(200);
expect(await off.json()).toMatchObject({ all: false, blocked: false });
const shared = await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(await shared.json()).toMatchObject({ status: 'success' });
// Cleaned up so a later assertion on this recipient isn't reading
// state this test left behind.
expect((await del({ all: true })).status).toBe(200);
});
it('keeps one caller\'s blocklist out of another\'s', async () => {
const res = await get('/share/blocks', env.users.admin.token, {});
const body = (await res.json()) as { all: boolean; items: unknown[] };
expect(body).toEqual({ all: false, items: [] });
});
});
+557 -345
View File
@@ -17,381 +17,593 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
// import type { Request, Response } from 'express';
// import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { Request, Response } from 'express';
import type { Actor } from '../../core/actor.js';
import { Controller, Delete, Get, Post } from '../../core/http/decorators.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import type {
ResolvedShare,
ShareRecipient,
ShareTarget,
} from '../../services/share/ShareService.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import { signEntryThumbnail } from '../fs/legacyFsHelpers.js';
import { runWithConcurrencyLimitSettled } from '../../util/concurrency.js';
import { normalizeLimit } from '../../util/pagination.js';
import { PuterController } from '../types.js';
// const SHARE_TOKEN_TYPE = 'share';
// const SHARE_TOKEN_EXPIRY = '14d';
/**
* Two windows: a burst ceiling, and a daily one so a slow drip can't add up to
* a mail-merge. Neither bounds _shares_ — one request carries many — which is
* what `ShareService`'s per-day quota is for.
*/
const SHARE_LIMIT = [
{ scope: 'share:mutate', limit: 60, window: 60_000, key: 'user' as const },
{
scope: 'share:mutate-daily',
limit: 500,
window: 24 * 60 * 60_000,
key: 'user' as const,
},
];
const SHARE_LIST_LIMIT = {
scope: 'share:list',
limit: 600,
window: 60_000,
key: 'user' as const,
};
/** Distinct (holder, item) pairs run together; see the note on grouping below. */
const SHARE_CONCURRENCY = 8;
const LIST_LIMIT_CAP = 200;
/**
* Share link endpoints — check, apply, and request access to pending shares.
* The main `POST /share` creation endpoint is also here.
*
* Shares are permission grants addressed to an email. When the recipient
* doesn't have a Puter account yet, the share row lives in the `share` table
* until they sign up and apply it. When they DO have an account, permissions
* are granted immediately and no row is stored.
* Caps on one request's fan-out. Recipients matter most: that number is how
* many people a single call can reach, so it stays small by default and only
* moves by configuration.
*/
export const DEFAULT_MAX_RECIPIENTS = 10;
export const DEFAULT_MAX_ITEMS = 50;
/**
* A success carries the created share; a failure carries why. `pending` is an
* invite: recorded, but not access anyone holds yet.
*/
interface ShareOutcome {
recipient: string;
status: 'success' | 'error' | 'pending';
path?: string;
uid?: string;
mode?: string;
uid_entry?: string;
is_dir?: boolean;
issuer?: string | null;
holder?: string | null;
created_at?: unknown;
message?: string;
code?: string;
}
/**
* Sharing endpoints. `ShareService` owns the semantics; this layer parses
* input, bounds fan-out, and shapes responses.
*
* Apps and tokens are admitted rather than gated out, because `ShareService`
* bounds them properly: authority to share comes from the user behind the
* actor, and the actor must additionally reach the node in its own right. An
* app therefore shares its own AppData and the files it was given, and nothing
* else its user happens to own.
*/
@Controller('/share')
export class ShareController extends PuterController {
registerRoutes(_router: PuterRouter): void {
// const api = { subdomain: 'api' } as const;
// router.post('/sharelink/check', api, this.#check);
// router.post(
// '/sharelink/apply',
// { ...api, requireAuth: true },
// this.#apply,
// );
// router.post(
// '/sharelink/request',
// { ...api, requireAuth: true },
// this.#request,
// );
// router.post('/share', { ...api, requireAuth: true }, this.#share);
/**
* POST /share — grant `mode` on one or more items to one or more
* recipients. Partial success is the contract: each pair reports its own
* outcome and the envelope summarizes.
*/
@Post('', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIMIT,
})
async createShares(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = this.#body(req);
const recipients = this.#recipients(body);
const items = this.#items(body, actor);
const mode = typeof body.mode === 'string' ? body.mode : 'read';
// Every (recipient, item) pair is a distinct (holder, entry) key, so
// they can run together. Two writes to the *same* pair could not —
// setUserUser is a read-modify-write, and a duplicated invite races
// itself into two pending rows — so duplicates in the request execute
// once and every original position reports that one outcome.
const pairs = recipients.flatMap((recipient) =>
items.map((item) => ({ recipient, item })),
);
const { unique, indexOf } = this.#dedupePairs(pairs);
const settledUnique = await runWithConcurrencyLimitSettled(
unique,
SHARE_CONCURRENCY,
async ({ recipient, item }) => {
const share = await this.services.share.share(actor, {
...item,
recipient,
mode: mode as never,
});
return share;
},
);
const settled = indexOf.map((i) => settledUnique[i]);
const results: ShareOutcome[] = await Promise.all(
settled.map(async (outcome, index) => {
const { recipient, item } = pairs[index];
const label = recipient.email ?? recipient.username ?? '';
if (outcome.status === 'fulfilled') {
// The whole share, not just an acknowledgement, so a caller
// needn't re-read to learn what it created.
const share = outcome.value as ResolvedShare;
return {
...(await this.#toClientShare(share)),
recipient: label,
status: share.pending ? 'pending' : 'success',
};
}
return {
recipient: label,
...(item.path ? { path: item.path } : {}),
status: 'error',
...this.#errorShape(outcome.reason),
};
}),
);
// Off the response path: a share that landed must not be reported as
// failed because telling the recipient didn't. Fanned out from the
// unique outcomes, so a duplicated pair is not counted twice.
void this.services.shareNotification.notifyShared(
actor,
settledUnique
.filter((o) => o.status === 'fulfilled')
.map((o) => (o as PromiseFulfilledResult<ResolvedShare>).value),
);
const succeeded = results.filter((r) => r.status !== 'error').length;
res.json({
status:
succeeded === results.length
? 'success'
: succeeded > 0
? 'mixed'
: 'aborted',
results,
});
}
// // -- POST /sharelink/check ---------------------------------------
// // Public — verify a share token from an email link.
/**
* POST /share/revoke — withdraw recipients' access to items. Same fan-out
* contract as POST /share: every (recipient, item) pair is its own revoke
* with its own outcome. Silently dropping pairs after the first would leave
* access standing that the caller believes is gone.
*/
@Post('/revoke', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIMIT,
})
async revokeShare(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = this.#body(req);
const recipients = this.#recipients(body);
const items = this.#items(body, actor);
// #check = async (req: Request, res: Response): Promise<void> => {
// const token = req.body?.token;
// if (typeof token !== 'string' || token.length === 0) {
// throw new HttpError(400, 'Missing `token`');
// }
const pairs = recipients.flatMap((recipient) =>
items.map((item) => ({ recipient, item })),
);
const { unique, indexOf } = this.#dedupePairs(pairs);
// let decoded: { uid?: string; type?: string };
// try {
// decoded = this.services.token.verify(SHARE_TOKEN_TYPE, token);
// } catch {
// throw new HttpError(400, 'Invalid or expired share token');
// }
// if (decoded.type !== `token:${SHARE_TOKEN_TYPE}` || !decoded.uid) {
// throw new HttpError(400, 'Invalid share token');
// }
const settledUnique = await runWithConcurrencyLimitSettled(
unique,
SHARE_CONCURRENCY,
({ recipient, item }) =>
this.services.share.unshare(actor, { ...item, recipient }),
);
const settled = indexOf.map((i) => settledUnique[i]);
// const share = await this.stores.share.getByUid(decoded.uid);
// if (!share) throw new HttpError(404, 'Share not found or expired');
// Totalled over the unique outcomes: a pair listed twice revoked its
// grants once.
const revoked = settledUnique.reduce(
(total, outcome) =>
outcome.status === 'fulfilled'
? total + outcome.value.revoked
: total,
0,
);
const results: ShareOutcome[] = settled.map((outcome, index) => {
const { recipient, item } = pairs[index];
const label = recipient.email ?? recipient.username ?? '';
if (outcome.status === 'fulfilled') {
return {
recipient: label,
...(item.path ? { path: item.path } : {}),
...(item.uid ? { uid: item.uid } : {}),
status: 'success',
};
}
return {
recipient: label,
...(item.path ? { path: item.path } : {}),
...(item.uid ? { uid: item.uid } : {}),
status: 'error',
...this.#errorShape(outcome.reason),
};
});
// res.json({
// $: 'api:share',
// uid: share.uid,
// email: share.recipient_email,
// });
// };
const succeeded = results.filter((r) => r.status === 'success').length;
res.json({
status:
succeeded === results.length
? 'success'
: succeeded > 0
? 'mixed'
: 'aborted',
revoked,
results,
});
}
// // -- POST /sharelink/apply ---------------------------------------
// // Auth required — apply a pending share's permissions to the caller.
/**
* GET /share/shared-with-me — paginated listing of what others have shared
* with the caller.
*/
@Get('/shared-with-me', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIST_LIMIT,
})
async listSharedWithMe(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = this.#query(req);
// #apply = async (req: Request, res: Response): Promise<void> => {
// const uid = req.body?.uid;
// if (typeof uid !== 'string') throw new HttpError(400, 'Missing `uid`');
const page = await this.services.share.listSharedWithMe(actor, {
limit: normalizeLimit(query.limit, { cap: LIST_LIMIT_CAP }),
cursor: typeof query.cursor === 'string' ? query.cursor : undefined,
includeTotal: query.includeTotal === 'true',
});
// const actor = req.actor;
// if (!actor?.user) throw new HttpError(401, 'Unauthorized');
res.json({
items: await Promise.all(
page.items.map((share) => this.#toClientShare(share)),
),
...(page.cursor ? { cursor: page.cursor } : {}),
...(page.total !== undefined ? { total: page.total } : {}),
});
}
// const share = await this.stores.share.getByUid(uid);
// if (!share) throw new HttpError(404, 'Share not found or expired');
/** GET /share/shares — who can reach one item. */
@Get('/shares', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIST_LIMIT,
})
async listSharesOf(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const query = this.#query(req);
const target: ShareTarget = {};
if (typeof query.uid === 'string') target.uid = query.uid;
if (typeof query.path === 'string')
target.path = expandTildePath(query.path, actor.user?.username);
if (!target.uid && !target.path) {
throw new HttpError(400, 'one of `uid` or `path` is required', {
legacyCode: 'bad_request',
});
}
// // Issuer must still exist
// const issuer = await this.stores.user.getById(share.issuer_user_id);
// if (!issuer)
// throw new HttpError(410, 'Share expired — issuer account gone');
const shares = await this.services.share.listSharesOf(actor, target);
res.json({
items: await Promise.all(
shares.map((share) => this.#toClientShare(share)),
),
});
}
// // Email must be confirmed
// if (
// actor.user.requires_email_confirmation &&
// !actor.user.email_confirmed
// ) {
// throw new HttpError(
// 403,
// 'Please confirm your email before applying shares',
// );
// }
// -- Blocking -----------------------------------------------------
// // Recipient email must match
// if (
// !actor.user.email ||
// actor.user.email.toLowerCase() !==
// share.recipient_email.toLowerCase()
// ) {
// throw new HttpError(
// 403,
// 'This share was sent to a different email address',
// );
// }
/**
* GET /share/blocks — who the caller is refusing shares from, and whether
* they are refusing everyone.
*/
@Get('/blocks', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIST_LIMIT,
})
async listBlocks(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const { all, items } =
await this.services.share.listBlockedSenders(actor);
res.json({
all,
items: items.map((item) => ({
username: item.username,
created_at: item.createdAt,
})),
});
}
// // Grant each permission
// const issuerActor = {
// user: {
// id: issuer.id,
// uuid: issuer.uuid,
// username: issuer.username,
// email: issuer.email ?? null,
// suspended: false,
// email_confirmed: true,
// requires_email_confirmation: false,
// },
// } as import('../../core/actor.js').Actor;
// const data = (share.data ?? {}) as {
// permissions?: Array<{
// permission: string;
// extra?: Record<string, unknown>;
// }>;
// };
// for (const perm of data.permissions ?? []) {
// try {
// await this.services.permission.grantUserUserPermission(
// issuerActor,
// actor.user.username ?? '',
// perm.permission,
// perm.extra ?? {},
// );
// } catch (err) {
// console.warn('[share] grant failed for', perm.permission, err);
// }
// }
/**
* POST /share/blocks — stop accepting shares. `{ all: true }` refuses
* everyone; `{ username }` refuses one person. Idempotent either way:
* blocking twice is the state the caller asked for.
*
* Access already granted is untouched — `POST /share/revoke` is what
* withdraws that.
*/
@Post('/blocks', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIMIT,
})
async createBlock(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = this.#body(req);
if (this.#isBlockAll(body)) {
const { all } = await this.services.share.setBlockAllSenders(
actor,
true,
);
res.json({ all, blocked: true });
return;
}
const { username, created } = await this.services.share.blockSender(
actor,
this.#username(body),
);
res.json({ username, blocked: true, created });
}
// // Share consumed — delete it
// await this.stores.share.deleteByUid(uid);
/**
* DELETE /share/blocks — accept shares again. `{ all: true }` lifts the
* blanket refusal, leaving the per-sender list as it was.
*/
@Delete('/blocks', {
subdomain: 'api',
requireVerified: true,
rateLimit: SHARE_LIMIT,
})
async deleteBlock(req: Request, res: Response): Promise<void> {
const actor = this.#requireActor(req);
const body = this.#body(req);
if (this.#isBlockAll(body)) {
const { all } = await this.services.share.setBlockAllSenders(
actor,
false,
);
res.json({ all, blocked: false });
return;
}
const { username, unblocked } = await this.services.share.unblockSender(
actor,
this.#username(body),
);
res.json({ username, blocked: false, unblocked });
}
// res.json({ $: 'api:status-report', status: 'success' });
// };
// -- Helpers ------------------------------------------------------
// // -- POST /sharelink/request -------------------------------------
// // Auth required — notify the issuer that someone is requesting access.
/**
* Collapse repeated (recipient, item) pairs to one execution. `indexOf`
* maps every original position onto its unique pair, so a request that
* names the same pair twice still gets a result in both positions — the
* same result, which is the truth of what happened.
*/
#dedupePairs<T extends { recipient: ShareRecipient; item: ShareTarget }>(
pairs: T[],
): { unique: T[]; indexOf: number[] } {
const unique: T[] = [];
const seen = new Map<string, number>();
const indexOf = pairs.map((pair) => {
const key = JSON.stringify([
pair.recipient.email ?? null,
pair.recipient.username ?? null,
pair.item.uid ?? null,
pair.item.path ?? null,
]);
let at = seen.get(key);
if (at === undefined) {
at = unique.length;
seen.set(key, at);
unique.push(pair);
}
return at;
});
return { unique, indexOf };
}
// #request = async (req: Request, res: Response): Promise<void> => {
// const uid = req.body?.uid;
// if (typeof uid !== 'string') throw new HttpError(400, 'Missing `uid`');
/**
* Whether this call is about the blanket switch rather than one person.
* Only the literal `true` counts, so a client sending `all: false`
* alongside a username still means the username.
*/
#isBlockAll(body: Record<string, unknown>): boolean {
return body.all === true;
}
// const actor = req.actor;
// if (!actor?.user) throw new HttpError(401, 'Unauthorized');
#username(body: Record<string, unknown>): string {
const username =
typeof body.username === 'string' ? body.username.trim() : '';
if (!username) {
throw new HttpError(400, '`username` is required', {
legacyCode: 'bad_request',
});
}
return username;
}
// const share = await this.stores.share.getByUid(uid);
// if (!share) throw new HttpError(404, 'Share not found or expired');
/**
* Only ever the username — never the internal id, and never an email the
* caller didn't already supply.
*
* `thumbnail` is stored as an `s3://bucket/key` URI, so it is swapped for a
* signed URL rather than emitted: the raw value names internal storage and
* no client can render it.
*/
async #toClientShare(share: ResolvedShare) {
const thumbnail =
share.thumbnail === undefined
? undefined
: await signEntryThumbnail(
this.clients.event,
share.entryUid,
share.thumbnail,
);
return {
uid: share.uid,
mode: share.mode,
path: share.path,
// A share listing has no fsentry behind it for a client to stat.
...(share.name === undefined ? {} : { name: share.name }),
...(share.type === undefined ? {} : { type: share.type }),
...(thumbnail === undefined ? {} : { thumbnail }),
...(share.owner === undefined
? {}
: { owner: share.owner.username }),
...(share.pending
? { pending: true, recipient_email: share.recipientEmail }
: {}),
uid_entry: share.entryUid,
is_dir: share.isDir,
issuer: share.issuer.username,
holder: share.holder.username,
created_at: share.createdAt,
issued_by_app: share.issuedByApp ?? null,
inherited_from: share.inheritedFrom ?? null,
modified: share.modified,
size: share.size,
};
}
// const issuer = await this.stores.user.getById(share.issuer_user_id);
// if (!issuer)
// throw new HttpError(410, 'Share expired — issuer account gone');
#requireActor(req: Request): Actor {
const actor = req.actor;
if (!actor?.user)
throw new HttpError(401, 'Unauthorized', {
legacyCode: 'unauthorized',
});
return actor;
}
// // If caller IS the intended recipient (confirmed email matches),
// // they should just /apply instead.
// if (
// actor.user.email_confirmed &&
// actor.user.email?.toLowerCase() ===
// share.recipient_email.toLowerCase()
// ) {
// throw new HttpError(
// 400,
// 'You are the intended recipient — use /sharelink/apply instead',
// );
// }
#body(req: Request): Record<string, unknown> {
const body = req.body;
if (!body || typeof body !== 'object' || Array.isArray(body)) {
throw new HttpError(400, 'body must be an object', {
legacyCode: 'bad_request',
});
}
return body as Record<string, unknown>;
}
// // Notify the issuer
// if (this.services.notification) {
// await this.services.notification.notify([issuer.id], {
// source: 'sharing',
// title: `User ${actor.user.username} is trying to open a share you sent to ${share.recipient_email}`,
// template: 'user-requesting-share',
// fields: {
// username: actor.user.username,
// intended_recipient: share.recipient_email,
// permissions:
// (share.data as Record<string, unknown>)?.permissions ??
// [],
// },
// });
// }
#query(req: Request): Record<string, unknown> {
return (req.query ?? {}) as Record<string, unknown>;
}
// res.json({ $: 'api:status-report', status: 'success' });
// };
#recipients(body: Record<string, unknown>): ShareRecipient[] {
const raw = body.recipients ?? body.recipient;
const list = Array.isArray(raw) ? raw : [raw];
const out: ShareRecipient[] = [];
for (const entry of list) {
if (typeof entry === 'string') {
const value = entry.trim();
if (!value) continue;
out.push(
value.includes('@')
? { email: value }
: { username: value },
);
continue;
}
if (entry && typeof entry === 'object') {
const rec = entry as Record<string, unknown>;
const email =
typeof rec.email === 'string' ? rec.email.trim() : '';
const username =
typeof rec.username === 'string' ? rec.username.trim() : '';
if (email || username) {
out.push(email ? { email } : { username });
}
}
}
if (out.length === 0) {
throw new HttpError(400, '`recipients` is required', {
legacyCode: 'bad_request',
});
}
const max = this.config.share_max_recipients ?? DEFAULT_MAX_RECIPIENTS;
if (out.length > max) {
throw new HttpError(400, `at most ${max} recipients per request`, {
legacyCode: 'too_many_recipients',
});
}
return out;
}
// // -- POST /share -------------------------------------------------
// // Auth required — create shares for recipients (users or emails).
#items(body: Record<string, unknown>, actor: Actor): ShareTarget[] {
const username = actor.user?.username;
const raw = body.items ?? body.item ?? body.path ?? body.uid;
const list = Array.isArray(raw) ? raw : [raw];
const out: ShareTarget[] = [];
for (const entry of list) {
if (typeof entry === 'string') {
const value = entry.trim();
if (!value) continue;
// Tilde-rooted strings are paths, as the legacy FS routes treat them.
const isPath = value.startsWith('/') || value.startsWith('~');
out.push(
isPath
? { path: expandTildePath(value, username) }
: { uid: value },
);
continue;
}
if (entry && typeof entry === 'object') {
const item = entry as Record<string, unknown>;
const path =
typeof item.path === 'string'
? expandTildePath(item.path, username)
: '';
const uid = typeof item.uid === 'string' ? item.uid : '';
if (path || uid) out.push(path ? { path } : { uid });
}
}
if (out.length === 0) {
throw new HttpError(400, '`items` is required', {
legacyCode: 'bad_request',
});
}
const max = this.config.share_max_items ?? DEFAULT_MAX_ITEMS;
if (out.length > max) {
throw new HttpError(400, `at most ${max} items per request`, {
legacyCode: 'too_many_items',
});
}
return out;
}
// #share = async (req: Request, res: Response): Promise<void> => {
// const actor = req.actor;
// if (!actor?.user) throw new HttpError(401, 'Unauthorized');
// const body = req.body ?? {};
// let recipients = body.recipients;
// let shares = body.shares;
// const dryRun = !!body.dry_run;
// if (!recipients) throw new HttpError(400, 'Missing `recipients`');
// if (!shares) throw new HttpError(400, 'Missing `shares`');
// if (!Array.isArray(recipients)) recipients = [recipients];
// if (!Array.isArray(shares)) shares = [shares];
// // Build the permissions list from share declarations.
// const permissions = this.#resolvePermissions(shares as unknown[]);
// const recipientResults: unknown[] = [];
// for (const recipient of recipients as unknown[]) {
// const recipientStr =
// typeof recipient === 'string' ? recipient.trim() : '';
// if (!recipientStr) {
// recipientResults.push({
// $: 'error',
// message: 'empty recipient',
// });
// continue;
// }
// try {
// // Try username first
// const targetUser =
// (await this.stores.user.getByUsername(recipientStr)) ??
// (recipientStr.includes('@')
// ? await this.stores.user.getByEmail(recipientStr)
// : null);
// if (targetUser) {
// // Direct grant — user exists
// if (!dryRun) {
// for (const perm of permissions) {
// try {
// await this.services.permission.grantUserUserPermission(
// actor,
// targetUser.username ?? '',
// perm.permission,
// perm.extra ?? {},
// );
// } catch (err) {
// console.warn(
// '[share] grant to user failed',
// perm.permission,
// err,
// );
// }
// }
// // Notify
// if (this.services.notification) {
// await this.services.notification.notify(
// [targetUser.id],
// {
// source: 'sharing',
// title: `${actor.user.username} shared items with you`,
// template: 'file-shared-with-you',
// fields: {
// username: actor.user.username,
// permissions: permissions.map(
// (p) => p.permission,
// ),
// },
// },
// );
// }
// }
// recipientResults.push({
// $: 'api:status-report',
// status: 'success',
// });
// } else if (recipientStr.includes('@')) {
// // Email recipient — store pending share
// if (!dryRun) {
// const share = await this.stores.share.create({
// issuerUserId: actor.user.id,
// recipientEmail: recipientStr.toLowerCase(),
// data: {
// permissions,
// metadata: body.metadata ?? {},
// },
// });
// // Sign a share token (14-day expiry)
// const token = this.services.token.sign(
// SHARE_TOKEN_TYPE,
// {
// type: `token:${SHARE_TOKEN_TYPE}`,
// uid: share.uid,
// },
// { expiresIn: SHARE_TOKEN_EXPIRY },
// );
// // Email the share link
// const origin = `https://${this.config.domain ?? 'puter.com'}`;
// try {
// await this.clients.email.sendRaw({
// to: recipientStr,
// subject: `${actor.user.username} shared something with you on Puter`,
// html: `<p>${actor.user.username} shared items with you.</p><p><a href="${origin}?share_token=${encodeURIComponent(token)}">Click here to accept</a></p>`,
// });
// } catch (err) {
// console.warn('[share] email send failed', err);
// }
// }
// recipientResults.push({
// $: 'api:status-report',
// status: 'success',
// });
// } else {
// recipientResults.push({
// $: 'error',
// message: 'User not found',
// });
// }
// } catch (err) {
// recipientResults.push({ $: 'error', message: String(err) });
// }
// }
// const allOk = recipientResults.every(
// (r: unknown) => (r as Record<string, unknown>).status === 'success',
// );
// const anyOk = recipientResults.some(
// (r: unknown) => (r as Record<string, unknown>).status === 'success',
// );
// res.json({
// $: 'api:share',
// $version: 'v0.0.0',
// status: allOk ? 'success' : anyOk ? 'mixed' : 'aborted',
// recipients: recipientResults,
// ...(dryRun ? { dry_run: true } : {}),
// });
// };
// // -- Helpers ------------------------------------------------------
// /**
// * Convert share declarations into a flat permission list.
// * Supports `fs-share` ({ path, access }) and `app-share` ({ uid, name }).
// */
// #resolvePermissions(
// shares: unknown[],
// ): Array<{ permission: string; extra?: Record<string, unknown> }> {
// const perms: Array<{
// permission: string;
// extra?: Record<string, unknown>;
// }> = [];
// for (const share of shares) {
// if (!share || typeof share !== 'object') continue;
// const s = share as Record<string, unknown>;
// if (s.$ === 'fs-share' || s.type === 'fs-share' || s.path) {
// const path = String(s.path ?? '');
// const access = String(s.access ?? 'read');
// if (path) {
// perms.push({ permission: `fs:${path}:${access}` });
// }
// } else if (
// s.$ === 'app-share' ||
// s.type === 'app-share' ||
// s.uid ||
// s.name
// ) {
// const appUid = String(s.uid ?? s.name ?? '');
// if (appUid) {
// perms.push({ permission: `app:uid#${appUid}:access` });
// }
// }
// }
// return perms;
// }
/**
* Report a failure without widening what the caller already knew. The
* service already decides 404-vs-403; anything unrecognized becomes a
* generic error rather than leaking an internal message.
*/
#errorShape(reason: unknown): { message: string; code?: string } {
if (!isHttpError(reason) || reason.statusCode >= 500) {
return { message: 'Request failed' };
}
const code = reason.legacyCode ?? reason.code;
return {
message: reason.message || 'Request failed',
...(code ? { code } : {}),
};
}
}
@@ -32,6 +32,17 @@ import { promoteToVerifiedGroup } from '../../util/userProvisioning.js';
* All root-subdomain-only, all unauthenticated (the confirm/unsubscribe tokens
* in the query string are the auth).
*/
/**
* Unauthenticated pages reached from a token-bearing email link. Matches what
* the emailSend extension already applies to its own link pages.
*/
const TOKEN_LINK_LIMIT = {
scope: 'token-link-page',
limit: 30,
window: 60_000,
key: 'ip' as const,
};
export class StaticPagesController extends PuterController {
registerRoutes(router: PuterRouter) {
const origin = this.config.origin ?? '';
@@ -159,151 +170,191 @@ export class StaticPagesController extends PuterController {
});
// -- /sitemap.xml --------------------------------------------
router.get('/sitemap.xml', {}, async (req, res) => {
const domain = this.config.domain ?? req.hostname;
const origin = `${req.protocol}://${domain}`;
const apps = (await this.clients.db.read(
`SELECT \`name\` FROM \`apps\` WHERE \`approved_for_listing\` = ${this.clients.db.booleanLiteral(true)}`,
)) as Array<{ name: string }>;
const urls = [
`<url><loc>${req.protocol}://docs.${domain}/</loc></url>`,
...apps.map(
(a) => `<url><loc>${origin}/app/${a.name}</loc></url>`,
),
];
const body =
'<?xml version="1.0" encoding="UTF-8"?>' +
'<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">' +
urls.join('') +
'</urlset>';
res.type('application/xml').send(body);
});
router.get(
'/sitemap.xml',
{
// Unauthenticated and runs a full-table scan over approved
// apps on every request, with no response cache in front.
rateLimit: {
scope: 'sitemap',
limit: 10,
window: 60_000,
key: 'ip',
},
},
async (req, res) => {
const domain = this.config.domain ?? req.hostname;
const origin = `${req.protocol}://${domain}`;
const apps = (await this.clients.db.read(
`SELECT \`name\` FROM \`apps\` WHERE \`approved_for_listing\` = ${this.clients.db.booleanLiteral(true)}`,
)) as Array<{ name: string }>;
const urls = [
`<url><loc>${req.protocol}://docs.${domain}/</loc></url>`,
...apps.map(
(a) => `<url><loc>${origin}/app/${a.name}</loc></url>`,
),
];
const body =
'<?xml version="1.0" encoding="UTF-8"?>' +
'<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">' +
urls.join('') +
'</urlset>';
res.type('application/xml').send(body);
},
);
// -- /unsubscribe --------------------------------------------
router.get('/unsubscribe', {}, async (req, res) => {
const userUuid =
typeof req.query.user_uuid === 'string'
? req.query.user_uuid
: undefined;
if (!userUuid) {
res.send(err('user_uuid is required'));
return;
}
router.get(
'/unsubscribe',
{ rateLimit: TOKEN_LINK_LIMIT },
async (req, res) => {
const userUuid =
typeof req.query.user_uuid === 'string'
? req.query.user_uuid
: undefined;
if (!userUuid) {
res.send(err('user_uuid is required'));
return;
}
const user = await this.stores.user.getByUuid(userUuid);
if (!user) {
res.send(err('User not found.'));
return;
}
if (user.unsubscribed) {
res.send(ok('You are already unsubscribed.'));
return;
}
const user = await this.stores.user.getByUuid(userUuid);
if (!user) {
res.send(err('User not found.'));
return;
}
if (user.unsubscribed) {
res.send(ok('You are already unsubscribed.'));
return;
}
await this.stores.user.update(user.id, { unsubscribed: 1 });
res.send(ok('You have successfully unsubscribed from all emails.'));
});
await this.stores.user.update(user.id, { unsubscribed: 1 });
res.send(
ok('You have successfully unsubscribed from all emails.'),
);
},
);
// -- /confirm-email-by-token ---------------------------------
router.get('/confirm-email-by-token', {}, async (req, res) => {
const userUuid =
typeof req.query.user_uuid === 'string'
? req.query.user_uuid
: undefined;
const token =
typeof req.query.token === 'string'
? req.query.token
: undefined;
if (!userUuid) {
res.send(err('user_uuid is required'));
return;
}
if (!token) {
res.send(err('token is required'));
return;
}
router.get(
'/confirm-email-by-token',
{ rateLimit: TOKEN_LINK_LIMIT },
async (req, res) => {
const userUuid =
typeof req.query.user_uuid === 'string'
? req.query.user_uuid
: undefined;
const token =
typeof req.query.token === 'string'
? req.query.token
: undefined;
if (!userUuid) {
res.send(err('user_uuid is required'));
return;
}
if (!token) {
res.send(err('token is required'));
return;
}
const user = await this.stores.user.getByProperty(
'uuid',
userUuid,
{ force: true },
);
if (!user) {
res.send(err('user not found.'));
return;
}
if (user.email_confirmed) {
res.send(ok('Email already confirmed.'));
return;
}
if (user.email_confirm_token !== token) {
res.send(err('invalid token.'));
return;
}
// v2 writes `clean_email` at signup (lowercased email). Older rows
// that predate that may be null — fall back to email.lower().
const cleanEmail =
(user.clean_email as string | null | undefined) ??
String(user.email ?? '').toLowerCase();
const [dupe] = (await this.clients.db.read(
`SELECT EXISTS(
SELECT 1 FROM \`user\` WHERE (\`email\` = ? OR \`clean_email\` = ?)
AND \`email_confirmed\` = ${this.clients.db.booleanLiteral(true)}
AND \`password\` IS NOT NULL
) AS email_exists`,
[user.email, cleanEmail],
)) as Array<{ email_exists: number }>;
if (dupe?.email_exists) {
res.send(
err('This email was confirmed on a different account.'),
const user = await this.stores.user.getByProperty(
'uuid',
userUuid,
{ force: true },
);
return;
}
if (!user) {
res.send(err('user not found.'));
return;
}
if (user.email_confirmed) {
res.send(ok('Email already confirmed.'));
return;
}
if (user.email_confirm_token !== token) {
res.send(err('invalid token.'));
return;
}
// Revoke any other accounts' pending change-email slots targeting
// this address — they're no longer valid once someone confirms it.
await this.clients.db.write(
'UPDATE `user` SET `unconfirmed_change_email` = NULL, `change_email_confirm_token` = NULL WHERE `unconfirmed_change_email` = ?',
[user.email],
);
// v2 writes `clean_email` at signup (lowercased email). Older rows
// that predate that may be null — fall back to email.lower().
const cleanEmail =
(user.clean_email as string | null | undefined) ??
String(user.email ?? '').toLowerCase();
await this.stores.user.update(user.id, {
email_confirmed: 1,
requires_email_confirmation: 0,
email_confirm_code: null,
email_confirm_token: null,
});
// An account that already confirmed this address proved access
// to the inbox. The strip below would take it away from them,
// so refuse here instead. Password-less accounts count: an
// identity provider verified the address for those, and they
// are exactly what the old `password IS NOT NULL` clause let
// through.
const confirmedRival =
await this.stores.user.findConfirmedOtherByEmail(
user.id as number,
user.email as string,
cleanEmail,
);
if (confirmedRival) {
res.send(
err('This email was confirmed on a different account.'),
);
return;
}
await promoteToVerifiedGroup(this.stores.group, this.config, user);
// Best-effort side-channels — don't fail the user-visible response
// if sockets or the event bus are unavailable.
try {
await this.services.socket.send(
{ room: user.id },
'user.email_confirmed',
{},
// Revoke any other accounts' pending change-email slots targeting
// this address — they're no longer valid once someone confirms it.
await this.clients.db.write(
'UPDATE `user` SET `unconfirmed_change_email` = NULL, `change_email_confirm_token` = NULL WHERE `unconfirmed_change_email` = ?',
[user.email],
);
} catch {
/* ignore */
}
try {
this.clients.event?.emit(
'user.email-confirmed',
{
user_id: user.id,
user_uid: user.uuid,
email: user.email,
},
{},
);
} catch {
/* ignore */
}
res.send(ok('Your email has been successfully confirmed.'));
});
// Take the address off every remaining row before confirming
// this one. The check above leaves only unconfirmed rows, and
// only one row may own an address once this one is confirmed.
await this.stores.user.unconfirmOthersByEmail(
user.id,
user.email as string,
cleanEmail,
);
await this.stores.user.update(user.id, {
email_confirmed: 1,
requires_email_confirmation: 0,
email_confirm_code: null,
email_confirm_token: null,
});
await promoteToVerifiedGroup(
this.stores.group,
this.config,
user,
);
// Best-effort side-channels — don't fail the user-visible response
// if sockets or the event bus are unavailable.
try {
await this.services.socket.send(
{ room: user.id },
'user.email_confirmed',
{},
);
} catch {
/* ignore */
}
try {
this.clients.event?.emit(
'user.email-confirmed',
{
user_id: user.id,
user_uid: user.uuid,
email: user.email,
},
{},
);
} catch {
/* ignore */
}
res.send(ok('Your email has been successfully confirmed.'));
},
);
}
}
@@ -25,6 +25,75 @@ import { PuterController } from '../types.js';
*
* These are all low-risk, authenticated or not, and mostly stateless.
*/
/**
* Liveness polling. The callers here are infrastructure, not people: a load
* balancer, an orchestrator and any external uptime prober all poll this, and
* they typically egress from a small set of addresses. A 429 here is read as an
* unhealthy node and takes the node out of rotation, so the ceiling is set
* where only a runaway loop can reach it. The handler itself reads a status
* snapshot refreshed on a background timer, so the per-request cost is close to
* nil.
*
* `memory` rather than the shared default, and that choice is load-bearing:
*
* - This route decides whether a node stays in rotation, so it must not depend on
* anything it isn't already reporting on. The default backend is redis, and
* the cluster is configured with an offline queue and no per-command timeout
* — so while redis is unreachable a gated request waits on it rather than
* failing fast. The gate does fail open, but only once the call rejects, and
* the ALB gives a target 4s per probe and evicts after two. A redis
* degradation could therefore empty every target group in every region, which
* is the outcome the `@dependencies` degrade rules in the health check query
* exist to prevent. Keeping the counter in-process removes redis from the
* liveness path entirely.
* - Per-node counting is also the more honest bucket here. The ceiling only ever
* needs to cover the pollers hitting _this_ node, not (pollers x fleet size)
* as a shared counter does.
*/
const HEALTHCHECK_LIMIT = {
scope: 'healthcheck',
limit: 30_000,
window: 60_000,
key: 'ip',
backend: 'memory',
};
/**
* Deploy-constant build info, polled by clients. One address is a NAT, a
* campus, a proxy or a server-side renderer, so this bucket aggregates every
* client behind it — sizing it for a single browser would throttle a whole
* office. The response is cached per-client for a minute, which bounds each
* client to roughly one hit per window; the ceiling is what is left to catch a
* client that ignores the cache.
*/
const VERSION_LIMIT = {
scope: 'version',
limit: 6_000,
window: 60_000,
key: 'ip',
};
/**
* Deploy-constant deployment identity, read once per page load to decide
* whether to offer signup. Same aggregation as `/version` — the bucket is a
* whole network's worth of clients — and the payload is four constants, so the
* limit only guards against an unbounded client loop.
*/
const WHOAREWE_LIMIT = {
scope: 'whoarewe',
limit: 6_000,
window: 60_000,
key: 'ip',
};
/** Static introspection output, read once at boot rather than in a loop. */
const LSMOD_LIMIT = {
scope: 'lsmod',
limit: 60,
window: 60_000,
key: 'user',
};
export class SystemController extends PuterController {
constructor(config, clients, stores, services, drivers) {
super(config, clients, stores, services, drivers);
@@ -44,7 +113,10 @@ export class SystemController extends PuterController {
// `?ignore=a,b` disregards the named checks for this request only.
// `?marked-degraded=a,b` demotes the named checks to a non-fatal
// `degraded` list: `ok` stays true but the response is 207 so the
// caller can tell the node is running in a degraded state.
// caller can tell the node is running in a degraded state. Either list
// accepts `@<group>` to stand for every check in a group — notably
// `@dependencies` for the backing-service probes — so a caller polling
// this route doesn't have to enumerate them.
const parseNames = (value) =>
typeof value === 'string'
? value
@@ -52,44 +124,53 @@ export class SystemController extends PuterController {
.map((name) => name.trim())
.filter(Boolean)
: [];
router.get('/healthcheck', { subdomain: '*' }, async (req, res) => {
const health = this.services.health;
if (!health || typeof health.getStatus !== 'function') {
// Fallback for boot ordering / missing service.
return res.send('ok');
}
const status = await health.getStatus({
ignore: parseNames(req.query.ignore),
degrade: parseNames(req.query['marked-degraded']),
});
if (!status.ok) return res.status(503).json(status);
if (status.degraded?.length) return res.status(207).json(status);
return res.json(status);
});
router.get(
'/healthcheck',
{ subdomain: '*', rateLimit: HEALTHCHECK_LIMIT },
async (req, res) => {
const health = this.services.health;
if (!health || typeof health.getStatus !== 'function') {
// Fallback for boot ordering / missing service.
return res.send('ok');
}
const status = await health.getStatus({
ignore: parseNames(req.query.ignore),
degrade: parseNames(req.query['marked-degraded']),
});
if (!status.ok) return res.status(503).json(status);
if (status.degraded?.length)
return res.status(207).json(status);
return res.json(status);
},
);
// -- Version -------------------------------------------------
router.get('/version', { subdomain: '*' }, (_req, res) => {
const version =
this.config.version ??
process.env.npm_package_version ??
'unknown';
const parts = String(version).split('.');
// Deploy-constant, and callers poll it. Cache per-client only:
// a shared cache could pin one region's `location` for everyone,
// and the short window still bounds how long a client can miss a
// new deploy.
res.setHeader('Cache-Control', 'private, max-age=60');
res.json({
version,
major: parts[0] ? Number(parts[0]) : null,
minor: parts[1] ? Number(parts[1]) : null,
patch: parts[2] ? Number(parts[2]) : null,
environment: this.config.env ?? 'prod',
location: this.config.serverId ?? null,
deploy_timestamp: this.bootTime,
});
});
router.get(
'/version',
{ subdomain: '*', rateLimit: VERSION_LIMIT },
(_req, res) => {
const version =
this.config.version ??
process.env.npm_package_version ??
'unknown';
const parts = String(version).split('.');
// Deploy-constant, and callers poll it. Cache per-client only:
// a shared cache could pin one region's `location` for everyone,
// and the short window still bounds how long a client can miss a
// new deploy.
res.setHeader('Cache-Control', 'private, max-age=60');
res.json({
version,
major: parts[0] ? Number(parts[0]) : null,
minor: parts[1] ? Number(parts[1]) : null,
patch: parts[2] ? Number(parts[2]) : null,
environment: this.config.env ?? 'prod',
location: this.config.serverId ?? null,
deploy_timestamp: this.bootTime,
});
},
);
// -- Contact us ----------------------------------------------
@@ -153,7 +234,7 @@ export class SystemController extends PuterController {
// -- GET /whoarewe -------------------------------------------
router.get('/whoarewe', {}, (_req, res) => {
router.get('/whoarewe', { rateLimit: WHOAREWE_LIMIT }, (_req, res) => {
res.json({
name: 'Puter',
version: this.config.version ?? null,
@@ -181,8 +262,16 @@ export class SystemController extends PuterController {
}
res.json({ interfaces });
};
router.get('/lsmod', { subdomain: 'api', requireAuth: true }, lsmod);
router.post('/lsmod', { subdomain: 'api', requireAuth: true }, lsmod);
router.get(
'/lsmod',
{ subdomain: 'api', requireAuth: true, rateLimit: LSMOD_LIMIT },
lsmod,
);
router.post(
'/lsmod',
{ subdomain: 'api', requireAuth: true, rateLimit: LSMOD_LIMIT },
lsmod,
);
}
onServerStart() {}
@@ -141,6 +141,19 @@ const callRoute = async (
// ── /healthcheck ────────────────────────────────────────────────────
describe('SystemController GET /healthcheck', () => {
// This route decides whether a node stays in rotation, so its rate limit
// must not reach for a backing service. The default backend is redis,
// whose client queues rather than fails fast while it is unreachable —
// enough to push a probe past the 4s the load balancer allows and evict
// every target during a redis degradation. In-process counting keeps the
// liveness path free of anything it is itself reporting on.
it('counts in-process, so liveness never waits on redis', () => {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === '/healthcheck',
);
expect(route?.options.rateLimit?.backend).toBe('memory');
});
it('returns the live ServerHealthService status payload', async () => {
const { res, captured } = makeRes();
await callRoute('get', '/healthcheck', makeReq({}), res);
@@ -461,6 +474,52 @@ describe('SystemController GET /lsmod', () => {
});
});
// ── rate-limit scopes ───────────────────────────────────────────────
describe('SystemController public route rate limits', () => {
const rateLimitOf = (path: string) => {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === path,
);
if (!route) throw new Error(`No GET ${path} route`);
return route.options.rateLimit as {
scope: string;
limit: number;
window: number;
key: string;
};
};
it('gives /healthcheck, /version and /whoarewe separate buckets', () => {
// These once shared one scope, which meant clients polling /version
// could exhaust the budget that liveness probes depend on. Keep them
// apart: a 429 on /healthcheck is read as an unhealthy node.
const scopes = [
rateLimitOf('/healthcheck').scope,
rateLimitOf('/version').scope,
rateLimitOf('/whoarewe').scope,
];
expect(new Set(scopes).size).toBe(3);
expect(scopes).toEqual(['healthcheck', 'version', 'whoarewe']);
});
it('sizes the unauthenticated buckets for a shared address, not one client', () => {
// All three key on IP, and an IP is a NAT, a campus or a carrier
// gateway — the bucket aggregates everyone behind it, and the
// limiter counts region-wide rather than per process.
for (const path of ['/healthcheck', '/version', '/whoarewe']) {
const limit = rateLimitOf(path);
expect(limit.key).toBe('ip');
expect(limit.window).toBe(60_000);
expect(limit.limit).toBeGreaterThanOrEqual(6_000);
}
// Liveness polling is the most generous of the three by design.
expect(rateLimitOf('/healthcheck').limit).toBe(30_000);
expect(rateLimitOf('/version').limit).toBe(6_000);
expect(rateLimitOf('/whoarewe').limit).toBe(6_000);
});
});
// ── lifecycle ───────────────────────────────────────────────────────
describe('SystemController.onServerPrepareShutdown', () => {
@@ -1,19 +1,21 @@
// This suite tests basic features of puter webdav. it is not a comprehensive webdav test suite unlike litmus
// but rather it performs some common sense checks to ensure that WebDAV support isn't irrevocably broken in puter
import type { Request, Response } from 'express';
import type { Request, RequestHandler, Response } from 'express';
import { Readable, Writable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { hash as bcryptHash } from 'bcrypt';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import type { RouteDescriptor } from '../../core/http/types.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { runWithContext } from '../../core/context.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { WebDAVController } from './WebDAVController.js';
let server: PuterServer;
let controller: WebDAVController;
let dispatchMiddleware: Function;
let routes: RouteDescriptor[];
beforeAll(async () => {
server = await setupTestServer();
@@ -21,12 +23,60 @@ beforeAll(async () => {
const router = new PuterRouter();
controller.registerRoutes(router);
// WebDAVController registers a single `use()` middleware on the `dav`
// subdomain. Grab it to call directly in tests.
dispatchMiddleware = router.routes[0]!.handler;
routes = router.routes;
});
/** Run one handler; true when it called `next` instead of answering. */
const runHandler = async (
handler: RequestHandler,
req: Request,
res: Response,
): Promise<boolean> => {
let advanced = false;
await handler(req, res, (() => {
advanced = true;
}) as never);
return advanced;
};
/**
* Stand in for express's router: walk the controller's collected routes in
* order and run the first one that matches the request's method, plus any
* `middleware` it declares.
*
* The options the real server materializes into gates — subdomain, rate limit,
* concurrency — are deliberately not applied here; those belong to the
* materializer's own tests, and reproducing them would make every case below
* share one DAV budget. `server.test.ts` covers the wiring on a real port.
*/
const dispatchMiddleware = async (
req: Request,
res: Response,
next: () => void,
): Promise<void> => {
const method = req.method.toLowerCase();
for (const route of routes) {
if (
route.method !== 'use' &&
route.method !== 'all' &&
route.method !== method
) {
continue;
}
let advanced = true;
for (const handler of [
...(route.options.middleware ?? []),
route.handler,
]) {
advanced = await runHandler(handler, req, res);
if (!advanced) break;
}
// A chain that stopped short of `next` answered the request.
if (!advanced) return;
}
next();
};
afterAll(async () => {
await server?.shutdown();
});
@@ -64,7 +114,20 @@ const makeRes = () => {
headers: {},
ended: false,
};
const listeners: Record<string, Array<() => void>> = {};
const res = {
// The DAV mount holds a concurrency slot for the life of the request
// and releases it on `finish` / `close`, so the stub has to behave
// like an emitter or every dispatch throws.
once: vi.fn((event: string, fn: () => void) => {
(listeners[event] ??= []).push(fn);
return res;
}),
emit: vi.fn((event: string) => {
const fns = listeners[event] ?? [];
listeners[event] = [];
for (const fn of fns) fn();
}),
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
@@ -89,10 +152,12 @@ const makeRes = () => {
}),
send: vi.fn((value: unknown) => {
captured.body = value;
res.emit('finish');
return res;
}),
end: vi.fn(() => {
captured.ended = true;
res.emit('finish');
return res;
}),
headersSent: false,
@@ -136,10 +201,45 @@ const makeUser = async () => {
describe('WebDAVController', () => {
describe('route registration', () => {
it('registers a single catch-all use() route', () => {
it('registers one catch-all per DAV verb, all on the dav subdomain', () => {
const router = new PuterRouter();
controller.registerRoutes(router);
expect(router.routes.length).toBeGreaterThanOrEqual(1);
for (const verb of [
'options',
'head',
'get',
'propfind',
'proppatch',
'mkcol',
'put',
'delete',
'copy',
'move',
'lock',
'unlock',
]) {
const route = router.routes.find((r) => r.method === verb);
expect(route, verb).toBeDefined();
expect(route!.path).toBe('/{*splat}');
expect(route!.options.subdomain).toBe('dav');
// The gates the materializer installs ahead of the handler.
expect(route!.options.rateLimit).toBeDefined();
expect(route!.options.concurrent).toBeDefined();
}
// `all` catches the verbs we don't implement, so it has to come
// after every route that does.
const allIndex = router.routes.findIndex((r) => r.method === 'all');
expect(allIndex).toBe(router.routes.length - 1);
});
it('registers HEAD ahead of GET so express does not fold the two', () => {
const router = new PuterRouter();
controller.registerRoutes(router);
const head = router.routes.findIndex((r) => r.method === 'head');
const get = router.routes.findIndex((r) => r.method === 'get');
expect(head).toBeLessThan(get);
});
});
@@ -225,6 +325,48 @@ describe('WebDAVController', () => {
});
});
describe('CORS preflight', () => {
it('answers a browser preflight before the auth gate', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'OPTIONS',
path: '/someone/.vscode/settings.json',
headers: {
origin: 'https://code.puter.com',
'access-control-request-method': 'PROPFIND',
},
}),
res,
noop,
);
// A non-2xx here is what made every browser DAV client give up
// before it could send a token.
expect(captured.statusCode).toBe(200);
expect(captured.headers['dav']).toContain('1');
expect(captured.headers['allow']).toContain('PROPFIND');
expect(captured.headers['access-control-max-age']).toBe('86400');
expect(captured.headers['www-authenticate']).toBeUndefined();
});
it('still authenticates an OPTIONS that is not a preflight', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'OPTIONS',
// An Origin alone isn't a preflight — only the
// `Access-Control-Request-Method` probe is.
headers: { origin: 'https://code.puter.com' },
}),
res,
noop,
);
expect(captured.statusCode).toBe(401);
// macOS opens a mount on this reply, so it keeps the DAV header.
expect(captured.headers['dav']).toContain('1');
});
});
describe('pending-verification gate', () => {
// WebDAV must enforce the same gate every other authenticated route
// gets from requireVerifiedAccount — it dispatches off a single use()
@@ -962,6 +1104,11 @@ describe('WebDAVController verbs', () => {
send: (value: unknown) => {
captured.body = value;
captured.ended = true;
// End the underlying Writable so `finish` fires, as it does
// on a real response. The DAV mount releases its concurrency
// slot on that event — without it every `send()` path would
// leak a slot and later requests would 429.
if (!sink.writableEnded) sink.end();
return res;
},
headersSent: false,
@@ -1102,6 +1249,54 @@ describe('WebDAVController verbs', () => {
);
});
it('reports timestamps in the present, not the epoch', async () => {
// Entry timestamps are stored as Unix seconds. Handing those to
// `Date` unscaled reads them as milliseconds and dates every file
// to January 1970, which clients sort and sync against.
const { actor, username } = await makeUser();
const path = `/${username}/Documents/dated.txt`;
await putFile(actor, path, 'x');
const captured = await dispatch({
method: 'PROPFIND',
path,
actor,
headers: { depth: '0' },
});
const xml = captured.body as string;
const modified = xml.match(
/<D:getlastmodified>(.*?)<\/D:getlastmodified>/,
)?.[1];
expect(modified).toBeDefined();
const skew = Math.abs(
new Date(modified as string).getTime() - Date.now(),
);
expect(skew).toBeLessThan(5 * 60 * 1000);
});
it('emits the same ETag for an entry over PROPFIND and GET', async () => {
// A client that sees the validator change between the two treats
// the file as modified and re-downloads it on every pass.
const { actor, username } = await makeUser();
const path = `/${username}/Documents/etag-match.txt`;
await putFile(actor, path, 'x');
const propfind = await dispatch({
method: 'PROPFIND',
path,
actor,
headers: { depth: '0' },
});
const propfindEtag = (propfind.body as string).match(
/<D:getetag>(.*?)<\/D:getetag>/,
)?.[1];
const get = await dispatch({ method: 'GET', path, actor });
expect(propfindEtag).toBeDefined();
expect(get.headers['etag']).toBe(propfindEtag);
});
it('omits children at depth 0', async () => {
const { actor, username } = await makeUser();
await putFile(actor, `/${username}/Documents/hidden.txt`, 'x');
@@ -1146,6 +1341,68 @@ describe('WebDAVController verbs', () => {
'<D:getcontenttype>text/markdown</D:getcontenttype>',
);
});
it("answers a share root's parent with a virtual collection", async () => {
const owner = await makeUser();
const holder = await makeUser();
await dispatch({
method: 'MKCOL',
path: `/${owner.username}/Documents/Album`,
actor: owner.actor,
});
const root = (await server.stores.fsEntry.getEntryByPath(
`/${owner.username}/Documents/Album`,
))!;
await server.services.acl.setUserUser(
owner.actor as never,
holder.actor as never,
{
path: root.path,
resolveAncestors: () =>
server.services.fs.getAncestorChain(root.path),
},
'read',
);
// `/<owner>/<uuid>` is where Up from a share root lands; it is not
// a real path, so it answers as a virtual collection whose only
// member is the share root at its masked path.
const captured = await dispatch({
method: 'PROPFIND',
path: `/${owner.username}/${root.uuid}`,
actor: holder.actor,
});
expect(captured.statusCode).toBe(207);
const xml = captured.body as string;
expect(xml).toContain(
`<D:href>/${owner.username}/${root.uuid}/</D:href>`,
);
expect(xml).toContain(
`<D:href>/${owner.username}/${root.uuid}/Album/</D:href>`,
);
// The owner's real tree never shows through.
expect(xml).not.toContain('/Documents/');
});
it("keeps a share root's parent a 404 for a caller with no share", async () => {
const owner = await makeUser();
const stranger = await makeUser();
await dispatch({
method: 'MKCOL',
path: `/${owner.username}/Documents/Private`,
actor: owner.actor,
});
const root = (await server.stores.fsEntry.getEntryByPath(
`/${owner.username}/Documents/Private`,
))!;
const captured = await dispatch({
method: 'PROPFIND',
path: `/${owner.username}/${root.uuid}`,
actor: stranger.actor,
});
expect(captured.statusCode).toBe(404);
});
});
describe('PROPPATCH', () => {
@@ -1633,4 +1890,368 @@ describe('WebDAVController verbs', () => {
expect(captured.body).toBe('Lock token does not match this path');
});
});
describe('shared-path masking', () => {
/** A `Secrets/plan.txt` folder in `owner`'s Documents, shared with `to`. */
const shareFolder = async (
owner: Awaited<ReturnType<typeof makeUser>>,
to: Awaited<ReturnType<typeof makeUser>>,
) => {
const dirUuid = uuidv4();
const dirPath = `/${owner.username}/Documents/Secrets`;
const now = Math.floor(Date.now() / 1000);
const parent = (await server.stores.fsEntry.getEntryByPath(
`/${owner.username}/Documents`,
))!;
await server.clients.db.write(
'INSERT INTO `fsentries` (`uuid`, `name`, `path`, `user_id`, `is_dir`, `modified`, `parent_id`, `parent_uid`) VALUES (?, ?, ?, ?, 1, ?, ?, ?)',
[
dirUuid,
'Secrets',
dirPath,
owner.userId,
now,
parent.id,
parent.uuid,
],
);
const dir = (await server.stores.fsEntry.getEntryByPath(dirPath))!;
await server.clients.db.write(
'INSERT INTO `fsentries` (`uuid`, `name`, `path`, `user_id`, `is_dir`, `modified`, `parent_id`, `parent_uid`) VALUES (?, ?, ?, ?, 0, ?, ?, ?)',
[
uuidv4(),
'plan.txt',
`${dirPath}/plan.txt`,
owner.userId,
now,
dir.id,
dir.uuid,
],
);
await runWithContext({ actor: owner.actor as never }, () =>
server.services.share.share(owner.actor as never, {
uid: dirUuid,
recipient: { username: to.username! },
mode: 'read',
}),
);
return { dirUuid, dirPath };
};
/**
* PROPFIND under the ALS wrap a real request gets, pre-auth actor
* empty.
*/
const propfind = async (path: string, actor: unknown, depth = '1') => {
const { res, captured } = makeRes();
await runWithContext({ actor: undefined }, () =>
dispatchMiddleware(
makeReq({
method: 'PROPFIND',
path,
headers: { depth },
actor,
}),
res,
noop,
),
);
return captured;
};
// The DAV controller authenticates in-controller, after the request
// context snapshotted an empty `req.actor` — so it has to place the
// actor into the context itself or outbound masking silently turns
// off and PROPFIND lists the owner's real paths.
it('lists a shared folder under its masked path, not the owner’s real one', async () => {
const owner = await makeUser();
const recipient = await makeUser();
const { dirUuid } = await shareFolder(owner, recipient);
const masked = `/${owner.username}/${dirUuid}/Secrets`;
const captured = await propfind(masked, recipient.actor);
expect(captured.statusCode).toBe(207);
const xml = String(captured.body);
expect(xml).toContain(`${masked}/plan.txt`);
expect(xml).not.toContain('/Documents/Secrets');
});
it('lists each sharer as a collection under the root', async () => {
const owner = await makeUser();
const recipient = await makeUser();
await shareFolder(owner, recipient);
const captured = await propfind('/', recipient.actor);
expect(captured.statusCode).toBe(207);
const xml = String(captured.body);
// The recipient's own home, plus the owner who shared with them.
expect(xml).toContain(`<D:href>/${recipient.username}/</D:href>`);
expect(xml).toContain(`<D:href>/${owner.username}/</D:href>`);
});
it('omits the sharers at depth 0', async () => {
const owner = await makeUser();
const recipient = await makeUser();
await shareFolder(owner, recipient);
const captured = await propfind('/', recipient.actor, '0');
expect(captured.statusCode).toBe(207);
expect(String(captured.body)).not.toContain(`/${owner.username}/`);
});
it('answers a sharer’s collection with the shares they issued', async () => {
const owner = await makeUser();
const recipient = await makeUser();
const { dirUuid } = await shareFolder(owner, recipient);
const captured = await propfind(
`/${owner.username}`,
recipient.actor,
);
expect(captured.statusCode).toBe(207);
const xml = String(captured.body);
expect(xml).toContain(`<D:href>/${owner.username}/</D:href>`);
// The `<uuid>` level, which the share root hangs off. It borrows
// the shared item's name so a client has something to show.
expect(xml).toContain(
`<D:href>/${owner.username}/${dirUuid}/</D:href>`,
);
expect(xml).toContain('<D:displayname>Secrets</D:displayname>');
// Nothing else of the owner's shows through — not their home
// directory's other children, not the folder the share sits in.
expect(xml).not.toContain('Documents');
});
it('keeps a sharer’s collection a 403 for a caller with no share', async () => {
const owner = await makeUser();
const stranger = await makeUser();
await shareFolder(owner, await makeUser());
const captured = await propfind(
`/${owner.username}`,
stranger.actor,
);
expect(captured.statusCode).toBe(403);
});
});
// The virtual collections invent two levels of path that no fsentry backs.
// Everything here checks that they only ever widen to what was actually
// shared — the rest of the owner's tree has to stay unreachable and
// unnamed.
describe('shared-space containment', () => {
/**
* `owner` shares `Shared/` (holding `in.txt`) with `to`, and keeps
* `Private/` (holding `out.txt`) to themselves.
*/
const shareOneOfTwo = async (
owner: Awaited<ReturnType<typeof makeUser>>,
to: Awaited<ReturnType<typeof makeUser>>,
) => {
const mk = async (
method: string,
path: string,
extra: Record<string, string> = {},
) => {
const { res, captured } = makeRes();
await runWithContext({ actor: undefined }, () =>
dispatchMiddleware(
makeReq({
method,
path,
actor: owner.actor,
headers: extra,
}),
res,
noop,
),
);
return captured;
};
await mk('MKCOL', `/${owner.username}/Documents/Shared`);
await mk('MKCOL', `/${owner.username}/Documents/Private`);
const shared = (await server.stores.fsEntry.getEntryByPath(
`/${owner.username}/Documents/Shared`,
))!;
const private_ = (await server.stores.fsEntry.getEntryByPath(
`/${owner.username}/Documents/Private`,
))!;
await runWithContext({ actor: owner.actor as never }, () =>
server.services.share.share(owner.actor as never, {
uid: shared.uuid,
recipient: { username: to.username! },
mode: 'read',
}),
);
return { shared, private: private_ };
};
const propfind = async (path: string, actor: unknown, depth = '1') => {
const { res, captured } = makeRes();
await runWithContext({ actor: undefined }, () =>
dispatchMiddleware(
makeReq({
method: 'PROPFIND',
path,
headers: { depth },
actor,
}),
res,
noop,
),
);
return captured;
};
it('lists only the items that were shared, not the owner’s siblings', async () => {
const owner = await makeUser();
const holder = await makeUser();
const { shared, private: hidden } = await shareOneOfTwo(
owner,
holder,
);
const captured = await propfind(`/${owner.username}`, holder.actor);
expect(captured.statusCode).toBe(207);
const xml = String(captured.body);
expect(xml).toContain(shared.uuid);
expect(xml).not.toContain(hidden.uuid);
expect(xml).not.toContain('Private');
});
it('does not recurse past the uuid level at Depth: infinity', async () => {
const owner = await makeUser();
const holder = await makeUser();
const { shared } = await shareOneOfTwo(owner, holder);
const captured = await propfind(
`/${owner.username}`,
holder.actor,
'infinity',
);
expect(captured.statusCode).toBe(207);
const xml = String(captured.body);
// The share's own href lives one segment deeper; asking for the
// world must not walk into it.
expect(xml).toContain(`/${shared.uuid}/`);
expect(xml).not.toContain(`/${shared.uuid}/Shared`);
});
it('refuses every way out of a share', async () => {
const owner = await makeUser();
const holder = await makeUser();
const { shared, private: hidden } = await shareOneOfTwo(
owner,
holder,
);
const root = `/${owner.username}/${shared.uuid}/Shared`;
for (const escape of [
// Up and sideways out of the shared subtree.
`${root}/../Private`,
`${root}/../../Documents`,
// Percent-encoded, because the path is decoded before it is
// resolved — the guard has to sit after that, not before.
`${root}/%2e%2e/Private`,
// The owner's real path, named directly rather than walked to.
`/${owner.username}/Documents/Private`,
`/${owner.username}/Documents`,
// A mask pointing at an entry the holder was never given —
// both a bad tail under a uuid they do hold, and the uuid of
// one they don't.
`/${owner.username}/${shared.uuid}/Private`,
`/${owner.username}/${hidden.uuid}`,
`/${owner.username}/${hidden.uuid}/Private`,
]) {
const captured = await propfind(escape, holder.actor);
expect(captured.statusCode, escape).toBeGreaterThanOrEqual(400);
expect(captured.statusCode, escape).toBeLessThan(500);
expect(String(captured.body), escape).not.toContain('Private');
}
});
it('does not leak one holder’s shares to another', async () => {
const owner = await makeUser();
const holder = await makeUser();
const outsider = await makeUser();
await shareOneOfTwo(owner, holder);
const captured = await propfind('/', outsider.actor);
expect(captured.statusCode).toBe(207);
expect(String(captured.body)).not.toContain(owner.username);
});
it('drops the sharer once the share is revoked', async () => {
const owner = await makeUser();
const holder = await makeUser();
const { shared } = await shareOneOfTwo(owner, holder);
expect(String((await propfind('/', holder.actor)).body)).toContain(
`/${owner.username}/`,
);
await runWithContext({ actor: owner.actor as never }, () =>
server.services.share.unshare(owner.actor as never, {
uid: shared.uuid,
recipient: { username: holder.username! },
}),
);
const root = await propfind('/', holder.actor);
expect(String(root.body)).not.toContain(`/${owner.username}/`);
// And the level it used to stand in for is a plain 403 again.
const owned = await propfind(`/${owner.username}`, holder.actor);
expect(owned.statusCode).toBe(403);
});
it('refuses writes and reads aimed at the invented levels', async () => {
const owner = await makeUser();
const holder = await makeUser();
const { shared } = await shareOneOfTwo(owner, holder);
const attempt = async (method: string, path: string) => {
const { res, captured } = makeRes();
await runWithContext({ actor: undefined }, () =>
dispatchMiddleware(
makeReq({
method,
path,
actor: holder.actor,
headers: { 'content-length': '3' },
}),
res,
noop,
),
);
return captured.statusCode;
};
// A collection, so GET is a 400 either way — what matters is that
// neither the owner level nor the uuid level accepts a write.
expect(await attempt('GET', `/${owner.username}`)).not.toBe(200);
expect(
await attempt('MKCOL', `/${owner.username}/intruder`),
).toBeGreaterThanOrEqual(400);
expect(
await attempt(
'MKCOL',
`/${owner.username}/${shared.uuid}/intruder`,
),
).toBeGreaterThanOrEqual(400);
expect(
await attempt('DELETE', `/${owner.username}/${shared.uuid}`),
).toBeGreaterThanOrEqual(400);
// Nothing landed in the owner's tree.
expect(
await server.stores.fsEntry.getEntryByPath(
`/${owner.username}/intruder`,
),
).toBeFalsy();
});
});
});
+555 -183
View File
@@ -21,15 +21,37 @@ import { compare as bcryptCompare } from 'bcrypt';
import type { Request, Response } from 'express';
import { posix as pathPosix } from 'node:path';
import { EventMap } from '../../clients/event/types.js';
import type { Actor } from '../../core/actor.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import {
assertNotSuspended,
assertVerifiedAccount,
} from '../../core/http/middleware/gates.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import {
All,
Controller,
Copy,
Delete,
Get,
Head,
Lock,
Mkcol,
Move,
Options,
Propfind,
Proppatch,
Put,
Unlock,
} from '../../core/http/decorators.js';
import type { RouteOptions } from '../../core/http/types.js';
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import {
maskEntryPath,
parseMaskedSharePath,
resolveSharePath,
} from '../../services/fs/sharePathMask.js';
import { Context } from '../../core/context.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import { toLegacyEntry } from '../fs/legacyFsHelpers.js';
import { PuterController } from '../types.js';
@@ -42,6 +64,9 @@ import {
hasWritePermission,
refreshLock,
} from './locks.js';
import { DAV_CONCURRENT, DAV_LIMIT } from '../fs/limits.js';
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
import type { ResolvedShare } from '../../services/share/ShareService.js';
const DAV_HEADERS = {
DAV: '1, 2, ordered-collections',
@@ -54,6 +79,43 @@ const ALLOW_METHODS =
// macOS creates these files; reject them to keep the FS clean.
const MACOS_JUNK_REGEX = /(?:^\.DS_Store$|^\._)/;
/**
* Every DAV verb mounts on the same catch-all: on a DAV host the path _is_ the
* resource, so there is nothing else to route on. `{*splat}` rather than plain
* `*splat` because only the braced form also matches `/`, which is the first
* collection a client PROPFINDs.
*/
const DAV_ROUTE = '/{*splat}';
/**
* Everything the route table declares. The subdomain keeps DAV off every other
* host, and both limits key on the network fingerprint because nothing has
* authenticated yet when they run — which is the point of putting them here
* rather than inside the handler. A DAV client sends credentials on every
* request anyway, so there is no unauthenticated browsing phase that a per-user
* key would protect.
*/
const DAV_ROUTE_OPTIONS: RouteOptions = {
subdomain: 'dav',
rateLimit: DAV_LIMIT,
concurrent: DAV_CONCURRENT,
};
/** How long a browser may cache a DAV preflight. */
const PREFLIGHT_MAX_AGE = '86400';
/** How many shared items the virtual share collections list. */
const SHARE_LISTING_CAP = 200;
/** What `#context` resolved for one request, handed to every verb handler. */
interface DavContext {
actor: Actor;
/** The real path being addressed, with `~` and any share mask resolved. */
davPath: string;
/** Lock token from `If:` / `Lock-Token:`, when the client sent one. */
lockToken: string | null;
}
/**
* WebDAV controller — full RFC 4918 surface on the `dav.*` subdomain.
*
@@ -65,93 +127,186 @@ const MACOS_JUNK_REGEX = /(?:^\.DS_Store$|^\._)/;
* `-token` username for token-based auth). Falls back to the global authProbe's
* `req.actor` if a session cookie is present.
*/
@Controller()
export class WebDAVController extends PuterController {
registerRoutes(router: PuterRouter): void {
// Single catch-all on the `dav` subdomain. We dispatch by req.method
// inside the handler because WebDAV uses non-standard HTTP verbs that
// Express doesn't have first-class router methods for in all versions.
router.use(
{ subdomain: 'dav' },
async (req: Request, res: Response, _next) => {
try {
await this.#dispatch(req, res);
} catch (err) {
if (err instanceof HttpError) {
res.status(err.statusCode).send(err.message);
return;
}
console.error('[webdav] unhandled error', err);
res.status(500).send('Internal Server Error');
}
// Don't call next — we always handle or error.
},
);
// -- Routes ------------------------------------------------------
//
// One route per verb, every one of them the same catch-all. Declaration
// order is registration order: HEAD before GET, because express falls a
// HEAD request back onto a GET route when it meets one first, and `@All`
// last, because it matches any method.
//
// `DAV_ROUTE_OPTIONS` is the whole declarative half of the chain. The
// credential work happens in `#serve` instead, which is what keeps the
// limits ahead of it: HTTP Basic means a bcrypt compare per request, and a
// caller already over budget shouldn't get one.
@Options(DAV_ROUTE, DAV_ROUTE_OPTIONS)
options(req: Request, res: Response): Promise<void> {
// Before the auth gate, and only for a real preflight — see
// `answerPreflight`.
if (answerPreflight(req, res)) return Promise.resolve();
return this.#serve(req, res, () => this.#options(res));
}
async #dispatch(req: Request, res: Response): Promise<void> {
// Authenticate
const actor = await this.#resolveActor(req, res);
if (!actor) return; // 401 already sent
// GET/HEAD/PUT/COPY move file content or duplicate it in the object store,
// so they carry the same budget gate the FS routes declare with
// `requireCredits`: DAV serves the same files over a metered host, and
// leaving it out would make mounting the drive the way around enforcement.
// The verbs that only describe or remove things stay open, as they do over
// HTTP. HEAD is metered with GET because a client asking for headers is a
// client about to fetch the body.
//
// `requireCredits` itself can't be used for this: it reads `req.actor`, and
// on the DAV host nothing has authenticated by the time route options run.
// Apply the same suspension + pending-verification gates every other
// authenticated route gets from `requireAuthGate` / `requireVerifiedAccount`.
// WebDAV dispatches every method off a single `router.use` with no route
// options, so that middleware is never inserted into its chain — without
// these calls a suspended account (or one still pending email / phone /
// card verification) could read, write, and delete its entire filesystem
// over the `dav` subdomain, bypassing the gates. Both throw a 403
// HttpError, surfaced by the catch in registerRoutes.
@Head(DAV_ROUTE, DAV_ROUTE_OPTIONS)
head(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#get(req, res, ctx, true), {
credits: true,
});
}
@Get(DAV_ROUTE, DAV_ROUTE_OPTIONS)
get(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#get(req, res, ctx, false), {
credits: true,
});
}
@Propfind(DAV_ROUTE, DAV_ROUTE_OPTIONS)
propfind(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#propfind(req, res, ctx));
}
@Proppatch(DAV_ROUTE, DAV_ROUTE_OPTIONS)
proppatch(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#proppatch(res, ctx));
}
@Mkcol(DAV_ROUTE, DAV_ROUTE_OPTIONS)
mkcol(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#mkcol(req, res, ctx));
}
@Put(DAV_ROUTE, DAV_ROUTE_OPTIONS)
put(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#put(req, res, ctx), {
credits: true,
});
}
@Delete(DAV_ROUTE, DAV_ROUTE_OPTIONS)
delete(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#delete(res, ctx));
}
@Copy(DAV_ROUTE, DAV_ROUTE_OPTIONS)
copy(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#copy(req, res, ctx), {
credits: true,
});
}
@Move(DAV_ROUTE, DAV_ROUTE_OPTIONS)
move(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#move(req, res, ctx));
}
@Lock(DAV_ROUTE, DAV_ROUTE_OPTIONS)
lock(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#lock(req, res, ctx));
}
@Unlock(DAV_ROUTE, DAV_ROUTE_OPTIONS)
unlock(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, (ctx) => this.#unlock(req, res, ctx));
}
/** POST, TRACE, an extension verb we don't implement. */
@All(DAV_ROUTE, DAV_ROUTE_OPTIONS)
unsupported(req: Request, res: Response): Promise<void> {
return this.#serve(req, res, () => {
res.status(405)
.set('Allow', ALLOW_METHODS)
.send('Method Not Allowed');
});
}
// -- Request plumbing ---------------------------------------------
/**
* Authenticate, gate, resolve the target path, and run `handler` — or
* answer the request instead when any of that fails. A thrown `HttpError`
* becomes the plain-text reply a DAV client reads; nothing calls `next`,
* because a request that gets here is ours to answer.
*/
async #serve(
req: Request,
res: Response,
handler: (ctx: DavContext) => void | Promise<void>,
opts: { credits?: boolean } = {},
): Promise<void> {
try {
const ctx = await this.#context(req, res, opts);
if (!ctx) return; // 401 already sent
await handler(ctx);
} catch (err) {
sendDavError(res, err);
}
}
/** The actor, path and lock token for one request; null once 401'd. */
async #context(
req: Request,
res: Response,
opts: { credits?: boolean },
): Promise<DavContext | null> {
const actor = await this.#resolveActor(req, res);
if (!actor) return null;
// The same suspension + pending-verification gates every other
// authenticated route gets from `requireAuthGate` /
// `requireVerifiedAccount`. DAV authenticates itself rather than
// through the auth probe, so that middleware is never in its chain —
// without these calls a suspended account (or one still pending email /
// phone / card verification) could read, write, and delete its entire
// filesystem over the `dav` subdomain, bypassing the gates.
assertNotSuspended(actor.user);
assertVerifiedAccount(actor.user);
// The actor was absent when the auth probe ran, so anything that reads
// it off the request or the context — shared-path masking, egress
// metering, which bills the `dav` host — is blind until it's published
// here.
req.actor = actor;
if (Context.current()) Context.set('actor', actor);
if (opts.credits) {
await assertActorHasCredits(
this.services.metering,
actor,
this.config,
);
}
// Expand `~`/`~/...` against the authenticated actor's username.
// WebDAV doesn't standardize `~`, but some clients do — and the
// pre-existing behaviour silently expanded it via the FS store.
const davPath = expandTildePath(
decodeURIComponent(req.path),
actor.user.username,
);
const redis = this.clients.redis;
const lockToken = extractLockToken(
(req.headers['if'] as string | undefined) ??
(req.headers['lock-token'] as string | undefined),
const davPath = await resolveSharePath(
this.stores.fsEntry,
actor,
expandTildePath(decodeURIComponent(req.path), actor.user.username),
);
switch (req.method.toUpperCase()) {
case 'OPTIONS':
return this.#options(res);
case 'HEAD':
case 'GET':
return this.#get(
req,
res,
actor,
davPath,
req.method === 'HEAD',
);
case 'PROPFIND':
return this.#propfind(req, res, actor, davPath);
case 'PROPPATCH':
return this.#proppatch(res, davPath, redis, lockToken);
case 'MKCOL':
return this.#mkcol(req, res, actor, davPath, redis, lockToken);
case 'PUT':
return this.#put(req, res, actor, davPath, redis, lockToken);
case 'DELETE':
return this.#delete(res, actor, davPath, redis, lockToken);
case 'COPY':
return this.#copy(req, res, actor, davPath, redis, lockToken);
case 'MOVE':
return this.#move(req, res, actor, davPath, redis, lockToken);
case 'LOCK':
return this.#lock(req, res, actor, davPath, redis, lockToken);
case 'UNLOCK':
return this.#unlock(req, res, davPath, redis);
default:
res.status(405)
.set('Allow', ALLOW_METHODS)
.send('Method Not Allowed');
}
return {
actor,
davPath,
lockToken: extractLockToken(
(req.headers['if'] as string | undefined) ??
(req.headers['lock-token'] as string | undefined),
),
};
}
// -- Auth ---------------------------------------------------------
@@ -238,7 +393,7 @@ export class WebDAVController extends PuterController {
}
// Build a session-less actor for the user
return {
return makeActor({
user: {
id: user.id,
uuid: user.uuid,
@@ -258,7 +413,7 @@ export class WebDAVController extends PuterController {
requires_card_verification:
user.requires_card_verification ?? false,
},
};
});
}
// -- OPTIONS ------------------------------------------------------
@@ -280,8 +435,7 @@ export class WebDAVController extends PuterController {
async #get(
req: Request,
res: Response,
actor: Actor,
davPath: string,
{ actor, davPath }: DavContext,
headOnly: boolean,
): Promise<void> {
const entry = await this.stores.fsEntry.getEntryByPath(davPath);
@@ -294,15 +448,14 @@ export class WebDAVController extends PuterController {
await this.#assertRead(actor, davPath);
const etag = `"${entry.uuid}-${Math.floor(entry.modified ?? entry.created ?? 0)}"`;
const modified = entry.modified ?? entry.created ?? 0;
const etag = entryEtag(entry.uuid, modified);
const size = entry.size ?? 0;
res.set({
'Accept-Ranges': 'bytes',
'Content-Length': String(size),
'Last-Modified': new Date(
entry.modified ?? entry.created ?? 0,
).toUTCString(),
'Last-Modified': entryDate(modified).toUTCString(),
ETag: etag,
});
@@ -330,43 +483,172 @@ export class WebDAVController extends PuterController {
async #propfind(
req: Request,
res: Response,
actor: Actor,
davPath: string,
{ actor, davPath }: DavContext,
): Promise<void> {
const depth = req.headers.depth ?? '1';
const entry =
davPath === '/'
? null // root always exists
: await this.stores.fsEntry.getEntryByPath(davPath);
if (davPath !== '/' && !entry)
if (davPath === '/') {
await this.#assertRead(actor, davPath);
this.#sendMultistatus(res, await this.#rootPropfind(actor, depth));
return;
}
const entry = await this.stores.fsEntry.getEntryByPath(davPath);
// The two levels a share mask invents — `/<owner>` and
// `/<owner>/<uuid>` — have no entry behind them, so they answer as
// virtual collections. Tried before the ACL check because `/<owner>`
// _is_ a real path (the owner's home directory) that the recipient
// can't read: a 403 there would hide every share they do have.
const virtual =
(await this.#shareOwnerPropfind(actor, davPath, entry, depth)) ??
(entry
? null
: await this.#shareRootParentPropfind(actor, davPath, depth));
if (virtual) {
this.#sendMultistatus(res, virtual);
return;
}
if (!entry) {
throw new HttpError(404, 'Not Found', { legacyCode: 'not_found' });
}
await this.#assertRead(actor, davPath);
const isDir = davPath === '/' || !!entry?.isDir;
const responses = [propfindEntry(davPath, entry, isDir)];
// `davPath` is the resolved real path; the href has to be the masked
// one, like every child below it, or the self-entry names the owner's
// real folder.
const responses = [
propfindEntry(maskEntryPath(entry), entry, entry.isDir),
];
if (depth !== '0' && isDir && entry) {
if (depth !== '0' && entry.isDir) {
const children = await this.services.fs.listDirectory(
entry.uuid,
{},
);
for (const child of children) {
responses.push(propfindEntry(child.path, child, child.isDir));
}
} else if (depth !== '0' && davPath === '/') {
// Root: list top-level user directories
const rootEntry = await this.stores.fsEntry.getEntryByPath(
`/${actor.user!.username}`,
);
if (rootEntry) {
responses.push(
propfindEntry(rootEntry.path, rootEntry, rootEntry.isDir),
propfindEntry(maskEntryPath(child), child, child.isDir),
);
}
}
this.#sendMultistatus(res, responses);
}
/**
* The root collection: the caller's own home directory, plus one collection
* per person who has shared something with them.
*
* A share is addressed `/<owner>/<uuid>/<name>`, where the uuid stands in
* for the folder the owner keeps it in. Only the `<owner>` segment is a
* child of the root, so without this level nothing shared is reachable from
* a mount at all — which is why a Finder mount showed only the caller's own
* files.
*/
async #rootPropfind(actor: Actor, depth: string | string[]) {
const responses = [propfindEntry('/', null, true)];
if (depth === '0') return responses;
const home = await this.stores.fsEntry.getEntryByPath(
`/${actor.user!.username}`,
);
if (home) {
responses.push(
propfindEntry(maskEntryPath(home), home, home.isDir),
);
}
const owners = new Set<string>();
for (const share of await this.#sharedWithMe(actor)) {
const owner = share.owner?.username;
// Self-owned entries never appear in the share index, but the home
// directory above already covers them if one ever did.
if (owner && owner !== actor.user!.username) owners.add(owner);
}
for (const owner of owners) {
responses.push(propfindEntry(`/${owner}`, { name: owner }, true));
}
return responses;
}
/**
* PROPFIND responses for the virtual collection at `/<owner>`, listing one
* child per item that owner shared with the actor, or null when `davPath`
* isn't a bare username, is the actor's own, is readable for real, or the
* actor holds nothing of theirs — the caller then continues down the normal
* path and gets the 403 or 404 it would otherwise have given.
*
* Children are the `<uuid>` level rather than the shared item itself: the
* item's own href sits one segment deeper, and a collection may only report
* its direct members. Nothing here reaches past that level, so `Depth:
* infinity` on a sharer's collection still enumerates only what they
* shared, and everything below it goes through the ordinary resolve + ACL
* path.
*/
async #shareOwnerPropfind(
actor: Actor,
davPath: string,
entry: FSEntry | null,
depth: string | string[],
): Promise<string[] | null> {
const segments = davPath.split('/').filter(Boolean);
if (segments.length !== 1) return null;
const owner = segments[0]!;
if (owner === actor.user?.username) return null;
// Every provisioned user's home directory is a real entry and a
// sharer is necessarily provisioned, so no directory here means no
// share to stand in for. Checked before the share lookup so an
// unknown name costs a 404 rather than a listing.
if (!entry?.isDir) return null;
// An owner who shared their whole home directory has a real listing
// here, and it beats the virtual one.
if (await this.#canRead(actor, davPath)) return null;
const shares = (await this.#sharedWithMe(actor)).filter(
(share) => share.owner?.username === owner,
);
if (shares.length === 0) return null;
const responses = [propfindEntry(davPath, { name: owner }, true)];
if (depth !== '0') {
for (const share of shares) {
responses.push(
propfindEntry(
`/${owner}/${share.entryUid}`,
{
name: share.name,
uuid: share.entryUid,
modified: share.modified,
},
true,
),
);
}
}
return responses;
}
/**
* What has been shared with the actor, for the virtual collections above.
* One page: WebDAV has no way to ask for the next one, so a caller holding
* more shares than {@link SHARE_LISTING_CAP} sees the oldest that many.
*
* Listing also teaches the request's path masker every root it returns, so
* an entry reached through one keeps that masked prefix on the way back out
* instead of being masked against itself.
*/
async #sharedWithMe(actor: Actor): Promise<ResolvedShare[]> {
const page = await this.services.share.listSharedWithMe(actor, {
limit: SHARE_LISTING_CAP,
});
return page.items;
}
#sendMultistatus(res: Response, responses: string[]): void {
res.status(207)
.set({ 'Content-Type': 'application/xml; charset=utf-8' })
.send(wrapMultistatus(responses.join('\n')));
@@ -376,16 +658,10 @@ export class WebDAVController extends PuterController {
async #proppatch(
res: Response,
davPath: string,
redis: unknown,
lockToken: string | null,
{ davPath, lockToken }: DavContext,
): Promise<void> {
if (
!(await hasWritePermission(
redis as import('ioredis').Cluster,
davPath,
lockToken,
))
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
) {
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
}
@@ -401,10 +677,7 @@ export class WebDAVController extends PuterController {
async #mkcol(
req: Request,
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
lockToken: string | null,
{ actor, davPath, lockToken }: DavContext,
): Promise<void> {
if (davPath === '/')
throw new HttpError(403, 'Cannot create at root', {
@@ -419,11 +692,7 @@ export class WebDAVController extends PuterController {
});
}
if (
!(await hasWritePermission(
redis as import('ioredis').Cluster,
davPath,
lockToken,
))
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
) {
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
}
@@ -451,10 +720,7 @@ export class WebDAVController extends PuterController {
async #put(
req: Request,
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
lockToken: string | null,
{ actor, davPath, lockToken }: DavContext,
): Promise<void> {
const name = pathPosix.basename(davPath);
if (MACOS_JUNK_REGEX.test(name)) {
@@ -462,11 +728,7 @@ export class WebDAVController extends PuterController {
return;
}
if (
!(await hasWritePermission(
redis as import('ioredis').Cluster,
davPath,
lockToken,
))
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
) {
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
}
@@ -511,13 +773,11 @@ export class WebDAVController extends PuterController {
);
const fe = writeResult.fsEntry;
const etag = `"${fe.uuid}-${Math.floor(fe.modified ?? fe.created ?? 0)}"`;
const modified = fe.modified ?? fe.created ?? 0;
res.status(existing ? 204 : 201)
.set({
ETag: etag,
'Last-Modified': new Date(
fe.modified ?? fe.created ?? 0,
).toUTCString(),
ETag: entryEtag(fe.uuid, modified),
'Last-Modified': entryDate(modified).toUTCString(),
})
.end();
}
@@ -526,17 +786,10 @@ export class WebDAVController extends PuterController {
async #delete(
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
lockToken: string | null,
{ actor, davPath, lockToken }: DavContext,
): Promise<void> {
if (
!(await hasWritePermission(
redis as import('ioredis').Cluster,
davPath,
lockToken,
))
!(await hasWritePermission(this.clients.redis, davPath, lockToken))
) {
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
}
@@ -557,18 +810,11 @@ export class WebDAVController extends PuterController {
async #copy(
req: Request,
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
lockToken: string | null,
{ actor, davPath, lockToken }: DavContext,
): Promise<void> {
const destPath = this.#parseDestination(req);
const destPath = await this.#parseDestination(req);
if (
!(await hasWritePermission(
redis as import('ioredis').Cluster,
destPath,
lockToken,
))
!(await hasWritePermission(this.clients.redis, destPath, lockToken))
) {
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
}
@@ -615,13 +861,10 @@ export class WebDAVController extends PuterController {
async #move(
req: Request,
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
lockToken: string | null,
{ actor, davPath, lockToken }: DavContext,
): Promise<void> {
const destPath = this.#parseDestination(req);
const r = redis as import('ioredis').Cluster;
const destPath = await this.#parseDestination(req);
const r = this.clients.redis;
if (!(await hasWritePermission(r, davPath, lockToken)))
throw new HttpError(423, 'Locked', { legacyCode: 'conflict' });
if (!(await hasWritePermission(r, destPath, lockToken)))
@@ -671,12 +914,9 @@ export class WebDAVController extends PuterController {
async #lock(
req: Request,
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
headerToken: string | null,
{ actor, davPath, lockToken: headerToken }: DavContext,
): Promise<void> {
const r = redis as import('ioredis').Cluster;
const r = this.clients.redis;
// ACL must succeed before any lock state is touched — otherwise
// an authenticated user could lock paths they don't own (e.g. `/`)
@@ -738,10 +978,9 @@ export class WebDAVController extends PuterController {
async #unlock(
req: Request,
res: Response,
davPath: string,
redis: unknown,
{ davPath }: DavContext,
): Promise<void> {
const r = redis as import('ioredis').Cluster;
const r = this.clients.redis;
const tokenHeader = req.headers['lock-token'] as string | undefined;
const token = extractLockToken(tokenHeader);
if (!token)
@@ -764,15 +1003,65 @@ export class WebDAVController extends PuterController {
res.status(204).end();
}
/**
* PROPFIND responses for the virtual collection at `/<owner>/<uuid>`, or
* null when `davPath` isn't that shape, the uuid doesn't resolve to an
* entry of `owner`'s, or the actor cannot read the share — the caller falls
* through to 404 in every null case, so an unauthorized probe learns
* nothing about whether the uuid exists.
*/
async #shareRootParentPropfind(
actor: Actor,
davPath: string,
depth: string | string[],
): Promise<string[] | null> {
const parsed = parseMaskedSharePath(davPath);
if (!parsed || parsed.tail !== '') return null;
if (parsed.ownerUsername === actor.user?.username) return null;
const root = await this.stores.fsEntry.getEntryByUuid(parsed.rootUuid);
if (!root) return null;
// Same guard as resolveSharePath: the mask names the owner, so the
// uuid must be theirs.
if (root.path.split('/')[1] !== parsed.ownerUsername) return null;
const allowed = await this.services.acl.check(
actor,
{
path: root.path,
resolveAncestors: () =>
this.services.fs.getAncestorChain(root.path),
},
'read',
);
if (!allowed) return null;
const maskedRoot = `/${parsed.ownerUsername}/${root.uuid}/${root.name}`;
// The uuid segment has no name of its own, so it borrows the shared
// item's — a client that reads `displayname` shows something
// recognizable rather than a bare uuid.
const responses = [propfindEntry(davPath, { name: root.name }, true)];
if (depth !== '0') {
responses.push(propfindEntry(maskedRoot, root, root.isDir));
}
return responses;
}
// -- ACL helpers -------------------------------------------------
async #canRead(actor: Actor, path: string): Promise<boolean> {
return await this.services.acl.check(
actor,
{
path,
resolveAncestors: () => this.services.fs.getAncestorChain(path),
},
'read',
);
}
async #assertRead(actor: Actor, path: string): Promise<void> {
const descriptor = {
path,
resolveAncestors: () => this.services.fs.getAncestorChain(path),
};
const ok = await this.services.acl.check(actor, descriptor, 'read');
if (!ok)
if (!(await this.#canRead(actor, path)))
throw new HttpError(403, 'Permission denied', {
legacyCode: 'permission_denied',
});
@@ -821,21 +1110,72 @@ export class WebDAVController extends PuterController {
// -- Misc helpers ------------------------------------------------
#parseDestination(req: Request): string {
async #parseDestination(req: Request): Promise<string> {
const dest = req.headers.destination as string | undefined;
if (!dest)
throw new HttpError(400, 'Missing Destination header', {
legacyCode: 'bad_request',
});
let raw: string;
try {
const url = new URL(dest, `http://${req.headers.host}`);
return decodeURIComponent(url.pathname);
raw = decodeURIComponent(url.pathname);
} catch {
return decodeURIComponent(dest);
raw = decodeURIComponent(dest);
}
// The destination is addressed the same way as the request path, so a
// client that browsed into a share names its target the same way too.
return resolveSharePath(this.stores.fsEntry, Context.get('actor'), raw);
}
}
// -- Middleware helpers -----------------------------------------------
/**
* Answer a CORS preflight before anything can reject it. True when this request
* was one and has been answered.
*
* A browser sends a credential-less `OPTIONS` ahead of any cross-origin
* PROPFIND/GET/PUT and treats a non-2xx reply as a failed preflight, never
* sending the real request. The authenticated OPTIONS path answers 401, which
* blocked every browser DAV client — the VS Code app's mount included — before
* it could present a token.
*
* Only an actual preflight is answered here: it's the one OPTIONS that carries
* `Access-Control-Request-Method`. A native client's plain OPTIONS still goes
* through auth, so macOS keeps getting the 401-with-`DAV:` reply it opens a
* mount with.
*/
function answerPreflight(req: Request, res: Response): boolean {
if (!req.headers.origin || !req.headers['access-control-request-method']) {
return false;
}
// `Allow-Origin`/`-Methods`/`-Headers` were set on the way in by the
// server's CORS middleware. `Max-Age` is what belongs here specifically:
// without it a browser re-preflights every URL every few seconds, doubling
// the request count of a mount and its share of the DAV rate limit.
res.status(200)
.set({
...DAV_HEADERS,
Allow: ALLOW_METHODS,
'Access-Control-Max-Age': PREFLIGHT_MAX_AGE,
'Content-Length': '0',
})
.end();
return true;
}
/** DAV clients read a status and a plain-text reason, not our JSON error shape. */
function sendDavError(res: Response, err: unknown): void {
if (res.headersSent) return;
if (err instanceof HttpError) {
res.status(err.statusCode).send(err.message);
return;
}
console.error('[webdav] unhandled error', err);
res.status(500).send('Internal Server Error');
}
// -- XML helpers ------------------------------------------------------
function escapeXml(text: string): string {
@@ -851,9 +1191,42 @@ function wrapMultistatus(inner: string): string {
return `<?xml version="1.0" encoding="utf-8"?>\n<D:multistatus xmlns:D="DAV:">\n${inner}\n</D:multistatus>`;
}
/**
* FSEntry timestamps are Unix seconds. Entries with nothing stored fall back to
* an ISO string literal, so both forms have to be accepted here.
*/
function toEpochSeconds(ts: number | string): number {
return typeof ts === 'number'
? Math.floor(ts)
: Math.floor(new Date(ts).getTime() / 1000);
}
/** `Date` takes milliseconds, so entry seconds must be scaled to format them. */
function entryDate(ts: number | string): Date {
return new Date(toEpochSeconds(ts) * 1000);
}
/**
* Opaque validator. Built from seconds so every verb emits the same ETag for an
* entry — a client that gets one value from PROPFIND and another from GET
* treats the resource as changed and re-fetches it on every pass.
*/
function entryEtag(uid: string, ts: number | string): string {
return `"${uid}-${toEpochSeconds(ts)}"`;
}
/**
* What a `<D:response>` needs to describe one resource. An {@link FSEntry}
* satisfies it; the virtual collections a share mask invents pass whatever they
* know, which is sometimes only a name.
*/
type PropfindTarget = Partial<
Pick<FSEntry, 'name' | 'uuid' | 'modified' | 'created' | 'size'>
>;
function propfindEntry(
href: string,
entry: FSEntry | null,
entry: PropfindTarget | null,
isDir: boolean,
): string {
const encodedHref =
@@ -863,14 +1236,13 @@ function propfindEntry(
const created = entry?.created ?? '2025-01-01T00:00:00Z';
const name = entry?.name ?? (pathPosix.basename(href) || '/');
const uid = entry?.uuid ?? 'root';
const modTs = Math.floor(new Date(modified as string).getTime());
let props = `
<D:displayname>${escapeXml(String(name))}</D:displayname>
<D:getlastmodified>${new Date(modified as string).toUTCString()}</D:getlastmodified>
<D:creationdate>${new Date(created as string).toISOString()}</D:creationdate>
<D:getlastmodified>${entryDate(modified).toUTCString()}</D:getlastmodified>
<D:creationdate>${entryDate(created).toISOString()}</D:creationdate>
<D:resourcetype>${isDir ? '<D:collection/>' : ''}</D:resourcetype>
<D:getetag>"${uid}-${modTs}"</D:getetag>
<D:getetag>${entryEtag(uid, modified)}</D:getetag>
<D:supportedlock>
<D:lockentry><D:lockscope><D:exclusive/></D:lockscope><D:locktype><D:write/></D:locktype></D:lockentry>
<D:lockentry><D:lockscope><D:shared/></D:lockscope><D:locktype><D:write/></D:locktype></D:lockentry>
@@ -880,7 +1252,7 @@ function propfindEntry(
if (!isDir && entry) {
props += `\n <D:getcontentlength>${entry.size ?? 0}</D:getcontentlength>`;
const mime = mimeFromExt(pathPosix.extname(entry.name));
const mime = mimeFromExt(pathPosix.extname(name));
props += `\n <D:getcontenttype>${escapeXml(mime)}</D:getcontenttype>`;
}
+33 -2
View File
@@ -21,6 +21,10 @@ import type { Request, Response } from 'express';
import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterController } from '../types.js';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
} from '../../services/metering/consts.js';
/**
* WISP relay token controller — create and verify short-lived JWT tokens for
@@ -32,12 +36,39 @@ export class WispController extends PuterController {
registerRoutes(router: PuterRouter): void {
router.post(
'/wisp/relay-token/create',
{ subdomain: 'api', requireAuth: true },
{
subdomain: 'api',
requireAuth: true,
// Auth is optional in practice (the handler tolerates an
// anonymous actor), so the key falls back to a fingerprint
// when there is no user to key on.
rateLimit: {
scope: 'wisp-token-create',
limit: 60,
window: 60_000,
key: 'user',
bySubscription: {
[DEFAULT_FREE_SUBSCRIPTION]: 30,
[DEFAULT_TEMP_SUBSCRIPTION]: 10,
},
},
},
this.#create,
);
router.post(
'/wisp/relay-token/verify',
{ subdomain: 'api', requireAuth: false },
{
subdomain: 'api',
requireAuth: false,
// Unauthenticated by design, which makes it a token-guessing
// oracle without a ceiling.
rateLimit: {
scope: 'wisp-token-verify',
limit: 300,
window: 60_000,
key: 'ip',
},
},
this.#verify,
);
}
+104
View File
@@ -0,0 +1,104 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import {
assertResolvedActor,
makeActor,
SYSTEM_ACTOR,
userRelatedActor,
type Actor,
} from './actor';
describe('makeActor / effectiveApp', () => {
const user = { uuid: 'u-1', id: 1, username: 'u' };
it('resolves a plain user actor to no app', () => {
expect(makeActor({ user }).effectiveApp).toBeNull();
});
it("resolves an app-under-user actor to its own app", () => {
expect(makeActor({ user, app: { uid: 'app-1' } }).effectiveApp).toEqual({
uid: 'app-1',
});
});
it("resolves a token to the app that issued it", () => {
// The whole point: a token actor carries no `app` of its own, so
// anything reading `app` sees a bare user token and skips app gating.
const issuer = makeActor({ user, app: { uid: 'app-1' } });
const token = makeActor({
user,
accessToken: { uid: 'tok-1', issuer },
});
expect(token.app).toBeUndefined();
expect(token.effectiveApp).toEqual({ uid: 'app-1' });
});
it('collapses a chain of tokens in one hop', () => {
const issuer = makeActor({ user, app: { uid: 'app-1' } });
const inner = makeActor({ user, accessToken: { uid: 't1', issuer } });
const outer = makeActor({
user,
accessToken: { uid: 't2', issuer: inner },
});
expect(outer.effectiveApp).toEqual({ uid: 'app-1' });
});
it('resolves a user-issued token to no app', () => {
const issuer = makeActor({ user });
const token = makeActor({
user,
accessToken: { uid: 'tok-1', issuer, fullAccess: true },
});
expect(token.effectiveApp).toBeNull();
});
it('drops the app when narrowing to the underlying user', () => {
const app = makeActor({ user, app: { uid: 'app-1' } });
expect(userRelatedActor(app).effectiveApp).toBeNull();
});
it('resolves the system actor', () => {
expect(SYSTEM_ACTOR.effectiveApp).toBeNull();
});
});
describe('assertResolvedActor', () => {
it('passes a resolved actor through unchanged', () => {
const actor = makeActor({ user: { uuid: 'u-1' } });
expect(assertResolvedActor(actor)).toBe(actor);
// `null` is a resolved answer, not a missing one.
expect(assertResolvedActor({ user: {}, effectiveApp: null })).toEqual({
user: {},
effectiveApp: null,
});
});
it('throws on an actor that skipped makeActor', () => {
// The field is optional so pre-existing literals still compile, which
// means an unresolved one can reach a gate. Fail loudly at the edge
// rather than let a gate read `undefined` as "no app" and wave it
// through — an app-under-user actor is the dangerous case.
const unresolved = { user: { uuid: 'u-1' }, app: { uid: 'app-1' } };
expect(() => assertResolvedActor(unresolved as Actor)).toThrow(
/effectiveApp/,
);
});
});
+53 -11
View File
@@ -47,6 +47,26 @@ export interface ActorAccessToken {
export interface Actor {
user: Partial<UserRow>;
app?: ActorApp | null;
/**
* The app this actor ultimately acts as: its own `app`, or failing that the
* app of whoever issued its access token.
*
* Read this — not `app` — in any gate asking "which app is doing this?". An
* access-token actor carries no `app` of its own, so `app` alone reads as
* "no app" even for a token an app minted, and a gate keyed off it fails
* open exactly where it must not.
*
* `null` and `undefined` are not the same thing here:
*
* - `null` — resolved, and this actor is not acting as any app.
* - Absent — never resolved, because the actor skipped `makeActor`.
*
* A gate must not read the second as the first: that is the fail-open this
* field exists to prevent. Optional only so an actor literal that predates
* the field still compiles; `assertResolvedActor` is what keeps the request
* path honest, and `makeActor` is the one place the derivation lives.
*/
effectiveApp?: ActorApp | null;
/** True for the system actor; skips metering / quota tracking. */
system?: boolean;
accessToken?: ActorAccessToken | null;
@@ -67,9 +87,41 @@ export const SYSTEM_ACTOR_UUID = '5d4adce0-a381-4982-9c02-6e2540026238';
/** The default system actor used when no actor is supplied. */
export const SYSTEM_ACTOR: Actor = {
user: { uuid: SYSTEM_ACTOR_UUID, username: 'system' },
effectiveApp: null,
system: true,
};
/**
* Build an actor, deriving `effectiveApp` from its own app and, failing that,
* from the app of whoever issued its access token.
*
* The issuer was itself built here, so its chain is already collapsed — one hop
* is enough, and no caller has to walk anything.
*/
export const makeActor = (actor: Omit<Actor, 'effectiveApp'>): Actor => ({
...actor,
effectiveApp: actor.app ?? actor.accessToken?.issuer.effectiveApp ?? null,
});
/**
* Fail closed on an actor whose `effectiveApp` was never derived.
*
* Every actor on the request path is built by `AuthService` through
* `makeActor`, so this cannot fire in production — which is the point. It turns
* a future actor literal that skips the builder into a loud 500 at the edge
* rather than a silent bypass deep inside a gate that read `undefined` as "no
* app". Call it once, where the request actor is established.
*/
export const assertResolvedActor = (actor: Actor): Actor => {
if (actor.effectiveApp === undefined) {
throw new Error(
'actor was built without `makeActor`: `effectiveApp` is unresolved, ' +
'and app-scoped gates would read that as "no app"',
);
}
return actor;
};
export const isSystemActor = (actor: Actor | undefined | null): boolean => {
return !!actor?.system || actor?.user?.uuid === SYSTEM_ACTOR_UUID;
};
@@ -106,15 +158,5 @@ export const actorUid = (actor: Actor): string => {
*/
export const userRelatedActor = (actor: Actor): Actor => {
if (!actor.app && !actor.accessToken) return actor;
return { user: actor.user };
};
/**
* Walk the access-token issuer chain and return the first app identity found,
* or null if no app appears anywhere in the chain.
*/
export const effectiveActorApp = (actor: Actor): ActorApp | null => {
if (actor.app) return actor.app;
if (actor.accessToken) return effectiveActorApp(actor.accessToken.issuer);
return null;
return { user: actor.user, effectiveApp: null };
};
+33 -19
View File
@@ -61,23 +61,22 @@ export type LegacyErrorCodes =
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
/**
* Options accepted by `HttpError`. All optional.
*/
/** Options accepted by `HttpError`. All optional. */
export interface HttpErrorOptions {
/** Underlying error. Set as the standard `Error.cause`. */
cause?: unknown;
@@ -89,25 +88,38 @@ export interface HttpErrorOptions {
legacyCode?: LegacyErrorCodes | (string & {});
/**
* Modern, structured error code. If both `legacyCode` and `code` are set,
* the legacy one takes the `code` slot in the response body and `code`
* is emitted as `errorCode`, so clients keying on either field find
* what they expect.
* the legacy one takes the `code` slot in the response body and `code` is
* emitted as `errorCode`, so clients keying on either field find what they
* expect.
*/
code?: string;
/** Additional fields merged into the response body. */
fields?: Record<string, unknown>;
/**
* Skip the terminal alarm gate for this error. Set it only where the call
* site already knows the failure is expected and arrives in proportion to
* traffic — an upstream rate limit on a zero-cost model, say. Never
* serialized to the client.
*/
noAlarm?: boolean;
}
/**
* The single error type controllers and services throw to surface an HTTP
* failure. The terminal `errorHandler` middleware catches it, serializes a
* JSON body, and sets the response status.
* failure. The terminal `errorHandler` middleware catches it, serializes a JSON
* body, and sets the response status.
*
* Usage:
*
* ```ts
* throw new HttpError(404, 'Item not found');
* throw new HttpError(409, 'Cannot overwrite directory', { legacyCode: 'is_directory' });
* throw new HttpError(403, 'Forbidden', { legacyCode: 'forbidden', fields: { target } });
* throw new HttpError(409, 'Cannot overwrite directory', {
* legacyCode: 'is_directory',
* });
* throw new HttpError(403, 'Forbidden', {
* legacyCode: 'forbidden',
* fields: { target },
* });
* ```
*
* Express 5 forwards thrown errors (sync and async) to error-handling
@@ -118,6 +130,7 @@ export class HttpError extends Error {
readonly legacyCode?: LegacyErrorCodes | (string & {});
readonly code?: string;
readonly fields?: Record<string, unknown>;
readonly noAlarm?: boolean;
constructor(
statusCode: number,
@@ -133,6 +146,7 @@ export class HttpError extends Error {
this.legacyCode = options.legacyCode;
this.code = options.code;
this.fields = options.fields;
this.noAlarm = options.noAlarm;
}
}
+79
View File
@@ -0,0 +1,79 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { Controller, Get, Propfind } from './decorators.ts';
import { PuterRouter } from './PuterRouter.ts';
interface Registrable {
registerRoutes: (router: PuterRouter) => void;
}
const collect = (controller: Registrable) => {
const router = new PuterRouter();
controller.registerRoutes(router);
return router.routes;
};
@Controller('/demo')
class DemoController {
@Get('/thing')
thing() {}
@Propfind('/{*splat}', { subdomain: 'dav' })
browse() {}
}
describe('route decorators', () => {
it('collects each decorated method once, in declaration order', () => {
const routes = collect(new DemoController() as unknown as Registrable);
expect(routes.map((r) => r.method)).toEqual(['get', 'propfind']);
expect(routes[1]!.options).toEqual({ subdomain: 'dav' });
});
it('does not re-collect when a second instance is constructed', () => {
// Method initializers run per instance but collect onto the shared
// prototype, so without a guard a process that builds two servers
// registers every route on the second one twice over.
const first = collect(new DemoController() as unknown as Registrable);
const second = collect(new DemoController() as unknown as Registrable);
const third = collect(new DemoController() as unknown as Registrable);
expect(second).toHaveLength(first.length);
expect(third).toHaveLength(first.length);
expect(third.map((r) => r.method)).toEqual(['get', 'propfind']);
});
it('binds handlers to the instance that registered them', () => {
@Controller()
class Bound {
marker = 'mine';
seen: string | undefined;
@Get('/x')
handle() {
this.seen = this.marker;
}
}
const instance = new Bound();
const [route] = collect(instance as unknown as Registrable);
(route!.handler as unknown as () => void)();
expect(instance.seen).toBe('mine');
});
});
+15 -1
View File
@@ -143,7 +143,21 @@ const createMethodDecorator = (method: Exclude<RouteMethod, 'use'>) => {
const proto = Object.getPrototypeOf(
this as object,
) as DecoratedPrototype;
getOrInitRoutes(proto).push({
const routes = getOrInitRoutes(proto);
// Initializers run once per *instance* but collect onto the
// shared prototype, so a process that builds a second server
// (every multi-server test file) would otherwise register
// every route twice. The decorated method and the path literal
// are the same references each time, so identity recognizes a
// repeat without comparing option objects.
const already = routes.some(
(r) =>
r.handler === (target as unknown as RequestHandler) &&
r.method === method &&
r.path === path,
);
if (already) return;
routes.push({
method,
path,
options,
Loaded 100 of 641 files, more files were not shown because too many files have changed in this diff. Show more