Compare commits

..
204 Commits
Author SHA1 Message Date
Daniel Salazar 1f1e5674d8 chore: bump puter js ver (#4073) 2026-10-04 17:17:38 -07:00
Daniel Salazar 8db4fb4f5b fix(ai): chat credit gate ceiling, in-flight holds and disconnects (#4071)
* docs(agents): no credits or bug-report mentions in commits and PRs

* fix(ai): chat credit gate ceiling, in-flight holds and disconnects

- Cap output at min(max_tokens, context - prompt). The gate used
  max_tokens - prompt, so any prompt longer than a model's output limit
  was refused with 402 regardless of balance. Gemini 2.5 Pro's output
  limit corrected to 65,536.
- Abort the upstream stream when the caller disconnects (driver calls and
  the OpenAI/Anthropic-compatible routes), which ends the credit hold.
- A request refused only because the account's own in-flight requests
  hold its balance gets 429 `credits_reserved` instead of 402; puter.js
  retries it with exponential backoff (1s to 30s, about 2 minutes).
- When the balance caps a completion and it uses the whole cap, the
  response carries `usage_limited` (non-stream `metadata`, stream `usage`
  line), which puter.js already turns into the upgrade prompt.

* perf(ai): read the balance and its holds once per credit gate

The gate re-read holds, and the whole balance when any were held, to tell
spent from committed. getUsageHeadroom returns both halves from the same
reads getRemainingUsage already did.
2026-10-04 17:10:51 -07:00
Daniel Salazar e7c89b0639 fix(apps): keep ids longer than an app uid out of the app-stats query (#4070)
ClickHouse's app_opens.app_uid is FixedString(40) and fails the whole
query on a longer value. Marketplace callers pass item ids into
getAppsStats, so one long listing-request id turned the seller's
listing-request list into a 500. Such ids can't have opens; they now
resolve to zero without being queried.
2026-10-04 17:03:19 -07:00
Daniel Salazar 83d64ac685 fix: follow-ups from the reviews of #4047, #4049, #4051, #4053, #4055 and #4057 (#4068)
* fix(auth): issue reauth tokens only for browser sessions

A rejected token's 401 carried a signed reauth_token, which /signup turns
into a session on a temp account. Any revoked app or access token, a stale
token from a deleted app whose uid was reused, or a worker credential could
get one. Only a GUI/browser session's rejection now carries a reauth_token;
everything else still gets reauth_required without it. The stale-app check
also runs before the session is touched.

* fix(events): refuse worker creation from an events handler

workers.create minted a worker token with no handler depth, so a handler
could deploy a worker and chain writes through it at depth 0. A handler
actor can no longer create a worker; delivery, deploys and hot reload mint
from plain owner actors and are unaffected.

* fix(ai-ocr): bound the page count's parsing and close its under-counts

- Object-stream headers are capped (/N, /First) and read incrementally, and
  the inflate budget scales with the file, so a crafted PDF can't spend
  seconds and gigabytes before the credit check.
- An escaped /ObjStm type, an indirect /Count, untyped page leaves and an
  object redefined to a smaller tree no longer lower the count.
- An account with no balance is refused before the file is parsed.

* fix(fs): add a switch to stop accepting pre-binding signed URLs

Signed URLs issued before owner binding are still accepted, and owner-signed
ones never expire, so the move-to-another-tree attack stays open for them
indefinitely. legacy_file_signatures (default on) can now turn them off, a
rate-limited warning counts their use so it's visible when they've died
out, and a non-UUID uid is refused before the entry lookup.

* fix(fs): hide the issuer's home uid from a scoped token's entries

A token scoped below the home could still learn the home's uid as the
parent_uid of a direct child (e.g. stat on ~/Documents). For access-token
actors that can't list the home, a direct child's parent uid is now null in
the v2 and legacy entry shapes, using the same rule as the root listing.

* fix(gui): ask for the 2FA code on the last step, right before enabling

The setup flow verified a code before the recovery-codes screen and sent
that same code to enable afterwards; by then it had usually expired, and
the user was sent back to the start. The code is now entered last and goes
straight to enable; a wrong code keeps the user on that step, and other
failures show the server's message. Enter submits the password step.

* test(fs): expect the legacy-signature flag in the signing config

* fix: apply the review's follow-ups to the merged-PR fixes

- Reauth tokens also require the session row to be a browser ('web')
  session, not just the absence of the worker claim.
- The OCR page count only treats an untyped object as a page when it sits in
  a page tree's /Kids, so outline items, widgets and name-tree leaves can't
  inflate an honest PDF's count.
- The signed-URL uid guard checks the uuid's shape only, so a legacy row
  whose uuid has nonstandard version/variant bits still resolves.
2026-10-04 14:46:21 -07:00
Daniel Salazar 5e34258da5 fix: drop database_id from the subdomain reclaim; MySQL has no such column (#4067)
reclaimByUuid (#4064) cleared `database_id`, which exists only in the SQLite
and Postgres schemas. On MySQL the UPDATE fails with "Unknown column", so
ensureSystemSite throws: the puter-profiles row stays registered to the
admin, the site keeps 404ing, and every profile save returns 500 as it
retries the bootstrap. puter-app-icons takes the same path if its row isn't
already protected. System rows never carry a database binding, so there is
nothing to clear.
2026-10-04 02:45:03 -07:00
Daniel Salazar 51eba174b3 fix: alert errors from timeouts (#4066) 2026-10-04 02:44:40 -07:00
Daniel Salazar f533eb84ae fix(ai-ocr): check credits for the whole document before calling the provider (#4055)
* fix(ai-ocr): check credits for the whole document before calling the provider

The pre-flight checked one page's cost because the page count was only
known from the provider's answer, so a 130-page PDF passed on a balance
worth a fraction of it and drove the account negative.

The driver now estimates pages before the call: a PDF's own count (read
from its page tree, including compressed object streams), capped by a
`pages` selection; one page for images and Textract, whose synchronous
API reads single-page documents. Other documents and PDFs it can't read
count 20 pages per MB, Mistral's 1,000-page limit at its 50 MB size cap.
The estimated cost is checked and held while the provider runs, so
concurrent calls see it. Usage is still metered from the pages the
provider reports.

* fix(ai-ocr): price the credit hold at the AI cost factor

Billing scales OCR usage by the ai.cost.factor hook, but the hold was sized
at the raw cost, so a balance covering N pages passed the check and was
then charged more. Hold through reserveAiCredits like the other AI drivers.
2026-10-04 00:53:19 -07:00
Nikhil Jangid 3dfc70cab7 fix: clarify team directory app access (#4031) 2026-10-04 00:51:02 -07:00
Daniel Salazar ebf5086e1c fix(fs): keep restore metadata out of reach of anyone but the tree's owner (PUT-2050) (#4063)
Follow-up to #4048. original_path/original_name decide where a GUI restore
lands, and the owner's restore is unchecked, so whoever sets them can aim a
file at another app's AppData root and block that app's launch. #4048 only
pinned them on move for non-account actors; a write could still set them,
and a share recipient's session counts as account context.

Writes now drop the trash keys for every caller. On move, only the account
that owns both source and destination may set them; anyone else moving into
the owner's Trash gets them recomputed from the entry, and other moves drop
them, including keys already on the row. Copies drop them on every branch.
The GUI only offers Restore for direct children of Trash, since nested rows
never carry legitimate trash metadata. The AppData/Trash checks for
cross-app deletes and create grants now ignore case, like the root guard.
2026-10-04 00:48:48 -07:00
Daniel Salazar 59e28bed23 fix(metering): keep a failed lookup's fallback out of the credit cache (PUT-2002) (#4062)
Follow-up to #4052. A failed subscription lookup was cached for 5 s, but the
credit cache built from its fallback plan lived 15 s, so a paid user who
had spent past the free allowance got 402s for ~15 s after one failed read.
Credit entries built from a provisional answer (a failed lookup, or an
actor missing its email or numeric id) are now capped at that answer's
expiry and don't drive credit-state alerts.

A failed lookup that finishes after a concurrent successful one no longer
replaces the good cached answer, actors without a numeric id aren't cached
(resolvers key on it), and a failed credit refresh fails open briefly
instead of replaying a stale answer or blocking on the failing store.
2026-10-04 00:48:35 -07:00
Daniel Salazar 73ebe3f47a fix(metering): decide app-usage access in MeteringService only (PUT-2015) (#4061)
The /metering/usage/:app handler repeated the "an app reads only its own
usage" check while the service still let any app read the global bucket.
The service now refuses an app actor anything but its own app, and the
handler only resolves names. Status codes for app actors are unchanged; a
user reading os-global gets their global usage instead of a name-lookup 404.

whoami no longer sets taskbar_items to undefined for app actors, and only
keys with a value count as built, so a listener can't fill one in for an
app. getMonthlyUsage's JSDoc notes allowanceInfo is account-wide.
2026-10-04 00:47:37 -07:00
Daniel Salazar c981a51f21 fix(fs): bind signed file URLs to the owner they were issued under (#4053)
New /sign signatures are an HMAC over uid, action, expiry and the entry's
owner at signing time. /file and /writeFile resolve the entry first and
verify against its current owner, so a move that hands the entry to another
account ends its URLs and stops /file minting child URLs from them. The URL
shape is unchanged; signatures in the previous format still verify, without
the owner check.
2026-10-04 00:47:16 -07:00
Daniel Salazar 4ef5e6c110 fix(socket): hold unlisted free plans to the free connection cap (PUT-1830) (#4060)
The per-account socket cap looked a plan up directly in its map, so
org_seat_free (team seats on a free team) got the paid cap. Route and
events limits already gave an unlisted free plan the default free entry;
that rule now lives in one helper, subscriptionOverride, used by all three.

Also publishes the socket connection caps, corrects the seat-plan line in
rate-limits-and-quotas.md, and makes the isFreeSubscription test assert
literal ids.
2026-10-04 00:46:59 -07:00
Daniel Salazar 794469845d fix(thumbnails): bind thumbnail keys to their entry and size-bound uploads (PUT-1999) (#4054) 2026-10-04 00:46:35 -07:00
Daniel Salazar 9ca77be5a9 fix(auth): require a step-up to set up 2FA and a valid code to enable it (#4051)
Setup returned a fresh secret and recovery codes to any session, and enable
flipped otp_enabled without proof of the secret, so a stolen session could
enroll its own authenticator and lock the owner out. Setup now lives at
/user-protected/setup-2fa behind the same password / OIDC revalidation gate as
disable-2fa, and enable requires a live code for the stored secret. The GUI
setup window asks for the password (or opens the revalidation popup) first,
and sends the code it already collected to enable.
2026-10-04 00:26:28 -07:00
Daniel Salazar 007d5391a1 fix(events): carry handler depth onto tokens a handler mints (#4049)
An events handler could mint a fresh token for its own app through /sign or
/auth/create-access-token, and writes made with it started a new chain. An app
token minted from a handler token now carries its handler_depth and expires
with it; an access token carries the depth at the lifetime asked for.
2026-10-04 00:25:54 -07:00
Daniel Salazar c0f4fe93b1 fix(auth): stop a deleted app's sessions authenticating as the next app with its uid (PUT-2049) (#4057) 2026-10-04 00:24:55 -07:00
Daniel Salazar d57f638eda fix: keep system sites writable and their names out of reach (#4064)
Follow-up to #4056. Icons and profiles are now written as the system user,
so on a storage-limited install they counted against its allowance (100 MB
by default) and every write past that returned 413, including a user's
profile save. Both writes now pass the unmetered allowance, as writes the
system makes on users' behalf already may.

ensureSystemSite reclaims a mismatched subdomain row in place instead of
delete+create, so the name is never free for a user to register, the row
keeps its uuid, and `protected` is never cleared; the previous holder's
bindings are dropped. If the row vanished since the read it falls through
to create. puter-app-icons is now protected like puter-profiles, both names
are reserved from user registration, and they live in util/systemSite so
the subdomain driver doesn't import the services. The icon service's
shutdown wait is capped at 5 s so a stalled run can't hold up the metering
flush.
2026-10-04 00:23:01 -07:00
Daniel Salazar 26abd4e4a5 fix(fs): hide the issuer's home from scoped-token root listings (PUT-2053) (#4047) 2026-10-04 00:22:43 -07:00
Daniel Salazar 3defae3cfc fix: own system sites by the system user (#4056)
* fix: register system sites to the owner of their directory

The hosting middleware refuses a site whose user doesn't own its root
directory (absent a manage grant). The app-icons and profiles bootstraps
always registered their subdomain to the admin, but the directory can belong
to another user: a carried-over `/system` owned by the `system` user, which a
new child directory inherits. Those sites then 404 on every request.

Both bootstraps now go through ensureSystemSite, which registers the
subdomain to the directory's owner and replaces a row registered to anyone
else or pointing at another directory.

Fixes #4045

* fix: own system sites by the system user

`/system`, `/system/app_icons` and `/system/profiles` and their subdomains
belong to the system user, and icon and profile files are written as it.
ensureSystemSite re-owns directories another account created and replaces a
subdomain row registered to anyone else, so installs bootstrapped under the
admin move over on boot.

The bootstrap no longer waits for an admin account, so DefaultUserService
stops re-running it, and deployments without an admin get icons and profiles.
AppIconService lets in-flight icon runs finish on shutdown.
2026-10-03 17:06:13 -07:00
Daniel Salazar 205f19a167 fix(metering): stop caching subscription answers from email-less actors and failed lookups (PUT-2002) (#4052) 2026-10-03 16:39:08 -07:00
Daniel Salazar 2cfc1461e4 fix: scope whoami billing details and monthly usage to the calling app (PUT-2015) (#4050)
* fix: scope whoami billing details and monthly usage to the calling app (PUT-2015)

* fix(metering): refuse an app another app's detailed usage (PUT-2015)
2026-10-03 16:38:31 -07:00
Daniel Salazar 3fa23bb809 fix(fs): refuse renaming or moving AppData roots (PUT-2050) (#4048)
* fix(fs): refuse renaming or moving AppData roots (PUT-2050)

* fix(fs): refuse a forged trash original_path from apps (PUT-2050)
2026-10-03 16:37:36 -07:00
Daniel Salazar e59f6285ba refactor(metering): route free-plan checks through isFreeSubscription (PUT-1830) (#4046) 2026-10-03 16:36:05 -07:00
Filip Kujundžić c3abec8663 Merge pull request #4035 from HeyPuter/filipkujundzic/put-1895-chat-credit-gate-cost-factor
fix(ai): price the chat credit gate at the factored cost (PUT-1895)
2026-10-03 19:44:51 +02:00
Neal Shah 04406373f4 event based FBL implementation (#3969) 2026-10-03 05:54:04 -04:00
Daniel Salazar 91aff909e1 fix: keep launch tokens out of URLs and storage (PUT-2040) (#4038)
* fix: keep launch tokens out of URLs and storage (PUT-2040)

- Strip the consumed launch token param from the app URL after load
- Never persist user session (godmode) tokens to localStorage; purge any stored one at boot
- Worker log socket sends the token as its first message instead of in the URL path

* fix: strip the private-app sign-in token from top-level URLs, keep godmode tokens across reloads (PUT-2040)

- Private-app gate: a top-level GET carrying ?puter.auth.token= that passes the gate gets the sticky cookie and a 302 to the same URL without the token. Framed launches are untouched.
- Godmode session tokens go to sessionStorage (never localStorage), so a frame reload stays signed in; cleared on sign-out or when an app token replaces it.
2026-10-03 00:26:31 -07:00
Daniel Salazar a242105540 fix(events): keep app-less scoped tokens off the account's durable rows and mailbox (#4042)
A personal access token scoped below the account (e.g. the one in an
account-minted getReadURL() URL) has effectiveApp === null, which the durable
list/unsubscribe/ack scope check and notif fetch read as account context. Gate
those on isAccountContext, as workers/handlers/kv-handles already do, and refuse
such tokens on durable subscribe.
2026-10-02 22:37:07 -07:00
Reynaldi Chernando 47510897ed add ai builder link to docs (#4044) 2026-10-03 10:27:23 +07:00
dependabot[bot] d0da8c5b1e chore(deps): bump engine.io from 6.6.9 to 6.6.11 (#4017)
Bumps [engine.io](https://github.com/socketio/socket.io) from 6.6.9 to 6.6.11.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@6.6.11)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-version: 6.6.11
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 15:03:34 -07:00
dependabot[bot] 303ee6c2c7 chore(deps-dev): bump brace-expansion from 1.1.18 to 1.1.21 (#4015)
Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 1.1.18 to 1.1.21.
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.18...v1.1.21)

---
updated-dependencies:
- dependency-name: brace-expansion
  dependency-version: 1.1.21
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 15:02:56 -07:00
Daniel Salazar 37c1678897 fix(fs): keep a vacated username off-limits to other accounts for an hour (#4039)
* fix(fs): keep a vacated username off-limits to other accounts for an hour

Renaming a home rewrites descendant paths in the database, but their
cached entries keep the old path until they expire. ACL grants a home by
path prefix, so whoever takes the old name next must not be able to
before those entries are gone. renameUserHome now records the vacated
name, and the claim check every username-claim site already runs treats
it as taken for anyone but the account that left it.

* test: seed recommended apps that are still in the default list

The default list no longer includes editor or camera, so the ordering
test found neither.
2026-10-02 14:53:34 -07:00
Filip KujundžićandClaude Opus 5.5 7b8b3752e4 fix(ai): price the chat credit gate at the factored cost
The chat gate read `model.costs` directly to decide affordability, cap
`max_tokens` and size the credit hold, while usage is recorded at cost
times the AI cost factor, so all three ran low by the factor on every
request. The gate now resolves the factor and applies it to both rates.

The factor is looked up by the key the provider records usage under.
Providers expose that key through `meteringModelKey` and use it when
recording, so the gate and the charge can't drift apart; the
unreported-stream backstop records under the same key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:21:18 +02:00
404oopsandClaude Opus 5.5 521812be02 refactor(ai): move the AI cost factor out of MeteringService
MeteringService knew about AI: it built the cost-factor facade, resolved
the `ai.cost.factor.*` hook and, since the previous commit, offered
`reserveAiCredits`. Metering stays agnostic to what it meters, so all of
that now lives with the AI drivers in `drivers/util/aiCostFactor.ts`.

`withAiCostFactor(metering, events, driver)` returns an
`AiMeteringService`: the service with recording scaled by the factor,
plus `costFactor` and `reserveAiCredits` for gates. The speech providers
take that type. MeteringService itself has no AI methods left.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 21:20:24 +02:00
Nariman Jelveh bb38e8c86f Update RecommendedAppsService.ts 2026-10-02 11:29:42 -07:00
Nariman Jelveh 74d89b64fd Update RecommendedAppsService.ts 2026-10-02 11:19:37 -07:00
Daniel Salazar 31e5b64a0f fix: make app-data delete grants imply write and read (PUT-1998) (#4022) 2026-10-02 10:47:58 -07:00
Daniel Salazar e96cf77982 feat(metering): per-month allowance override for partly billed plan changes (PUT-1996) (#4027) 2026-10-02 10:40:52 -07:00
Filip KujundžićandClaude Opus 5.5 22d52cf544 fix(ai): name data-URL audio with its real extension
A data URL's filename came from its MIME subtype, so `audio/mpeg`
became `input.mpeg` and `audio/x-wav` became `input.x-wav`. OpenAI's
transcription API picks the decoder from the extension and rejected
both. Map the MIME types whose subtype isn't the extension.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 19:28:13 +02:00
Daniel Salazar 86bb94d3d9 fix: cap FS batch array lengths (PUT-1894) (#4024) 2026-10-02 10:06:21 -07:00
Daniel Salazar a4f1e665df fix: return 400 when a streamed upload is shorter than its declared size (#4023)
A streamed write sends the caller's declared size to the object store as
the body length. A body that ends short of it was rejected as
IncompleteBody and surfaced as a 500.
2026-10-02 10:05:13 -07:00
Filip KujundžićandClaude Opus 5.5 abd0d809d5 fix(ai): price speech credit gates at the charged cost and hold them in flight
Text-to-speech, speech-to-text and voice-changer providers checked
affordability against the unscaled provider cost and never reserved it,
so the gate ran low by the AI cost factor and concurrent requests could
each spend the whole balance. They now go through
`reserveAiCredits`, which scales the amount by the model's cost factor,
checks it, and holds it until the request finishes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:56:53 +02:00
Filip KujundžićandClaude Opus 5.5 82bd7a7360 chore(ai): sync model catalogs with provider listings and deprecations
Remove models providers have deprecated or retired, add newly released
ones, and move provider defaults off retired models. Gemini video (Veo
via the Gemini API) is removed; Infron and OpenRouter skip models their
listings mark deprecated. xAI bills cached prompt tokens only at the
cached rate. Claude Sonnet 4.6 max output raised to 128K.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 18:52:35 +02:00
Reynaldi Chernando b32ae7dcfc Add rate limit detail for contact form recipe (#4030)
* Add rate limit detail for contact form recipe

* handle ipv6
2026-10-02 23:19:41 +07:00
Juan Fernando Castro d5e6d5ea07 Merge pull request #4007 from HeyPuter/juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant
fix: refuse opaque origins cleanly instead of a logged 400 (PUT-1896)
2026-10-02 09:58:13 -04:00
Reynaldi ChernandoandCopilot Autofix powered by AI f46d187d51 contact form recipe (#4029)
* contact form recipe

* Potential fix for pull request finding 'Network failures cause unhandled fetch rejections'

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* minor

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-10-02 20:24:05 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 0fcee62884 Update recipes for storing a small list and store per id (#4028)
* Update recipes for storing a small list and store per id

* Potential fix for pull request finding 'Clarify that IDs must be path-safe or properly escaped'

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-10-02 19:23:39 +07:00
Reynaldi Chernando 384524a027 improve teams docs (#4025) 2026-10-02 14:45:54 +07:00
Daniel Salazar 42b204f46e fix: don't count failed username change towards rate limit (#4021) 2026-10-02 00:26:37 -07:00
Daniel Salazar 91c8b2385b fix: run a broadcast persistent handler in the worker or the clients, not both (PUT-1889) (#4016) 2026-10-02 00:08:02 -07:00
jelveh 05e2e7428a feat(gui): let users remove their profile picture
The account tab only offered changing the avatar. Add a "Remove photo"
action, shown while a picture is set, that clears it via
update_profile({ picture: null }) and resets every avatar to the default.
On failure the hint line says so and the picture stays.

The tab can render before the profile loads, so the profile loader also
reveals the button once a picture arrives. Also move the avatar hint
into i18n.
2026-10-01 23:21:57 -07:00
dependabot[bot] f387a7aa68 chore(deps): bump fast-uri from 3.1.7 to 3.1.8 (#4014)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.7 to 3.1.8.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.8)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 18:39:45 -07:00
Daniel Salazar e6bda950f0 perf: batch per-app origin, canonical and hosted-backing lookups in app listings (PUT-1892) (#4013) 2026-10-01 18:30:06 -07:00
Daniel Salazar 4f31388dad fix: cap app lookups and fields, dedupe icon processing, rate-limit app landing pages (PUT-1891) (#4012) 2026-10-01 17:54:25 -07:00
dependabot[bot] 376a047ecc chore(deps): bump @grpc/grpc-js from 1.14.4 to 1.14.5 (#4010)
Bumps [@grpc/grpc-js](https://github.com/grpc/grpc-node) from 1.14.4 to 1.14.5.
- [Release notes](https://github.com/grpc/grpc-node/releases)
- [Commits](https://github.com/grpc/grpc-node/compare/@grpc/[email protected]...@grpc/[email protected])

---
updated-dependencies:
- dependency-name: "@grpc/grpc-js"
  dependency-version: 1.14.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:50:14 -07:00
dependabot[bot] fe7b0ce535 chore(deps): bump axios from 1.18.1 to 1.20.0 (#4008)
Bumps [axios](https://github.com/axios/axios) from 1.18.1 to 1.20.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.18.1...v1.20.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.20.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:49:54 -07:00
dependabot[bot] e85c6917f7 chore(deps): bump fastify from 5.12.3 to 5.12.5 (#4009)
Bumps [fastify](https://github.com/fastify/fastify) from 5.12.3 to 5.12.5.
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/commits/v5.12.5)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.12.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 17:49:42 -07:00
Juan Castro 989399a857 Merge branch 'main' into juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant 2026-10-01 20:30:41 -04:00
Juan Castro 143948ed2c Merge branch 'main' into juancastro/put-1896-file-and-sandboxed-iframe-origins-get-a-hard-400-and-cant 2026-10-01 20:30:00 -04:00
Juan Fernando Castro e48dc0cbf3 Merge pull request #4011 from HeyPuter/juancastro/put-1997-bound-grant-and-token-route-inputs
fix: bound the inputs on the grant and token routes (PUT-1997)
2026-10-01 20:29:42 -04:00
Juan Castro a67cdac380 fix: bound the inputs on the grant and token routes
Three permission routes sized their work by the request body rather than by
the route, so one call could cost far more than its rate-limit slot implies.

- `extra` and `meta` were only type-checked. They ride every row a grant
  writes (up to 16) and are re-serialised into the per-(user, app) cache on
  each miss, so they are now capped at 4 KiB.
- `/auth/create-access-token` took a `permissions` array of any length: one
  permission check and one sequential INSERT each. It now uses the same
  16-per-request cap the grant routes already had, and runs every entry
  through the validator those routes use.
- Withdrawing an app's cross-app data grants looks them up by permission
  text, and no index on `user_to_app_permissions` led with `permission`.
  Indexed per engine, mirroring what mysql_mig_22 / postgres_mig_11 /
  sqlite 0067 did for `user_to_user_permissions`.

`grant-dev-app` validated none of its input — not even the type of `extra` —
so it now runs the same validator as its user-app sibling.

Two paths could also carry a permission wider than the `varchar(255)` column
it lands in. `grantUserAppPermission` already rejected that after rewriting;
`grantDevAppPermission` now does the same, and `createAccessToken` checks it
before the session row a failing INSERT would otherwise orphan.

Caps are published in rate-limits-and-quotas.md. Every in-tree caller sends
one permission and a small `extra`, so none of them change behaviour.
2026-10-01 16:51:39 -04:00
dependabot[bot] c6da95beb0 chore(deps-dev): bump undici from 7.29.0 to 7.30.0 (#3991)
Bumps [undici](https://github.com/nodejs/undici) from 7.29.0 to 7.30.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.29.0...v7.30.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.30.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 09:52:01 -07:00
Daniel Salazar ed7b8f828a fix: kv path errors and ttl hardening, events subscription end and race fixes (#4004) 2026-10-01 09:50:34 -07:00
Juan Castro 29c6f78b7c fix: refuse opaque origins cleanly instead of a logged 400
A page the browser gives no origin to — one opened straight from disk, or an
iframe sandboxed without `allow-same-origin` — serialises its origin as
`"null"`. `appUidFromOrigin` logged that at error level on every attempt and
the GUI sent it on three endpoints, so each refusal cost a 400 and an
error-level line while the user got nothing useful.

An opaque origin cannot name an app to mint a token for, and is not a valid
`postMessage` target to deliver one to, so these clients are refused rather
than supported. Make the refusal quiet, early and legible instead:

- AuthService: drop the `console.error` and say what the caller should do.
  The accept/reject set is unchanged; a 400 is already counted per route.
- puter.js: `signIn()` rejects with `unsupported_origin` before opening
  anything, which covers implicit auth too since it routes through `signIn`.
  The load-time warning now covers sandboxed iframes, console-only — a modal
  belongs in nobody else's embed.
- GUI: a popup whose opener has no attested origin says so and closes,
  instead of rendering a blank window and wedging on a failed exchange.

Also fixes two faults this made reachable: `PuterDialog.open()` prefers the
popup branch, which reaches the `puter` global before the constructor that
assigns it has returned (`puter is not defined`, no SDK at all); and
`showModal()` throws where modals are blocked. `openNotice()` does neither.

Ordinary http(s) sign-in is unaffected.
2026-10-01 11:09:04 -04:00
Juan Fernando Castro c542e4675b Merge pull request #4000 from HeyPuter/juancastro/put-2036-shared-file-issue
fix: let a share recipient create entries in a folder shared with them
2026-10-01 09:56:09 -04:00
Daniel SalazarandReynaldi Chernando d3b46a6b86 docs: easy kv recipes (PUT-1962) (#3999)
* docs: easy kv recipes (PUT-1962)

* docs: publish the append-to-list kv recipe (PUT-1962)

* docs: bare-object add rejects with invalid_path (PUT-1962)

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 18:03:07 +07:00
Juan Fernando Castro fd8b8288d3 Merge pull request #3988 from HeyPuter/juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-30 20:28:02 -04:00
Daniel Salazar fc62ea2ee1 Revert "variable color for text in input fields (#3996)" (#4001)
This reverts commit 8862420594.
2026-09-30 16:12:19 -07:00
Juan Castro d1b11c793c Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a
One conflict, in workerInvoker.integration.test.ts: #3997 fixed the same
gap-marker flake this branch did, with a different barrier. Took theirs
— waiting on the `ev:qf` counter, which `#handlerFailed` writes strictly
after `discard`, so the marker is in and the lease cleared by the time it
reads 1. Their ordering also asserts the counter before flipping `answer`
back to 200, which closes the redelivery-clears-failures hole this
branch's version had removed the line to avoid.

Converted the two `vi.waitFor` calls #3997 added to `waitUntil`, the
helper this branch introduced for the rest of the file. Neither is wrong
today, since the new block fakes no timers, but a `jump()` added near
them later would hit exactly the drained-backoff trap the helper exists
for.
2026-09-30 17:49:36 -04:00
Juan Castro d6b7c8ff52 docs: drop the replica-lag testing runbook from the open-source tree
It describes our local two-server MySQL replication setup, which is
internal tooling rather than anything a self-hoster or contributor
needs. It now lives in the heyputer repo alongside the other internal
docs.

The suite it documented stays: the header comment now states the
env-var gate directly instead of pointing at a file this repo no
longer carries.
2026-09-30 17:42:04 -04:00
Daniel Salazar 4448103ef0 fix: misc events and kv issues found during recipes (#3997)
* fix: misc events and kv issues found during recipes

* test: cover handler depth on narrowed actors and created notifications

* test: fix stale legacy batch directory upload test and seat user-agent flake

* test: wait for the refusal to be recorded before claiming its gap marker
2026-09-30 14:40:47 -07:00
Juan Castro 79e496e79d test: wait on the gap marker rather than a depth that never moves
The refused-delivery test waited for `depth(subId)` to reach 1 before
claiming the marker, but `discard` removes the entry and appends the
marker in its place, so the depth is 1 before, during and after. The
wait returned at once, and the claim came back `inflight` — the refused
delivery still held the lease it was invoked on — leaving the assertion
reading `undefined`. It only passed on the slack in `invoked`'s poll
interval, which a loaded runner takes away.

Waiting on the claim itself is the barrier the depth was standing in
for. Dropping `answer = 200` with it: `beforeEach` already resets it,
and a redelivery settling before the last assertion would clear the
failure count it reads.
2026-09-30 16:36:57 -04:00
Juan Castro 2e0bf26cac Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-30 16:31:42 -04:00
Juan Castro b126cf9c5a fix: carry the owner's path into WebDAV events and expand tilde before touch
WebDAV emits the same GUI events the FS controllers do, and had the same
confusion: the payload was masked for the actor and addressed to the owner. A
recipient mounting a shared folder in Finder reproduced the disappearing-item
symptom exactly.

/touch prefixed `/` before expanding, so `~/AppData/<app>/f.txt` became a literal
`/~/...` that no longer looked like a tilde path — the shape puter.js sends for
every app-relative write answered 404.
2026-09-30 16:21:39 -04:00
Juan Castro ec4d3b31cf fix: let a share recipient create entries in a folder shared with them
A recipient addresses the owner's entries by a masked `/<owner>/<uuid>/<name>`
path. Routes that resolve an existing row unmask it on the way in, but the ones
that create a new entry had nothing to resolve and ran the ACL check against the
mask itself, which names no row — so mkdir, touch and the batch write/mkdir/
shortcut ops answered 404 `subject_does_not_exist` inside any shared folder.
They now expand client paths the same way every other legacy route does.

The GUI events carried the same confusion the other way: the payload was masked
for whoever made the request but addressed to the entry's owner, who cannot
resolve another user's mask. The owner's client dropped the row by uid and then
failed to re-add it under a path it had never heard of, so anything a recipient
touched vanished from the owner's open window until a refresh. Those events now
publish the owner's real path; responses to the actor stay masked.

Also fixes the desktop's `item.moved` handler reading `metadata` as an object
when the wire carries a JSON string: the name of a trashed item came out empty
(trashing renames the entry to its uid), and re-encoding the string left the
restore path parsing a string instead of an object. The parse the dashboard and
the explorer already open-coded is now one helper.
2026-09-30 15:28:02 -04:00
hairyEagle 8862420594 variable color for text in input fields (#3996) 2026-09-30 11:41:32 -07:00
Neal ShahandReynaldi Chernando 3b4f3f6d06 PUT-1959 PUT-1960 subdomains & worker recipes (#3993)
* PUT-1960 worker recipes

* subdomain recipes

* fixes

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 01:33:29 +07:00
Juan Fernando CastroandReynaldi Chernando 3d1357455f docs: teams recipe and playground examples (PUT-1966) (#3967)
* docs: add the easy teams recipe and playground examples

Covers the read-only surface an app actually has: list(), listMembers()
and listDirectory(). Adds the teams recipe tag the build validates
against, and a Teams section to the playground.

Each example handles the two states that are easy to confuse: teams
turned off on the deployment (not_found) and a team that has not opened
its directory to apps (team_not_found).

* docs: open the teams recipe with what it lets an app build

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-10-01 00:11:42 +07:00
Juan Castro f8d8003fea test: stop the delivery-backoff test reading holds against a moving clock
`holds one it could not answer, for longer each time` freezes `Date` so a
hold written as `now + backoff` reads back as exactly `backoff`. But every
wait in the file went through `vi.waitFor`, which advances the faked clock
once per poll — measured at 50ms a poll. Each poll between the hold being
written and read therefore ate into the value: CI read a 2s hold as 525ms
and failed, on main one run and on this branch the next.

Polling now goes through `waitUntil`, which retries on `performance.now`
and real `setTimeout` — neither faked — so the clock only moves where
`jump` moves it. The holds then read exactly 2s/4s/8s/16s, so the
assertion is an equality rather than a one-second window that a slow
enough runner could always slide out of.
2026-09-30 12:25:40 -04:00
Juan Castro 23c8f73bcb test: verify the replica-lag fix against real MySQL replication
The unit tests inject the driver error because sqlite has neither a read
replica nor foreign-key enforcement, so neither half of this bug can
occur locally — they prove the handling, not that the situation arises.

This suite runs two MySQL servers in real replication and freezes the
follower with STOP REPLICA, so the row really is gone on the primary and
really is still served by the replica, and MySQL raises the foreign-key
errors itself. Against the pre-fix code all four cases fail, including
the unhandled "Cannot add or update a child row" the reported 500s were.

Opt-in behind PUTER_TEST_REPLICA_LAG, matching the env-gated provider
integration tests and their skipUnlessEnv helper, so CI and anyone
without the containers skip it. It builds and drops its own database
rather than touching local dev data.
2026-09-30 10:02:36 -04:00
Daniel Salazar b9efac2363 fix: misc fixes from alert logs (#3995) 2026-09-30 00:51:27 -07:00
Daniel Salazar ad12da2942 feat: add gpt 6.1 sol to our direct apis (#3994) 2026-09-29 17:38:40 -07:00
Neal Shah 9aa3989e9d fix PUT-1888 (#3992) 2026-09-29 17:06:28 -04:00
dependabot[bot] cd37cf147f chore(deps): bump socket.io-client from 4.8.3 to 4.8.4 (#3982)
Bumps [socket.io-client](https://github.com/socketio/socket.io) from 4.8.3 to 4.8.4.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@4.8.4)

---
updated-dependencies:
- dependency-name: socket.io-client
  dependency-version: 4.8.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 12:56:15 -07:00
Filip Kujundžić f60d94ef3f Merge pull request #3987 from HeyPuter/fix/put-1874-chat-test-mode
Fix chat test mode routing to fake provider
2026-09-29 19:19:44 +02:00
Juan Castro 8bce62e880 Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a 2026-09-29 13:13:01 -04:00
Juan Castro abd27c0192 fix: replicate tombstones, and attribute foreign-key failures correctly
Review of the first pass turned up two holes that made it ineffective.

CacheReplicationService always deleted the keys a peer broadcast and
never adopted a payload, so tombstones stopped at the region boundary:
peers dropped the marker, kept caching deleted rows off their own
replicas, and the multi-region case the fix was for stayed broken.
Tombstones are now the one payload a peer adopts — they carry no row,
only the fact of a delete.

user_to_app_permissions keys both the app and the user, and the drivers
don't all name the constraint, so a deleted *account* was being read as a
deleted app: a live app got tombstoned, the request 404'd for the wrong
reason, and the dead account was never retired, so the client retried and
re-tombstoned indefinitely. The insert site now asks the primary which
parent actually went away and answers 404 or 401 to match.

Also: re-check the tombstone after a cache write lands, since a row
cached under a live tombstone is never looked at again; extend the guard
to #refreshCache, which update() could otherwise use to undo a delete;
stop markDeleted tombstoning an address the deleted row never owned,
which was blocking the account that did own it; let the primary rather
than the tombstone decide in batched lookups; tombstone the id key when
no cached row names the others; and correct the leaf-store comment in
stores/index.ts now that SessionStore depends on UserStore.
2026-09-29 13:12:31 -04:00
dependabot[bot] 6a31b2af9e chore(deps): bump libphonenumber-js from 1.13.13 to 1.13.14 (#3984)
Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.13.13 to 1.13.14.
- [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.13.13...v1.13.14)

---
updated-dependencies:
- dependency-name: libphonenumber-js
  dependency-version: 1.13.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 10:11:03 -07:00
dependabot[bot] 3ebb753eb1 chore(deps): bump socket.io from 4.8.3 to 4.8.4 (#3985)
Bumps [socket.io](https://github.com/socketio/socket.io) from 4.8.3 to 4.8.4.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/[email protected]@4.8.4)

---
updated-dependencies:
- dependency-name: socket.io
  dependency-version: 4.8.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 10:08:32 -07:00
Juan Fernando CastroandReynaldi Chernando ee4d55027d docs: easy perms recipe (PUT-1965) (#3972)
* docs: add the easy perms recipe

Covers the supported surface: request() and check(). Adds the perms
recipe tag the build validates against.

* docs: add a playground example per operation in the perms recipe

One runnable example for each section: request a folder, request the
email address, request apps or subdomains, and check before prompting.

* docs: open the perms recipe with what it lets an app build

* docs: ground the perms examples in the UI that would ask

A photo editor exporting to Pictures, a checkout emailing a receipt, a
launcher listing apps, and a settings toggle that reads its own state
without raising a prompt.

* refine

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-09-30 00:00:34 +07:00
dependabot[bot] 7eb9ace733 chore(deps-dev): bump prettier from 3.9.8 to 3.9.9 (#3986)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.8 to 3.9.9.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.8...3.9.9)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 09:51:06 -07:00
dependabot[bot] 14ae6a69ee chore(deps): bump postal-mime from 3.0.0 to 3.0.1 (#3983)
Bumps [postal-mime](https://github.com/postalsys/postal-mime) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/postalsys/postal-mime/releases)
- [Changelog](https://github.com/postalsys/postal-mime/blob/master/CHANGELOG.md)
- [Commits](https://github.com/postalsys/postal-mime/compare/v3.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: postal-mime
  dependency-version: 3.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 09:50:45 -07:00
Juan Fernando CastroandReynaldi Chernando 3b00cd2d14 docs: easy fs sharing recipe (PUT-1958) (#3973)
* docs: add the easy fs sharing recipe

share(), listShared(), getShares() and unshare() — the fs ground the
existing recipes do not already cover.

* docs: add a playground example per operation in the fs sharing recipe

One runnable example for each section of the recipe: share, mode,
listShared, getShares and unshare.

* docs: open the sharing recipe with what it lets an app build

* docs: ground the sharing examples in a document-editor flow

Each example now sits in the UI that would call it — a Share button, a
role picker, a shared-with-me inbox, an access list, ending a review.

* remove redundant examples and refine recipe

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-09-29 23:47:23 +07:00
Juan Castro 785304e807 fix: stop deleted apps and users from being resurrected by a lagging replica
A delete lands on the primary and invalidates the cache, but a concurrent
request that misses the cache reads a replica that hasn't applied it yet,
finds the row, and writes it back under a fresh TTL — 24h for apps, 15m
for users. Every write referencing that resurrected row then fails its
foreign key and escapes as a raw driver error, so the route 500s until
the TTL expires. Origin-bootstrapped app uids are deterministic, so the
same origin keeps hitting the stale row and never re-bootstraps.

Deletes now tombstone their cache keys before dropping them. While a
tombstone is live the read path goes to the primary and the write path
refuses to cache; re-creating a row clears it. As a backstop, a write
rejected for a missing parent is answered as one: 404 for a vanished app,
401 for a vanished account, with the stale entry retired either way.

Also fixes cascadeDelete's identity lookup, which selected a column some
schemas lack and so threw into its catch, skipping the invalidation
entirely; and drops SessionStore's private copy of isUniqueViolation in
favour of the shared helper.
2026-09-29 10:16:48 -04:00
Juan Fernando Castro 8e31d9149a Merge pull request #3966 from HeyPuter/juancastro/put-1994-group-scan-cycle
fix: stop group permission scans re-entering the sharer's own grant (PUT-1994)
2026-09-29 10:12:54 -04:00
Juan Castro 9539c7b553 test: cover team-share access through the real authorization path
The existing team-share suite resolves shares through listings and store
reads, neither of which enters the permission scan, so a scan that never
terminates leaves it green. These ask the question the product asks —
can this member open the file — via ACLService.check.
2026-09-29 09:41:05 -04:00
Juan Fernando Castro 27c4873042 Merge pull request #3971 from HeyPuter/juancastro/put-2033-team-create-read-after-write
fix: read a team back from the primary after writing it (PUT-2033)
2026-09-29 09:06:43 -04:00
404oops 0b1a4283f5 fix(ai): honor chat test mode 2026-09-29 12:17:33 +02:00
Daniel Salazar 1cc091206c fix: PUT-1885 PUT 1950 (#3981)
* fix: app uuid fixes

* fix: bound kv update/add createPaths writes (PUT-1885)

* fix: upload progress socket events on the v2 tracker (PUT-1950)
2026-09-28 23:38:34 -07:00
Daniel Salazar 09887747e0 fix: app uuid fixes (#3980) 2026-09-28 19:43:23 -07:00
Daniel Salazar f80d4ae136 fix: events docs (#3968)
* fix: events docs

* fix: docs
2026-09-28 17:24:37 -07:00
Filip KujundžićandClaude Opus 5.5 42e0c28d0a feat(ai): add Claude Sonnet 5.5 (#3970)
Add claude-sonnet-5-5 ($2/$10 per MTok, 1M context, 128K output) and move
the bare claude-sonnet / claude-sonnet-latest aliases to it. Sonnet 5.5
rejects non-default sampling params and disabled thinking, so it joins the
temperature-omit, effort, and adaptive-thinking gates.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-09-28 15:29:21 -07:00
Juan Castro d888377138 Merge remote-tracking branch 'origin/main' into juancastro/put-2033-team-create-read-after-write 2026-09-28 17:20:43 -04:00
Juan Castro 8a53d14540 fix: read a team back from the primary after writing it (PUT-2033)
create() inserted on the primary then read the row back through the
replica, so a replica that had not caught up answered the same as a team
that was not there and the call threw. Nothing rolled the insert back,
and the owner membership was never written, leaving a team its owner
could not see but which still counted against their cap.

Read back through pread in create() and update(), seed the row cache
with the result, and delete the row if the read-back still finds
nothing. #cached no longer stores a null, which would otherwise hold a
lagging replica's answer for the full TTL.
2026-09-28 16:55:19 -04:00
Juan Fernando Castro 12e6bc3d71 Merge pull request #3950 from HeyPuter/juancastro/put-1988-team-force-2fa
feat: let a team require two-factor authentication of its accounts (PUT-1988)
2026-09-28 16:52:34 -04:00
Filip Kujundžić 359d68e51f Merge pull request #3949 from HeyPuter/fix/ai-ocr-model-sync-and-normalization 2026-09-28 21:23:28 +02:00
Filip Kujundžić 020e61065f Merge pull request #3947 from AnujShrivastava01/anuj/fix/remove-deprecated-ai-chat-models 2026-09-28 21:22:33 +02:00
Juan Castro 480d9d032b Merge remote-tracking branch 'origin/main' into juancastro/put-1988-team-force-2fa
# Conflicts:
#	src/backend/clients/database/SqliteDatabaseClient.test.ts
#	src/backend/clients/database/SqliteDatabaseClient.ts
#	src/backend/clients/database/migrations/mysql/mysql_mig_42.sql
#	src/backend/clients/database/migrations/postgres/postgres_mig_31.sql
2026-09-28 10:24:39 -04:00
Juan Castro 528eb192b5 fix: stop group permission scans re-entering the sharer's own grant
The group path recursed without the anti-cycle state, so a grant issued
into a group the issuer belongs to resolved back into itself and the
scan did not terminate. Thread the scan state through the group scanner
and skip an issuer already on the path, matching the user-to-user path.

Readings produced under a non-empty path are pruned for that path only,
so they are no longer written to or served from the shared scan cache.
2026-09-28 10:05:54 -04:00
Rupanjana ChoudhuryandDaniel Salazar ca52b5236c Fix/3638 shared link account (#3845)
* fix: open shared links with the intended account

* refactor: validate shared account hints in the GUI

* fix: scope the shared-link account hint to plain top-level visits

The hint no longer switches accounts in popups, embeds or action flows, and
only counts next to a share path the GUI would open. It is dropped from the
URL before any switch or prompt, so a reload never asks again. An unsaved
recipient gets a dismissible choice (sign in, or continue as the current
account) instead of a picker with no way out.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-27 18:25:28 -07:00
Neal Shah 153f581011 fix PUT-1872 (#3952) 2026-09-25 22:34:53 -04:00
Neal Shah 1fc2eae1ae self hosted mail (#3932)
* self hosted mail

* update lock

* address daniel review
2026-09-25 21:48:15 -04:00
Daniel Salazar dcb61bc4fd fix: PUT-1824 + PUT-1884 + PUT-1952 + PUT-1827 (#3951) 2026-09-25 17:47:03 -07:00
Juan Fernando Castro b445680f10 Merge pull request #3937 from HeyPuter/juancastro/put-1879-oidc-popup-return-proof-binding
fix(auth): bind the OIDC popup-return proof to its purpose, browser, and popup
2026-09-25 18:04:59 -04:00
Juan Castro 95ee796157 Merge branch 'main' into juancastro/put-1879-oidc-popup-return-proof-binding 2026-09-25 17:22:12 -04:00
Nariman JelvehandDaniel Salazar 3d3678a555 feat: let the Contact Us form carry screenshots and recordings (#3563)
* feat: let the Contact Us form carry screenshots and recordings

The form doubles as our bug-report channel, and the bugs worth reporting
are often the ones that need a picture: a visual glitch, or something that
takes five steps to reach. Up to 5 images or videos may now ride along
with the message, delivered as attachments on the support email.

Nothing the client says about a file is believed. The endpoint takes bare
base64 and re-derives all three of the things that matter from the decoded
bytes:

- Type, sniffed from magic numbers and matched against an allow-list of
  PNG/JPEG/GIF/WebP and MP4/QuickTime/WebM. A declared MIME is never read,
  so it cannot smuggle anything past the list. SVG is excluded on purpose
  (it carries script, and support tooling renders what it is sent), as are
  the non-video brands that share MP4's `ftyp` box -- HEIC, M4A, JPEG 2000.
- The file name, reduced to a display label with the extension taken from
  the sniffed type. PNG bytes named `payload.html` arrive as
  `payload.png`; a name can never carry the CR/LF that would break out of
  a Content-Disposition header, nor the bidi overrides that make
  `report<RLO>gnp.exe` render as `report.exe.mp4`.
- Size: 10 MB per file, 15 MB per submission, counted on decoded bytes.
  Encoded length is capped before decoding, so an oversized payload costs
  a length check rather than a 10 MB allocation. The total stays well
  under the 25 MB most providers enforce, since the outgoing mail base64s
  these again.

Base64 is decoded strictly, reusing the round-tripping decoder that
already guards app icons -- `Buffer.from(s, 'base64')` silently drops
characters it does not recognise, and the round-trip is what rejects
bytes smuggled after the payload. That decoder and the image sniffer move
from appIcon.ts to a new mediaSniff.ts, which gains the video counterpart.

One request is now worth megabytes of parsing and outbound mail, so the
existing per-user rate limit gains a per-IP backstop (which the per-user
counter cannot see through freshly minted accounts), a concurrency cap,
and a Content-Length gate that refuses an impossible body before anything
decodes it.

Payloads are not stored. They ride the email; the new
`feedback.attachments` column records names, types and sizes only, so an
abusive submission stays attributable once the mail has been dealt with.

Also fixes the form posting an empty message when Send was pressed with
nothing typed, and surfaces submit failures instead of leaving the button
disabled with no explanation.

* fix: stream Contact Us attachments as multipart instead of base64 JSON

Base64 in a JSON body went through the global JSON parser before the route
ran: raw buffer, rawBody copy, decoded string, parsed strings, then decoded
Buffers plus a re-encode for the strict check. A max-size submission peaked
around 110-135 MB of heap for 15 MB of files, and the route's Content-Length
check ran after all of it.

Attachments now arrive as multipart/form-data, which the global parser skips.
Auth, rate limit, concurrency and the Content-Length 413 all run before the
body is read, and busboy enforces the count, per-file and total caps while
streaming, so only the decoded file bytes are held (~16 MB peak). On a broken
limit the reader stops and the response closes the connection.

JSON stays supported for message-only posts; a JSON `attachments` field is
rejected.

* fix: deliver Contact Us 413s instead of resetting the upload

Closing the socket on a mid-stream limit sent the 413 and then reset the
connection with the client still uploading, so the client could see a
reset instead of the error. After a limit trips, the rest of the body is
now read and discarded, bounded by the body budget; past the budget the
request is destroyed.

Adds an HTTP-level test through the full middleware stack (FormData
upload, unauthenticated refusal, 413 on declared length before the body
is sent, 413 on a chunked upload over the per-file cap), trims comments
to the AGENTS.md length rule, and drops a box-drawing test divider.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-25 14:19:35 -07:00
hairyEagle a9ee3d1ea6 fix(selfhost): configure bucket CORS in s3-init so browser uploads work (#3934)
Browser uploads (Dev Center deploy, puter.fs.upload) PUT file bytes
directly to presigned URLs on the public S3 endpoint. That cross-origin
PUT is preflighted, and the bucket had no CORS rules, so RustFS answered
without Access-Control-Allow-Origin and the browser blocked the upload.

s3-init now runs put-bucket-cors on every boot after ensuring the bucket
exists (idempotent). Origins are open because the presigned URL is the
credential and the SDK sends no cookies. Document how to re-run and
verify in doc/self-hosting.md.
2026-09-25 12:12:02 -07:00
Juan Castro ee3f928b0e fix: annotate the seat-lookup catch so the baselined typecheck passes 2026-09-25 14:39:23 -04:00
Juan Castro 176c8848f5 fix: let a seat enrol in 2FA from the console, and hold it once required
The Security tab only rendered the 2FA card for accounts with a confirmed
email, so a seat — provisioned without one by design — had no way to
enrol from the UI at all. It was the GUI twin of the enable-route check
fixed earlier: the rule could be required of accounts that could not
satisfy it. The card now shows for a seat too, giving it the ordinary
authenticator flow: QR, manual secret, six-digit code, recovery codes.

Disabling is refused while the seat's team requires 2FA. Without that a
member could switch off the factor their team mandates and lock
themselves out of everything until they enrolled again.
2026-09-25 14:31:58 -04:00
Juan Castro beaf4a9a7c docs: record the team 2FA rule where the other member-facing gates are
No method pages: requiring 2FA and resetting a member's factor are team
administration, which the docs deliberately do not publish — only
`list`, `listMembers` and `listDirectory` are app-callable.

But an app acting for a gated seat does meet `two_factor_required`, and
an undocumented 403 leaves it nothing to go on. It goes beside
`password_change_required` in rate-limits-and-quotas, which already
explains what a team's administration does to a member's requests.
2026-09-25 14:18:30 -04:00
404oopsandClaude Opus 5.5 d36da7522b fix(ai-ocr): address review: playground examples, Textract options, inline size
Adds the Mistral and document-annotation img2txt examples to the
playground, states that Textract takes no options, and drops the
normalize mentions from the img2txt docs and types.

The SDK now checks Mistral inline sources against the size a base64
upload can reach inside the 50 MB JSON body (36 MB) instead of the
10 MB Textract limit; 50 MB let 38-50 MB inputs through to a body-parser
413. Retired-model messages no longer cite vendor dates.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 18:56:29 +02:00
Daniel Salazar e6cfe8c492 fix: codeowners (#3948) 2026-09-25 09:23:26 -07:00
404oopsandClaude Opus 5.5 eef8258539 fix(ai-ocr): sync OCR models and prices, route by model, cap input size
Model sync against Mistral's listing, docs and a live call per id:
mistral-ocr-latest now serves OCR 4.1, so the catalog pins it (with
mistral-ocr-4 and the retired-but-redirected 2503) to mistral-ocr-4-1,
adds mistral-ocr-4-0 and OCR 3 (mistral-ocr-2512 / -3 / -3-0), and
rejects the retired mistral-ocr-2505 before any upstream call. Mistral
usage is metered per model at vendor rates: $4/1000 pages for OCR 4.x
with $5 annotations, $2 and $3 for OCR 3, replacing the flat $1 and $3.

Textract now calls DetectDocumentText instead of AnalyzeDocument with
LAYOUT. The layout feature cost $4/1000 pages while we billed the
$1.50 detect rate, and it only added text-less LAYOUT_* blocks that put
blank lines into img2txt output. Multi-page PDFs and unsupported
formats become a 400 that names Mistral as the alternative.

Request shape: a model name alone now selects its provider, so calls
differ only by model; an explicit provider still picks its default and
must agree with the model. Mistral options gain tableFormat,
extractHeader, extractFooter and documentAnnotationPrompt; annotation
formats accept the REST (json_schema.schema) or SDK
(jsonSchema.schemaDefinition) spelling and are validated before
billing. Non-image documents such as DOCX go as document_url.

Response shape: both providers return model, text and blocks with a
0-based page; Mistral also returns document_annotation, which was
billed but dropped. img2txt resolves to that annotation when
documentAnnotationFormat is set and to the text otherwise; the return
is a string for every model, so normalize has no effect on it.

Input size: OCR input is capped at the provider limit (10 MB Textract,
50 MB Mistral), and web inputs in loadFileInput are streamed under the
cap instead of buffered whole, which also bounds speech2txt URLs.

Docs: img2txt models, options, input limits, return value and errors;
OCR limits in the quotas page.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 18:19:54 +02:00
Juan Castro 3105378fdc fix: let a team seat enrol in 2FA, and fit the rule's note on its card
Found by driving the flow against a live instance: a provisioned seat
could reach `configure-2fa/setup` but never `enable`, which demands a
confirmed email. Seats are provisioned without one on purpose (PUT-1792,
the team being the trust anchor), so requiring 2FA of a team would have
locked out precisely the accounts the rule is for — permanently, since a
seat cannot enrol and the rule never lifts.

The check now passes for an account a team provisioned; every other
account still needs its address confirmed, with tests either way.

The card's note was also a line too long and rendered clipped, so it says
the same thing in the space the layout gives it.
2026-09-25 11:58:32 -04:00
Juan Castro 695a26b272 feat: surface the team 2FA rule in the console and SDK (PUT-1988)
An owner toggle beside the directory one, confirmed on the way on since
it locks out every account without 2FA until they enrol, and unconfirmed
on the way off since that only lifts a requirement.

A per-member Reset 2FA action, shown only while the rule is on — with it
off a member can clear their own factor and needs nobody's help. The
confirmation says what it does and what it leaves alone: signed out
everywhere, told about it, password unchanged.

`require2fa` rides the Team shape and `update()`, and `resetTwoFactor()`
joins the module beside `resetPassword()`, whose docblock already says
2FA is left alone — the two now read as the pair they are.
2026-09-25 11:28:59 -04:00
Juan Castro cab70254b1 feat: let a team owner reset a seat's second factor (PUT-1988)
Forcing 2FA makes lockout materially likelier, and nothing could clear a
factor except the account holding it — `resetMemberPassword` leaves 2FA
alone on purpose, and admin only displays it. A seat that lost both its
authenticator and its recovery codes had no way back.

Its own route and its own audit action rather than folded into the
password reset, so that method keeps the property its comment claims: a
password reset alone is not takeover. The two together are, and for an
account the team created and pays for that is the team's to do — but it
has to be chosen and recorded, not arrived at sideways.

Bounded to `org_owned = 1`: a joined member's own login is not a team's
to reach into. The member is told, and signed out everywhere.
2026-09-25 11:15:23 -04:00
Juan Castro db1a95260a feat: let a team require 2FA of the accounts it provisioned (PUT-1988)
`group.require_2fa`, toggled by the owner through `PUT /teams/:uid` and
recorded in the audit log like `directory_enabled` — it changes what
every seat must hold to sign in, so it is not something a team should be
able to alter silently.

Enforced by deriving, not by stamping a flag on the user: a stamp left
behind by a member leaving, or by the rule being turned off, would lock
someone out of their own account with no team left to clear it. The seat
row already joins `group` and is already read per request, so it carries
the flag and the gate costs nothing extra. Toggling busts those rows, or
turning the rule off would leave members locked out until the TTL lapsed.

The gate rides the same `allowUnconfirmed` opt-out as the other pending-
requirement gates, so the routes a gated member must still reach stay
reachable — `configure-2fa` among them, which would otherwise be walled
off behind the very requirement it clears.

Scoped to `org_owned = 1`: the owner is not a seat of their own team, and
a joined member is an ordinary self-paying account whose personal login
is not a team's to dictate. The owner is held to the rule at the point it
matters — they cannot turn it on without 2FA of their own.
2026-09-25 10:35:24 -04:00
Anuj Shrivastava e041fb6b9b Remove deprecated AI chat models flagged by live model check
OpenAI's o3-pro, Mistral's mistral-large-2512 and zai-glm-5-2, and
DeepSeek's deepseek-v4-flash / deepseek-v4-flash-vision-exp are no
longer served by their upstream APIs, so drop them from the hardcoded
provider model lists (closes HeyPuter/puter#3945).

DeepSeek's legacy chat/reasoner aliases and default model previously
pointed at deepseek-v4-flash; they now resolve to deepseek-v4-pro, the
remaining model in that catalog, so existing callers keep working.

Signed-off-by: Anuj Shrivastava <[email protected]>
Signed-off-by: Anuj Shrivastava <[email protected]>
2026-09-25 19:40:02 +05:30
Nariman Jelveh 5721bdeb1d Add link to AI Builder in README (#3946) 2026-09-24 21:00:04 -07:00
Daniel Salazar 132cae9f6a Bug bash fixes (PUT-1876, 1877, 1880, 1881, 1948, 1949, 1951) and KV event fan-out by plan (#3943)
* fix(kv): keep disableSharing entries out of other apps' events (PUT-1876)

Private kv keys and values are no longer delivered to cross-app subscriptions, including forwarded and durable deliveries.

* fix(fs): refuse usernames whose home path still holds another account's rows (PUT-1877)

Username claims (signup, save_account, change-username, OIDC, team provisioning) now also check for leftover rows under /<name>/, and the root listing only returns the actor's own home.

* fix(puter-js): reconnect the events channel after a server-side disconnect (PUT-1881)

A server hang-up is retried with a bounded backoff; persistent handlers survive it and get an optional onError when the connection can't be restored.

* fix(backend): let the graceful shutdown finish before telemetry exits (PUT-1948)

The telemetry preload no longer exits on signals; the entry point drains, tears down top-down, flushes telemetry last, then exits. DB pools stay open until teardown.

* fix(fs): reserve quota for signed uploads in progress (PUT-1880)

startWrite/startBatchWrite hold each upload's declared size in a per-owner cache reservation until it completes, aborts or expires, so a burst of starts can't all read the same committed usage. Caps in-flight uploads at 10,000 per account.

* fix(fs): refuse to complete a signed upload whose object never arrived (PUT-1951)

Completion now 400s when the object store reports the key missing (transient errors stay lenient), abort no longer deletes an object a live entry still uses, and a ghost entry is only removed after a second miss at its recorded location.

* fix(metering): move usage to v2 keys with a small totals item, per-model detail shards and a 3-month TTL (PUT-1949)

The billed item holds only totals; per-model detail lives in 100 hash-sharded items read in one batch and cached for 60s, and more than 5,000 distinct usage types in a month fold into other. Global and per-app aggregates store totals only, appTotals comes from a prefix listing, a refused path only skips the key that refused it, exact reads near the allowance are throttled per key, and every metering record expires after 3 months. September usage restarts at deploy; the monthly-charge claim stays on the v1 key through September so recurring charges don't fire twice.

* test(events): let the worker backoff test tolerate a loaded run

* feat(events): scale KV event fan-out with the key owner's plan

A KV change delivers to up to 512 matching subscriptions (2,048 filter evaluations) for a paid key owner and 128 (512) otherwise, inline values are dropped once more than 128 match, and paid accounts may hold up to 512 share handles per app. The owner's plan is only looked up when a change has more candidates than any cap.

* test(metering): seed the 150-app listing test directly so it fits under coverage
2026-09-24 16:00:52 -07:00
Neal Shah 4a9e5a2440 Ns/worker no socket (#3944)
* remove socket creation from puter worker startup

* Remove really extra comment
2026-09-24 18:55:10 -04:00
Juan Fernando Castro 600064dcee Merge pull request #3942 from HeyPuter/juancastro/put-1944-teams-hardening
fix: Teams lifecycle robustness — owner deletion, cap serialization, provisioned-account scope (PUT-1944)

fixes requested by Daniel pushed
2026-09-24 16:51:43 -04:00
Juan Castro 66ee50317c style: restore prettier formatting displaced by the main merge 2026-09-24 16:35:09 -04:00
Juan Castro 8f0a317a97 Merge remote-tracking branch 'origin/juancastro/put-1944-teams-hardening' into juancastro/put-1944-teams-hardening 2026-09-24 16:33:40 -04:00
Juan Castro 8c816060a3 fix: answer a cap-lock failure as 409, like contention, per review 2026-09-24 16:33:12 -04:00
Juan Castro 0131ebfe9b Merge branch 'main' into juancastro/put-1944-teams-hardening 2026-09-24 12:20:12 -04:00
Juan Castro ca48477db9 Merge branch 'main' into juancastro/put-1879-oidc-popup-return-proof-binding 2026-09-24 12:19:59 -04:00
Juan Fernando Castro 2cb7516141 Merge pull request #3936 from HeyPuter/juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and
fix(hosting): keep alternate-host bootstrap stubs from shadowing private apps
2026-09-24 12:19:31 -04:00
Juan Castro 0c5c1be777 fix: Teams lifecycle robustness ahead of wider enablement
Closing an account that owns a team left `owner_user_id` NULL while
`deleted_at` stayed NULL, so the team remained live while every team
route resolves by membership — which the deletion cascades away. The team
could then not be administered by anyone, and the accounts it provisioned
could not be closed. Declined now in both directions: live teams via
`countOwned`, and soft-deleted ones still holding accounts via a new
store read, since a deleted team's accounts are suspended rather than
removed.

The cap lock gave up after 200ms and proceeded unserialized, so
overlapping provisioning could each read the seat count before any insert
committed. It declines instead; a single create never contends. An
unreachable lock backend is a 503 rather than a silent pass.

An account a team provisioned can no longer create a team of its own
(PUT-1890) — that is not the intended lifecycle, since such accounts do
not go through the normal sign-up path. Keyed on `org_owned = 1`, so the
owner and joined members are unaffected, and not via `teamsAvailableTo`,
which is permissive by design.

The directory declines scoped access tokens, as `listMembers` already
bounds non-account callers.
2026-09-24 11:06:00 -04:00
Juan Castro 8ecaa77a0e Merge remote-tracking branch 'origin/main' into juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and
# Conflicts:
#	src/backend/controllers/auth/AuthController.test.ts
2026-09-24 10:04:00 -04:00
Neal Shah 07a11240ea PUT-1868 (#3940) 2026-09-24 00:30:47 -04:00
Daniel Salazar 7758aea78c fix: acl check (#3939) 2026-09-23 19:03:16 -07:00
Rasad RegmiandDaniel Salazar 83de67cca9 fix(tools): spawn npm.cmd through a shell on Windows (#3923)
* fix(tools): spawn npm.cmd through a shell on Windows

* fix(tools): join the npm command line on Windows to avoid DEP0190

Passing args alongside shell: true warns on Node 24. Test the spawn
tuple as data and run a real npm spawn instead of mocking child_process.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-23 18:54:43 -07:00
Echa ApriliyantoandDaniel Salazar 9f69483593 feat: allow browser extension origins in auth requests (#3907)
* feat: allow browser extension origins in auth requests

Add chrome-extension://, moz-extension://, safari-extension://, safari-web-extension://, and extension:// to the protocol allow-list so browser extensions can obtain app tokens via /auth/get-user-app-token.

Extract WEB_AND_EXTENSION_PROTOCOLS constant in validation.js so the allow-list is defined once and shared by AuthService, AppStore, and AppDriver.

* fix: harden the extension-origin allow-list

Review follow-ups on the extension-origin change:

- Require a host in `validateUrl`. Only "special" schemes need an
  authority, so `chrome-extension:` parsed with an empty hostname and
  slipped past the reserved-system-host guard in AppDriver.
- Lowercase the host in `AuthService#normalizedOrigin`. `new URL()`
  lowercases http(s) hosts but leaves opaque ones alone, so one
  extension in two spellings resolved to two app uids — two AppData
  trees, two permission sets — and missed the origin blocklist.
- Drop `extension:`. No browser emits it, and it accepted
  `extension://evil.com` as an app origin.
- Freeze the allow-list and derive `validateUrl`'s http(s) default from
  `WEB_PROTOCOLS` so the two spellings can't drift apart.
- Pin the tests to the real uid derivation, use unique extension ids so
  a row left by another test can't mask the bootstrap path, and cover
  the host-less and near-miss schemes.
- Fix the prettier/eslint failure in AppStore.js.

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-09-23 18:24:31 -07:00
Daniel Salazar 606f5a27a3 chore: CODEOWNERS (#3938) 2026-09-23 18:24:09 -07:00
Daniel Salazar 58ed2f485f feat(auth): signup bonus codes
Signup links can carry a bonusCode. Core checks its shape and previews it through `puter.signup-bonus.check`, refuses a dead code before the abuse hook records the attempt, then hands a live one to `puter.signup-bonus.validate` after `puter.signup.validate`, which may raise the phone/card gates; the accepted code rides on `puter.signup.success`, and OIDC carries it in the signed state. `user.card-verified` is now awaited like `user.phone-verified`. The signup form shows the offer and sends the code.
2026-09-23 14:47:34 -07:00
Juan Castro d73933b8a5 fix(auth): bind the OIDC popup-return proof to its purpose, browser, and popup
Tokens under the `oidc-state` scope share one signing key, so the payload is
what says which job a token belongs to. The popup-return proof now carries a
`purpose` claim, and each verifier accepts exactly one kind of token.

The return leg also sets a single-use companion cookie, mirroring the nonce
`/start` already uses, and stamps the action its redirect lands on. Redeeming
a proof requires the matching cookie — consumed on success only, so a rejected
call can't invalidate an in-flight return — and the popup honors only a proof
minted for its own action.

The cookie is host-only, so `/auth/oidc/verify-popup-return` is also served on
the GUI origin and the popup redeems it same-origin. The return leg always
lands the popup on `config.origin`, where both the cookie and the route live.
2026-09-23 17:21:35 -04:00
Daniel Salazar 4c3c684cb4 fix: refuse app-issued scoped tokens on events handler and worker routes (#3935)
* fix: new cache costs for claude

* fix: refuse app-issued scoped tokens on events handler and worker routes

An access token an app mints (e.g. a getReadURL() token) resolves
effectiveApp to the issuing app, so #handlerApp and listEventsWorkers
treated it as the app: it could publish, remove and list handlers, and
list or destroy the app's worker, regardless of its permission manifest.
Refuse any scoped token there, as the kv-handle routes already do.

* test: freeze the clock before the first backoff read in workerInvoker

The first hold was read on real time, so a slow run measured the 2s
backoff as 1s.
2026-09-23 13:58:49 -07:00
Juan Castro 438a4584de fix(hosting): keep alternate-host bootstrap stubs from shadowing private apps
A public origin-bootstrap app row planted on an alternate hosting domain
could resolve ahead of the owner's private app and skip the private
access gate (PUT-1883).

- get-user-app-token canonicalizes hosted-subdomain origins before uid
  derivation and bootstrap, so every hosting variant shares one app row
- the app create/update conflict check crosses hosting-domain variants,
  so an existing alt-host stub is absorbed by the owner's create
- resolveOwnedAppForHostedSite and the canonical-uid lookup order
  matches private-first with a deterministic id tiebreak
2026-09-23 16:39:22 -04:00
Daniel Salazar b3fda186e7 fix: let apps read opted-in teams, scope worker listing, fix revokeReadURL (#3933)
- GET /teams and /teams/:uid/members admit apps for teams whose owner opened
  the directory; apps get active members' usernames only
- events.workers.list() from an app returns only its own worker
- hide account-session-only SDK methods from the public types
- unpublish the updateProfile docs page (redirects to getProfile)
- revokeReadURL() works through a new /auth/revoke-own-access-token route
2026-09-23 12:06:56 -07:00
Filip KujundžićandClaude Opus 5.5 bc0f4dc279 feat(ai): add GPT-6 Sol and Luna and sync Claude pricing (#3931)
* feat(ai): sync GPT-6 Sol, GPT-6 Luna, and Claude Opus 5.5

* chore: remove documentation changes from model sync

* fix(ai): bill OpenAI cache writes and long-context pricing

GPT-5.6 and later bill prompt-cache writes at 1.25x input and report them
in `cache_write_tokens`, inside the input total. Both OpenAI calculators now
split them out of the prompt count and meter them under their own key, and
the six GPT-5.6+ models carry the rate.

GPT-6, GPT-5.6, GPT-5.5 and GPT-5.4 (incl. Pro) bill a request with more
than 272K input tokens at 2x input (cached reads and cache writes included)
and 1.5x output for the whole request. Models declare this as
`long_context_pricing`, and the ledger overrides, the reported `usd_cents`
and the credit gate all apply the multipliers.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ai): bill GPT-5.6 Sol at OpenAI's promotional pricing

The catalog carried GPT-5.5's $5/$0.50/$30, but OpenAI bills GPT-5.6 Sol
at $4/$0.40/$20 per million input/cached/output tokens, promotional
through at least 2026-11-21. PUT-1943 tracks re-checking before then.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-23 01:59:10 -07:00
Juan Fernando Castro 17f5454265 Merge pull request #3915 from HeyPuter/juancastro/put-1871-authcheck-app-reports-every-app-as-already-authorized
fix: check-app reported every app as already authorized
2026-09-22 18:24:20 -04:00
Filip Kujundžić 5ddff2b3d6 Merge pull request #3927 from HeyPuter/404oops/put-1870-ai-upstream-credit-exhaustion-is-the-largest-error-class-in
fix(ai): classify upstream credit exhaustion, contain the route, redact vendor text (PUT-1870)
2026-09-22 22:29:58 +02:00
404oops 52f0176854 fix(ai): classify and sanitize upstream credit exhaustion 2026-09-22 21:51:44 +02:00
Daniel Salazar 300fb115a1 feat: opus 5.5 (#3930)
* feat: opus 5.5

* fix: bad typing
2026-09-22 12:31:44 -07:00
dependabot[bot] 6debd7df76 chore(deps-dev): bump prettier from 3.9.6 to 3.9.8 (#3925)
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.6 to 3.9.8.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.6...3.9.8)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-22 11:33:49 -07:00
dependabot[bot] 70404579f2 chore(deps-dev): bump @types/node from 24.13.4 to 24.13.6 (#3926)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 24.13.4 to 24.13.6.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.13.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-22 11:33:40 -07:00
Juan Fernando Castro 7462a8eda9 Merge pull request #3914 from HeyPuter/juancastro/put-1869-unhandled-rejection-on-the-hottest-permission-path-can-take
fix: stop leaking an unawaited permission traversal on every check
2026-09-22 14:28:44 -04:00
Daniel Salazar 718de8a9c6 fix: bump server timeouts for better error handling (#3929) 2026-09-22 10:13:46 -07:00
Juan Fernando Castro 8abdcf4f7f Merge pull request #3928 from HeyPuter/juancastro/put-1873-restrict-godmode-app-launches-to-godmode-callers
fix: restrict godmode app launches to godmode callers
2026-09-22 12:32:32 -04:00
Juan Fernando Castro 476f7e075d Merge pull request #3913 from HeyPuter/juancastro/put-1865-apps-repeatedly-ask-permission-when-it-wants-to-access-files
fix: settle a URL file-access prompt from a grant the app already holds
2026-09-22 12:32:06 -04:00
Juan Castro 1bc8cb964e fix: restrict godmode app launches to godmode callers
`ExecService.launchApp`, reachable from `puter.ui.launchApp`, launched any
named app with no check on the target. Gate it: a godmode target may only
be launched by a godmode caller. Desktop launches (tile, double-click,
URL) call `launch_app` directly and are unaffected; `connectToInstance`
already had its own allowlist.
2026-09-22 09:12:25 -04:00
Daniel Salazar 2690f4934c fix: tests (#3922) 2026-09-21 23:33:23 -07:00
Daniel Salazar d07ddeaaf5 fix(fs): keep a home directory on its account's username (#3920)
A root was free to be renamed or moved, so it could be parked on a
username no account held yet. The next account to take that name
provisioned a second root at the same path, and from then on either row
could answer a lookup of it — new entries inherit their owner from
whichever row resolved as the parent, so one account's files were
created owned by the other, and renaming the parked root dragged the
other account's subtree along with it.

Rename and move now refuse a root, every username claim checks the home
path before the user row is written, and renameUserHome refuses to heal
onto a path another account holds.
2026-09-21 23:29:09 -07:00
Daniel Salazar 1261976088 feat: user profiles in a system-owned store, public only for paid accounts (#3919)
- ProfileService keeps each profile as /system/profiles/<uuid>.profile,
  admin-owned and served by the protected puter-profiles subdomain
- GET/POST /profile; puter.auth.getProfile/updateProfile, with
  getProfilePicture reading through them
- another user's profile is served only while its owner is on a paid
  plan (profileGate.enabled kill switch); the hosted file is gated
  through the new site.access.check hook the hosting middleware asks
  before streaming any file
- SubdomainStore.create takes isProtected
- the GUI reads and writes the profile through the SDK
- docs: getProfile, updateProfile, UserProfile, limits

PUT-1859 PUT-1860 PUT-1861 PUT-1862
2026-09-21 23:28:56 -07:00
Daniel Salazar 2abd5533ac fix(puter-js): restore file pickers on third-party sites (#3917)
The clickjacking guard in #3912 pinned `event.source` to `messageTarget`,
which is only assigned when `env === 'app'`. On a third-party site it is
undefined, so the UI listener dropped every message and
showOpenFilePicker, showSaveFilePicker and showDirectoryPicker never
settled.

Split the two environments. `app` still pins the host frame, whose origin
is whatever the deployment is served from. `web` has no host frame — the
picker popups post back directly — so pin the GUI origin we opened them
on, plus the set of popups we opened. Origin pinning is safe there,
unlike the parent-frame case, because we chose the URL, so self-hosted
and local deployments keep working.

Falls back to the origin alone when `event.source` is null: the picker
calls window.close() right after posting and a discarded browsing
context can drop the source.

PUT-1867
2026-09-21 21:43:58 -07:00
922d203e18 fix(ai): send stable, non-sequential user identifiers to AI providers (#3856)
* fix(ai): send stable, non-sequential user identifiers to AI providers

A precedence bug in the AI providers' identifier expression made every
request send `user: ":undefined"` (the ternary bound the app-uid suffix to
the whole `actor.user.id + actor.app?.uid` sum instead of just the suffix),
or read `actor.user.id` on a missing user. The same expression also shipped
the sequential internal user id, letting AI vendors correlate a single
account across apps and sessions.

All eight OpenAI-, Azure-, xAI-, Meta- and ZAI-style providers now build
the identifier through one shared helper, `aiUserIdentifier()`:

- `puter-<user-uuid>[-<app-token>]`: the random user UUID is always
  preserved in full; `maxLength` constrains only the app-bearing form
- app attribution reads `effectiveApp`, so access-token requests name the
  issuing app instead of looking like direct user traffic
- the app token is truncated to fit the budget, and omitted entirely when
  the remaining budget is below 8 chars, where a truncation could collide
  with another app's uid
- nothing is sent for the system actor
- Meta and ZAI keep a caller-supplied `safety_identifier` / `user_id`
  override, applied before the helper result

`user` is deprecated by OpenAI; the SDK types direct callers to
`safety_identifier` (abuse detection) and `prompt_cache_key` (cache-hit
bucketing). The four OpenAI/Azure chat providers and MetaProvider now send
`prompt_cache_key` as well, defaulting it to the same per-user identifier
unless the caller supplies one; Azure's Grok branch drops both fields,
matching its rejection of unknown args. The cap comment cites only verified
limits: OpenAI's 64 for `safety_identifier` (from the SDK types) and Z.AI's
6-128 for `user_id` (from Z.AI's docs); Meta and xAI document none, so none
is claimed.

The xAI image `#edit` path now carries the identifier like generation, and
takes a named-options param so `user` cannot be transposed with the
adjacent same-typed `aspectRatio`.

Tests share a four-actor matrix (`user` / `user+app` / `access token` /
`system`) with `assertActorMatrixIdentifiers()` across the six
OpenAI-style suites; the helper has exact-string and boundary coverage
(size caps, zero-budget and sub-base cases, no dangling separator, UUID
never truncated, collision guard); the Azure Grok assertions run under a
real user actor so they cannot pass vacuously. 212 provider-suite tests
pass; typecheck and ESLint are clean.

* fix(ai): lock the vendor identifier down and keep vitest out of the test util

Meta and Z.AI no longer let `custom` override the abuse identifier; it is
Puter's attribution, not the caller's. The shared test util exposes pure
field pickers instead of importing vitest into a file the production
tsconfig compiles. The helper's length-cap comment now matches vendor docs
(Meta does cap `safety_identifier` at 64), the redundant budget branch and
the unused export are gone, and the per-user `prompt_cache_key` trade-off is
stated once in the helper instead of five times in providers.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: 404oops <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 16:21:47 -07:00
404oopsandClaude Fable 5.1 027e9a71f3 fix(ai-image): refresh model catalogs, unify txt2img requests, expand docs (#3889)
* fix(ai-image): sync image catalogs, unify request shape, document models

Model sync against every vendor listing and a live generation sweep:
Gemini stable ids replace the retired preview spellings (2.5 Flash Image
delisted ahead of its 2026-10-02 shutdown), OpenAI gpt-image-1/-1-mini/-1.5
are delisted but routable until their shutdown dates with gpt-image-2 as
the default, xAI gains grok-imagine-image-2.0 with its quality tiers,
BytePlus gains the 3K/4K tiers and per-model pixel bounds, Cloudflare
gains SDXL Lightning/Base and SD 1.5 Inpainting, Replicate gains a
schema-driven catalog of 88 additional models with version-pinned
community predictions, and every Together image route is excluded for
the third-party data-sharing requirement.

Request normalization: the driver collapses ratio/width/height/aspect_ratio
into one imageSize with aspect-versus-pixel intent, validates prompt,
quality and resolution once, resolves provider hints (short names or
full driver ids) and aliases with exact ids winning over resellers, and
hands each provider an immutable copy of the caller's args. Providers
share prompt validation, aspect snapping, and content-sniffed data URIs
so responses carry the right MIME type. Replicate predictions are created
once, cancelled on abort or deadline, and bounded in input fan-out.

SDK: txt2img copies caller options, rejects blank prompts with
prompt_required in every call form, and documents that normalize has no
effect because the image return shape is already uniform.

Docs: txt2img options per provider, provider defaults and discovery,
availability notes, a new image model catalog and pricing page, and the
image-generation limits in the quotas page.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(ai-image): sniff SVG outputs correctly, reject hex/exponent dimensions

imageDataUri only decoded the first 24 base64 chars (18 bytes) before
sniffImageMime, which cannot see an <svg> root behind an XML prolog -
SVG outputs from recraft-v*-svg models were being labeled image/png.
Decode enough of the payload to cover the 8 KB SVG sniff window.

dimension() accepted string number literals ('0x10', '1e3') as if they
were decimal dimensions; restrict to plain decimal notation.

* docs(ai): link to the model directory instead of a static catalog

The image model catalog and pricing page duplicated the always up to date
model directory at developer.puter.com/ai/models. Link to the directory
from txt2img-related docs and keep the Together data-sharing exclusion
note self-contained.

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 15:00:02 -07:00
Juan Castro 259cb69fde fix: check-app reported every app as already authorized
The token grant writes `flag:app-is-authenticated` on the user->app row,
but the check asked for `service:<app_uid>:ii:flag:app-is-authenticated`.
Those never match, and the parent walk reduces the question to a bare
`service` — a root every user holds by default — so the check answered
true for any app_uid and handed back an app-under-user token for an app
the user had never opened.

Read the row the grant writes, through one shared constant so the two
cannot drift again, and mint against the resolved uid.

A cancelled account picker no longer leaves `popup_signin_consent` set:
with no relationship there is no token to flip it.
2026-09-21 16:13:53 -04:00
404oopsandClaude Fable 5.1 081941a6f3 fix: null-prototype model maps in chat/video drivers; txt2vid prompt and option guards (#3887)
Chat and video drivers keyed their provider and model maps on plain objects,
so `model` or `provider` values such as `__proto__` reached Object.prototype
and surfaced as 500s. Both maps are now null-prototype objects and reject
those names as ordinary unknown models.

txt2vid now rejects blank or non-string prompts with `prompt_required` before
any request, tolerates `null` in either argument slot, and copies the caller's
options before resolving the `duration` alias and output path so frozen
option objects work and caller objects are never mutated.

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 13:12:35 -07:00
404oops e2a5ab8cdc docs: document local Docker builds with Compose overrides (#3867) 2026-09-21 13:12:16 -07:00
Juan Castro 07e889f135 fix: stop leaking an unawaited permission traversal on every check
`validateUserPerms` started the linked SQL traversal beside the flat read
and returned on a flat hit without awaiting it. The hit is the common
case, so nearly every user-permission check left a floating promise: with
`keep_alive_on_uncaught` unset no `unhandledRejection` listener is
registered, so a DB wobble inside the traversal exits the process.

Run it only when the flat read misses. That also stops a SQL walk the
cache had already answered.
2026-09-21 16:07:26 -04:00
Juan Castro ab7a5f325a fix: refuse an empty app_uid and survive a failed grant check
A present-but-empty `app_uid` fell through to checking the user, where
every `fs:` scope on their own file answers `true` — a prompt settled by
a question nobody answered. It is a 400 now.

The launch-path check also sat outside any catch, so a rejecting read
would have taken the whole launch down instead of falling through to the
prompt. Renamed the seam it is injected through to match the function it
defaults to, since wiring the token-based sibling in its place would send
an app uid as a bearer token and silently always prompt.
2026-09-21 15:56:39 -04:00
Juan Castro b4b464554b fix: settle a URL file-access prompt from a grant the app already holds
Opening `/app/<name>?file=<path>` asked for consent on every launch: the
gate prompted unconditionally and never read the `fs:<uid>:write` grant
its own Allow had written.

`/auth/check-permissions` takes an optional `app_uid` so the account can
ask what one of its apps holds, and the launch gate skips the dialog when
the answer is yes. Sessions only — an app or a scoped token asking would
be a window onto its neighbours' grants.
2026-09-21 15:00:02 -04:00
Nariman Jelveh 4de1d1fb39 Redesign the Teams tab in the dashboard (#3901)
* Redesign the Teams tab in the dashboard

Hero card with a lettered tile, role pill, handle and headcount; the
directory setting becomes a settings row with the shared toggle switch;
the add-account form gets labels, Enter to submit, and a copy button on
the one-time password. The accounts and record tables drop the grid for
hairlines, avatar tiles and status pills, and stack into labelled rows
on phones so actions stay reachable. Loading, empty and error states
get a skeleton, a create call to action, and a retry.

Also fixes audit dates showing 1970 (the audit routes send unix
seconds), the actions cell breaking the row border by being a flex
table cell, and the "0 suspended" clause on a healthy team.

* Use the dashboard's overlay dialog in the Teams tab, not UIAlert

Adds UIDashboardDialog, a confirm / alert / prompt card built like the
share and properties modals: mounted in the dashboard window, revealed
after a frame, dismissed by Escape, the close button or a backdrop
press-and-release, with a focus trap and focus restored on close. On
phones it docks to the bottom as a sheet. The Teams tab's confirms,
error alerts, seat-limit warning and rename/create prompts all go
through it; UIAlert and UIPrompt are no longer imported there.
2026-09-21 09:30:03 -07:00
Daniel Salazar 4e9b03bf5c fix: harden clickjacking (#3912) 2026-09-21 09:19:37 -07:00
Juan Fernando Castro a63172e9ae Merge pull request #3892 from HeyPuter/juancastro/put-1806-invite-email-addresses-are-disclosed-to-apps-manage
fix: stop disclosing invite addresses to apps, tokens and delegates

approvals already in place
2026-09-21 12:05:08 -04:00
Reynaldi Chernando 29d805d793 show app icon when login from external app (#3905)
* show app icon when login from external app

* fix
2026-09-21 22:33:38 +07:00
Juan Fernando Castro 059fae3d15 Merge pull request #3872 from HeyPuter/juancastro/put-1813-team-share-blocklist-and-unshare-paging
fix: make the block list bite on team shares, and unshare every re-share (PUT-1813)

Approvals in place to move with the merge
2026-09-21 11:31:24 -04:00
Juan Castro 18ada4764a chore: drop a stray lockfile marker
A local `npm install` (needed to pick up main's new `postal-mime`) wrote
`"extraneous": true` onto a bundled `@clack/prompts` dependency. It was the
branch's only lockfile change and it does not belong to this PR.
2026-09-21 11:26:27 -04:00
Juan Castro 72203152d9 fix: bound a scoped token to what it issued, which is nothing
CI caught three HTTP-level tests the earlier merge left behind, and they
were right to fail: dropping this branch's `manage` gate in favour of
main's row filter lost a case main's filter does not cover.

Main bounds an app to the rows it issued. A *scoped* access token is not
an app, so it was falling through unbounded — `/fs/stat` with
`return_shares` handed a `fs:<uid>:list` token the owner's whole share
list. Addresses stayed withheld, but who else can reach a file is no
more a narrow token's business than the addresses are.

So the filter is generalised rather than the gate restored: a scoped
token is bounded to nothing, since it issues nothing under its own name.
Filtering it by a null app would have been worse than not filtering —
that matches the owner's own rows. Apps and sessions behave exactly as
they do on main, and a full-access token still holds the account's reach.

The three tests now assert the answer instead of a refusal, including
the one whose name had always promised a refusal its body never checked.
2026-09-21 11:12:37 -04:00
Juan Castro cec8382d4d Merge branch 'main' into juancastro/put-1806-invite-email-addresses-are-disclosed-to-apps-manage
Two textual conflicts, both additive on each side: `isAccountContext`
(here) and `isPlainUserActor` (main) are both imported and both used,
and the `stat()` note keeps both sentences — this branch's on who may
read an invite address, main's on the share-read limit it spends.

The rest is adapting to main, which grew its own answer to half of what
this branch was for. `listSharesOf` there bounds an app to the rows it
issued itself; this branch instead required the credential to hold
`manage` and refused it otherwise. Main's is the better mechanism — it
answers the app rather than turning it away, and it hides other
issuers' rows outright rather than redacting a field on them — so the
`manage` gate goes, and with it the two tests that asserted the
refusal. They are replaced by tests that hold main's line: an app sees
none of the invites it did not send, with or without `manage`.

What this branch still carries is the gap main does not close. Its row
filter only applies to apps, so a plain manage delegate still reads the
owner's invite addresses; `#maySeeInviteAddress` is what withholds
those, and its delegate tests pass unchanged. The `stat()` note is
corrected to describe main's behaviour rather than the removed gate.
2026-09-21 10:46:07 -04:00
Juan Castro 3f335939b3 fix: team shares are not subject to a recipient's per-sender block
Review call from the ticket's author: a team share is the team's, not
one colleague's to withhold from another, so a personal block should not
hide it. This drops the enforcement added earlier on the branch — the
listing, its total and the fs-event fan-out no longer filter team rows
by the recipient's block list, and the SQL fragment that did it goes
with them.

What a block still does for a team share is suppress the notification,
which was already true before this branch: it stops the interruption
without pretending to stop the access. Leaving the team is what ends
that. Said so in the block API docs, the settings copy and the code,
since the mismatch between the two was the original complaint.

The unshare sweep is untouched — that half of the ticket stands.
2026-09-21 10:28:50 -04:00
Juan Castro 4e821c128c Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging
# Conflicts:
#	src/backend/services/share/ShareService.ts
#	src/backend/stores/share/ShareStore.js
2026-09-21 10:28:33 -04:00
Juan Fernando Castro c313a381b6 Merge pull request #3896 from HeyPuter/juancastro/put-1843-plan-card-unknown-not-free
fix: a team seat's plan card names its team, and an unreadable plan is not Free (PUT-1843)
2026-09-21 09:58:41 -04:00
Juan Fernando Castro 76e0d2ec93 Merge pull request #3871 from HeyPuter/juancastro/put-1798-sharing-email-notification-changes
feat: name the issuing app in share emails, and cut the digest window to 5s
2026-09-21 09:58:09 -04:00
Juan Castro dfd7925872 fix: say which team a seat's plan belongs to, paid or free
A provisioned account read as a plain "Free" plan with an "Upgrade for
more features" badge: nothing said the account belongs to a team, and
the prompt asked for an upgrade only its owner can buy. Its free tier is
not the free plan either — org_seat_free is half the allowance.

The card now names the team on both paths: a paid seat keeps its tier
and status and gains "Managed by <team>", and a seat with no tier reads
"Team account" instead of an upgrade pitch.
2026-09-17 14:15:01 -04:00
Juan Castro e50d210423 fix: an unreadable plan is not a free plan (PUT-1843)
The Home tab read /marketplace/subscriptions/current and folded every
failure into the free state: a non-OK response left `subscription` null,
and the catch said so explicitly. So a refusal rendered as a confident
"Free" with an Upgrade badge.

That is reachable: verification gates are default-on for authenticated
routes, and a team seat that still owes its password change is refused
there — it then reads Free while its team pays for a tier. A rate limit
or a blip does the same to anyone.

The plan card moves into its own method so a failed read can return
without touching it, leaving the card as it was rather than naming a
plan the account does not have. A 200 carrying no subscription still
reads Free, which is the one case that actually means it.
2026-09-17 13:52:03 -04:00
Juan Castro 461cb7d0a9 fix: close two gaps the review found in the invite-address fix
- A full-access token was denied the address while `shared-by-me` still
  handed it the same rows, so the clause bought no privacy and cost an
  API client the address `unshare()` takes. `isAccountContext` is the
  boundary the rest of the codebase already uses for 'acting as the
  account': plain session or full-access token, never a scoped one.
- The Dashboard share modal dropped a withheld invite entirely, since
  its aggregate keys a pending row on the address — so a delegate saw no
  sign of an outstanding invite and accessCount under-reported who could
  reach the item. It is now kept, keyed on the share uid, labelled, and
  without the controls that would need a recipient to address.
2026-09-17 13:21:13 -04:00
Juan Castro 3e3d02bafe fix: stop disclosing invite addresses to apps, tokens and delegates
getShares (and stat's return_shares, which runs the same listing) gated
on #assertCanManage's default 'see' mode, so any credential that could
see the node got every unclaimed invite's raw email — including an app
handed one file by the picker, a list-scoped token on the stat surface,
and a manage delegate reading the owner's invitees.

Two bounds, matching the invariant clientShare.ts already claimed:

- An invite's address goes only to the item's owner and to whoever sent
  it, and never to an app or token. A delegate can revoke only what they
  issued, so withholding costs them nothing they could act on.
- An app or token must hold manage reach of its own to read the listing
  at all; it answers for the ancestors too, which is not what being
  handed one file grants. tryListSharesOf turns that into an empty
  shares array, so stat itself keeps working.

The share dialog names an unattributable invite rather than rendering a
blank row with a dead revoke button.

Closes PUT-1806.
2026-09-16 18:59:10 -04:00
Juan Castro 728966bfb1 Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 18:05:58 -04:00
Juan Castro 3f7ee34706 Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 18:05:39 -04:00
Juan Castro 23c33b2e06 Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging
# Conflicts:
#	src/backend/stores/share/ShareStore.js
2026-09-16 15:27:09 -04:00
Juan Castro 0593844acf Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 13:05:03 -04:00
Juan Castro 67d048148c Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 13:04:43 -04:00
Juan Castro 5d20e054ac Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-16 10:33:03 -04:00
Juan Castro 3bd3955e0d Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-16 10:32:05 -04:00
Juan Castro 27e858d760 fix: gate the group-share index delete on authority, qualify the live filter
Review follow-ups on #3872: deleteActiveGroup ran even when every
canManagePermission check was denied, deleting the index row while the
team's grant survived — unlisted live access, the class this PR exists
to remove. And memberIdsAmong's join left deleted_at unqualified, which
only works while jct_user_group has no such column; #live now takes an
alias.
2026-09-15 18:57:50 -04:00
Juan Castro 0bd6458441 test: pin the mixed-digest button link carrying no app
Review follow-up on #3871: the flush-level rule — shared_app on the
button only when every entry came through one app — had no direct
coverage. An app share and a plain share folded into one digest now
prove the line and item link keep their attribution while the button
stays clean.
2026-09-15 18:56:38 -04:00
Juan Castro af6547724d Merge branch 'main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging 2026-09-15 15:23:22 -04:00
Juan Castro e9922aea26 Merge branch 'main' into juancastro/put-1798-sharing-email-notification-changes 2026-09-15 15:23:11 -04:00
Juan Castro aadcda073d fix: keep blocks out of the authority graph, and sweep what unshare missed
Code review on the previous commit confirmed three ways the block filter
inside readUserGroupPerms turned a reversible block into permanent loss:
canManagePermission reads the same rows, so a block-suspended grant
looked revoked to #revokeDownstream's cascade, to unshare's authority
gate, and to invite claiming — deleting re-shares and invites that were
supposed to come back on unblock. The scan is now deliberately blind to
blocks; a block suspends delivery only (listing, count, fan-out,
telling), which the share store filters itself through one shared
notBlockedSql fragment owned by UserBlockStore.

The team-unshare sweep also now covers what it claimed to: a member's
re-share *to another team* is revoked with them (group rows swept in
 #revokeDownstream, user path included), and the sweep is skipped
entirely while another issuer's grant still backs the team — members'
re-shares rest on that authority and revoking them would orphan live
access. The blanket block-all note in settings now says team shares
still arrive, matching what the code deliberately does.
2026-09-15 14:49:54 -04:00
Juan Castro 9da03555ae fix: make the block list bite on team shares, and unshare every re-share (PUT-1813)
A sender a member blocked could still land items in that member's
shared-with-me — and grant them real access — by sharing with a common
team. The group grant is one row and cannot exclude a member, so the
block is now enforced where delivery is derived per member: the
permission scan, the inbound listing and its count, and the fs-event
fan-out all skip team rows whose issuer the member blocked. Nothing is
revoked, so unblocking restores everything. Blocking now also bumps the
blocker's permission cache so it bites immediately. Direct shares keep
their contract: existing ones stand.

#unshareTeam swept re-shares by listing members with a 200 cap and
never following the cursor, so members past the cap kept their orphaned
rows. It now walks the share rows in the subtree — the set that can
actually need sweeping — and filters those issuers to members, which
has no cap by construction.
2026-09-15 14:49:54 -04:00
Juan Castro 577693bf84 feat: name the issuing app in share emails, and cut the digest window to 5s
When an app shares on a user's behalf, the mail now says so — the
digest line reads 'alice shared report.txt via Mail App', preferring
the app's title, then its name, then its index_url host. Both the
per-item links and the 'Open Puter' button carry a new shared_app
query param (the app's name, or its uid) so the GUI can match the
share back to the app; the button only carries it when the whole
digest came from one app. Plain shares are byte-for-byte unchanged.

Also drops SHARE_EMAIL_BATCH_SECONDS from 30 to 5: the window only
has to fold one gesture's worth of calls, and every second there is
a second the common single-share email arrives late.

Closes PUT-1798.
2026-09-15 12:15:30 -04:00
715 changed files with 77981 additions and 12364 deletions

No files matched your search

+5
View File
@@ -16,3 +16,8 @@ MARIADB_PASSWORD=replace-with-strong-password
S3_ACCESS_KEY=puter
S3_SECRET_KEY=replace-with-strong-secret
S3_BUCKET=puter-local
# ── Inbound SMTP (compose profile `smtp`) ------------------------------
# Host port mapped to the receiver's 2525. 25 is what an MX record means;
# many hosting providers block it.
SMTP_PORT=25
+56 -45
View File
@@ -1,61 +1,72 @@
# Default owner, including backend and SDK areas without a specialist below.
# The last matching rule takes precedence.
* @Salazareo
- @Salazareo
# GUI
/src/gui/ @jelveh @Salazareo
# Documentation, developer site, and CLI
/doc/ @reynaldichernando
/src/docs/ @reynaldichernando
/src/dev-center/ @reynaldichernando
/src/cli/ @reynaldichernando
/doc/ @reynaldichernando @Salazareo
/src/docs/ @reynaldichernando @Salazareo
/src/dev-center/ @reynaldichernando @Salazareo
/src/cli/ @reynaldichernando @Salazareo
# Backend AI
/src/backend/controllers/puterai/ @404oops @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/ai-\*/ @404oops @ProgrammerIn-wonderland @Salazareo
# Backend: AI, hosting, workers, networking, and WebDAV
/src/backend/controllers/puterai/ @ProgrammerIn-wonderland
/src/backend/controllers/hosting/ @ProgrammerIn-wonderland
/src/backend/controllers/webdav/ @ProgrammerIn-wonderland
/src/backend/controllers/wisp/ @ProgrammerIn-wonderland
/src/backend/drivers/ai-*/ @ProgrammerIn-wonderland
/src/backend/drivers/util/ @ProgrammerIn-wonderland
/src/backend/drivers/workers/ @ProgrammerIn-wonderland
/src/backend/services/localworker/ @ProgrammerIn-wonderland
/src/backend/stores/subdomain/ @ProgrammerIn-wonderland
/extensions/workerSandbox* @ProgrammerIn-wonderland
/src/worker/ @ProgrammerIn-wonderland
/src/worker-types/ @ProgrammerIn-wonderland
/src/backend/controllers/peer/ @ProgrammerIn-wonderland
/src/backend/controllers/hosting/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/webdav/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/wisp/ @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/util/ @ProgrammerIn-wonderland @Salazareo
/src/backend/drivers/workers/ @ProgrammerIn-wonderland @Salazareo
/src/backend/services/localworker/ @ProgrammerIn-wonderland @Salazareo
/src/backend/stores/subdomain/ @ProgrammerIn-wonderland @Salazareo
/extensions/workerSandbox\* @ProgrammerIn-wonderland @Salazareo
/src/worker/ @ProgrammerIn-wonderland @Salazareo
/src/worker-types/ @ProgrammerIn-wonderland @Salazareo
/src/backend/controllers/peer/ @ProgrammerIn-wonderland @Salazareo
# Backend: sharing, permissions, and teams
/src/backend/controllers/share/ @jfcastro92
/src/backend/controllers/team/ @jfcastro92
/src/backend/services/share/ @jfcastro92
/src/backend/services/permission/ @jfcastro92
/src/backend/services/acl/ @jfcastro92
/src/backend/services/team/ @jfcastro92
/src/backend/services/apps/AppPermissionService* @jfcastro92
/src/backend/stores/share/ @jfcastro92
/src/backend/stores/permission/ @jfcastro92
/src/backend/stores/group/ @jfcastro92
/src/backend/stores/team/ @jfcastro92
/src/backend/controllers/share/ @jfcastro92 @Salazareo
/src/backend/controllers/team/ @jfcastro92 @Salazareo
/src/backend/services/share/ @jfcastro92 @Salazareo
/src/backend/services/permission/ @jfcastro92 @Salazareo
/src/backend/services/acl/ @jfcastro92 @Salazareo
/src/backend/services/team/ @jfcastro92 @Salazareo
/src/backend/services/apps/AppPermissionService\* @jfcastro92 @Salazareo
/src/backend/stores/share/ @jfcastro92 @Salazareo
/src/backend/stores/permission/ @jfcastro92 @Salazareo
/src/backend/stores/group/ @jfcastro92 @Salazareo
/src/backend/stores/team/ @jfcastro92 @Salazareo
# SDK: AI, hosting, workers, and networking
/src/puter-js/src/modules/ai/ @ProgrammerIn-wonderland
/src/puter-js/src/modules/hosting/ @ProgrammerIn-wonderland
/src/puter-js/src/modules/Workers* @ProgrammerIn-wonderland
/src/puter-js/src/modules/networking/ @ProgrammerIn-wonderland
/src/puter-js/src/lib/networkUtils* @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/ai.suite.ts @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/hosting.suite.ts @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/workers.suite.ts @ProgrammerIn-wonderland
/src/puter-js/tests/api/suites/net.suite.ts @ProgrammerIn-wonderland
/src/puter-js/src/modules/Peer* @ProgrammerIn-wonderland
/src/puter-js/src/modules/ai/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/hosting/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Workers* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/networking/ @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/lib/networkUtils* @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/ai.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/hosting.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/workers.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/tests/api/suites/net.suite.ts @ProgrammerIn-wonderland @Salazareo
/src/puter-js/src/modules/Peer\* @ProgrammerIn-wonderland @Salazareo
# SDK: sharing and permissions
/src/puter-js/src/modules/perms/ @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/share* @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/unshare* @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/getShares* @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/listShared* @jfcastro92
/src/puter-js/tests/api/suites/perms.suite.ts @jfcastro92
/src/puter-js/tests/api/suites/sharing.suite.ts @jfcastro92
/src/puter-js/src/modules/FileSystem/operations/share* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/unshare* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/getShares* @jfcastro92 @Salazareo
/src/puter-js/src/modules/FileSystem/operations/listShared* @jfcastro92 @Salazareo
/src/puter-js/tests/api/suites/perms.suite.ts @jfcastro92 @Salazareo
/src/puter-js/tests/api/suites/sharing.suite.ts @jfcastro92 @Salazareo
+3
View File
@@ -72,3 +72,6 @@ servers.json
config.*.json
volatile/
# Local build override
docker-compose.override.yml
+32
View File
@@ -47,6 +47,8 @@ Before opening a PR, scan the diff for:
When in doubt, return less. Auth-, permission-, or data-export-related changes deserve an explicit callout in the PR description.
Commit messages and PR descriptions never credit anyone or mention a bug report. For security fixes, state what the change does mechanically ("scoped tokens do xyz now"), not the vulnerability or how it could be abused.
### Working rules of thumb
- **Run it, don't just compile it.** "It type-checks" is not "it works." Exercise the code path end-to-end at least once.
@@ -58,6 +60,36 @@ When in doubt, return less. Auth-, permission-, or data-export-related changes d
---
## Local development configuration
To test a local Docker build, create `docker-compose.override.yml` in the repository
root (or merge into the existing file):
```yaml
services:
puter:
pull_policy: never
build:
context: .
```
Compose merges it automatically. Run `docker compose up -d --build` to build and
start the stack, and repeat after source changes. Keep local build settings in
this gitignored override instead of editing `docker-compose.yml`, so they stay
out of pull requests and do not conflict when pulling updates. See
[Self-Hosting Puter](doc/self-hosting.md#building-from-source-instead-of-pulling)
for the required runtime configuration.
For local API testing, agents may decide whether to set `"unlimitedMetering": true`
when budget checks block unrelated tests; no additional approval is needed.
Merge it into the ignored `config.json` for `npm start` or
`puter/config/config.json` for Docker, then restart Puter. It applies to all
accounts on that local instance, including guests, and usage is still recorded.
Keep normal settings when testing budget or subscription enforcement, and leave
shared defaults unchanged.
---
## Backend
A layered stack with explicit dependency injection: each layer depends only on the layers beneath it, receives them through its constructor, and `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires the whole thing together. [doc/architecture.md](doc/architecture.md) is the full reference.
+2
View File
@@ -79,6 +79,8 @@ ENV PUTER_CONFIG_PATH=/etc/puter/config.json
ENV NODE_OPTIONS=--enable-source-maps
EXPOSE 4100
# The inbound SMTP receiver, when run from this image with the `smtp` profile.
EXPOSE 2525
USER node
+4
View File
@@ -10,6 +10,8 @@
·
<a href="https://apps.puter.com/">App Store</a>
·
<a href="https://builder.puter.com/">AI Builder</a>
·
<a href="https://developer.puter.com" target="_blank">Developers</a>
·
<a href="https://twitter.com/HeyPuter">X</a>
@@ -46,6 +48,8 @@ npm start
```
**→** This should launch Puter at http://puter.localhost:4100
To run this checkout with Docker, follow [Building from source](./doc/self-hosting.md#building-from-source-instead-of-pulling). Create a local `docker-compose.override.yml` to select the local build; keeping these settings out of `docker-compose.yml` avoids conflicts when pulling updates and keeps local configuration out of pull requests.
<br/>
+42
View File
@@ -32,6 +32,10 @@
"port": 4100,
// Externally-visible port (set this when behind a reverse proxy on 80/443).
"pub_port": 4100,
// Idle keep-alive timeout in ms. Must stay above the idle timeout of any
// proxy in front, or the proxy reuses connections this server has closed
// and its clients see 502s.
"keep_alive_timeout": 620000,
"protocol": "http",
"domain": "puter.localhost",
// Fully-qualified externally-visible URL. Computed from protocol/domain/
@@ -251,6 +255,38 @@
}
},
// ── Inbound mail (separate process) ─────────────────────────────────
// Mailboxes addressed at one of your own domains. `secret` gates the
// ingress endpoint that files a message into the recipient's ~/.mail.
//
// `localServer` additionally runs an SMTP listener that accepts mail from
// other mail servers and hands each message to that endpoint. It runs as
// its own process - `npm run start:smtp`, or the `smtp` compose profile -
// and stores nothing itself. Without it, nothing delivers to the endpoint
// unless you point something else at it.
//
// The listener is unauthenticated by nature: anything on the internet may
// hand it a message for an address in `localDomains`. It performs no spam,
// SPF, DKIM or DMARC checks. Read the self-hosting guide before exposing
// it.
// "userEmail": {
// "secret": "a long random string",
// "localServer": true,
// "localDomains": ["example.com"],
// // Container-internal by default; publish 25 onto it from outside.
// "localPort": 2525,
// // Defaults to <api_base_url>/email/ingress. The endpoint is served
// // on the `api` subdomain, so when this points somewhere internal,
// // `localIngressHost` has to name that virtual host.
// "localIngressUrl": "http://puter:4100/email/ingress",
// "localIngressHost": "api.example.com",
// "localHostname": "mx.example.com",
// "localMaxRecipients": 50,
// // Concurrent connections. Each message is held in memory while it
// // is delivered, so this also bounds memory use.
// "localMaxClients": 20
// },
// ── Sharing ─────────────────────────────────────────────────────────
// Email a recipient who already has an account about a new share. On
// unless set to false; they opt out via the unsubscribe link, or by
@@ -538,4 +574,10 @@
// what this deployment's normal traffic costs, so it has to be measured
// rather than guessed, and a stale number here alarms on healthy growth.
// "maxGlobalUsagePerMinute": 200000000
// How many full months past its own a monthly usage record is kept before
// it expires. Defaults to 3; set to 0 to keep every month's records
// forever. Changing it, including to 0, applies to records first written
// after the change; records already stamped keep their expiry.
// "meteringRetentionMonths": 3
}
+147 -1
View File
@@ -35,6 +35,12 @@ Optional services (compose profile `ai`, opt-in):
| `puter-ollama` | `ollama/ollama` | Local LLM provider (CPU; GPU passthrough opt-in) |
| `puter-ollama-init` | `ollama/ollama` | One-shot — pulls the default model (`tinyllama`) on first boot |
Optional services (compose profile `smtp`, opt-in):
| Container | Image | Role |
| ------------ | ------------------------ | -------------------------------------------------------- |
| `puter-smtp` | `ghcr.io/heyputer/puter` | Receives mail over SMTP and files it into user mailboxes |
State lives under `./puter/data/<service>/`.
---
@@ -144,6 +150,16 @@ Why these knobs:
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
- `s3.s3Config.forcePathStyle: true` — RustFS / MinIO / fauxqs need path-style URLs (`<endpoint>/<bucket>`). Real AWS S3 wants virtual-hosted (`<bucket>.<endpoint>`) — drop this flag (or set `false`) when you swap to real S3.
- `s3.s3Config.publicEndpoint` — `endpoint` (`http://s3:9000`) only resolves inside the docker network; presigned upload/download URLs handed to the browser need a host-reachable URL. Caddy routes the `s3.<domain>` subdomain to RustFS internally and preserves the Host header end-to-end (required for S3 signature validation), so the browser hits the same port/protocol as the rest of the app — no separate published port, no mixed-content surprises when you turn on TLS. Switch to `https://s3.<your-domain>` once you enable TLS in Step 3. Real AWS S3 doesn't need this — its endpoint is already public; drop the field entirely.
- **Bucket CORS** — the browser uploads file bytes straight to RustFS with a cross-origin `PUT` to the presigned URL (Dev Center deploy, `puter.fs.upload`). That only works if the bucket answers the `OPTIONS` preflight with `Access-Control-Allow-*` headers. The `s3-init` container applies those rules on every boot (`put-bucket-cors`, origins `*` — the signed URL is the credential, and the SDK sends no cookies). If deploys fail with `No 'Access-Control-Allow-Origin' header is present`, re-run it: `docker compose run --rm s3-init`. Verify with:
```bash
curl -sk -X OPTIONS "https://s3.<domain>/<bucket>/test" \
-H "Origin: https://<domain>" \
-H "Access-Control-Request-Method: PUT" \
-H "Access-Control-Request-Headers: content-type" -D - -o /dev/null | grep -i access-control
```
You should see `access-control-allow-origin`, `allow-methods: GET, HEAD, PUT, POST, DELETE`, `allow-headers: *`, and `max-age: 3600`.
- `trust_proxy: 1` — Caddy terminates TLS and forwards `X-Forwarded-For`. Without this, `req.ip` is the docker-network address of the Caddy container instead of the real client IP, which breaks rate limiting and IP-based audit logs. `1` = one trusted hop (Caddy). Bump to `2` if you put Cloudflare in front of Caddy; never set `true` (it trusts every hop and makes XFF forgeable).
> If you ever change `MARIADB_PASSWORD` after first boot, `.env` alone won't update MariaDB — its credentials are baked into `./puter/data/mariadb/` on first init. Either rotate the password inside MariaDB by hand or `docker compose down && rm -rf ./puter/data/mariadb` to start fresh.
@@ -408,6 +424,12 @@ Every account then resolves to an unlimited policy. Usage is still recorded, so
the dashboard still shows what is being consumed; nothing is ever refused for
lack of budget.
For API testing, you can also set `"unlimitedMetering": true` in your
ignored runtime config: `puter/config/config.json` for Docker, or the repository's
`config.json` for `npm start`. Merge it into the existing config and restart Puter.
This includes guest accounts and applies to the whole local instance. Keep normal
metering settings when testing budget or subscription enforcement.
To keep the budgets but stop them blocking anything — recording only:
```json
@@ -418,6 +440,17 @@ Calls driven by a deployed worker are exempt from enforcement by default,
because a worker has no prompt to show and nobody watching it fail. Set
`"meteringEnforcement": { "workers": true }` to include them.
Monthly usage records expire on their own once they're old enough to be of no
further use — `meteringRetentionMonths` full months past the month they
belong to, 3 by default. Set it to `0` to keep every month's records forever:
```json
"meteringRetentionMonths": 6
```
Changing it, including to `0`, applies to records first written after the
change; records already stamped keep their expiry.
### Captcha on signup / login
Built-in proof-of-work captcha — no external service needed.
@@ -486,14 +519,127 @@ Without `--profile ai`, the `ollama` containers stay down and Puter (with `enabl
For GPU acceleration (NVIDIA), uncomment the `deploy:` block under the `ollama` service in [docker-compose.yml](../docker-compose.yml). Requires `nvidia-container-toolkit` on the host.
## Optional: receiving mail over SMTP
Puter can give each account a mailbox at a domain you control. Two pieces are
involved, and they are configured together under `userEmail`:
- an **ingress endpoint** (`POST /email/ingress`) that files a message into the
recipient's `~/.mail`. It exists whenever `userEmail.secret` is set.
- the **`puter-smtp` receiver**, which accepts mail from other mail servers and
hands each message to that endpoint. It runs as its own process, behind the
`smtp` compose profile.
> **Read this before you expose it.** An inbound mail server is unauthenticated
> by nature: anyone on the internet may hand it a message for any address in
> `localDomains`. Puter performs **no** spam filtering and **no** SPF, DKIM or
> DMARC checks — whatever is delivered is stored in the recipient's mailbox and
> counts against their storage. If that is not acceptable, put a filtering relay
> in front of it and point `localDomains` at that instead.
1. Configure it in `puter/config/config.json`:
```json
"userEmail": {
"secret": "a long random string",
"localServer": true,
"localDomains": ["example.com"]
}
```
`secret` is shared between the two halves; generate it with
`openssl rand -hex 32`. `localDomains` lists the domains this server accepts
mail for — anything else is refused at `RCPT TO`, which is what stops it
being used as an open relay. It has no default, and the receiver refuses to
start without it.
The receiver posts to `<api_base_url>/email/ingress` unless you set
`localIngressUrl`. The endpoint is served on the `api` subdomain, so if you
point it at an internal address to skip the reverse proxy, name that virtual
host too or the request will 404:
```json
"localIngressUrl": "http://puter:4100/email/ingress",
"localIngressHost": "api.example.com"
```
`localIngressHost` defaults to the host of `api_base_url`, which is usually
what you want.
2. Bring it up with the `smtp` profile:
```bash
docker compose --profile smtp up -d
docker compose logs -f puter-smtp
```
3. Point DNS at it:
```dns
mx.example.com. A 203.0.113.10
example.com. MX 10 mx.example.com.
```
Also set a `PTR` record for the address that resolves back to
`mx.example.com` — many senders refuse mail from a server without one.
Inbound TCP port 25 must reach the host. **Most cloud providers block port
25 by default and residential ISPs block it permanently**; if you cannot get
it unblocked, this will not work no matter how it is configured. The
container itself binds 2525 (it runs unprivileged) and compose publishes
`${SMTP_PORT:-25}` onto it, so you can move the host port if you need to.
A message is held in memory while it is being delivered, so peak memory is
bounded by `localMaxClients` (default 20) times the 25 MiB message ceiling.
Lower `localMaxClients` on a small host. There is no queue and no retry
schedule — if delivery fails, the sending server is told to try again later and
retrying is its job.
Mail is stored as `message/rfc822` objects under `~/.mail/objects/<date>/` and
read back through `puter.email` in the SDK.
Every setting is documented in
[config.template.jsonc](../config.template.jsonc). Two deliberate limits, both
in the name of keeping the self-hosted path simple: it does not offer STARTTLS,
so mail arrives in the clear, and it applies no per-sender rate limiting. Put a
filtering relay in front if either matters to you.
Note this is a *receiving* server only: it never sends, and it must not be used
for authenticated submission. Outbound transactional mail is the separate
`email` block above.
## Building from source instead of pulling
If you want to test local Dockerfile changes against the full stack, uncomment the `build:` block in [docker-compose.yml](../docker-compose.yml) under the `puter` service, change `pull_policy: always` → `pull_policy: never`, then:
To run a local build against the full stack, use a source checkout and complete
the configuration steps above. Create `docker-compose.override.yml` in the
repository root, next to [docker-compose.yml](../docker-compose.yml):
```yaml
services:
puter:
pull_policy: never
build:
context: .
```
If that file already exists, merge these settings into its `puter` service.
Compose loads and merges the override automatically. `build.context: .` selects
this checkout, and `pull_policy: never` prevents pulling the published Puter image.
Use this override for local build settings instead of editing `docker-compose.yml`.
The override is ignored by Git, keeping local configuration out of pull requests
and avoiding conflicts in the shared Compose file when pulling updates.
From the repository root, build and start the stack:
```bash
docker compose up -d --build
```
Run the same command after changing or pulling source code to rebuild the image.
To return to the published image, remove the local build settings (or the override
file if those are its only settings), then run `docker compose up -d`.
---
## Managing running backend
+52 -11
View File
@@ -139,8 +139,10 @@ services:
start_period: 5s
s3-init:
# One-shot container that creates the `puter-local` bucket on first
# boot. Exits 0 once the bucket exists; stays exited 0 thereafter.
# One-shot container that creates the `puter-local` bucket and
# applies its CORS rules on every boot (put-bucket-cors is
# idempotent). Without CORS, browser uploads to presigned URLs
# fail the preflight check.
image: amazon/aws-cli:latest
container_name: puter-s3-init
depends_on:
@@ -163,6 +165,26 @@ services:
echo "creating bucket $$bucket"
aws --endpoint-url "$$endpoint" s3 mb "s3://$$bucket"
fi
# Browser uploads (Dev Center deploy, puter.fs.upload) PUT to
# presigned URLs cross-origin, so the bucket must answer the
# OPTIONS preflight. Origins are open because the presigned URL
# itself is the credential and the SDK sends no cookies.
cat > /tmp/cors.json <<'EOF'
{
"CORSRules": [
{
"AllowedOrigins": ["*"],
"AllowedMethods": ["GET", "HEAD", "PUT", "POST", "DELETE"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag", "x-amz-request-id"],
"MaxAgeSeconds": 3600
}
]
}
EOF
aws --endpoint-url "$$endpoint" s3api put-bucket-cors \
--bucket "$$bucket" --cors-configuration file:///tmp/cors.json
echo "bucket CORS configured"
restart: "no"
# ── Optional: local LLM ───────────────────────────────────────────
@@ -230,15 +252,8 @@ services:
puter:
image: ghcr.io/heyputer/puter:main
pull_policy: always
# Uncomment to build from this directory instead of pulling the published
# image. Also flip pull_policy to `never` so compose doesn't overwrite
# your local build by re-pulling :latest.
# build:
# context: .
# # buildx-only: cross-compile to both archs in a single push
# # platforms:
# # - linux/amd64
# # - linux/arm64
# For local builds, create a docker-compose.override.yml;
# see doc/self-hosting.md#building-from-source-instead-of-pulling.
container_name: puter
restart: unless-stopped
depends_on:
@@ -271,6 +286,32 @@ services:
retries: 3
start_period: 30s
# Inbound mail. Behind the `smtp` profile, so it only starts when asked for:
# docker compose --profile smtp up -d
# Accepts SMTP from other mail servers and hands each message to the ingress
# endpoint, which files it into the recipient's mailbox. Needs `userEmail`
# configured — see doc/self-hosting.md#optional-receiving-mail-over-smtp.
puter-smtp:
profiles: ["smtp"]
image: ghcr.io/heyputer/puter:main
pull_policy: always
container_name: puter-smtp
restart: unless-stopped
command: ["node", "-r", "./dist/src/backend/telemetry.js", "./dist/src/backend/smtp/index.js"]
depends_on:
puter:
condition: service_healthy
ports:
# The container runs unprivileged and binds 2525; 25 is what an MX record
# means, and many providers block it.
- "${SMTP_PORT:-25}:2525"
environment:
PUID: 1000
PGID: 1000
volumes:
# The same config.json the backend reads; only `userEmail` applies here.
- ./puter/config:/etc/puter:z
caddy:
image: caddy:2.11-alpine
container_name: puter-caddy
+165
View File
@@ -8,6 +8,7 @@ import {
it,
vi,
} from 'vitest';
import { makeActor, type Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
@@ -88,6 +89,80 @@ describe('metering extension — handleMeteringUsage', () => {
(captured.body as Record<string, unknown>).allowanceInfo,
).toBeDefined();
});
describe('usage split across apps', () => {
const seedUsage = async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const metering = server.services.metering;
await metering.incrementUsage(
makeActor({ user: owner, app: { uid: 'app-mine' } }),
'kv:read',
1,
10,
);
await metering.incrementUsage(
makeActor({ user: owner, app: { uid: 'app-other' } }),
'kv:write',
1,
20,
);
await metering.incrementUsage(
makeActor({ user: owner }),
'kv:list',
1,
40,
);
await server.stores.meteringBuffer.flushCycle();
return owner;
};
const usageAs = async (actor: Actor) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleMeteringUsage(makeReq(), res),
);
return captured.body as {
usage: Record<string, unknown> & { total: number };
appTotals: Record<string, unknown>;
allowanceInfo: Record<string, unknown>;
};
};
it('reports only the calling app’s own usage to an app actor', async () => {
const owner = await seedUsage();
const body = await usageAs(
makeActor({ user: owner, app: { uid: 'app-mine' } }),
);
expect(body.usage.total).toBe(10);
expect(Object.keys(body.usage).sort()).toEqual([
'kv:read',
'total',
]);
expect(body.appTotals).toEqual({
'app-mine': { total: 10, count: 1 },
});
expect(body.allowanceInfo).toMatchObject({
remaining: expect.any(Number),
monthUsageAllowance: expect.any(Number),
});
});
it('still reports the whole account to the user', async () => {
const owner = await seedUsage();
const body = await usageAs(makeActor({ user: owner }));
expect(body.usage.total).toBe(70);
expect(body.usage).toHaveProperty('kv:read');
expect(body.usage).toHaveProperty('kv:write');
expect(body.usage).toHaveProperty('kv:list');
expect(body.appTotals).toMatchObject({
'app-mine': { total: 10, count: 1 },
'app-other': { total: 20, count: 1 },
});
});
});
});
describe('metering extension — handleMeteringUsageForApp', () => {
@@ -132,6 +207,96 @@ describe('metering extension — handleMeteringUsageForApp', () => {
),
).rejects.toMatchObject({ statusCode: 404 });
});
describe('scoped to the calling app', () => {
const appUsageAs = async (actor: Actor, appIdOrName: string) => {
const { res, captured } = makeRes();
await runWithContext({ actor }, () =>
handleMeteringUsageForApp(
makeReq({ params: { appIdOrName } }),
res,
),
);
return captured.body as { total: number };
};
const seedApps = async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const mine = `app-${uuidv4()}`;
const other = `app-${uuidv4()}`;
await server.services.metering.incrementUsage(
makeActor({ user: owner, app: { uid: mine } }),
'kv:read',
1,
10,
);
await server.services.metering.incrementUsage(
makeActor({ user: owner, app: { uid: other } }),
'kv:write',
1,
20,
);
await server.stores.meteringBuffer.flushCycle();
return {
owner,
mine,
other,
asMine: makeActor({ user: owner, app: { uid: mine } }),
};
};
it('lets an app read its own usage', async () => {
const { mine, asMine } = await seedApps();
expect((await appUsageAs(asMine, mine)).total).toBe(10);
});
it('refuses an app another app’s usage', async () => {
const { other, asMine } = await seedApps();
await expect(appUsageAs(asMine, other)).rejects.toMatchObject({
statusCode: 403,
});
});
it('refuses an app another app’s usage by name', async () => {
const { owner, other, asMine } = await seedApps();
const name = `other-${uuidv4()}`;
await server.clients.db.write(
'INSERT INTO `apps` (`uid`, `name`, `title`, `index_url`, `owner_user_id`) VALUES (?, ?, ?, ?, ?)',
[other, name, name, `https://${name}.example/`, owner.id],
);
await expect(appUsageAs(asMine, name)).rejects.toMatchObject({
statusCode: 403,
});
});
it('refuses an app the usage outside any app', async () => {
const { asMine } = await seedApps();
await expect(appUsageAs(asMine, 'os-global')).rejects.toMatchObject(
{ statusCode: 403 },
);
});
it('still lets the user read any of their apps', async () => {
const { owner, mine, other } = await seedApps();
const asUser = makeActor({ user: owner });
expect((await appUsageAs(asUser, mine)).total).toBe(10);
expect((await appUsageAs(asUser, other)).total).toBe(20);
});
it('lets the user read their usage outside any app', async () => {
const { owner } = await seedApps();
const asUser = makeActor({ user: owner });
await server.services.metering.incrementUsage(
asUser,
'kv:read',
1,
5,
);
await server.stores.meteringBuffer.flushCycle();
expect((await appUsageAs(asUser, 'os-global')).total).toBe(5);
});
});
});
describe('metering extension — handleMeteringGlobalUsage', () => {
+4 -2
View File
@@ -6,6 +6,7 @@ import {
servicesContainers,
} from '@heyputer/backend/src/exports';
import { extension } from '@heyputer/backend/src/extensions';
import { GLOBAL_APP_KEY } from '@heyputer/backend/src/services/metering/consts';
import {
creditMultiplierFrom,
toCredits,
@@ -151,8 +152,9 @@ export const handleMeteringUsageForApp = async (
let appId = String(req.params.appIdOrName ?? '');
if (!appId) throw new HttpError(400, 'appId parameter is required');
// If not a UUID-shaped app UID, look up by name
if (!appId.startsWith('app-')) {
// If not a UUID-shaped app UID or the global sentinel, look up by name.
// Which apps an actor may read is MeteringService's call, not this route's.
if (!appId.startsWith('app-') && appId !== GLOBAL_APP_KEY) {
const appRows = (await clients.db.read(
'SELECT `uid` FROM `apps` WHERE `name` = ? LIMIT 1',
[appId],
+496 -173
View File
@@ -1,8 +1,10 @@
import {
GetObjectCommand,
HeadObjectCommand,
PutObjectCommand,
type S3Client,
} from '@aws-sdk/client-s3';
import type { Request, Response } from 'express';
import crypto from 'node:crypto';
import {
afterAll,
@@ -12,8 +14,17 @@ import {
it,
vi,
} from 'vitest';
import type { FSController } from '../src/backend/controllers/fs/FSController.ts';
import type {
ClientSignedWriteResponse,
CompleteWriteRequest,
SignedWriteRequest,
} from '../src/backend/controllers/fs/requestTypes.ts';
import type { Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import { generateDefaultFsentries } from '../src/backend/util/userProvisioning.ts';
import {
handleFsCopyNodeThumbnail,
handleFsRemoveNodeThumbnail,
@@ -27,9 +38,33 @@ const TINY_PNG_BASE64 =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
const BUCKET = 'puter-local';
const BUCKET_ENDPOINT = 'http://127.0.0.1:4566/puter-local/';
// Keys the extension will accept back: its own `thumbnails/<uuid>` namespace.
const mintedKey = () => `thumbnails/${crypto.randomUUID()}`;
// Keys the extension mints: `thumbnails/<entry uuid>/<uuid>`.
const mintedKey = (entryUuid: string = crypto.randomUUID()) =>
`thumbnails/${entryUuid}/${crypto.randomUUID()}`;
// Minted before keys were bound to an entry.
const legacyKey = () => `thumbnails/${crypto.randomUUID()}`;
const putObject = async (s3: S3Client, key: string, body: Buffer) => {
await s3.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: key,
Body: body,
ContentType: 'image/png',
}),
);
};
const objectExists = async (s3: S3Client, key: string) => {
try {
await s3.send(new HeadObjectCommand({ Bucket: BUCKET, Key: key }));
return true;
} catch {
return false;
}
};
const streamToBuffer = async (
body: { transformToByteArray: () => Promise<Uint8Array> } | undefined,
@@ -40,28 +75,42 @@ const streamToBuffer = async (
describe('thumbnails extension — handleThumbnailCreated', () => {
let server: PuterServer;
let s3: S3Client;
beforeAll(async () => {
server = await setupTestServer();
s3 = server.clients.s3.get();
});
afterAll(async () => {
await server?.shutdown();
});
it('uploads a valid data: URL thumbnail to S3 and rewrites event.url to an s3:// pointer', async () => {
const s3 = server.clients.s3.get();
const event: Record<string, unknown> = {
url: `data:image/png;base64,${TINY_PNG_BASE64}`,
};
const created = async (url: unknown, uuid?: unknown) => {
const event: Record<string, unknown> = { url, uuid };
await handleThumbnailCreated(event, {
s3,
bucketName: BUCKET,
bucketEndpoint: BUCKET_ENDPOINT,
});
return event.url;
};
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
it('uploads a valid data: URL thumbnail under a key bound to the entry', async () => {
const entryUuid = crypto.randomUUID();
const newUrl = await created(
`data:image/png;base64,${TINY_PNG_BASE64}`,
entryUuid,
);
expect(typeof event.url).toBe('string');
const newUrl = event.url as string;
expect(newUrl.startsWith(`s3://${BUCKET}/`)).toBe(true);
expect(typeof newUrl).toBe('string');
expect(
(newUrl as string).startsWith(
`s3://${BUCKET}/thumbnails/${entryUuid}/`,
),
).toBe(true);
const key = newUrl.slice(`s3://${BUCKET}/`.length);
const key = (newUrl as string).slice(`s3://${BUCKET}/`.length);
const obj = await s3.send(
new GetObjectCommand({ Bucket: BUCKET, Key: key }),
);
@@ -73,34 +122,77 @@ describe('thumbnails extension — handleThumbnailCreated', () => {
});
it('sets event.url to null when the data: URL does not decode to a valid image', async () => {
const s3 = server.clients.s3.get();
const event: Record<string, unknown> = {
url: `data:image/png;base64,${Buffer.from('not an image').toString('base64')}`,
};
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
expect(event.url).toBeNull();
const url = await created(
`data:image/png;base64,${Buffer.from('not an image').toString('base64')}`,
crypto.randomUUID(),
);
expect(url).toBeNull();
});
it('leaves event.url untouched when the URL is not a data: URL', async () => {
const s3 = server.clients.s3.get();
it('sets event.url to null for a data: URL with no entry to bind it to', async () => {
const url = await created(`data:image/png;base64,${TINY_PNG_BASE64}`);
expect(url).toBeNull();
});
// A documented SDK input: `thumbnail` / `thumbnailGenerator` may return
// a plain URL. It never names our storage, so it is stored as given.
it('leaves an external image URL untouched', async () => {
const original = 'https://example.com/thumb.png';
const event: Record<string, unknown> = { url: original };
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
expect(event.url).toBe(original);
expect(await created(original, crypto.randomUUID())).toBe(original);
});
it('returns without writing to S3 when event.url is missing', async () => {
const s3 = server.clients.s3.get();
const event: Record<string, unknown> = {};
await handleThumbnailCreated(event, { s3, bucketName: BUCKET });
await handleThumbnailCreated(event, {
s3,
bucketName: BUCKET,
bucketEndpoint: BUCKET_ENDPOINT,
});
expect(event.url).toBeUndefined();
});
it('keeps an uploaded pointer minted for the entry, normalised to our bucket', async () => {
const entryUuid = crypto.randomUUID();
const key = mintedKey(entryUuid);
await putObject(s3, key, Buffer.from(TINY_PNG_BASE64, 'base64'));
expect(
await created(`s3://client-named-bucket/${key}`, entryUuid),
).toBe(`s3://${BUCKET}/${key}`);
});
// A thumbnail key is visible to anyone who can list the entry holding it.
// Planting it on a file of one's own used to hand its deletion to them.
it.each([
['a key minted for another entry', () => mintedKey()],
['a key that predates entry binding', legacyKey],
])('drops a pointer naming %s', async (_label, makeKey) => {
const key = makeKey();
await putObject(s3, key, Buffer.from(TINY_PNG_BASE64, 'base64'));
expect(
await created(`s3://${BUCKET}/${key}`, crypto.randomUUID()),
).toBeNull();
expect(await objectExists(s3, key)).toBe(true);
});
it('drops a pointer whose object was never uploaded', async () => {
const entryUuid = crypto.randomUUID();
expect(
await created(`s3://${BUCKET}/${mintedKey(entryUuid)}`, entryUuid),
).toBeNull();
});
it('drops and removes an uploaded object over the size bound', async () => {
const entryUuid = crypto.randomUUID();
const key = mintedKey(entryUuid);
await putObject(s3, key, Buffer.alloc(2 * 1024 * 1024 + 1));
expect(await created(`s3://${BUCKET}/${key}`, entryUuid)).toBeNull();
expect(await objectExists(s3, key)).toBe(false);
});
});
describe('thumbnails extension — handleThumbnailUploadPrepare', () => {
@@ -148,10 +240,15 @@ describe('thumbnails extension — handleThumbnailUploadPrepare', () => {
expect(item.thumbnailUrl).toBeUndefined();
});
it('skips items whose size exceeds the max thumbnail bytes', async () => {
it.each([
['exceeds the max thumbnail bytes', { size: 999_999_999 }],
['is not declared', {}],
['is not a whole number', { size: 10.5 }],
])('skips items whose size %s', async (_label, sizeField) => {
const item: Record<string, unknown> = {
contentType: 'image/png',
size: 999_999_999,
item_uid: crypto.randomUUID(),
...sizeField,
};
await handleThumbnailUploadPrepare(
{ items: [item] },
@@ -161,10 +258,26 @@ describe('thumbnails extension — handleThumbnailUploadPrepare', () => {
expect(item.thumbnailUrl).toBeUndefined();
});
it('mints a presigned uploadUrl and an s3:// thumbnailUrl for valid items', async () => {
it('skips items with no entry to bind the key to', async () => {
const item: Record<string, unknown> = {
contentType: 'image/png',
size: 1024,
item_uid: 'not-a-uuid',
};
await handleThumbnailUploadPrepare(
{ items: [item] },
{ s3Presign, bucketName: BUCKET },
);
expect(item.uploadUrl).toBeUndefined();
expect(item.thumbnailUrl).toBeUndefined();
});
it('mints a size-bound upload URL for a key bound to the entry', async () => {
const entryUuid = crypto.randomUUID();
const item: Record<string, unknown> = {
contentType: 'image/png',
size: 1024,
item_uid: entryUuid,
};
await handleThumbnailUploadPrepare(
{ items: [item] },
@@ -174,8 +287,15 @@ describe('thumbnails extension — handleThumbnailUploadPrepare', () => {
expect((item.uploadUrl as string).startsWith('http')).toBe(true);
expect(typeof item.thumbnailUrl).toBe('string');
expect(
(item.thumbnailUrl as string).startsWith(`s3://${BUCKET}/`),
(item.thumbnailUrl as string).startsWith(
`s3://${BUCKET}/thumbnails/${entryUuid}/`,
),
).toBe(true);
// A signed `content-length` makes the store refuse any other body size.
const signedHeaders = new URL(
item.uploadUrl as string,
).searchParams.get('X-Amz-SignedHeaders');
expect(signedHeaders).toContain('content-length');
});
});
@@ -196,34 +316,41 @@ describe('thumbnails extension — handleThumbnailRead', () => {
await server?.shutdown();
});
it('rewrites an s3:// thumbnail into a presigned https URL', async () => {
// Seed an object so the presigned URL points at something real
// (the signer itself doesn't validate existence, but this keeps
// the test honest).
const key = mintedKey();
await s3.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: key,
Body: Buffer.from(TINY_PNG_BASE64, 'base64'),
ContentType: 'image/png',
}),
);
// Thumbnails stored before keys were bound to entries keep displaying.
it.each([
['an entry-bound key', () => mintedKey()],
['a key that predates entry binding', legacyKey],
])(
'rewrites an s3:// thumbnail naming %s into a presigned https URL',
async (_label, makeKey) => {
// Seed an object so the presigned URL points at something real
// (the signer itself doesn't validate existence, but this keeps
// the test honest).
const key = makeKey();
await s3.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: key,
Body: Buffer.from(TINY_PNG_BASE64, 'base64'),
ContentType: 'image/png',
}),
);
const entry: Record<string, unknown> = {
thumbnail: `s3://${BUCKET}/${key}`,
};
await handleThumbnailRead(entry, {
s3,
s3Presign,
bucketName: BUCKET,
bucketEndpoint: 'http://127.0.0.1:4566/puter-local/',
db: stubDb,
});
const entry: Record<string, unknown> = {
thumbnail: `s3://${BUCKET}/${key}`,
};
await handleThumbnailRead(entry, {
s3,
s3Presign,
bucketName: BUCKET,
bucketEndpoint: 'http://127.0.0.1:4566/puter-local/',
db: stubDb,
});
expect(typeof entry.thumbnail).toBe('string');
expect((entry.thumbnail as string).startsWith('http')).toBe(true);
});
expect(typeof entry.thumbnail).toBe('string');
expect((entry.thumbnail as string).startsWith('http')).toBe(true);
},
);
// `fsentries.thumbnail` is writable through the FS API, so a stored
// pointer is attacker input. Signing one the extension didn't mint would
@@ -245,6 +372,10 @@ describe('thumbnails extension — handleThumbnailRead', () => {
`s3://${BUCKET}/thumbnails/../${crypto.randomUUID()}`,
],
['a non-uuid inside the namespace', `s3://${BUCKET}/thumbnails/etc`],
[
'a key nested deeper than the namespace allows',
`s3://${BUCKET}/${mintedKey()}/${crypto.randomUUID()}`,
],
])('refuses to presign a pointer naming %s', async (_label, thumbnail) => {
const entry: Record<string, unknown> = { thumbnail };
await handleThumbnailRead(entry, {
@@ -301,8 +432,9 @@ describe('thumbnails extension — handleThumbnailRead', () => {
});
it('migrates an inline data: URL by uploading to S3 and updating the DB row', async () => {
const entryUuid = crypto.randomUUID();
const entry: Record<string, unknown> = {
uuid: 'fs-entry-uuid',
uuid: entryUuid,
thumbnail: `data:image/png;base64,${TINY_PNG_BASE64}`,
};
@@ -322,9 +454,27 @@ describe('thumbnails extension — handleThumbnailRead', () => {
await Promise.resolve();
expect(stubDb.write).toHaveBeenCalledWith(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[expect.stringMatching(/^s3:\/\//), 'fs-entry-uuid'],
[
expect.stringMatching(
new RegExp(`^s3://${BUCKET}/thumbnails/${entryUuid}/`),
),
entryUuid,
],
);
});
it('leaves an inline data: URL as is when there is no entry to bind it to', async () => {
const thumbnail = `data:image/png;base64,${TINY_PNG_BASE64}`;
const entry: Record<string, unknown> = { thumbnail };
await handleThumbnailRead(entry, {
s3,
s3Presign,
bucketName: BUCKET,
bucketEndpoint: BUCKET_ENDPOINT,
db: stubDb,
});
expect(entry.thumbnail).toBe(thumbnail);
});
});
describe('thumbnails extension — handleFsRemoveNodeThumbnail', () => {
@@ -340,66 +490,63 @@ describe('thumbnails extension — handleFsRemoveNodeThumbnail', () => {
await server?.shutdown();
});
it('deletes the S3 object referenced by an s3:// thumbnail URL', async () => {
const key = mintedKey();
await s3.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: key,
Body: Buffer.from(TINY_PNG_BASE64, 'base64'),
ContentType: 'image/png',
}),
const remove = (target: { thumbnail?: string | null; uuid?: string }) =>
handleFsRemoveNodeThumbnail(
{ target },
{ s3, bucketName: BUCKET, bucketEndpoint: BUCKET_ENDPOINT },
);
await handleFsRemoveNodeThumbnail(
{ target: { thumbnail: `s3://${BUCKET}/${key}` } },
{ s3, bucketName: BUCKET },
);
it('deletes the object minted for the removed entry', async () => {
const entryUuid = crypto.randomUUID();
const key = mintedKey(entryUuid);
await putObject(s3, key, Buffer.from(TINY_PNG_BASE64, 'base64'));
// GetObject should now error because the key was deleted.
await expect(
s3.send(new GetObjectCommand({ Bucket: BUCKET, Key: key })),
).rejects.toThrow();
await remove({ thumbnail: `s3://${BUCKET}/${key}`, uuid: entryUuid });
expect(await objectExists(s3, key)).toBe(false);
});
// The destructive half of the same confused deputy: the stored pointer
// decides which object is deleted, so a key the extension didn't mint
// must never reach DeleteObject.
it('does not delete an object the pointer names but we did not mint', async () => {
const victimKey = crypto.randomUUID(); // shaped like an fs object key
await s3.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: victimKey,
Body: Buffer.from(TINY_PNG_BASE64, 'base64'),
ContentType: 'image/png',
}),
);
// decides which object is deleted, so only a key minted for the removed
// entry may reach DeleteObject.
it.each([
// Shaped like an fs object key.
[
'an object outside the thumbnails namespace',
() => crypto.randomUUID(),
],
// Visible to anyone who can list the entry it belongs to.
["another entry's thumbnail", () => mintedKey()],
// Can't be told apart from a planted or shared pointer.
['a thumbnail that predates entry binding', legacyKey],
])('does not delete %s', async (_label, makeKey) => {
const victimKey = makeKey();
await putObject(s3, victimKey, Buffer.from(TINY_PNG_BASE64, 'base64'));
await handleFsRemoveNodeThumbnail(
{ target: { thumbnail: `s3://${BUCKET}/${victimKey}` } },
{ s3, bucketName: BUCKET },
);
await remove({
thumbnail: `s3://${BUCKET}/${victimKey}`,
uuid: crypto.randomUUID(),
});
const survivor = await s3.send(
new GetObjectCommand({ Bucket: BUCKET, Key: victimKey }),
);
expect(survivor.ContentType).toBe('image/png');
expect(await objectExists(s3, victimKey)).toBe(true);
});
it('does not delete through an external URL that names our key', async () => {
const entryUuid = crypto.randomUUID();
const key = mintedKey(entryUuid);
await putObject(s3, key, Buffer.from(TINY_PNG_BASE64, 'base64'));
await remove({
thumbnail: `https://cdn.example.com/${BUCKET}/${key}`,
uuid: entryUuid,
});
expect(await objectExists(s3, key)).toBe(true);
});
it('is a no-op when the target has no thumbnail', async () => {
// Should not throw or attempt a delete.
await handleFsRemoveNodeThumbnail(
{ target: {} },
{ s3, bucketName: BUCKET },
);
});
it('is a no-op when the thumbnail URL is not an s3:// pointer', async () => {
await handleFsRemoveNodeThumbnail(
{ target: { thumbnail: 'https://cdn.example.com/x.png' } },
{ s3, bucketName: BUCKET },
);
await remove({});
});
});
@@ -416,64 +563,58 @@ describe('thumbnails extension — handleFsCopyNodeThumbnail', () => {
await server?.shutdown();
});
it('duplicates the shared thumbnail object and repoints the copied row', async () => {
const sourceKey = mintedKey();
const body = Buffer.from(TINY_PNG_BASE64, 'base64');
await s3.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: sourceKey,
Body: body,
ContentType: 'image/png',
}),
);
const copyUuid = crypto.randomUUID();
const copyNode = async (thumbnail: string | null, copyUuid: string) => {
const db = { write: vi.fn().mockResolvedValue(undefined) };
await handleFsCopyNodeThumbnail(
{
copy: {
thumbnail: `s3://${BUCKET}/${sourceKey}`,
uuid: copyUuid,
},
},
{ s3, bucketName: BUCKET, db },
{ copy: { thumbnail, uuid: copyUuid } },
{ s3, bucketName: BUCKET, bucketEndpoint: BUCKET_ENDPOINT, db },
);
return db;
};
// The copied row was repointed at a fresh object...
expect(db.write).toHaveBeenCalledTimes(1);
const [, params] = db.write.mock.calls[0] as [string, [string, string]];
const [newPointer, updatedUuid] = params;
expect(updatedUuid).toBe(copyUuid);
expect(newPointer.startsWith(`s3://${BUCKET}/thumbnails/`)).toBe(true);
expect(newPointer).not.toBe(`s3://${BUCKET}/${sourceKey}`);
it.each([
['an entry-bound key', () => mintedKey()],
['a key that predates entry binding', legacyKey],
])(
'duplicates a thumbnail naming %s under a key bound to the copy',
async (_label, makeKey) => {
const sourceKey = makeKey();
const body = Buffer.from(TINY_PNG_BASE64, 'base64');
await putObject(s3, sourceKey, body);
// ...whose content matches, while the source object survives — so
// deleting either entry can no longer break the other's thumbnail.
const newKey = newPointer.slice(`s3://${BUCKET}/`.length);
const duplicated = await s3.send(
new GetObjectCommand({ Bucket: BUCKET, Key: newKey }),
);
expect(
(await streamToBuffer(duplicated.Body as never)).equals(body),
).toBe(true);
const original = await s3.send(
new GetObjectCommand({ Bucket: BUCKET, Key: sourceKey }),
);
expect(original.ContentType).toBe('image/png');
});
const copyUuid = crypto.randomUUID();
const db = await copyNode(`s3://${BUCKET}/${sourceKey}`, copyUuid);
// The copied row was repointed at a fresh object of its own...
expect(db.write).toHaveBeenCalledTimes(1);
const [, params] = db.write.mock.calls[0] as [
string,
[string, string],
];
const [newPointer, updatedUuid] = params;
expect(updatedUuid).toBe(copyUuid);
expect(
newPointer.startsWith(`s3://${BUCKET}/thumbnails/${copyUuid}/`),
).toBe(true);
// ...whose content matches, while the source object survives — so
// deleting either entry can no longer break the other's thumbnail.
const newKey = newPointer.slice(`s3://${BUCKET}/`.length);
const duplicated = await s3.send(
new GetObjectCommand({ Bucket: BUCKET, Key: newKey }),
);
expect(
(await streamToBuffer(duplicated.Body as never)).equals(body),
).toBe(true);
expect(await objectExists(s3, sourceKey)).toBe(true);
},
);
it('drops the pointer when the shared object is already gone', async () => {
const copyUuid = crypto.randomUUID();
const db = { write: vi.fn().mockResolvedValue(undefined) };
await handleFsCopyNodeThumbnail(
{
copy: {
thumbnail: `s3://${BUCKET}/${mintedKey()}`, // never uploaded
uuid: copyUuid,
},
},
{ s3, bucketName: BUCKET, db },
const db = await copyNode(
`s3://${BUCKET}/${mintedKey()}`, // never uploaded
copyUuid,
);
expect(db.write).toHaveBeenCalledTimes(1);
@@ -482,27 +623,209 @@ describe('thumbnails extension — handleFsCopyNodeThumbnail', () => {
expect(params).toEqual([copyUuid]);
});
// Only reachable for objects uploaded before upload URLs were size-bound;
// copying one would multiply bytes no allowance counts.
it('drops the pointer instead of duplicating an object over the size bound', async () => {
const sourceKey = legacyKey();
await putObject(s3, sourceKey, Buffer.alloc(2 * 1024 * 1024 + 1));
const copyUuid = crypto.randomUUID();
const db = await copyNode(`s3://${BUCKET}/${sourceKey}`, copyUuid);
expect(db.write).toHaveBeenCalledTimes(1);
const [sql, params] = db.write.mock.calls[0] as [string, [string]];
expect(sql).toContain('NULL');
expect(params).toEqual([copyUuid]);
});
it('does not duplicate an object the pointer names but we did not mint', async () => {
const foreignKey = crypto.randomUUID(); // shaped like an fs object key
const db = { write: vi.fn().mockResolvedValue(undefined) };
await handleFsCopyNodeThumbnail(
{
copy: {
thumbnail: `s3://${BUCKET}/${foreignKey}`,
uuid: crypto.randomUUID(),
},
},
{ s3, bucketName: BUCKET, db },
const db = await copyNode(
`s3://${BUCKET}/${foreignKey}`,
crypto.randomUUID(),
);
expect(db.write).not.toHaveBeenCalled();
});
it('leaves an external image URL to ride along as is', async () => {
const sourceKey = legacyKey();
await putObject(s3, sourceKey, Buffer.from(TINY_PNG_BASE64, 'base64'));
const db = await copyNode(
`https://cdn.example.com/${BUCKET}/${sourceKey}`,
crypto.randomUUID(),
);
expect(db.write).not.toHaveBeenCalled();
});
it('is a no-op when the copy has no thumbnail', async () => {
const db = { write: vi.fn().mockResolvedValue(undefined) };
await handleFsCopyNodeThumbnail(
{ copy: { thumbnail: null, uuid: crypto.randomUUID() } },
{ s3, bucketName: BUCKET, db },
);
const db = await copyNode(null, crypto.randomUUID());
expect(db.write).not.toHaveBeenCalled();
});
});
// The flow `puter.fs.upload()` runs — start, PUT to the signed URLs, complete
// with the returned pointer — through the real controller and the listeners
// this extension registers.
describe('thumbnails extension — signed batch upload through /fs', () => {
let server: PuterServer;
let controller: FSController;
const png = Buffer.from(TINY_PNG_BASE64, 'base64');
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.fs as unknown as FSController;
});
afterAll(async () => {
await server?.shutdown();
});
const makeActor = async (): Promise<{ actor: Actor; username: string }> => {
const username = `thumb-${Math.random().toString(36).slice(2, 10)}`;
const user = await server.stores.user.create({
username,
uuid: crypto.randomUUID(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
user,
);
return {
username,
actor: {
user: {
id: user.id,
uuid: user.uuid,
username,
email: user.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
};
};
const call = async <B>(
actor: Actor,
handler: (req: Request, res: Response) => Promise<void>,
body: B,
): Promise<unknown> => {
let captured: unknown;
const res = {
json: (value: unknown) => {
captured = value;
return res;
},
status: () => res,
setHeader: () => res,
};
const req = {
body,
query: {},
headers: { 'content-type': 'application/json' },
actor,
user: { id: actor.user!.id, username: actor.user!.username },
};
await runWithContext({ actor }, () =>
handler.call(
controller,
req as unknown as Request,
res as unknown as Response,
),
);
return captured;
};
const upload = async (
actor: Actor,
path: string,
thumbnailData: (
started: ClientSignedWriteResponse,
) => string | undefined,
) => {
const [started] = (await call<SignedWriteRequest[]>(
actor,
controller.startBatchWrites,
[
{
fileMetadata: { path, size: 4 },
thumbnailMetadata: {
contentType: 'image/png',
size: png.length,
},
},
],
)) as ClientSignedWriteResponse[];
await fetch(started!.url!, { method: 'PUT', body: 'abcd' });
if (started!.thumbnailUploadUrl) {
await fetch(started!.thumbnailUploadUrl, {
method: 'PUT',
body: png,
headers: { 'content-type': 'image/png' },
});
}
const thumbnail = thumbnailData(started!);
await call<CompleteWriteRequest[]>(
actor,
controller.completeBatchWrites,
[
{
uploadId: started!.sessionId,
...(thumbnail === undefined
? {}
: { thumbnailData: thumbnail }),
},
],
);
return {
started: started!,
entry: (await server.stores.fsEntry.getEntryByPath(path))!,
};
};
it('stores the pointer minted for the uploaded entry', async () => {
const { actor, username } = await makeActor();
const { started, entry } = await upload(
actor,
`/${username}/Documents/photo.png`,
(s) => s.thumbnailUrl,
);
expect(started.thumbnailUrl).toMatch(
new RegExp(`^s3://${BUCKET}/thumbnails/${entry.uuid}/`),
);
expect(entry.thumbnail).toBe(started.thumbnailUrl);
});
it("drops a pointer minted for another user's entry", async () => {
const victim = await makeActor();
const { entry: victimEntry } = await upload(
victim.actor,
`/${victim.username}/Documents/photo.png`,
(s) => s.thumbnailUrl,
);
const attacker = await makeActor();
const { entry } = await upload(
attacker.actor,
`/${attacker.username}/Documents/decoy.png`,
() => victimEntry.thumbnail!,
);
expect(entry.thumbnail).toBeNull();
});
it('keeps an external image URL as given', async () => {
const { actor, username } = await makeActor();
const { entry } = await upload(
actor,
`/${username}/Documents/linked.png`,
() => 'https://example.com/thumb.png',
);
expect(entry.thumbnail).toBe('https://example.com/thumb.png');
});
});
+144 -51
View File
@@ -2,11 +2,13 @@ import {
CopyObjectCommand,
DeleteObjectCommand,
GetObjectCommand,
HeadObjectCommand,
PutObjectCommand,
S3Client,
} from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { extension } from '@heyputer/backend/src/extensions';
import { isMissingObjectError } from '@heyputer/backend/src/stores/fs/S3ObjectStore';
import crypto from 'node:crypto';
import sharp from 'sharp';
const clients = extension.import('client');
@@ -22,8 +24,13 @@ const THUMBNAIL_KEY_PREFIX = 'thumbnails/';
const UUID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
const mintThumbnailKey = (): string =>
`${THUMBNAIL_KEY_PREFIX}${crypto.randomUUID()}`;
const isUuid = (value: unknown): value is string =>
typeof value === 'string' && UUID_PATTERN.test(value);
// `thumbnails/<entry uuid>/<random uuid>`: the entry segment binds the object
// to the entry it was minted for, which is what writes and deletes check.
const mintThumbnailKey = (entryUuid: string): string =>
`${THUMBNAIL_KEY_PREFIX}${entryUuid}/${crypto.randomUUID()}`;
/**
* Extract the object key from a stored thumbnail pointer, or null when the
@@ -31,12 +38,13 @@ const mintThumbnailKey = (): string =>
*
* `fsentries.thumbnail` is writable through the FS API, so neither half of the
* stored string is trusted: the bucket is discarded (callers always pass their
* own) and the key must sit under {@link THUMBNAIL_KEY_PREFIX} with a random
* uuid. Honouring an arbitrary key would lend this extension's storage
* credentials to whatever object the caller named — in the shared-bucket layout
* that is every user's file, since an fs object's key is its fsentry uuid.
* Legacy bare-uuid thumbnails fail the check and are treated as absent; they
* are indistinguishable from a planted pointer, so there is nothing safer to do
* own) and the key must sit under {@link THUMBNAIL_KEY_PREFIX} as either
* `<uuid>` (minted before keys were bound to entries) or `<entry uuid>/<uuid>`.
* Honouring an arbitrary key would lend this extension's storage credentials to
* whatever object the caller named — in the shared-bucket layout that is every
* user's file, since an fs object's key is its fsentry uuid. Bare-uuid keys
* from before the prefix fail the check and are treated as absent; they are
* indistinguishable from a planted pointer, so there is nothing safer to do
* with them than stop signing them.
*/
const resolveThumbnailKey = (pointer: string): string | null => {
@@ -58,10 +66,26 @@ const resolveThumbnailKey = (pointer: string): string | null => {
key = segments.join('/');
}
if (!key.startsWith(THUMBNAIL_KEY_PREFIX)) return null;
const id = key.slice(THUMBNAIL_KEY_PREFIX.length);
return UUID_PATTERN.test(id) ? key : null;
const segments = key.slice(THUMBNAIL_KEY_PREFIX.length).split('/');
return segments.length <= 2 && segments.every(isUuid) ? key : null;
};
// The entry a resolved key was minted for. Null for an unbound key, whose
// holder can't be told apart from a row that copied or planted the pointer.
const thumbnailKeyOwner = (key: string): string | null => {
const segments = key.slice(THUMBNAIL_KEY_PREFIX.length).split('/');
return segments.length === 2 ? segments[0] : null;
};
// Whether a stored thumbnail names an object in our storage. Anything else is
// an external image URL the client supplied: shown as given, never copied or
// deleted.
const isStoragePointer = (pointer: string, bucketEndpoint: string): boolean =>
pointer.startsWith('s3://') ||
// Legacy format — remove after full migration
(pointer.startsWith('https') &&
pointer.includes(new URL(bucketEndpoint).hostname));
// S3 client + bucket config — lazily resolved after boot from config.
let s3Client: S3Client | null = null;
let s3PresignClient: S3Client | null = null;
@@ -144,16 +168,73 @@ async function decodeAndValidateThumbnail(
return { mimeType, data };
}
// A storage pointer a client hands back after a presigned upload is kept only
// if it names a key minted for this entry and an object within the size bound.
async function vetUploadedThumbnail(
pointer: string,
entryUuid: unknown,
deps: { s3: S3Client; bucketName: string },
): Promise<string | null> {
const key = resolveThumbnailKey(pointer);
if (!key || thumbnailKeyOwner(key) !== entryUuid) return null;
let size: number | undefined;
try {
const head = await deps.s3.send(
new HeadObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
size = head.ContentLength;
} catch (err) {
if (!isMissingObjectError(err)) {
console.warn(
'[thumbnails] failed to check uploaded thumbnail',
err,
);
}
return null;
}
if (typeof size !== 'number') return null;
if (size > MAX_THUMBNAIL_BYTES) {
// Only reachable on a store that doesn't enforce the signed length.
// The key is this entry's own, so removing it is safe.
try {
await deps.s3.send(
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
);
} catch (err) {
console.warn(
'[thumbnails] failed to remove oversized thumbnail',
err,
);
}
return null;
}
return `s3://${deps.bucketName}/${key}`;
}
// -- thumbnail.created -----------------------------------------------
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer.
// and replace the URL with an s3:// pointer bound to the entry.
export async function handleThumbnailCreated(
event: Record<string, unknown>,
deps: { s3: S3Client; bucketName: string },
deps: { s3: S3Client; bucketName: string; bucketEndpoint: string },
): Promise<void> {
const url = event.url;
if (typeof url !== 'string' || !url.startsWith('data:')) return;
if (typeof url !== 'string') return;
if (!url.startsWith('data:')) {
if (isStoragePointer(url, deps.bucketEndpoint)) {
event.url = await vetUploadedThumbnail(url, event.uuid, deps);
}
return;
}
const entryUuid = event.uuid;
if (!isUuid(entryUuid)) {
event.url = null;
return;
}
const decoded = await decodeAndValidateThumbnail(url);
if (!decoded) {
@@ -161,7 +242,7 @@ export async function handleThumbnailCreated(
return;
}
const key = mintThumbnailKey();
const key = mintThumbnailKey(entryUuid);
event.url = `s3://${deps.bucketName}/${key}`;
await deps.s3.send(
@@ -190,21 +271,24 @@ export const handleThumbnailUploadPrepare = async (
typeof item.contentType === 'string' ? item.contentType.trim() : '';
if (!contentType) continue;
if (item.size !== undefined) {
const size = Number(item.size);
if (
!Number.isFinite(size) ||
size < 0 ||
size > MAX_THUMBNAIL_BYTES
)
continue;
}
const size = item.size;
if (
typeof size !== 'number' ||
!Number.isInteger(size) ||
size < 0 ||
size > MAX_THUMBNAIL_BYTES
)
continue;
if (!isUuid(item.item_uid)) continue;
const key = mintThumbnailKey();
const key = mintThumbnailKey(item.item_uid);
const command = new PutObjectCommand({
Bucket: deps.bucketName,
Key: key,
ContentType: contentType,
// Signing covers `content-length`, so a body of any other size
// fails the signature instead of landing.
ContentLength: size,
});
item.uploadUrl = await getSignedUrl(presignClient, command, {
expiresIn: 900,
@@ -227,12 +311,7 @@ export const handleThumbnailRead = async (
if (typeof thumb !== 'string' || !thumb) return;
const presignClient = deps.s3Presign;
if (
thumb.startsWith('s3://') ||
// Legacy format — remove after full migration
(thumb.startsWith('https') &&
thumb.includes(new URL(deps.bucketEndpoint).hostname))
) {
if (isStoragePointer(thumb, deps.bucketEndpoint)) {
const key = resolveThumbnailKey(thumb);
if (!key) {
// Not a pointer we minted — refuse to sign it rather than hand
@@ -247,7 +326,9 @@ export const handleThumbnailRead = async (
);
} else if (thumb.startsWith('data')) {
// Inline data-URL migration: upload to S3 and update the DB entry.
const key = mintThumbnailKey();
const uuid = entry.uuid ?? entry.uid;
if (!isUuid(uuid)) return;
const key = mintThumbnailKey(uuid);
const { mimeType, data } = base64ParseDataUrl(thumb);
const newUrl = `s3://${deps.bucketName}/${key}`;
@@ -261,7 +342,6 @@ export const handleThumbnailRead = async (
);
// Best-effort async DB update
const uuid = entry.uuid ?? entry.uid;
if (uuid) {
deps.db
.write(
@@ -286,24 +366,36 @@ export const handleFsCopyNodeThumbnail = async (
deps: {
s3: S3Client;
bucketName: string;
bucketEndpoint: string;
db: { write: (sql: string, params: unknown[]) => Promise<unknown> };
},
): Promise<void> => {
const copy = payload.copy;
const thumbnailUrl = copy?.thumbnail;
if (!copy || !copy.uuid || typeof thumbnailUrl !== 'string') return;
if (!copy || !isUuid(copy.uuid) || typeof thumbnailUrl !== 'string') return;
if (!isStoragePointer(thumbnailUrl, deps.bucketEndpoint)) return;
// Same trust rule as the read and remove paths: only touch objects this
// extension minted.
const sourceKey = resolveThumbnailKey(thumbnailUrl);
if (!sourceKey) return;
// The copied row points at the SAME S3 object as its source, and
// fs.remove.node deletes the pointed-to object — so the first removal
// among the sharers (an overwrite, a trash purge) would break every
// other sharer's thumbnail. Give the copy an object of its own.
const newKey = mintThumbnailKey();
// The copied row points at the SAME object as its source. Give the copy
// an object of its own, bound to it, so removing either entry leaves the
// other's thumbnail alone.
const newKey = mintThumbnailKey(copy.uuid);
try {
// Nothing larger than an upload may produce gets duplicated; such an
// object came in through an unbounded upload URL.
const head = await deps.s3.send(
new HeadObjectCommand({ Bucket: deps.bucketName, Key: sourceKey }),
);
if (
typeof head.ContentLength !== 'number' ||
head.ContentLength > MAX_THUMBNAIL_BYTES
) {
throw new Error('thumbnail exceeds the size bound');
}
await deps.s3.send(
new CopyObjectCommand({
Bucket: deps.bucketName,
@@ -312,10 +404,8 @@ export const handleFsCopyNodeThumbnail = async (
}),
);
} catch (err) {
// The shared object is already gone (e.g. a sharer was removed
// before this fix existed) — the pointer is dead either way, so
// drop it rather than leave the row advertising a thumbnail it
// doesn't have.
// The shared object is gone or oversized — drop the pointer rather
// than leave the row advertising a thumbnail it doesn't have.
await deps.db.write(
'UPDATE `fsentries` SET `thumbnail` = NULL WHERE `uuid` = ?',
[copy.uuid],
@@ -331,18 +421,18 @@ export const handleFsCopyNodeThumbnail = async (
};
export const handleFsRemoveNodeThumbnail = async (
payload: { target: { thumbnail?: string | null } },
deps: { s3: S3Client; bucketName: string },
payload: { target: { thumbnail?: string | null; uuid?: string } },
deps: { s3: S3Client; bucketName: string; bucketEndpoint: string },
): Promise<void> => {
const thumbnailUrl = payload.target.thumbnail;
if (!thumbnailUrl) return;
const { thumbnail: thumbnailUrl, uuid } = payload.target;
if (!thumbnailUrl || !isStoragePointer(thumbnailUrl, deps.bucketEndpoint))
return;
// Same trust rule as the read path, and load-bearing for the same reason:
// the pointer decides which object gets deleted, so a key we didn't mint
// would let the owner of one file destroy an object belonging to someone
// else just by naming it here.
// The pointer decides which object gets deleted, and a key is visible to
// anyone who can list the entry holding it. Only a key minted for this
// entry is deleted; an unbound one may be shared or planted, so it stays.
const key = resolveThumbnailKey(thumbnailUrl);
if (!key) return;
if (!key || thumbnailKeyOwner(key) !== uuid) return;
await deps.s3.send(
new DeleteObjectCommand({ Bucket: deps.bucketName, Key: key }),
@@ -355,6 +445,7 @@ extension.on(
await handleThumbnailCreated(event, {
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
});
},
);
@@ -397,6 +488,7 @@ extension.on('fs.copy.node', async (_key, payload) => {
{
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
db: clients.db,
},
);
@@ -409,5 +501,6 @@ extension.on('fs.remove.node', async (_key, payload) => {
await handleFsRemoveNodeThumbnail(payload, {
s3: getClient(),
bucketName: thumbnailBucketName,
bucketEndpoint: extensionBucketEndpoint,
});
});
+355
View File
@@ -0,0 +1,355 @@
import { EventEmitter } from 'node:events';
import { Readable } from 'node:stream';
import type { Request, Response } from 'express';
import { beforeEach, describe, expect, test, vi } from 'vitest';
const SECRET = 'ingress-secret';
// Only `extension` is stubbed; the secret comparison and the routing are the
// real code paths, and the fake service/store layer is the boundary.
const state = vi.hoisted(() => ({
config: { userEmail: { secret: 'ingress-secret' } } as Record<
string,
unknown
>,
routes: [] as Array<{
path: string;
handler: (req: unknown, res: unknown) => unknown;
}>,
users: new Map<string, Record<string, unknown>>(),
writes: [] as Array<{
userId: number;
path: string;
homeRegion?: string;
content?: unknown;
}>,
feedback: [] as Array<{ to: string; from: string | null; raw: Buffer }>,
takeFeedback: true,
}));
vi.mock('@heyputer/backend/src/extensions', () => ({
extension: {
get config() {
return state.config;
},
post(
path: string,
_opts: unknown,
handler: (req: unknown, res: unknown) => unknown,
) {
state.routes.push({ path, handler });
},
import(layer: string) {
if (layer === 'client') {
return {
event: {
async emitAndWait(
key: string,
data: {
to: string;
from: string | null;
raw: Buffer;
handled: boolean;
},
) {
if (key !== 'email.ingress.feedback') return;
state.feedback.push(data);
if (state.takeFeedback) data.handled = true;
},
},
};
}
if (layer === 'store') {
return {
user: {
getByUsername: async (username: string) =>
state.users.get(username) ?? null,
},
};
}
return {
fs: {
async write(
userId: number,
{
fileMetadata,
fileContent,
}: {
fileMetadata: { path: string };
fileContent: unknown;
},
_uploadTracker?: unknown,
_storageAllowanceMax?: number,
homeRegion?: string,
) {
state.writes.push({
userId,
path: fileMetadata.path,
homeRegion,
content: fileContent,
});
return { fsEntry: { path: fileMetadata.path } };
},
},
};
},
},
}));
import './index.js';
const permanent = (id: number, username: string) => ({
id,
username,
email: `${username}@example.com`,
password: '$2b$10$hash',
});
/** No password and no email is what makes an account temporary. */
const temp = (id: number, username: string) => ({
id,
username,
email: null,
password: null,
});
const ingress = async (
to: string,
over: { secret?: string; contentLength?: string | null } = {},
) => {
const route = state.routes.find((r) => r.path === '/email/ingress');
if (!route) throw new Error('route not registered');
const body = Buffer.from('From: [email protected]\r\n\r\nhello\r\n');
const req = Readable.from([body], { objectMode: false }) as unknown as {
headers: Record<string, string>;
query: Record<string, string>;
};
// `contentLength: null` sends no length header at all, which is the one
// framing the HTTP parser does not bound for us.
req.headers =
over.contentLength === null
? {}
: {
'content-length': over.contentLength ?? String(body.length),
};
req.query = {
SECRET: over.secret ?? SECRET,
to,
from: '[email protected]',
subject: 'hi',
};
let status = 200;
let ended = false;
// `end` takes (chunk, encoding) and no callback, the way the compression
// middleware leaves it, and emits `finish` — which is what the refusal
// paths hang their drain/destroy on. A stub that accepts a callback
// instead hides the bug that took the 413 path to production as a 500.
const emitter = new EventEmitter();
const res = {
status(code: number) {
status = code;
return res;
},
end(chunk?: unknown, encoding?: BufferEncoding) {
if (chunk) Buffer.byteLength(chunk as string, encoding);
ended = true;
emitter.emit('finish');
},
once(event: string, listener: () => void) {
emitter.once(event, listener);
return res;
},
} as unknown as Response;
await route.handler(req as unknown as Request, res);
return {
status,
ended,
drained: (req as unknown as Readable).readableFlowing,
destroyed: (req as unknown as Readable).destroyed,
};
};
beforeEach(() => {
state.users.clear();
state.writes.length = 0;
state.feedback.length = 0;
state.takeFeedback = true;
});
describe('temporary accounts cannot receive', () => {
test('mail for a temp account is accepted and dropped', async () => {
state.users.set('tempy', temp(1, 'tempy'));
const { status, ended, drained } = await ingress('[email protected]');
// Accepted, so the relay neither retries nor bounces to the sender.
expect(status).toBe(200);
expect(ended).toBe(true);
// The body is read off the wire rather than left unconsumed.
expect(drained).toBe(true);
expect(state.writes).toEqual([]);
});
test('mail for a permanent account still lands in the mailbox', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status } = await ingress('[email protected]');
expect(status).toBe(200);
expect(state.writes).toHaveLength(1);
expect(state.writes[0].userId).toBe(2);
expect(state.writes[0].path).toMatch(/^\/bob\/\.mail\/objects\//);
});
test('an unknown address is still rejected outright', async () => {
const { status } = await ingress('[email protected]');
expect(status).toBe(404);
expect(state.writes).toEqual([]);
});
});
describe('inbound mail is placed in the recipient home region', () => {
/** A permanent account with explicit placement columns. */
const placed = (
id: number,
username: string,
over: { home?: string | null; signup_server?: string | null },
) => ({
...permanent(id, username),
home: null,
signup_server: null,
...over,
});
test("uses the recipient's own home region when they have one", async () => {
state.users.set('bob', placed(2, 'bob', { home: 'frankfurt' }));
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('frankfurt');
});
test('falls back to the server that served their signup', async () => {
// Every account that predates the `home` column is this case.
state.users.set(
'bob',
placed(2, 'bob', { home: null, signup_server: 'london' }),
);
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('london');
});
test('falls back to the primary region when neither is set', async () => {
state.users.set('bob', placed(2, 'bob', {}));
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('oregon');
});
test('home wins over signup_server when both are set', async () => {
state.users.set(
'bob',
placed(2, 'bob', { home: 'sydney', signup_server: 'london' }),
);
await ingress('[email protected]');
expect(state.writes[0].homeRegion).toBe('sydney');
});
});
describe('refusals dispose of the body nobody read', () => {
test('an unknown recipient drains it and keeps the connection', async () => {
// The secret checked out and the length is under the cap, so the body
// was ours to read: discard it rather than resetting a usable socket.
const { status, drained, destroyed } =
await ingress('[email protected]');
expect(status).toBe(404);
expect(drained).toBe(true);
expect(destroyed).toBe(false);
});
test('a bad secret closes rather than reading the upload', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
secret: 'not-the-secret',
});
expect(status).toBe(403);
// Nothing authenticated this caller; their bytes are not worth reading.
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
test('no content-length is refused with 411', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
contentLength: null,
});
expect(status).toBe(411);
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
test('a declared length over the cap is refused with 413', async () => {
state.users.set('bob', permanent(2, 'bob'));
const { status, destroyed } = await ingress('[email protected]', {
contentLength: String(25 * 1024 * 1024 + 1),
});
expect(status).toBe(413);
expect(destroyed).toBe(true);
expect(state.writes).toEqual([]);
});
});
describe('feedback addresses', () => {
test('are handed to a listener and reach no mailbox', async () => {
const { status, ended } = await ingress('[email protected]');
expect(status).toBe(200);
expect(ended).toBe(true);
expect(state.feedback).toHaveLength(1);
expect(state.feedback[0].to).toBe('[email protected]');
expect(state.feedback[0].from).toBe('[email protected]');
expect(state.feedback[0].raw.toString()).toContain('hello');
expect(state.writes).toEqual([]);
});
test('match the local part case-insensitively', async () => {
await ingress('[email protected]');
expect(state.feedback).toHaveLength(1);
});
test('fall through to normal routing when nobody takes them', async () => {
state.takeFeedback = false;
state.users.set('abuse', permanent(3, 'abuse'));
const { status } = await ingress('[email protected]');
expect(status).toBe(200);
expect(state.feedback).toHaveLength(1);
// The body was already read, so the buffered copy is what is stored.
expect(state.writes).toHaveLength(1);
expect(Buffer.isBuffer(state.writes[0].content)).toBe(true);
});
test('an unclaimed one with no such account is still a 404', async () => {
state.takeFeedback = false;
const { status } = await ingress('[email protected]');
expect(status).toBe(404);
});
test('only count on a Puter mail domain', async () => {
state.users.set('abuse', permanent(3, 'abuse'));
await ingress('[email protected]');
expect(state.feedback).toEqual([]);
});
test('are refused over the feedback size cap without being read', async () => {
const { status, destroyed } = await ingress('[email protected]', {
contentLength: String(5 * 1024 * 1024 + 1),
});
expect(status).toBe(413);
expect(destroyed).toBe(true);
expect(state.feedback).toEqual([]);
});
});
+161
View File
@@ -0,0 +1,161 @@
import { extension } from '@heyputer/backend/src/extensions';
import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
import type { Request } from 'express';
import {
MAX_MESSAGE_BYTES,
isPuterEmailAddress,
isTempUser,
puterEmailUsername,
storeInboxMessage,
} from '@heyputer/backend/src/services/email/mailbox.js';
// copied form peer secrets checker
const COMPARE_KEY = randomBytes(32);
const secretsEqual = (a: string, b: string): boolean =>
timingSafeEqual(
createHmac('sha256', COMPARE_KEY).update(a).digest(),
createHmac('sha256', COMPARE_KEY).update(b).digest(),
);
const USER_EMAIL_CONFIG = (extension.config as Record<string, unknown>)
.userEmail as { secret?: string; feedbackAddresses?: unknown } | undefined;
const INGRESS_SECRET = USER_EMAIL_CONFIG?.secret;
/**
* Local parts that take complaint feedback rather than naming a mailbox. The
* same names are reserved at signup, so no account can be handed them.
*/
const DEFAULT_FEEDBACK_ADDRESSES = ['fbl', 'abuse', 'postmaster'];
const FEEDBACK_ADDRESSES = new Set(
(Array.isArray(USER_EMAIL_CONFIG?.feedbackAddresses)
? USER_EMAIL_CONFIG.feedbackAddresses
: DEFAULT_FEEDBACK_ADDRESSES
).map((local) => String(local).toLowerCase()),
);
/**
* Feedback is handed over whole, so it is read into memory; a complaint report
* quotes one message and has no business near the mailbox ceiling.
*/
const MAX_FEEDBACK_BYTES = 5 * 1024 * 1024;
const isFeedbackAddress = (to: string): boolean =>
isPuterEmailAddress(to) &&
FEEDBACK_ADDRESSES.has(puterEmailUsername(to).toLowerCase());
const readBody = async (req: Request): Promise<Buffer> => {
const chunks: Buffer[] = [];
for await (const chunk of req) chunks.push(chunk as Buffer);
return Buffer.concat(chunks);
};
if (!INGRESS_SECRET) {
console.warn(
'[useremailrx] config.userEmail.secret unset - email ingress disabled',
);
} else {
extension.post(
'/email/ingress',
{
subdomain: 'api',
},
async (req, res) => {
// Answer first, then deal with the body nobody read. Touching the
// request before the response has flushed takes the response down
// with the connection and the caller sees a reset with no status,
// so this hangs off `finish` — and it has to be the event rather
// than `res.end(cb)`, because the compression middleware replaces
// `res.end` with an `(chunk, encoding)` signature that has no
// callback parameter, so a callback lands in `chunk` and dies in
// `Buffer.byteLength`.
//
// `drain` discards the rest of a body we were entitled to read and
// leaves the connection reusable. `close` is for bodies we do not
// want to read at all: an unauthenticated caller's, or one whose
// declared length is missing or over the cap.
const refuse = (status: number, how: 'drain' | 'close') => {
res.once('finish', () =>
how === 'drain' ? req.resume() : req.destroy(),
);
res.status(status).end();
};
const presented = req.query.SECRET;
if (
typeof presented !== 'string' ||
!secretsEqual(presented, INGRESS_SECRET)
) {
// Nothing has authenticated this caller, so refuse to spend
// bandwidth reading whatever they were uploading.
refuse(403, 'close');
return;
}
const size = Number(req.headers['content-length']);
if (!Number.isInteger(size) || size <= 0) {
refuse(411, 'close');
return;
}
if (size > MAX_MESSAGE_BYTES) {
refuse(413, 'close');
return;
}
const stores = extension.import('store');
const services = extension.import('service');
const to = String(req.query.to ?? '');
let content: Buffer | Request = req;
if (isFeedbackAddress(to)) {
if (size > MAX_FEEDBACK_BYTES) {
refuse(413, 'close');
return;
}
const raw = await readBody(req);
const event = {
to,
from:
typeof req.query.from === 'string'
? req.query.from
: null,
raw,
handled: false,
};
await extension
.import('client')
.event.emitAndWait('email.ingress.feedback', event, {});
if (event.handled) {
res.end();
return;
}
// Nobody took it: route it like any other message.
content = raw;
}
const user = await stores.user.getByUsername(
puterEmailUsername(to),
);
if (!user) {
refuse(404, 'drain');
return;
}
if (isTempUser(user)) {
if (content === req) req.resume();
res.end();
return;
}
await storeInboxMessage(services.fs, user, {
subject: req.query.subject,
content,
size: Buffer.isBuffer(content) ? content.length : size,
});
const { SECRET: _secret, ..._loggableQuery } = req.query;
// console.log('got email: ', puterUser, loggableQuery, size);
res.end();
},
);
}
+12
View File
@@ -0,0 +1,12 @@
{
"name": "emailrx",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "emailrx",
"version": "1.0.0"
}
}
}
+6
View File
@@ -0,0 +1,6 @@
{
"name": "emailrx",
"version": "1.0.0",
"main": "index.ts",
"description": "Internal service which handles routing puter emails to users"
}
+99 -1
View File
@@ -8,7 +8,7 @@ import {
it,
vi,
} from 'vitest';
import { makeActor } from '../src/backend/core/actor.ts';
import { makeActor, type Actor } from '../src/backend/core/actor.ts';
import { runWithContext } from '../src/backend/core/context.ts';
import { configContainer } from '../src/backend/exports.ts';
import { PuterServer } from '../src/backend/server.ts';
@@ -285,6 +285,104 @@ describe('whoami extension — handleWhoami', () => {
expect(body.directories).toBeUndefined();
});
describe('fields added by whoami.details listeners', () => {
const addAccountDetails = (
_key: unknown,
event: { details: Record<string, unknown> },
) => {
Object.assign(event.details, {
subscribed: true,
paid_storage: 1024,
subscription: { tier: 'pro', status: 'trialing' },
accountEligible: true,
});
};
const whoamiWithListener = async (
actor: Actor,
listener: typeof addAccountDetails = addAccountDetails,
) => {
const { res, captured } = makeRes();
server.clients.event.on('whoami.details', listener);
try {
await runWithContext({ actor }, () =>
handleWhoami(makeReq(), res),
);
} finally {
server.clients.event.off('whoami.details', listener);
}
return captured.body as Record<string, unknown>;
};
it('reach an app actor only as the documented subscribed / paid_storage', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
);
expect(body.subscribed).toBe(true);
expect(body.paid_storage).toBe(1024);
expect(body).not.toHaveProperty('subscription');
expect(body).not.toHaveProperty('accountEligible');
// Fields the handler built itself still ship.
expect(body.username).toBe(user.username);
expect(body.app_name).toBe('app-test-actor');
});
it('are withheld from an app reached through an access token it issued', async () => {
const user = await seedUser();
const owner = { uuid: user.uuid, id: user.id as number };
const body = await whoamiWithListener(
makeActor({
user: owner,
accessToken: {
uid: 'tok-1',
issuer: makeActor({
user: owner,
app: { uid: 'app-issuer' },
}),
},
}),
);
expect(body.subscribed).toBe(true);
expect(body).not.toHaveProperty('subscription');
expect(body).not.toHaveProperty('accountEligible');
});
it('all reach a user actor', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({ user: { uuid: user.uuid, id: user.id as number } }),
);
expect(body).toMatchObject({
subscribed: true,
paid_storage: 1024,
subscription: { tier: 'pro', status: 'trialing' },
accountEligible: true,
});
});
it('cannot add taskbar_items for an app actor, which core omits', async () => {
const user = await seedUser();
const body = await whoamiWithListener(
makeActor({
user: { uuid: user.uuid, id: user.id as number },
app: { uid: 'app-test-actor' },
}),
(_key, event) => {
event.details.taskbar_items = [{ name: 'injected' }];
},
);
expect(body).not.toHaveProperty('taskbar_items');
});
});
it('redacts tmp_password from metadata for user actors', async () => {
const user = await seedUser();
await server.stores.user.updateMetadata(user.id as number, {
+36 -15
View File
@@ -71,6 +71,13 @@ const SENSITIVE_KEYS: ReadonlySet<string> = new Set([
'audit_metadata',
]);
// What an app actor may receive from `whoami.details` listeners. Anything
// else a listener adds describes the account, and only ships to user actors.
const APP_VISIBLE_LISTENER_KEYS: ReadonlySet<string> = new Set([
'subscribed',
'paid_storage',
]);
// Depth-limited, cycle-safe walk deleting every SENSITIVE_KEYS entry it finds
// at any level (`metadata` and `taskbar_items` are both nested structures).
const scrubSensitive = (
@@ -186,19 +193,6 @@ export const handleWhoami = async (
// this endpoint is polled, and a mint is a write.
referral_code: user.referral_code,
oidc_only: oidcOnly,
taskbar_items: isUser
? await getTaskbarItems(
user,
{
clients,
stores,
services,
apiBaseUrl: String(extension.config.api_base_url ?? ''),
config: extension.config,
},
{ iconSize, noIcons },
)
: undefined,
otp: !!user.otp_enabled,
feature_flags,
created_ts: toUnixSeconds(user.timestamp),
@@ -227,6 +221,21 @@ export const handleWhoami = async (
}
}
// Taskbar items — only sent to user actors
if (isUser) {
details.taskbar_items = await getTaskbarItems(
user,
{
clients,
stores,
services,
apiBaseUrl: String(extension.config.api_base_url ?? ''),
config: extension.config,
},
{ iconSize, noIcons },
);
}
// Directories — only sent to user actors
if (isUser) {
const directories: Record<string, unknown> = {};
@@ -292,6 +301,12 @@ export const handleWhoami = async (
details.app_name = app.uid;
}
// A key core left undefined isn't built, so a listener can't fill it for an app.
const builtKeys = isUser
? null
: new Set(
Object.keys(details).filter((key) => details[key] !== undefined),
);
try {
await clients.event.emitAndWait(
'whoami.details',
@@ -301,10 +316,16 @@ export const handleWhoami = async (
} catch {
/* best-effort */
}
if (builtKeys) {
for (const key of Object.keys(details)) {
if (!builtKeys.has(key) && !APP_VISIBLE_LISTENER_KEYS.has(key)) {
delete details[key];
}
}
}
const subscription = details.subscription as
| { offering?: Record<string, unknown> }
| undefined;
{ offering?: Record<string, unknown> } | undefined;
if (subscription?.offering) {
delete subscription.offering.group;
delete subscription.offering.benefits;
+114 -83
View File
@@ -16,7 +16,7 @@
"@aws-sdk/s3-request-presigner": "^3.1028.0",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.13",
"libphonenumber-js": "1.13.14",
"miniflare": "^4.20260617.1",
"open": "^10.1.0"
},
@@ -2043,9 +2043,9 @@
"license": "MIT"
},
"node_modules/@eslint/config-array/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.21",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -2124,9 +2124,9 @@
"license": "MIT"
},
"node_modules/@eslint/eslintrc/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.21",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -2412,9 +2412,9 @@
}
},
"node_modules/@grpc/grpc-js": {
"version": "1.14.4",
"resolved": "https://registry.npmjs.org/@grpc/grpc-js/-/grpc-js-1.14.4.tgz",
"integrity": "sha512-k9Dj3DV/itK9D06Y8f190Qgop7/Ui+D0njFV3LHMPwPT75DpXLQohE9Wmz0QElrJnzsjB7KPWiKJbOl7IPDArQ==",
"version": "1.14.5",
"resolved": "https://registry.npmjs.org/@grpc/grpc-js/-/grpc-js-1.14.5.tgz",
"integrity": "sha512-7VZM+SVdEcUUqSQeNI3zM8Qs/BhQKZndPo2h5VkYkAM8Iz0wJIa8mKV5ekQGqG8UUsnkQ0NMxIxwkIHYvj0qOw==",
"license": "Apache-2.0",
"dependencies": {
"@grpc/proto-loader": "^0.8.0",
@@ -6055,9 +6055,9 @@
}
},
"node_modules/@types/node": {
"version": "24.13.4",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.4.tgz",
"integrity": "sha512-YJ7EqCstVTzIr0fMr7qul/977en+pQHrfmuKIo6Zr9i75Be21dr3MovcfvGtyvi2HAUrRerWps5sMO9I7WaxDw==",
"version": "24.13.6",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.6.tgz",
"integrity": "sha512-SGrw/h3KPFshy3OE6ZL53LMBG5vGQQ8/gIpiqz/kRZhPJ7HgwCEs8LBuNtWLa8dvGZVpSF7+Bf+c11HUrCb/yg==",
"license": "MIT",
"dependencies": {
"undici-types": "~7.18.0"
@@ -6153,6 +6153,17 @@
"@types/node": "*"
}
},
"node_modules/@types/smtp-server": {
"version": "3.5.13",
"resolved": "https://registry.npmjs.org/@types/smtp-server/-/smtp-server-3.5.13.tgz",
"integrity": "sha512-S3rGl2KbViH+98/CgHipPIWgtAFnjxLsppxIRbgJHNuZL7Y+py+7kZjT7xS+wOmCxYTn4O7IqLqkvekg99MDVQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*",
"@types/nodemailer": "*"
}
},
"node_modules/@types/tedious": {
"version": "4.0.14",
"resolved": "https://registry.npmjs.org/@types/tedious/-/tedious-4.0.14.tgz",
@@ -7185,13 +7196,13 @@
}
},
"node_modules/axios": {
"version": "1.18.1",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.18.1.tgz",
"integrity": "sha512-3nTvFlvpn9Zu/RkHUqtc7/+al4UpRW5az71ap5zccp6e8RAYEzhMTecX8Dz1wWDYrPpUoB1HAQEGEAEvUr7S9g==",
"version": "1.20.0",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.20.0.tgz",
"integrity": "sha512-r8aOh8j9cGKpgQAqpzrUHnSIc6a59Y3Xf/cv8sy1DrHCkZHzQGEuoq1tARk6qSyDdtQGSDgpb9kFlruzPvrgwg==",
"license": "MIT",
"dependencies": {
"follow-redirects": "^1.16.0",
"form-data": "^4.0.5",
"form-data": "^4.0.6",
"https-proxy-agent": "^5.0.1",
"proxy-from-env": "^2.1.0"
}
@@ -7272,15 +7283,6 @@
],
"license": "MIT"
},
"node_modules/base64id": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/base64id/-/base64id-2.0.0.tgz",
"integrity": "sha512-lGe34o6EHj9y3Kts9R4ZYs/Gr+6N7MCaMlIFA3F1R2O5/m7K06AxfSeO5530PEERE6/WyEg3lsuyw4GHlPZHog==",
"license": "MIT",
"engines": {
"node": "^4.5.0 || >= 5.9"
}
},
"node_modules/baseline-browser-mapping": {
"version": "2.11.21",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.21.tgz",
@@ -7506,9 +7508,9 @@
"license": "MIT"
},
"node_modules/brace-expansion": {
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
"integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
"version": "5.0.12",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.12.tgz",
"integrity": "sha512-YovQ3rzhaLMIrDjNDMkNS01tea93qhEhG5xy8f6+R0l+dw3Ki+5sCoIoI942iuLZTHWogWktgwVDhU09iNEimQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -8756,16 +8758,15 @@
}
},
"node_modules/engine.io": {
"version": "6.6.9",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.6.9.tgz",
"integrity": "sha512-clKkw4C7nJ22mGgoVcCg6V/W/TxdNyIOTr89k2ONZu81qqkddPFDF0LXcbAwhzPD8DjkiRCjzuiO6Y+fkpD4vg==",
"version": "6.6.11",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.6.11.tgz",
"integrity": "sha512-z34mGpLoY/82ZbkX1LqGMIaf3wjVMPL6gVC681QDqnmzJ26synI8qAE32P+a7UazmpJZ8UGrFIeAavgRfXzYrg==",
"license": "MIT",
"dependencies": {
"@types/cors": "^2.8.12",
"@types/node": ">=10.0.0",
"@types/ws": "^8.5.12",
"accepts": "~1.3.4",
"base64id": "2.0.0",
"cookie": "~0.7.2",
"cors": "~2.8.5",
"debug": "~4.4.1",
@@ -9181,9 +9182,9 @@
"license": "MIT"
},
"node_modules/eslint/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.21",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -9542,9 +9543,9 @@
"license": "Unlicense"
},
"node_modules/fast-uri": {
"version": "3.1.7",
"resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.7.tgz",
"integrity": "sha512-dOvZVzjdZdz7phd9v6jCbwxrBW3fK6n8Rc0CtdmM4bumzMnxywBYhuph6J819RRw/ku+rLbelwfMunktuzVVHg==",
"version": "3.1.8",
"resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.8.tgz",
"integrity": "sha512-GZMtZUTNRpOVIECoXwLNZS5xUGE+mVNbTB8h/7Rwh2TFWcBQiPzTgyZi05BF9UMZKkLJv8XBRJTlU7zg8+ZfMg==",
"funding": [
{
"type": "github",
@@ -9568,9 +9569,9 @@
}
},
"node_modules/fastify": {
"version": "5.12.3",
"resolved": "https://registry.npmjs.org/fastify/-/fastify-5.12.3.tgz",
"integrity": "sha512-reZ8wce5VNCcufIt9AVtzZa3L4u1j8esikn7OEgHWLVpRpL5R7Y2+Xzj70OUkv5zDfzUAxXZT6cu4Rt0zr3EKA==",
"version": "5.12.5",
"resolved": "https://registry.npmjs.org/fastify/-/fastify-5.12.5.tgz",
"integrity": "sha512-OB2k1dlxs5/NAABqeKV2FUHkSD2BbENsCak8yULVcymn3fHIPDVa9TI3SDnJSWYSllZmSYuZXy2gTnsT+Sut1A==",
"funding": [
{
"type": "github",
@@ -10797,6 +10798,12 @@
"node": ">= 10"
}
},
"node_modules/ipv6-normalize": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/ipv6-normalize/-/ipv6-normalize-1.0.1.tgz",
"integrity": "sha512-Bm6H79i01DjgGTCWjUuCjJ6QDo1HB96PT/xCYuyJUP9WFbVDrLSbG4EZCvOCun2rNswZb0c3e4Jt/ws795esHA==",
"license": "MIT"
},
"node_modules/is-binary-path": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/is-binary-path/-/is-binary-path-2.1.0.tgz",
@@ -11492,9 +11499,9 @@
}
},
"node_modules/libphonenumber-js": {
"version": "1.13.13",
"resolved": "https://registry.npmjs.org/libphonenumber-js/-/libphonenumber-js-1.13.13.tgz",
"integrity": "sha512-hV6SmWnyQ8U1QXhhqbqYBHIStNnscG7A5XbGodgdHhf/8N5PwhYpJA5U/yHDjYwPYTd8uNGXqq416XctuvK2PA==",
"version": "1.13.14",
"resolved": "https://registry.npmjs.org/libphonenumber-js/-/libphonenumber-js-1.13.14.tgz",
"integrity": "sha512-llihgCcx0BFLksecLP+x1J+6JDE1GsXS1RN/LoPF6qcwpeQcnjj0lcvZxY8AzbEpYwyZWPZW/nDuqkqzm3amiw==",
"license": "MIT"
},
"node_modules/light-my-request": {
@@ -14225,9 +14232,9 @@
}
},
"node_modules/postal-mime": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/postal-mime/-/postal-mime-3.0.0.tgz",
"integrity": "sha512-Z4a9ar2Bv3YpK3IXag+Yda30k7bMZfpRuUGyqtHnZ2pjHG8Bl62EhZIk4n1dzv00gfzP9g+94e9kd8+XmjVWLA==",
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/postal-mime/-/postal-mime-3.0.1.tgz",
"integrity": "sha512-N86EnYOxZjW1uHQIgSXnnRdnJz+wxmtDsnk3C4R/9rQ1utj8A2znauXZ3Q3+stCHxG5ckY4/NrO09WiPU6rcdQ==",
"license": "MIT-0"
},
"node_modules/postcss": {
@@ -14336,9 +14343,9 @@
}
},
"node_modules/prettier": {
"version": "3.9.6",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.6.tgz",
"integrity": "sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g==",
"version": "3.9.9",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.9.tgz",
"integrity": "sha512-Z/CJHIkdujO/OtN7nXUii0Rf3VT5SRuhjBA82Xvu2XhBUgX3nhP67T0LHceBdQLex7OOFGTox+Q5Yg8Jk2Qivg==",
"dev": true,
"license": "MIT",
"bin": {
@@ -14488,6 +14495,15 @@
"node": ">=6"
}
},
"node_modules/punycode.js": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/punycode.js/-/punycode.js-2.3.1.tgz",
"integrity": "sha512-uxFIHU0YlHYhDQtV4R9J6a52SLx28BCjT+4ieh7IGbgwVJWO+km431c4yRlREUAsAmt/uMjQUyQHNEPf0M39CA==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/puter-mcp-connector": {
"resolved": "src/mcp-connector",
"link": true
@@ -14845,9 +14861,9 @@
"license": "MIT"
},
"node_modules/rimraf/node_modules/brace-expansion": {
"version": "2.1.4",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.4.tgz",
"integrity": "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg==",
"version": "2.1.7",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.7.tgz",
"integrity": "sha512-uZbew1NqdmPDTMJ8ah1y+b+9QEJrfkXFk3RcTQw3X0jW/xRUvFKsg1CfQdSYGdTbXZWExtU3J3ccxtnfw1Fi0g==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0"
@@ -15530,14 +15546,36 @@
"integrity": "sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==",
"license": "MIT"
},
"node_modules/smtp-server": {
"version": "3.19.13",
"resolved": "https://registry.npmjs.org/smtp-server/-/smtp-server-3.19.13.tgz",
"integrity": "sha512-W1CjNcrPqW4+1gUOYRyjPvTOxu7GgKR9+7l50yQBF0GM8v8jpYwheHFCHX7s+p7pYI8iXKQQZiF2Rv2Na/3i2g==",
"license": "MIT-0",
"dependencies": {
"ipv6-normalize": "1.0.1",
"nodemailer": "10.0.10",
"punycode.js": "2.3.1"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/smtp-server/node_modules/nodemailer": {
"version": "10.0.10",
"resolved": "https://registry.npmjs.org/nodemailer/-/nodemailer-10.0.10.tgz",
"integrity": "sha512-He9XskOFms62SyAKkLu8CcGcYHAo+BSHSsORI8s4PJH8qZgZY4L4lDfvyN0twvmbpyG+eGrxpyBlskj9d9rJ8A==",
"license": "MIT-0",
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/socket.io": {
"version": "4.8.3",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.8.3.tgz",
"integrity": "sha512-2Dd78bqzzjE6KPkD5fHZmDAKRNe3J15q+YHDrIsy9WEkqttc7GY+kT9OBLSMaPbQaEd0x1BjcmtMtXkfpc+T5A==",
"version": "4.8.4",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.8.4.tgz",
"integrity": "sha512-9ZV9zg8vN8hO3Y25DhR7nUkJ1VDnBhq5O/JrXhFT9s6+F0gqytSASTHVfCFc2mJXaTfARroErplXlzFWwETZqQ==",
"license": "MIT",
"dependencies": {
"accepts": "~1.3.4",
"base64id": "~2.0.0",
"cors": "~2.8.5",
"debug": "~4.4.1",
"engine.io": "~6.6.0",
@@ -15576,9 +15614,9 @@
}
},
"node_modules/socket.io-client": {
"version": "4.8.3",
"resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-4.8.3.tgz",
"integrity": "sha512-uP0bpjWrjQmUt5DTHq9RuoCBdFJF10cdX9X+a368j/Ft0wmaVgxlrjvK3kjvgCODOMMOz9lcaRzxmso0bTWZ/g==",
"version": "4.8.4",
"resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-4.8.4.tgz",
"integrity": "sha512-7wjKAvVXYOJsbmUgkWbnhPD4bFpkqrUR0++XOjpGFmqDqsqV1CI93VTraeExLxpjxEqrWgFoSRZu1ihjLWfkAA==",
"license": "MIT",
"dependencies": {
"@socket.io/component-emitter": "~3.1.0",
@@ -16133,9 +16171,9 @@
"license": "MIT"
},
"node_modules/test-exclude/node_modules/brace-expansion": {
"version": "2.1.4",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.4.tgz",
"integrity": "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg==",
"version": "2.1.7",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.7.tgz",
"integrity": "sha512-uZbew1NqdmPDTMJ8ah1y+b+9QEJrfkXFk3RcTQw3X0jW/xRUvFKsg1CfQdSYGdTbXZWExtU3J3ccxtnfw1Fi0g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -16538,9 +16576,9 @@
"license": "MIT"
},
"node_modules/undici": {
"version": "7.29.1",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.1.tgz",
"integrity": "sha512-RYONW2MeafgYlkVOKYKkA/Ag7BmXqgIWCa8t1m0JcxrQg9pI9lEqRhAOruOBCbAohOa/gkCF+iPi9hrgvTzu6Q==",
"version": "7.30.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.30.0.tgz",
"integrity": "sha512-dkrQXeHSaoamnItlYbmzG0wFYrM0ZwDxCIg0A7aKjTyyhh9svRzCNFEzV+Vm05/yehjCzjDZ31KXfGEjYSztDQ==",
"license": "MIT",
"engines": {
"node": ">=20.18.1"
@@ -17338,16 +17376,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/wrangler/node_modules/undici": {
"version": "7.29.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
"integrity": "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=20.18.1"
}
},
"node_modules/wrangler/node_modules/workerd": {
"version": "1.20260915.1",
"resolved": "https://registry.npmjs.org/workerd/-/workerd-1.20260915.1.tgz",
@@ -17624,7 +17652,7 @@
"@pagerduty/pdjs": "^2.2.4",
"@smithy/node-http-handler": "^2.5.0",
"@socket.io/redis-streams-adapter": "^0.3.1",
"axios": "^1.15.0",
"axios": "^1.20.0",
"bcrypt": "^6.0.0",
"better-sqlite3": "^12.6.0",
"busboy": "^1.6.0",
@@ -17650,9 +17678,11 @@
"openai": "^6.34.0",
"otpauth": "^9.5.2",
"pg": "^8.21.0",
"postal-mime": "^3.0.1",
"replicate": "^1.0.0",
"sharp": "^0.35.4",
"socket.io": "^4.8.3",
"smtp-server": "^3.19.13",
"socket.io": "^4.8.4",
"svg-captcha": "^1.4.0",
"together-ai": "^0.33.0",
"ua-parser-js": "^1.0.41",
@@ -17667,9 +17697,10 @@
"@types/express": "^5.0.0",
"@types/jsonwebtoken": "^9.0.10",
"@types/mime-types": "^2.1.4",
"@types/node": "^24.13.4",
"@types/node": "^24.13.6",
"@types/nodemailer": "^8.0.1",
"@types/pg": "^8.6.1",
"@types/smtp-server": "^3.5.13",
"@types/validator": "^13.15.10",
"pgmock": "^1.0.3",
"typescript": "^5.9.3",
@@ -18437,14 +18468,14 @@
},
"src/puter-js": {
"name": "@heyputer/puter.js",
"version": "2.6.3",
"version": "2.6.4",
"license": "Apache-2.0",
"dependencies": {
"@heyputer/kv.js": "^0.2.1",
"open": "^10.2.0",
"path-browserify": "1.0.1",
"postal-mime": "^3.0.0",
"socket.io-client": "^4.8.3"
"postal-mime": "^3.0.1",
"socket.io-client": "^4.8.4"
},
"devDependencies": {
"@playwright/test": "^1.49.0",
+3 -2
View File
@@ -50,6 +50,7 @@
"build:workerLib:coverage": "cd src/puter-js && npm run build:coverage && cd ../worker && npm run build",
"start:gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node ./tools/start.mjs",
"start:smtp": "node ./tools/start.mjs --smtp",
"dev": "npm start",
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
@@ -76,7 +77,7 @@
"@aws-sdk/s3-request-presigner": "^3.1028.0",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.13",
"libphonenumber-js": "1.13.14",
"miniflare": "^4.20260617.1",
"open": "^10.1.0"
},
@@ -96,6 +97,6 @@
"workerd": true
},
"overrides": {
"undici": "^7.29.0"
"undici": "^7.30.0"
}
}
@@ -184,7 +184,8 @@ describe('compareMigrationFilenames', () => {
//
// `read()` normally goes to the replica batcher; when the replica side is
// degraded (batcher load-shed or a transient connection error) and a real
// replica is configured, the read retries once on the primary batcher.
// replica is configured, the read retries once on the primary read batcher.
// The write batcher is left unset, so a failover that reached it would throw.
type Batcher = { execute: ReturnType<typeof vi.fn> };
@@ -200,7 +201,7 @@ const makeClient = (opts: {
// inject the batchers directly. Configuration enum: SINGLE=0, REPLICA=1.
Object.assign(client as unknown as Record<string, unknown>, {
dbReplica: opts.replica,
db: opts.primary,
dbPrimaryRead: opts.primary,
configuration: opts.multiNode === false ? 0 : 1,
});
return client;
@@ -284,6 +285,10 @@ const startClient = async (
// must answer with one row-set per statement plus one for the SELECT 1.
const rowSets = (...sets: unknown[][]) => [[...sets, [{ 1: 1 }]], undefined];
// Writes go out one statement at a time, so the driver answers with just the
// result header.
const writeHeader = (header: Record<string, unknown>) => [header, undefined];
beforeEach(() => {
createdPools.length = 0;
createPoolMock.mockClear();
@@ -303,6 +308,9 @@ describe('MySQLDatabaseClient — pool construction', () => {
expect(createdPools[0].poolConfig).toEqual({
maxPreparedStatements: 900,
connectionLimit: 30,
maxIdle: 29,
idleTimeout: 60_000,
gracefulEnd: true,
enableKeepAlive: true,
host: 'db.internal',
port: 3307,
@@ -313,6 +321,32 @@ describe('MySQLDatabaseClient — pool construction', () => {
});
});
// mysql2 skips its idle reaper unless maxIdle < connectionLimit, and an
// unreaped connection is closed by the server under the pool's feet.
it('keeps the idle reaper on when a pool sets its own connection limit', async () => {
await startClient({
replica: { host: 'replica.internal', connectionLimit: 10 },
});
expect(createdPools[1].poolConfig).toMatchObject({
connectionLimit: 10,
maxIdle: 9,
idleTimeout: 60_000,
});
});
// mysql2 reads 0 as unlimited; a maxIdle of -1 would crash its reaper.
it('leaves maxIdle to mysql2 for an unlimited pool', async () => {
await startClient({
replica: { host: 'replica.internal', connectionLimit: 0 },
});
expect(createdPools[1].poolConfig).toMatchObject({
connectionLimit: 0,
});
expect(createdPools[1].poolConfig).not.toHaveProperty('maxIdle');
});
it('falls back to loopback defaults when the endpoint is unspecified', async () => {
await startClient();
@@ -401,10 +435,30 @@ describe('MySQLDatabaseClient — query interface', () => {
]);
});
it('sends a write on its own, outside a transaction', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () => writeHeader({ affectedRows: 1 });
await client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
'ada',
1,
]);
expect(pool.calls).toEqual([
{
sql: 'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]);
expect(pool.connections[0].beginTransaction).not.toHaveBeenCalled();
expect(pool.connections[0].commit).not.toHaveBeenCalled();
});
it('maps the mysql result header onto the shared write result', async () => {
const client = await startClient();
createdPools[0].respond = () =>
rowSets({ insertId: 42, affectedRows: 2 } as unknown as unknown[]);
writeHeader({ insertId: 42, affectedRows: 2 });
await expect(
client.write('UPDATE `user` SET `username` = ? WHERE `id` = ?', [
@@ -420,7 +474,7 @@ describe('MySQLDatabaseClient — query interface', () => {
it('reports no rows affected when the header omits the counters', async () => {
const client = await startClient();
createdPools[0].respond = () => rowSets({} as unknown as unknown[]);
createdPools[0].respond = () => writeHeader({});
await expect(client.write('DELETE FROM `user`')).resolves.toEqual({
insertId: 0,
@@ -432,17 +486,14 @@ describe('MySQLDatabaseClient — query interface', () => {
it('builds an INSERT with quoted identifiers and positional params', async () => {
const client = await startClient();
const pool = createdPools[0];
pool.respond = () =>
rowSets({ insertId: 7, affectedRows: 1 } as unknown as unknown[]);
pool.respond = () => writeHeader({ insertId: 7, affectedRows: 1 });
await expect(
client.insert('user', { username: 'ada', email: null }),
).resolves.toMatchObject({ insertId: 7 });
expect(pool.calls[0]).toEqual({
sql:
'INSERT INTO `user` (`username`, `email`) VALUES (?, ?); ' +
'SELECT 1',
sql: 'INSERT INTO `user` (`username`, `email`) VALUES (?, ?)',
values: ['ada', null],
});
});
@@ -693,31 +744,19 @@ describe('MySQLDatabaseClient — pool lifecycle', () => {
expect(createdPools[1].ended).toBe(true);
});
it('closes the pools once the drain window elapses', async () => {
it('leaves the pools open through prepare-shutdown; only onServerShutdown closes them', async () => {
vi.useFakeTimers();
try {
const client = await startClient();
await client.onServerPrepareShutdown();
expect(createdPools[0].ended).toBe(false);
// No timer closes the pools on its own — the layers above are
// still draining through them for as long as the process drains.
await vi.advanceTimersByTimeAsync(60_000);
expect(createdPools[0].ended).toBe(true);
} finally {
vi.useRealTimers();
}
});
it('cancels the drain timer when shutdown arrives first', async () => {
vi.useFakeTimers();
try {
const client = await startClient();
await client.onServerPrepareShutdown();
// A second prepare is a no-op — only one drain timer may exist.
await client.onServerPrepareShutdown();
expect(vi.getTimerCount()).toBe(1);
expect(createdPools[0].ended).toBe(false);
await client.onServerShutdown();
expect(vi.getTimerCount()).toBe(0);
expect(createdPools[0].ended).toBe(true);
} finally {
vi.useRealTimers();
@@ -30,6 +30,8 @@ import { compareMigrationFilenames } from './migrationFilenames.js';
import type { IConfig } from '../../types';
const DEFAULT_SELECT_TIMEOUT_MS = 30_000;
const DEFAULT_CONNECTION_LIMIT = 30;
const IDLE_TIMEOUT_MS = 60_000;
const replicaFailoverCounter = metrics
.getMeter('puter-backend')
@@ -54,11 +56,10 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
private replicaPool!: Pool;
private db!: SQLBatcher;
private dbReplica!: SQLBatcher;
/** Primary pool, SELECT-only: same rows as `db`, without the transaction. */
/** Primary pool, SELECT-only: same rows as `db`, coalesced like the replica. */
private dbPrimaryRead!: SQLBatcher;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(config: IConfig) {
super(config);
@@ -101,29 +102,12 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
// Allow in-flight queries to drain before closing pools
const drainMs = 60_000;
console.log(
`[mysql] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools('drain').catch((e) =>
console.error('[mysql] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
// Blocks reinitPrimary/reinitReplica from here on. Pools stay open;
// onServerShutdown closes them after the layers above have drained.
console.log('[mysql] entering drain mode');
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools('shutdown');
}
@@ -154,7 +138,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
throw error;
}
replicaFailoverCounter.add(1);
result = await this.db.execute(query, params);
result = await this.dbPrimaryRead.execute(query, params);
}
if (!result) return [];
return (result[0] as Record<string, unknown>[]) ?? [];
@@ -318,9 +302,19 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
// ------------------------------------------------------------------
private createPool(poolConfig: PoolConfig): Pool {
const connectionLimit =
poolConfig.connectionLimit ?? DEFAULT_CONNECTION_LIMIT;
const pool = createPool({
maxPreparedStatements: 900,
connectionLimit: 30,
connectionLimit,
// mysql2 only reaps idle connections when maxIdle < connectionLimit.
// Unreaped, they sit until the server's wait_timeout closes them,
// and the next statement sent on one fails. 0 means unlimited,
// and a negative maxIdle would crash the reaper.
...(connectionLimit > 1 && { maxIdle: connectionLimit - 1 }),
idleTimeout: IDLE_TIMEOUT_MS,
// Reaped connections say goodbye instead of dropping the socket.
gracefulEnd: true,
enableKeepAlive: true,
...poolConfig,
multipleStatements: true,
@@ -348,21 +342,20 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
return pool;
}
/**
* One write per flush. Coalesced writes share a transaction, so unrelated
* callers hold each other's row locks until COMMIT and deadlock; a lone
* statement needs no BEGIN/COMMIT round trips.
*/
private createPrimaryBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 30,
maxBatchSize: 5,
maxBatchSize: 1,
poolLabel: 'primary',
acquireTimeoutMs: this.config.database?.acquireTimeoutMs,
});
}
/**
* Reads that must see the primary still only read, so they skip the
* batcher's transaction wrapper. Worth a separate batcher because BEGIN and
* COMMIT are round trips: against a primary in another region they cost
* more than the query.
*/
/** Reads that must see the primary, still coalesced. */
private createPrimaryReadBatcher(pool: Pool): SQLBatcher {
return new SQLBatcher(pool, {
maxTimeInQueue: 30,
@@ -18,7 +18,7 @@
*/
import type { FieldDef, QueryResult } from 'pg';
import { describe, expect, it } from 'vitest';
import { describe, expect, it, vi } from 'vitest';
import type { IConfig } from '../../types';
import { DatabaseClientFactory } from './index.js';
import {
@@ -357,4 +357,33 @@ describe('PostgresDatabaseClient', () => {
]);
expect(conn.released).toBe(true);
});
it('leaves the pool open through prepare-shutdown; only onServerShutdown closes it', async () => {
vi.useFakeTimers();
try {
const pool = new RecordingPool();
let ended = false;
pool.end = async () => {
ended = true;
};
const client = new PostgresDatabaseClient(
postgresConfig(),
() => pool,
);
await client.onServerStart();
await client.onServerPrepareShutdown();
expect(ended).toBe(false);
// No timer closes the pool on its own — the layers above are
// still draining through it for as long as the process drains.
await vi.advanceTimersByTimeAsync(60_000);
expect(ended).toBe(false);
await client.onServerShutdown();
expect(ended).toBe(true);
} finally {
vi.useRealTimers();
}
});
});
@@ -138,7 +138,6 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
private replicaPool!: PostgresPool;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(
config: IConfig,
@@ -172,28 +171,12 @@ export class PostgresDatabaseClient extends AbstractDatabaseClient {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
const drainMs = 60_000;
console.log(
`[postgres] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools().catch((e) =>
console.error('[postgres] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
// Pools stay open; onServerShutdown closes them after the layers above
// have drained through them.
console.log('[postgres] entering drain mode');
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools();
}
+58 -24
View File
@@ -258,16 +258,11 @@ export class SQLBatcher {
}
if (batch.length === 0) return;
const query = `${batch.map((b) => b.sql.replace(/;+\s*$/, '')).join(';')}; SELECT 1`; // SELECT 1 forces mysql2 to return array
const values = batch.map((b) => b.values ?? []).flat();
let connection;
try {
connection = await this.#acquireConnection();
} catch (error) {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1, this.#metricAttrs);
this.#recordFlushFailure();
console.warn(
'SQLBatcher could not acquire connection for flush:',
error,
@@ -278,6 +273,14 @@ export class SQLBatcher {
return;
}
if (batch.length === 1 && !this.readOnly) {
await this.#flushLoneWrite(batch[0], connection);
return;
}
const query = `${batch.map((b) => b.sql.replace(/;+\s*$/, '')).join(';')}; SELECT 1`; // SELECT 1 forces mysql2 to return array
const values = batch.map((b) => b.values ?? []).flat();
// Run the coalesced multi-statement inside an explicit transaction so
// a single bad statement (e.g. a duplicate-key INSERT) rolls back the
// whole batch atomically, leaving us free to re-run each item
@@ -333,7 +336,7 @@ export class SQLBatcher {
async () => {
while (cursor < batch.length) {
const i = cursor++;
settled[i] = await this.#runFallbackItem(batch[i]);
settled[i] = await this.#runItem(batch[i]);
}
},
);
@@ -367,7 +370,34 @@ export class SQLBatcher {
}
}
// Run one fallback item, retrying transient failures with backoff.
// A single statement is atomic under autocommit, so it skips BEGIN/COMMIT
// and their round trips. With no transaction to roll back, a failure is
// not re-run wholesale; only the per-item retry policy applies.
async #flushLoneWrite(item, connection) {
const r = await this.#runItem(item, connection);
if (r.ok) {
this.#consecutiveFailures = 0;
item.resolve(r.value);
return;
}
// An error the server answered with (duplicate key, bad SQL) means it
// is up; only failing to reach it counts toward the breaker.
if (isRetriableError(r.error)) {
this.#recordFlushFailure();
} else {
this.#consecutiveFailures = 0;
flushFailureCounter.add(1, this.#metricAttrs);
}
item.reject(r.error);
}
#recordFlushFailure() {
this.#consecutiveFailures++;
this.#lastFailureAt = Date.now();
flushFailureCounter.add(1, this.#metricAttrs);
}
// Run one item on its own, retrying transient failures with backoff.
// A read-only batcher may retry anything transient; a batcher that
// carries writes only retries failures where the statement provably
// did not apply — either it never reached the server, or the server
@@ -378,33 +408,37 @@ export class SQLBatcher {
// than surfacing: an item is a single statement, so a deadlock victim has
// been fully undone, and the caller sees an unhandled 500 for what the
// database is telling us to just run again.
async #runFallbackItem(b) {
async #runItem(b, connection) {
let attempt = 0;
while (true) {
let connection;
try {
connection = await this.#acquireConnection();
} catch (error) {
return { ok: false, error };
if (!connection) {
try {
connection = await this.#acquireConnection();
} catch (error) {
return { ok: false, error };
}
}
let error;
try {
return {
ok: true,
value: await connection.query(b.sql, b.values ?? []),
};
} catch (error) {
const canRetry = this.readOnly
? isRetriableError(error) || isRolledBackError(error)
: isNeverSentError(error) || isRolledBackError(error);
if (!canRetry || attempt >= ITEM_RETRY_ATTEMPTS) {
return { ok: false, error };
}
attempt++;
fallbackItemRetriesCounter.add(1, this.#metricAttrs);
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
} catch (e) {
error = e;
} finally {
connection.release();
connection = undefined;
}
const canRetry = this.readOnly
? isRetriableError(error) || isRolledBackError(error)
: isNeverSentError(error) || isRolledBackError(error);
if (!canRetry || attempt >= ITEM_RETRY_ATTEMPTS) {
return { ok: false, error };
}
attempt++;
fallbackItemRetriesCounter.add(1, this.#metricAttrs);
await sleep(RETRY_BASE_BACKOFF_MS * attempt);
}
}
}
@@ -126,7 +126,7 @@ describe('SQLBatcher', () => {
});
it('drops the oldest item with reason queueOverflow at the high-water mark', async () => {
const conn = makeConnection(happyBatch);
const conn = makeConnection(() => [[{ n: 0 }], undefined]);
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxTimeInQueue: 5,
@@ -277,6 +277,65 @@ describe('SQLBatcher', () => {
);
});
it('sends a lone write as-is, outside a transaction', async () => {
const conn = makeConnection((sql) => {
if (isBatchQuery(sql)) throw new Error('unexpected batch query');
return [{ affectedRows: 1 }, undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxBatchSize: 1 });
const result = await batcher.query('UPDATE t SET x = ?', [1]);
expect(result[0]).toEqual({ affectedRows: 1 });
expect(conn.query).toHaveBeenCalledTimes(1);
expect(conn.query).toHaveBeenCalledWith('UPDATE t SET x = ?', [1]);
expect(conn.beginTransaction).not.toHaveBeenCalled();
expect(conn.commit).not.toHaveBeenCalled();
expect(conn.release).toHaveBeenCalledTimes(1);
});
it('runs a lone write the server rejects once, without tripping the breaker', async () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
const conn = makeConnection((sql) => {
if (sql === 'INSERT dup') throw makeError('ER_DUP_ENTRY');
return [{ affectedRows: 1 }, undefined];
});
const { pool } = makePool(conn);
const batcher = new SQLBatcher(pool, {
maxBatchSize: 1,
failureThreshold: 1,
cooldownMs: 60_000,
});
await expect(batcher.query('INSERT dup', [])).rejects.toMatchObject({
code: 'ER_DUP_ENTRY',
});
expect(conn.query).toHaveBeenCalledTimes(1);
expect(conn.rollback).not.toHaveBeenCalled();
expect(warnSpy).not.toHaveBeenCalled();
// The server answered, so it is up: the next write goes through.
await expect(batcher.query('INSERT fine', [])).resolves.toBeTruthy();
warnSpy.mockRestore();
});
it('retries a lone write on a connection the server closed for inactivity', async () => {
let attempts = 0;
const conn = makeConnection(() => {
attempts++;
if (attempts === 1) {
throw makeError('ER_CLIENT_INTERACTION_TIMEOUT');
}
return [{ affectedRows: 1 }, undefined];
});
const { pool, getConnection } = makePool(conn);
const batcher = new SQLBatcher(pool, { maxBatchSize: 1 });
const result = await batcher.query('INSERT x', []);
expect(result[0]).toEqual({ affectedRows: 1 });
expect(attempts).toBe(2);
expect(getConnection).toHaveBeenCalledTimes(2);
});
it('never retries deterministic row-level errors and does not escalate the breaker', async () => {
let fallbackAttempts = 0;
const conn = makeConnection((sql) => {
@@ -27,7 +27,7 @@ import { DatabaseClientFactory } from './index.js';
import { SqliteDatabaseClient } from './SqliteDatabaseClient.js';
/** Highest schema version the migration table can reach. */
const CURRENT_SCHEMA_VERSION = 81;
const CURRENT_SCHEMA_VERSION = 86;
/**
* These suites migrate real files on disk. Idle they finish in well under a
@@ -276,6 +276,68 @@ describe('SqliteDatabaseClient — boot and migrations', { timeout: DISK_MIGRATI
]);
});
it('deletes the subdomain rows an app owns when the app is deleted', async () => {
const [user] = (await client.read(
'SELECT MIN(`id`) AS id FROM `user`',
)) as { id: number }[];
await client.write(
'INSERT INTO `apps` (`uid`, `owner_user_id`, `name`, `title`, `index_url`) ' +
'VALUES (?, ?, ?, ?, ?)',
[
'app-cascade-test',
user.id,
'cascade-test',
'cascade-test',
'https://cascade.test/',
],
);
const [app] = (await client.read(
'SELECT `id` FROM `apps` WHERE `uid` = ?',
['app-cascade-test'],
)) as { id: number }[];
const insertSubdomain = (
uuid: string,
name: string,
appOwner: number | null,
) =>
client.write(
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`, `app_owner`) ' +
'VALUES (?, ?, ?, ?)',
[uuid, name, user.id, appOwner],
);
await insertSubdomain('sd-cascade-owned', 'cascade-owned', app.id);
await insertSubdomain('sd-cascade-unowned', 'cascade-unowned', null);
await client.write('DELETE FROM `apps` WHERE `id` = ?', [app.id]);
await expect(
client.read(
'SELECT `subdomain` FROM `subdomains` WHERE `subdomain` LIKE ? ORDER BY `id`',
['cascade-%'],
),
).resolves.toEqual([{ subdomain: 'cascade-unowned' }]);
});
it('keeps every subdomains column through the 0086 rebuild', async () => {
const columns = (await client.read(
"SELECT `name` FROM pragma_table_info('subdomains') ORDER BY `cid`",
)) as { name: string }[];
expect(columns.map((c) => c.name)).toEqual([
'id',
'uuid',
'subdomain',
'user_id',
'root_dir_id',
'associated_app_id',
'ts',
'app_owner',
'protected',
'domain',
'database_id',
'preamble_version',
]);
});
it('constrains team shares that the user-holder index cannot', async () => {
const [user] = (await client.read(
'SELECT MIN(`id`) AS id FROM `user`',
@@ -115,6 +115,11 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[78, ['0083_team-directory.sql']],
[79, ['0084_share-anyone-with-link.sql']],
[80, ['0085_event-subscriptions-include-value.sql']],
[81, ['0086_subdomains-app-owner-cascade.sql']],
[82, ['0087_feedback-attachments.sql']],
[83, ['0088_apps-index-url.sql']],
[84, ['0089_team-require-2fa.sql']],
[85, ['0090_user-app-permission-index.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
@@ -0,0 +1,75 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0086: `subdomains.app_owner` cascades on app
-- delete instead of nulling. The existing constraint is found by column
-- rather than by name, since an older database need not carry the name
-- mysql_mig_1 declares. Guarded procedure as mysql_mig_34: a replay sees
-- DELETE_RULE already CASCADE and does nothing.
DROP PROCEDURE IF EXISTS _puter_subdomains_app_owner_cascade;
DELIMITER //
CREATE PROCEDURE _puter_subdomains_app_owner_cascade()
BEGIN
DECLARE fk_name VARCHAR(64) DEFAULT NULL;
-- A subquery, not SELECT ... INTO: no matching row yields NULL quietly.
SET fk_name = (
SELECT rc.CONSTRAINT_NAME
FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS rc
JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu
ON kcu.CONSTRAINT_SCHEMA = rc.CONSTRAINT_SCHEMA
AND kcu.CONSTRAINT_NAME = rc.CONSTRAINT_NAME
AND kcu.TABLE_NAME = rc.TABLE_NAME
WHERE rc.CONSTRAINT_SCHEMA = DATABASE()
AND rc.TABLE_NAME = 'subdomains'
AND rc.REFERENCED_TABLE_NAME = 'apps'
AND kcu.COLUMN_NAME = 'app_owner'
AND rc.DELETE_RULE <> 'CASCADE'
LIMIT 1
);
IF fk_name IS NOT NULL THEN
SET @s := CONCAT('ALTER TABLE `subdomains` DROP FOREIGN KEY `', fk_name, '`');
PREPARE stmt FROM @s;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
IF NOT EXISTS (
SELECT 1
FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS rc
JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu
ON kcu.CONSTRAINT_SCHEMA = rc.CONSTRAINT_SCHEMA
AND kcu.CONSTRAINT_NAME = rc.CONSTRAINT_NAME
AND kcu.TABLE_NAME = rc.TABLE_NAME
WHERE rc.CONSTRAINT_SCHEMA = DATABASE()
AND rc.TABLE_NAME = 'subdomains'
AND rc.REFERENCED_TABLE_NAME = 'apps'
AND kcu.COLUMN_NAME = 'app_owner'
AND rc.DELETE_RULE = 'CASCADE'
) THEN
ALTER TABLE `subdomains` ADD CONSTRAINT `fk_subdomains_app_owner`
FOREIGN KEY (`app_owner`) REFERENCES `apps` (`id`)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END//
DELIMITER ;
CALL _puter_subdomains_app_owner_cascade();
DROP PROCEDURE IF EXISTS _puter_subdomains_app_owner_cascade;
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0087: Contact Us attachment metadata, a JSON array
-- of `{name, type, size}` (NULL when none). Guarded by a throwaway procedure,
-- as in mysql_mig_21.
DROP PROCEDURE IF EXISTS _puter_add_feedback_attachments;
DELIMITER //
CREATE PROCEDURE _puter_add_feedback_attachments()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'feedback'
AND COLUMN_NAME = 'attachments'
) THEN
ALTER TABLE `feedback` ADD COLUMN `attachments` text DEFAULT NULL;
END IF;
END//
DELIMITER ;
CALL _puter_add_feedback_attachments();
DROP PROCEDURE IF EXISTS _puter_add_feedback_attachments;
@@ -0,0 +1,44 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0088: index `apps.index_url` for origin lookups.
-- TEXT needs a prefix; 191 characters keeps the key under 767 bytes in
-- utf8mb4 whatever the table's row format, and URLs diverge well before it.
--
-- Idempotent: the guarded procedure, as mig_36. The 1061 handler covers two
-- nodes booting at once, where the loser finds the index already built.
DROP PROCEDURE IF EXISTS _puter_add_apps_index_url_index;
DELIMITER //
CREATE PROCEDURE _puter_add_apps_index_url_index()
BEGIN
DECLARE CONTINUE HANDLER FOR 1061 BEGIN END;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'apps'
AND INDEX_NAME = 'idx_apps_index_url'
) THEN
ALTER TABLE `apps`
ADD INDEX `idx_apps_index_url` (`index_url`(191)),
ALGORITHM=INPLACE, LOCK=NONE;
END IF;
END//
DELIMITER ;
CALL _puter_add_apps_index_url_index();
DROP PROCEDURE IF EXISTS _puter_add_apps_index_url_index;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0089: a team may require 2FA of the accounts it
-- provisioned. Added through `_puter_add_col` as mysql_mig_32 does, so a
-- replay finds the column and does nothing.
CALL _puter_add_col('group', 'require_2fa', '`require_2fa` tinyint(1) NOT NULL DEFAULT 0');
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0090. Guarded like mysql_mig_22, because there is
-- no per-file applied-state tracking and a replay must do nothing.
DROP PROCEDURE IF EXISTS _puter_add_user_app_permission_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_app_permission_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user_to_app_permissions'
AND INDEX_NAME = 'idx_user_to_app_permissions_permission'
) THEN
ALTER TABLE `user_to_app_permissions`
ADD INDEX `idx_user_to_app_permissions_permission` (`permission`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_app_permission_index();
DROP PROCEDURE IF EXISTS _puter_add_user_app_permission_index;
@@ -0,0 +1,59 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0086: `subdomains.app_owner` cascades on app
-- delete instead of nulling. The existing constraint is found by column, not
-- by name. Idempotent: there is no per-file applied-state tracking, and a
-- replay sees the cascade already in place and does nothing.
DO $$
DECLARE
fk_name text;
BEGIN
SELECT c.conname INTO fk_name
FROM pg_constraint c
JOIN pg_attribute a
ON a.attrelid = c.conrelid
AND a.attnum = ANY (c.conkey)
WHERE c.conrelid = 'subdomains'::regclass
AND c.contype = 'f'
AND c.confrelid = 'apps'::regclass
AND a.attname = 'app_owner'
AND c.confdeltype <> 'c'
LIMIT 1;
IF fk_name IS NOT NULL THEN
EXECUTE format('ALTER TABLE subdomains DROP CONSTRAINT %I', fk_name);
END IF;
IF NOT EXISTS (
SELECT 1
FROM pg_constraint c
JOIN pg_attribute a
ON a.attrelid = c.conrelid
AND a.attnum = ANY (c.conkey)
WHERE c.conrelid = 'subdomains'::regclass
AND c.contype = 'f'
AND c.confrelid = 'apps'::regclass
AND a.attname = 'app_owner'
AND c.confdeltype = 'c'
) THEN
ALTER TABLE subdomains ADD CONSTRAINT subdomains_app_owner_fkey
FOREIGN KEY (app_owner) REFERENCES apps (id)
ON DELETE CASCADE ON UPDATE CASCADE;
END IF;
END $$;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0087: Contact Us attachment metadata, a JSON array
-- of `{name, type, size}` (NULL when none).
ALTER TABLE feedback ADD COLUMN IF NOT EXISTS attachments text DEFAULT NULL;
@@ -0,0 +1,24 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0088: index `apps.index_url` for origin lookups.
-- Hash rather than btree: `index_url` allows 3000 characters, past btree's
-- ~2.7 KB row limit, and every lookup is equality (`=` / `IN`).
--
-- Idempotent via IF NOT EXISTS.
CREATE INDEX IF NOT EXISTS idx_apps_index_url ON apps USING hash (index_url);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0089: a team may require 2FA of the accounts it
-- provisioned. Idempotent -- there is no per-file applied-state tracking, so
-- a replay must find the column already present and do nothing.
ALTER TABLE "group" ADD COLUMN IF NOT EXISTS require_2fa smallint NOT NULL DEFAULT 0;
@@ -0,0 +1,24 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0090, as postgres_mig_11 did for user_to_user.
-- text_pattern_ops is what makes the left-anchored LIKE a range scan under a
-- non-C collation -- the plain index dev_to_app already has does not.
CREATE INDEX IF NOT EXISTS idx_user_to_app_permissions_permission
ON user_to_app_permissions (permission text_pattern_ops);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_permission_pattern
ON dev_to_app_permissions (permission text_pattern_ops);
@@ -0,0 +1,52 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- A deleted app takes the subdomain rows it owns (its hosted sites and its
-- workers) with it. `SET NULL` left them behind as unowned rows, and an
-- unowned worker row is redeployed with the account's own credential.
-- SQLite cannot alter a constraint, so the table is rebuilt as 0043 does.
PRAGMA foreign_keys = OFF;
CREATE TABLE `subdomains_new` (
`id` INTEGER PRIMARY KEY,
`uuid` varchar(40) DEFAULT NULL,
`subdomain` varchar(64) NOT NULL,
`user_id` int(10) NOT NULL,
`root_dir_id` int(10) DEFAULT NULL,
`associated_app_id` int(10) DEFAULT NULL,
`ts` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`app_owner` int(10) DEFAULT NULL,
`protected` tinyint(1) DEFAULT '0',
`domain` varchar(256) DEFAULT NULL,
`database_id` varchar(40) DEFAULT NULL,
`preamble_version` varchar(64) DEFAULT NULL,
FOREIGN KEY (`app_owner`) REFERENCES `apps` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
);
INSERT INTO `subdomains_new`
(`id`, `uuid`, `subdomain`, `user_id`, `root_dir_id`, `associated_app_id`, `ts`,
`app_owner`, `protected`, `domain`, `database_id`, `preamble_version`)
SELECT
`id`, `uuid`, `subdomain`, `user_id`, `root_dir_id`, `associated_app_id`, `ts`,
`app_owner`, `protected`, `domain`, `database_id`, `preamble_version`
FROM `subdomains`;
DROP TABLE `subdomains`;
ALTER TABLE `subdomains_new` RENAME TO `subdomains`;
PRAGMA foreign_keys = ON;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Contact Us attachment metadata: a JSON array of `{name, type, size}`, NULL
-- when none. The files themselves go out with the support email.
ALTER TABLE `feedback` ADD COLUMN "attachments" TEXT DEFAULT NULL;
@@ -0,0 +1,22 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Resolving an origin to its app looks `apps` up by `index_url` on every
-- origin-based app token request (`AuthService#findCanonicalAppUidForOrigin`),
-- plus `AppStore.existsByIndexUrl`. With no index each lookup scans the table.
CREATE INDEX IF NOT EXISTS `idx_apps_index_url` ON `apps` (`index_url`);
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Whether every account this team provisioned must hold 2FA before it can
-- use Puter. Off until the owner turns it on, and the owner must hold 2FA
-- themselves first, or enabling it would lock out the account that did so.
ALTER TABLE `group` ADD COLUMN `require_2fa` INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Withdrawing an app's cross-app data grants looks them up by permission text,
-- and no existing index on either table leads with `permission`. Same fix as
-- `idx_user_to_user_permissions_permission` in 0067.
CREATE INDEX IF NOT EXISTS `idx_user_to_app_permissions_permission`
ON `user_to_app_permissions` (`permission`);
-- The sweep reads both tables; MySQL and Postgres already index this one.
CREATE INDEX IF NOT EXISTS `idx_dev_to_app_permissions_permission`
ON `dev_to_app_permissions` (`permission`);
@@ -32,6 +32,7 @@ const RETRIABLE_ERROR_CODES = new Set([
'ENETUNREACH',
'EAI_AGAIN',
POOL_ACQUIRE_TIMEOUT,
'ER_CLIENT_INTERACTION_TIMEOUT',
]);
const RETRIABLE_ERROR_MESSAGES = [
@@ -52,6 +53,9 @@ const NEVER_SENT_ERROR_CODES = new Set([
'ENETUNREACH',
'EAI_AGAIN',
POOL_ACQUIRE_TIMEOUT,
// The server closed the pooled connection for inactivity before the
// statement arrived, so nothing ran.
'ER_CLIENT_INTERACTION_TIMEOUT',
]);
/**
@@ -142,6 +142,42 @@ describe('DDBClient — item operations', () => {
expect(result.Item).toBeUndefined();
});
it('deletes only while its condition holds', async () => {
await client.put(TABLE, { pk: 'u6', sk: 'guarded', v: 1 });
await expect(
client.del(
TABLE,
{ pk: 'u6', sk: 'guarded' },
{
condition: {
expression: '#v = :wrong',
names: { '#v': 'v' },
values: { ':wrong': 2 },
},
},
),
).rejects.toMatchObject({ name: 'ConditionalCheckFailedException' });
expect(
(await client.get(TABLE, { pk: 'u6', sk: 'guarded' })).Item,
).toBeDefined();
await client.del(
TABLE,
{ pk: 'u6', sk: 'guarded' },
{
condition: {
expression: '#v = :right',
names: { '#v': 'v' },
values: { ':right': 1 },
},
},
);
expect(
(await client.get(TABLE, { pk: 'u6', sk: 'guarded' })).Item,
).toBeUndefined();
});
it('fetches keys from several tables in one batch', async () => {
await client.put(TABLE, { pk: 'b1', sk: 'a', v: 1 });
await client.put(TABLE, { pk: 'b1', sk: 'b', v: 2 });
@@ -364,6 +400,59 @@ describe('DDBClient — expired item sweep', () => {
expect((await client.get(table, { pk: 'fresh' })).Item).toBeDefined();
expect((await client.get(table, { pk: 'eternal' })).Item).toBeDefined();
});
it('keeps an item whose ttl is 0, which never expires', async () => {
const client = new DDBClient(localConfig());
const table = 'ttl-zero-items';
const schema = {
TableName: table,
KeySchema: [{ AttributeName: 'pk', KeyType: 'HASH' as const }],
AttributeDefinitions: [
{ AttributeName: 'pk', AttributeType: 'S' as const },
],
ProvisionedThroughput: {
ReadCapacityUnits: 5,
WriteCapacityUnits: 5,
},
};
await client.createTableIfNotExists(schema, 'expireAt');
await awaitTable(client, table, { pk: 'ready' });
await client.put(table, { pk: 'zero', expireAt: 0 });
// Second call finds the table in use, so the sweep runs.
await client.createTableIfNotExists(schema, 'expireAt');
expect((await client.get(table, { pk: 'zero' })).Item).toBeDefined();
});
it('keeps an item whose ttl is not a number, which the sweep can never purge', async () => {
const client = new DDBClient(localConfig());
const table = 'ttl-nan-items';
const schema = {
TableName: table,
KeySchema: [{ AttributeName: 'pk', KeyType: 'HASH' as const }],
AttributeDefinitions: [
{ AttributeName: 'pk', AttributeType: 'S' as const },
],
ProvisionedThroughput: {
ReadCapacityUnits: 5,
WriteCapacityUnits: 5,
},
};
await client.createTableIfNotExists(schema, 'expireAt');
await awaitTable(client, table, { pk: 'ready' });
const past = Math.floor(Date.now() / 1000) - 60;
await client.put(table, { pk: 'stringTtl', expireAt: String(past) });
// Second call finds the table in use, so the sweep runs.
await client.createTableIfNotExists(schema, 'expireAt');
expect(
(await client.get(table, { pk: 'stringTtl' })).Item,
).toBeDefined();
});
});
describe('DDBClient — expired item sweep on an empty table', () => {
@@ -559,3 +648,118 @@ describe('DDBClient — unprocessed batch items', () => {
).resolves.toEqual({ ConsumedCapacity: [] });
});
});
// UnprocessedKeys are batchGet's analogue of UnprocessedItems above, driven
// against the same kind of wire-format stub.
describe('DDBClient — unprocessed batch keys', () => {
let server: Server;
let endpoint: string;
let responses: unknown[];
let requestCount = 0;
beforeAll(async () => {
server = createServer((req, res) => {
let body = '';
req.on('data', (chunk) => {
body += chunk;
});
req.on('end', () => {
const next = responses.shift() ?? {};
requestCount += 1;
res.writeHead(200, {
'content-type': 'application/x-amz-json-1.0',
});
res.end(JSON.stringify(next));
});
});
await new Promise<void>((resolve) =>
server.listen(0, '127.0.0.1', resolve),
);
endpoint = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
});
afterAll(async () => {
await new Promise<void>((resolve) => server.close(() => resolve()));
});
const stubClient = () =>
new DDBClient({
port: 0,
extensions: [],
dynamo: {
aws: { access_key: 'a', secret_key: 'b', region: 'us-west-2' },
endpoint,
},
} as unknown as IConfig);
const unprocessed = (pk: string) => ({
UnprocessedKeys: {
[TABLE]: { Keys: [{ pk: { S: pk } }] },
},
ConsumedCapacity: [{ TableName: TABLE, CapacityUnits: 1 }],
});
it('retries leftover keys and merges responses and capacity across attempts', async () => {
requestCount = 0;
responses = [
unprocessed('retry-me'),
{
Responses: {
[TABLE]: [{ pk: { S: 'retry-me' }, v: { N: '1' } }],
},
UnprocessedKeys: {},
ConsumedCapacity: [{ TableName: TABLE, CapacityUnits: 2 }],
},
];
const result = await stubClient().batchGet([
{ table: TABLE, items: { pk: 'retry-me' } },
]);
expect(requestCount).toBe(2);
expect(result.Responses?.[TABLE]).toEqual([{ pk: 'retry-me', v: 1 }]);
expect(result.ConsumedCapacity).toEqual([
{ TableName: TABLE, CapacityUnits: 3 },
]);
});
it('gives up with a bad-request error when keys never drain', async () => {
requestCount = 0;
responses = Array.from({ length: 12 }, () => unprocessed('stuck'));
await expect(
stubClient().batchGet([{ table: TABLE, items: { pk: 'stuck' } }]),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
message: 'Failed to batch get all items from DynamoDB',
});
// One initial attempt plus the full retry budget.
expect(requestCount).toBe(9);
}, 30_000);
it('tolerates a response that reports no items or capacity at all', async () => {
requestCount = 0;
responses = [{ Responses: {}, UnprocessedKeys: {} }];
await expect(
stubClient().batchGet([{ table: TABLE, items: { pk: 'quiet' } }]),
).resolves.toEqual({ Responses: {}, ConsumedCapacity: [] });
expect(requestCount).toBe(1);
});
it('ignores capacity entries with no table name', async () => {
requestCount = 0;
responses = [
{
Responses: {},
UnprocessedKeys: {},
ConsumedCapacity: [{ CapacityUnits: 7 }],
},
];
await expect(
stubClient().batchGet([{ table: TABLE, items: { pk: 'anon' } }]),
).resolves.toEqual({ Responses: {}, ConsumedCapacity: [] });
});
});
+108 -22
View File
@@ -56,6 +56,8 @@ const localDynaliteEndpointPromises = new Map<string, Promise<string>>();
const MAX_BATCH_WRITE_ITEMS = 25;
const MAX_BATCH_WRITE_RETRIES = 8;
const BATCH_WRITE_RETRY_BASE_MS = 25;
const MAX_BATCH_GET_RETRIES = 8;
const BATCH_GET_RETRY_BASE_MS = 25;
// In-memory mode gives each DDBClient its own unique key, which keys
// into a fresh dynalite server. This is what test parallelism needs:
@@ -234,6 +236,9 @@ export class DDBClient extends PuterClient {
);
}
// UnprocessedKeys retried with capped exponential backoff, same posture as
// #batchWrite's UnprocessedItems: a batch this size can be throttled or
// partially served, and the caller shouldn't have to know that.
@Span('ddb.batchGet', (params: unknown[]) => ({
'db.batch_size': params.length,
}))
@@ -250,25 +255,84 @@ export class DDBClient extends PuterClient {
{} as Record<string, Record<string, unknown>[]>,
);
const requestItems: BatchGetCommandInput['RequestItems'] =
Object.entries(allRequestItemsPerTable).reduce(
(acc, [table, keyList]) => {
acc[table] = {
Keys: keyList,
ConsistentRead: consistentRead,
};
return acc;
},
{} as NonNullable<BatchGetCommandInput['RequestItems']>,
);
const command = new BatchGetCommand({
RequestItems: requestItems,
ReturnConsumedCapacity: 'TOTAL',
});
let requestItems: BatchGetCommandInput['RequestItems'] = Object.entries(
allRequestItemsPerTable,
).reduce(
(acc, [table, keyList]) => {
acc[table] = {
Keys: keyList,
ConsistentRead: consistentRead,
};
return acc;
},
{} as NonNullable<BatchGetCommandInput['RequestItems']>,
);
const client = await this.#getDocumentClient();
return client.send(command);
const responsesByTable = new Map<string, Record<string, unknown>[]>();
const consumedCapacityByTable = new Map<string, number>();
for (let attempt = 0; attempt <= MAX_BATCH_GET_RETRIES; attempt++) {
if (Object.keys(requestItems).length === 0) break;
const response = await client.send(
new BatchGetCommand({
RequestItems: requestItems,
ReturnConsumedCapacity: 'TOTAL',
}),
);
for (const [table, items] of Object.entries(
response.Responses ?? {},
)) {
const existing = responsesByTable.get(table) ?? [];
existing.push(...items);
responsesByTable.set(table, existing);
}
for (const entry of response.ConsumedCapacity ?? []) {
if (!entry.TableName) continue;
consumedCapacityByTable.set(
entry.TableName,
(consumedCapacityByTable.get(entry.TableName) ?? 0) +
Number(entry.CapacityUnits ?? 0),
);
}
const unprocessedKeys = response.UnprocessedKeys ?? {};
if (Object.keys(unprocessedKeys).length === 0) {
requestItems = {};
break;
}
requestItems = unprocessedKeys as NonNullable<
BatchGetCommandInput['RequestItems']
>;
if (attempt < MAX_BATCH_GET_RETRIES) {
const delayMs = Math.min(
1000,
BATCH_GET_RETRY_BASE_MS * 2 ** attempt,
);
await sleep(delayMs);
}
}
if (Object.keys(requestItems).length > 0) {
throw new HttpError(
400,
'Failed to batch get all items from DynamoDB',
{ legacyCode: 'bad_request' },
);
}
return {
Responses: Object.fromEntries(responsesByTable),
ConsumedCapacity: Array.from(consumedCapacityByTable.entries()).map(
([TableName, CapacityUnits]) => ({
TableName,
CapacityUnits,
}),
),
};
}
@Span('ddb.batchPut', (params: unknown[]) => ({
@@ -417,7 +481,15 @@ export class DDBClient extends PuterClient {
async del<T extends Record<string, unknown>>(
table: string,
key: T,
opts?: { returnOld?: boolean },
opts?: {
returnOld?: boolean;
/** Refused (`ConditionalCheckFailedException`) unless it holds. */
condition?: {
expression: string;
names?: Record<string, string>;
values?: Record<string, unknown>;
};
},
) {
const command = new DeleteCommand({
TableName: table,
@@ -426,6 +498,20 @@ export class DDBClient extends PuterClient {
// ALL_OLD makes the delete an atomic claim: exactly one caller
// gets the attributes back.
...(opts?.returnOld ? { ReturnValues: 'ALL_OLD' as const } : {}),
...(opts?.condition
? {
ConditionExpression: opts.condition.expression,
...(opts.condition.names
? { ExpressionAttributeNames: opts.condition.names }
: {}),
...(opts.condition.values
? {
ExpressionAttributeValues:
opts.condition.values,
}
: {}),
}
: {}),
});
const client = await this.#getDocumentClient();
@@ -697,14 +783,15 @@ export class DDBClient extends PuterClient {
const scan = await client.send(
new ScanCommand({
TableName: table,
FilterExpression: '#ttl < :now',
// 0 means no expiry, so it must never sweep.
FilterExpression: '#ttl < :now AND #ttl <> :none',
ExpressionAttributeNames: {
'#ttl': ttlAttribute,
...Object.fromEntries(
keyNames.map((key) => [`#k_${key}`, key]),
),
},
ExpressionAttributeValues: { ':now': now },
ExpressionAttributeValues: { ':now': now, ':none': 0 },
ProjectionExpression: keyNames
.map((key) => `#k_${key}`)
.join(', '),
@@ -715,8 +802,7 @@ export class DDBClient extends PuterClient {
);
lastEvaluatedKey = scan.LastEvaluatedKey as
| Record<string, unknown>
| undefined;
Record<string, unknown> | undefined;
const items = scan.Items;
if (!items || items.length === 0) continue;
+19 -3
View File
@@ -184,7 +184,7 @@ const SHARE_LIST_ROW = `
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="width: 100%;">
{{#each shares}}
<tr>
<td class="ink rule" style="padding: 12px 0;{{#unless @first}} border-top: 1px solid ${RULE};{{/unless}} font-family: ${FONT}; font-size: 16px; line-height: 24px; color: ${INK};"><strong style="font-weight: 600;">{{this.sender}}</strong> shared ${SHARE_ITEM_LIST}</td>
<td class="ink rule" style="padding: 12px 0;{{#unless @first}} border-top: 1px solid ${RULE};{{/unless}} font-family: ${FONT}; font-size: 16px; line-height: 24px; color: ${INK};"><strong style="font-weight: 600;">{{this.sender}}</strong> shared ${SHARE_ITEM_LIST}{{#if this.via}} via {{this.via}}{{/if}}</td>
</tr>
{{/each}}
</table>
@@ -404,6 +404,22 @@ will be asked to choose your own the next time you sign in.</p>
<p>If you did not expect this, the record of who reset it and when is in your
team activity, along with every sign-in to your account.</p>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
team_2fa_reset: {
subject:
'Two-factor authentication was reset on your {{team_name}} account',
html: `
<p>Hi there,</p>
<p>An administrator of {{team_name}} reset two-factor authentication on your
Puter account <b>{{username}}</b>, so you can set it up again from a device you
still have. You have been signed out everywhere.</p>
<p>Your password is unchanged. If your team requires two-factor authentication,
you will be asked to enrol the next time you sign in.</p>
<p>If you did not expect this, the record of who reset it and when is in your
team activity, along with every sign-in to your account.</p>
<p>Sincerely,</p>
<p>Puter</p>
`,
},
@@ -458,7 +474,7 @@ immediately</p>
Shared with you on Puter:
{{#each shares}}
- {{this.sender}} shared {{this.what}}
- {{this.sender}} shared {{this.what}}{{#if this.via}} via {{this.via}}{{/if}}
{{#each this.items}}{{#if this.link}} {{this.name}}: {{this.link}}
{{/if}}{{/each}}{{/each}}
@@ -496,7 +512,7 @@ immediately</p>
Shared with you on Puter:
{{#each shares}}
- {{this.sender}} shared {{this.what}}
- {{this.sender}} shared {{this.what}}{{#if this.via}} via {{this.via}}{{/if}}
{{/each}}
There's no Puter account for {{email}} yet. Create one with this
+67 -1
View File
@@ -227,14 +227,57 @@ export type EventMap = {
fingerprint?: string | null;
/** True when the created account is a temp user (no email/password). */
is_temp?: boolean;
/** The bonus code `puter.signup-bonus.validate` accepted, if any. */
bonus_code?: string;
[key: string]: unknown;
};
// Signup bonus codes are pure mechanism here: an extension decides what a
// code is worth and fills these in (both emitted via `emitAndWait`).
'puter.signup-bonus.check': {
code: string;
ip: string | null;
fingerprint: string | null;
valid: boolean;
/** Opaque to core; forwarded to the client when `valid` is false. */
reason: string | null;
display: { title: string; description: string } | null;
/** Verification the code will require after signup. */
requirements: { phone: boolean; card: boolean } | null;
};
/**
* Emitted once a signup has passed `puter.signup.validate`, so listeners
* see the harness's verdict. A listener sets `accepted` to honor the code,
* and may raise (never lower) the verification requirements.
*/
'puter.signup-bonus.validate': {
code: string;
email?: string;
clean_email?: string;
ip?: string | null;
fingerprint?: string | null;
reputation?: number | null;
source?: 'oidc';
requires_phone_verification: boolean;
requires_card_verification: boolean;
accepted: boolean;
};
'email.validate': {
email: string;
allow: boolean;
message: string | null;
[key: string]: unknown;
};
// Inbound mail to a feedback address (`abuse@`, `fbl@`, ...), emitted via
// `emitAndWait`. A listener that takes it sets `handled` and it reaches no
// mailbox; otherwise it is routed like any other message.
'email.ingress.feedback': {
/** The envelope recipient, as the relay presented it. */
to: string;
from: string | null;
/** The whole message, exactly as received. */
raw: Buffer;
handled: boolean;
};
'user.save_account': {
user_id: number;
old_username?: string;
@@ -429,6 +472,10 @@ export type EventMap = {
* did not have it in hand, as an `expire` does not.
*/
values?: unknown[];
/** Keys among `keys` private to the namespace's app. */
noShareKeys?: string[];
/** Events handler runs behind the write; see `Actor.handlerDepth`. */
handlerDepth?: number;
};
/**
* A whole namespace was emptied. Namespace-level on purpose: `flush`'s own
@@ -452,6 +499,8 @@ export type EventMap = {
appUid: string | null;
value: Record<string, unknown>;
createdAt: number;
/** Events handler runs behind whatever created it. */
handlerDepth?: number;
};
/**
* One notification, addressed at whatever sockets each region holds for the
@@ -555,6 +604,11 @@ export type EventMap = {
generation: number;
/** Whether the table changed; only then does a peer need to re-read it. */
durable: boolean;
/**
* Session subscriptions this bump ended, so siblings stop queueing for
* them.
*/
ended?: string[];
};
/**
* A user's sockets moved between regions, so every node must drop what it
@@ -605,6 +659,16 @@ export type EventMap = {
requestHash?: string;
mime: string;
};
// Asked before a hosted file is streamed. A listener that owns the site
// sets `result.allowed = false` to withhold the entry; the visitor then
// sees the same 404 as for a missing file.
'site.access.check': {
subdomain: string;
host: string;
requestPath: string;
entry: { name: string; path: string };
result: { allowed: boolean };
};
// ---- Thumbnails ----
// The listener rewrites `thumbnail` in place (an s3:// key or legacy
@@ -616,7 +680,8 @@ export type EventMap = {
uid?: string;
thumbnail?: string | null;
};
'thumbnail.created': { url: string };
/** `uuid` is the entry the thumbnail belongs to. */
'thumbnail.created': { url: string; uuid: string };
'thumbnail.upload.prepare': {
items: { index: number; item_uid: string }[];
uploadUrl?: string;
@@ -664,6 +729,7 @@ export type EventMap = {
email?: string;
app_name?: string;
};
/** False for an app actor: add only `subscribed` / `paid_storage` then. */
isUser: boolean;
};
'wisp.get-policy': {
+159
View File
@@ -0,0 +1,159 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { existsSync, readFileSync } from 'node:fs';
import path from 'node:path';
import type { IConfig } from './types';
// Config resolution order:
// 1. `process.env.PUTER_CONFIG_PATH` — absolute path to a config file. Used
// by deployments whose outer bootstrap writes a merged config to a known
// location.
// 2. `<PACKAGE_ROOT>/config.json` — user's runtime override (gitignored),
// deep-merged over config.default.json so users can omit keys they
// don't care to override (e.g. gui_assets_root, database).
// 3. `<PACKAGE_ROOT>/config.default.json` — bundled OSS defaults.
//
// Post-flatten depth: compiled file is at `packages/puter/dist/src/backend/index.js`,
// so three `..`s land at `packages/puter/`.
const PACKAGE_ROOT = path.resolve(__dirname, '../../..');
// Root of the running code tree. Matches PACKAGE_ROOT for a source run, but
// points at `dist/` for a compiled run — so config-declared paths like
// `./extensions` resolve to `dist/extensions` at runtime without the config
// having to know about the build layout.
const RUNTIME_ROOT = path.resolve(__dirname, '../..');
const isPlainObject = (v: unknown): v is Record<string, unknown> =>
typeof v === 'object' && v !== null && !Array.isArray(v);
const deepMerge = <T extends Record<string, unknown>>(
base: T,
override: Record<string, unknown>,
): T => {
const out: Record<string, unknown> = { ...base };
for (const [k, v] of Object.entries(override)) {
out[k] =
isPlainObject(v) && isPlainObject(out[k])
? deepMerge(out[k] as Record<string, unknown>, v)
: v;
}
return out as T;
};
export const loadConfig = (): IConfig => {
const envPath = process.env.PUTER_CONFIG_PATH;
const runtimePath = path.join(PACKAGE_ROOT, 'config.json');
const defaultPath = path.join(PACKAGE_ROOT, 'config.default.json');
const defaults = existsSync(defaultPath)
? (JSON.parse(readFileSync(defaultPath, 'utf8')) as Record<
string,
unknown
>)
: {};
// Runtime override path: env wins, then config.json, else no override
// (we still return defaults so single-file installs work).
const overridePath =
envPath && existsSync(envPath)
? envPath
: existsSync(runtimePath)
? runtimePath
: null;
console.log(`[config] defaults from ${defaultPath}`);
if (overridePath) console.log(`[config] override from ${overridePath}`);
const override = overridePath
? (JSON.parse(readFileSync(overridePath, 'utf8')) as Record<
string,
unknown
>)
: {};
const config = deepMerge(defaults, override) as IConfig;
if (!config.version) {
const pkgPath = path.join(PACKAGE_ROOT, 'package.json');
if (existsSync(pkgPath)) {
try {
const pkg = JSON.parse(readFileSync(pkgPath, 'utf8')) as {
version?: string;
};
if (pkg.version) config.version = pkg.version;
} catch {
// fall through — /version returns 'unknown'
}
}
}
// Computed defaults. `origin` and `pub_port` are the externally-visible
// URL+port — what the browser sees. Separate from `port`, which is the
// bind port (can differ when behind a reverse proxy). Code paths that
// build self-referential URLs (GUI bootstrap, email links, OIDC callbacks)
// depend on `origin` having the right port baked in.
if (config.pub_port === undefined) config.pub_port = config.port;
const protocol = config.protocol ?? 'http';
const domain = config.domain ?? 'localhost';
const portSuffix =
config.pub_port === 80 || config.pub_port === 443
? ''
: `:${config.pub_port}`;
if (config.origin === undefined) {
config.origin = `${protocol}://${domain}${portSuffix}`;
}
// API lives on the `api.` subdomain on the same host+port as the main
// origin (see PuterRouter subdomain handling in server.ts). Compute it
// from pub_port/domain so a single-port override (e.g. port: 5101) flows
// through to the GUI bootstrap without users having to restate the URL.
if (config.api_base_url === undefined) {
config.api_base_url = `${protocol}://api.${domain}${portSuffix}`;
}
// Resolve path-valued config fields. Two different roots:
// - `extensions` uses RUNTIME_ROOT because extensions ship inside the
// build output (dist/extensions) and the loader's dynamic import()
// resolves relative paths against the *importing* module file.
// - GUI/puter-js/builtin-apps use PACKAGE_ROOT because those assets
// live only in the source tree (not copied into dist/) and are served
// via express.static at runtime.
const resolveRuntime = (p: string): string =>
path.isAbsolute(p) ? p : path.resolve(RUNTIME_ROOT, p);
const resolvePackage = (p: string): string =>
path.isAbsolute(p) ? p : path.resolve(PACKAGE_ROOT, p);
if (Array.isArray(config.extensions)) {
config.extensions = config.extensions.map(resolveRuntime);
}
if (typeof config.gui_assets_root === 'string') {
config.gui_assets_root = resolvePackage(config.gui_assets_root);
}
if (typeof config.puterjs_root === 'string') {
config.puterjs_root = resolvePackage(config.puterjs_root);
}
if (isPlainObject(config.builtin_apps)) {
for (const [k, v] of Object.entries(config.builtin_apps)) {
if (typeof v === 'string') {
(config.builtin_apps as Record<string, string>)[k] =
resolvePackage(v);
}
}
}
return config;
};
+18 -6
View File
@@ -69,6 +69,11 @@ const APP_ICON_LIMIT = {
key: 'ip',
};
// Shared by `GET /apps/:name` (pipe-separated) and `POST /query/app`, which
// both fan out one lookup per name.
const APP_LOOKUP_MAX_ENTRIES = 200;
const APP_LOOKUP_MAX_SELECTOR_LEN = 200;
export class AppController extends PuterController {
get appStore() {
return this.stores.app;
@@ -266,11 +271,21 @@ export class AppController extends PuterController {
async (req, res) => {
const raw = req.params.name;
const names = raw.split('|').filter(Boolean);
if (names.length > APP_LOOKUP_MAX_ENTRIES) {
throw new HttpError(
400,
`at most ${APP_LOOKUP_MAX_ENTRIES} app names per request`,
{ legacyCode: 'bad_request' },
);
}
const userUid = req.actor?.user?.uuid ?? null;
const results = await Promise.all(
names.map(async (name) => {
if (name.length > APP_LOOKUP_MAX_SELECTOR_LEN) {
return null;
}
const app = await this.appStore.getByName(name);
if (!app) return null;
let shaped;
@@ -327,9 +342,6 @@ export class AppController extends PuterController {
// included for public/owned apps only, consistent with
// marketplace semantics.
const QUERY_APP_MAX_ENTRIES = 200;
const QUERY_APP_MAX_SELECTOR_LEN = 200;
router.post(
'/query/app',
{
@@ -339,10 +351,10 @@ export class AppController extends PuterController {
},
async (req, res) => {
const appList = Array.isArray(req.body) ? req.body : [];
if (appList.length > QUERY_APP_MAX_ENTRIES) {
if (appList.length > APP_LOOKUP_MAX_ENTRIES) {
throw new HttpError(
400,
`request body must contain at most ${QUERY_APP_MAX_ENTRIES} selectors`,
`request body must contain at most ${APP_LOOKUP_MAX_ENTRIES} selectors`,
{ legacyCode: 'bad_request' },
);
}
@@ -354,7 +366,7 @@ export class AppController extends PuterController {
if (
typeof selector !== 'string' ||
selector.length === 0 ||
selector.length > QUERY_APP_MAX_SELECTOR_LEN
selector.length > APP_LOOKUP_MAX_SELECTOR_LEN
) {
continue;
}
@@ -538,6 +538,57 @@ describe('AppController GET /apps/:name', () => {
});
});
it('throws 400 when the pipe-batch exceeds the 200-name cap', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
const names = new Array(201).fill('x').join('|');
await expect(
withActor(actor, () =>
callRoute(
'get',
'/apps/:name',
makeReq({ params: { name: names }, actor }),
res,
),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('accepts a pipe-batch at exactly the 200-name cap', async () => {
const owner = await makeUser();
const app = await createApp(owner.actor);
const names = [app.name, ...new Array(199).fill('missing-app')].join(
'|',
);
const { res, captured } = makeRes();
await withActor(owner.actor, () =>
callRoute(
'get',
'/apps/:name',
makeReq({ params: { name: names }, actor: owner.actor }),
res,
),
);
expect(Array.isArray(captured.body)).toBe(true);
expect(captured.body as unknown[]).toHaveLength(200);
});
it('throws 404 for a single name over the per-name length cap', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
callRoute(
'get',
'/apps/:name',
makeReq({ params: { name: 'x'.repeat(201) }, actor }),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('omits the stale index_url for a non-owner', async () => {
const owner = await makeUser();
const stranger = await makeUser();
@@ -0,0 +1,614 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { TOTP } from 'otpauth';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { makeActor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { createSecret as otpCreateSecret } from '../../services/auth/OTPUtil.js';
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
import {
createTestUser,
setupPuterTestEnv,
type PuterTestEnv,
} from '../../testUtil.js';
import type { IConfig } from '../../types';
/**
* HTTP-level coverage for `/auth/revoke-own-access-token` — the route that
* lets an app revoke a read-URL token it minted itself, without touching the
* account-only `/auth/revoke-access-token`.
*/
describe('revoke-own-access-token over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const call = (
method: string,
path: string,
token: string,
body?: unknown,
) =>
fetch(new URL(path, env.apiOrigin), {
method,
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
...(body === undefined ? {} : { body: JSON.stringify(body) }),
});
const tokenReadStatus = async (fileUid: string, token: string) => {
const url = new URL('/token-read', env.apiOrigin);
url.searchParams.set('uid', fileUid);
url.searchParams.set('token', token);
return (await fetch(url)).status;
};
/** A file with real content in `owner`'s home, readable by uid. */
const makeFile = async (owner: { username: string }) => {
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const content = Buffer.from(`revoke-http-${crypto.randomUUID()}`);
const path = `/${owner.username}/Documents/${crypto.randomUUID()}.txt`;
await env.server.services.fs.write(user!.id, {
fileMetadata: {
path,
size: content.byteLength,
contentType: 'text/plain',
},
fileContent: content,
});
const entry = await env.server.stores.fsEntry.getEntryByPath(path);
return { uid: entry!.uid };
};
/** An app of `owner`'s, with a token and read reach over `file`. */
const makeApp = async (
owner: { username: string },
file: { uid: string },
) => {
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const actor = makeActor({ user: user! });
const app = await env.server.stores.app.create(
{
name: `auth-http-app-${crypto.randomUUID()}`,
title: 'Auth app',
index_url: `https://auth-${crypto.randomUUID()}.test/`,
},
{ ownerUserId: actor.user.id! },
);
await runWithContext({ actor }, () =>
env.server.services.permission.grantUserAppPermission(
actor,
app.uid,
`fs:${file.uid}:read`,
),
);
const token = await env.server.services.auth.getUserAppToken(
actor,
app.uid,
);
return { ...app, token };
};
/** A scoped (non-full-access) access token for `owner`, carrying no app. */
const makeScopedToken = async (owner: {
username: string;
token: string;
}) => {
const res = await call(
'POST',
'/auth/create-access-token',
owner.token,
{ permissions: ['service:foo:ii:read'], expiresIn: '1h' },
);
expect(res.status).toBe(200);
return ((await res.json()) as { token: string }).token;
};
/** Mints a read-URL token the way `getReadURL()` does. */
const mintReadToken = async (callerToken: string, fileUid: string) => {
const res = await call(
'POST',
'/auth/create-access-token',
callerToken,
{ permissions: [`fs:${fileUid}:read`], expiresIn: '1h' },
);
expect(res.status).toBe(200);
return ((await res.json()) as { token: string }).token;
};
it('lets an app revoke a read token it minted itself', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const app = await makeApp(owner, file);
const readToken = await mintReadToken(app.token, file.uid);
expect(await tokenReadStatus(file.uid, readToken)).toBe(200);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
app.token,
{ token: readToken },
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true });
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
});
it('404s a second app of the same user, and the URL still works', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const app = await makeApp(owner, file);
const otherApp = await makeApp(owner, file);
const readToken = await mintReadToken(app.token, file.uid);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
otherApp.token,
{ token: readToken },
);
expect(res.status).toBe(404);
expect(await res.json()).toMatchObject({ code: 'not_found' });
expect(await tokenReadStatus(file.uid, readToken)).toBe(200);
});
it('lets the account session revoke an app-minted token', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const app = await makeApp(owner, file);
const readToken = await mintReadToken(app.token, file.uid);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
owner.token,
{ token: readToken },
);
expect(res.status).toBe(200);
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
});
it('404s another user entirely', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const app = await makeApp(owner, file);
const readToken = await mintReadToken(app.token, file.uid);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
env.users.other.token,
{ token: readToken },
);
expect(res.status).toBe(404);
expect(await res.json()).toMatchObject({ code: 'not_found' });
expect(await tokenReadStatus(file.uid, readToken)).toBe(200);
});
it('404s an app presenting an account-issued token', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const app = await makeApp(owner, file);
// Minted with the owner's own session, so it carries no `app_uid`.
const accountToken = await mintReadToken(owner.token, file.uid);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
app.token,
{ token: accountToken },
);
expect(res.status).toBe(404);
expect(await res.json()).toMatchObject({ code: 'not_found' });
expect(await tokenReadStatus(file.uid, accountToken)).toBe(200);
});
it('refuses a scoped access token as caller', async () => {
const owner = env.users.user;
const scoped = await makeScopedToken(owner);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
scoped,
{ token: scoped },
);
expect(res.status).toBe(403);
expect(await res.json()).toMatchObject({ code: 'forbidden' });
});
it('rejects a non-JWT string with token_invalid', async () => {
const owner = env.users.user;
const res = await call(
'POST',
'/auth/revoke-own-access-token',
owner.token,
{ token: 'not-a-jwt' },
);
expect(res.status).toBe(400);
expect(await res.json()).toMatchObject({ code: 'token_invalid' });
});
it('still requires `token` in the body', async () => {
const owner = env.users.user;
const res = await call(
'POST',
'/auth/revoke-own-access-token',
owner.token,
{},
);
expect(res.status).toBe(400);
expect(await res.json()).toMatchObject({ code: 'bad_request' });
});
/** A second personal API token for `owner`, minted the way the dashboard does. */
const mintPersonalToken = async (owner: { token: string }) => {
const { actor } = await env.server.services.auth.authenticate(
owner.token,
);
return env.server.services.auth.createAccessToken(actor!, [
[FULL_API_ACCESS],
]);
};
const whoamiStatus = async (token: string) =>
(await call('GET', '/whoami', token)).status;
it('refuses to revoke a personal API token, even for its own account', async () => {
const owner = env.users.user;
const pat = await mintPersonalToken(owner);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
owner.token,
{ token: pat },
);
expect(res.status).toBe(403);
expect(await res.json()).toMatchObject({ code: 'forbidden' });
expect(await whoamiStatus(pat)).toBe(200);
});
it('keeps a personal API token from revoking a sibling one', async () => {
const owner = env.users.user;
const sibling = await mintPersonalToken(owner);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
owner.apiToken,
{ token: sibling },
);
expect(res.status).toBe(403);
expect(await res.json()).toMatchObject({ code: 'forbidden' });
expect(await whoamiStatus(sibling)).toBe(200);
});
it('lets a personal API token revoke its account`s read token', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
// Access tokens can't mint, so the account session creates the URL.
const readToken = await mintReadToken(owner.token, file.uid);
expect(await tokenReadStatus(file.uid, readToken)).toBe(200);
const res = await call(
'POST',
'/auth/revoke-own-access-token',
owner.apiToken,
{ token: readToken },
);
expect(res.status).toBe(200);
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
});
it('resolves a second revoke of the same token', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const app = await makeApp(owner, file);
const readToken = await mintReadToken(app.token, file.uid);
for (let i = 0; i < 2; i++) {
const res = await call(
'POST',
'/auth/revoke-own-access-token',
app.token,
{ token: readToken },
);
expect(res.status).toBe(200);
}
expect(await tokenReadStatus(file.uid, readToken)).not.toBe(200);
});
// These three run last in the file's shared `env.users.user`: each one
// revokes a session belonging to `owner.token` / `owner.workerToken`, so
// later tests can't rely on those credentials still being live.
it('a revoked access token yields 401 with no reauth_token or auth_id', async () => {
const owner = env.users.user;
const file = await makeFile(owner);
const readToken = await mintReadToken(owner.token, file.uid);
const revoke = await call(
'POST',
'/auth/revoke-own-access-token',
owner.token,
{ token: readToken },
);
expect(revoke.status).toBe(200);
const res = await call('GET', '/whoami', readToken);
expect(res.status).toBe(401);
const body = (await res.json()) as Record<string, unknown>;
expect(body.code).toBe('reauth_required');
expect(body.reauth_token).toBeUndefined();
expect(body.auth_id).toBeUndefined();
});
it('a revoked worker session gets no reauth_token even though it rides the session token type', async () => {
const owner = env.users.user;
const decoded = env.server.services.token.verify(
'auth',
owner.workerToken,
) as { session_uid: string };
await env.server.stores.session.removeByUuid(decoded.session_uid);
const res = await call('GET', '/whoami', owner.workerToken);
expect(res.status).toBe(401);
const body = (await res.json()) as Record<string, unknown>;
expect(body.code).toBe('reauth_required');
expect(body.reauth_token).toBeUndefined();
expect(body.auth_id).toBeUndefined();
});
it('a revoked GUI session still gets a reauth_token', async () => {
const owner = env.users.user;
const decoded = env.server.services.token.verify('auth', owner.token) as {
session_uid: string;
};
await env.server.stores.session.removeByUuid(decoded.session_uid);
const res = await call('GET', '/whoami', owner.token);
expect(res.status).toBe(401);
const body = (await res.json()) as Record<string, unknown>;
expect(body.code).toBe('reauth_required');
expect(typeof body.reauth_token).toBe('string');
const user = await env.server.stores.user.getByUsername(
owner.username,
);
expect(
env.server.services.auth.verifyReauthToken(
body.reauth_token as string,
).authId,
).toBe(user!.uuid);
});
});
/**
* Enrolling a second factor: setup sits behind the user-protected gate, and
* enable needs a live code for the secret setup issued.
*/
describe('2FA enrollment over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv({ teams_enabled: true } as IConfig);
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const PASSWORD = 'tfa-http-password';
const uniq = () => Math.random().toString(36).slice(2, 10);
/** A signed-in password account with a confirmed address. */
const makeAccount = async () => {
const username = `tfa_${uniq()}`;
const { token } = await createTestUser(env.server, {
username,
password: PASSWORD,
});
const row = (await env.server.stores.user.getByUsername(username))!;
await env.server.stores.user.update(row.id, {
email: `${username}@test.local`,
email_confirmed: 1,
});
await env.server.stores.user.invalidateById(row.id);
return { username, token, userId: row.id };
};
/** The user-protected gate takes the session cookie on the GUI origin. */
const protectedPost = (
path: string,
token: string,
body: Record<string, unknown>,
credential: 'cookie' | 'bearer' = 'cookie',
) =>
fetch(new URL(path, env.origin), {
method: 'POST',
headers: {
'content-type': 'application/json',
...(credential === 'cookie'
? { cookie: `puter_auth_token=${token}` }
: { authorization: `Bearer ${token}` }),
},
body: JSON.stringify(body),
});
const setup = (
token: string,
body: Record<string, unknown>,
credential: 'cookie' | 'bearer' = 'cookie',
) => protectedPost('/user-protected/setup-2fa', token, body, credential);
const configure = (
token: string,
action: string,
body: Record<string, unknown> = {},
) =>
fetch(new URL(`/auth/configure-2fa/${action}`, env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
const otpState = async (userId: number) => {
const row = await env.server.stores.user.getByProperty('id', userId, {
force: true,
});
return {
secret: row?.otp_secret ?? null,
enabled: Boolean(row?.otp_enabled),
};
};
const liveTotp = (username: string, secret: string) =>
new TOTP({
issuer: 'puter.com',
label: username,
algorithm: 'SHA1',
digits: 6,
secret,
}).generate();
it('refuses setup to a session that has not re-proved the password', async () => {
const account = await makeAccount();
const noPassword = await setup(account.token, {});
expect(noPassword.status).toBe(403);
expect(await noPassword.text()).toContain('password_required');
const wrongPassword = await setup(account.token, { password: 'nope' });
expect(wrongPassword.status).toBe(400);
expect(await wrongPassword.text()).toContain('password_mismatch');
// Right password, but a bearer token is not the session cookie.
const bearer = await setup(
account.token,
{ password: PASSWORD },
'bearer',
);
expect(bearer.status).toBe(401);
// The old ungated path no longer hands out a secret.
expect((await configure(account.token, 'setup')).status).toBe(400);
expect((await otpState(account.userId)).secret).toBeNull();
});
it('enrolls after the password, and enables only on a live code', async () => {
const account = await makeAccount();
const res = await setup(account.token, { password: PASSWORD });
expect(res.status).toBe(200);
const { secret, codes } = (await res.json()) as {
secret: string;
codes: string[];
};
expect(codes).toHaveLength(10);
expect((await otpState(account.userId)).secret).toBe(secret);
expect((await configure(account.token, 'enable')).status).toBe(400);
const foreign = otpCreateSecret(account.username).secret;
const wrong = await configure(account.token, 'enable', {
code: liveTotp(account.username, foreign),
});
expect(wrong.status).toBe(400);
expect(await wrong.text()).toContain('code_mismatch');
expect((await otpState(account.userId)).enabled).toBe(false);
const enabled = await configure(account.token, 'enable', {
code: liveTotp(account.username, secret),
});
expect(enabled.status).toBe(200);
expect((await otpState(account.userId)).enabled).toBe(true);
});
it('lets a seat its team holds for 2FA enroll through the same gate', async () => {
const owner = await makeAccount();
// The rule can only be turned on by an owner who has 2FA.
await env.server.stores.user.update(owner.userId, { otp_enabled: 1 });
await env.server.stores.user.invalidateById(owner.userId);
const team = await env.server.services.team.createTeam(owner.userId, {
name: 'Acme',
});
const seatName = `seat_${uniq()}`;
const { userId, temporaryPassword } =
await env.server.services.team.provisionAccount(
team.uid,
owner.userId,
{ username: seatName },
);
await env.server.services.team.updateTeam(team.uid, owner.userId, {
require2fa: true,
});
const { token } = await env.server.services.auth.createSessionToken(
(await env.server.stores.user.getById(userId))!,
);
// First sign-in replaces the temporary password.
const changed = await protectedPost(
'/user-protected/change-password',
token,
{ password: temporaryPassword, new_pass: 'seat-own-password' },
);
expect(changed.status).toBe(200);
const held = await fetch(new URL('/teams', env.apiOrigin), {
headers: { authorization: `Bearer ${token}` },
});
expect(held.status).toBe(403);
expect(await held.text()).toContain('two_factor_required');
const res = await setup(token, { password: 'seat-own-password' });
expect(res.status).toBe(200);
const { secret } = (await res.json()) as { secret: string };
const enabled = await configure(token, 'enable', {
code: liveTotp(seatName, secret),
});
expect(enabled.status).toBe(200);
expect((await otpState(userId)).enabled).toBe(true);
const released = await fetch(new URL('/teams', env.apiOrigin), {
headers: { authorization: `Bearer ${token}` },
});
expect(released.status).toBe(200);
});
});
@@ -0,0 +1,413 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Signup bonus codes through the real controller: the check route, the signup
* hand-off to `puter.signup-bonus.validate`, and the awaited
* `user.card-verified` a grant on card verification depends on. Tests stand in
* for the extension with shared listeners (EventClient has no `off()`).
*/
import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest';
import type { Actor } from '../../core/actor.js';
import type { EventMap } from '../../clients/event/types.js';
import type { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
let server: PuterServer;
let controller: any;
type Handler<K extends keyof EventMap> = ((data: EventMap[K]) => void) | null;
let onBonusCheck: Handler<'puter.signup-bonus.check'> = null;
let onBonusValidate: Handler<'puter.signup-bonus.validate'> = null;
let onSignupValidate: ((data: Record<string, unknown>) => void) | null = null;
let onCardConfirm: ((data: Record<string, unknown>) => void) | null = null;
let onCardVerified: (() => Promise<void>) | null = null;
const heardSignupSuccess: Array<Record<string, unknown>> = [];
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.auth;
const events = server.clients.event;
events.on('puter.signup-bonus.check', (_k, data) => onBonusCheck?.(data));
events.on('puter.signup-bonus.validate', (_k, data) =>
onBonusValidate?.(data),
);
events.on('puter.signup.validate', (_k, data) =>
onSignupValidate?.(data as Record<string, unknown>),
);
events.on('puter.signup.success', (_k, data) => {
heardSignupSuccess.push(data as Record<string, unknown>);
});
events.on('puter.card-verification.confirm', (_k, data) =>
onCardConfirm?.(data as Record<string, unknown>),
);
events.on('user.card-verified' as never, async () => {
await onCardVerified?.();
});
});
afterAll(async () => {
await server?.shutdown();
});
afterEach(() => {
onBonusCheck = null;
onBonusValidate = null;
onSignupValidate = null;
onCardConfirm = null;
onCardVerified = null;
});
const uniq = () => Math.random().toString(36).slice(2, 10);
/** A check listener that finds the code open, as an extension would. */
const openCheck: Handler<'puter.signup-bonus.check'> = (data) => {
data.valid = true;
data.display = { title: 'T', description: 'D' };
};
const makeReq = (
body: Record<string, unknown> = {},
extra: { actor?: Actor; ip?: string } = {},
) => ({
body,
headers: {},
connection: { remoteAddress: extra.ip ?? '127.0.0.1' },
socket: { remoteAddress: extra.ip ?? '127.0.0.1' },
ip: extra.ip ?? '127.0.0.1',
params: {},
actor: extra.actor,
});
const makeRes = () => {
const res = {
statusCode: 200,
body: undefined as unknown,
status(code: number) {
this.statusCode = code;
return this;
},
json(b: unknown) {
this.body = b;
return this;
},
cookie() {
return this;
},
clearCookie() {
return this;
},
send() {
return this;
},
end() {
return this;
},
};
return res;
};
const signupBody = (extra: Record<string, unknown> = {}) => {
const username = `b_${uniq()}`;
return {
username,
email: `${username}@test.local`,
password: 'correct-horse-battery',
...extra,
};
};
const signup = async (body: Record<string, unknown>) => {
const res = makeRes();
await controller.handleSignup(makeReq(body), res);
return res;
};
const findUser = (username: unknown) =>
server.stores.user.getByUsername(username as string, { force: true });
/** `puter.signup.success` is fire-and-forget; wait for it to land. */
const waitForSuccess = async (userUuid: string) => {
const deadline = Date.now() + 2000;
while (Date.now() < deadline) {
const hit = heardSignupSuccess.find((e) => e.user_uuid === userUuid);
if (hit) return hit;
await new Promise((r) => setTimeout(r, 10));
}
throw new Error('puter.signup.success never arrived');
};
// -- POST /signup/bonus-code/check -----------------------------------
describe('AuthController.handleSignupBonusCheck', () => {
const check = async (body: Record<string, unknown>) => {
const res = makeRes();
await controller.handleSignupBonusCheck(makeReq(body), res);
return res.body;
};
it('answers invalid when no extension is listening', async () => {
expect(await check({ bonusCode: 'startup3m-abcd1234' })).toEqual({
valid: false,
});
});
it('returns the display and requirements a listener fills in', async () => {
let seen: EventMap['puter.signup-bonus.check'] | null = null;
onBonusCheck = (data) => {
seen = { ...data };
data.valid = true;
data.display = { title: '3 months of Basic', description: 'Free' };
data.requirements = { phone: true, card: false };
};
expect(
await check({
bonusCode: 'Startup3M-ABCD1234',
fingerprint: 'fp1',
}),
).toEqual({
valid: true,
display: { title: '3 months of Basic', description: 'Free' },
requirements: { phone: true, card: false },
});
// Listeners only ever see the canonical form.
expect(seen).toMatchObject({
code: 'startup3mabcd1234',
fingerprint: 'fp1',
ip: '127.0.0.1',
});
});
it('forwards an opaque reason for a refused code', async () => {
onBonusCheck = (data) => {
data.reason = 'ended';
};
expect(await check({ bonusCode: 'startup3m-abcd1234' })).toEqual({
valid: false,
reason: 'ended',
});
});
it('treats valid without display as invalid', async () => {
onBonusCheck = (data) => {
data.valid = true;
};
expect(await check({ bonusCode: 'startup3m-abcd1234' })).toEqual({
valid: false,
});
});
it('answers a malformed code without consulting listeners', async () => {
let called = false;
onBonusCheck = () => {
called = true;
};
expect(await check({ bonusCode: 'a:b' })).toEqual({ valid: false });
expect(called).toBe(false);
});
it('400s a non-string bonusCode', async () => {
await expect(
controller.handleSignupBonusCheck(
makeReq({ bonusCode: 42 }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
});
});
// -- POST /signup with bonusCode -------------------------------------
describe('AuthController.handleSignup with a bonus code', () => {
it('refuses the signup when nothing accepts the code', async () => {
const body = signupBody({ bonusCode: 'startup3m-abcd1234' });
await expect(signup(body)).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bonus_code_invalid',
});
expect(await findUser(body.username)).toBeFalsy();
});
it('refuses a dead code before the abuse gate records the attempt', async () => {
let validated = false;
onSignupValidate = () => {
validated = true;
};
await expect(
signup(signupBody({ bonusCode: 'startup3m-abcd1234' })),
).rejects.toMatchObject({ legacyCode: 'bonus_code_invalid' });
expect(validated).toBe(false);
});
it('still refuses a code that passed the check but is refused after the gate', async () => {
onBonusCheck = openCheck;
const body = signupBody({ bonusCode: 'startup3m-abcd1234' });
await expect(signup(body)).rejects.toMatchObject({
legacyCode: 'bonus_code_invalid',
});
expect(await findUser(body.username)).toBeFalsy();
});
it('refuses a malformed code before any listener runs', async () => {
let called = false;
onBonusValidate = () => {
called = true;
};
await expect(
signup(signupBody({ bonusCode: 'nope' + '!'.repeat(3) })),
).rejects.toMatchObject({ legacyCode: 'bonus_code_invalid' });
expect(called).toBe(false);
});
it('400s a non-string bonusCode', async () => {
await expect(
signup(signupBody({ bonusCode: ['startup3m'] })),
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
});
it('applies the requirements an accepted code raises and reports the code', async () => {
onBonusCheck = openCheck;
let seen: EventMap['puter.signup-bonus.validate'] | null = null;
onBonusValidate = (data) => {
seen = { ...data };
data.accepted = true;
data.requires_card_verification = true;
};
const body = signupBody({
bonusCode: 'STARTUP6M-abcd1234',
fingerprint: 'fp-bonus',
});
const res = await signup(body);
expect(
(res.body as { user: Record<string, unknown> }).user,
).toMatchObject({ requires_card_verification: true });
expect(seen).toMatchObject({
code: 'startup6mabcd1234',
email: body.email,
fingerprint: 'fp-bonus',
requires_phone_verification: false,
requires_card_verification: false,
});
const user = await findUser(body.username);
expect(Boolean(user!.requires_card_verification)).toBe(true);
expect(Boolean(user!.requires_phone_verification)).toBe(false);
const success = await waitForSuccess(user!.uuid);
expect(success.bonus_code).toBe('startup6mabcd1234');
});
it('keeps a requirement the abuse harness set even if the listener clears it', async () => {
onBonusCheck = openCheck;
onSignupValidate = (data) => {
data.requires_phone_verification = true;
};
onBonusValidate = (data) => {
expect(data.requires_phone_verification).toBe(true);
data.accepted = true;
data.requires_phone_verification = false;
};
const body = signupBody({ bonusCode: 'startup3m-abcd1234' });
await signup(body);
const user = await findUser(body.username);
expect(Boolean(user!.requires_phone_verification)).toBe(true);
});
it('never offers the code for a signup the abuse harness blocked', async () => {
onBonusCheck = openCheck;
let called = false;
onSignupValidate = (data) => {
data.allow = false;
data.message = 'Signup blocked';
};
onBonusValidate = () => {
called = true;
};
await expect(
signup(signupBody({ bonusCode: 'startup3m-abcd1234' })),
).rejects.toMatchObject({ statusCode: 403 });
expect(called).toBe(false);
});
it('leaves the success event without a code when none was presented', async () => {
const body = signupBody();
await signup(body);
const user = await findUser(body.username);
const success = await waitForSuccess(user!.uuid);
expect(success).not.toHaveProperty('bonus_code');
});
it('ignores a bonus code on a temp signup', async () => {
let called = false;
onBonusValidate = () => {
called = true;
};
const res = makeRes();
await controller.handleSignup(
makeReq({ is_temp: true, bonusCode: 'startup3m-abcd1234' }),
res,
);
expect((res.body as { user: { is_temp: boolean } }).user.is_temp).toBe(
true,
);
expect(called).toBe(false);
});
});
// -- /card-verification/confirm awaits user.card-verified ------------
describe('AuthController.handleCardVerificationConfirm', () => {
it('finishes user.card-verified listeners before responding', async () => {
const body = signupBody();
await signup(body);
const created = await findUser(body.username);
await server.stores.user.update(created!.id, {
requires_card_verification: 1,
});
const actor = {
user: {
id: created!.id,
uuid: created!.uuid,
username: created!.username,
email: created!.email ?? null,
email_confirmed: false,
},
} as Actor;
onCardConfirm = (data) => {
data.enabled = true;
data.verified = true;
data.fingerprint = 'fp_card_1';
};
let listenerDone = false;
onCardVerified = async () => {
await new Promise((r) => setTimeout(r, 30));
listenerDone = true;
};
const res = makeRes();
await controller.handleCardVerificationConfirm(
makeReq({ setup_intent_id: 'seti_1' }, { actor }),
res,
);
expect(res.body).toMatchObject({ card_verified: true });
expect(listenerDone).toBe(true);
});
});
File diff suppressed because it is too large. Load diff
+457 -72
View File
@@ -33,7 +33,12 @@ import {
hasVerifiedPhone,
} from '../../core/http/middleware/gates.js';
import type { Actor } from '../../core/actor.js';
import { checkRateLimit } from '../../core/http/middleware/rateLimit.js';
import { isPlainUserActor, makeActor } from '../../core/actor.js';
import {
checkRateLimit,
consumeRouteRateLimit,
peekRouteRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import {
signStepUpToken,
STEP_UP_COOKIE_NAME,
@@ -75,6 +80,13 @@ import { sessionCookieFlags } from '../../util/cookieFlags.js';
import { cleanEmail, isBlockedEmail } from '../../util/email.js';
import { generate_identifier } from '../../util/identifier.js';
import { parsePhone } from '../../util/phone.js';
import {
bonusCodeInvalidError,
checkSignupBonus,
isAbsentBonusCode,
normalizeBonusCode,
validateSignupBonus,
} from '../../util/signupBonus.js';
import { isTemporaryPasswordExpired } from '../../util/temporaryPassword.js';
import { getTaskbarItems } from '../../util/taskbarItems.js';
import {
@@ -108,7 +120,11 @@ const FINGERPRINT_MAX_LENGTH = 128;
// One consent prompt covers a handful of scopes at most. The cap keeps a
// crafted request from turning a single grant call into a bulk write.
const MAX_PERMISSIONS_PER_REQUEST = 16;
// Rides every row a request writes, and the per-(user, app) cache after that.
const GRANT_EXTRA_MAX_BYTES = 4096;
const DISPATCH_ID_MAX_LENGTH = 128;
// One name for the flag, so the write and the read cannot drift apart.
const APP_AUTHENTICATED_FLAG = 'flag:app-is-authenticated';
// -- Post-login route limits -----------------------------------------
//
@@ -130,6 +146,14 @@ const CREDENTIAL_MINT_LIMIT = {
key: 'user',
} as const;
/** Revoking is cheap and undoing a mistake shouldn't fight the mint budget. */
const ACCESS_TOKEN_REVOKE_LIMIT = {
scope: 'auth-access-token-revoke',
limit: 60,
window: 60_000,
key: 'user',
} as const;
/**
* Second-factor configuration, including the verify leg. Shorter window than
* the mint limit because enabling 2FA legitimately involves a few attempts in a
@@ -197,6 +221,26 @@ const SESSION_LIMIT = {
window: 60_000,
key: 'user',
} as const;
/**
* Completed renames. Charged by the handler after the rename lands, so a wrong
* password or a taken name doesn't spend it.
*/
const CHANGE_USERNAME_LIMIT = {
scope: 'change-username-done',
limit: 2,
window: 30 * 24 * 60 * 60_000,
key: 'user',
} as const;
/** Rename attempts, success or not. Bounds password guessing through the gate. */
const CHANGE_USERNAME_ATTEMPT_LIMIT = {
scope: 'change-username-attempt',
limit: 10,
window: 60 * 60_000,
key: 'user',
} as const;
// How long a failed-SMS-send record stays readable by its error_id — long
// enough to cover the typical support round-trip.
const SMS_SEND_ERROR_TTL_SECONDS = 7 * 24 * 60 * 60;
@@ -215,6 +259,15 @@ export const RESERVED_USERNAMES = new Set([
'contact',
'mail',
'email',
// Role mailboxes: a Puter address names its account, so these must never
// be ownable.
'abuse',
'postmaster',
'hostmaster',
'fbl',
'security',
'noreply',
'no-reply',
'null',
'undefined',
'test',
@@ -228,7 +281,7 @@ export const RESERVED_USERNAMES = new Set([
* Auth controller — login/logout, permission grants/revokes, session
* management, OTP, and permission checks.
*
* Routes are declared via decorators (@Get/@Post on each handler). The five
* Routes are declared via decorators (@Get/@Post on each handler). The
* `/user-protected/*` and `/user-protected/delete-own-user` routes also need a
* per-instance `createUserProtectedGate(...)` middleware built from
* `this.config / this.stores / this.services`, which can't live in a static
@@ -804,6 +857,18 @@ export class AuthController extends PuterController {
}
}
// Only the shape is judged here; whether the code is honored is up to
// `puter.signup-bonus.validate`, after the abuse checks below.
let bonusCode: string | null = null;
if (!is_temp && !isAbsentBonusCode(body.bonusCode)) {
if (typeof body.bonusCode !== 'string')
throw new HttpError(400, 'bonusCode must be a string.', {
legacyCode: 'bad_request',
});
bonusCode = normalizeBonusCode(body.bonusCode);
if (!bonusCode) throw bonusCodeInvalidError();
}
// Signup-disabled gate. Runs before the duplicate checks so a
// disabled endpoint doesn't reveal which usernames or emails
// exist. Claiming a pre-existing placeholder row is still
@@ -836,6 +901,22 @@ export class AuthController extends PuterController {
);
}
// ...and the same against the filesystem: a free username whose home
// path is occupied (exactly or by leftover rows underneath it) would
// provision a second root there, and the two trees then resolve
// interchangeably.
if (
await this.stores.fsEntry.findHomePathConflict(
body.username,
undefined,
{ includeDescendants: true },
)
) {
throw new HttpError(400, 'This username is not available.', {
legacyCode: 'bad_request',
});
}
// Duplicate confirmed-email check. A confirmed account (any
// credential type — password OR OIDC) on this email → reject.
//
@@ -868,6 +949,20 @@ export class AuthController extends PuterController {
? null
: await this.#resolveSignupEmailClaim(body.email);
// A dead code fails here rather than after the gate below, which records
// an allowed attempt against the address as if an account followed.
if (
bonusCode &&
!(
await checkSignupBonus(this.clients.event, bonusCode, {
ip: clientIp,
fingerprint,
})
).valid
) {
throw bonusCodeInvalidError();
}
// Extension-level validation gate. Abuse-prevention extensions
// inspect the incoming signup and can:
// - block it outright via `event.allow = false`
@@ -954,18 +1049,37 @@ export class AuthController extends PuterController {
const force_email_confirmation = Boolean(
validateEvent.requires_email_confirmation,
);
const force_phone_verification =
let force_phone_verification =
Boolean(validateEvent.requires_phone_verification) ||
// Test/QA switch: force the SMS gate on every signup regardless of
// reputation (see config.always_require_phone_verification).
Boolean(this.config.always_require_phone_verification);
const force_card_verification = Boolean(
let force_card_verification = Boolean(
validateEvent.requires_card_verification ||
// Test/QA switch: force the card gate on every signup regardless of
// reputation (see config.always_require_card_verification).
this.config.always_require_card_verification,
);
if (bonusCode) {
const verdict = await validateSignupBonus(
this.clients.event,
bonusCode,
{
email: body.email,
clean_email: cleanEmail(body.email),
ip: clientIp,
fingerprint,
reputation: validateEvent.reputation,
requires_phone_verification: force_phone_verification,
requires_card_verification: force_card_verification,
},
);
if (!verdict.accepted) throw bonusCodeInvalidError();
force_phone_verification = verdict.requiresPhoneVerification;
force_card_verification = verdict.requiresCardVerification;
}
// Prepare shared fields
const user_uuid = uuidv4();
const email_confirm_code = String(crypto.randomInt(100000, 1000000));
@@ -1186,6 +1300,7 @@ export class AuthController extends PuterController {
// have to agree or per-IP counters are written under one
// key and read under another.
ip: clientIp,
...(bonusCode ? { bonus_code: bonusCode } : {}),
} as never,
{},
);
@@ -1207,6 +1322,64 @@ export class AuthController extends PuterController {
await this.#completeLogin(req, res, user!);
}
/**
* Preview a signup bonus code before the account exists, so the signup form
* can say what it grants. Answers only valid/invalid: an extension owns the
* codes, and with none installed every code is invalid.
*/
@Post('/signup/bonus-code/check', {
rateLimit: [
{ scope: 'signup-bonus-check', limit: 20, window: 15 * 60_000 },
{
scope: 'signup-bonus-check-ip',
limit: 60,
window: 15 * 60_000,
key: 'ip',
},
],
})
async handleSignupBonusCheck(req: Request, res: Response): Promise<void> {
const raw = req.body?.bonusCode;
if (typeof raw !== 'string')
throw new HttpError(400, 'bonusCode must be a string.', {
legacyCode: 'bad_request',
});
const code = normalizeBonusCode(raw);
if (!code) {
res.json({ valid: false });
return;
}
const fingerprint =
typeof req.body?.fingerprint === 'string' &&
req.body.fingerprint.length <= FINGERPRINT_MAX_LENGTH
? req.body.fingerprint
: null;
const checkEvent = await checkSignupBonus(this.clients.event, code, {
ip: (req.ip || req.socket?.remoteAddress || null) as string | null,
fingerprint,
});
if (checkEvent.valid !== true || !checkEvent.display) {
res.json({
valid: false,
...(checkEvent.reason ? { reason: checkEvent.reason } : {}),
});
return;
}
res.json({
valid: true,
display: {
title: String(checkEvent.display.title),
description: String(checkEvent.display.description),
},
requirements: {
phone: checkEvent.requirements?.phone === true,
card: checkEvent.requirements?.card === true,
},
});
}
// -- Logout ------------------------------------------------------
@Post('/logout', {
@@ -2200,8 +2373,11 @@ export class AuthController extends PuterController {
...(clearedPhoneGate ? { requires_phone_verification: 0 } : {}),
});
// Awaited like `user.phone-verified`, so whatever a listener grants on
// verification is in place before the client refreshes. emitAndWait
// swallows listener errors, so confirm never fails on one.
try {
this.clients.event?.emit(
await this.clients.event?.emitAndWait(
'user.card-verified' as never,
{
user_id: user.id,
@@ -2214,7 +2390,7 @@ export class AuthController extends PuterController {
{},
);
} catch {
// ignore — event is a side-channel signal, not load-bearing
// ignore — listeners are best-effort
}
// Notify other tabs/devices for this user so they refresh + drop the gate.
try {
@@ -2484,7 +2660,7 @@ export class AuthController extends PuterController {
// -- User-protected mutations ------------------------------------
//
// The five `/user-protected/*` and `/user-protected/delete-own-user`
// The `/user-protected/*` and `/user-protected/delete-own-user`
// routes are wired in the `registerRoutes` override below because
// their `middleware: createUserProtectedGate(...)` argument depends
// on `this.config / this.stores / this.services` and so can't live
@@ -2587,10 +2763,33 @@ export class AuthController extends PuterController {
legacyCode: 'username_already_in_use',
});
}
// Before the username is written, not after: the rename below is what
// keeps the account's files reachable, and it can't run onto a taken
// path.
if (
await this.stores.fsEntry.findHomePathConflict(
new_username,
req.actor!.user.id!,
{ includeDescendants: true },
)
) {
throw new HttpError(400, 'This username is not available.', {
legacyCode: 'username_already_in_use',
});
}
if (!(await peekRouteRateLimit(req, CHANGE_USERNAME_LIMIT))) {
throw new HttpError(
429,
'You can only change your username twice every 30 days.',
{ legacyCode: 'too_many_requests' },
);
}
await this.stores.user.update(req.actor!.user.id!, {
username: new_username,
});
await consumeRouteRateLimit(req, CHANGE_USERNAME_LIMIT);
// Rename the user's FS home from `/<old>` to `/<new>` and
// cascade the prefix to all descendants. Without this, any
@@ -2911,6 +3110,15 @@ export class AuthController extends PuterController {
legacyCode: 'username_already_in_use',
});
}
if (
await this.stores.fsEntry.findHomePathConflict(username, user.id, {
includeDescendants: true,
})
) {
throw new HttpError(400, 'This username is not available.', {
legacyCode: 'username_already_in_use',
});
}
// Match raw + canonical to catch gmail-alias collisions, and
// reject on ANY confirmed account (OIDC accounts have
// password=null but are real) — not just password-holders.
@@ -3133,6 +3341,16 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
}
if (
Buffer.byteLength(JSON.stringify(value), 'utf8') >
GRANT_EXTRA_MAX_BYTES
) {
throw new HttpError(
400,
`\`${key}\` may not exceed ${GRANT_EXTRA_MAX_BYTES} bytes`,
{ legacyCode: 'bad_request' },
);
}
}
}
@@ -3535,27 +3753,66 @@ export class AuthController extends PuterController {
// -- Permission checks -------------------------------------------
/**
* The caller's account acting as `appIdentifier`, resolved uid-or-name like
* the grant handlers.
*/
async #appUnderUserActor(
actor: Actor,
appIdentifier: unknown,
): Promise<Actor> {
this.#validateAppPermissionParams({ app_uid: appIdentifier });
// Sessions only: for an app, this would be a window onto its neighbours' grants.
if (!isPlainUserActor(actor)) {
throw new HttpError(403, 'actor must be a user', {
legacyCode: 'forbidden',
});
}
const app = await this.stores.app.resolveApp(appIdentifier as string);
if (!app) {
throw new HttpError(404, `App ${appIdentifier} does not exist`, {
legacyCode: 'not_found',
});
}
return makeActor({
user: actor.user,
app: { id: app.id, uid: app.uid },
});
}
@Post('/auth/check-permissions', {
subdomain: 'api',
requireAuth: true,
rateLimit: AUTH_CHECK_LIMIT,
})
async handleCheckPermissions(req: Request, res: Response): Promise<void> {
const { permissions } = req.body ?? {};
const { permissions, app_uid } = req.body ?? {};
if (!Array.isArray(permissions)) {
throw new HttpError(400, 'Missing or invalid `permissions` array', {
legacyCode: 'bad_request',
});
}
// Present but empty must not fall through to checking the user: on their own file every `fs:` scope answers `true`.
if (
app_uid !== undefined &&
(typeof app_uid !== 'string' || !app_uid)
) {
throw new HttpError(400, 'Invalid `app_uid`', {
legacyCode: 'bad_request',
});
}
// `app_uid` asks what an app of mine holds, not what I hold.
const actor = app_uid
? await this.#appUnderUserActor(req.actor!, app_uid)
: req.actor!;
const unique = [...new Set(permissions)] as string[];
const result: Record<string, boolean> = {};
let granted: Map<string, boolean>;
try {
granted = await this.services.permission.checkMany(
req.actor!,
unique,
);
granted = await this.services.permission.checkMany(actor, unique);
} catch {
granted = new Map<string, boolean>();
}
@@ -3661,6 +3918,13 @@ export class AuthController extends PuterController {
async handleGrantDevApp(req: Request, res: Response): Promise<void> {
let { app_uid } = req.body ?? {};
const { origin, permission, extra, meta } = req.body ?? {};
this.#validateAppPermissionParams({
app_uid,
origin,
permission,
extra,
meta,
});
if (origin && !app_uid) {
// Registered apps only, for the same reason the user-app handlers
// insist on it: a synthesised `app-<uuidv5(origin)>` is resolved
@@ -3839,11 +4103,18 @@ export class AuthController extends PuterController {
if (!app && resolvedFromOrigin) {
// Hosted-subdomain origins get the site owner stamped as the
// app's creator at bootstrap; external origins stay unowned.
// Canonical origin only, so alternate hosts share one row.
const canonicalOrigin =
this.services.auth.canonicalizeOrigin(origin);
const ownerUserId =
await this.services.auth.subdomainOwnerIdFromOrigin(origin);
app = await this.stores.app.createFromOrigin(app_uid, origin, {
ownerUserId,
});
await this.services.auth.subdomainOwnerIdFromOrigin(
canonicalOrigin,
);
app = await this.stores.app.createFromOrigin(
app_uid,
canonicalOrigin,
{ ownerUserId },
);
// An origin's uid is a deterministic uuidv5, so a deleted app
// reappears here under the identical uid. Withdraw any cross-app
// data grants left pointing at it before this new row can inherit
@@ -3871,7 +4142,7 @@ export class AuthController extends PuterController {
this.services.permission.grantUserAppPermission(
req.actor!,
app_uid,
'flag:app-is-authenticated',
APP_AUTHENTICATED_FLAG,
{},
{},
);
@@ -3955,13 +4226,15 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
// Check if the app is authenticated for this user
const authenticated = await this.services.permission
.check(
req.actor!,
`service:${app_uid}:ii:flag:app-is-authenticated`,
)
.catch(() => false);
// The exact row, not a scan: a `service:`-shaped check falls open on the `service` root.
const app = await this.stores.app.resolveApp(app_uid);
const userId = req.actor!.user?.id;
const authenticated =
!!app?.id &&
!!userId &&
(await this.stores.permission
.hasUserAppPerm(userId, app.id, APP_AUTHENTICATED_FLAG)
.catch(() => false));
const result: {
app_uid: string;
@@ -3969,9 +4242,10 @@ export class AuthController extends PuterController {
token?: string;
} = { app_uid, authenticated };
if (authenticated) {
// The resolved uid: the token carries it as the app's identity.
result.token = await this.services.auth.getUserAppToken(
req.actor!,
app_uid,
app!.uid,
);
}
res.json(result);
@@ -3991,6 +4265,11 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
}
if (permissions.length > MAX_PERMISSIONS_PER_REQUEST) {
throw new HttpError(400, 'Too many `permissions`', {
legacyCode: 'bad_request',
});
}
// Optional user-facing name for the manage-sessions UI. Trim and clamp
// to the same 64-char limit the rename endpoint enforces.
@@ -4024,13 +4303,25 @@ export class AuthController extends PuterController {
// Normalize specs: string → [string], [string] → [string, {}], [string, extra] → as-is
const normalized = permissions.map((spec) => {
if (typeof spec === 'string') return [spec];
if (Array.isArray(spec)) return spec;
throw new HttpError(
400,
'Each permission must be a string or [string, extra?]',
{ legacyCode: 'bad_request' },
);
const entry =
typeof spec === 'string'
? [spec]
: Array.isArray(spec)
? spec
: null;
if (!entry || typeof entry[0] !== 'string' || !entry[0]) {
throw new HttpError(
400,
'Each permission must be a string or [string, extra?]',
{ legacyCode: 'bad_request' },
);
}
// Same caps the grant routes apply, before a session or row exists.
this.#validateAppPermissionParams({
permission: entry[0],
extra: entry[1],
});
return entry;
});
const token = await this.services.auth.createAccessToken(
@@ -4044,10 +4335,36 @@ export class AuthController extends PuterController {
res.json({ token });
}
/**
* Revokes an access token given as its JWT. Unlike
* `/auth/revoke-access-token`, apps may call this; the service limits an
* app to tokens it issued and refuses personal API tokens.
*/
@Post('/auth/revoke-own-access-token', {
subdomain: 'api',
requireAuth: true,
rateLimit: ACCESS_TOKEN_REVOKE_LIMIT,
})
async handleRevokeOwnAccessToken(
req: Request,
res: Response,
): Promise<void> {
const { token } = req.body ?? {};
if (!token || typeof token !== 'string') {
throw new HttpError(400, 'Missing `token`', {
legacyCode: 'bad_request',
});
}
await this.services.auth.revokeOwnAccessToken(req.actor!, token);
res.json({ ok: true });
}
// Wired imperatively in `registerRoutes` so the cookie-only gate
// (built from `this.config`) can be composed in. Cookie-only is
// mandatory: a leaked access token must not be able to silently
// revoke its own siblings.
// mandatory: a leaked access token must not be able to revoke a
// personal API token, or revoke by raw uuid — those stay web-session-only
// here; `/auth/revoke-own-access-token` covers revoking scoped tokens by
// JWT and already refuses PATs itself.
async handleRevokeAccessToken(req: Request, res: Response): Promise<void> {
let { tokenOrUuid } = req.body ?? {};
if (!tokenOrUuid || typeof tokenOrUuid !== 'string') {
@@ -4064,11 +4381,51 @@ export class AuthController extends PuterController {
res.json({ ok: true });
}
// -- 2FA: setup (user-protected, wired in registerRoutes below) ---
async handleSetup2fa(req: Request, res: Response): Promise<void> {
const user = await this.stores.user.getById(req.actor!.user.id!, {
force: true,
});
if (!user)
throw new HttpError(404, 'User not found', {
legacyCode: 'not_found',
});
if (user.otp_enabled) {
throw new HttpError(409, '2FA is already enabled.', {
legacyCode: 'conflict',
});
}
const result = otpCreateSecret(user.username);
// Generate 10 recovery codes
const codes: string[] = [];
for (let i = 0; i < 10; i++) {
codes.push(createRecoveryCode());
}
const hashedCodes = codes.map((c) => hashRecoveryCode(c));
await this.clients.db.write(
'UPDATE `user` SET `otp_secret` = ?, `otp_recovery_codes` = ? WHERE `uuid` = ?',
[result.secret, hashedCodes.join(','), user.uuid],
);
await this.stores.user.invalidateById(user.id);
res.json({
url: result.url,
secret: result.secret,
codes,
});
}
// -- 2FA: configure ----------------------------------------------
@Post('/auth/configure-2fa/:action', {
subdomain: 'api',
requireUserActor: true,
// A member owing their team's 2FA reaches nothing else until this.
allowUnconfirmed: true,
rateLimit: TWO_FACTOR_LIMIT,
})
async handleConfigure2fa(req: Request, res: Response): Promise<void> {
@@ -4081,36 +4438,6 @@ export class AuthController extends PuterController {
legacyCode: 'not_found',
});
if (action === 'setup') {
if (user.otp_enabled) {
throw new HttpError(409, '2FA is already enabled.', {
legacyCode: 'conflict',
});
}
const result = otpCreateSecret(user.username);
// Generate 10 recovery codes
const codes: string[] = [];
for (let i = 0; i < 10; i++) {
codes.push(createRecoveryCode());
}
const hashedCodes = codes.map((c) => hashRecoveryCode(c));
await this.clients.db.write(
'UPDATE `user` SET `otp_secret` = ?, `otp_recovery_codes` = ? WHERE `uuid` = ?',
[result.secret, hashedCodes.join(','), user.uuid],
);
await this.stores.user.invalidateById(user.id);
res.json({
url: result.url,
secret: result.secret,
codes,
});
return;
}
if (action === 'test') {
const { code } = req.body ?? {};
if (!code)
@@ -4123,7 +4450,9 @@ export class AuthController extends PuterController {
}
if (action === 'enable') {
if (!user.email_confirmed) {
// A seat has no email by design, so this would bar it forever.
const seat = await this.stores.team.getOrgSeat(user.id);
if (!user.email_confirmed && !seat) {
throw new HttpError(
403,
'Email must be confirmed before enabling 2FA.',
@@ -4142,6 +4471,18 @@ export class AuthController extends PuterController {
{ legacyCode: 'conflict' },
);
}
// Proves the caller holds the secret setup issued; a session alone
// can't turn 2FA on.
const { code } = req.body ?? {};
if (typeof code !== 'string' || !code)
throw new HttpError(400, 'Missing `code`', {
legacyCode: 'bad_request',
});
if (!verifyOtp(user.username, user.otp_secret, code)) {
throw new HttpError(400, 'Incorrect code.', {
legacyCode: 'code_mismatch',
});
}
await this.clients.db.write(
'UPDATE `user` SET `otp_enabled` = ? WHERE `uuid` = ?',
@@ -4179,6 +4520,15 @@ export class AuthController extends PuterController {
legacyCode: 'not_found',
});
const seat = await this.stores.team.getOrgSeat(user.id);
if (seat && Number(seat.require_2fa) === 1) {
throw new HttpError(
409,
'Your team requires two-factor authentication, so it cannot be turned off.',
{ legacyCode: 'conflict' },
);
}
await this.clients.db.write(
'UPDATE `user` SET `otp_enabled` = ?, `otp_recovery_codes` = NULL, `otp_secret` = NULL WHERE `uuid` = ?',
[this.clients.db.booleanValue(false), user.uuid],
@@ -4399,6 +4749,25 @@ export class AuthController extends PuterController {
{ legacyCode: 'forbidden' },
);
}
// The other direction: deleting an owner sets `owner_user_id` NULL and
// cascades their membership, leaving a team no one can administer —
// not even an admin, since every team route resolves by membership.
if (await this.stores.team.countOwned(userId)) {
throw new HttpError(
403,
'Delete the teams you own before closing this account.',
{ legacyCode: 'forbidden' },
);
}
// A deleted team's seats are suspended, not gone, and only its owner
// can retire them.
if (await this.stores.team.countOwnedTeamsWithAccounts(userId)) {
throw new HttpError(
403,
'Close the accounts your teams created before closing this one.',
{ legacyCode: 'forbidden' },
);
}
res.clearCookie(this.config.cookie_name ?? 'puter_token');
res.clearCookie('puter_token_v2');
res.clearCookie('puter_revalidation');
@@ -4413,7 +4782,7 @@ export class AuthController extends PuterController {
//
// The `@Controller('')` decorator would normally install a default
// `registerRoutes` walker that iterates `prototype[__puterRoutes]`.
// We override it here so we can ALSO wire the five
// We override it here so we can ALSO wire the
// `/user-protected/*` (and `/user-protected/delete-own-user`) routes
// whose `middleware: createUserProtectedGate(...)` argument is
// built from instance state — not expressible inside a static
@@ -4487,12 +4856,7 @@ export class AuthController extends PuterController {
{
requireUserActor: true,
requireVerified: true,
rateLimit: {
scope: 'change-username',
limit: 2,
window: 30 * 24 * 60 * 60_000,
key: 'user',
},
rateLimit: CHANGE_USERNAME_ATTEMPT_LIMIT,
middleware: [
createUserProtectedGate(
userProtectedDeps as never,
@@ -4540,6 +4904,27 @@ export class AuthController extends PuterController {
(req, res) => this.handleDisable2fa(req, res),
);
router.post(
'/user-protected/setup-2fa',
{
requireUserActor: true,
// A member owing their team's 2FA reaches nothing else until this.
allowUnconfirmed: true,
rateLimit: {
scope: 'setup-2fa',
limit: 10,
window: 60 * 60_000,
key: 'user',
},
middleware: [
createUserProtectedGate(
userProtectedDeps as never,
) as unknown as RequestHandler,
],
},
(req, res) => this.handleSetup2fa(req, res),
);
router.post(
'/user-protected/delete-own-user',
{
@@ -179,6 +179,28 @@ const throwing = (payload: unknown) => () => {
// -- Upstream status extraction --------------------------------------
describe('DriverController upstream error translation', () => {
it('maps an upstream 402 to sanitized credit exhaustion', async () => {
const { err } = await callWith(
throwing(
Object.assign(
new Error(
'<html>Insufficient credits. Add more using https://openrouter.ai/settings/credits (request id: req-secret)</html>',
),
{ status: 402 },
),
),
);
expect(err).toMatchObject({
statusCode: 503,
legacyCode: 'upstream_credits_exhausted',
message: 'AI provider out of credits',
fields: { upstreamStatus: 402 },
});
expect(JSON.stringify(err)).not.toContain('http');
expect(JSON.stringify(err)).not.toMatch(/request id|<html>/i);
});
it('maps an upstream 429 to a Puter 429 with upstream_rate_limited', async () => {
const { err } = await callWith(
throwing(
@@ -21,6 +21,7 @@ import { metrics } from '@opentelemetry/api';
import type { Request, Response } from 'express';
import { actorUid } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { abortOnDisconnect } from '../../core/http/abortOnDisconnect.js';
import { Controller } from '../../core/http/decorators.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import { assertNotUserSession } from '../../core/http/middleware/gates.js';
@@ -31,7 +32,11 @@ import {
} from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { DriverMeta } from '../../drivers/meta.js';
import { isUpstreamTimeoutError } from '../../drivers/util/upstreamErrors.js';
import {
isCreditExhaustion,
isUpstreamTimeoutError,
sanitizeUpstreamMessage,
} from '../../drivers/util/upstreamErrors.js';
import {
isDriverStreamResult,
resolveCallableMethods,
@@ -120,6 +125,18 @@ const translateProviderError = (err: unknown): unknown => {
cause?: unknown;
};
const status = extractUpstreamStatus(e);
const msg = sanitizeUpstreamMessage(
e.error?.message ?? e.message ?? 'Upstream provider error',
);
const upstreamCode = e.error?.code ?? e.code;
const fields = { upstreamStatus: status, upstreamCode };
if (isCreditExhaustion(status, upstreamCode, msg)) {
return new HttpError(503, 'AI provider out of credits', {
legacyCode: 'upstream_credits_exhausted',
fields,
});
}
if (typeof status !== 'number') {
if (isUpstreamTimeoutError(e)) {
const cause = e.cause as { code?: string } | undefined;
@@ -132,10 +149,6 @@ const translateProviderError = (err: unknown): unknown => {
return err;
}
const msg = e.error?.message ?? e.message ?? 'Upstream provider error';
const upstreamCode = e.error?.code ?? e.code;
const fields = { upstreamStatus: status, upstreamCode };
if (status === 429) {
return new HttpError(429, msg, {
legacyCode: 'upstream_rate_limited',
@@ -313,7 +326,8 @@ export class DriverController extends PuterController {
if (req.actor) {
const permService = this.services.permission as unknown as
PermissionService | undefined;
| PermissionService
| undefined;
if (permService) {
// Build via PermissionUtil.join so any `:` in a driver or
// interface name is escaped — raw interpolation would let a
@@ -447,12 +461,7 @@ export class DriverController extends PuterController {
// A caller that hangs up mid-call gets nothing back, so long-running
// drivers watch this to stop working (and metering) as soon as it does.
// `close` after `finish` is the normal end of a response, not an abort.
const abort = new AbortController();
res.once('close', () => {
if (!res.writableFinished) abort.abort();
});
Context.set('abortSignal', abort.signal);
Context.set('abortSignal', abortOnDisconnect(res));
// Per-method lifecycle events, scoped to `driver.<iface>.<method>`.
// Subscribers can listen on `driver.*`, `driver.<iface>.*`, or the
@@ -287,12 +287,14 @@ export class EventsController extends PuterController {
// -- Events workers ------------------------------------------------
//
// The billable artifact a published handler set implies. Account-scoped
// like the kv-handle surface above: this is the owner's own view of what
// it is paying for, so — unlike the handler routes — an app token cannot
// act here on its owner's behalf.
// The billable artifact a published handler set implies. An account
// session or full-access token sees every app it owns; an app sees only
// its own worker, the same scope publish and destroy use.
/** GET /events/workers — the caller's own events workers, one per app. */
/**
* GET /events/workers — an account's events workers, one per app, or an
* app's own worker (0 or 1 item).
*/
@Get('/workers', {
subdomain: 'api',
requireAuth: true,
@@ -36,6 +36,8 @@ import {
EVENTS_DURABLE_SUBSCRIPTIONS_PER_USER,
EVENTS_KV_HANDLES_PER_APP,
EVENTS_KV_HANDLES_PER_USER,
EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT,
EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT,
EVENTS_SINGLE_DELIVERY_LIMIT,
EVENTS_WORKER_INVOCATION_LIMIT,
limitFor,
@@ -119,6 +121,34 @@ describe('the tiered subscription quotas', () => {
});
describe('the delivery budgets', () => {
it('sets KV fan-out and filter work by the namespace owner tier', () => {
for (const [plan, matched, evaluated] of [
[null, 512, 2048],
[DEFAULT_FREE_SUBSCRIPTION, 128, 512],
[DEFAULT_TEMP_SUBSCRIPTION, 128, 512],
[ORG_SEAT_FREE_SUBSCRIPTION, 128, 512],
] as const) {
expect(
limitFor(EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT, plan),
).toBe(matched);
expect(
limitFor(EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT, plan),
).toBe(evaluated);
expect(evaluated).toBeGreaterThanOrEqual(matched);
}
});
it('pins the share-handle quotas by plan', () => {
expect(EVENTS_KV_HANDLES_PER_USER.limit).toBe(512);
expect(EVENTS_KV_HANDLES_PER_APP.limit).toBe(512);
expect(
limitFor(EVENTS_KV_HANDLES_PER_USER, DEFAULT_FREE_SUBSCRIPTION),
).toBe(200);
expect(
limitFor(EVENTS_KV_HANDLES_PER_APP, DEFAULT_FREE_SUBSCRIPTION),
).toBe(128);
});
it('keeps `single` well under broadcast — each one costs far more', () => {
expect(EVENTS_SINGLE_DELIVERY_LIMIT.limit).toBeLessThan(
EVENTS_BROADCAST_DELIVERY_LIMIT.limit,
+45 -20
View File
@@ -21,7 +21,7 @@ import type { RouteRateLimit } from '../../core/http/types';
import {
DEFAULT_FREE_SUBSCRIPTION,
DEFAULT_TEMP_SUBSCRIPTION,
FREE_SUBSCRIPTION_IDS,
subscriptionOverride,
} from '../../services/metering/consts.js';
// -- Shared event limits ---------------------------------------------
@@ -45,7 +45,7 @@ const userWindow = (
/**
* A plan-varying cap in route-gate shape: `limit` is what a subscribed account
* gets, `bySubscription` carves out the free tiers, unlisted plans get the
* gets, `bySubscription` carves out the free tiers, unlisted paid plans get the
* base.
*/
export interface TieredLimit {
@@ -67,11 +67,9 @@ export const limitFor = (
subscriptionId: string | null,
): number => {
if (subscriptionId === null) return tier.limit;
const own = tier.bySubscription[subscriptionId];
if (typeof own === 'number') return own;
return FREE_SUBSCRIPTION_IDS.has(subscriptionId)
? (tier.bySubscription[DEFAULT_FREE_SUBSCRIPTION] ?? tier.limit)
: tier.limit;
return (
subscriptionOverride(tier.bySubscription, subscriptionId) ?? tier.limit
);
};
/**
@@ -128,14 +126,20 @@ export const EVENTS_DURABLE_SUBSCRIPTIONS_MAX =
export const SUSPENDED_ROW_TTL_DAYS = 30;
/**
* Subscribe + unsubscribe calls per minute, per user.
* Subscribe calls per minute, per user.
*
* Both resolve a path and take a write, so a loop over them is a write loop.
* Resolves a path and takes a write, so a loop over subscribes is a write loop.
* Sized like the sharing verbs, which are the closest existing analogue: a
* client sets its subscriptions up once and then leaves them alone.
*/
export const EVENTS_SUBSCRIBE_LIMIT = userWindow('events:subscribe', 60);
/**
* Unsubscribe calls per minute, per user — its own budget, never the subscribe
* one.
*/
export const EVENTS_UNSUBSCRIBE_LIMIT = userWindow('events:unsubscribe', 600);
/**
* Subscription listings per minute, per user. Reads an index and returns one
* page, so it is budgeted well above the verbs that write.
@@ -167,16 +171,15 @@ export const EVENTS_KV_HANDLE_LIMIT = userWindow('events:kvHandles', 60);
* grow forever. A temporary account holds none: the grant outlives the session
* that made it, and no account is left to take it back.
*/
export const EVENTS_KV_HANDLES_PER_USER = tiered(500, 200, 0);
export const EVENTS_KV_HANDLES_PER_USER = tiered(512, 200, 0);
/**
* Live share handles one app may hold out for one account.
*
* Below the per-account cap so that one app cannot spend an account's whole
* budget: a handle names one namespace, so without this a second app on the
* same account would be left with nothing.
* Separately bounded from the account cap so one namespace cannot consume the
* whole budget on free accounts.
*/
export const EVENTS_KV_HANDLES_PER_APP = tiered(100, 50, 0);
export const EVENTS_KV_HANDLES_PER_APP = tiered(512, 128, 0);
// -- Handler surface -------------------------------------------------
@@ -256,17 +259,30 @@ export const EVENTS_WORKER_LIST_LIMIT = userWindow('events:workers:list', 120);
// -- Dispatch fan-out ------------------------------------------------
/**
* Subscriptions one event may deliver to before dispatch stops and reports a
* gap. The amplification ceiling: without it, one write costs as many
* Subscriptions one non-KV event may deliver to before dispatch stops and
* reports a gap. The amplification ceiling: without it, one write costs as many
* deliveries as an account cared to register.
*/
export const EVENTS_MATCHED_SUBSCRIPTIONS_PER_EVENT = 50;
/**
* Largest key-value value a delivery inlines, in serialized bytes. A value over
* this is left out and the subscriber re-reads the key: a delivery fans out to
* many rows, may cross regions and may sit in a backlog, none of which is sized
* for the store's own ceiling.
* Key-value subscriptions one mutation may deliver to. A namespace owner's plan
* sets this ceiling; the listeners' plans do not affect it.
*/
export const EVENTS_KV_MATCHED_SUBSCRIPTIONS_PER_EVENT = tiered(512, 128, 128);
/**
* Filter evaluations a key-value mutation may spend. This leaves room for
* non-matching rows while retaining a finite bound on dispatch work.
*/
export const EVENTS_KV_FILTER_EVALUATIONS_PER_EVENT = tiered(2048, 512, 512);
/** Omit all inline KV values when the matched audience exceeds this count. */
export const EVENTS_KV_VALUE_OMIT_MATCHED_SUBSCRIPTIONS = 128;
/**
* Largest inline KV value in serialized bytes. Larger values are omitted;
* deliveries retain the key metadata.
*/
export const EVENTS_KV_VALUE_MAX_BYTES = 16 * 1024;
@@ -309,6 +325,15 @@ export const EVENTS_WORKER_INVOCATION_LIMIT = userWindow(
60,
);
/**
* Handler runs one chain of events may take, by the holder's plan.
*
* A write made with a handler's token is one run deeper than the event that
* invoked it. An event at this depth still reaches sockets but runs no handler,
* which is what ends handlers whose writes trigger each other.
*/
export const EVENTS_HANDLER_DEPTH = tiered(12, 4, 4);
/**
* Filter evaluations one event may spend. Lives with the matcher because the
* primitive that enforces it does; re-exported here so every published number
@@ -54,12 +54,7 @@ import { EVENTS_WORKER_DEPLOYS_PER_HOUR } from './limits.js';
/** Why a deploy could not happen, for the callers that answer differently. */
export type EventsDeployOutcome =
| 'deployed'
| 'stale'
| 'no-handlers'
| 'no-owner'
| 'throttled'
| 'failed';
'deployed' | 'stale' | 'no-handlers' | 'no-owner' | 'throttled' | 'failed';
interface DeployLayers {
config: IConfig;
@@ -353,4 +353,199 @@ describe('GET /fs/readdir over HTTP', () => {
expect(stat.shares).toEqual([]);
});
});
describe('the root listing', () => {
const mint = async (permissions: string[]) => {
const response = await fetch(
new URL('/auth/create-access-token', env.apiOrigin),
{
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${env.users.user.token}`,
},
body: JSON.stringify({ permissions }),
},
);
expect(response.status).toBe(200);
return ((await response.json()) as { token: string }).token;
};
const listRoot = async (token: string) => {
const response = await fetch(
readdirUrl({ path: '/', auth_token: token }),
);
expect(response.status).toBe(200);
return response.text();
};
const rootPaths = async (token: string) => {
const entries = JSON.parse(await listRoot(token)) as Array<{
path: string;
}>;
return entries.map((entry) => entry.path);
};
const entryAt = async (path: string) =>
(await env.server.stores.fsEntry.getEntryByPath(path))!;
it('lists the home for a session and a full-access token', async () => {
const { username, token, apiToken } = env.users.user;
for (const credential of [token, apiToken]) {
expect(await rootPaths(credential)).toEqual([`/${username}`]);
}
});
it('leaves the home out for a token scoped below it', async () => {
const { username } = env.users.user;
const home = await entryAt(`/${username}`);
const documents = await entryAt(`/${username}/Documents`);
const scoped = await mint([`fs:${documents.uuid}:read`]);
const raw = await listRoot(scoped);
expect(JSON.parse(raw)).toEqual([]);
expect(raw).not.toContain(home.uuid);
expect(raw).not.toContain(username);
});
it('lists the home for a token that can list it', async () => {
const { username } = env.users.user;
const home = await entryAt(`/${username}`);
const scoped = await mint([`fs:${home.uuid}:list`]);
expect(await rootPaths(scoped)).toEqual([`/${username}`]);
});
});
});
describe('POST /fs/completeWrite over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
it('refuses completion of a session whose object never arrived', async () => {
const { username, token } = env.users.user;
const started = await fetch(new URL('/fs/startWrite', env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify({
fileMetadata: {
path: `/${username}/Documents/http-nobytes-${Date.now()}.bin`,
size: 4,
},
}),
});
expect(started.status).toBe(200);
const { sessionId } = (await started.json()) as { sessionId: string };
// No PUT to the presigned URL.
const response = await fetch(
new URL('/fs/completeWrite', env.apiOrigin),
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify({ uploadId: sessionId }),
},
);
expect(response.status).toBe(400);
const body = (await response.json()) as {
code?: string;
message?: string;
};
expect(body.code).toBe('bad_request');
expect(body.message).toBe('Upload content was not received');
});
});
describe('POST /fs/startWrite storage quota over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv({
is_storage_limited: true,
storage_capacity: 10 * 1024,
} as never);
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
it('rejects the excess of a parallel burst of startWrite calls', async () => {
const { username, token } = env.users.user;
const user = await env.server.stores.user.getByUsername(username);
const fs = env.server.services.fs as unknown as {
getUsersStorageAllowance: (
userId: number,
) => Promise<{ curr: number; max: number }>;
};
const { curr, max } = await fs.getUsersStorageAllowance(user!.id);
const size = Math.floor((max - curr) / 3) + 1;
const responses = await Promise.all(
Array.from({ length: 8 }, (_, i) =>
fetch(new URL('/fs/startWrite', env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify({
fileMetadata: {
path: `/${username}/Documents/http-quota-burst-${i}.bin`,
size,
},
}),
}),
),
);
const ok = responses.filter((response) => response.status === 200);
const rejected = responses.filter(
(response) => response.status === 413,
);
expect(ok).toHaveLength(2);
expect(rejected).toHaveLength(6);
for (const response of rejected) {
const body = (await response.json()) as { code?: string };
expect(body.code).toBe('storage_limit_reached');
}
});
// A raw JSON body carries `1e400` as a numeral, not a computed value, so
// `JSON.stringify` can't produce it (it turns `Infinity` into `null`) —
// the wire text is built by hand to exercise what the server's own
// `JSON.parse` does with it.
it('rejects a declared size of 1e400 (parses to Infinity) with 400', async () => {
const { username, token } = env.users.user;
const body = `{"fileMetadata":{"path":"/${username}/Documents/http-huge.bin","size":1e400}}`;
const response = await fetch(new URL('/fs/startWrite', env.apiOrigin), {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body,
});
expect(response.status).toBe(400);
const responseBody = (await response.json()) as { code?: string };
expect(responseBody.code).toBe('bad_request');
});
});
@@ -71,7 +71,7 @@ const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
actor: {
actor: makeActor({
user: {
id: refreshed.id,
uuid: refreshed.uuid,
@@ -79,7 +79,7 @@ const makeUser = async (): Promise<{ actor: Actor; userId: number }> => {
email: refreshed.email ?? null,
email_confirmed: true,
} as Actor['user'],
},
}),
};
};
@@ -342,9 +342,16 @@ describe('FSController.completeBatchWrites', () => {
.body as ClientSignedWriteResponse[];
expect(startResponses).toHaveLength(2);
// 2) Complete via the controller. Single-mode completion only
// needs the session row → it doesn't read the S3 object back,
// so we can skip the actual upload step in this test.
// 2) PUT the declared bytes through each presigned URL, then
// complete via the controller.
await fetch(startResponses[0]!.url!, {
method: 'PUT',
body: '12345',
});
await fetch(startResponses[1]!.url!, {
method: 'PUT',
body: '1234567',
});
const { res, captured } = makeRes();
const completeBody: CompleteWriteRequest[] = startResponses.map(
(r) => ({ uploadId: r.sessionId }),
@@ -416,6 +423,7 @@ describe('FSController.completeBatchWrites', () => {
);
const [firstResponse] = firstStart.captured
.body as ClientSignedWriteResponse[];
await fetch(firstResponse!.url!, { method: 'PUT', body: 'x' });
const firstComplete = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
@@ -547,6 +555,7 @@ describe('FSController.completeBatchWrites', () => {
);
const [firstResponse] = firstStart.captured
.body as ClientSignedWriteResponse[];
await fetch(firstResponse!.url!, { method: 'PUT', body: 'x' });
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
@@ -834,6 +843,90 @@ describe('FSController.statEntry', () => {
);
expect((captured.body as { name: string }).name).toBe('share-budget');
});
describe('parentUid of a home child', () => {
const tokenActorFor = async (userId: number, permission: string) => {
const user = (await server.stores.user.getById(userId))!;
const token = await server.services.auth.createAccessToken(
makeActor({ user }),
[[permission]],
);
return (await server.services.auth.authenticateFromToken(token))!;
};
it('nulls it for a token scoped below the home', async () => {
const { userId } = await makeUser();
const user = (await server.stores.user.getById(userId))!;
const username = user.username!;
const documents = (await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
))!;
const scoped = await tokenActorFor(
userId,
`fs:${documents.uuid}:read`,
);
const { res, captured } = makeRes();
await withActor(scoped, () =>
controller.statEntry(
makeReq({
body: { path: `/${username}/Documents` },
actor: scoped,
}),
res,
),
);
expect((captured.body as { parentUid: unknown }).parentUid).toBe(
null,
);
});
it('keeps it for a token that can list the home', async () => {
const { userId } = await makeUser();
const user = (await server.stores.user.getById(userId))!;
const username = user.username!;
const home = (await server.stores.fsEntry.getEntryByPath(
`/${username}`,
))!;
const scoped = await tokenActorFor(userId, `fs:${home.uuid}:list`);
const { res, captured } = makeRes();
await withActor(scoped, () =>
controller.statEntry(
makeReq({
body: { path: `/${username}/Documents` },
actor: scoped,
}),
res,
),
);
expect((captured.body as { parentUid: unknown }).parentUid).toBe(
home.uuid,
);
});
it('keeps it for a session actor', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const home = (await server.stores.fsEntry.getEntryByPath(
`/${username}`,
))!;
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.statEntry(
makeReq({
body: { path: `/${username}/Documents` },
actor,
}),
res,
),
);
expect((captured.body as { parentUid: unknown }).parentUid).toBe(
home.uuid,
);
});
});
});
// ── /readdir (readdirEntries) ───────────────────────────────────────
@@ -2731,6 +2824,7 @@ describe('FSController associatedAppId entitlement gate', () => {
),
);
const [started] = startRes.captured.body as ClientSignedWriteResponse[];
await fetch(started.url!, { method: 'PUT', body: 'abc' });
const completeRes = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
+72 -8
View File
@@ -25,6 +25,7 @@ import type { Actor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import { clientParentUid } from '../../services/fs/rootListing.js';
import {
expandTildePath,
isOwnersTrash,
@@ -52,7 +53,9 @@ import { consumeRouteRateLimit } from '../../core/http/middleware/rateLimit.js';
import { PuterController } from '../types.js';
import { STORAGE_OP_COSTS } from '../../services/metering/costs.js';
import {
FS_BATCH_WRITE_MAX_ITEMS,
FS_MULTIPART_LIMIT,
FS_MULTIPART_MAX_PARTS,
FS_MUTATE_LIMIT,
FS_READ_CONCURRENT,
FS_READ_LIMIT,
@@ -222,6 +225,7 @@ export class FSController extends PuterController {
const userId = this.#getActorUserId(req);
const storageAllowanceMax = this.#getStorageAllowanceMaxOverride(req);
const appUidLookupCache = new Map<string, Promise<number | null>>();
this.#assertBatchItemCount(req.body);
const requests = Array.isArray(req.body)
? await Promise.all(
req.body.map(async (requestBody) => {
@@ -367,6 +371,7 @@ export class FSController extends PuterController {
res: Response<ClientCompleteWriteResponse[]>,
) {
const userId = this.#getActorUserId(req);
this.#assertBatchItemCount(req.body);
const requests = Array.isArray(req.body)
? req.body.map((requestBody) => {
return this.#withGuiMetadata(
@@ -443,6 +448,17 @@ export class FSController extends PuterController {
res: Response<ClientSignMultipartPartsResponse>,
) {
const userId = this.#getActorUserId(req);
const partNumbers = req.body?.partNumbers;
if (
Array.isArray(partNumbers) &&
partNumbers.length > FS_MULTIPART_MAX_PARTS
) {
throw new HttpError(
400,
`Too many partNumbers in one request (max ${FS_MULTIPART_MAX_PARTS})`,
{ legacyCode: 'bad_request' },
);
}
await this.#assertUploadSessionWriteAccess(req, userId, [
req.body?.uploadId,
]);
@@ -863,6 +879,7 @@ export class FSController extends PuterController {
return;
}
this.#assertBatchItemCount(req.body);
const requests = Array.isArray(req.body)
? await Promise.all(
req.body.map(async (requestBody) => {
@@ -1015,7 +1032,7 @@ export class FSController extends PuterController {
legacyCode: 'too_many_requests',
});
}
const [subtreeSize, suggestedApps, shareFlags, shares] =
const [subtreeSize, suggestedApps, shareFlags, shares, parentUid] =
await Promise.all([
entry.isDir && wantsSize
? this.services.fs.getSubtreeSize(userId, entry.path)
@@ -1030,11 +1047,21 @@ export class FSController extends PuterController {
entry.uuid,
)
: undefined,
clientParentUid(
actor,
entry,
this.services.acl,
this.stores.permission,
),
]);
entry.suggestedApps = suggestedApps;
res.json({
...this.#toClientEntry(entry, shareFlags.get(entry.uuid) ?? null),
...this.#toClientEntry(
entry,
shareFlags.get(entry.uuid) ?? null,
parentUid,
),
...(subtreeSize !== undefined ? { size: subtreeSize } : {}),
...(shares !== undefined ? { shares } : {}),
});
@@ -1047,8 +1074,16 @@ export class FSController extends PuterController {
* callers who only hold `see`/`list` on the entry — a share recipient, or
* (with public folders enabled) any authenticated user. The legacy read
* path already curates its output; this does the same for the v2 routes.
*
* `parentUid` is `entry.parentUid` unless `parentUidOverride` is passed
* (even as `null`) — callers reading a scoped access token's home pass the
* result of `clientParentUid` to hide it there.
*/
#toClientEntry(entry: FSEntry, isShared?: boolean | null): ClientFSEntry {
#toClientEntry(
entry: FSEntry,
isShared?: boolean | null,
parentUidOverride?: string | null,
): ClientFSEntry {
// Allowlist, not a denylist: a denylist silently ships every column
// added to `fsentries` later. Omits the numeric primary keys (`id`,
// `parentId`, `associatedAppId`), the storage columns, the owning
@@ -1060,7 +1095,10 @@ export class FSController extends PuterController {
return {
uuid: entry.uuid,
uid: entry.uid ?? entry.uuid,
parentUid: entry.parentUid ?? null,
parentUid:
parentUidOverride !== undefined
? parentUidOverride
: (entry.parentUid ?? null),
path: maskEntryPath(entry),
name: entry.name,
isDir: entry.isDir,
@@ -1181,6 +1219,8 @@ export class FSController extends PuterController {
const rootChildren = await listRootEntries(
actor,
this.stores.fsEntry,
this.services.acl,
this.stores.permission,
);
const rootSuggestions =
await this.services.suggestedApps.getSuggestedAppsForEntries(
@@ -1322,6 +1362,12 @@ export class FSController extends PuterController {
...this.#toClientEntry(
entry,
shareFlags.get(entry.uuid) ?? null,
await clientParentUid(
actor,
entry,
this.services.acl,
this.stores.permission,
),
),
// Fields the client cannot derive on its own.
type: fsEntryMimeType(entry),
@@ -2382,6 +2428,16 @@ export class FSController extends PuterController {
return guiMetadata;
}
#assertBatchItemCount(items: unknown): void {
if (Array.isArray(items) && items.length > FS_BATCH_WRITE_MAX_ITEMS) {
throw new HttpError(
400,
`Too many items in one request (max ${FS_BATCH_WRITE_MAX_ITEMS})`,
{ legacyCode: 'bad_request' },
);
}
}
/**
* The write handlers build on `req.body` being an object. A request whose
* body never parsed leaves it undefined, and the first field read after
@@ -2596,8 +2652,9 @@ export class FSController extends PuterController {
};
}
/** `forOwner` — these events go to the owner, not to the acting user. */
async #toGuiFsEntry(entry: FSEntry): Promise<Record<string, unknown>> {
return toLegacyEntry(this.clients.event, entry);
return toLegacyEntry(this.clients.event, entry, { forOwner: true });
}
async #emitGuiWriteEvent(
@@ -2703,10 +2760,13 @@ export class FSController extends PuterController {
return fsEntry;
}
const thumbnailPayload = { url: requestedThumbnail };
const thumbnailPayload = {
url: requestedThumbnail,
uuid: fsEntry.uuid,
};
// emitAndWait — the thumbnails extension may rewrite `url` from a
// data URL to an `s3://` pointer; plain `emit` races with the DB
// update below.
// data URL to an `s3://` pointer bound to `uuid`, or drop a pointer
// not minted for it; plain `emit` races with the DB update below.
await this.clients.event.emitAndWait(
'thumbnail.created',
thumbnailPayload,
@@ -2795,6 +2855,9 @@ export class FSController extends PuterController {
index: item.index,
contentType: item.contentType,
...(item.size !== undefined ? { size: item.size } : {}),
// The entry the upload lands on; its thumbnail key is
// bound to it.
item_uid: responses[item.index]?.objectKey ?? '',
}) as ThumbnailUploadPrepareItem,
),
};
@@ -3018,6 +3081,7 @@ export class FSController extends PuterController {
{ legacyCode: 'bad_request' },
);
}
this.#assertBatchItemCount(manifest.items);
const manifestGuiMetadata = this.#extractGuiMetadata(
manifest,
@@ -28,6 +28,7 @@ import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { FSController } from './FSController.js';
import { FS_BATCH_WRITE_MAX_ITEMS, FS_MULTIPART_MAX_PARTS } from './limits.js';
import type {
ClientSignedWriteResponse,
CompleteWriteRequest,
@@ -166,8 +167,18 @@ const buildMultipartBody = (parts: MultipartPart[]): Buffer => {
return Buffer.from(chunks.join(''), 'utf8');
};
const makeMultipartReq = (parts: MultipartPart[], actor: Actor): Request => {
const stream = Readable.from([buildMultipartBody(parts)]);
const makeMultipartReq = (
parts: MultipartPart[],
actor: Actor,
chunkSize?: number,
): Request => {
const body = buildMultipartBody(parts);
const chunks: Buffer[] = [];
const step = chunkSize ?? body.length;
for (let i = 0; i < body.length; i += step) {
chunks.push(body.subarray(i, i + step));
}
const stream = Readable.from(chunks);
return Object.assign(stream, {
body: undefined,
query: {},
@@ -228,6 +239,46 @@ describe('FSController.write', () => {
expect(stored?.size).toBe(5);
});
it('emits an upload.progress socket event reaching loaded === total', async () => {
const { actor, userId, username } = await makeUser();
const target = `/${username}/Documents/write-progress.txt`;
const content = 'progress-bytes';
const send = vi.spyOn(server.services.socket, 'send');
const { res } = makeRes();
try {
await withActor(actor, () =>
controller.write(
makeReq<WriteRequest>({
body: {
fileMetadata: {
path: target,
size: content.length,
contentType: 'text/plain',
},
fileContent: content,
} as WriteRequest,
actor,
}),
res,
),
);
expect(send).toHaveBeenCalledWith(
{ room: userId },
'upload.progress',
expect.objectContaining({
total: content.length,
loaded: content.length,
loaded_diff: content.length,
item_path: target,
}),
);
} finally {
send.mockRestore();
}
});
it('reports wasOverwrite and updates the row when overwriting', async () => {
const { actor, username } = await makeUser();
const target = `/${username}/Documents/write-overwrite.txt`;
@@ -611,6 +662,60 @@ describe('batch write handlers with a malformed element', () => {
});
});
describe('batch write handlers at the item cap', () => {
// Null elements fail the per-item body check, so which 400 comes back
// shows whether the cap ran before any item was read.
const handlers: Array<
[string, (c: typeof controller, r: Request, res: Response) => unknown]
> = [
[
'startBatchWrites',
(c, r, res) => c.startBatchWrites(r as never, res as never),
],
['batchWrites', (c, r, res) => c.batchWrites(r as never, res as never)],
[
'completeBatchWrites',
(c, r, res) => c.completeBatchWrites(r as never, res as never),
],
];
const callWith = async (
call: (typeof handlers)[number][1],
length: number,
) => {
const { actor } = await makeUser();
const req = makeReq({ actor });
(req as { body?: unknown }).body = Array(length).fill(null);
const { res } = makeRes();
return withActor(actor, async () => call(controller, req, res));
};
it.each(handlers)(
'%s rejects one item over the cap up front',
async (_label, call) => {
await expect(
callWith(call, FS_BATCH_WRITE_MAX_ITEMS + 1),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
message: `Too many items in one request (max ${FS_BATCH_WRITE_MAX_ITEMS})`,
});
},
);
it.each(handlers)(
'%s lets exactly the cap through to the per-item checks',
async (_label, call) => {
await expect(
callWith(call, FS_BATCH_WRITE_MAX_ITEMS),
).rejects.toMatchObject({
statusCode: 400,
message: 'A request body is required',
});
},
);
});
describe('FSController.write storage allowance', () => {
let limitedServer: PuterServer;
let limitedController: FSController;
@@ -807,15 +912,18 @@ describe('FSController.startWrite', () => {
it('attaches signed thumbnail upload targets published by a listener', async () => {
const { actor, username } = await makeUser();
const itemUids: string[] = [];
const listener = (_key: string, data: unknown) => {
const payload = data as {
items: Array<{
index: number;
item_uid: string;
uploadUrl?: string;
thumbnailUrl?: string;
}>;
};
for (const item of payload.items) {
itemUids.push(item.item_uid);
item.uploadUrl = `https://thumbs.test/put/${item.index}`;
item.thumbnailUrl = `https://thumbs.test/get/${item.index}`;
}
@@ -844,6 +952,13 @@ describe('FSController.startWrite', () => {
const body = captured.body as ClientSignedWriteResponse;
expect(body.thumbnailUploadUrl).toBe('https://thumbs.test/put/0');
expect(body.thumbnailUrl).toBe('https://thumbs.test/get/0');
// Listeners are told the entry the upload will complete into, so
// the thumbnail can be bound to it.
const session =
await server.stores.fsEntry.getPendingEntryBySessionId(
body.sessionId,
);
expect(itemUids).toEqual([session?.objectKey]);
} finally {
server.clients.event.off(
'thumbnail.upload.prepare',
@@ -1024,6 +1139,7 @@ describe('FSController.completeWrite', () => {
const { actor, username } = await makeUser();
const target = `/${username}/Documents/complete-single.txt`;
const started = await startSignedWrite(actor, target, 4);
await fetch(started.url!, { method: 'PUT', body: 'wxyz' });
const { res, captured } = makeRes();
await withActor(actor, () =>
@@ -1073,6 +1189,7 @@ describe('FSController.completeWrite', () => {
`/${username}/Documents/complete-thumb.txt`,
2,
);
await fetch(started.url!, { method: 'PUT', body: 'yz' });
const { res } = makeRes();
await withActor(actor, () =>
controller.completeWrite(
@@ -1092,6 +1209,38 @@ describe('FSController.completeWrite', () => {
expect(stored?.thumbnail).toBe('https://thumbs.test/x.png');
});
it('tells thumbnail listeners which entry the thumbnail is for', async () => {
const { actor, username } = await makeUser();
const target = `/${username}/Documents/complete-bound-thumb.txt`;
const started = await startSignedWrite(actor, target, 2);
await fetch(started.url!, { method: 'PUT', body: 'yz' });
const seen: unknown[] = [];
const listener = (_key: string, data: unknown) => {
seen.push(data);
};
server.clients.event.on('thumbnail.created', listener as never);
try {
const { res } = makeRes();
await withActor(actor, () =>
controller.completeWrite(
makeReq<CompleteWriteRequest>({
body: {
uploadId: started.sessionId,
thumbnailData: 'https://thumbs.test/x.png',
},
actor,
}),
res,
),
);
} finally {
server.clients.event.off('thumbnail.created', listener as never);
}
const stored = await server.stores.fsEntry.getEntryByPath(target);
expect(seen).toEqual([expect.objectContaining({ uuid: stored?.uuid })]);
});
it('404s an unknown upload id', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
@@ -1231,6 +1380,29 @@ describe('FSController.signMultipartParts', () => {
),
).rejects.toMatchObject({ statusCode: 404 });
});
// The session is unknown, so reaching the lookup would answer 404.
it.each([
[FS_MULTIPART_MAX_PARTS + 1, 400],
[FS_MULTIPART_MAX_PARTS, 404],
])('answers %i part numbers with %i', async (length, statusCode) => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.signMultipartParts(
makeReq<SignMultipartPartsRequest>({
body: {
uploadId: uuidv4(),
partNumbers: Array(length).fill(1),
},
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode });
});
});
// -- /fs/batchWrite (JSON mode) ---------------------------------------
@@ -1467,6 +1639,60 @@ describe('FSController.batchWrites (multipart)', () => {
expect(stored?.size).toBe(5);
});
it('coalesces streamed upload progress when the declared size is 0', async () => {
const { actor, userId, username } = await makeUser();
const path = `/${username}/Documents/mp-progress.bin`;
const content = 'x'.repeat(64 * 1024);
const chunkSize = 4 * 1024;
const send = vi.spyOn(server.services.socket, 'send');
const { res } = makeRes();
// Scoped to this user: earlier tests' trailing flushes can land here.
const progressSends = () =>
send.mock.calls.filter(
([spec, key]) =>
key === 'upload.progress' &&
(spec as { room?: unknown }).room === userId,
);
try {
await withActor(actor, () =>
controller.batchWrites(
makeMultipartReq(
[
{
kind: 'field',
name: 'manifest',
value: manifestFor([path]),
},
{
kind: 'file',
name: 'file-0',
filename: 'progress.bin',
content,
},
],
actor,
chunkSize,
),
res,
),
);
const sumDiffs = () =>
progressSends().reduce(
(sum, [, , data]) =>
sum + (data as { loaded_diff: number }).loaded_diff,
0,
);
await vi.waitFor(() => expect(sumDiffs()).toBe(content.length));
expect(progressSends().length).toBeLessThan(
content.length / chunkSize,
);
} finally {
send.mockRestore();
}
});
it('maps positional `file` parts onto manifest order', async () => {
const { actor, username } = await makeUser();
const paths = [`/${username}/Documents/mp-pos.txt`];
@@ -1610,6 +1836,35 @@ describe('FSController.batchWrites (multipart)', () => {
});
});
it('rejects a manifest over the item cap', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.batchWrites(
makeMultipartReq(
[
{
kind: 'field',
name: 'manifest',
value: JSON.stringify({
items: Array(
FS_BATCH_WRITE_MAX_ITEMS + 1,
).fill(null),
}),
},
],
actor,
),
res,
),
),
).rejects.toMatchObject({
statusCode: 400,
message: `Too many items in one request (max ${FS_BATCH_WRITE_MAX_ITEMS})`,
});
});
it('rejects a manifest item without fileMetadata', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
@@ -2089,6 +2344,7 @@ describe('FSController upload session access re-checks', () => {
const target = `${folder}/kept-complete.bin`;
const single = await startUpload(recipient.actor, target);
await fetch(single.url!, { method: 'PUT', body: '12345678' });
await withActor(recipient.actor, () =>
controller.completeWrite(
makeReq<CompleteWriteRequest>({
@@ -2113,6 +2369,9 @@ describe('FSController upload session access re-checks', () => {
await startUpload(recipient.actor, targets[0]!),
await startUpload(recipient.actor, targets[1]!),
];
for (const s of started) {
await fetch(s.url!, { method: 'PUT', body: '12345678' });
}
await withActor(recipient.actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
@@ -18,6 +18,7 @@
*/
import type { Request, RequestHandler, Response } from 'express';
import { createHash } from 'node:crypto';
import { Readable, Writable } from 'node:stream';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
@@ -534,6 +535,36 @@ describe('LegacyFSController.updateFsentryThumbnail', () => {
expect(row?.thumbnail).toBe(thumbnail);
});
it('tells thumbnail listeners which entry the thumbnail is for', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/bound-thumb.txt`;
const { uuid } = await writeFileEntry(actor, path, 'body');
const seen: unknown[] = [];
const listener = (_key: string, data: unknown) => {
seen.push(data);
};
server.clients.event.on('thumbnail.created', listener as never);
try {
const { res } = makeRes();
await withActor(actor, () =>
controller.updateFsentryThumbnail(
makeReq({
body: {
uid: uuid,
thumbnail: 'data:image/png;base64,aGVsbG8=',
},
actor,
}),
res,
),
);
} finally {
server.clients.event.off('thumbnail.created', listener as never);
}
expect(seen).toEqual([expect.objectContaining({ uuid })]);
});
it("refuses to retag another user's entry", async () => {
const owner = await makeUser();
const stranger = await makeUser();
@@ -735,6 +766,26 @@ describe('LegacyFSController.file (signed streaming)', () => {
);
});
// Metering resolves the billed account's plan from this actor, and some
// plans are keyed on the account's email.
it('bills an anonymous read to the owner, email included', async () => {
const { actor, userId, username } = await makeUser();
const { query } = await signedQuery(
actor,
`/${username}/Documents/signed-billed.txt`,
'billed',
);
const req = makeReq({ query });
const { res, finished } = makeStreamRes();
await controller.file(req, res);
await finished;
expect(req.egressActor?.user).toMatchObject({
id: userId,
uuid: actor.user.uuid,
email: `${username}@test.local`,
});
});
it('rejects a signature minted before the owner was suspended', async () => {
const { actor, userId, username } = await makeUser();
const { query } = await signedQuery(
@@ -755,6 +806,341 @@ describe('LegacyFSController.file (signed streaming)', () => {
});
});
// -- signed URLs across an ownership change ----------------------------
describe('LegacyFSController signed URLs across an ownership change', () => {
type Signed = { uid: string; read_url: string; write_url?: string };
const queryOf = (url: string) => {
const parsed = new URL(url);
return {
uid: parsed.searchParams.get('uid')!,
expires: parsed.searchParams.get('expires')!,
signature: parsed.searchParams.get('signature')!,
};
};
// No actor: a link holder needs no account.
const listSigned = async (url: string) => {
const { res, captured } = makeRes();
await controller.file(makeReq({ query: queryOf(url) }), res);
return captured.body as Array<Signed & { fsentry_name: string }>;
};
const readSigned = async (url: string) => {
const { res, captured, finished } = makeStreamRes();
await controller.file(makeReq({ query: queryOf(url) }), res);
await finished;
return captured.text();
};
const signAs = async (actor: Actor, path: string): Promise<Signed> => {
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.sign(
makeReq({
body: { items: [{ path, action: 'write' }] },
actor,
}),
res,
),
);
return (captured.body as { signatures: Signed[] }).signatures[0];
};
const moveAs = async (
actor: Actor,
body: { source: string; destination: string },
) =>
withActor(actor, () =>
controller.move(makeReq({ body, actor }), makeRes().res),
);
it('refuses the URL and the child URLs it minted once a write share hands the entry to another user', async () => {
const alice = await makeUser();
const bob = await makeUser();
const dir = `/${alice.username}/Documents/signed-dir`;
await writeFileEntry(alice.actor, `${dir}/a.txt`, 'alice-bytes');
const signed = await signAs(alice.actor, dir);
const [child] = await listSigned(signed.read_url);
expect(child.fsentry_name).toBe('a.txt');
expect(await readSigned(child.read_url)).toBe('alice-bytes');
// Bob lets Alice write into his folder, and moving hers there makes
// it Bob's. Then he revokes the share and adds a file of his own.
const inbox = await server.services.fs.mkdir(bob.userId, {
path: `/${bob.username}/Documents/inbox`,
});
const grant = `fs:${inbox.uuid}:write`;
await server.services.permission.grantUserUserPermission(
bob.actor,
alice.username,
grant,
{},
);
await moveAs(alice.actor, { source: dir, destination: inbox.path });
await server.services.permission.revokeUserUserPermission(
bob.actor,
alice.username,
grant,
);
const movedDir = `${inbox.path}/signed-dir`;
await writeFileEntry(bob.actor, `${movedDir}/bob.txt`, 'bob-bytes');
expect(
(await server.stores.fsEntry.getEntryByPath(movedDir))!.userId,
).toBe(bob.userId);
await expect(listSigned(signed.read_url)).rejects.toMatchObject({
statusCode: 403,
});
// Refused on the signature, even for a caller allowed to write there.
await expect(
withActor(bob.actor, () =>
controller.writeFile(
makeReq({
query: {
...queryOf(signed.write_url!),
operation: 'mkdir',
},
body: { name: 'via-old-url' },
actor: bob.actor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 403 });
// Moved descendants keep a cached owner until the entry cache expires.
await server.stores.fsEntry.invalidateEntryCacheByUuid(child.uid);
await expect(readSigned(child.read_url)).rejects.toMatchObject({
statusCode: 403,
});
});
it('keeps the URL working across moves and renames by the same owner', async () => {
const { actor, username } = await makeUser();
const dir = `/${username}/Documents/kept-dir`;
await writeFileEntry(actor, `${dir}/a.txt`, 'kept-bytes');
const signed = await signAs(actor, dir);
const [child] = await listSigned(signed.read_url);
await moveAs(actor, {
source: dir,
destination: `/${username}/Pictures`,
});
await withActor(actor, () =>
controller.rename(
makeReq({
body: { uid: signed.uid, new_name: 'renamed-dir' },
actor,
}),
makeRes().res,
),
);
const [listed] = await listSigned(signed.read_url);
expect(listed.fsentry_name).toBe('a.txt');
expect(await readSigned(listed.read_url)).toBe('kept-bytes');
expect(await readSigned(child.read_url)).toBe('kept-bytes');
});
it('still serves a URL signed before signatures were bound to the owner', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/legacy.txt`;
const { uuid } = await writeFileEntry(actor, path, 'legacy-bytes');
const expires = Math.ceil(Date.now() / 1000) + 9_999_999_999_999;
const signature = createHash('sha256')
.update(`${uuid}/read/${signingCfg().secret}/${expires}`)
.digest('hex');
const url = `${signingCfg().apiBaseUrl}/file?uid=${uuid}&expires=${expires}&signature=${signature}`;
expect(await readSigned(url)).toBe('legacy-bytes');
});
it('refuses that same pre-binding URL once legacy_file_signatures is off', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/legacy-off.txt`;
const { uuid } = await writeFileEntry(actor, path, 'legacy-bytes');
const expires = Math.ceil(Date.now() / 1000) + 9_999_999_999_999;
const signature = createHash('sha256')
.update(`${uuid}/read/${signingCfg().secret}/${expires}`)
.digest('hex');
const url = `${signingCfg().apiBaseUrl}/file?uid=${uuid}&expires=${expires}&signature=${signature}`;
const config = (
controller as unknown as { config: { legacy_file_signatures?: boolean } }
).config;
config.legacy_file_signatures = false;
try {
await expect(readSigned(url)).rejects.toMatchObject({
statusCode: 403,
});
} finally {
config.legacy_file_signatures = undefined;
}
});
it('still serves an owner-bound signature while legacy_file_signatures is off', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/owner-bound-while-off.txt`;
await writeFileEntry(actor, path, 'owner-bytes');
const signed = await signAs(actor, path);
const config = (
controller as unknown as { config: { legacy_file_signatures?: boolean } }
).config;
config.legacy_file_signatures = false;
try {
expect(await readSigned(signed.read_url)).toBe('owner-bytes');
} finally {
config.legacy_file_signatures = undefined;
}
});
});
// -- /file: non-UUID uid skips the entry-store lookup -------------------
describe('LegacyFSController.file (non-UUID uid)', () => {
it('rejects a non-UUID uid without touching the entry store', async () => {
const spy = vi.spyOn(server.stores.fsEntry, 'getEntryByUuid');
const expires = Math.ceil(Date.now() / 1000) + 9_999_999_999_999;
const { res } = makeRes();
await expect(
controller.file(
makeReq({
query: {
uid: 'not-a-uuid',
expires: String(expires),
signature: 'deadbeef',
},
}),
res,
),
).rejects.toMatchObject({ statusCode: 403 });
expect(spy).not.toHaveBeenCalled();
spy.mockRestore();
});
it('still looks up a UUID-shaped uid that does not exist', async () => {
const spy = vi.spyOn(server.stores.fsEntry, 'getEntryByUuid');
const expires = Math.ceil(Date.now() / 1000) + 9_999_999_999_999;
const { res } = makeRes();
await expect(
controller.file(
makeReq({
query: {
uid: uuidv4(),
expires: String(expires),
signature: 'deadbeef',
},
}),
res,
),
).rejects.toMatchObject({ statusCode: 403 });
expect(spy).toHaveBeenCalledTimes(1);
spy.mockRestore();
});
it('looks up a uuid whose version and variant nibbles are nonstandard', async () => {
const spy = vi.spyOn(server.stores.fsEntry, 'getEntryByUuid');
const expires = Math.ceil(Date.now() / 1000) + 9_999_999_999_999;
const { res } = makeRes();
await expect(
controller.file(
makeReq({
query: {
uid: '0123abcd-0000-0000-0000-0123456789ab',
expires: String(expires),
signature: 'deadbeef',
},
}),
res,
),
).rejects.toMatchObject({ statusCode: 403 });
expect(spy).toHaveBeenCalledTimes(1);
spy.mockRestore();
});
});
// -- /file via a signature minted by someone other than the owner -------
//
// `/sign` binds the signature to the entry's owner, not to whoever calls
// `/sign` — a grantee reading a shared file, or an app signing inside its
// own AppData, both mint signatures over an entry they don't own. Covers
// that those still read through `/file`.
describe('LegacyFSController /file via a non-owner-minted signature', () => {
const queryOf = (url: string) => {
const parsed = new URL(url);
return {
uid: parsed.searchParams.get('uid')!,
expires: parsed.searchParams.get('expires')!,
signature: parsed.searchParams.get('signature')!,
};
};
it('reads through /file when a grantee with read access signs the URL', async () => {
const owner = await makeUser();
const grantee = await makeUser();
const path = `/${owner.username}/Documents/grantee-read.txt`;
const { uuid } = await writeFileEntry(owner.actor, path, 'grantee-bytes');
await server.services.permission.grantUserUserPermission(
owner.actor,
grantee.username,
`fs:${uuid}:read`,
{},
);
const { res: signRes, captured: signCaptured } = makeRes();
await withActor(grantee.actor, () =>
controller.sign(
makeReq({
body: { items: [{ uid: uuid, action: 'read' }] },
actor: grantee.actor,
}),
signRes,
),
);
const signed = (
signCaptured.body as { signatures: Array<{ read_url: string }> }
).signatures[0];
const { res, captured, finished } = makeStreamRes();
await controller.file(makeReq({ query: queryOf(signed.read_url) }), res);
await finished;
expect(captured.statusCode).toBe(200);
expect(captured.text()).toBe('grantee-bytes');
});
it('reads through /file when an app actor signs inside its AppData root', async () => {
const { actor, username } = await makeUser();
const appUid = `routes-sign-app-${uuidv4()}`;
const appActor = makeActor({ ...actor, app: { uid: appUid } });
const path = `/${username}/AppData/${appUid}/app-read.txt`;
await writeFileEntry(actor, path, 'app-bytes');
const { res: signRes, captured: signCaptured } = makeRes();
await withActor(appActor, () =>
controller.sign(
makeReq({
body: { items: [{ path, action: 'read' }] },
actor: appActor,
}),
signRes,
),
);
const signed = (
signCaptured.body as { signatures: Array<{ read_url: string }> }
).signatures[0];
const { res, captured, finished } = makeStreamRes();
await controller.file(makeReq({ query: queryOf(signed.read_url) }), res);
await finished;
expect(captured.statusCode).toBe(200);
expect(captured.text()).toBe('app-bytes');
});
});
// -- /writeFile operation dispatch -------------------------------------
describe('LegacyFSController.writeFile operations', () => {
File diff suppressed because it is too large. Load diff
+421 -166
View File
@@ -32,19 +32,31 @@ import { HttpError } from '../../core/http/HttpError.js';
import { RouteOptions } from '../../core/http/index.js';
import {
assertNotSuspended,
assertTeam2fa,
assertVerifiedAccount,
} from '../../core/http/middleware/gates.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { ACLService } from '../../services/acl/ACLService.js';
import { assertActorHasCredits } from '../../services/metering/enforcement.js';
import type { SignedFile } from '../../util/fileSigning.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import type {
SignAction,
SignedFile,
SigningConfig,
} from '../../util/fileSigning.js';
import {
NON_OWNER_SIGNATURE_TTL_SECONDS,
parseSignedQuery,
verifySignature,
} from '../../util/fileSigning.js';
import { APP_ICON_SIZES, getAppIconCdnUrl } from '../../util/appIcon.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import {
joinChildPath,
normalizeAbsolutePath,
splitParentAndName,
} from '../../services/fs/resolveNode.js';
import { maskEntryPath } from '../../services/fs/sharePathMask.js';
import { clientParentUid } from '../../services/fs/rootListing.js';
import {
buildHostedBackingDenial,
hostedIndexUrlBackingIsUnavailable,
@@ -67,6 +79,7 @@ import {
FS_SEARCH_CONCURRENT,
FS_SEARCH_LIMIT,
FS_SIGN_LIMIT,
FS_SIGN_MAX_ITEMS,
FS_SIGNED_CONCURRENT,
FS_SIGNED_READ_LIMIT,
FS_SIGNED_WRITE_LIMIT,
@@ -77,6 +90,7 @@ import {
assertAccess,
assertCanCreate,
assertCanMoveInto,
expandClientPath,
getBoolean,
getString,
loadLegacyAssociatedApps,
@@ -90,15 +104,55 @@ type RouterCache = Map<string, RequestHandler | null>;
const additionalRoutePaths: Record<string, string> = {};
// Shape only: version and variant nibbles aren't checked, so a row whose uuid
// wasn't minted by an RFC-conforming generator still resolves.
const UUID_SHAPE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
// Legacy `/batch` multipart upload caps. Each file is buffered fully into
// memory before any quota / storage check runs, so without these limits an
// authenticated caller could grow the process heap proportional to whatever
// they sent. Streaming uploads go through `/writeFile`; this path is for
// pre-v2 clients only.
const BATCH_MAX_FILE_SIZE = 100 * 1024 * 1024; // 100 MiB per file
const BATCH_MAX_TOTAL_SIZE = 256 * 1024 * 1024; // 256 MiB buffered per request
const BATCH_MAX_FILES = 64;
const BATCH_MAX_PARTS = 256;
const BATCH_MAX_FIELD_SIZE = 1 * 1024 * 1024; // 1 MiB per operation/fileinfo JSON
// Multipart carries at most one op per part, so JSON mode gets the same bound.
const BATCH_MAX_OPS = BATCH_MAX_PARTS;
const BATCH_MUTATING_OPS = new Set(['mkdir', 'shortcut', 'move', 'delete']);
// An op's `as` names the path it produced; later ops in the same batch refer
// to it as `$name` or `$name/rest`, since a deduped mkdir can land elsewhere.
function resolveBatchRef(raw: string, refs: Map<string, string>): string {
if (!raw.startsWith('$')) return raw;
const slashIdx = raw.indexOf('/');
const name = slashIdx === -1 ? raw.slice(1) : raw.slice(1, slashIdx);
const rest = slashIdx === -1 ? '' : raw.slice(slashIdx);
if (!name) {
throw new HttpError(400, 'Invalid batch reference', {
legacyCode: 'bad_request',
});
}
const base = refs.get(name);
if (base === undefined) {
throw new HttpError(400, `Unknown batch reference: $${name}`, {
legacyCode: 'bad_request',
});
}
// Rejects non-normalized results, so `$ref/../x` can't escape.
return normalizeAbsolutePath(base + rest);
}
const malformedMultipart = (err: unknown): HttpError =>
err instanceof HttpError
? err
: new HttpError(
400,
`Malformed multipart body${err instanceof Error ? `: ${err.message}` : ''}`,
{ legacyCode: 'bad_request' },
);
async function loadAdditionalRouter(
key: string,
@@ -461,11 +515,18 @@ export class LegacyFSController extends PuterController {
'see',
);
const [suggestedApps, appsById, shareFlags] = await Promise.all([
this.services.suggestedApps.getSuggestedApps(entry),
loadLegacyAssociatedApps(this.stores.app, [entry]),
this.services.share.shareFlags(actor, [entry]),
]);
const [suggestedApps, appsById, shareFlags, parentUid] =
await Promise.all([
this.services.suggestedApps.getSuggestedApps(entry),
loadLegacyAssociatedApps(this.stores.app, [entry]),
this.services.share.shareFlags(actor, [entry]),
clientParentUid(
actor,
entry,
this.services.acl,
this.stores.permission,
),
]);
entry.suggestedApps = suggestedApps;
const shaped = await toLegacyEntry(this.clients.event, entry, {
@@ -477,6 +538,7 @@ export class LegacyFSController extends PuterController {
},
appsById,
isShared: shareFlags.get(entry.uuid) ?? null,
parentUid,
});
// Optional hydrations:
@@ -525,6 +587,8 @@ export class LegacyFSController extends PuterController {
const rootChildren = await listRootEntries(
actor,
this.stores.fsEntry,
this.services.acl,
this.stores.permission,
);
const rootSuggestions =
await this.services.suggestedApps.getSuggestedAppsForEntries(
@@ -661,14 +725,21 @@ export class LegacyFSController extends PuterController {
// When `parent` is a path string, use it directly without requiring
// the entry to exist — `services.fs.mkdir` honors `create_missing_parents`
// and will materialize any missing intermediate directories.
let targetPath = rawPath;
const username = actor.user?.username;
let targetPath = rawPath.startsWith('/')
? await expandClientPath(this.stores.fsEntry, rawPath, username)
: rawPath;
if (body.parent !== undefined && !rawPath.startsWith('/')) {
let parentPath: string;
if (
typeof body.parent === 'string' &&
(body.parent.startsWith('/') || body.parent.startsWith('~'))
) {
parentPath = expandTildePath(body.parent, actor.user?.username);
parentPath = await expandClientPath(
this.stores.fsEntry,
body.parent,
username,
);
} else {
const parent = await resolveV1Selector(
this.stores.fsEntry,
@@ -1002,9 +1073,13 @@ export class LegacyFSController extends PuterController {
legacyCode: 'bad_request',
});
const parentPath = pathPosix.dirname(
rawPath.startsWith('/') ? rawPath : `/${rawPath}`,
const expanded = await expandClientPath(
this.stores.fsEntry,
rawPath,
actor.user?.username,
);
const targetPath = expanded.startsWith('/') ? expanded : `/${expanded}`;
const parentPath = pathPosix.dirname(targetPath);
if (parentPath === '/') {
throw new HttpError(400, 'Cannot touch in root', {
legacyCode: 'bad_request',
@@ -1019,7 +1094,7 @@ export class LegacyFSController extends PuterController {
);
await this.services.fs.touch(userId, {
path: rawPath,
path: targetPath,
setAccessed: getBoolean(body, 'set_accessed_to_now') ?? false,
setModified: getBoolean(body, 'set_modified_to_now') ?? false,
setCreated: getBoolean(body, 'set_created_to_now') ?? false,
@@ -1128,7 +1203,7 @@ export class LegacyFSController extends PuterController {
// emitAndWait is required: the thumbnails extension rewrites
// `event.url` from a data URL to an `s3://` pointer, and the DB
// write below needs to see that rewrite.
const event = { url: thumbnail };
const event = { url: thumbnail, uuid: entry.uuid };
await this.clients.event.emitAndWait('thumbnail.created', event, {});
await this.clients.db.write(
@@ -1264,6 +1339,7 @@ export class LegacyFSController extends PuterController {
// that guard every other authenticated FS route have to run here.
assertNotSuspended(actor!.user);
assertVerifiedAccount(actor!.user);
await assertTeam2fa(actor!.user, this.stores.team);
req.actor = assertResolvedActor(actor!);
Context.set('actor', actor);
@@ -1298,6 +1374,12 @@ export class LegacyFSController extends PuterController {
throw new HttpError(400, '`items` is required', {
legacyCode: 'bad_request',
});
if (items.length > FS_SIGN_MAX_ITEMS)
throw new HttpError(
400,
`Too many items in one request (max ${FS_SIGN_MAX_ITEMS})`,
{ legacyCode: 'bad_request' },
);
const actingApp = actor.effectiveApp;
const signingCfg = signingConfigFromAppConfig(this.config);
@@ -1433,25 +1515,12 @@ export class LegacyFSController extends PuterController {
writeFile = async (req: Request, res: Response): Promise<void> => {
const query = asRecord(req.query);
const signingCfg = signingConfigFromAppConfig(this.config);
verifySignature(
{
uid: query.uid as string,
expires: query.expires as string,
signature: query.signature as string,
},
const targetEntry = await this.#resolveSignedEntry(
query,
'write',
signingCfg,
);
const uid = typeof query.uid === 'string' ? query.uid : '';
const targetEntry = await resolveV1Selector(this.stores.fsEntry, {
uid,
});
if (!targetEntry)
throw new HttpError(404, 'Item not found', {
legacyCode: 'not_found',
});
// Owner suspension check.
const owner = await this.stores.user.getById(targetEntry.userId);
if (!owner)
@@ -1675,18 +1744,7 @@ export class LegacyFSController extends PuterController {
file = async (req: Request, res: Response): Promise<void> => {
const query = asRecord(req.query);
const signingCfg = signingConfigFromAppConfig(this.config);
verifySignature(
{
uid: query.uid as string,
expires: query.expires as string,
signature: query.signature as string,
},
'read',
signingCfg,
);
const uid = typeof query.uid === 'string' ? query.uid : '';
const entry = await resolveV1Selector(this.stores.fsEntry, { uid });
const entry = await this.#resolveSignedEntry(query, 'read', signingCfg);
// Owner-suspension guard — matches v1's /file. A signed URL stays
// valid forever by default, so a signature minted before a suspension
@@ -1710,6 +1768,7 @@ export class LegacyFSController extends PuterController {
uuid: owner.uuid,
id: owner.id,
username: owner.username,
email: owner.email ?? null,
suspended: !!(owner as { suspended?: unknown }).suspended,
},
};
@@ -1932,7 +1991,7 @@ export class LegacyFSController extends PuterController {
* access to a subject FS entry.
*/
checkAppAcl = async (req: Request, res: Response): Promise<void> => {
this.#requireActor(req);
const actor = this.#requireActor(req);
const body = asRecord(req.body);
const subjectRef = body.subject;
@@ -1944,10 +2003,28 @@ export class LegacyFSController extends PuterController {
legacyCode: 'bad_request',
});
const subject = await resolveV1Selector(
this.stores.fsEntry,
subjectRef,
);
// A missing subject and one the caller can't see get the same 404,
// so this can't be used to probe for other users' entries.
let subject: import('../../stores/fs/FSEntry.js').FSEntry | null = null;
try {
subject = await resolveV1Selector(this.stores.fsEntry, subjectRef);
} catch (err) {
if (!(err instanceof HttpError && err.statusCode === 404))
throw err;
}
let ancestors: Promise<Array<{ uid: string; path: string }>> | null =
null;
const descriptor = subject && {
path: subject.path,
resolveAncestors: () =>
(ancestors ??= this.services.fs.getAncestorChain(subject.path)),
};
const acl = this.services.acl as ACLService;
if (!descriptor || !(await acl.check(actor, descriptor, 'see')))
throw new HttpError(404, 'Subject does not exist', {
legacyCode: 'subject_does_not_exist',
});
let app: { uid: string } | null = null;
if (typeof appRef === 'string') {
app =
@@ -1961,19 +2038,10 @@ export class LegacyFSController extends PuterController {
// Build an actor-under-user shape for the check.
const actorForApp = makeActor({
user: req.actor!.user,
user: actor.user,
app: { uid: (app as { uid: string }).uid },
});
const descriptor = {
path: subject.path,
resolveAncestors: () =>
this.services.fs.getAncestorChain(subject.path),
};
const allowed = await (this.services.acl as ACLService).check(
actorForApp,
descriptor,
mode,
);
const allowed = await acl.check(actorForApp, descriptor, mode);
res.json({ allowed });
};
@@ -2057,9 +2125,12 @@ export class LegacyFSController extends PuterController {
// silently ignores. Run the entry through `toLegacyEntry` first so
// the event payload matches what /stat et al. return, then overlay
// per-op extras (e.g. `old_path` for moves).
// `forOwner` — the audience is the owner, who can't read the actor's mask.
try {
const response = {
...(await toLegacyEntry(this.clients.event, entry)),
...(await toLegacyEntry(this.clients.event, entry, {
forOwner: true,
})),
...extra,
from_new_service: true,
};
@@ -2189,6 +2260,7 @@ export class LegacyFSController extends PuterController {
const results: unknown[] = [];
let hasError = false;
let sequentialFileIdx = 0;
const pathRefs = new Map<string, string>();
for (const spec of operationSpecs) {
try {
@@ -2196,6 +2268,16 @@ export class LegacyFSController extends PuterController {
const op = typeof record.op === 'string' ? record.op : '';
let shaped: unknown;
// Non-write ops draw the same budget as the standalone routes.
if (
BATCH_MUTATING_OPS.has(op) &&
!(await consumeRouteRateLimit(req, FS_MUTATE_LIMIT))
) {
throw new HttpError(429, 'Too many requests.', {
legacyCode: 'too_many_requests',
});
}
if (op === 'write') {
// Pair with a file part — prefer `item_upload_id`
// index (what puter-js sets), fall back to the op's
@@ -2229,15 +2311,22 @@ export class LegacyFSController extends PuterController {
legacyCode: 'bad_request',
});
}
const parentPath = getString(record, 'path') ?? '';
const expandedParent = expandTildePath(
const parentPath = resolveBatchRef(
getString(record, 'path') ?? '',
pathRefs,
);
const expandedParent = await expandClientPath(
this.stores.fsEntry,
parentPath,
username,
);
const targetPath =
expandedParent && expandedParent !== '/'
? `${expandedParent.replace(/\/+$/, '')}/${name}`
: `/${name}`;
// joinChildPath rejects a `name` carrying a slash or a
// `.`/`..` segment, so it can't reach outside the parent
// the ACL check below actually authorized.
const targetPath = joinChildPath(
expandedParent || '/',
name,
);
// Mirrors the per-op /write|/mkdir routes: assert write
// on the parent dir, but when the parent resolves to `/`
// fall back to the target path so the ACL check rides
@@ -2288,25 +2377,12 @@ export class LegacyFSController extends PuterController {
response.fsEntry,
);
} else if (op === 'mkdir') {
const parentPath = getString(record, 'path') ?? '';
const name = getString(record, 'name');
if (!name) {
throw new HttpError(400, 'mkdir op missing `name`', {
legacyCode: 'bad_request',
});
}
const expandedParent = expandTildePath(
parentPath,
username,
);
const targetPath =
expandedParent && expandedParent !== '/'
? `${expandedParent.replace(/\/+$/, '')}/${name}`
: `/${name}`;
const writeAclPath =
expandedParent && expandedParent !== '/'
? expandedParent.replace(/\/+$/, '')
: targetPath;
const { targetPath, writeAclPath } =
await this.#resolveBatchMkdirTarget(
record,
username,
pathRefs,
);
await assertAccess(
this.services.acl,
this.services.fs,
@@ -2324,12 +2400,13 @@ export class LegacyFSController extends PuterController {
'create_missing_parents',
) ?? false,
});
this.#registerBatchRef(record, pathRefs, entry.path);
await this.#emitGuiEvent('outer.gui.item.added', entry);
shaped = await toLegacyEntry(this.clients.event, entry);
} else if (op === 'shortcut') {
const parentPath = getString(record, 'path') ?? '';
const name = getString(record, 'name');
const shortcutToUid =
const shortcutToRaw =
getString(record, 'shortcut_to_uid') ??
getString(record, 'shortcut_to');
if (!name) {
@@ -2337,19 +2414,21 @@ export class LegacyFSController extends PuterController {
legacyCode: 'shortcut_target_not_found',
});
}
if (!shortcutToUid) {
if (!shortcutToRaw) {
throw new HttpError(
400,
'shortcut op missing `shortcut_to_uid`',
{ legacyCode: 'shortcut_target_not_found' },
);
}
// A path or a uid; the desktop sends a path.
const target = await resolveV1Selector(
this.stores.fsEntry,
{ uid: shortcutToUid },
resolveBatchRef(shortcutToRaw, pathRefs),
);
const expandedParent = expandTildePath(
parentPath,
const expandedParent = await expandClientPath(
this.stores.fsEntry,
resolveBatchRef(parentPath, pathRefs),
username,
);
const parent = await resolveV1Selector(
@@ -2379,13 +2458,21 @@ export class LegacyFSController extends PuterController {
await this.#emitGuiEvent('outer.gui.item.added', link);
shaped = await toLegacyEntry(this.clients.event, link);
} else if (op === 'move') {
const sourceRef =
typeof record.source === 'string'
? resolveBatchRef(record.source, pathRefs)
: record.source;
const destinationRef =
typeof record.destination === 'string'
? resolveBatchRef(record.destination, pathRefs)
: record.destination;
const source = await resolveV1Selector(
this.stores.fsEntry,
record.source,
sourceRef,
);
const destinationParent = await resolveV1Selector(
this.stores.fsEntry,
record.destination,
destinationRef,
);
await assertAccess(
this.services.acl,
@@ -2408,14 +2495,20 @@ export class LegacyFSController extends PuterController {
overwrite: getBoolean(record, 'overwrite') ?? false,
dedupeName: getBoolean(record, 'dedupe_name') ?? false,
});
this.#registerBatchRef(record, pathRefs, moved.path);
await this.#emitGuiEvent('outer.gui.item.moved', moved, {
old_path: source.path,
});
shaped = await toLegacyEntry(this.clients.event, moved);
} else if (op === 'delete') {
const pathField = getString(record, 'path');
const selectorRef =
pathField !== undefined
? resolveBatchRef(pathField, pathRefs)
: record;
const entry = await resolveV1Selector(
this.stores.fsEntry,
getString(record, 'path') ?? record,
selectorRef,
);
await assertAccess(
this.services.acl,
@@ -2464,64 +2557,100 @@ export class LegacyFSController extends PuterController {
}> = [];
const fileinfos: Array<Record<string, unknown>> = [];
let parseError: Error | null = null;
const bb = Busboy({
headers: req.headers,
limits: {
fileSize: BATCH_MAX_FILE_SIZE,
files: BATCH_MAX_FILES,
parts: BATCH_MAX_PARTS,
fieldSize: BATCH_MAX_FIELD_SIZE,
},
});
let totalBufferedBytes = 0;
// First error wins. Buffered files are dropped right away rather
// than held while the rest of the body drains.
const fail = (err: Error) => {
if (parseError) return;
parseError = err;
files.length = 0;
};
const tooLarge = (message: string) =>
new HttpError(413, message, {
legacyCode: 'too_large' as never,
});
const reserve = (bytes: number): boolean => {
totalBufferedBytes += bytes;
if (totalBufferedBytes <= BATCH_MAX_TOTAL_SIZE) return true;
fail(
tooLarge(
`Batch exceeds ${BATCH_MAX_TOTAL_SIZE} bytes in total`,
),
);
return false;
};
let bb: ReturnType<typeof Busboy>;
try {
bb = Busboy({
headers: req.headers,
limits: {
fileSize: BATCH_MAX_FILE_SIZE,
files: BATCH_MAX_FILES,
parts: BATCH_MAX_PARTS,
fieldSize: BATCH_MAX_FIELD_SIZE,
},
});
} catch (err) {
// Missing boundary or unparseable content-type.
reject(malformedMultipart(err));
return;
}
// Busboy emits these `*Limit` events when a configured cap is
// hit. Capture the first one as a 413 so callers get a clean
// signal instead of a silently-truncated upload.
bb.on('filesLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
bb.on('filesLimit', () =>
fail(
tooLarge(
`Too many files in batch (max ${BATCH_MAX_FILES})`,
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('partsLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
),
),
);
bb.on('partsLimit', () =>
fail(
tooLarge(
`Too many parts in batch (max ${BATCH_MAX_PARTS})`,
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('fieldsLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
'Too many fields in batch',
{ legacyCode: 'too_large' as never },
);
}
});
),
),
);
bb.on('fieldsLimit', () =>
fail(tooLarge('Too many fields in batch')),
);
bb.on('field', (fieldName, value) => {
bb.on('field', (fieldName, value, info) => {
if (parseError) return;
if (info?.valueTruncated) {
fail(
tooLarge(
`\`${fieldName}\` field exceeds ${BATCH_MAX_FIELD_SIZE} bytes`,
),
);
return;
}
if (!reserve(Buffer.byteLength(value))) return;
// Ignore operation_id / socket_id / misc fields — not
// needed for v2 batch semantics.
if (fieldName !== 'operation' && fieldName !== 'fileinfo') {
return;
}
let parsed: unknown;
try {
if (fieldName === 'operation') {
ops.push(JSON.parse(value));
} else if (fieldName === 'fileinfo') {
const parsed = JSON.parse(value);
fileinfos.push(
parsed && typeof parsed === 'object'
? (parsed as Record<string, unknown>)
: {},
);
}
// Ignore operation_id / socket_id / misc fields — not
// needed for v2 batch semantics.
} catch (err) {
parseError =
err instanceof Error ? err : new Error(String(err));
parsed = JSON.parse(value);
} catch {
fail(
new HttpError(
400,
`\`${fieldName}\` field is not valid JSON`,
{ legacyCode: 'bad_request' },
),
);
return;
}
if (fieldName === 'operation') {
ops.push(parsed);
} else {
fileinfos.push(
parsed && typeof parsed === 'object'
? (parsed as Record<string, unknown>)
: {},
);
}
});
@@ -2530,24 +2659,26 @@ export class LegacyFSController extends PuterController {
// For streaming uploads use the signed `/writeFile` endpoint.
bb.on('file', (_fieldName, stream, info) => {
const chunks: Buffer[] = [];
let truncated = false;
stream.on('data', (chunk: Buffer) => chunks.push(chunk));
// Busboy emits `limit` after writing the first byte past
// `fileSize`. The stream continues being drained so the
// multipart parser stays in sync, but we discard the (now
// truncated) buffer and mark the batch as failed.
stream.on('limit', () => {
truncated = true;
if (!parseError) {
parseError = new HttpError(
413,
`File in batch exceeds ${BATCH_MAX_FILE_SIZE} bytes`,
{ legacyCode: 'too_large' as never },
);
// After a failure the stream keeps draining so the multipart
// parser reaches `close`, but nothing more is held.
stream.on('data', (chunk: Buffer) => {
if (parseError || !reserve(chunk.length)) {
chunks.length = 0;
return;
}
chunks.push(chunk);
});
// Busboy emits `limit` after the first byte past `fileSize`.
stream.on('limit', () => {
chunks.length = 0;
fail(
tooLarge(
`File in batch exceeds ${BATCH_MAX_FILE_SIZE} bytes`,
),
);
});
stream.on('end', () => {
if (truncated) return;
if (parseError) return;
files.push({
content: Buffer.concat(chunks),
mimeType:
@@ -2560,16 +2691,19 @@ export class LegacyFSController extends PuterController {
: undefined,
});
});
stream.on('error', (err: Error) => {
parseError = err;
});
stream.on('error', (err: Error) => fail(err));
});
bb.on('close', () => {
if (parseError) reject(parseError);
if (parseError) reject(malformedMultipart(parseError));
else resolve({ ops, files, fileinfos });
});
bb.on('error', (err) => reject(err));
bb.on('error', (err) => reject(malformedMultipart(err)));
// A client that disconnects mid-body never lets busboy close.
req.on('error', (err) => reject(malformedMultipart(err)));
req.on('aborted', () =>
reject(malformedMultipart(new Error('Request aborted'))),
);
req.pipe(bb);
});
@@ -2577,9 +2711,100 @@ export class LegacyFSController extends PuterController {
#parseJsonBatch(req: Request): unknown[] {
const body = asRecord(req.body);
if (Array.isArray(body.operations)) return body.operations;
if (Array.isArray(body.ops)) return body.ops;
return [];
const ops = Array.isArray(body.operations)
? body.operations
: Array.isArray(body.ops)
? body.ops
: [];
if (ops.length > BATCH_MAX_OPS) {
throw new HttpError(
413,
`Too many operations in batch (max ${BATCH_MAX_OPS})`,
{ legacyCode: 'too_large' as never },
);
}
return ops;
}
/**
* Batch `mkdir` shapes: `{ parent, path }` with `path` relative to
* `parent`, `{ path, name }` with `path` as the parent, or an absolute `{
* path }`.
*/
async #resolveBatchMkdirTarget(
record: Record<string, unknown>,
username: string | undefined,
pathRefs: Map<string, string>,
): Promise<{ targetPath: string; writeAclPath: string }> {
// Write is checked on the immediate parent: a multi-segment relative
// path must not be authorized by a grant on the target itself. Under
// `/` there is no parent to check, so the target stands in.
const withAclPath = (targetPath: string) => {
const parentPath = pathPosix.dirname(targetPath);
return {
targetPath,
writeAclPath: parentPath === '/' ? targetPath : parentPath,
};
};
const name = getString(record, 'name');
if (typeof record.parent === 'string' && !name) {
const parentPath = await expandClientPath(
this.stores.fsEntry,
resolveBatchRef(record.parent, pathRefs),
username,
);
const segments = (getString(record, 'path') ?? '')
.split('/')
.filter((s) => s.length > 0);
if (segments.length === 0) {
throw new HttpError(400, 'mkdir op missing `name`', {
legacyCode: 'bad_request',
});
}
let targetPath = parentPath || '/';
for (const segment of segments) {
targetPath = joinChildPath(targetPath, segment);
}
return withAclPath(targetPath);
}
if (name) {
const parentPath = await expandClientPath(
this.stores.fsEntry,
resolveBatchRef(getString(record, 'path') ?? '', pathRefs),
username,
);
return withAclPath(joinChildPath(parentPath || '/', name));
}
const rawPath = getString(record, 'path');
if (!rawPath) {
throw new HttpError(400, 'mkdir op missing `name`', {
legacyCode: 'bad_request',
});
}
const { parentPath, name: derivedName } = splitParentAndName(
await expandClientPath(
this.stores.fsEntry,
resolveBatchRef(rawPath, pathRefs),
username,
),
);
return withAclPath(joinChildPath(parentPath, derivedName));
}
/** Record an op's `as` (a bare name, no slash) for later `$name` refs. */
#registerBatchRef(
record: Record<string, unknown>,
pathRefs: Map<string, string>,
path: string,
): void {
const as = record.as;
if (typeof as === 'string' && as.length > 0 && !as.includes('/')) {
pathRefs.set(as, path);
}
}
#serializeBatchError(err: unknown): Record<string, unknown> {
@@ -2607,6 +2832,36 @@ export class LegacyFSController extends PuterController {
}
// -- Helpers ---------------------------------------------------------
/**
* Resolve the entry a signed URL names and verify the signature against its
* current owner. Malformed or expired params fail before the lookup.
*/
async #resolveSignedEntry(
query: Record<string, unknown>,
action: SignAction,
signingCfg: SigningConfig,
): Promise<FSEntry> {
const signed = {
uid: query.uid as string,
expires: query.expires as string,
signature: query.signature as string,
};
const { uid } = parseSignedQuery(signed);
// A non-UUID uid can never match a real entry — skip the cache/DB
// round trip a forged request would otherwise pay for on every hit.
const entry = UUID_SHAPE.test(uid)
? await this.stores.fsEntry.getEntryByUuid(uid)
: null;
verifySignature(signed, action, signingCfg, entry?.userId ?? null);
// Only a signature from before owner binding verifies without an entry.
if (!entry)
throw new HttpError(404, 'Subject does not exist', {
legacyCode: 'subject_does_not_exist',
});
return entry;
}
#requireActor(req: Request) {
const actor = req.actor;
if (!actor) {
@@ -0,0 +1,173 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
// Launch adopts whatever directory already sits at `AppData/<appUid>`, and an
// origin app's uid is derived from its origin, so another app knows where a
// not-yet-opened app's root will be.
describe('AppData roots over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const call = (path: string, token: string, body?: unknown) =>
fetch(new URL(path, env.apiOrigin), {
method: body === undefined ? 'GET' : 'POST',
headers: {
authorization: `Bearer ${token}`,
...(body === undefined
? {}
: { 'content-type': 'application/json' }),
},
...(body === undefined ? {} : { body: JSON.stringify(body) }),
});
const origin = (label: string) =>
`https://${label}-${Math.random().toString(36).slice(2, 10)}.example.com`;
const launch = async (userToken: string, appOrigin: string) => {
const res = await call('/auth/get-user-app-token', userToken, {
origin: appOrigin,
});
expect(res.status).toBe(200);
return (await res.json()) as { token: string; app_uid: string };
};
const appRoot = async (app: { token: string; app_uid: string }) => {
const res = await call('/auth/request-app-root-dir', app.token, {
app_uid: app.app_uid,
});
expect(res.status).toBe(200);
return (await res.json()) as { uid: string; path: string };
};
it('refuses an app renaming its root onto another app’s uid', async () => {
const { username, token } = env.users.user;
const a = await launch(token, origin('appdata-a'));
const bOrigin = origin('appdata-b');
const bUid = await env.server.services.auth.appUidFromOrigin(bOrigin);
const root = await appRoot(a);
// A uid-bound grant on its own root, which would follow the root.
const sign = await call('/sign', a.token, {
items: [{ uid: root.uid, action: 'write' }],
app_uid: a.app_uid,
});
expect(sign.status).toBe(200);
const rename = await call('/rename', a.token, {
uid: root.uid,
new_name: bUid,
});
expect(rename.status).toBe(403);
const unmoved = await env.server.stores.fsEntry.getEntryByUuid(
root.uid,
);
expect(unmoved?.path).toBe(`/${username}/AppData/${a.app_uid}`);
// B gets a root of its own, out of A's reach.
const b = await launch(token, bOrigin);
expect(b.app_uid).toBe(bUid);
const bRoot = await appRoot(b);
expect(bRoot.uid).not.toBe(root.uid);
const secret = `/${username}/AppData/${bUid}/secret.txt`;
const write = await call('/fs/write', b.token, {
fileMetadata: {
path: secret,
size: Buffer.byteLength('b-private'),
contentType: 'text/plain',
},
fileContent: 'b-private',
encoding: 'utf8',
});
expect(write.status).toBe(200);
const read = await call(
`/read?file=${encodeURIComponent(secret)}`,
a.token,
);
expect(read.status).toBe(404);
});
it('refuses an app moving its root into the Trash', async () => {
const { username, token } = env.users.user;
const a = await launch(token, origin('appdata-trash'));
const root = await appRoot(a);
const move = await call('/move', a.token, {
source: root.uid,
destination: `/${username}/Trash`,
});
expect(move.status).toBe(403);
const unmoved = await env.server.stores.fsEntry.getEntryByUuid(
root.uid,
);
expect(unmoved?.path).toBe(`/${username}/AppData/${a.app_uid}`);
});
it('rewrites a forged trash original_path from an app instead of trusting it', async () => {
const { username, token } = env.users.user;
const a = await launch(token, origin('appdata-origin'));
const bUid = await env.server.services.auth.appUidFromOrigin(
origin('appdata-origin-b'),
);
const root = await appRoot(a);
const made = await call('/mkdir', a.token, {
path: `${root.path}/inner`,
});
expect(made.status).toBe(200);
const inner = (await made.json()) as { uid: string; path: string };
// Forging the other app's root as the trash origin would let the
// owner's (unchecked) restore recreate the file there. Refusing the
// move outright would also break an ordinary delete by anyone who
// isn't the account, so the forged keys are corrected instead.
const trash = await call('/move', a.token, {
source: inner.uid,
destination: `/${username}/Trash`,
new_name: inner.uid,
new_metadata: {
original_name: bUid,
original_path: `/${username}/AppData/${bUid}`,
trashed_ts: 1,
},
});
expect(trash.status).toBe(200);
const trashed = await env.server.stores.fsEntry.getEntryByUuid(
inner.uid,
);
const metadata = JSON.parse(trashed!.metadata!) as Record<
string,
unknown
>;
expect(metadata.original_path).toBe(inner.path);
expect(metadata.original_name).toBe('inner');
expect(metadata.trashed_ts).not.toBe(1);
});
});
@@ -256,7 +256,21 @@ describe('signingConfigFromAppConfig', () => {
url_signature_secret: 's3cret',
api_base_url: 'https://api.test',
} as never),
).toEqual({ secret: 's3cret', apiBaseUrl: 'https://api.test' });
).toEqual({
secret: 's3cret',
apiBaseUrl: 'https://api.test',
allowLegacySignatures: true,
});
});
it('stops accepting pre-binding signatures when legacy_file_signatures is false', () => {
expect(
signingConfigFromAppConfig({
url_signature_secret: 's3cret',
api_base_url: 'https://api.test',
legacy_file_signatures: false,
} as never).allowLegacySignatures,
).toBe(false);
});
it('fails loudly when the signing secret is missing', () => {
@@ -412,6 +426,23 @@ describe('toLegacyEntry', () => {
);
expect(shaped.associated_app).toBeNull();
});
it('uses the parentUid override, including null, over the entry’s own', async () => {
const hidden = await toLegacyEntry(undefined, baseEntry(), {
parentUid: null,
});
expect(hidden.parent_id).toBeNull();
expect(hidden.parent_uid).toBeNull();
const overridden = await toLegacyEntry(undefined, baseEntry(), {
parentUid: 'other-parent',
});
expect(overridden.parent_id).toBe('other-parent');
expect(overridden.parent_uid).toBe('other-parent');
const defaulted = await toLegacyEntry(undefined, baseEntry());
expect(defaulted.parent_uid).toBe('parent-1');
});
});
describe('loadLegacyAssociatedApps short-circuit', () => {
+72 -32
View File
@@ -22,6 +22,7 @@ import { contentType as contentTypeFromMime } from 'mime-types';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import type { FSEntryStore } from '../../stores/fs/FSEntryStore.js';
import type { FSService } from '../../services/fs/FSService.js';
import type { NodeRef } from '../../services/fs/resolveNode.js';
import type { ACLService, AclMode } from '../../services/acl/ACLService.js';
import { isAppActor, type Actor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
@@ -31,6 +32,7 @@ import {
resolveNode,
normalizeAbsolutePath,
isOwnersTrash,
assertChildName,
joinChildPath,
expandTildePath,
} from '../../services/fs/resolveNode.js';
@@ -115,9 +117,32 @@ export async function expandClientPath(
);
}
export async function resolveV1Selector(
// Nested `{parent, name}` selectors recurse; unbounded depth overflows the stack.
const MAX_SELECTOR_DEPTH = 4;
/**
* Resolve `ref`, reporting any miss as the same 404 an unreadable entry gets,
* so a lookup never echoes a resolved path or tells missing from hidden.
*/
async function resolveOrNotFound(
fsEntryStore: FSEntryStore,
ref: NodeRef,
): Promise<FSEntry> {
try {
const entry = await resolveNode(fsEntryStore, ref, { required: true });
if (entry) return entry;
} catch (err) {
if (!(err instanceof HttpError && err.statusCode === 404)) throw err;
}
throw new HttpError(404, 'Subject does not exist', {
legacyCode: 'subject_does_not_exist',
});
}
async function resolveV1SelectorAtDepth(
fsEntryStore: FSEntryStore,
raw: unknown,
depth: number,
): Promise<FSEntry> {
const username = Context.get('actor')?.user?.username;
@@ -130,30 +155,29 @@ export async function resolveV1Selector(
const ref = isPath
? { path: await expandClientPath(fsEntryStore, raw, username) }
: { uid: raw };
const entry = await resolveNode(fsEntryStore, ref, { required: true });
if (!entry)
throw new HttpError(404, `Entry not found: ${raw}`, {
legacyCode: 'not_found',
});
return entry;
return resolveOrNotFound(fsEntryStore, ref);
}
const record = asRecord(raw);
// {parent, name}: "child selector" — resolve parent, then child by name.
if (record.parent !== undefined && typeof record.name === 'string') {
const parent = await resolveV1Selector(fsEntryStore, record.parent);
const childPath = joinChildPath(parent.path, record.name);
const child = await resolveNode(
fsEntryStore,
{ path: childPath },
{ required: true },
);
if (!child)
throw new HttpError(404, `Entry not found: ${childPath}`, {
legacyCode: 'not_found',
if (depth >= MAX_SELECTOR_DEPTH) {
throw new HttpError(400, 'Selector is nested too deeply', {
legacyCode: 'bad_request',
});
return child;
}
// Before the parent lookup, so a bad name can't tell whether the
// parent exists.
assertChildName(record.name);
const parent = await resolveV1SelectorAtDepth(
fsEntryStore,
record.parent,
depth + 1,
);
return resolveOrNotFound(fsEntryStore, {
path: joinChildPath(parent.path, record.name),
});
}
const rawPath = typeof record.path === 'string' ? record.path : undefined;
@@ -173,12 +197,14 @@ export async function resolveV1Selector(
? record.id
: undefined,
};
const entry = await resolveNode(fsEntryStore, ref, { required: true });
if (!entry)
throw new HttpError(404, 'Entry not found', {
legacyCode: 'not_found',
});
return entry;
return resolveOrNotFound(fsEntryStore, ref);
}
export async function resolveV1Selector(
fsEntryStore: FSEntryStore,
raw: unknown,
): Promise<FSEntry> {
return resolveV1SelectorAtDepth(fsEntryStore, raw, 0);
}
// -- ACL --------------------------------------------------------------
@@ -225,7 +251,7 @@ export async function assertAccess(
// through the error code. User-actor denials keep the real 403.
if (isAppActor(actor)) {
throw new HttpError(404, `Entry not found: path=${path}`, {
throw new HttpError(404, 'Subject does not exist', {
legacyCode: 'subject_does_not_exist',
});
}
@@ -438,7 +464,9 @@ export async function signEntryThumbnail(
* URL for a signed one. Pass `fsEntryStore`/`userStore` to hydrate `is_empty`
* (directories) and `owner` — both are required fields per the legacy stat
* contract but need extra DB lookups. Pass `appsById` (built via
* `loadLegacyAssociatedApps`) to populate `associated_app`.
* `loadLegacyAssociatedApps`) to populate `associated_app`. Pass `parentUid`
* (even as `null`, via `clientParentUid`) to override `parent_id`/`parent_uid`
* — callers reading a scoped access token's home use it to hide the uuid.
*/
export async function toLegacyEntry(
eventClient: EventClient | undefined,
@@ -451,11 +479,14 @@ export async function toLegacyEntry(
appsById?: Map<number, Record<string, unknown>>;
/** Omitted from the response when undefined. */
isShared?: boolean | null;
/** Skip the mask: the mask is the actor's, and only they can read it. */
forOwner?: boolean;
parentUid?: string | null;
} = {},
): Promise<Record<string, unknown>> {
// Someone else's entry is published under its masked path; the owner's
// real one, and everything above the share, stays server-side.
const publishedPath = maskEntryPath(entry);
const publishedPath = opts.forOwner ? entry.path : maskEntryPath(entry);
const dirname = pathPosix.dirname(publishedPath);
const mimeType = fsEntryMimeType(entry);
@@ -463,12 +494,17 @@ export async function toLegacyEntry(
const appdata_app =
pathComponents[2] === 'AppData' ? pathComponents[3] : undefined;
const parentUid =
opts.parentUid !== undefined
? opts.parentUid
: (entry.parentUid ?? null);
const response: Record<string, unknown> = {
id: entry.uuid,
uid: entry.uuid,
uuid: entry.uuid,
parent_id: entry.parentUid,
parent_uid: entry.parentUid,
parent_id: parentUid,
parent_uid: parentUid,
path: publishedPath,
dirname,
dirpath: dirname,
@@ -563,7 +599,11 @@ export function signingConfigFromAppConfig(config: IConfig): SigningConfig {
{ legacyCode: 'internal_error' },
);
}
return { secret, apiBaseUrl };
return {
secret,
apiBaseUrl,
allowLegacySignatures: config.legacy_file_signatures !== false,
};
}
/**
@@ -571,7 +611,7 @@ export function signingConfigFromAppConfig(config: IConfig): SigningConfig {
*
* Pass `actorUserId` so a signature over someone else's entry — a shared file —
* expires rather than outliving the share (see
* NON_OWNER_SIGNATURE_TTL_SECONDS).
* NON_OWNER_SIGNATURE_TTL_SECONDS). The signature is bound to `entry.userId`.
*/
export function signEntry(
entry: {
@@ -582,7 +622,7 @@ export function signEntry(
accessed: number | null;
modified: number;
created: number | null;
userId?: number;
userId: number;
},
config: SigningConfig,
opts: { actorUserId?: number; ttlSeconds?: number } = {},
+4 -1
View File
@@ -28,7 +28,10 @@ import type { RouteOptions, RouteRateLimit } from '../../core/http/types';
type Spec = RouteRateLimit | NonNullable<RouteOptions['concurrent']>;
const all = Object.entries(limits) as Array<[string, Spec | RouteRateLimit[]]>;
// Rate and concurrency specs only; the per-request item caps are plain numbers.
const all = Object.entries(limits).filter(
([, value]) => typeof value === 'object',
) as Array<[string, Spec | RouteRateLimit[]]>;
const flat: Array<[string, Spec]> = all.flatMap(([name, value]) =>
Array.isArray(value)
? value.map((v, i): [string, Spec] => [`${name}[${i}]`, v])
+27 -5
View File
@@ -185,15 +185,37 @@ export const FS_HELPER_LIMIT = userWindow('fs:helper', 120, 60, 30);
*/
export const FS_POLL_LIMIT = userWindow('fs:poll', 240, 240, 120);
// -- Per-request item caps -------------------------------------------
//
// The windows above count requests; these bound what one request can fan out
// to. Each item costs its own path resolution and ACL check, and an upload item
// also a pending session and presigned URLs. Checked before any per-item work.
/**
* `startBatchWrite` / `completeBatchWrite` / `batchWrite`. Puter.js chunks
* uploads at this size.
*/
export const FS_BATCH_WRITE_MAX_ITEMS = 500;
/** Entries per `/sign`. Each may also write an app grant. */
export const FS_SIGN_MAX_ITEMS = 500;
/**
* Part numbers per `signMultipartParts`: every part a multipart upload can have
* (1-10,000).
*/
export const FS_MULTIPART_MAX_PARTS = 10_000;
// -- Legacy multipart upload -----------------------------------------
/**
* `/batch` buffers every file fully into memory before any quota or storage
* check runs, up to BATCH_MAX_FILES × BATCH_MAX_FILE_SIZE. The concurrency slot
* is doing the real work here; the window is secondary, and sized to say so —
* one upload is one call, so a per-minute ceiling in the tens is a cap on how
* many files someone may upload rather than a bound on cost. What actually
* bounds the memory this route can tie up is how many run at once.
* check runs, up to BATCH_MAX_FILES × BATCH_MAX_FILE_SIZE, capped in aggregate
* per request by BATCH_MAX_TOTAL_SIZE. The concurrency slot is doing the real
* work here; the window is secondary, and sized to say so — one upload is one
* call, so a per-minute ceiling in the tens is a cap on how many files someone
* may upload rather than a bound on cost. What actually bounds the memory this
* route can tie up is how many run at once.
*/
export const FS_BATCH_LIMIT = userWindow('fs:batch', 600, 300, 300);
export const FS_BATCH_CONCURRENT = userConcurrent('fs:batch', 5, 2, 2);
@@ -0,0 +1,197 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import {
createTestUser,
setupPuterTestEnv,
type PuterTestEnv,
} from '../../testUtil.js';
// A recipient addresses the owner's entries by a masked `/<owner>/<uuid>/<name>`
// path. Routes that create an entry have no row to resolve, so they unmask it
// themselves before the ACL check runs against the result.
describe('writes inside a folder shared with write', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 180_000);
afterAll(async () => {
await env?.shutdown();
});
const post = async (route: string, token: string, body: unknown) => {
const res = await fetch(new URL(route, env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
const text = await res.text();
return {
status: res.status,
json: text ? (JSON.parse(text) as Record<string, unknown>) : null,
};
};
const makeUser = () =>
createTestUser(env.server, {
username: `sfw${Math.random().toString(36).slice(2, 9)}`,
password: 'puter-test-user-password',
});
/** `/<owner>/SharedDir` with `note.txt`, shared `write` with a recipient. */
const shareAFolder = async () => {
const owner = await makeUser();
const recipient = await makeUser();
const dir = await post('/mkdir', owner.token, {
path: `/${owner.username}/SharedDir`,
});
await post('/touch', owner.token, {
path: `/${owner.username}/SharedDir/note.txt`,
});
const note = await post('/stat', owner.token, {
path: `/${owner.username}/SharedDir/note.txt`,
});
await post('/share', owner.token, {
recipients: [recipient.username],
items: [{ uid: dir.json!.uid }],
mode: 'write',
});
return {
owner,
recipient,
realDir: `/${owner.username}/SharedDir`,
maskedDir: `/${owner.username}/${dir.json!.uid}/SharedDir`,
noteUid: note.json!.uid as string,
};
};
const ownerListing = async (s: { owner: { token: string; username: string } }) => {
const res = await post('/readdir', s.owner.token, {
path: `/${s.owner.username}/SharedDir`,
});
return (res.json as unknown as Array<{ name: string }>).map(
(entry) => entry.name,
);
};
// `{parent, path}` is the shape puter.js sends — the GUI's "New Folder".
it('mkdir accepts a masked parent', async () => {
const s = await shareAFolder();
const made = await post('/mkdir', s.recipient.token, {
parent: s.maskedDir,
path: 'New Folder',
});
expect(made.status).toBe(200);
expect(made.json).toMatchObject({
name: 'New Folder',
is_dir: true,
// Anchored to the share root the request came through, so the
// recipient can navigate into what they just made.
path: `${s.maskedDir}/New Folder`,
});
expect(await ownerListing(s)).toContain('New Folder');
}, 180_000);
it('mkdir accepts a masked absolute path', async () => {
const s = await shareAFolder();
const made = await post('/mkdir', s.recipient.token, {
path: `${s.maskedDir}/Reports`,
});
expect(made.status).toBe(200);
expect(await ownerListing(s)).toContain('Reports');
}, 180_000);
it('touch accepts a masked path', async () => {
const s = await shareAFolder();
const touched = await post('/touch', s.recipient.token, {
path: `${s.maskedDir}/from-recipient.txt`,
});
expect(touched.status).toBe(200);
expect(await ownerListing(s)).toContain('from-recipient.txt');
}, 180_000);
it('touch still expands a tilde path', async () => {
const s = await shareAFolder();
const touched = await post('/touch', s.owner.token, {
path: '~/SharedDir/from-tilde.txt',
});
expect(touched.status).toBe(200);
expect(await ownerListing(s)).toContain('from-tilde.txt');
}, 180_000);
// Addressed to the owner, so it carries the path the owner can resolve.
it('publishes the owner’s real path on events about their entries', async () => {
const s = await shareAFolder();
const seen: Array<Record<string, unknown>> = [];
const emit = env.server.clients.event.emit.bind(
env.server.clients.event,
);
vi.spyOn(env.server.clients.event, 'emit').mockImplementation(
((name: string, payload: Record<string, unknown>, meta: unknown) => {
if (name.startsWith('outer.gui.item.')) {
seen.push(payload);
}
return emit(
name as never,
payload as never,
meta as never,
);
}) as never,
);
const moved = await post('/move', s.recipient.token, {
source: s.noteUid,
destination: s.maskedDir,
new_name: 'renamed.txt',
});
expect(moved.status).toBe(200);
const ownerId = (await env.server.stores.user.getByUsername(
s.owner.username,
))!.id;
const event = seen.find(
(payload) =>
(payload.response as { uid?: string })?.uid === s.noteUid,
);
expect(event?.user_id_list).toEqual([ownerId]);
expect(event?.response).toMatchObject({
path: `${s.realDir}/renamed.txt`,
dirpath: s.realDir,
});
// The recipient's own response stays masked.
expect(
(moved.json!.moved as { path: string }).path,
).toBe(`${s.maskedDir}/renamed.txt`);
}, 180_000);
});
@@ -427,6 +427,25 @@ describe('HomepageController GET /app/:name', () => {
});
});
// -- app landing rate limit --
describe('HomepageController app landing rate limit', () => {
it('rate-limits /app/:name and /desktop/app/:name per network', () => {
for (const path of ['/app/:name', '/desktop/app/:name']) {
const route = router.routes.find(
(r) => r.method === 'get' && r.path === path,
);
if (!route) throw new Error(`No GET ${path} route`);
expect(route.options.rateLimit).toEqual({
scope: 'app-landing',
limit: 600,
window: 60_000,
key: 'ip',
});
}
});
});
// ── /show/* ─────────────────────────────────────────────────────────
describe('HomepageController GET /show/*splat', () => {
@@ -31,6 +31,17 @@ import type {
/** Longest description/og:description/twitter:description on `/app/<name>`. */
export const APP_META_DESCRIPTION_MAX = 150;
/**
* Unauthenticated, rendered on every request, so the bucket is the network
* address rather than an account.
*/
const APP_LANDING_LIMIT = {
scope: 'app-landing',
limit: 600,
window: 60_000,
key: 'ip' as const,
};
/**
* App descriptions can run long; search snippets and social cards want a
* short blurb. Collapses whitespace, then cuts at a word boundary with an
@@ -160,8 +171,16 @@ export class HomepageController extends PuterController {
});
};
router.get('/app/:name', {}, sendAppShell);
router.get('/desktop/app/:name', {}, sendAppShell);
router.get(
'/app/:name',
{ rateLimit: APP_LANDING_LIMIT },
sendAppShell,
);
router.get(
'/desktop/app/:name',
{ rateLimit: APP_LANDING_LIMIT },
sendAppShell,
);
// -- /show/* - launch explorer with the requested file path --
@@ -93,6 +93,22 @@ export class HostingController extends PuterController {
userId: req.actor.user.id,
});
// Same event the `puter-subdomains` driver fires on delete —
// listeners (CDN purge, abuse tracking, worker cleanup)
// depend on it.
try {
this.clients.event.emit(
'subdomain.delete',
{
subdomain: row.subdomain,
uid: String(row.uuid),
},
{},
);
} catch {
// Non-critical.
}
res.json({});
},
);
@@ -181,4 +181,24 @@ describe('HostingController /delete-site', () => {
expect(captured.body).toEqual({});
expect(await server.stores.subdomain.getByUuid(site.uuid)).toBeNull();
});
it('emits subdomain.delete so listeners can purge caches', async () => {
const { userId, actor } = await makeUser();
const site = await makeSite(userId);
const emitSpy = vi.spyOn(server.clients.event, 'emit');
try {
await deleteSite(
makeReq({ body: { site_uuid: site.uuid }, actor }),
makeRes().res,
);
expect(emitSpy).toHaveBeenCalledWith(
'subdomain.delete',
{ subdomain: site.subdomain, uid: String(site.uuid) },
{},
);
} finally {
emitSpy.mockRestore();
}
});
});
+2
View File
@@ -40,11 +40,13 @@ import { WebDAVController } from './webdav/WebDAVController.js';
import { WispController } from './wisp/WispController.js';
import type { IPuterControllerRegistry } from './types.js';
import { PeerController } from './peer/PeerController.js';
import { ProfileController } from './profile/ProfileController.js';
export const puterControllers = {
staticAssets: StaticAssetsController,
staticPages: StaticPagesController,
auth: AuthController,
profile: ProfileController,
apps: AppController,
appFeedback: AppFeedbackController,
desktop: DesktopController,
@@ -58,6 +58,10 @@ let router: PuterRouter;
// override in and out (same pattern as AuthController.test.ts).
type SignupValidateOverride = (data: Record<string, unknown>) => void;
let signupValidateOverride: SignupValidateOverride | null = null;
// Stand-in for the extension that honors signup bonus codes.
let bonusValidateOverride: SignupValidateOverride | null = null;
let bonusCheckOverride: SignupValidateOverride | null = null;
const heardSignupSuccess: Array<Record<string, unknown>> = [];
beforeAll(async () => {
server = await setupTestServer({
@@ -90,6 +94,24 @@ beforeAll(async () => {
}
},
);
server.clients.event.on(
'puter.signup-bonus.check',
(_k: unknown, data: unknown) => {
bonusCheckOverride?.(data as Record<string, unknown>);
},
);
server.clients.event.on(
'puter.signup-bonus.validate',
(_k: unknown, data: unknown) => {
bonusValidateOverride?.(data as Record<string, unknown>);
},
);
server.clients.event.on(
'puter.signup.success',
(_k: unknown, data: unknown) => {
heardSignupSuccess.push(data as Record<string, unknown>);
},
);
});
afterAll(async () => {
@@ -99,6 +121,8 @@ afterAll(async () => {
afterEach(() => {
vi.restoreAllMocks();
signupValidateOverride = null;
bonusValidateOverride = null;
bonusCheckOverride = null;
});
interface CapturedResponse {
@@ -424,9 +448,14 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
return String(oidc().verifyState(state!)?.redirect_uri);
};
const returnToFor = (paths: string[], path = '/') => {
const returnToFor = (
paths: string[],
path = '/',
recipientUuid?: string,
) => {
const params = new URLSearchParams();
for (const p of paths) params.append('shared', p);
if (recipientUuid) params.set('user_uuid', recipientUuid);
return `${path}?${params.toString()}`;
};
@@ -460,6 +489,16 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
]);
});
it('carries a valid recipient account hint with the share', async () => {
const recipientUuid = uuidv4();
const uri = await redirectUriFor(
returnToFor([sharedPath('Report.pdf')], '/', recipientUuid),
);
expect(new URL(uri).searchParams.get('user_uuid')).toBe(
recipientUuid,
);
});
it('carries no more items than a share link may name', async () => {
const paths = Array.from({ length: 25 }, (_, i) =>
sharedPath(`file-${i}.txt`),
@@ -481,6 +520,8 @@ describe('OIDCController GET /auth/oidc/:provider/start', () => {
// a parameter that isn't `shared`, alone or alongside one
'/?x=1',
`${returnToFor([sharedPath('a.txt')])}&x=1`,
`${returnToFor([sharedPath('a.txt')])}&user_uuid=not-a-uuid`,
`${returnToFor([sharedPath('a.txt')])}&user_uuid=${uuidv4()}&user_uuid=${uuidv4()}`,
// the root is only a destination when it names something
'/',
// still no origin smuggling, share link or not
@@ -660,6 +701,26 @@ describe('OIDCController login callback', () => {
expect(captured.redirectUrl).toContain('auth_error=1');
});
it('redirects with auth_error when the provider is unreachable', async () => {
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockRejectedValue(
new TypeError('fetch failed'),
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/login',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.redirectStatus).toBe(302);
expect(captured.redirectUrl).toContain('auth_error=1');
});
it('parses code/state from the POST body (Apple form_post)', async () => {
const state = oidc().signState({
provider: 'custom',
@@ -830,7 +891,7 @@ describe('OIDCController login callback', () => {
const state = oidc().signState({
provider: 'custom',
redirect_uri: `${TEST_ORIGIN}/?shared=${encodeURIComponent(shared)}`,
redirect_uri: `${TEST_ORIGIN}/?shared=${encodeURIComponent(shared)}&user_uuid=${encodeURIComponent(created.user!.uuid)}`,
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
@@ -853,6 +914,7 @@ describe('OIDCController login callback', () => {
expect(url.searchParams.get('auth_error')).toBe('1');
expect(url.searchParams.get('action')).toBe('login');
expect(url.searchParams.getAll('shared')).toEqual([shared]);
expect(url.searchParams.get('user_uuid')).toBe(created.user!.uuid);
});
it('redirects back to an /app/<name> landing after sign-in', async () => {
@@ -952,7 +1014,6 @@ describe('OIDCController login callback', () => {
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.cookies).toHaveLength(1);
expect(captured.redirectUrl).toContain('embedded_in_popup=true');
expect(captured.redirectUrl).toContain('oidc_login=true');
@@ -966,6 +1027,15 @@ describe('OIDCController login callback', () => {
const user = await server.stores.user.getByEmail(email);
expect(user?.uuid).toBeTruthy();
expect(proof?.user_uuid).toBe(user!.uuid);
// Session cookie plus the proof's binding companion.
expect(captured.cookies).toHaveLength(2);
const binding = captured.cookies.find(
(c) => c.name === 'puter_oidc_popup_return',
);
expect(binding?.value).toBeTruthy();
expect(proof?.nonce).toBe(binding!.value);
expect(proof?.action).toBe('sign-in');
});
it('uses popup-style error URL (msg_id + opener_origin) when the popup-state user is suspended', async () => {
@@ -1017,6 +1087,14 @@ describe('OIDCController login callback', () => {
expect(captured.redirectUrl).toContain(
`opener_origin=${encodeURIComponent('http://opener.test')}`,
);
// The error leg must stamp the action its own redirect names.
const url = new URL(captured.redirectUrl!);
const proof = oidc().verifyPopupReturn(
url.searchParams.get('opener_state')!,
);
expect(proof?.oidc_login).toBe(false);
expect(proof?.action).toBe(url.searchParams.get('action'));
});
it('links an OIDC identity to an existing CONFIRMED password account via email match', async () => {
@@ -1668,6 +1746,30 @@ describe('OIDCController revalidate callback', () => {
expect(String(captured.body)).toContain('Missing');
});
it('returns 400 when the userinfo request cannot reach the provider', async () => {
const state = oidc().signState({
provider: 'custom',
flow: 'revalidate',
user_uuid: uuidv4(),
});
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockRejectedValue(
new TypeError('fetch failed'),
);
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/revalidate',
makeReq({ query: { code: 'c', state } }),
res,
);
expect(captured.statusCode).toBe(400);
expect(String(captured.body)).toContain('Please try again');
});
it('returns 403 when the OIDC sub resolves to a different account than the session', async () => {
// The linked account exists, but state.user_uuid points at someone else.
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
@@ -1835,19 +1937,34 @@ describe('OIDCController GET /auth/revalidate-done', () => {
* attested rather than read.
*/
describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
const redeem = async (opener_state: unknown) => {
const NONCE = 'binding-nonce';
const mint = (payload: Record<string, unknown>) =>
server.services.oidc.signPopupReturn({
nonce: NONCE,
action: 'sign-in',
...payload,
});
const redeemWith = (openerState: unknown, nonce: string | null = NONCE) =>
makeReq({
body: { opener_state: openerState },
cookies: nonce === null ? {} : { puter_oidc_popup_return: nonce },
});
const redeem = async (openerState: unknown) => {
const { res, captured } = makeRes();
await callRoute(
'post',
'/auth/oidc/verify-popup-return',
makeReq({ body: { opener_state } }),
redeemWith(openerState),
res,
);
return captured;
};
it('hands back what a genuine proof attests', async () => {
const proof = server.services.oidc.signPopupReturn({
const proof = mint({
opener_origin: 'https://opener.test',
msg_id: '77',
oidc_login: true,
@@ -1856,6 +1973,7 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
expect(captured.body).toEqual({
opener_origin: 'https://opener.test',
msg_id: '77',
action: 'sign-in',
oidc_login: true,
user_uuid: null,
});
@@ -1864,7 +1982,7 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
it('hands back the account a proof is bound to', async () => {
// The popup compares this against its current user to reject a proof
// replayed from another account's login.
const proof = server.services.oidc.signPopupReturn({
const proof = mint({
opener_origin: 'https://opener.test',
msg_id: '77',
oidc_login: true,
@@ -1875,9 +1993,13 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
});
it('rejects a proof signed with someone else’s key', async () => {
// The whole point: only the server can mint one of these.
const forged = jwt.sign(
{ opener_origin: 'https://console.puter.com', oidc_login: true },
{
opener_origin: 'https://console.puter.com',
oidc_login: true,
purpose: 'popup-return',
nonce: NONCE,
},
'not-the-server-secret',
{ keyid: 'v2' },
);
@@ -1885,7 +2007,7 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
callRoute(
'post',
'/auth/oidc/verify-popup-return',
makeReq({ body: { opener_state: forged } }),
redeemWith(forged),
makeRes().res,
),
).rejects.toMatchObject({ statusCode: 400 });
@@ -1896,14 +2018,19 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
// redeemed on the very next request, so a stale one is never genuine.
const stale = server.services.token.sign(
'oidc-state',
{ opener_origin: 'https://opener.test', oidc_login: true },
{
opener_origin: 'https://opener.test',
oidc_login: true,
purpose: 'popup-return',
nonce: NONCE,
},
{ expiresIn: -600 },
);
await expect(
callRoute(
'post',
'/auth/oidc/verify-popup-return',
makeReq({ body: { opener_state: stale } }),
redeemWith(stale),
makeRes().res,
),
).rejects.toMatchObject({ statusCode: 400 });
@@ -1915,7 +2042,7 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
callRoute(
'post',
'/auth/oidc/verify-popup-return',
makeReq({ body: { opener_state: bad } }),
redeemWith(bad),
makeRes().res,
),
).rejects.toMatchObject({ statusCode: 400 });
@@ -1925,13 +2052,82 @@ describe('OIDCController POST /auth/oidc/verify-popup-return', () => {
it('reports oidc_login false when the proof does not claim a login', async () => {
// The error leg mints one of these: a real return, but nothing was
// signed in on it, so it must not suppress the account picker.
const proof = server.services.oidc.signPopupReturn({
const proof = mint({
opener_origin: 'https://opener.test',
msg_id: '77',
oidc_login: false,
});
expect((await redeem(proof)).body).toMatchObject({ oidc_login: false });
});
it('rejects an authorization state presented as a proof', async () => {
const state = server.services.oidc.signState({
provider: 'google',
redirect_uri: 'https://puter.test/action/sign-in',
embedded_in_popup: true,
msg_id: '77',
opener_origin: 'https://opener.test',
nonce: NONCE,
});
await expect(
callRoute(
'post',
'/auth/oidc/verify-popup-return',
redeemWith(state),
makeRes().res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rejects a proof redeemed without its binding cookie', async () => {
const proof = mint({
opener_origin: 'https://opener.test',
msg_id: '77',
oidc_login: true,
});
for (const cookie of [null, 'a-different-nonce']) {
await expect(
callRoute(
'post',
'/auth/oidc/verify-popup-return',
redeemWith(proof, cookie),
makeRes().res,
),
).rejects.toMatchObject({ statusCode: 400 });
}
});
it('clears the binding cookie so a proof redeems once', async () => {
const proof = mint({
opener_origin: 'https://opener.test',
msg_id: '77',
oidc_login: true,
});
const { res, captured } = makeRes();
await callRoute(
'post',
'/auth/oidc/verify-popup-return',
redeemWith(proof),
res,
);
expect(captured.clearedCookies).toContainEqual(
expect.objectContaining({ name: 'puter_oidc_popup_return' }),
);
});
it('leaves the binding cookie in place when redemption fails', async () => {
// A junk POST must not consume a cookie an in-flight return needs.
const { res, captured } = makeRes();
await expect(
callRoute(
'post',
'/auth/oidc/verify-popup-return',
redeemWith('not.a.proof'),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
expect(captured.clearedCookies).toEqual([]);
});
});
// ── rate-limit scopes ───────────────────────────────────────────────
@@ -1965,3 +2161,143 @@ describe('OIDCController rate limits', () => {
expect(rateLimitOf('get', '/auth/oidc/providers').limit).toBe(1_200);
});
});
// -- Signup bonus codes ----------------------------------------------
describe('OIDC signup bonus codes', () => {
const rand = () => Math.random().toString(36).slice(2, 8);
const startState = async (query: Record<string, unknown>) => {
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/:provider/start',
makeReq({ params: { provider: 'custom' }, query }),
res,
);
const state = new URL(captured.redirectUrl ?? '').searchParams.get(
'state',
);
return oidc().verifyState(state!) as Record<string, unknown>;
};
const stubIdp = (sub: string, email: string) => {
vi.spyOn(oidc(), 'exchangeCodeForTokens').mockResolvedValue({
access_token: 'access',
id_token: 'id',
} as never);
vi.spyOn(oidc(), 'getUserInfo').mockResolvedValue({
sub,
email,
email_verified: true,
} as never);
};
const openCheck = (data: Record<string, unknown>) => {
data.valid = true;
data.display = { title: 'T', description: 'D' };
};
const signupCallback = async (bonusCode: string) => {
const state = oidc().signState({
provider: 'custom',
redirect_uri: TEST_ORIGIN + '/',
bonus_code: bonusCode,
});
const { res, captured } = makeRes();
await callRoute(
'get',
'/auth/oidc/callback/signup',
makeReq({ query: { code: 'c', state } }),
res,
);
return captured;
};
it('carries a canonical code in the signed state', async () => {
const decoded = await startState({
flow: 'signup',
bonusCode: 'Startup3M-ABCD1234',
});
expect(decoded.bonus_code).toBe('startup3mabcd1234');
});
it('drops a malformed code at start', async () => {
const decoded = await startState({ flow: 'signup', bonusCode: 'a:b' });
expect(decoded).not.toHaveProperty('bonus_code');
});
it('creates the account with the requirements an accepted code raises', async () => {
const email = `bonus-${rand()}@test.local`;
stubIdp(`sub-${rand()}`, email);
bonusCheckOverride = openCheck;
let seen: Record<string, unknown> | null = null;
bonusValidateOverride = (data) => {
seen = { ...data };
data.accepted = true;
data.requires_phone_verification = true;
};
const captured = await signupCallback('startup3mabcd1234');
expect(captured.cookies).toHaveLength(1);
expect(seen).toMatchObject({
code: 'startup3mabcd1234',
source: 'oidc',
email,
});
const user = await server.stores.user.findEmailOwner(email, {
force: true,
});
expect(Boolean(user!.requires_phone_verification)).toBe(true);
const deadline = Date.now() + 2000;
let success: Record<string, unknown> | undefined;
while (!success && Date.now() < deadline) {
success = heardSignupSuccess.find((e) => e.email === email);
if (!success) await new Promise((r) => setTimeout(r, 10));
}
expect(success?.bonus_code).toBe('startup3mabcd1234');
});
it('refuses a dead code before the validate hook runs', async () => {
const email = `bonus-${rand()}@test.local`;
stubIdp(`sub-${rand()}`, email);
let validated = false;
signupValidateOverride = (data) => {
if (data.email === email) validated = true;
};
const captured = await signupCallback('startup3mabcd1234');
expect(captured.redirectUrl).toContain('message=bonus_code_invalid');
expect(validated).toBe(false);
});
it('refuses the account when nothing accepts the code, without echoing it', async () => {
const email = `bonus-${rand()}@test.local`;
stubIdp(`sub-${rand()}`, email);
const captured = await signupCallback('startup3mabcd1234');
expect(captured.redirectUrl).toContain('message=bonus_code_invalid');
expect(captured.redirectUrl).not.toContain('bonusCode=');
expect(captured.cookies).toHaveLength(0);
expect(
await server.stores.user.findEmailOwner(email, { force: true }),
).toBeFalsy();
});
it('keeps the code on the retry redirect when something else failed', async () => {
const email = `bonus-${rand()}@test.local`;
stubIdp(`sub-${rand()}`, email);
bonusCheckOverride = openCheck;
signupValidateOverride = (data) => {
if (data.email === email) data.allow = false;
};
const captured = await signupCallback('startup3mabcd1234');
expect(captured.redirectUrl).toContain('message=signup_blocked');
expect(captured.redirectUrl).toContain('bonusCode=startup3mabcd1234');
});
});
+153 -48
View File
@@ -18,15 +18,18 @@
*/
import crypto from 'node:crypto';
import { validate as validateUuid } from 'uuid';
import type { Request, Response } from 'express';
import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterController } from '../types.js';
import { sessionCookieFlags } from '../../util/cookieFlags.js';
import { normalizeBonusCode } from '../../util/signupBonus.js';
import { parseMaskedSharePath } from '../../services/fs/sharePathMask.js';
import {
SHARE_DEEP_LINK_ITEMS_LIMIT,
SHARE_DEEP_LINK_PARAM,
SHARE_RECIPIENT_PARAM,
} from '../../services/share/shareDeepLink.js';
const REVALIDATION_COOKIE_NAME = 'puter_revalidation';
@@ -38,6 +41,14 @@ const REVALIDATION_EXPIRY_SEC = 300;
const OIDC_NONCE_COOKIE_NAME = 'puter_oidc_nonce';
const OIDC_NONCE_EXPIRY_SEC = 600;
// Single-use, and binds a popup-return proof to the browser that earned it.
// Expiry mirrors POPUP_RETURN_EXPIRY_SEC in OIDCService.
const OIDC_POPUP_RETURN_COOKIE_NAME = 'puter_oidc_popup_return';
const OIDC_POPUP_RETURN_EXPIRY_SEC = 300;
// The popup branch below hard-codes this return target.
const POPUP_RETURN_ACTION = 'sign-in';
const OIDC_ERROR_REDIRECT_MAP: Record<string, Record<string, string>> = {
login: { account_not_found: 'signup', other: 'login' },
signup: { account_already_exists: 'login', other: 'signup' },
@@ -50,6 +61,7 @@ const ALLOWED_ERRORS = [
'account_suspended',
'unauthorized',
'signup_blocked',
'bonus_code_invalid',
] as const;
/**
@@ -69,6 +81,23 @@ function resolutionErrorCode(code: string | undefined): string {
: 'signup_blocked';
}
/** What a new account created by an OIDC callback inherits from its flow. */
interface OIDCSignupOptions {
referrer?: string | null;
bonusCode?: string | null;
}
function signupOptionsFromState(
stateDecoded: Record<string, unknown>,
): OIDCSignupOptions {
return {
referrer: (stateDecoded.referrer as string) ?? null,
// Normalized when the state was signed; re-checked since this is the
// value signup acts on.
bonusCode: normalizeBonusCode(stateDecoded.bonus_code),
};
}
// GUI pages an OIDC flow may return to: the root (where a share email lands),
// /desktop, /dashboard, and direct app landings (/app/<name> and its
// desktop-booted twin /desktop/app/<name>, mirroring APP_NAME_REGEX in
@@ -93,14 +122,23 @@ function isWhitelistedReturnPath(path: string): boolean {
* or the recipient returns to a bare Home with nothing to say what they were
* sent.
*
* `shared` is the only parameter that makes the trip, and only values shaped
* like the masked path the mail was built from — the value is user-visible
* text, so a hand-edited one is refused rather than reflected back into the
* browser.
* Only `shared` makes the trip, with values shaped like the masked path the
* mail was built from — the value is user-visible text, so a hand-edited one is
* refused rather than reflected back into the browser. The mail's one recipient
* hint (`user_uuid`) rides along when it is a single uuid.
*/
function sharedPathsFromReturnQuery(query: string): string[] | null {
function sharedReturnQuery(query: string): {
paths: string[];
recipientUuid?: string;
} | null {
const paths: string[] = [];
let recipientUuid: string | undefined;
for (const [key, value] of new URLSearchParams(query)) {
if (key === SHARE_RECIPIENT_PARAM) {
if (recipientUuid || !validateUuid(value)) return null;
recipientUuid = value;
continue;
}
if (key !== SHARE_DEEP_LINK_PARAM) return null;
const parsed = parseMaskedSharePath(value);
// The segment after the uuid is the shared item itself. A mask without
@@ -116,7 +154,7 @@ function sharedPathsFromReturnQuery(query: string): string[] | null {
paths.push(value);
}
}
return paths;
return { paths, recipientUuid };
}
/**
@@ -135,15 +173,19 @@ function sanitizeReturnTo(raw: string): string | null {
const shared =
separator === -1
? []
: sharedPathsFromReturnQuery(raw.slice(separator + 1));
? { paths: [] }
: sharedReturnQuery(raw.slice(separator + 1));
if (shared === null) return null;
// The root is only a destination when it names something: on its own it is
// where the flow already lands.
if (shared.length === 0) return path === '/' ? null : path;
if (shared.paths.length === 0) return path === '/' ? null : path;
const params = new URLSearchParams();
for (const value of shared) params.append(SHARE_DEEP_LINK_PARAM, value);
for (const value of shared.paths)
params.append(SHARE_DEEP_LINK_PARAM, value);
if (shared.recipientUuid) {
params.set(SHARE_RECIPIENT_PARAM, shared.recipientUuid);
}
return `${path}?${params.toString()}`;
}
@@ -154,11 +196,8 @@ function buildErrorRedirectUrl(
message: string,
stateDecoded?: Record<string, unknown>,
requestCode?: string,
// Signs the popup-return proof. Passed in because this is a module-level
// helper with no access to services; omitted by callers that have no
// state to attest (the proof is simply absent then, and the popup falls
// back to its browser-attested sources).
signPopupReturn?: (payload: Record<string, unknown>) => string,
// Omitted by callers with no state to attest.
mintPopupReturn?: (payload: Record<string, unknown>) => string,
): string {
const targetFlow =
OIDC_ERROR_REDIRECT_MAP[sourceFlow]?.[errorCondition] ?? sourceFlow;
@@ -175,12 +214,15 @@ function buildErrorRedirectUrl(
// its success reloads it, so they have to be on it to survive.
let pagePath = '/';
let sharedPaths: string[] = [];
let shareRecipientUuid: string | undefined;
if (typeof stateDecoded?.redirect_uri === 'string') {
try {
const stateUrl = new URL(stateDecoded.redirect_uri);
if (isWhitelistedReturnPath(stateUrl.pathname)) {
pagePath = stateUrl.pathname;
sharedPaths = sharedPathsFromReturnQuery(stateUrl.search) ?? [];
const shared = sharedReturnQuery(stateUrl.search);
sharedPaths = shared?.paths ?? [];
shareRecipientUuid = shared?.recipientUuid;
}
} catch {
// unparsable redirect_uri: fall back to the root page
@@ -202,13 +244,15 @@ function buildErrorRedirectUrl(
// Same reasoning as the success leg: the popup cannot tell a verified
// `opener_origin` from a typed one, so attest it. The error leg is a
// real return from the provider too — the flow failed, not the hop.
if (signPopupReturn) {
if (mintPopupReturn) {
params.set(
'opener_state',
signPopupReturn({
mintPopupReturn({
opener_origin: stateDecoded?.opener_origin ?? null,
msg_id: stateDecoded?.msg_id ?? null,
oidc_login: false,
// The popup checks this against the action it lands on.
action: targetFlow,
}),
);
}
@@ -222,9 +266,19 @@ function buildErrorRedirectUrl(
if (requestCode) {
params.set('request_code', requestCode);
}
// A retry from the error page keeps the bonus code, unless it's what failed.
if (
typeof stateDecoded?.bonus_code === 'string' &&
clamped !== 'bonus_code_invalid'
) {
params.set('bonusCode', stateDecoded.bonus_code);
}
for (const path of sharedPaths) {
params.append(SHARE_DEEP_LINK_PARAM, path);
}
if (shareRecipientUuid) {
params.set(SHARE_RECIPIENT_PARAM, shareRecipientUuid);
}
return `${base}${pagePath}?${params.toString()}`;
}
@@ -267,18 +321,21 @@ export class OIDCController extends PuterController {
// A sign-in popup returning from a provider is told the opener's
// origin and that a login completed. It cannot check either: the
// values arrive as query parameters, and a URL built from a verified
// `state` looks exactly like one an attacker typed. The opener's
// `state` looks exactly like one anybody can type. The opener's
// origin decides which app a token gets minted for, so the popup
// redeems the signed proof here instead of believing the raw
// parameters.
//
// Served on the GUI origin too: the binding cookie is host-only.
//
// Unauthenticated on purpose — it reveals nothing the caller did not
// already hand over, and a forged or expired proof yields nothing.
// already hand over, and a proof that isn't this browser's yields
// nothing.
router.post(
'/auth/oidc/verify-popup-return',
{
subdomain: 'api',
subdomain: ['api', ''],
rateLimit: {
scope: 'oidc-verify-popup-return',
limit: 60,
@@ -293,14 +350,25 @@ export class OIDCController extends PuterController {
});
}
const decoded = this.services.oidc.verifyPopupReturn(proof);
if (!decoded) {
const cookieNonce =
req.cookies?.[OIDC_POPUP_RETURN_COOKIE_NAME];
if (
!decoded ||
typeof decoded.nonce !== 'string' ||
typeof cookieNonce !== 'string' ||
!constantTimeEqual(cookieNonce, decoded.nonce)
) {
throw new HttpError(400, 'Invalid `opener_state`', {
legacyCode: 'bad_request',
});
}
// Single-use; a rejected call must not burn the cookie.
res.clearCookie(OIDC_POPUP_RETURN_COOKIE_NAME, { path: '/' });
res.json({
opener_origin: decoded.opener_origin ?? null,
msg_id: decoded.msg_id ?? null,
action: decoded.action ?? null,
oidc_login: decoded.oidc_login === true,
user_uuid: decoded.user_uuid ?? null,
});
@@ -404,7 +472,7 @@ export class OIDCController extends PuterController {
: null;
if (embeddedInPopup && msgId) {
appRedirectUri = `${origin}/action/sign-in?embedded_in_popup=true&msg_id=${encodeURIComponent(msgId)}`;
appRedirectUri = `${origin}/action/${POPUP_RETURN_ACTION}?embedded_in_popup=true&msg_id=${encodeURIComponent(msgId)}`;
if (openerOrigin) {
appRedirectUri += `&opener_origin=${encodeURIComponent(openerOrigin)}`;
}
@@ -423,6 +491,15 @@ export class OIDCController extends PuterController {
redirect_uri: appRedirectUri,
};
if (referrer) statePayload.referrer = referrer ?? openerOrigin;
// Login can create an account too, so both flows carry it. A
// malformed code is dropped here and never reaches signup.
const rawBonusCode = Array.isArray(req.query.bonusCode)
? req.query.bonusCode[0]
: req.query.bonusCode;
const bonusCode = normalizeBonusCode(rawBonusCode);
if (bonusCode && (flow === 'login' || flow === 'signup')) {
statePayload.bonus_code = bonusCode;
}
if (embeddedInPopup && msgId) {
statePayload.embedded_in_popup = true;
statePayload.msg_id = msgId;
@@ -507,7 +584,7 @@ export class OIDCController extends PuterController {
const resolved = await this.#resolveOrCreateOIDCUser(
provider,
userinfo,
(stateDecoded.referrer as string) ?? null,
signupOptionsFromState(stateDecoded),
);
if ('error' in resolved) {
console.warn(
@@ -522,7 +599,7 @@ export class OIDCController extends PuterController {
resolutionErrorCode(resolved.code),
stateDecoded,
resolved.requestCode,
(p) => this.services.oidc.signPopupReturn(p),
(p) => this.#mintPopupReturn(res, p),
),
);
}
@@ -541,7 +618,7 @@ export class OIDCController extends PuterController {
'account_suspended',
stateDecoded,
undefined,
(p) => this.services.oidc.signPopupReturn(p),
(p) => this.#mintPopupReturn(res, p),
),
);
}
@@ -574,7 +651,7 @@ export class OIDCController extends PuterController {
const resolved = await this.#resolveOrCreateOIDCUser(
provider,
userinfo,
(stateDecoded.referrer as string) ?? null,
signupOptionsFromState(stateDecoded),
);
if ('error' in resolved) {
console.warn(
@@ -589,7 +666,7 @@ export class OIDCController extends PuterController {
resolutionErrorCode(resolved.code),
stateDecoded,
resolved.requestCode,
(p) => this.services.oidc.signPopupReturn(p),
(p) => this.#mintPopupReturn(res, p),
),
);
}
@@ -605,7 +682,7 @@ export class OIDCController extends PuterController {
'account_suspended',
stateDecoded,
undefined,
(p) => this.services.oidc.signPopupReturn(p),
(p) => this.#mintPopupReturn(res, p),
),
);
}
@@ -722,6 +799,23 @@ if (window.opener) {
// -- Shared helpers ----------------------------------------------
/** Sign a popup-return proof, bound to this browser and this popup. */
#mintPopupReturn(res: Response, payload: Record<string, unknown>): string {
const nonce = crypto.randomBytes(32).toString('base64url');
res.cookie(OIDC_POPUP_RETURN_COOKIE_NAME, nonce, {
// Redeemed same-origin.
...sessionCookieFlags(this.config, { crossSite: false }),
httpOnly: true,
maxAge: OIDC_POPUP_RETURN_EXPIRY_SEC * 1000,
path: '/',
});
return this.services.oidc.signPopupReturn({
action: POPUP_RETURN_ACTION,
...payload,
nonce,
});
}
/**
* Resolve an OIDC callback to a Puter user. In order:
*
@@ -742,7 +836,7 @@ if (window.opener) {
async #resolveOrCreateOIDCUser(
provider: string,
userinfo: { sub: string; email?: unknown; [k: string]: unknown },
referrer?: string | null,
signup: OIDCSignupOptions = {},
attempt = 0,
): Promise<
| { error: string; code?: string; requestCode?: string }
@@ -792,7 +886,8 @@ if (window.opener) {
const outcome = await this.services.oidc.createUserFromOIDC(
provider,
userinfo as { sub: string; email?: string },
referrer,
signup.referrer ?? null,
{ bonusCode: signup.bonusCode ?? null },
);
// A concurrent callback (a second tab, a provider retry) created the
// account between step 2 and the insert. Nothing went wrong for the
@@ -802,7 +897,7 @@ if (window.opener) {
return this.#resolveOrCreateOIDCUser(
provider,
userinfo,
referrer,
signup,
attempt + 1,
);
}
@@ -868,23 +963,33 @@ if (window.opener) {
const callbackUrl = this.services.oidc.getCallbackUrl(flow);
if (!callbackUrl) return { error: 'Invalid flow.' };
const tokens = await this.services.oidc.exchangeCodeForTokens(
provider,
code,
callbackUrl,
);
if (!tokens || !tokens.access_token)
return { error: 'Token exchange failed.' };
try {
const tokens = await this.services.oidc.exchangeCodeForTokens(
provider,
code,
callbackUrl,
);
if (!tokens || !tokens.access_token)
return { error: 'Token exchange failed.' };
const userinfo = await this.services.oidc.getUserInfo(
provider,
tokens.access_token,
typeof tokens.id_token === 'string' ? tokens.id_token : undefined,
);
if (!userinfo || !userinfo.sub)
return { error: 'Could not get user info.' };
const userinfo = await this.services.oidc.getUserInfo(
provider,
tokens.access_token,
typeof tokens.id_token === 'string'
? tokens.id_token
: undefined,
);
if (!userinfo || !userinfo.sub)
return { error: 'Could not get user info.' };
return { provider, userinfo, stateDecoded };
return { provider, userinfo, stateDecoded };
} catch (e) {
// Network failures reaching the provider; the user can retry.
console.warn(`[oidc] ${provider} request failed`, e);
return {
error: 'Could not reach the sign-in provider. Please try again.',
};
}
}
async #finishLogin(
@@ -933,7 +1038,7 @@ if (window.opener) {
target = appendQueryParam(
target,
'opener_state',
this.services.oidc.signPopupReturn({
this.#mintPopupReturn(res, {
opener_origin: stateDecoded.opener_origin ?? null,
msg_id: stateDecoded.msg_id ?? null,
oidc_login: true,
@@ -458,6 +458,7 @@ describe('PeerController TURN', () => {
uuid: created.uuid,
id: created.id,
username: created.username,
email: created.email,
},
effectiveApp: null,
});
@@ -456,6 +456,7 @@ export class PeerController extends PuterController {
uuid: user.uuid,
id: user.id,
username: user.username,
email: user.email ?? null,
},
});
await this.services.metering.incrementUsage(
@@ -0,0 +1,152 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
const PICTURE =
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aD1sAAAAASUVORK5CYII=';
describe('profile routes over HTTP', () => {
let env: PuterTestEnv;
let userUuid = '';
/** The plan `env.users.user` is on; see the resolver below. */
let plan: string | null = null;
beforeAll(async () => {
env = await setupPuterTestEnv();
userUuid = (await env.server.stores.user.getByUsername(
env.users.user.username,
))!.uuid;
env.server.services.metering.registerPolicy({
id: 'business',
monthUsageAllowance: 45 * 1_000_000 * 100,
monthlyStorageAllowance: 1024 ** 3,
});
env.server.services.metering.registerSubscriptionResolver((actor) =>
actor.user?.uuid === userUuid ? plan : null,
);
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const withSubscription = async <T>(
id: string | null,
fn: () => Promise<T>,
): Promise<T> => {
plan = id;
env.server.services.metering.invalidateActorSubscription(userUuid);
try {
return await fn();
} finally {
plan = null;
env.server.services.metering.invalidateActorSubscription(userUuid);
}
};
const get = (query: string, token?: string) =>
fetch(new URL(`/profile${query}`, env.apiOrigin), {
headers: token ? { authorization: `Bearer ${token}` } : {},
});
const post = (body: unknown, token: string) =>
fetch(new URL('/profile', env.apiOrigin), {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${token}`,
},
body: JSON.stringify(body),
});
it('lets a user write and read their own profile on any plan', async () => {
const write = await post(
{ picture: PICTURE, displayName: 'Test' },
env.users.user.token,
);
expect(write.status).toBe(200);
expect(await write.json()).toEqual({
picture: PICTURE,
displayName: 'Test',
bio: null,
});
const own = await get('', env.users.user.token);
expect(own.status).toBe(200);
expect(await own.json()).toMatchObject({ picture: PICTURE });
const byName = await get(
`?username=${env.users.user.username}`,
env.users.user.token,
);
expect(byName.status).toBe(200);
expect(await byName.json()).toMatchObject({ displayName: 'Test' });
});
it('hides a free user from anonymous and third-party readers as a 404', async () => {
const anonymous = await get(`?username=${env.users.user.username}`);
expect(anonymous.status).toBe(404);
const other = await get(
`?username=${env.users.user.username}`,
env.users.other.token,
);
expect(other.status).toBe(404);
// Same shape as an account that does not exist.
const missing = await get('?username=nobody-here');
expect(missing.status).toBe(404);
});
it('serves a paid user to anyone, and hides them again when the plan ends', async () => {
await withSubscription('business', async () => {
const anonymous = await get(`?username=${env.users.user.username}`);
expect(anonymous.status).toBe(200);
expect(await anonymous.json()).toMatchObject({ picture: PICTURE });
const other = await get(
`?username=${env.users.user.username}`,
env.users.other.token,
);
expect(other.status).toBe(200);
});
expect((await get(`?username=${env.users.user.username}`)).status).toBe(
404,
);
});
it('requires a username or a signed-in caller, and validates the username shape', async () => {
expect((await get('')).status).toBe(401);
expect((await get('?username=not%20valid')).status).toBe(400);
});
it('rejects a bad patch with a stable code and refuses app and anonymous writers', async () => {
const bad = await post({ name: 'x' }, env.users.user.token);
expect(bad.status).toBe(400);
expect(await bad.json()).toMatchObject({
code: 'profile_field_not_allowed',
});
const anonymous = await fetch(new URL('/profile', env.apiOrigin), {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ bio: 'x' }),
});
expect(anonymous.status).toBe(401);
});
});
@@ -0,0 +1,103 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { Request, Response } from 'express';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { ProfileService } from '../../services/profile/ProfileService.js';
import { PuterController } from '../types.js';
// Shape only; the store decides whether the account exists.
const USERNAME_PATTERN = /^[a-z0-9_-]{1,64}$/i;
const notFound = () =>
new HttpError(404, 'Profile not found', { legacyCode: 'not_found' });
/**
* User profiles. Reads are open so an app can show another user's profile, but
* a profile other than the caller's own is served only while its owner is on a
* paid plan (`ProfileService.isPubliclyVisible`); a hidden one 404s like a
* missing account, so the response says nothing about the plan.
*/
@Controller('/profile')
export class ProfileController extends PuterController {
/**
* GET /profile?username=<name> — a user's profile. Without `username`, the
* signed-in user's own.
*/
@Get('/', {
subdomain: 'api',
rateLimit: {
scope: 'profile-read',
limit: 120,
window: 60_000,
key: 'ip',
},
})
async get(req: Request, res: Response): Promise<void> {
const service = this.services.profile as ProfileService;
const caller = req.actor?.user;
const username = req.query.username;
let target;
if (username === undefined) {
if (!caller?.uuid) {
throw new HttpError(401, 'Sign in, or pass a username', {
legacyCode: 'unauthorized',
});
}
target = await this.stores.user.getByUuid(caller.uuid);
} else {
if (
typeof username !== 'string' ||
!USERNAME_PATTERN.test(username)
) {
throw new HttpError(400, 'Invalid username', {
legacyCode: 'bad_request',
});
}
target = await this.stores.user.getByUsername(username);
}
if (!target) throw notFound();
const isOwner = caller?.uuid === target.uuid;
if (!isOwner && !(await service.isPubliclyVisible(target))) {
throw notFound();
}
res.json(await service.getProfile(target));
}
/** POST /profile — update the signed-in user's own profile. */
@Post('/', {
subdomain: 'api',
requireUserActor: true,
rateLimit: {
scope: 'profile-write',
limit: 30,
window: 60_000,
key: 'user',
},
})
async update(req: Request, res: Response): Promise<void> {
const service = this.services.profile as ProfileService;
const user = await this.stores.user.getByUuid(req.actor!.user.uuid!);
if (!user) throw notFound();
res.json(await service.updateProfile(user, req.body));
}
}
@@ -45,7 +45,7 @@ import {
import type { Actor } from '../../core/actor.js';
import type { ChatCompletionDriver } from '../../drivers/ai-chat/ChatCompletionDriver.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { inRequestScope, setupTestServer } from '../../testUtil.js';
import { PuterAIController } from './PuterAIController.js';
let server: PuterServer;
@@ -53,7 +53,9 @@ let controller: PuterAIController;
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.puterAi as unknown as PuterAIController;
controller = inRequestScope(
server.controllers.puterAi as unknown as PuterAIController,
);
});
afterAll(async () => {
@@ -48,7 +48,7 @@ import type { RouteOptions } from '../../core/http/index.js';
import type { ChatCompletionDriver } from '../../drivers/ai-chat/ChatCompletionDriver.js';
import { AI_CONCURRENT, AI_RATE_LIMIT } from '../../drivers/util/aiLimits.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { inRequestScope, setupTestServer } from '../../testUtil.js';
import { PuterAIController } from './PuterAIController.js';
// ── Test harness ────────────────────────────────────────────────────
@@ -58,7 +58,9 @@ let controller: PuterAIController;
beforeAll(async () => {
server = await setupTestServer();
controller = server.controllers.puterAi as unknown as PuterAIController;
controller = inRequestScope(
server.controllers.puterAi as unknown as PuterAIController,
);
});
afterAll(async () => {
@@ -130,6 +132,7 @@ const makeRes = () => {
captured.ended = true;
return res;
}),
once: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
@@ -20,6 +20,8 @@
import type { Request, Response } from 'express';
import crypto from 'node:crypto';
import { Readable } from 'node:stream';
import { Context } from '../../core/context.js';
import { abortOnDisconnect } from '../../core/http/abortOnDisconnect.js';
import { HttpError } from '../../core/http/HttpError.js';
import { RouteOptions } from '../../core/http/index.js';
import { computeNetworkFingerprint } from '../../core/http/middleware/rateLimit.js';
@@ -342,7 +344,7 @@ export class PuterAIController extends PuterController {
...openaiCompatProvider(body),
};
const result = await this.#driver().complete(completeArgs);
const result = await this.#complete(res, completeArgs);
const effectiveModel = completeArgs.model || '';
if (stream) {
@@ -500,7 +502,7 @@ export class PuterAIController extends PuterController {
const completionId = `cmpl-${randomId()}`;
const created = Math.floor(Date.now() / 1000);
const result = await this.#driver().complete(completeArgs);
const result = await this.#complete(res, completeArgs);
const effectiveModel = completeArgs.model || '';
if (stream) {
@@ -583,7 +585,8 @@ export class PuterAIController extends PuterController {
text: extractTextContent(
(
messageResult.message as
Record<string, unknown> | undefined
| Record<string, unknown>
| undefined
)?.content,
),
index: 0,
@@ -706,7 +709,7 @@ export class PuterAIController extends PuterController {
const responseId = generateId('resp');
const createdAt = Math.floor(Date.now() / 1000);
const result = await this.#driver().complete(completeArgs);
const result = await this.#complete(res, completeArgs);
const effectiveModel = completeArgs.model || '';
if (stream) {
@@ -996,7 +999,7 @@ export class PuterAIController extends PuterController {
};
const messageId = `msg_${randomId()}`;
const result = await this.#driver().complete(completeArgs);
const result = await this.#complete(res, completeArgs);
const effectiveModel = completeArgs.model || '';
if (stream) {
@@ -1194,6 +1197,15 @@ export class PuterAIController extends PuterController {
});
return driver;
}
/** A completion that stops, and stops billing, when the caller hangs up. */
#complete(
res: Response,
args: ICompleteArguments,
): Promise<IChatCompleteResult> {
Context.set('abortSignal', abortOnDisconnect(res));
return this.#driver().complete(args);
}
}
// -- Shared helpers --------------------------------------------------
@@ -26,6 +26,7 @@ import {
type PuterTestEnv,
} from '../../testUtil.js';
import { ShareController } from './ShareController.js';
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
/**
* Route-level coverage for the sharing endpoints. The service unit tests drive
@@ -1069,4 +1070,156 @@ describe('share endpoints over HTTP', () => {
expect(route.options?.allowFullAccessToken).toBeUndefined();
}
});
describe('an invited address over the wire', () => {
/** An app of the owner's, holding `mode` on one file, with a token. */
const appHolding = async (
owner: { username: string },
uid: string,
permission: (fileUid: string) => string,
) => {
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const actor = makeActor({ user: user! });
const app = await env.server.stores.app.create(
{
name: `peek-app-${crypto.randomUUID().slice(0, 8)}`,
title: 'Peek app',
index_url: `https://peek-${crypto.randomUUID()}.test/`,
},
{ ownerUserId: user!.id },
);
await runWithContext({ actor }, () =>
env.server.services.permission.grantUserAppPermission(
actor,
app.uid,
permission(uid),
),
);
const token = await env.server.services.auth.getUserAppToken(
actor,
app.uid,
);
return { ...app, token };
};
const invitedFile = async (owner: { username: string; token: string }) => {
const file = await makeFile(owner);
const email = `wire-${crypto.randomUUID().slice(0, 8)}@test.local`;
const res = await post('/share', owner.token, {
recipients: [email],
items: [{ uid: file.uid }],
mode: 'read',
});
expect(res.status).toBe(200);
return { file, email };
};
it('reaches the owner but not an app the file was handed to', async () => {
const owner = env.users.user;
const { file, email } = await invitedFile(owner);
// The owner asked, so the owner is told.
const mine = await get('/share/shares', owner.token, {
uid: file.uid,
});
expect(mine.status).toBe(200);
expect(await mine.text()).toContain(email);
// The app is answered, but bounded to the rows it issued — and it
// issued none, so the owner's invite is not among them.
const app = await appHolding(owner, file.uid, (u) => `fs:${u}:read`);
const peeked = await get('/share/shares', app.token, {
uid: file.uid,
});
expect(peeked.status).toBe(200);
const peekedBody = await peeked.text();
expect(peekedBody).not.toContain(email);
expect(JSON.parse(peekedBody).items).toEqual([]);
// `return_shares` is the same listing on an access-token surface.
const stat = await post('/fs/stat', app.token, {
uid: file.uid,
return_shares: true,
});
expect(stat.status).toBe(200);
const body = await stat.text();
expect(body).not.toContain(email);
expect(JSON.parse(body).shares).toEqual([]);
});
// The ticket's second route in: `ShareController` admits no access
// token, but `/fs/stat` does, and `return_shares` is the same listing.
it('withholds the listing from a list-scoped access token', async () => {
const owner = env.users.user;
const { file, email } = await invitedFile(owner);
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const actor = makeActor({ user: user! });
const scoped = await runWithContext({ actor }, () =>
env.server.services.auth.createAccessToken(actor, [
[`fs:${file.uid}:list`],
]),
);
const stat = await post('/fs/stat', scoped, {
uid: file.uid,
return_shares: true,
});
expect(stat.status).toBe(200);
const body = await stat.text();
expect(body).not.toContain(email);
expect(JSON.parse(body).shares).toEqual([]);
});
// Withholding here would buy nothing: `shared-by-me` hands the same
// addresses to the same credential.
it('tells a full-access token what it could read from shared-by-me anyway', async () => {
const owner = env.users.user;
const { file, email } = await invitedFile(owner);
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const actor = makeActor({ user: user! });
const pat = await runWithContext({ actor }, () =>
env.server.services.auth.createAccessToken(actor, [
[FULL_API_ACCESS],
]),
);
const outbound = await get('/share/shared-by-me', pat, {});
expect(outbound.status).toBe(200);
expect(await outbound.text()).toContain(email);
const listed = await get('/share/shares', pat, { uid: file.uid });
expect(listed.status).toBe(200);
expect(await listed.text()).toContain(email);
});
it('answers an app that was handed the item to manage', async () => {
const owner = env.users.user;
const { file, email } = await invitedFile(owner);
const app = await appHolding(
owner,
file.uid,
(u) => `manage:fs:${u}`,
);
const res = await get('/share/shares', app.token, {
uid: file.uid,
});
expect(res.status).toBe(200);
const text = await res.text();
// `manage` buys an app authority over the item, not sight of who
// else the owner invited to it.
expect(
JSON.parse(text).items.some(
(i: { pending?: boolean }) => i.pending,
),
).toBe(false);
expect(text).not.toContain(email);
});
});
});
+7 -1
View File
@@ -54,8 +54,14 @@ export async function toClientShare(
...(share.type === undefined ? {} : { type: share.type }),
...(thumbnail === undefined ? {} : { thumbnail }),
...(share.owner === undefined ? {} : { owner: share.owner.username }),
// Withheld from a listing whose caller it isn't for, hence the omission.
...(share.pending
? { pending: true, recipient_email: share.recipientEmail }
? {
pending: true,
...(share.recipientEmail === undefined
? {}
: { recipient_email: share.recipientEmail }),
}
: {}),
// A link share: no holder of any kind, this is what says so.
...(share.anyone ? { anyone: true } : {}),
@@ -18,6 +18,12 @@
*/
import { HttpError } from '../../core/http/HttpError.js';
import {
MAX_TOTAL_ATTACHMENT_BYTES,
attachmentMetadata,
attachmentSummary,
readContactSubmission,
} from '../../util/contactAttachments.js';
import { PuterController } from '../types.js';
/**
@@ -94,6 +100,38 @@ const LSMOD_LIMIT = {
key: 'user',
};
/** Longest message the Contact Us form will carry, in characters. */
const CONTACT_MESSAGE_MAX_LENGTH = 100_000;
/** UTF-8 worst case for {@link CONTACT_MESSAGE_MAX_LENGTH} UTF-16 code units. */
const CONTACT_MESSAGE_MAX_BYTES = CONTACT_MESSAGE_MAX_LENGTH * 3;
/** Largest valid multipart body: attachments, message, part-header slack. */
const CONTACT_BODY_MAX_BYTES =
MAX_TOTAL_ATTACHMENT_BYTES + CONTACT_MESSAGE_MAX_BYTES + 64 * 1024;
const isMultipart = (req) =>
/^multipart\/form-data\b/i.test(req.headers?.['content-type'] ?? '');
/**
* Per user, plus a per-IP backstop against one machine cycling fresh accounts.
* The IP limit sits well above what a shared office would send.
*/
const CONTACT_US_LIMITS = [
{
scope: 'contact-us',
limit: 10,
window: 15 * 60_000,
key: 'user',
},
{
scope: 'contact-us-ip',
limit: 40,
window: 24 * 60 * 60_000,
key: 'ip',
},
];
export class SystemController extends PuterController {
constructor(config, clients, stores, services, drivers) {
super(config, clients, stores, services, drivers);
@@ -180,33 +218,78 @@ export class SystemController extends PuterController {
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
rateLimit: {
scope: 'contact-us',
limit: 10,
window: 15 * 60_000,
key: 'user',
},
rateLimit: CONTACT_US_LIMITS,
// Each in-flight request holds its attachments in memory until
// the mail is sent.
concurrent: { limit: 2, scope: 'contact-us', key: 'user' },
},
async (req, res) => {
const { message } = req.body ?? {};
let message;
let attachments = [];
if (isMultipart(req)) {
const declaredLength = Number(
req.headers['content-length'],
);
if (
Number.isFinite(declaredLength) &&
declaredLength > CONTACT_BODY_MAX_BYTES
) {
// Don't read a body already known to be too large.
res.setHeader('Connection', 'close');
throw new HttpError(413, 'Request body is too large', {
legacyCode: 'bad_request',
});
}
const submission = await readContactSubmission(req, {
maxMessageBytes: CONTACT_MESSAGE_MAX_BYTES,
maxBodyBytes: CONTACT_BODY_MAX_BYTES,
});
if (!submission.ok) {
throw new HttpError(
submission.status,
submission.reason,
{ legacyCode: 'bad_request' },
);
}
({ message, attachments } = submission);
} else {
message = req.body?.message;
if (req.body?.attachments !== undefined) {
throw new HttpError(
400,
'`attachments` must be sent as multipart/form-data',
{ legacyCode: 'bad_request' },
);
}
}
if (!message || typeof message !== 'string') {
throw new HttpError(400, '`message` is required', {
legacyCode: 'bad_request',
});
}
if (message.length > 100_000) {
if (message.length > CONTACT_MESSAGE_MAX_LENGTH) {
throw new HttpError(
400,
'`message` is too long (max 100,000 characters)',
`\`message\` is too long (max ${CONTACT_MESSAGE_MAX_LENGTH.toLocaleString('en-US')} characters)`,
{ legacyCode: 'bad_request' },
);
}
// Persist to feedback table for durability
// Persist to feedback table for durability. Attachments are
// recorded by name and size only; the mail carries the bytes.
try {
await this.clients.db.write(
'INSERT INTO `feedback` (`user_id`, `message`) VALUES (?, ?)',
[req.actor.user.id, message],
'INSERT INTO `feedback` (`user_id`, `message`, `attachments`) VALUES (?, ?, ?)',
[
req.actor.user.id,
message,
attachments.length
? JSON.stringify(
attachmentMetadata(attachments),
)
: null,
],
);
} catch (e) {
console.warn('[contactUs] feedback insert failed:', e);
@@ -221,7 +304,16 @@ export class SystemController extends PuterController {
to: supportEmail,
replyTo: req.actor.user.email,
subject: `Contact from ${req.actor.user.username}`,
text: message,
text: attachments.length
? `${message}\n\n${attachmentSummary(attachments)}`
: message,
// Never rendered inline by the mail client.
attachments: attachments.map((a) => ({
filename: a.filename,
content: a.content,
contentType: a.contentType,
contentDisposition: 'attachment',
})),
});
} catch (e) {
console.warn('[contactUs] email send failed:', e);
@@ -18,6 +18,7 @@
*/
import type { Request, RequestHandler, Response } from 'express';
import { Readable } from 'node:stream';
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
@@ -82,11 +83,12 @@ const makeReq = (init: {
body?: unknown;
actor?: Actor;
query?: Record<string, unknown>;
headers?: Record<string, string>;
}): Request => {
return {
body: init.body ?? {},
query: init.query ?? {},
headers: {},
headers: init.headers ?? {},
actor: init.actor,
} as unknown as Request;
};
@@ -430,11 +432,253 @@ describe('SystemController POST /contactUs', () => {
// The row landed in the real `feedback` table for the right user.
const rows = (await server.clients.db.read(
'SELECT `user_id`, `message` FROM `feedback` WHERE `user_id` = ? AND `message` = ?',
'SELECT `user_id`, `message`, `attachments` FROM `feedback` WHERE `user_id` = ? AND `message` = ?',
[userId, message],
)) as Array<{ user_id: number; message: string }>;
)) as Array<{
user_id: number;
message: string;
attachments: string | null;
}>;
expect(rows).toHaveLength(1);
expect(rows[0]?.message).toBe(message);
// No files sent — the column stays null rather than an empty array.
expect(rows[0]?.attachments ?? null).toBeNull();
});
});
describe('SystemController POST /contactUs — attachments', () => {
// A real PNG signature with a filler body; the sniffer only reads the
// first eight bytes, and nothing downstream decodes the image.
const pngBytes = (size = 64): Buffer =>
Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
Buffer.alloc(size - 8, 0x61),
]);
type Part =
| { field: string; value: string }
| { field: string; filename: string; data: Buffer };
const BOUNDARY = '----contact-us-test';
const multipart = (parts: Part[]): Buffer =>
Buffer.concat([
...parts.flatMap((part) =>
'value' in part
? [
Buffer.from(
`--${BOUNDARY}\r\nContent-Disposition: form-data; name="${part.field}"\r\n\r\n${part.value}\r\n`,
),
]
: [
Buffer.from(
`--${BOUNDARY}\r\nContent-Disposition: form-data; name="${part.field}"; filename="${part.filename}"\r\nContent-Type: application/octet-stream\r\n\r\n`,
),
part.data,
Buffer.from('\r\n'),
],
),
Buffer.from(`--${BOUNDARY}--\r\n`),
]);
const submitMultipart = async (
parts: Part[],
headers: Record<string, string> = {},
) => {
const { actor, userId } = await makeUser();
const { res, captured } = makeRes();
const body = multipart(parts);
const req = Object.assign(Readable.from([body]), {
headers: {
'content-type': `multipart/form-data; boundary=${BOUNDARY}`,
'content-length': String(body.length),
...headers,
},
actor,
query: {},
}) as unknown as Request;
const call = callRoute('post', '/contactUs', req, res);
return { call, captured, userId };
};
const readAttachments = async (userId: number, message: string) => {
const rows = (await server.clients.db.read(
'SELECT `attachments` FROM `feedback` WHERE `user_id` = ? AND `message` = ?',
[userId, message],
)) as Array<{ attachments: string | null }>;
return rows[0]?.attachments ?? null;
};
it('stores attachment metadata — names, types and sizes, no payloads', async () => {
const message = `attached ${Math.random().toString(36).slice(2)}`;
const { call, captured, userId } = await submitMultipart([
{ field: 'message', value: message },
{
field: 'attachments',
filename: 'repro-step-3.png',
data: pngBytes(128),
},
]);
await call;
expect(captured.body).toEqual({});
const stored = await readAttachments(userId, message);
expect(JSON.parse(stored!)).toEqual([
{ name: 'repro-step-3.png', type: 'image/png', size: 128 },
]);
});
it('emails the files to support with sanitized names and a manifest', async () => {
const sendRaw = vi
.spyOn(server.clients.email, 'sendRaw')
.mockResolvedValue(null);
try {
const message = `attached ${Math.random().toString(36).slice(2)}`;
const { call } = await submitMultipart([
{ field: 'message', value: message },
// Declares .html, but the bytes are a PNG: the stored and
// emailed extension must follow the bytes.
{
field: 'attachments',
filename: 'payload.html',
data: pngBytes(64),
},
]);
await call;
expect(sendRaw).toHaveBeenCalledTimes(1);
const sent = sendRaw.mock.calls[0][0] as {
text: string;
attachments: Array<{
filename: string;
content: Buffer;
contentType: string;
contentDisposition: string;
}>;
};
expect(sent.attachments).toHaveLength(1);
expect(sent.attachments[0].filename).toBe('payload.png');
expect(sent.attachments[0].contentType).toBe('image/png');
expect(sent.attachments[0].contentDisposition).toBe('attachment');
expect(sent.attachments[0].content.equals(pngBytes(64))).toBe(true);
// The body records what should have arrived, in case a gateway
// strips the files on the way.
expect(sent.text).toContain(message);
expect(sent.text).toContain('payload.png');
} finally {
sendRaw.mockRestore();
}
});
it('accepts a multipart submission with no files', async () => {
const sendRaw = vi
.spyOn(server.clients.email, 'sendRaw')
.mockResolvedValue(null);
try {
const message = `plain ${Math.random().toString(36).slice(2)}`;
const { call, captured } = await submitMultipart([
{ field: 'message', value: message },
]);
await call;
expect(captured.body).toEqual({});
const sent = sendRaw.mock.calls[0][0] as {
text: string;
attachments: unknown[];
};
expect(sent.attachments).toEqual([]);
expect(sent.text).toBe(message);
} finally {
sendRaw.mockRestore();
}
});
it('rejects base64 attachments in a JSON body', async () => {
const { actor } = await makeUser();
const { res } = makeRes();
await expect(
callRoute(
'post',
'/contactUs',
makeReq({
body: {
message: 'hi',
attachments: [{ data: pngBytes().toString('base64') }],
},
actor,
}),
res,
),
).rejects.toMatchObject({ statusCode: 400 });
});
it.each([
[
'too many files',
[
{ field: 'message', value: 'hi' },
...Array.from({ length: 6 }, (_, i) => ({
field: 'attachments',
filename: `${i}.png`,
data: pngBytes(),
})),
],
],
[
'an unsupported type',
[
{ field: 'message', value: 'hi' },
{
field: 'attachments',
filename: 'a.pdf',
data: Buffer.from('%PDF-1.7 x'),
},
],
],
[
'a script-capable SVG',
[
{ field: 'message', value: 'hi' },
{
field: 'attachments',
filename: 'a.svg',
data: Buffer.from('<svg><script/></svg>'),
},
],
],
[
'a missing message',
[{ field: 'attachments', filename: 'a.png', data: pngBytes() }],
],
] as [string, Part[]][])('rejects %s with 400', async (_label, parts) => {
const { call, captured } = await submitMultipart(parts);
await expect(call).rejects.toMatchObject({ statusCode: 400 });
expect(captured.body).toBeUndefined();
});
it('refuses a body whose declared length is over budget before reading it', async () => {
const { call, captured } = await submitMultipart(
[{ field: 'message', value: 'hi' }],
{ 'content-length': String(64 * 1024 * 1024) },
);
await expect(call).rejects.toMatchObject({ statusCode: 413 });
expect(captured.headers.Connection).toBe('close');
});
it('rejects an upload that breaks a limit mid-stream', async () => {
const { call, captured } = await submitMultipart(
[
{ field: 'message', value: 'hi' },
{
field: 'attachments',
filename: 'big.png',
data: pngBytes(10 * 1024 * 1024 + 1),
},
],
// Chunked uploads carry no length to check up front.
{ 'content-length': '' },
);
await expect(call).rejects.toMatchObject({ statusCode: 413 });
// The rest of the body is drained, so the connection can stay open.
expect(captured.headers.Connection).toBeUndefined();
});
});
@@ -0,0 +1,136 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import http from 'node:http';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
/**
* Contact Us attachments over real HTTP. Multipart bodies must skip the global
* JSON parser so the route's gates run before any of the body is read; only a
* listening server shows that.
*/
describe('POST /contactUs over HTTP', () => {
let env: PuterTestEnv;
beforeAll(async () => {
env = await setupPuterTestEnv();
}, 120_000);
afterAll(async () => {
await env?.shutdown();
});
const png = (size: number): Buffer =>
Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
Buffer.alloc(size - 8, 0x61),
]);
const BOUNDARY = 'contact-us-http-test';
const multipartHead = Buffer.from(
`--${BOUNDARY}\r\nContent-Disposition: form-data; name="message"\r\n\r\nhi\r\n` +
`--${BOUNDARY}\r\nContent-Disposition: form-data; name="attachments"; filename="a.png"\r\n` +
'Content-Type: image/png\r\n\r\n',
);
/** Send raw bytes and resolve with the response, however early it comes. */
const rawPost = (
headers: Record<string, string>,
write: (req: http.ClientRequest) => void,
) =>
new Promise<{ status: number; connection?: string }>(
(resolve, reject) => {
const url = new URL('/contactUs', env.apiOrigin);
const req = http.request(
url,
{
method: 'POST',
headers: {
Authorization: `Bearer ${env.users.user.token}`,
'Content-Type': `multipart/form-data; boundary=${BOUNDARY}`,
...headers,
},
},
(res) => {
res.resume();
resolve({
status: res.statusCode ?? 0,
connection: res.headers.connection,
});
},
);
req.on('error', reject);
write(req);
},
);
it('accepts the FormData the Contact Us window sends', async () => {
const message = `http ${Math.random().toString(36).slice(2)}`;
const body = new FormData();
body.append('message', message);
body.append(
'attachments',
new Blob([png(4096)], { type: 'image/png' }),
'shot.png',
);
const res = await fetch(new URL('/contactUs', env.apiOrigin), {
method: 'POST',
headers: { Authorization: `Bearer ${env.users.user.token}` },
body,
});
expect(res.status).toBe(200);
const rows = (await env.server.clients.db.read(
'SELECT `attachments` FROM `feedback` WHERE `message` = ?',
[message],
)) as Array<{ attachments: string | null }>;
expect(JSON.parse(rows[0]!.attachments!)).toEqual([
{ name: 'shot.png', type: 'image/png', size: 4096 },
]);
});
it('refuses an unauthenticated upload', async () => {
const body = new FormData();
body.append('message', 'hi');
const res = await fetch(new URL('/contactUs', env.apiOrigin), {
method: 'POST',
body,
});
expect(res.status).toBe(401);
});
it('answers an oversized Content-Length before the body is sent', async () => {
const res = await rawPost(
{ 'Content-Length': String(64 * 1024 * 1024) },
(req) => req.flushHeaders(),
);
expect(res).toEqual({ status: 413, connection: 'close' });
});
it('answers a chunked upload that breaks the per-file cap with 413', async () => {
const file = png(12 * 1024 * 1024);
const res = await rawPost({ 'Transfer-Encoding': 'chunked' }, (req) => {
req.write(multipartHead);
req.end(file);
});
expect(res.status).toBe(413);
});
});
@@ -18,6 +18,7 @@
*/
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { makeActor } from '../../core/actor.js';
import type { IConfig } from '../../types';
import { setupPuterTestEnv, type PuterTestEnv } from '../../testUtil.js';
@@ -53,6 +54,47 @@ describe('team endpoints over HTTP', () => {
...(body === undefined ? {} : { body: JSON.stringify(body) }),
});
/** An app token acting for `owner`, minted the way the dashboard does. */
const makeAppToken = async (owner: { username: string }) => {
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const actor = makeActor({ user: user! });
const app = await env.server.stores.app.create(
{
name: `team-http-app-${crypto.randomUUID()}`,
title: 'Team app',
index_url: `https://team-${crypto.randomUUID()}.test/`,
},
{ ownerUserId: actor.user.id! },
);
return env.server.services.auth.getUserAppToken(actor, app.uid);
};
/** A scoped (non-full-access) access token for `owner`, carrying no app. */
const makeScopedToken = async (owner: {
username: string;
token: string;
}) => {
const user = await env.server.stores.user.getByUsername(
owner.username,
);
const anchor = `/${owner.username}/${crypto.randomUUID()}`;
await env.server.services.fs.mkdir(user!.id, {
path: anchor,
createMissingParents: true,
});
const entry = await env.server.stores.fsEntry.getEntryByPath(anchor);
const { actor } = await env.server.services.auth.authenticate(
owner.token,
);
return env.server.services.auth.createAccessToken(
actor!,
[[`fs:${entry!.uid}:list`]],
{ label: 'teams-scope' },
);
};
/** A team owned by `user`, with one provisioned member. */
const makeTeam = async () => {
const res = await call('POST', '/teams', env.users.user.token, {
@@ -199,6 +241,15 @@ describe('team endpoints over HTTP', () => {
expect(after.status).toBe(200);
const body = (await after.json()) as { items: { username: string }[] };
expect(body.items.some((m) => m.username === env.users.user.username)).toBe(true);
// A scoped token was minted for one thing; a roster is not it.
const scoped = await makeScopedToken(env.users.user);
const narrow = await call(
'GET',
`/teams/${team.uid}/directory`,
typeof scoped === 'string' ? scoped : scoped.token,
);
expect(narrow.status).toBe(403);
});
it('refuses the directory to someone outside the team', async () => {
@@ -215,6 +266,188 @@ describe('team endpoints over HTTP', () => {
expect(res.status).toBe(404);
});
// -- app actors, gated by the same opt-in as the directory --------
//
// Built through the service, not the wire: these routes share a
// rate-limit budget with the mutation tests above, and every HTTP
// create/update here would eat into that same window.
/** A team owned by `user`, created and toggled through the service. */
const makeServiceTeam = async () => {
const owner = (await env.server.stores.user.getByUsername(
env.users.user.username,
))!;
const team = await env.server.services.team.createTeam(owner.id, {
name: 'Acme',
handle: randomHandle(),
});
return { team, ownerId: owner.id };
};
it('omits a team from an app token until the directory opts in', async () => {
const { team, ownerId } = await makeServiceTeam();
const appToken = await makeAppToken(env.users.user);
const before = await call('GET', '/teams', appToken);
expect(before.status).toBe(200);
const beforeBody = (await before.json()) as { items: { uid: string }[] };
expect(beforeBody.items.some((t) => t.uid === team.uid)).toBe(false);
await env.server.services.team.updateTeam(team.uid, ownerId, {
directoryEnabled: true,
});
const after = await call('GET', '/teams', appToken);
expect(after.status).toBe(200);
const afterBody = (await after.json()) as { items: { uid: string }[] };
expect(afterBody.items.some((t) => t.uid === team.uid)).toBe(true);
});
it('still lists a team to its owner session with the directory off', async () => {
const { team } = await makeServiceTeam();
const res = await call('GET', '/teams', env.users.user.token);
expect(res.status).toBe(200);
const body = (await res.json()) as { items: { uid: string }[] };
expect(body.items.some((t) => t.uid === team.uid)).toBe(true);
});
it('lists a team to a full-access API token with the directory off', async () => {
const { team } = await makeServiceTeam();
const res = await call('GET', '/teams', env.users.user.apiToken);
expect(res.status).toBe(200);
const body = (await res.json()) as { items: { uid: string }[] };
expect(body.items.some((t) => t.uid === team.uid)).toBe(true);
});
it('404s /members to an app token until the directory opts in, then gives usernames only', async () => {
const { team, ownerId } = await makeServiceTeam();
const username = `svc_${Math.random().toString(36).slice(2, 9)}`;
await env.server.services.team.provisionAccount(team.uid, ownerId, {
username,
});
const appToken = await makeAppToken(env.users.user);
const before = await call(
'GET',
`/teams/${team.uid}/members`,
appToken,
);
expect(before.status).toBe(404);
expect(await before.json()).toMatchObject({ code: 'team_not_found' });
await env.server.services.team.updateTeam(team.uid, ownerId, {
directoryEnabled: true,
});
const after = await call(
'GET',
`/teams/${team.uid}/members`,
appToken,
);
expect(after.status).toBe(200);
const body = (await after.json()) as {
items: Record<string, unknown>[];
};
expect(body.items.length).toBeGreaterThan(0);
// An app gets username only — no org_owned, created_at, or uuid.
for (const m of body.items) expect(Object.keys(m)).toEqual(['username']);
});
it('still refuses an app token on GET /teams/:uid', async () => {
const { team } = await makeServiceTeam();
const appToken = await makeAppToken(env.users.user);
const res = await call('GET', `/teams/${team.uid}`, appToken);
expect(res.status).toBe(403);
expect(await res.json()).toMatchObject({ code: 'forbidden' });
});
it('excludes a never-activated or suspended seat from an app`s view of /members', async () => {
const { team, ownerId } = await makeServiceTeam();
const neverActivated = `svc_na_${Math.random().toString(36).slice(2, 9)}`;
await env.server.services.team.provisionAccount(team.uid, ownerId, {
username: neverActivated,
});
const suspendedUsername = `svc_su_${Math.random().toString(36).slice(2, 9)}`;
const suspended = await env.server.services.team.provisionAccount(
team.uid,
ownerId,
{ username: suspendedUsername },
);
await env.server.services.team.disableMember(
team.uid,
ownerId,
suspended.userId,
);
await env.server.services.team.updateTeam(team.uid, ownerId, {
directoryEnabled: true,
});
const appToken = await makeAppToken(env.users.user);
const appView = await call(
'GET',
`/teams/${team.uid}/members`,
appToken,
);
expect(appView.status).toBe(200);
const appItems = (
(await appView.json()) as { items: Record<string, unknown>[] }
).items;
for (const m of appItems) expect(Object.keys(m)).toEqual(['username']);
const appUsernames = appItems.map((m) => m.username);
expect(appUsernames).not.toContain(neverActivated);
expect(appUsernames).not.toContain(suspendedUsername);
// The owner's own seat is active, so it still shows.
expect(appUsernames).toContain(env.users.user.username);
// The owner's own session sees every seat regardless of status.
const ownerView = await call(
'GET',
`/teams/${team.uid}/members`,
env.users.user.token,
);
expect(ownerView.status).toBe(200);
const ownerUsernames = (
(await ownerView.json()) as { items: { username: string }[] }
).items.map((m) => m.username);
expect(ownerUsernames).toContain(neverActivated);
expect(ownerUsernames).toContain(suspendedUsername);
});
it('gives a full-access API token the owner`s uuid on /members', async () => {
const { team } = await makeServiceTeam();
const res = await call(
'GET',
`/teams/${team.uid}/members`,
env.users.user.apiToken,
);
expect(res.status).toBe(200);
const body = (await res.json()) as {
items: { username: string; uuid?: string }[];
};
const owner = body.items.find(
(m) => m.username === env.users.user.username,
);
expect(owner?.uuid).toEqual(expect.any(String));
});
it('refuses a scoped access token on GET /teams and /members', async () => {
const { team } = await makeServiceTeam();
const scoped = await makeScopedToken(env.users.user);
const teamsRes = await call('GET', '/teams', scoped);
expect(teamsRes.status).toBe(403);
expect(await teamsRes.json()).toMatchObject({ code: 'forbidden' });
const membersRes = await call(
'GET',
`/teams/${team.uid}/members`,
scoped,
);
expect(membersRes.status).toBe(403);
expect(await membersRes.json()).toMatchObject({ code: 'forbidden' });
});
it('lists members with org_owned distinguishing the owner', async () => {
const { team, memberUsername } = await makeTeam();
@@ -499,12 +732,17 @@ describe('team endpoints over HTTP', () => {
return { username, password, userId: row.id, token };
};
/** Cookie-credentialed on the GUI origin, as the user-protected gate insists. */
/**
* Cookie-credentialed on the GUI origin, as the user-protected gate insists.
* Sent with the seat's own user agent: the session touch otherwise rewrites
* it to Node's once a second boundary passes.
*/
const changePassword = (token: string, password: string, next: string) =>
fetch(new URL('/user-protected/change-password', env.origin), {
method: 'POST',
headers: {
'content-type': 'application/json',
'user-agent': 'puter-test-seat',
authorization: `Bearer ${token}`,
cookie: `puter_auth_token=${token}`,
},
+66 -16
View File
@@ -25,6 +25,7 @@ import {
Post,
Put,
} from '../../core/http/decorators.js';
import { isAccountContext } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { TeamRow } from '../../stores/team/TeamStore.js';
import { PuterController } from '../types.js';
@@ -66,11 +67,14 @@ const toClientTeam = (team: TeamRow, isOwner: boolean) => ({
is_owner: isOwner,
created_at: team.created_at,
directory_enabled: Number(team.directory_enabled) === 1,
require_2fa: Number(team.require_2fa) === 1,
});
@Controller('/teams')
export class TeamController extends PuterController {
// `requireUserActor` installs the auth gates; reads need it too.
// Most routes gate with `requireUserActor`; `listTeams` and `listMembers`
// use `requireAuth` so an app can read a team whose owner opted into the
// directory (see `directory_enabled`).
/** Off means `/teams` 404s and no team route is registered at all. */
isEnabled(): boolean {
@@ -100,7 +104,7 @@ export class TeamController extends PuterController {
@Get('', {
subdomain: 'api',
requireUserActor: true,
requireAuth: true,
requireVerified: true,
rateLimit: TEAM_READ_LIMIT,
})
@@ -108,8 +112,13 @@ export class TeamController extends PuterController {
const userId = this.#requireUserId(req);
await this.#requireTeamsAvailable(req, userId);
const teams = await this.stores.team.listTeamsForUser(userId);
// An app only sees a team whose owner opened the directory to apps;
// a team that hasn't is indistinguishable from no team at all.
const visible = isAccountContext(req.actor)
? teams
: teams.filter((t) => this.services.team.isDirectoryOpen(t));
res.json({
items: teams.map((t) =>
items: visible.map((t) =>
toClientTeam(t, t.owner_user_id === userId),
),
});
@@ -145,6 +154,7 @@ export class TeamController extends PuterController {
name?: string;
handle?: string | null;
directoryEnabled?: boolean;
require2fa?: boolean;
} = {};
if (body.name !== undefined)
changes.name = this.#requireString(body.name, 'name');
@@ -152,6 +162,8 @@ export class TeamController extends PuterController {
changes.handle = body.handle === null ? null : String(body.handle);
if (body.directory_enabled !== undefined)
changes.directoryEnabled = body.directory_enabled === true;
if (body.require_2fa !== undefined)
changes.require2fa = body.require_2fa === true;
// Through the service: turning the directory on writes an audit row.
const team = await this.services.team.updateTeam(
@@ -178,7 +190,7 @@ export class TeamController extends PuterController {
@Get('/:uid/members', {
subdomain: 'api',
requireUserActor: true,
requireAuth: true,
requireVerified: true,
rateLimit: TEAM_READ_LIMIT,
})
@@ -188,9 +200,13 @@ export class TeamController extends PuterController {
this.#param(req, 'uid'),
userId,
);
// Only the owner: the uuid is what billing keys a seat's plan on, and
// one member has no business identifying another.
const isOwner = team.owner_user_id === userId;
const accountContext = isAccountContext(req.actor);
// Same consent as the directory: an app only reads a team that opted in.
if (!accountContext) this.services.team.assertDirectoryOpen(team);
// uuid only when the team owner calls with their own session or API
// token; an app needing a stable id uses `listDirectory()` instead.
const showUuid = accountContext && team.owner_user_id === userId;
const page = await this.stores.team.listMembers(
this.#param(req, 'uid'),
@@ -200,23 +216,30 @@ export class TeamController extends PuterController {
typeof req.query.cursor === 'string'
? req.query.cursor
: undefined,
// An app's view matches the directory's: active seats only.
activeOnly: !accountContext,
},
);
// An app only ever gets a username; org_owned/created_at are account-only.
res.json({
items: page.items.map((m) => ({
username: m.username,
org_owned: Number(m.org_owned) === 1,
created_at: m.created_at,
...(isOwner ? { uuid: m.uuid } : {}),
})),
items: page.items.map((m) =>
accountContext
? {
username: m.username,
org_owned: Number(m.org_owned) === 1,
created_at: m.created_at,
...(showUuid ? { uuid: m.uuid } : {}),
}
: { username: m.username },
),
...(page.cursor ? { cursor: page.cursor } : {}),
});
}
/**
* The only team route that admits an app actor. It discloses nothing a
* colleague cannot already read through `/members`, and the team has to
* have opted in, so an app cannot enumerate a team by default.
* Admits an app actor, like `listTeams` and `listMembers`, once the team
* has opted in. Discloses nothing a colleague cannot already read through
* `/members`.
*/
@Get('/:uid/directory', {
subdomain: 'api',
@@ -226,6 +249,15 @@ export class TeamController extends PuterController {
async listDirectory(req: Request, res: Response): Promise<void> {
// The membership is the person's, never the app's.
const userId = this.#requireUserId(req);
// A scoped token holds only what it was minted for; a roster is not that.
const token = req.actor?.accessToken;
if (token && token.fullAccess !== true) {
throw new HttpError(
403,
'This endpoint is not available to scoped access tokens',
{ legacyCode: 'forbidden' },
);
}
const page = await this.services.team.listDirectory(
this.#param(req, 'uid'),
userId,
@@ -304,6 +336,24 @@ export class TeamController extends PuterController {
res.json({ temporary_password: temporaryPassword });
}
// Same budget as a password reset: both are a step toward the account.
@Post('/:uid/members/:username/2fa-reset', {
subdomain: 'api',
requireUserActor: true,
requireVerified: true,
rateLimit: TEAM_RESET_LIMIT,
})
async resetMemberTwoFactor(req: Request, res: Response): Promise<void> {
const userId = this.#requireUserId(req);
const uid = this.#param(req, 'uid');
// Authority first, or resolving `:username` is an existence oracle.
await this.services.team.requireOwner(uid, userId);
const target = await this.#requireTargetUserId(req);
await this.services.team.resetMemberTwoFactor(uid, userId, target);
res.json({ success: true });
}
@Post('/:uid/members/:username/disable', {
subdomain: 'api',
requireUserActor: true,
+2 -1
View File
@@ -73,7 +73,8 @@ export const PuterController =
public onServerShutdown() {
return;
}
public getReportedCosts(): // eslint-disable-next-line @typescript-eslint/no-explicit-any
public getReportedCosts():
// eslint-disable-next-line @typescript-eslint/no-explicit-any
| Promise<Record<string, any>[]>
// eslint-disable-next-line @typescript-eslint/no-explicit-any
| Record<string, any>[] {
@@ -2,6 +2,7 @@
// but rather it performs some common sense checks to ensure that WebDAV support isn't irrevocably broken in puter
import type { Request, RequestHandler, Response } from 'express';
import { Readable, Writable } from 'node:stream';
import { GetObjectCommand } from '@aws-sdk/client-s3';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { hash as bcryptHash } from 'bcrypt';
@@ -9,7 +10,9 @@ import { PuterRouter } from '../../core/http/PuterRouter.js';
import type { RouteDescriptor } from '../../core/http/types.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { makeActor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { FULL_API_ACCESS } from '../../services/permission/consts.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { WebDAVController } from './WebDAVController.js';
@@ -1197,7 +1200,7 @@ describe('WebDAVController verbs', () => {
* A response double that is also a Writable, so handlers that end with
* `body.pipe(res)` (GET) work without a socket.
*/
const makeStreamRes = () => {
const makeStreamRes = ({ abortOnFirstChunk = false } = {}) => {
const chunks: Buffer[] = [];
const captured = {
statusCode: 200,
@@ -1209,6 +1212,9 @@ describe('WebDAVController verbs', () => {
const sink = new Writable({
write(chunk, _encoding, callback) {
chunks.push(Buffer.from(chunk as Buffer));
// A client that hangs up mid-download surfaces as the
// response being destroyed, not as a write error.
if (abortOnFirstChunk) sink.destroy();
callback();
},
});
@@ -1334,6 +1340,87 @@ describe('WebDAVController verbs', () => {
expect(captured.text()).toBe('2345');
});
/**
* Swap the object store's answer for one GET so the body stream is
* under the test's control. Intercepts at the S3 SDK client, the real
* boundary; every other command still hits the in-memory S3.
*/
const serveBody = (body: Readable) => {
const client = server.clients.s3.get(
server.stores.s3Object.resolveRegion(null),
);
const original = client.send.bind(client);
return vi.spyOn(client, 'send').mockImplementation(((
command: unknown,
) =>
command instanceof GetObjectCommand
? Promise.resolve({
Body: body,
ContentLength: 7,
ContentType: 'text/plain',
})
: original(command as never)) as never);
};
it('tears down the upstream stream when the client disconnects mid-download', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/abort-me.txt`;
await putFile(actor, path, 'streamed');
// Never ends on its own: only a teardown can finish it.
const body = new Readable({ read() {} });
body.push('partial');
const send = serveBody(body);
const { res } = makeStreamRes({ abortOnFirstChunk: true });
try {
await dispatchMiddleware(
makeReq({ method: 'GET', path, actor }),
res,
noop,
);
} finally {
send.mockRestore();
}
expect((res as unknown as Writable).destroyed).toBe(true);
expect(body.destroyed).toBe(true);
});
it('survives the upstream stream failing mid-download', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/reset-me.txt`;
await putFile(actor, path, 'streamed');
// First read hands over a chunk, the next one fails the way a
// dropped storage connection does. Without an error listener on
// the body that event would take the process down.
let reads = 0;
const body = new Readable({
read() {
reads += 1;
if (reads === 1) this.push('partial');
else this.destroy(new Error('upstream reset'));
},
});
const send = serveBody(body);
const { res, captured } = makeStreamRes();
try {
await expect(
dispatchMiddleware(
makeReq({ method: 'GET', path, actor }),
res,
noop,
),
).resolves.toBeUndefined();
} finally {
send.mockRestore();
}
expect(captured.statusCode).toBe(200);
expect((res as unknown as Writable).destroyed).toBe(true);
expect(body.destroyed).toBe(true);
});
it('refuses to GET a directory', async () => {
const { actor, username } = await makeUser();
const captured = await dispatch({
@@ -1461,6 +1548,98 @@ describe('WebDAVController verbs', () => {
);
});
describe('root collection per credential', () => {
// Own address, so the per-IP auth failure bucket the rest of the
// file draws on can't refuse these.
const propfindRoot = (authorization: string) =>
dispatch({
method: 'PROPFIND',
path: '/',
headers: { authorization },
ip: '203.0.113.53',
});
const mintToken = async (userId: number, permission: string) => {
const user = (await server.stores.user.getById(userId))!;
return server.services.auth.createAccessToken(
makeActor({ user }),
[[permission]],
);
};
it('lists the home for a password login and a full-access token', async () => {
const username = `webdav-root-${Math.random()
.toString(36)
.slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: await bcryptHash('correct-horse', 4),
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const fullAccess = await mintToken(created.id, FULL_API_ACCESS);
for (const authorization of [
basicAuth(username, 'correct-horse'),
basicAuth('-token', fullAccess),
]) {
const captured = await propfindRoot(authorization);
expect(captured.statusCode).toBe(207);
expect(captured.body as string).toContain(
`<D:href>/${username}/</D:href>`,
);
}
});
it('leaves the home out for a token scoped below it', async () => {
const { userId, username } = await makeUser();
const home = (await server.stores.fsEntry.getEntryByPath(
`/${username}`,
))!;
const documents = (await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
))!;
const scoped = await mintToken(
userId,
`fs:${documents.uuid}:read`,
);
const captured = await propfindRoot(
basicAuth('-token', scoped),
);
expect(captured.statusCode).toBe(207);
const xml = captured.body as string;
expect(xml).toContain('<D:href>/</D:href>');
expect(xml).not.toContain(`/${username}/`);
expect(xml).not.toContain(home.uuid);
});
it('lists the home for a token that can read it', async () => {
const { userId, username } = await makeUser();
const home = (await server.stores.fsEntry.getEntryByPath(
`/${username}`,
))!;
const scoped = await mintToken(userId, `fs:${home.uuid}:read`);
const captured = await propfindRoot(
basicAuth('-token', scoped),
);
expect(captured.statusCode).toBe(207);
expect(captured.body as string).toContain(
`<D:href>/${username}/</D:href>`,
);
});
});
it('describes a single file when asked for one', async () => {
const { actor, username } = await makeUser();
const path = `/${username}/Documents/single.md`;
@@ -20,11 +20,13 @@
import { compare as bcryptCompare } from 'bcrypt';
import type { Request, Response } from 'express';
import { posix as pathPosix } from 'node:path';
import { pipeline } from 'node:stream/promises';
import { EventMap } from '../../clients/event/types.js';
import { makeActor, type Actor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import {
assertNotSuspended,
assertTeam2fa,
assertVerifiedAccount,
} from '../../core/http/middleware/gates.js';
import {
@@ -46,6 +48,7 @@ import {
import type { RouteOptions } from '../../core/http/types.js';
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import { rootShowsHome } from '../../services/fs/rootListing.js';
import {
maskEntryPath,
parseMaskedSharePath,
@@ -283,6 +286,7 @@ export class WebDAVController extends PuterController {
// filesystem over the `dav` subdomain, bypassing the gates.
assertNotSuspended(actor.user);
assertVerifiedAccount(actor.user);
await assertTeam2fa(actor.user, this.stores.team);
// The actor was absent when the auth probe ran, so anything that reads
// it off the request or the context — shared-path masking, egress
@@ -551,7 +555,12 @@ export class WebDAVController extends PuterController {
'Content-Length': String(result.contentLength ?? 0),
});
}
result.body.pipe(res);
try {
await pipeline(result.body, res);
} catch {
// Client disconnect or upstream stream error: pipeline already
// tore down both ends, so nothing is left dangling to crash on.
}
}
// -- PROPFIND ----------------------------------------------------
@@ -616,7 +625,8 @@ export class WebDAVController extends PuterController {
/**
* The root collection: the caller's own home directory, plus one collection
* per person who has shared something with them.
* per person who has shared something with them. A scoped access token sees
* the home only when it can read it.
*
* A share is addressed `/<owner>/<uuid>/<name>`, where the uuid stands in
* for the folder the owner keeps it in. Only the `<owner>` segment is a
@@ -631,7 +641,16 @@ export class WebDAVController extends PuterController {
const home = await this.stores.fsEntry.getEntryByPath(
`/${actor.user!.username}`,
);
if (home) {
if (
home &&
(await rootShowsHome(
actor,
home,
'read',
this.services.acl,
this.stores.permission,
))
) {
responses.push(
propfindEntry(maskEntryPath(home), home, home.isDir),
);
@@ -1166,7 +1185,9 @@ export class WebDAVController extends PuterController {
void Promise.resolve()
.then(async () => {
const response = {
...(await toLegacyEntry(this.clients.event, entry)),
...(await toLegacyEntry(this.clients.event, entry, {
forOwner: true,
})),
...extra,
from_new_service: true,
};
+9
View File
@@ -75,6 +75,15 @@ describe('makeActor / effectiveApp', () => {
expect(userRelatedActor(app).effectiveApp).toBeNull();
});
it('keeps the handler depth when narrowing to the underlying user', () => {
const app = makeActor({ user, app: { uid: 'app-1' } });
app.handlerDepth = 3;
expect(userRelatedActor(app).handlerDepth).toBe(3);
expect(
userRelatedActor(makeActor({ user, app: { uid: 'app-1' } })),
).not.toHaveProperty('handlerDepth');
});
it('resolves the system actor', () => {
expect(SYSTEM_ACTOR.effectiveApp).toBeNull();
});
Loaded 100 of 715 files, more files were not shown because too many files have changed in this diff. Show more