Compare commits

..
Author SHA1 Message Date
Sourabh Sharma 15134c99bf Fix mkdir dedupeName for existing directories (#3215) 2026-06-07 12:49:00 -07:00
Daniel Salazar 8262e3193e perf: don't batch get app stats (#3219) 2026-06-06 18:28:59 -07:00
Reynaldi Chernando 98a21c4ac0 More workers docs improvement (#3218)
* More workers docs improvement

* Add mention of workers alongside mention of user scope
2026-06-06 01:27:24 +07:00
ProgrammerIn-wonderland c52eb1b903 Ns/bug fixes 060426 (#3214)
* Harden backend auth

* Puter.js auth hardening
2026-06-05 13:55:23 -04:00
mohit_parmar_007 d6a71e50bb fix: remove unnecessary cache console.log statements (#3216) 2026-06-05 10:01:59 -07:00
Reynaldi Chernando a4b6ee0dd2 [PUT-1090] Improve worker docs (#3217)
* Improve worker docs

* refine
2026-06-05 21:54:56 +07:00
Daniel Salazar bb61103080 feat: add apps api into mcp (#3213) 2026-06-04 17:19:53 -07:00
Daniel Salazar 9bbc76b2dd drop metering alerts to low severity (#3212) 2026-06-04 14:01:44 -07:00
Daniel Salazar 918aba9c80 fix: reduce alarms further (#3211)
* feat; update bug-bounty

* fix: reduce alarms further
2026-06-04 10:43:50 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 3ceba9e3b3 Minor fix deployment docs (#3210)
* Minor fix deployment docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-04 23:48:25 +07:00
Daniel Salazar a480679a2d vis: make metering service alarm only go off on actual abuse (#3208) 2026-06-03 22:45:18 -07:00
Daniel Salazar e99ae40e93 chore: bump puter-js (#3207) 2026-06-03 18:59:07 -07:00
Daniel Salazar 3cd50fbaf3 move the token migration to root (#3206) 2026-06-03 18:54:22 -07:00
ProgrammerIn-wonderland 60dd1453d4 Fix issues (#3205) 2026-06-03 21:19:33 -04:00
ProgrammerIn-wonderland 07781cd681 allow overriding origin from workers.dev on mcp server (#3204) 2026-06-03 17:48:37 -04:00
ProgrammerIn-wonderland 2387d8dcc9 MCP whoami (#3203) 2026-06-03 17:13:13 -04:00
Reynaldi Chernando 723608f4db Add new puter cli (#3199)
* Add new cli

* update package lock

* fix readme and license
2026-06-04 01:51:31 +07:00
Reynaldi Chernando bd1dcde7b0 Add info for puterjs deployments (#3202)
* Add info for puterjs deployments

* update external links

* hide github actions method
2026-06-04 01:25:57 +07:00
ProgrammerIn-wonderland 5b8e41e66d MCP Server (#3197)
* beginnings of mcp

* oauth support

* add workers and better prompt engineering
2026-06-03 12:23:06 -04:00
Nariman Jelveh acc4796b99 Allow Cache-Control and Pragma headers (#3198) 2026-06-03 14:30:33 +02:00
jelveh 106e85d6ca Force revalidation on FileSystem reads 2026-06-03 14:15:22 +02:00
ProgrammerIn-wonderland 0ffc32f19e add whitelist for signed batch uploads (#3196) 2026-06-02 21:30:34 -04:00
Daniel Salazar d897cc5dd0 refactor: move subscriptions to webhook system (#3193)
* fix: dont sign write urls if they don't have write perm

* wip: new subscriptions

* refactor: move subscriptions to
2026-06-02 17:58:56 -07:00
Daniel Salazar 6d2f277ce2 fix: some error handling (#3190) 2026-05-31 18:58:27 -07:00
ProgrammerIn-wonderland 86f80b29c0 tighten RAO policy (#3188) 2026-05-29 20:52:21 -04:00
Daniel SalazarandProgrammerIn-wonderland 39243735e8 feat (put-1049): worker types (#3185)
* feat (put-1049): worket types

* Implement changes to worker types

---------

Co-authored-by: ProgrammerIn-wonderland <[email protected]>
2026-05-29 20:51:57 -04:00
ProgrammerIn-wonderland b3ff9d74b7 add browserjs to taskbar and RecommendedAppService (#3189) 2026-05-29 20:32:53 -04:00
ProgrammerIn-wonderland 7b7604440f Add serverless only filtering for togetherai chat models (#3187) 2026-05-29 14:33:55 -04:00
ProgrammerIn-wonderland 382d2b3c14 add minimax provider (#3171)
* add minimax provider
2026-05-29 13:36:10 -04:00
Anurag Pappula eda5379c17 fix(docs/search): handle search index loading and failure states (#3184)
* fix(docs/search): show loading state and replay query when index isn't ready

Previously performSearch rendered "No results found" while the
MiniSearch index was still loading, which is misleading on slow
networks. Track the last query as pendingQuery and replay it from
fetchSearchIndex once the index is ready; render a distinct
"Failed to load search index" message on fetch error so the
loading message doesn't persist indefinitely.

Fixes #3180.

* fix(docs/search): close stale-replay race and consolidate status messages

Drop pendingQuery synchronously in the input handler when the query
falls below the minimum length, so a fetchSearchIndex that resolves
within the 300ms debounce window after the user clears the field can
no longer replay the cleared query into an empty input.

Move the idle, loading, and failed copy into a single
renderSearchStatus(kind) helper so the four .search-no-results
renderings can only drift in one place.
2026-05-29 16:00:59 +07:00
Ron HernausandDaniel Salazar d2fee51844 Support PostgreSQL database backend (#3167)
* feat(database): add postgres database client

Adds the PostgreSQL database client, native bootstrap migration, SQL preparation helpers, and database config/factory wiring.\n\nRefs #3165.

* feat(database): make backend queries postgres-aware

Updates runtime SQL call sites for database-specific booleans, identifiers, insert-ignore, upserts, JSON extraction, intervals, and Postgres insert ids.\n\nRefs #3165.

* test(database): cover postgres client behavior

Adds unit coverage for SQL preparation, factory selection, write-result mapping, and transaction rollback/commit ordering, plus an env-gated PostgreSQL integration flow.\n\nRefs #3165.

* docs(self-hosting): document postgres database setup

Adds PostgreSQL configuration examples and migration path guidance for self-hosted deployments.\n\nRefs #3165.

* fix: harden postgres oidc tests

* fix(postgres): normalize query results and SQL prep

* fix(user): preserve normalized cache booleans

* test(postgres): run integration coverage with pgmock

* tests: add way to run all tests with postgres though slow

Also adding note that postgres is not in active use so might not work out the box

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-05-28 14:12:17 -07:00
Daniel Salazar 8ac4bad196 fix: readUrl perm check (#3183) 2026-05-28 11:44:16 -07:00
Reynaldi Chernando f140d7221c Add opus 4.8 (#3182)
* Add opus 4.8

* update alias

* handling opus 48 omit temperature
2026-05-28 13:27:46 -04:00
Daniel Salazar 631f0bda1b chore:deploy latest puter.js (#3181) 2026-05-28 09:58:33 -07:00
Reynaldi Chernando 1121314582 Docs search improvement (#3179)
* Init search improvement

* Add fuzzy implementation

* update package lock

* package lock matched with runner 24.16.0
2026-05-28 23:11:34 +07:00
Daniel Salazar dda6b82f0f fix: missing english i18n (#3176) 2026-05-27 16:31:40 -07:00
ProgrammerIn-wonderland 6c757c3fc8 Don't use passed in authorization in workers (#3175) 2026-05-27 18:09:59 -04:00
Daniel Salazar eba194936e tests: more tests for session and auth (#3174) 2026-05-27 12:32:15 -07:00
Daniel Salazar a7bdac16da tests: Add unit tests for XAISpeechToTextDriver (#3173)
closes #3002
2026-05-27 11:49:16 -07:00
Daniel Salazar 716a8d06e2 fix: session store kind column + duplicate error handling (#3172) 2026-05-27 11:44:08 -07:00
Daniel Salazar e95cf44fec fix: small fixes for perf and username checks (#3169) 2026-05-27 03:10:56 -07:00
Daniel Salazar 8f8efcb349 feat (PUT-1025 PUT-1026 and PUT-1017): better handling old token auth required in gui (#3166)
* feat (PUT-1025 PUT-1026 and PUT-1017): better handling old token forcing in gui

* fix: misc issues with token inval project
2026-05-27 02:26:35 -07:00
Daniel Salazar b188942436 feat (PUT-1016 & PUT-1020) (#3164)
* feat (PUT-1016 & PUT-1020)
temp account preservation on forced relogin
hosted asset cookies to v2 token too

* fix: remove llm dashes and ugly comments

* update agents
2026-05-26 23:35:16 -07:00
Daniel Salazar 9b7e9a98c8 fix: session revocation (#3163) 2026-05-26 22:13:12 -07:00
Daniel Salazar d6f7fb6155 chore: package-lock update (#3161) 2026-05-26 20:33:11 -07:00
Daniel Salazar bd91f5e192 feat: worker sessions get their own kind + per-(user, app, worker_name) row (#3160)
* feat: worker sessions get their own kind + per-(user, app, worker_name) row

Schema
------
- mysql_mig_11.sql + sqlite 0054: add idx_sessions_user_worker_active,
  a partial unique index over (user_id, app_uid, meta.worker_name) for
  kind='worker' rows. Active worker sessions are deduped by that triple
  so each named worker gets its own session row and they don't fight
  the existing idx_sessions_user_app_active (which still constrains
  kind='app' only). app_uid is allowed NULL for user-scoped workers
  with no app binding.

SessionStore
------------
- getOrCreateWorker(userId, { appUid, workerName, ... }): mirrors the
  getOrCreateApp pattern — cache lookup, partial-unique re-SELECT on
  insert-ignore, all keyed on the worker triple. expires_at lands at
  WORKER_WINDOW_SECONDS (~99y) so the worker doesn't have to re-mint
  on any cadence.
- #cacheKeyWorker + #allCacheKeysForRow worker branch so revoke /
  update invalidates the worker cache view alongside the by-uuid one.

AuthService
-----------
- createWorkerSessionToken(user, workerName, meta?) now takes the
  workerName explicitly and routes through getOrCreateWorker. Emits
  the same { session, token, gui_token } shape but both JWTs carry
  { worker: true, worker_name }.
- createWorkerAppToken(actor, appUid, workerName) likewise — JWT
  carries the worker_name claim so a verifier can tell two workers
  under the same app apart without a DB round-trip.
- Both methods 400 on empty workerName.

WorkerDriver
------------
- Five auth-mint call sites swapped over: app-bound deploy (3x:
  appId branch, actor.app fallback, hot-reload redeploy), user-bound
  fallback (2x: cold deploy, hot-reload). All pass `workerName` so
  the worker's session row is naturally idempotent across redeploys.

GUI manage-sessions
-------------------
- sessionTitle adds a kind='worker' branch ("name (app)" for
  app-scoped workers, just "name" for user-scoped), pulling worker_name
  from the meta-spread that listSessions already surfaces.
- en.js adds ui_session_kind_worker.

* fix(workers): MySQL JSON_EXTRACT quoting + revoke cache invalidation +
SQLite NULL-distinct in worker index

Three real bugs in the worker session plumbing from the prior commit,
plus a misleading comment. Schema design kept (worker_name lives in
`meta` rather than a dedicated column) per offline review:

1. `#selectWorkerRow` compared `JSON_EXTRACT(meta, '$.worker_name')`
   directly to a bind parameter. MySQL's `JSON_EXTRACT` returns a
   JSON-typed value with embedded quotes (`"name"`, not `name`), so
   the comparison never matched. After the first INSERT, every
   follow-up getOrCreateWorker call missed the existing row in the
   SELECT, hit INSERT-IGNORE, then missed again in the re-SELECT —
   the caller would receive whatever the INSERT-IGNORE returned (a
   no-op row in conflict cases). Wrap with `JSON_UNQUOTE` on MySQL
   via `db.case`; SQLite's `json_extract` already returns the
   unwrapped scalar so it keeps the literal form.

2. mig_11's generated `worker_unique_key` had the same JSON-quoting
   bug. Mirror the fix: `IFNULL(JSON_UNQUOTE(JSON_EXTRACT(...)), '')`
   so the concatenated unique key is a plain string that lines up
   with what `#selectWorkerRow` now binds against.

3. SQLite UNIQUE indexes treat NULL columns as distinct (per the SQL
   standard), so two user-scoped workers (app_uid NULL) with the same
   worker_name would both insert. Wrap the index expression with
   `IFNULL(app_uid, '')` so they correctly conflict — matches the
   MySQL side's `IFNULL` in the generated column.

4. `removeByUuid` / `revokeCascade` SELECTed only the identity
   columns (no `meta`), so `#allCacheKeysForRow`'s worker branch
   couldn't read `meta.worker_name` and the composite
   `sessions:v2:worker:<user>:<app>:<name>` cache key survived
   revocation. Up to CACHE_TTL_SECONDS (15min) afterwards,
   getOrCreateWorker would short-circuit to the cached (revoked) row.
   Add `meta` to both SELECTs; the existing meta-parsing logic in
   `#allCacheKeysForRow` handles the rest.
2026-05-26 20:05:31 -07:00
Daniel SalazarandClaude Opus 4.7 971a1b5071 chore: drop linear/issue ticket references from source comments (#3159)
54 references across 18 files (PUT-1010, PUT-1014, PUT-1019, PUT-1021,
PUT-1022, PUT-1023, PUT-1024 + sub-tags AUTH-2/4/5, SDK-1, PJS-1/2,
GUI-1/2, ROLLOUT-1) removed from inline comments, doc comments,
test describe blocks, and SQL migration headers. The substantive
explanations stay; only the ticket pointers go.

No behavior change. Full backend test suite: 2172 passed / 16 skipped.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:50:43 -07:00
Daniel SalazarandClaude Opus 4.7 ac5eecb7f3 feat (put-1019 put-1021): v2 auth revoke endpoints + silent v1->v2 to… (#3158)
* feat (put-1019 put-1021): v2 auth revoke endpoints + silent v1->v2 token migration

PUT-1019 (AUTH-5): full revoke-endpoint coverage
- /logout: soft-revoke web session + its asset cookies via revokeCascade
  (app sessions and access tokens survive)
- POST /auth/revoke-session: cascade per row kind (web/app/access_token/asset)
- POST /auth/revoke-all-sessions: revoke all web rows for user; optional
  include_apps=true nuclear option; gated by userProtected (cookie-only)
- revokeAccessToken: soft-revoke matching access_token row in addition to
  removing access_token_permissions
- All revokes are UPDATE revoked_at = now(); no DELETE statements remain

PUT-1021 (SDK-1): backend POST /auth/migrate-token
- v1 access_token/app -> mint matching-kind v2 token, idempotent on
  (auth_id, kind, token_uid)
- v1 web/session -> 409 { code: "reauth_required" } (interactive relogin only)
- Same-origin / signed-referer hardening; rate-limited per IP and auth_id
- Gated by auth.allow_v1_tokens; emits puter_token_v2 cookie for app-in-browser

DB migrations: mysql_mig_10, sqlite 0053 (sessions.access_token_uid column)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1019): reject self-revoke; enrich list-sessions response

- handleRevokeSession refuses uuid === req.actor.session.uid (use /logout
  instead). The cookie that authenticated the call should never be the
  target of a self-revoke — the response can't write fresh auth state
  and the client ends up with an ambiguous identity. revoke-all-sessions
  still has the explicit include_current opt-in for the nuclear case.

- AuthService.listSessions now joins the apps table for kind='app' rows
  (returning { uid, name, title, icon } so the manage-sessions UI can
  render the authorizing app without a second round trip), surfaces
  kind / expires_at / label / last_ip / created_via, and filters out
  asset rows (per-cookie children of web rows, revoked transitively via
  cascade — surfacing them as standalone entries would be confusing).

- Sort order: current session first, then most-recently-active. UI
  relies on this to anchor "you are here" at the top.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1019 put-1021): review nits — origin normalization, cookie fallback, types, migration order

B1: createAccessToken.options.expiresIn widened to string | number.
    The impl (#hardExpiryFromExpiresIn) and existing callers/tests use
    jsonwebtoken-style strings ('1h', '30d'); narrowing to number forced
    unsafe casts at every call site. Cast at the single sign() boundary
    where jsonwebtoken's typed template-literal SignOptions clashes
    with the wider runtime contract.

B2: Inline comment on the DELETE in access_token_permissions. The
    AUTH-5 "no DELETE on revoke" rule scoped to the `sessions` table
    (where the cascade graph + audit trail matter). Permissions rows
    are the grant manifest for an active token — once its session is
    soft-revoked they're dead-weight cache entries. A future audit
    requirement would land as a `revoked_at` column on this table,
    not a behavior change in this PR.

B3: handleMigrateToken now sets the puter_token_v2 cookie (with the
    shared sessionCookieFlags + httpOnly) when the migration result
    is kind='app'. The endpoint is already gated on Origin so the
    caller is by definition in a browser; access tokens deliberately
    skip the cookie since they're programmatic.

B4: #isMigrateTokenOriginAllowed normalizes both incoming origin and
    config.origin / allowlist entries (trim + strip trailing slash +
    lowercase) before equality. A misconfigured `config.origin =
    "https://puter.com/"` would otherwise reject every same-origin
    browser call.

B5: Replaced 4x `this.config.cookie_name!` non-null assertions in
    AuthController with `(this.config.cookie_name ?? 'puter_token')`.
    IConfig is `Partial<IConfigOptional>` so cookie_name is undefined
    at runtime in some deployments / test setups; the fallback matches
    the pattern in userProtected / OIDCController / puterSite.

B6: MySQLDatabaseClient sorts migrations numerically by trailing
    integer instead of lexically. Existing files use unpadded names
    (`mysql_mig_<N>.sql`), so plain `.sort()` ran mysql_mig_10 before
    mysql_mig_2 — a future migration that depended on _2..9 running
    first would break. Non-numeric filenames fall through to
    localeCompare for determinism.

B7: Restored the docstring for SessionStore.getOrCreateApp's
    `opts.auth_id` ("Stable per-user identity (survives re-login);
    carried on every v2 JWT so manage-sessions can group by identity")
    — the previous edit truncated it to a fragment.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test + feat: backend test coverage for PUT-1019/1021 review fixes + worker session methods

Tests
-----
- compareMigrationFilenames (new) covering B6 numeric-sort: confirms
  mysql_mig_10.sql lands after mysql_mig_9.sql; non-numeric files sort
  after numbered ones; stable for already-ordered input.
- listSessions (AuthService.test.ts): excludes kind="asset" rows;
  enriches with kind/expires_at/last_ip/created_via; joins kind="app"
  rows with the apps table; sorts current first then by last_activity
  desc.
- handleRevokeSession (AuthController.test.ts): refuses self-revoke
  with 400; still allows revoking a sibling session.
- handleMigrateToken (AuthController.test.ts): rejects missing/
  disallowed Origin; tolerates trailing slash and uppercase Origin
  (B4 normalization); returns 409 reauth_required for v1 web tokens;
  does NOT set the cookie for access-token migration; DOES set the
  puter_token_v2 cookie (httpOnly + sessionCookieFlags) for
  app-under-user migration.
- SessionStore tests updated to import APP_WINDOW_SECONDS /
  WEB_WINDOW_SECONDS rather than hardcoded 30/90 day values — the
  windows just got bumped to 1y and the assertions need to follow
  the constant.

Refactor
--------
- MySQLDatabaseClient exports compareMigrationFilenames so the sort
  logic is unit-testable in isolation.

Worker tokens
-------------
- AuthService.createWorkerSessionToken(user, meta?): mints a new
  kind="web" row tagged meta.worker=true, expires_at =
  WORKER_WINDOW_SECONDS, returns { session, token, gui_token }
  with worker: true on each JWT.
- AuthService.createWorkerAppToken(actor, appUid): mints a new
  kind="app" row tagged meta.worker=true, expires_at =
  WORKER_WINDOW_SECONDS, returns an app-under-user JWT with
  worker: true. Note the existing idx_sessions_user_app_active
  unique index will collide with an existing non-worker app
  session for the same (user, app) — future schema work can
  carve workers out of that uniqueness.

SessionStore.js: WEB/APP_WINDOW_SECONDS now 1y;
WORKER_WINDOW_SECONDS = 99y added for the worker path.

Full backend suite: 2172 passed / 16 skipped / 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:49:36 -07:00
Daniel SalazarandClaude Opus 4.7 3ae076b73e feat (put-1022 put-1023 put-1024): puter.js reauth handling + GUI v2 modal + silent v1 migration (#3157)
* feat (put-1022 put-1023 put-1024): puter.js reauth handling + GUI v2 modal + silent v1 migration

PUT-1022 (PJS-1): puter.js handles 401 reauth_required
- Single apiCall wrapper intercepts 401 { code: "reauth_required" }
- Clears local token, emits puter.auth.reauth_required { reason, auth_id }
- Queues in-flight requests, re-issues after re-auth completes
- web/app: opens puter.com login popup forwarding auth_id
- gui: no-op (handled by GUI-1 modal)
- workers: surfaces as structured exception
- All token writes go through setAuthToken() (future-proofed for PJS-2)

PUT-1023 (GUI-1): web GUI reauth modal + v2 token storage
- Detects 401 reauth_required across http + websocket connect paths
- Soft modal preserves URL/window state; auth_id forwarded into login form
- Storage key migrated: auth_token -> auth_token_v2
- Cookie cleared on logout: puter_token (legacy) cleared, backend writes puter_token_v2
- Cross-tab propagation via localStorage storage events
- WebSocket revocation surfaces as same modal (no silent failure)

PUT-1024 (PJS-2): puter.js storage versioning + silent v1->v2 migration
- New storage key puter.auth.token.v2
- On SDK init: prefer v2 key; else legacy v1 -> silent POST /auth/migrate-token (SDK-1)
- On success: store v2, clear v1. On failure: fall back to PJS-1 reauth flow
- URL-param tokens (?puter.auth.token=, ?auth_token=) also run through silent migration
- Logout clears both keys; setAuthToken writes v2 only

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1023): manage-sessions polish — hide self-revoke, fix modal stack, richer rows

Three issues surfaced during local testing of the manage-sessions UI:

1. The session row representing the caller's own cookie had a Revoke
   button that, if used, left the client in an ambiguous identity state
   (backend now rejects it too — see put-1019 backend fix). The button
   is now omitted entirely when session.current=true; /logout remains
   the right path for ending the active session.

2. The confirm-revoke prompt rendered behind the manage-sessions
   window because both share the dominant z-index pool. UIAlert calls
   now pass parent_uuid (the manage-sessions window's data-element_uuid)
   plus stay_on_top, so the prompt stacks above its parent.

3. Each row only showed the bare uuid. Now renders: title (app
   title for app sessions, label / "Browser session" / "Access token"
   otherwise), app icon when applicable, kind / current badges,
   created / last-active / expires (timeago, with absolute on hover),
   and last_ip. App metadata is joined server-side per the backend
   listSessions change.

Adds en.js strings for the new labels (ui_session_*).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1022): tighten reauth replay + iframe postMessage hygiene

driverCall_ replay (G1): on 401 reauth_required, re-enter driverCall_
instead of calling replayXhrAfterReauth. The generic helper wires the
retried XHR through setupXhrEventHandlers which resolves with the
parsed response and silently skips driverCall_'s streaming detection,
usage-limit / email-confirmation handling, settings.transform, and
resp.result unwrapping — i.e. it would change the driver call API
contract on retry. One-shot via settings._reauthReplayed so a
fresh-token rejection bubbles up instead of looping.

responseType drift (G2): driverCall_ sets xhr.responseType from
settings AFTER initXhr (which captured the stale value into
xhr._puterReq). Mirror the mutation onto _puterReq so any replay path
builds the retry with the live config rather than the snapshot.

targetOrigin lockdown (G3): triggerReauth's parent.postMessage now
targets this.defaultGUIOrigin instead of '*'. The payload carries
reauth metadata + auth_id and is only meaningful to the GUI; '*'
would leak the signal to whatever frame happened to be embedding us.

event.source pinning (G4): the reauth wait listener also matches
event.source against globalThis.parent, not just event.origin —
origin alone admits any same-origin frame on the GUI domain.

event name alignment (G5): the SDK now emits and listens for
'puter.auth.reauth_required' (matches the documented name in the
PR / commit description). The old 'auth.reauth_required' key isn't
yet consumed externally so this is a safe rename.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:49:17 -07:00
Daniel Salazar 5394ccc45c feat: add verification for v2 auth (#3155) 2026-05-26 15:34:54 -07:00
Daniel Salazar 3649109f40 fix: add back AGENTS.md (#3154) 2026-05-25 16:01:35 -07:00
Daniel Salazar 0ff0ea7743 fix: fs event emissions (#3153) 2026-05-25 14:25:27 -07:00
Daniel Salazar 8be115096f feat (put-1012 put-1014): new tokens version signature and logic (#3152) 2026-05-25 14:19:23 -07:00
Daniel Salazar e66fd2373f feat: start adding support for token expiry and invalidation (#3151)
* feat: new expirable + revokable session

* fix: msql migration
2026-05-25 12:30:20 -07:00
jelveh 0e2cce152d Update PuterAlert.js 2026-05-24 17:03:34 -07:00
Daniel Salazar 101e0bc930 fix: search leaking metadata (#3150) 2026-05-24 16:21:33 -07:00
jelveh ea078250bf Strip non-essential media from docs' markdown 2026-05-24 16:15:24 -07:00
jelveh db79f8e484 Add note: serve Puter.js via HTTP server 2026-05-24 15:48:21 -07:00
Daniel Salazar 6422f2d513 fix: associated app ids as input (#3149) 2026-05-23 13:01:03 -07:00
ProgrammerIn-wonderland c60626f772 Add preamble versioning (#3147)
* Add preamble versioning

* update migrations index
2026-05-23 01:15:58 -04:00
ProgrammerIn-wonderland cafdaf7b50 fix bug with signed url directory listings (#3146) 2026-05-22 18:48:41 -04:00
Daniel Salazar bb0f56d758 fix: bring back puter site config logic (#3145) 2026-05-22 11:15:45 -07:00
NighttandReynaldi Chernando 7438db7a90 Add OpenRouter model release dates (#3144)
* Add OpenRouter model release dates

* simplify implementation

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-05-22 12:58:59 -04:00
Daniel Salazar 948f771838 feat: ai limits tightning and add severity alert (#3142) 2026-05-21 20:46:19 -07:00
Nariman Jelveh 82e34f014d Warn on file:// loads using PuterDialog 2026-05-21 19:15:21 -07:00
Nariman Jelveh 9e4c3f8ba3 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-05-21 14:39:44 -07:00
Nariman JelvehandClaude Opus 4.7 e6acd7386a Bump @heyputer/puter.js to 2.4.2
Record the 2.4.2 release in the repo. The version was published to npm
(via prepublishOnly's `npm version patch`) but the bump was never
committed, leaving package.json stuck at 2.4.1 on GitHub.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-21 14:39:18 -07:00
Daniel Salazar b04e8b046a fix: fixing misc issues around alerting and bugs (#3141)
- decrease ai related alerts and make errors not 500
- decreased bad fs controller checks to be 404s instead of 500s
- force bucket regions for now
- validate webdav perms for locking/unlocking
2026-05-21 14:26:47 -07:00
Nariman Jelveh 3262b984b7 Unify Email/Usage dialogs with Puter dialog style 2026-05-21 14:26:26 -07:00
Nariman Jelveh 71af88c73c Bump @heyputer/puter.js to 2.4.1 2026-05-21 14:06:48 -07:00
Nariman Jelveh 2275f65169 Centralize auth popup handling and consent dialog 2026-05-21 12:06:47 -07:00
ProgrammerIn-wonderland 71b7d6367f fix together provider issues (#3140) 2026-05-21 08:58:48 -04:00
Nariman Jelveh 08369d840b Add cssstyle to docs dependencies (#3139) 2026-05-20 17:41:02 -07:00
Daniel Salazar 94236133b5 fix: event types (#3138)
* fix: agents.md

* fix weird names

* fix: weird names again

* fix: event types

* cleanup types
2026-05-20 16:26:16 -07:00
Daniel Salazar 84b48541bd fix: event types + emit subdomain serve event (#3137)
* fix: agents.md

* fix weird names

* fix: weird names again
2026-05-20 12:47:56 -07:00
Daniel Salazar a33d02f522 fix: agents.md name (#3136) 2026-05-20 12:18:35 -07:00
Reynaldi Chernando f94d4aa7d9 Update gemini 3.5 flash release date (#3135) 2026-05-20 13:25:46 -04:00
Daniel Salazar 89cfb0c634 fix: subdomain htmls (#3134) 2026-05-19 22:04:57 -07:00
267 changed files with 24986 additions and 5423 deletions

No files matched your search

+2
View File
@@ -70,3 +70,5 @@ coverage/
undefined
servers.json
config.*.json
volatile/
+4
View File
@@ -66,6 +66,10 @@ Keep comments light. Prefer self-documenting code — clear names, small functio
Don't restate what the code already says. Don't write comments that reference the current task or PR — those rot.
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the *why* in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
**Use plain ASCII `-` in comment section dividers, not box-drawing characters.** Write `// -- Section name --`, not `// ── Section name ──`. The `─` (U+2500) character renders inconsistently in some editors and diff tools and adds nothing over `-`.
### Tests
When adding new behavior (function, endpoint, driver method, branch of logic), add a test for it.
+34 -1
View File
@@ -17,18 +17,51 @@ Out-of-scope:
* Denial of Service (DoS), spam, or volumetric attacks.
* Physical security issues.
## Known Non-Issues (Please Check Before Submitting)
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
* **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we *do* evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
* **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
* **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
* **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
* **App metadata or app user-count "leaks".** This information is currently public by design.
* **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
* **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
* **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
* **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
* **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
* **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
* **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
## Rules of Engagement
To participate, you must:
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code or screenshots where applicable.
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see *Proof of Reproduction* below).
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
5. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
6. **Check the Known Non-Issues list**: Reports matching an item in the "Known Non-Issues" section above are not eligible and will be closed.
Reports that do not meet these guidelines may not be eligible for a reward.
## Proof of Reproduction
Reports must demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
* Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
* The **observed** result versus the **expected** result.
* For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
* For server-side bugs: a runnable proof-of-concept.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and are generally not eligible**. If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Please submit **one issue per report**. Bundled "audit packs" of many speculative findings will be declined; send each confirmed issue separately.
## Reporting Process
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
+2
View File
@@ -6,6 +6,8 @@
"domain": "puter.localhost",
"cookie_name": "puter_auth_token",
"jwt_secret": "dev-jwt-secret-change-me",
"jwt_secret_v2": "dev-jwt-secret-v2-change-me",
"allow_v1_tokens": true,
"url_signature_secret": "dev-url-signature-secret-change-me",
"allow_all_host_values": true,
"allow_no_host_header": true,
+26 -4
View File
@@ -67,7 +67,11 @@
// ── Auth / session ──────────────────────────────────────────────────
// ALWAYS replace these for any public install — `openssl rand -hex 64`.
// `jwt_secret` is legacy verify-only; new tokens sign with `jwt_secret_v2`.
"jwt_secret": "change-me",
"jwt_secret_v2": "change-me",
// Set false to retire v1 tokens entirely (ROLLOUT-1).
"allow_v1_tokens": true,
"url_signature_secret": "change-me",
"cookie_name": "puter_auth_token",
"min_pass_length": 6,
@@ -155,17 +159,30 @@
// ── Database ────────────────────────────────────────────────────────
"database": {
// `sqlite` for single-node / dev; `mysql` for self-host with MariaDB.
// `sqlite` for single-node/dev, `mysql` for MariaDB/MySQL,
// or `postgres` for PostgreSQL.
//
// NOTE: `postgres` support is a community contribution and is not
// exercised by Puter.com production. It boots, passes its own
// integration tests against pgmock, and runs the common user/app/
// fsentry/session/permission/OIDC flows — but less-traveled code
// paths may surface MySQL/SQLite-isms that haven't been ported yet.
// Expect rough edges and please file issues if you hit one. For
// production self-hosting today, `mysql` (MariaDB) and `sqlite` are
// the supported defaults.
"engine": "sqlite",
// sqlite — file path on disk
"path": "volatile/runtime/puter-database.sqlite",
"targetVersion": 0,
// mysql — connection details
// mysql/postgres — connection details. PostgreSQL defaults to port
// 5432 when `engine` is `postgres`; MySQL/MariaDB normally use 3306.
"host": "",
"port": 3306,
"user": "",
"password": "",
"database": "",
// postgres may also use a URL instead of host/user/password fields:
// "connectionString": "postgres://puter:secret@localhost:5432/puter",
// Optional read-replica pool. Reads route here when populated.
"replica": {
"host": "",
@@ -174,9 +191,10 @@
"password": "",
"database": ""
}
// mysql self-host bootstrap: set `migrationPaths` to apply the bundled
// schema on first boot. Idempotent — safe to leave on.
// mysql/postgres self-host bootstrap: set `migrationPaths` to apply the
// bundled schema on first boot. Idempotent — safe to leave on.
// "migrationPaths": ["./src/backend/clients/database/migrations/mysql"]
// "migrationPaths": ["./src/backend/clients/database/migrations/postgres"]
},
// ── DynamoDB (KV store) ─────────────────────────────────────────────
@@ -248,6 +266,10 @@
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
},
"openrouter": {
"apiKey": "",
"apiBaseUrl": "https://openrouter.ai/api/v1"
+25 -1
View File
@@ -48,6 +48,7 @@ MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
cat > .env <<EOF
@@ -78,6 +79,8 @@ cat > puter/config/config.json <<EOF
"private_app_hosting_domain_alt": "dev.puter.localhost",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"allow_v1_tokens": true,
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
@@ -131,8 +134,9 @@ Replace `puter.localhost`, `site.puter.localhost`, `host.puter.localhost`, `dev.
Why these knobs:
- `jwt_secret` + `jwt_secret_v2` — Puter signs new auth tokens with `jwt_secret_v2` (v2 token format, `kid: 'v2'` JWT header). `jwt_secret` is verify-only and lets tokens minted before this version (cookies still in users' browsers, stored API tokens) keep working. `allow_v1_tokens: true` keeps that fallback enabled. Fresh installs can leave it as-is; future versions will flip the flag to `false` and retire `jwt_secret` entirely once legacy tokens have drained.
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start=gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL schema on boot. `mysql_mig_1.sql` (tables) and `mysql_mig_2.sql` (default apps: editor, viewer, pdf, camera, player, recorder, git, dev-center, puter-linux). Idempotent — safe to re-run.
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
@@ -228,6 +232,26 @@ Change it in Settings after first login.
All optional. Drop any of the blocks below into `puter/config/config.json` and `docker compose restart puter`. See [config.template.jsonc](../config.template.jsonc) for the full list. Per-key documentation lives in [src/backend/types.ts](../src/backend/types.ts).
### PostgreSQL database
> **Community contribution — expect rough edges.** PostgreSQL support was contributed by the community and is not run by Puter.com production. It boots, applies the bundled schema, and exercises the common user/app/fsentry/session/permission/OIDC flows in the integration tests, but less-traveled SQL call sites may still need porting. If you hit a query that doesn't work on Postgres, please open an issue. For production self-hosting today, MariaDB/MySQL and SQLite are the supported defaults.
The bundled Docker Compose stack still defaults to MariaDB. To use PostgreSQL instead, run a PostgreSQL service yourself, point Puter at it, and use the PostgreSQL migration path:
```json
"database": {
"engine": "postgres",
"host": "postgres",
"port": 5432,
"user": "puter",
"password": "...",
"database": "puter",
"migrationPaths": ["/opt/puter/dist/src/backend/clients/database/migrations/postgres"]
}
```
You may use `"connectionString": "postgres://puter:...@postgres:5432/puter"` instead of the host/user/password fields. Puter only needs normal PostgreSQL connection details and the migration path; CloudNativePG is one possible Kubernetes operator, but no operator-specific manifests are required by Puter.
### Email (SMTP)
Used for password resets, email confirmation, and notifications. Without it those flows silently fail.
+1 -1
View File
@@ -84,7 +84,7 @@ export const handleAppTelemetryUsers = async (
const users = await clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
INNER JOIN user u ON p.user_id = u.id
INNER JOIN ${clients.db.quoteIdentifier('user')} u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
+4 -4
View File
@@ -95,7 +95,7 @@ async function decodeAndValidateThumbnail(
return { mimeType, data };
}
// ── thumbnail.created ───────────────────────────────────────────────
// -- thumbnail.created -----------------------------------------------
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer.
@@ -253,7 +253,7 @@ extension.on(
},
);
// ── thumbnail.upload.prepare ────────────────────────────────────────
// -- thumbnail.upload.prepare ----------------------------------------
// Generate pre-signed upload URLs so the client can PUT directly to S3.
extension.on(
@@ -266,7 +266,7 @@ extension.on(
},
);
// ── thumbnail.read ──────────────────────────────────────────────────
// -- thumbnail.read --------------------------------------------------
// Convert s3:// or legacy https:// thumbnails to signed URLs.
extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
@@ -279,7 +279,7 @@ extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
});
});
// ── fs.remove.node ──────────────────────────────────────────────────
// -- fs.remove.node --------------------------------------------------
// Delete S3 thumbnail when the file is removed.
extension.on(
+5
View File
@@ -113,7 +113,10 @@ if ($writeConfig) {
$mariadbRootPw = New-HexSecret 32
$mariadbPw = New-HexSecret 32
$s3SecretKey = New-HexSecret 32
# Two JWT secrets: $jwtSecret is verify-only for legacy v1 tokens
# already in circulation; $jwtSecretV2 signs every new token.
$jwtSecret = New-HexSecret 64
$jwtSecretV2 = New-HexSecret 64
$urlSigSecret = New-HexSecret 64
$envContent = @"
@@ -142,6 +145,8 @@ S3_BUCKET=puter-local
private_app_hosting_domain = "app.$PuterDomain"
private_app_hosting_domain_alt = "dev.$PuterDomain"
jwt_secret = $jwtSecret
jwt_secret_v2 = $jwtSecretV2
allow_v1_tokens = $true
url_signature_secret = $urlSigSecret
database = [ordered]@{
engine = 'mysql'
+6
View File
@@ -92,7 +92,11 @@ if [ "$write_config" = "1" ]; then
MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
# Two JWT secrets: `jwt_secret` is verify-only for legacy v1 tokens
# already in circulation; `jwt_secret_v2` signs every new token.
# Both are required at boot (`jwt_secret` only when verifying v1).
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
cat > .env <<EOF
@@ -123,6 +127,8 @@ EOF
"private_app_hosting_domain_alt": "dev.$PUTER_DOMAIN",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"allow_v1_tokens": true,
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
+2012 -2596
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -45,6 +45,7 @@
},
"scripts": {
"test:backend": "vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run build:ts",
+19 -8
View File
@@ -23,7 +23,7 @@ import { createHash } from 'node:crypto';
import type { IConfig } from '../../types';
import { PuterClient } from '../types';
// ── Types ────────────────────────────────────────────────────────────
// -- Types ------------------------------------------------------------
export interface AlarmFields {
error?: Error;
@@ -49,7 +49,7 @@ interface Alarm {
noAlert?: boolean;
}
type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
export type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
export interface AlertPayload {
id: string;
@@ -73,7 +73,7 @@ interface KnownErrorRule {
};
}
// ── Helpers ──────────────────────────────────────────────────────────
// -- Helpers ----------------------------------------------------------
/**
* Deterministic short identifier derived from an alarm ID.
@@ -166,7 +166,7 @@ function cleanFields(fields: AlarmFields): Record<string, string> {
return out;
}
// ── AlarmClient ──────────────────────────────────────────────────────
// -- AlarmClient ------------------------------------------------------
/**
* Manages system alarms and dispatches alerts to external paging
@@ -184,7 +184,7 @@ export class AlarmClient extends PuterClient {
super(config);
}
// ── Lifecycle ────────────────────────────────────────────────────
// -- Lifecycle ----------------------------------------------------
override async onServerStart(): Promise<void> {
const pagerConf = this.config.pager;
@@ -228,13 +228,23 @@ export class AlarmClient extends PuterClient {
console.log('[alarm] entering drain mode — suppressing new alarms');
}
// ── Public API ───────────────────────────────────────────────────
// -- Public API ---------------------------------------------------
/**
* Create or update an alarm. If the alarm ID already exists, the
* occurrence count is incremented and a repeat alert is dispatched.
*
* `severity` is the PagerDuty severity for this alarm; omit for the
* default 'critical'. Use 'info' / 'warning' for expected-but-worth-
* tracking signals (e.g. a user hitting a rate limit) so they record
* and de-dupe like any other alarm but don't page on-call.
*/
create(id: string, message: string, fields: AlarmFields = {}): void {
create(
id: string,
message: string,
fields: AlarmFields = {},
severity?: PagerSeverity,
): void {
if (this.draining) {
if (!this.drainLogged) {
this.drainLogged = true;
@@ -256,6 +266,7 @@ export class AlarmClient extends PuterClient {
shortId: shortId(id),
message,
fields,
severity,
started: Date.now(),
timestamps: [Date.now()],
occurrences: [],
@@ -298,7 +309,7 @@ export class AlarmClient extends PuterClient {
this.knownErrors = rules;
}
// ── Internals ────────────────────────────────────────────────────
// -- Internals ----------------------------------------------------
private recordOccurrence(
alarm: Alarm,
@@ -0,0 +1,50 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Optional ClickHouse client.
*
* ClickHouse is NOT a hard dependency. Core Puter runs without it — the app
* stats read path (open / unique-user counts) falls back to the primary SQL
* database. `this.clients.clickhouse` is therefore typed as optional and is
* `undefined` in default and self-hosted setups; callers MUST branch on its
* presence and fall back to SQL.
*
* A production deployment can register a real ClickHouse client via an
* extension (`extension.registerClient('clickhouse', client)`) to offload the
* analytics queries off the primary database and keep the stats path fast at
* scale. When registered, the instance flows into `this.clients.clickhouse`
* everywhere it's typed.
*
* Only the surface the stats path actually consumes is declared here. Extend
* this interface (don't widen to `any`) when a new query shape is needed.
*/
export interface ClickhouseQueryResult {
json<T = Record<string, unknown>>(): Promise<T[]>;
}
export interface ClickhouseQueryParams {
query: string;
query_params?: Record<string, unknown>;
format?: string;
}
export interface ClickhouseClient {
query(params: ClickhouseQueryParams): Promise<ClickhouseQueryResult>;
}
+90 -3
View File
@@ -31,6 +31,8 @@ export interface BatchEntry {
values: unknown[];
}
type SqlJsonPath = readonly [string, ...string[]];
/**
* Base database client. Subclasses must override every method that throws here.
*
@@ -101,9 +103,10 @@ export class AbstractDatabaseClient extends PuterClient {
const cols = Object.keys(data);
const values = Object.values(data);
const sql =
`INSERT INTO \`${tableName}\` ` +
`(${cols.map((c) => `\`${c}\``).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`;
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
}
@@ -161,4 +164,88 @@ export class AbstractDatabaseClient extends PuterClient {
}
return choices.otherwise as T;
}
quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `\`${part.replaceAll('`', '``')}\``;
})
.join('.');
}
booleanLiteral(value: boolean): string {
return value ? '1' : '0';
}
booleanValue(value: boolean): boolean | 0 | 1 {
return value ? 1 : 0;
}
insertIgnoreInto(tableName: string): string {
const table = this.quoteIdentifier(tableName);
return this.case({
sqlite: `INSERT OR IGNORE INTO ${table}`,
postgres: `INSERT INTO ${table}`,
otherwise: `INSERT IGNORE INTO ${table}`,
});
}
insertIgnoreSuffix(): string {
return this.case({
postgres: ' ON CONFLICT DO NOTHING',
otherwise: '',
});
}
upsertClause(
conflictColumns: readonly string[],
updateColumns: readonly string[],
): string {
if (updateColumns.length === 0) {
throw new Error('upsertClause requires at least one update column');
}
const updateList = updateColumns
.map((column) => `${this.quoteIdentifier(column)} = ?`)
.join(', ');
return this.case({
mysql: `ON DUPLICATE KEY UPDATE ${updateList}`,
otherwise: `ON CONFLICT(${conflictColumns
.map((column) => this.quoteIdentifier(column))
.join(', ')}) DO UPDATE SET ${updateList}`,
});
}
jsonTextExtract(jsonExpression: string, path: SqlJsonPath): string {
const sqlitePath = `$${path.map((part) => `.${part}`).join('')}`;
return this.case({
sqlite: `json_extract(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)})`,
mysql: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
postgres: `${jsonExpression} #>> ARRAY[${path
.map((part) => this.sqlStringLiteral(part))
.join(', ')}]`,
otherwise: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
});
}
nullCoalesce(...expressions: readonly string[]): string {
if (expressions.length === 0) {
throw new Error('nullCoalesce requires at least one expression');
}
return `COALESCE(${expressions.join(', ')})`;
}
returningIdClause(): string {
return this.case({
postgres: ' RETURNING id',
otherwise: '',
});
}
protected sqlStringLiteral(value: string): string {
return `'${value.replaceAll("'", "''")}'`;
}
}
@@ -0,0 +1,95 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { compareMigrationFilenames } from './MySQLDatabaseClient.js';
describe('compareMigrationFilenames', () => {
it('orders numbered migrations numerically, not lexically', () => {
const input = [
'mysql_mig_10.sql',
'mysql_mig_2.sql',
'mysql_mig_1.sql',
'mysql_mig_9.sql',
'mysql_mig_3.sql',
];
const sorted = [...input].sort(compareMigrationFilenames);
expect(sorted).toEqual([
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_3.sql',
'mysql_mig_9.sql',
'mysql_mig_10.sql',
]);
});
it('keeps mig_10 after mig_9 when the real prod set is shuffled', () => {
// Reflects the current migrations/mysql/ listing — guards against
// a future "let's just rename to padded" suggestion accidentally
// re-introducing the lex bug if the rename is incomplete.
const real = [
'mysql_mig_9.sql',
'mysql_mig_3.sql',
'mysql_mig_10.sql',
'mysql_mig_1.sql',
'mysql_mig_7.sql',
'mysql_mig_5.sql',
'mysql_mig_2.sql',
'mysql_mig_4.sql',
'mysql_mig_8.sql',
'mysql_mig_6.sql',
];
const sorted = [...real].sort(compareMigrationFilenames);
for (let i = 1; i <= sorted.length; i += 1) {
expect(sorted[i - 1]).toBe(`mysql_mig_${i}.sql`);
}
});
it('sorts non-numeric filenames after numbered ones, lexically among themselves', () => {
// Numbered files always run first (they're the canonical history);
// unmatched names follow in localeCompare order. Mixing the two
// sets prevents a vendor dump from accidentally wedging itself
// between mig_4 and mig_5 if it happened to lex-sort there.
const mixed = [
'mysql_mig_2.sql',
'mysql_vendor_dump.sql',
'mysql_mig_10.sql',
'mysql_bootstrap.sql',
'mysql_mig_1.sql',
];
const sorted = [...mixed].sort(compareMigrationFilenames);
expect(sorted).toEqual([
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_10.sql',
'mysql_bootstrap.sql',
'mysql_vendor_dump.sql',
]);
});
it('is stable for already-sorted input', () => {
const sorted = [
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_10.sql',
'mysql_mig_11.sql',
];
expect([...sorted].sort(compareMigrationFilenames)).toEqual(sorted);
});
});
@@ -23,6 +23,7 @@ import { createPool, type Pool } from 'mysql2';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { SQLBatcher } from './SQLBatcher.js';
import { splitMysqlStatements } from './splitMysqlStatements.js';
import { compareMigrationFilenames } from './migrationFilenames.js';
import type { IConfig } from '../../types';
const RETRIABLE_ERROR_CODES = new Set([
@@ -44,6 +45,8 @@ const RETRIABLE_ERROR_MESSAGES = [
'ETIMEDOUT',
];
export { compareMigrationFilenames };
type PoolConfig = Parameters<typeof createPool>[0];
enum Configuration {
@@ -247,7 +250,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
.filter(
(f) => f.endsWith('.sql') && f.startsWith('mysql'),
)
.sort();
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[mysql] migration path is unreadable: ${dir}`,
@@ -0,0 +1,328 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Pool } from 'pg';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { IConfig } from '../../types';
import type { PuterServer } from '../../server';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import {
createPgMockPostgresDatabaseClient,
POSTGRES_TEST_MIGRATIONS_PATH,
setupTestServer,
} from '../../testUtil.js';
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
const postgresUrl = process.env.PUTER_TEST_POSTGRES_URL;
const postgresMigrationsPath = POSTGRES_TEST_MIGRATIONS_PATH;
const postgresTestSchemaPattern = /^puter_test_[a-f0-9]{32}$/u;
const postgresIntegrationTimeoutMs = 180_000;
let postgresTestSchema: string | undefined;
let postgresTestUrl: string | undefined;
const postgresConfig = (overrides: Partial<IConfig> = {}): IConfig => {
if (postgresUrl && !postgresTestUrl) {
throw new Error('Postgres test schema was not initialized');
}
const { database: databaseOverrides, ...rootOverrides } = overrides;
const database = postgresUrl
? {
engine: 'postgres' as const,
inMemory: false,
connectionString: postgresTestUrl,
migrationPaths: [postgresMigrationsPath],
}
: {
engine: 'postgres' as const,
inMemory: true,
migrationPaths: [postgresMigrationsPath],
};
return {
port: 0,
extensions: [],
database: { ...database, ...(databaseOverrides ?? {}) },
...rootOverrides,
};
};
const quoteTestSchemaIdentifier = (schema: string): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
return `"${schema}"`;
};
const postgresConnectionStringForSchema = (
connectionString: string,
schema: string,
): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
const url = new URL(connectionString);
url.searchParams.set('options', `-c search_path=${schema}`);
return url.toString();
};
const createPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl) return;
const schema = `puter_test_${uuidv4().replaceAll('-', '')}`;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(`CREATE SCHEMA ${quoteTestSchemaIdentifier(schema)}`);
postgresTestSchema = schema;
postgresTestUrl = postgresConnectionStringForSchema(
postgresUrl,
schema,
);
} finally {
await pool.end();
}
};
const dropPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl || !postgresTestSchema) return;
const schema = postgresTestSchema;
postgresTestSchema = undefined;
postgresTestUrl = undefined;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(
`DROP SCHEMA IF EXISTS ${quoteTestSchemaIdentifier(schema)} CASCADE`,
);
} finally {
await pool.end();
}
};
describe('PostgresDatabaseClient integration', () => {
let server: PuterServer | undefined;
beforeEach(async () => {
await createPostgresTestSchema();
});
afterEach(async () => {
try {
await server?.shutdown();
} finally {
server = undefined;
await dropPostgresTestSchema();
}
});
it(
'applies the native migrations idempotently to an empty database',
async () => {
const config = postgresConfig();
const pgMockClient = postgresUrl
? undefined
: await createPgMockPostgresDatabaseClient(config);
try {
const firstClient =
pgMockClient?.client ?? new PostgresDatabaseClient(config);
try {
await firstClient.onServerStart();
} finally {
await firstClient.onServerShutdown();
}
const secondClient =
pgMockClient?.createClient() ??
new PostgresDatabaseClient(config);
try {
await secondClient.onServerStart();
const [systemUser] = await secondClient.read(
'SELECT `id`, `username` FROM `user` WHERE `username` = ?',
['system'],
);
const [devCenter] = await secondClient.read(
'SELECT `name`, `index_url` FROM `apps` WHERE `name` = ?',
['dev-center'],
);
expect(systemUser).toMatchObject({
id: 1,
username: 'system',
});
expect(devCenter).toMatchObject({
name: 'dev-center',
index_url:
'https://builtins.namespaces.puter.com/dev-center',
});
} finally {
await secondClient.onServerShutdown();
}
} finally {
pgMockClient?.destroy();
}
},
postgresIntegrationTimeoutMs,
);
it('starts the server and exercises user, app, fsentry, permission, and session flows', async () => {
const userStorageAllowance = 123_456_789;
server = await setupTestServer(
postgresConfig({
no_default_user: false,
is_storage_limited: true,
}),
);
const admin = await server.stores.user.getByUsername('admin');
expect(admin?.username).toBe('admin');
const username = `pg-${uuidv4().slice(0, 8)}`;
const createdUser = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: userStorageAllowance,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
createdUser,
);
const user = await server.stores.user.getById(createdUser.id);
if (!user) throw new Error('created user was not readable');
expect(user.username).toBe(username);
await expect(
server.stores.fsEntry.getUserStorageAllowance(user.id),
).resolves.toMatchObject({
max: userStorageAllowance,
});
const otherUsername = `pg-other-${uuidv4().slice(0, 8)}`;
const otherUser = await server.stores.user.create({
username: otherUsername,
uuid: uuidv4(),
password: null,
email: `${otherUsername}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const authResult = await server.services.auth.createSessionToken(
user,
{
ip: '127.0.0.1',
user_agent: 'postgres-integration-test',
},
);
const authenticated =
await server.services.auth.authenticateFromToken(authResult.token);
expect(authenticated?.user?.id).toBe(user?.id);
const devCenter = await server.stores.app.getByName('dev-center');
if (!devCenter) throw new Error('dev-center app was not seeded');
expect(devCenter.name).toBe('dev-center');
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
if (!documents) throw new Error('Documents directory was not created');
expect(documents.isDir).toBe(true);
const folder = await server.stores.fsEntry.createNonFileEntry({
userId: user.id,
parent: documents,
name: 'postgres-folder',
kind: 'directory',
});
const renamedFolder = await server.stores.fsEntry.updateEntry(
folder.uuid,
{
name: 'postgres-folder-renamed',
path: `/${username}/Documents/postgres-folder-renamed`,
},
);
expect(renamedFolder.name).toBe('postgres-folder-renamed');
await server.stores.permission.upsertUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
{ ok: true },
);
await expect(
server.stores.permission.hasUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
),
).resolves.toBe(true);
await server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
);
await expect(
server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
),
).resolves.toBeUndefined();
await expect(
server.stores.oidc.link(
otherUser.id,
'postgres-integration-test',
'subject-1',
null,
),
).rejects.toMatchObject({
statusCode: 409,
legacyCode: 'conflict',
});
const session = await server.stores.session.create(user.id, {
meta: { source: 'postgres-integration-test' },
});
const activeSession = await server.stores.session.getByUuid(
session.uuid,
);
expect(activeSession?.uuid).toBe(session.uuid);
const workerName = `worker-${uuidv4().slice(0, 8)}`;
const workerSession = await server.stores.session.getOrCreateWorker(
user.id,
{ workerName },
);
expect(workerSession?.kind).toBe('worker');
expect(workerSession?.meta?.worker_name).toBe(workerName);
await server.stores.session.removeByUuid(session.uuid);
await expect(
server.stores.session.getByUuid(session.uuid),
).resolves.toBeNull();
}, postgresIntegrationTimeoutMs);
});
@@ -0,0 +1,360 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { FieldDef, QueryResult } from 'pg';
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types';
import { DatabaseClientFactory } from './index.js';
import {
mapPostgresWriteResult,
PostgresDatabaseClient,
type PostgresPool,
type PostgresPoolClient,
} from './PostgresDatabaseClient.js';
type QueryCall = {
text: string;
values?: unknown[];
};
const postgresConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
},
});
const postgresReplicaConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
replica: {},
},
});
const field = (name: string, dataTypeID: number): FieldDef => ({
name,
tableID: 0,
columnID: 0,
dataTypeID,
dataTypeSize: -1,
dataTypeModifier: -1,
format: 'text',
});
const int8Field = (name: string): FieldDef => field(name, 20);
const textField = (name: string): FieldDef => field(name, 25);
const queryResult = (
rows: Record<string, unknown>[] = [],
rowCount = rows.length,
fields: FieldDef[] = [],
): QueryResult<Record<string, unknown>> => ({
command: '',
fields,
oid: 0,
rowCount,
rows,
});
class RecordingPoolClient implements PostgresPoolClient {
readonly calls: QueryCall[] = [];
released = false;
constructor(private readonly failOnText?: string) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (this.failOnText && text.includes(this.failOnText)) {
throw new Error(`forced query failure: ${text}`);
}
return queryResult([], text === 'ROLLBACK' ? 0 : 1);
}
release(): void {
this.released = true;
}
}
class RecordingPool implements PostgresPool {
readonly calls: QueryCall[] = [];
constructor(
private readonly client: RecordingPoolClient =
new RecordingPoolClient(),
private readonly nextResult: QueryResult<Record<string, unknown>> =
queryResult(),
) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (text === 'SELECT 1') return queryResult([{ ok: 1 }], 1);
return this.nextResult;
}
async connect(): Promise<PostgresPoolClient> {
return this.client;
}
async end(): Promise<void> {}
}
describe('PostgresDatabaseClient', () => {
it('is selected by the database factory', () => {
const client = new DatabaseClientFactory(postgresConfig());
expect(client).toBeInstanceOf(PostgresDatabaseClient);
});
it('maps pg write results to the shared WriteResult shape', () => {
expect(
mapPostgresWriteResult(queryResult([{ id: '42' }], 1)),
).toEqual({
insertId: 42,
affectedRows: 1,
anyRowsAffected: true,
});
expect(mapPostgresWriteResult(queryResult([], 0))).toEqual({
insertId: 0,
affectedRows: 0,
anyRowsAffected: false,
});
});
it('prepares write SQL at the pg boundary', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ id: '7' }], 1),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
const result = await client.write(
'INSERT INTO `apps` (`name`) VALUES (?) RETURNING id',
['editor'],
);
expect(result.insertId).toBe(7);
expect(pool.calls.at(-1)).toEqual({
text: 'INSERT INTO "apps" ("name") VALUES ($1) RETURNING id',
values: ['editor'],
});
});
it('normalizes int8 fields on read and primary read rows', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[
{
uuid: 'session-1',
created_at: '1710000000',
last_activity: '1710000001',
expires_at: '1710000002',
revoked_at: null,
},
],
1,
[
textField('uuid'),
int8Field('created_at'),
int8Field('last_activity'),
int8Field('expires_at'),
int8Field('revoked_at'),
],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
await expect(client.pread('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
});
it('rejects unsafe int8 values instead of losing precision', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[{ id: '9007199254740992' }],
1,
[int8Field('id')],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT `id` FROM `sessions`')).rejects.toThrow(
'safe integer',
);
});
it('normalizes tryHardRead rows returned by a replica', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '1' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '2' }], 1, [
int8Field('created_at'),
]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 2 }]);
});
it('normalizes tryHardRead rows returned by primary fallback', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '3' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([], 0, [int8Field('created_at')]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 3 }]);
});
it('runs batch writes in order and commits', async () => {
const conn = new RecordingPoolClient();
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
{
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
values: ['session-1'],
},
]);
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{
text: 'DELETE FROM "sessions" WHERE "uuid" = $1',
values: ['session-1'],
},
{ text: 'COMMIT', values: undefined },
]);
expect(conn.released).toBe(true);
});
it('rolls back and releases the pg connection when a batch write fails', async () => {
const conn = new RecordingPoolClient('UPDATE "user"');
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await expect(
client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]),
).rejects.toThrow('forced query failure');
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{ text: 'ROLLBACK', values: undefined },
]);
expect(conn.released).toBe(true);
});
});
@@ -0,0 +1,390 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { Pool, type PoolConfig, type QueryResult } from 'pg';
import {
AbstractDatabaseClient,
type BatchEntry,
type WriteResult,
} from './DatabaseClient';
import { compareMigrationFilenames } from './migrationFilenames.js';
import { preparePostgresSql } from './preparePostgresSql.js';
import { splitPostgresStatements } from './splitPostgresStatements.js';
import type { IConfig } from '../../types';
type PostgresEndpointConfig = {
host?: string;
port?: number;
user?: string;
password?: string;
database?: string;
connectionString?: string;
url?: string;
};
export interface PostgresQueryable {
query(query: string, values?: unknown[]): Promise<QueryResult>;
}
export interface PostgresPoolClient extends PostgresQueryable {
release(): void;
}
export interface PostgresPool extends PostgresQueryable {
connect(): Promise<PostgresPoolClient>;
end(): Promise<void>;
}
type PostgresPoolFactory = (poolConfig: PoolConfig) => PostgresPool;
enum Configuration {
SINGLE,
REPLICA,
}
const POSTGRES_INT8_OID = 20;
const INTEGER_TEXT_PATTERN = /^-?\d+$/u;
const normalizePostgresInt8 = (
value: unknown,
columnName: string,
): number | null | undefined => {
if (value === null || value === undefined) return value;
const parsed =
typeof value === 'bigint'
? Number(value)
: typeof value === 'number'
? value
: typeof value === 'string' && INTEGER_TEXT_PATTERN.test(value)
? Number(value)
: Number.NaN;
if (!Number.isSafeInteger(parsed)) {
throw new Error(
`[postgres] int8 column ${columnName} is outside JavaScript's safe integer range`,
);
}
return parsed;
};
const normalizePostgresRows = (
result: QueryResult,
): Record<string, unknown>[] => {
const int8Fields = result.fields.filter(
(field) => field.dataTypeID === POSTGRES_INT8_OID,
);
if (int8Fields.length === 0) {
return result.rows as Record<string, unknown>[];
}
return result.rows.map((row) => {
const normalized: Record<string, unknown> = { ...row };
for (const field of int8Fields) {
normalized[field.name] = normalizePostgresInt8(
normalized[field.name],
field.name,
);
}
return normalized;
});
};
export const mapPostgresWriteResult = (result: QueryResult): WriteResult => {
const affectedRows = result.rowCount ?? 0;
const rowId = result.rows[0]?.id;
const insertId =
typeof rowId === 'bigint'
? rowId
: typeof rowId === 'number'
? rowId
: typeof rowId === 'string' && rowId !== ''
? Number(rowId)
: 0;
const normalizedInsertId =
typeof insertId === 'number' && Number.isNaN(insertId) ? 0 : insertId;
return {
insertId: normalizedInsertId,
affectedRows,
anyRowsAffected: affectedRows > 0,
};
};
export class PostgresDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'postgres';
private primaryPool!: PostgresPool;
private replicaPool!: PostgresPool;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(
config: IConfig,
private readonly poolFactory: PostgresPoolFactory = (poolConfig) =>
new Pool(poolConfig) as unknown as PostgresPool,
) {
super(config);
}
override async onServerStart(): Promise<void> {
const dbConf = this.config.database!;
this.primaryPool = this.createPool(dbConf);
await this.primaryPool.query('SELECT 1');
console.log('[postgres] connected to primary');
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
await this.replicaPool.query('SELECT 1');
this.configuration = Configuration.REPLICA;
console.log('[postgres] connected to read-replica');
} else {
this.replicaPool = this.primaryPool;
this.configuration = Configuration.SINGLE;
}
await this.runMigrations();
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
const drainMs = 60_000;
console.log(
`[postgres] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools().catch((e) =>
console.error('[postgres] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools();
}
override quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `"${part.replaceAll('"', '""')}"`;
})
.join('.');
}
override booleanLiteral(value: boolean): string {
return value ? 'TRUE' : 'FALSE';
}
override booleanValue(value: boolean): boolean {
return value;
}
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.replicaPool, query, params);
return normalizePostgresRows(result);
}
override async pread(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.primaryPool, query, params);
return normalizePostgresRows(result);
}
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
const result = await this.query(this.primaryPool, query, params);
return mapPostgresWriteResult(result);
}
override async batchWrite(entries: BatchEntry[]): Promise<void> {
if (entries.length === 0) return;
const conn = await this.primaryPool.connect();
try {
await conn.query('BEGIN');
try {
for (const { statement, values } of entries) {
await this.query(conn, statement, values);
}
await conn.query('COMMIT');
} catch (err) {
await conn.query('ROLLBACK').catch(() => {});
throw err;
}
} finally {
conn.release();
}
}
override async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
if (this.configuration === Configuration.SINGLE) {
return this.read(query, params);
}
const primaryPromise = this.query(this.primaryPool, query, params);
try {
const replicaResult = await this.query(
this.replicaPool,
query,
params,
);
if (replicaResult.rows.length > 0) {
primaryPromise.catch(() => {});
return normalizePostgresRows(replicaResult);
}
} catch {
// fall through to primary
}
const primaryResult = await primaryPromise;
return normalizePostgresRows(primaryResult);
}
private async runMigrations(): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const conn = await this.primaryPool.connect();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) =>
f.endsWith('.sql') && f.startsWith('postgres'),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[postgres] migration path is unreadable: ${dir}`,
{ cause: e },
);
}
if (files.length === 0) {
console.log(`[postgres] no migrations in ${dir}`);
continue;
}
console.log(
`[postgres] running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const filePath = resolvePath(dir, file);
const contents = readFileSync(filePath, 'utf8');
const statements = splitPostgresStatements(contents);
await conn.query('BEGIN');
try {
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`[postgres] failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
await conn.query('COMMIT');
} catch (e) {
await conn.query('ROLLBACK').catch(() => {});
throw e;
}
console.log(
`[postgres] applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
}
private createPool(dbConf: PostgresEndpointConfig): PostgresPool {
const connectionString = dbConf.connectionString ?? dbConf.url;
if (connectionString) {
return this.poolFactory({
connectionString,
max: 30,
});
}
return this.poolFactory({
host: dbConf.host ?? '127.0.0.1',
port: dbConf.port ?? 5432,
user: dbConf.user ?? 'postgres',
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
max: 30,
});
}
private async query(
target: PostgresQueryable,
query: string,
params: unknown[] = [],
): Promise<QueryResult> {
const prepared = preparePostgresSql(query);
return target.query(prepared.text, params);
}
private async closeCurrentPools(): Promise<void> {
const tasks: Promise<void>[] = [];
if (this.primaryPool) tasks.push(this.primaryPool.end());
if (this.replicaPool && this.replicaPool !== this.primaryPool) {
tasks.push(this.replicaPool.end());
}
await Promise.all(tasks);
}
}
@@ -78,6 +78,13 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[43, ['0047_app-url-updates.sql']],
[44, ['0048_old-app-names-unique-tuple.sql']],
[45, ['0049_music-player-pdf-player-updates.sql']],
[46, ['0050_add_preamble_version.sql']],
[47, ['0051_sessions_v2.sql']],
[48, ['0052_sessions_v2_lookups.sql']],
[49, ['0053_sessions_access_token_uid.sql']],
[50, ['0054_sessions_workers.sql']],
[50, ['0055_username_nocase_unique.sql']],
[51, ['0056_sessions_kind_worker.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
+4
View File
@@ -24,10 +24,12 @@ export {
} from './DatabaseClient';
export { SqliteDatabaseClient } from './SqliteDatabaseClient';
export { MySQLDatabaseClient } from './MySQLDatabaseClient';
export { PostgresDatabaseClient } from './PostgresDatabaseClient';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient } from './DatabaseClient';
import { MySQLDatabaseClient } from './MySQLDatabaseClient';
import { PostgresDatabaseClient } from './PostgresDatabaseClient';
import { SqliteDatabaseClient } from './SqliteDatabaseClient';
/**
@@ -41,6 +43,8 @@ export const DatabaseClientFactory = class DatabaseClientFactory {
switch (engine) {
case 'mysql':
return new MySQLDatabaseClient(config);
case 'postgres':
return new PostgresDatabaseClient(config);
case 'sqlite':
return new SqliteDatabaseClient(config);
default:
@@ -0,0 +1,43 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Comparator for migration filenames named `<engine>_mig_<N>.sql`.
*
* Existing MySQL files use unpadded integers, so lexical sorting places
* `*_mig_10.sql` before `*_mig_2.sql`. Pull the trailing integer out and sort
* numerically. Anything that does not match the `_<digits>.sql` shape falls
* back to lexical comparison and sorts after numbered files.
*/
export const compareMigrationFilenames = (a: string, b: string): number => {
const numericIndex = (name: string): number => {
const m = /_(\d+)\.sql$/.exec(name);
return m ? Number.parseInt(m[1], 10) : Number.NaN;
};
const na = numericIndex(a);
const nb = numericIndex(b);
if (Number.isFinite(na) && Number.isFinite(nb)) {
if (na !== nb) return na - nb;
} else if (Number.isFinite(na)) {
return -1;
} else if (Number.isFinite(nb)) {
return 1;
}
return a.localeCompare(b);
};
@@ -0,0 +1,53 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0053. Adds the
-- `access_token_uid` reverse-lookup column on `sessions` so raw-uuid
-- revoke can find the matching session row when only the v2 token_uid
-- (no JWT) is presented.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_access_token_uid;
DELIMITER //
CREATE PROCEDURE _puter_sessions_access_token_uid()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'access_token_uid'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `access_token_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_access_token_uid'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_access_token_uid` (`access_token_uid`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_access_token_uid();
DROP PROCEDURE IF EXISTS _puter_sessions_access_token_uid;
@@ -0,0 +1,70 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Worker session uniqueness. Each Puter worker is a separately-deployed
-- code unit (its own subdomain), so one user can have many workers under
-- the same app — distinguished by `meta.worker_name`. The natural unique
-- key for an active worker session is therefore
-- (user_id, app_uid, worker_name), and app_uid is allowed NULL for
-- user-scoped workers that aren't bound to any specific app.
--
-- Implemented the same way as `app_unique_key` from mig_9: a VIRTUAL
-- generated column that's non-NULL only for the rows under the rule,
-- then a UNIQUE INDEX on the column. NULLs don't conflict in MySQL
-- UNIQUE indexes, so soft-revoked / non-worker rows fall out
-- automatically. IFNULL normalises NULL `app_uid` so two user-scoped
-- workers with the same name still dedupe. JSON_UNQUOTE strips the
-- JSON value quoting from JSON_EXTRACT so the concatenated key is a
-- plain string that matches the SELECT path's binding.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_worker_unique;
DELIMITER //
CREATE PROCEDURE _puter_sessions_worker_unique()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'worker_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `worker_unique_key` VARCHAR(550)
GENERATED ALWAYS AS (
IF(`kind` = 'worker' AND `revoked_at` IS NULL,
CONCAT(`user_id`, '|', IFNULL(`app_uid`, ''), '|',
IFNULL(JSON_UNQUOTE(JSON_EXTRACT(`meta`, '$.worker_name')), '')),
NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_worker_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_user_worker_active` (`worker_unique_key`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_worker_unique();
DROP PROCEDURE IF EXISTS _puter_sessions_worker_unique;
@@ -0,0 +1,50 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Add 'worker' to the `sessions.kind` ENUM. mig_11 already added the
-- worker uniqueness index and the application code in SessionStore
-- inserts rows with kind='worker', but the ENUM defined in mig_8 never
-- listed 'worker' as a permitted value. Under STRICT_TRANS_TABLES the
-- INSERT fails outright; under a relaxed sql_mode the value is coerced
-- to '' (and the worker SELECT-by-kind path then misses the row).
-- Mirrors SQLite migration 0056.
--
-- Idempotent: guarded against COLUMN_TYPE so re-running the directory is
-- a no-op once 'worker' is in the ENUM.
DROP PROCEDURE IF EXISTS _puter_sessions_kind_worker;
DELIMITER //
CREATE PROCEDURE _puter_sessions_kind_worker()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND COLUMN_NAME = 'kind'
AND FIND_IN_SET('worker', REPLACE(REPLACE(REPLACE(COLUMN_TYPE, 'enum(', ''), ')', ''), '''', '')) > 0
) THEN
ALTER TABLE `sessions`
MODIFY COLUMN `kind`
ENUM('web', 'app', 'access_token', 'asset', 'worker')
NOT NULL DEFAULT 'web';
END IF;
END//
DELIMITER ;
CALL _puter_sessions_kind_worker();
DROP PROCEDURE IF EXISTS _puter_sessions_kind_worker;
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
--
-- Idempotent: the ADD COLUMN is guarded by an INFORMATION_SCHEMA check,
-- so re-running the migration directory is safe (required — the runner
-- has no per-file tracking).
DROP PROCEDURE IF EXISTS _puter_add_subdomains_preamble_version;
DELIMITER //
CREATE PROCEDURE _puter_add_subdomains_preamble_version()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'subdomains' AND COLUMN_NAME = 'preamble_version'
) THEN
ALTER TABLE `subdomains`
ADD COLUMN `preamble_version` varchar(64) DEFAULT NULL;
END IF;
END//
DELIMITER ;
CALL _puter_add_subdomains_preamble_version();
DROP PROCEDURE IF EXISTS _puter_add_subdomains_preamble_version;
@@ -0,0 +1,98 @@
-- Extend `sessions` so a single row can represent any token kind
-- (web/app/access_token/asset), carry display metadata for the
-- manage-sessions UI, and be soft-revoked with row-level expiry.
-- Mirrors SQLite migration 0050.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded by an
-- INFORMATION_SCHEMA check, so re-running the migration directory
-- is safe (required — the runner has no per-file tracking).
DROP PROCEDURE IF EXISTS _puter_extend_sessions_v2;
DELIMITER //
CREATE PROCEDURE _puter_extend_sessions_v2()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'kind'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `kind` ENUM('web', 'app', 'access_token', 'asset')
NOT NULL DEFAULT 'web';
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'label'
) THEN
ALTER TABLE `sessions` ADD COLUMN `label` VARCHAR(255) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'parent_session_id'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `parent_session_id` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'last_ip'
) THEN
ALTER TABLE `sessions` ADD COLUMN `last_ip` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'last_user_agent'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `last_user_agent` VARCHAR(512) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'revoked_at'
) THEN
ALTER TABLE `sessions` ADD COLUMN `revoked_at` BIGINT DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'expires_at'
) THEN
ALTER TABLE `sessions` ADD COLUMN `expires_at` BIGINT DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_revoked'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_user_revoked` (`user_id`, `revoked_at`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_parent'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_parent` (`parent_session_id`);
END IF;
END//
DELIMITER ;
CALL _puter_extend_sessions_v2();
DROP PROCEDURE IF EXISTS _puter_extend_sessions_v2;
@@ -0,0 +1,130 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Composite-key lookups + audit columns. Mirrors
-- SQLite migration 0052. MySQL has no partial unique indexes, so the
-- "at most one active row per (user_id, app_uid)" / "one active row
-- per legacy_token_uid" semantics are encoded via VIRTUAL generated
-- columns that are NULL when the row isn't subject to the rule —
-- MySQL allows multiple NULLs in a UNIQUE index, so non-applicable
-- rows don't conflict.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_v2_lookups;
DELIMITER //
CREATE PROCEDURE _puter_sessions_v2_lookups()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'app_uid'
) THEN
ALTER TABLE `sessions` ADD COLUMN `app_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'legacy_token_uid'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `legacy_token_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'created_via'
) THEN
ALTER TABLE `sessions` ADD COLUMN `created_via` VARCHAR(32) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'auth_id'
) THEN
ALTER TABLE `sessions` ADD COLUMN `auth_id` VARCHAR(64) DEFAULT NULL;
END IF;
-- Generated discriminant: non-NULL only for active app-authorization rows,
-- so UNIQUE(app_unique_key) enforces "one active app session per
-- (user_id, app_uid)" while permitting any number of revoked rows.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'app_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `app_unique_key` VARCHAR(150)
GENERATED ALWAYS AS (
IF(`kind` = 'app' AND `revoked_at` IS NULL,
CONCAT(`user_id`, '|', `app_uid`),
NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'legacy_token_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `legacy_token_unique_key` VARCHAR(64)
GENERATED ALWAYS AS (
IF(`revoked_at` IS NULL, `legacy_token_uid`, NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_app_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_user_app_active` (`app_unique_key`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_legacy_token_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_legacy_token_active`
(`legacy_token_unique_key`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_kind_user'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_kind_user` (`kind`, `user_id`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_v2_lookups();
DROP PROCEDURE IF EXISTS _puter_sessions_v2_lookups;
@@ -0,0 +1,724 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
CREATE TABLE IF NOT EXISTS "user" (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(36) NOT NULL UNIQUE,
username varchar(50) UNIQUE,
email varchar(256),
password varchar(225),
free_storage bigint,
max_subdomains integer,
taskbar_items text,
desktop_uuid varchar(36),
appdata_uuid varchar(36),
documents_uuid varchar(36),
pictures_uuid varchar(36),
videos_uuid varchar(36),
trash_uuid varchar(36),
trash_id integer,
appdata_id integer,
desktop_id integer,
documents_id integer,
pictures_id integer,
videos_id integer,
referrer varchar(64),
desktop_bg_url text,
desktop_bg_color varchar(20),
desktop_bg_fit varchar(16),
pass_recovery_token varchar(36),
requires_email_confirmation boolean NOT NULL DEFAULT FALSE,
email_confirm_code varchar(8),
email_confirm_token varchar(36),
email_confirmed boolean NOT NULL DEFAULT FALSE,
dev_first_name varchar(100),
dev_last_name varchar(100),
dev_paypal varchar(100),
dev_approved_for_incentive_program boolean DEFAULT FALSE,
dev_joined_incentive_program boolean DEFAULT FALSE,
suspended boolean,
unsubscribed boolean NOT NULL DEFAULT FALSE,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_activity_ts timestamp,
referral_code varchar(16) UNIQUE,
referred_by integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
unconfirmed_change_email varchar(256),
change_email_confirm_token varchar(256),
otp_secret text,
otp_enabled boolean DEFAULT FALSE,
otp_recovery_codes text,
stripe_customer_id varchar(40),
public_uuid varchar(36),
public_id integer,
clean_email varchar(256),
audit_metadata jsonb,
signup_ip varchar(45),
signup_ip_forwarded varchar(45),
signup_user_agent varchar(512),
signup_origin varchar(255),
signup_server varchar(255),
metadata jsonb DEFAULT '{}'::jsonb,
reputation smallint DEFAULT 100
);
CREATE INDEX IF NOT EXISTS idx_user_email ON "user" (email);
CREATE INDEX IF NOT EXISTS idx_user_pass_recovery_token ON "user" (pass_recovery_token);
CREATE INDEX IF NOT EXISTS idx_user_referrer ON "user" (referrer);
CREATE INDEX IF NOT EXISTS idx_user_email_confirm_token ON "user" (email_confirm_token);
CREATE INDEX IF NOT EXISTS idx_user_last_activity_ts ON "user" (last_activity_ts);
CREATE INDEX IF NOT EXISTS idx_user_referred_by ON "user" (referred_by);
CREATE INDEX IF NOT EXISTS idx_user_stripe_customer_id ON "user" (stripe_customer_id);
CREATE INDEX IF NOT EXISTS idx_user_clean_email ON "user" (clean_email);
CREATE INDEX IF NOT EXISTS idx_user_signup_ip ON "user" (signup_ip);
CREATE INDEX IF NOT EXISTS idx_user_signup_ip_forwarded ON "user" (signup_ip_forwarded);
CREATE INDEX IF NOT EXISTS idx_user_signup_user_agent ON "user" (signup_user_agent);
CREATE INDEX IF NOT EXISTS idx_user_signup_origin ON "user" (signup_origin);
CREATE INDEX IF NOT EXISTS idx_user_signup_server ON "user" (signup_server);
CREATE TABLE IF NOT EXISTS apps (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
owner_user_id integer REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
icon text,
name varchar(100) NOT NULL UNIQUE,
title varchar(100) NOT NULL,
description text,
godmode boolean DEFAULT FALSE,
maximize_on_start boolean DEFAULT FALSE,
index_url text NOT NULL,
approved_for_listing boolean DEFAULT FALSE,
approved_for_opening_items boolean DEFAULT FALSE,
approved_for_incentive_program boolean DEFAULT FALSE,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_review timestamp,
tags varchar(255),
app_owner integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
background boolean DEFAULT FALSE,
metadata jsonb,
protected boolean DEFAULT FALSE,
is_private boolean DEFAULT FALSE
);
CREATE INDEX IF NOT EXISTS idx_apps_owner_user_id ON apps (owner_user_id);
CREATE INDEX IF NOT EXISTS idx_apps_app_owner ON apps (app_owner);
CREATE INDEX IF NOT EXISTS idx_apps_owner_timestamp ON apps (owner_user_id, "timestamp" DESC);
CREATE INDEX IF NOT EXISTS idx_apps_listing_timestamp ON apps (approved_for_listing, "timestamp" DESC);
CREATE TABLE IF NOT EXISTS "group" (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) UNIQUE,
owner_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
extra jsonb,
metadata jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_group_owner_user_id ON "group" (owner_user_id);
CREATE TABLE IF NOT EXISTS app_filetype_association (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
type varchar(60) NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_app_filetype_association_app_id ON app_filetype_association (app_id);
CREATE INDEX IF NOT EXISTS idx_app_filetype_association_type ON app_filetype_association (type);
CREATE TABLE IF NOT EXISTS app_opens (
_id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_uid varchar(40) NOT NULL REFERENCES apps (uid) ON DELETE CASCADE ON UPDATE CASCADE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
ts integer NOT NULL,
human_ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_app_opens_user_id ON app_opens (user_id);
CREATE INDEX IF NOT EXISTS idx_app_opens_app_uid ON app_opens (app_uid);
CREATE INDEX IF NOT EXISTS idx_app_opens_uid_ts ON app_opens (app_uid, ts);
CREATE INDEX IF NOT EXISTS idx_app_opens_app_user ON app_opens (app_uid, user_id);
CREATE TABLE IF NOT EXISTS fsentries (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(36) NOT NULL UNIQUE,
bucket varchar(50),
bucket_region varchar(30),
public_token varchar(36) UNIQUE,
file_request_token varchar(36) UNIQUE,
is_shortcut boolean DEFAULT FALSE,
shortcut_to integer REFERENCES fsentries (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
parent_id integer REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
associated_app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
is_dir boolean DEFAULT FALSE,
layout varchar(30),
sort_by varchar(20),
sort_order varchar(10),
is_public boolean,
thumbnail text,
immutable boolean NOT NULL DEFAULT FALSE,
name varchar(767) NOT NULL,
metadata text,
modified integer NOT NULL,
created integer,
accessed integer,
size bigint,
symlink_path varchar(260),
is_symlink boolean DEFAULT FALSE,
parent_uid varchar(36),
path varchar(4096),
CONSTRAINT fsentries_parent_name_unique UNIQUE (parent_id, name)
);
CREATE INDEX IF NOT EXISTS idx_fsentries_name ON fsentries (name);
CREATE INDEX IF NOT EXISTS idx_fsentries_modified ON fsentries (modified);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_id ON fsentries (parent_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_is_dir ON fsentries (is_dir);
CREATE INDEX IF NOT EXISTS idx_fsentries_user_id ON fsentries (user_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_shortcut_to ON fsentries (shortcut_to);
CREATE INDEX IF NOT EXISTS idx_fsentries_associated_app_id ON fsentries (associated_app_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_bucket ON fsentries (bucket);
CREATE INDEX IF NOT EXISTS idx_fsentries_bucket_region ON fsentries (bucket_region);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_uid ON fsentries (parent_uid);
CREATE INDEX IF NOT EXISTS idx_fsentries_path ON fsentries (path);
CREATE INDEX IF NOT EXISTS idx_fsentries_accessed ON fsentries (accessed);
CREATE INDEX IF NOT EXISTS idx_fsentries_user_parent_name ON fsentries (user_id, parent_uid, name);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_uid_name ON fsentries (parent_uid, name);
CREATE TABLE IF NOT EXISTS fsentry_versions (
id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
fsentry_id integer NOT NULL REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_uuid varchar(36) NOT NULL,
version_id varchar(60) NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
message text,
ts_epoch integer
);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_fsentry_id ON fsentry_versions (fsentry_id);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_fsentry_uuid ON fsentry_versions (fsentry_uuid);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_user_id ON fsentry_versions (user_id);
CREATE TABLE IF NOT EXISTS subdomains (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(40) UNIQUE,
subdomain varchar(64) NOT NULL UNIQUE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
root_dir_id integer REFERENCES fsentries (id) ON DELETE SET NULL ON UPDATE CASCADE,
associated_app_id integer REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
ts timestamp DEFAULT CURRENT_TIMESTAMP,
app_owner integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
protected boolean DEFAULT FALSE,
domain varchar(265),
database_id varchar(40),
preamble_version varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_subdomains_user_id ON subdomains (user_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_root_dir_id ON subdomains (root_dir_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_associated_app_id ON subdomains (associated_app_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_app_owner ON subdomains (app_owner);
CREATE INDEX IF NOT EXISTS idx_subdomains_domain ON subdomains (domain);
CREATE INDEX IF NOT EXISTS idx_subdomains_root_user ON subdomains (root_dir_id, user_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_app_user ON subdomains (associated_app_id, user_id);
CREATE TABLE IF NOT EXISTS kv (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app varchar(40),
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
kkey_hash bigint NOT NULL,
kkey text NOT NULL,
value text,
migrated boolean DEFAULT FALSE,
"expireAt" timestamp,
CONSTRAINT kv_app_user_kkey_hash_unique UNIQUE (app, user_id, kkey_hash)
);
CREATE INDEX IF NOT EXISTS idx_kv_app ON kv (app);
CREATE INDEX IF NOT EXISTS idx_kv_user_id ON kv (user_id);
CREATE INDEX IF NOT EXISTS idx_kv_kkey_hash ON kv (kkey_hash);
CREATE TABLE IF NOT EXISTS sessions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
uuid varchar(40) NOT NULL,
meta jsonb,
created_at bigint DEFAULT 0,
last_activity bigint DEFAULT 0,
kind varchar(32) NOT NULL DEFAULT 'web'
CHECK (kind IN ('web', 'app', 'access_token', 'asset', 'worker')),
label varchar(255),
parent_session_id varchar(64),
last_ip varchar(64),
last_user_agent varchar(512),
revoked_at bigint,
expires_at bigint,
app_uid varchar(64),
legacy_token_uid varchar(64),
created_via varchar(32),
auth_id varchar(64),
access_token_uid varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions (user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_uuid ON sessions (uuid);
CREATE INDEX IF NOT EXISTS idx_sessions_user_revoked ON sessions (user_id, revoked_at);
CREATE INDEX IF NOT EXISTS idx_sessions_parent ON sessions (parent_session_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_user_app_active
ON sessions (user_id, app_uid)
WHERE kind = 'app' AND revoked_at IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_legacy_token_active
ON sessions (legacy_token_uid)
WHERE legacy_token_uid IS NOT NULL AND revoked_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_sessions_kind_user ON sessions (kind, user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_access_token_uid
ON sessions (access_token_uid)
WHERE access_token_uid IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_user_worker_active
ON sessions (user_id, COALESCE(app_uid, ''), (meta #>> ARRAY['worker_name']))
WHERE kind = 'worker' AND revoked_at IS NULL;
CREATE TABLE IF NOT EXISTS user_to_app_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
dt timestamp DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, app_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_utap_user_permission ON user_to_app_permissions (user_id, permission);
CREATE INDEX IF NOT EXISTS idx_utap_app_permission ON user_to_app_permissions (app_id, permission);
CREATE TABLE IF NOT EXISTS user_to_group_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (user_id, group_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_user_to_group_permissions_group_id ON user_to_group_permissions (group_id);
CREATE TABLE IF NOT EXISTS user_to_user_permissions (
issuer_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
holder_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (issuer_user_id, holder_user_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_user_to_user_permissions_holder_user_id ON user_to_user_permissions (holder_user_id);
CREATE TABLE IF NOT EXISTS dev_to_app_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (user_id, app_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_app_id ON dev_to_app_permissions (app_id);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_permission ON dev_to_app_permissions (permission);
CREATE TABLE IF NOT EXISTS access_token_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
token_uid varchar(40) NOT NULL,
authorizer_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
authorizer_app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_access_token_permissions_token_uid ON access_token_permissions (token_uid);
CREATE TABLE IF NOT EXISTS old_app_names (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_uid varchar(40) NOT NULL REFERENCES apps (uid) ON DELETE CASCADE,
name varchar(255) NOT NULL,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT unique_old_app_names_app_uid_name UNIQUE (app_uid, name)
);
CREATE INDEX IF NOT EXISTS idx_old_app_names_app_uid ON old_app_names (app_uid);
CREATE INDEX IF NOT EXISTS idx_old_app_names_name ON old_app_names (name);
CREATE TABLE IF NOT EXISTS jct_user_group (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE,
extra jsonb,
metadata jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_jct_user_group_user_id ON jct_user_group (user_id);
CREATE INDEX IF NOT EXISTS idx_jct_user_group_group_id ON jct_user_group (group_id);
CREATE TABLE IF NOT EXISTS notification (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
uid varchar(40) UNIQUE,
value jsonb NOT NULL,
acknowledged bigint,
shown bigint,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_notification_user_id ON notification (user_id);
CREATE TABLE IF NOT EXISTS share (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) UNIQUE,
issuer_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
recipient_email varchar(255) NOT NULL,
data jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_share_issuer_user_id ON share (issuer_user_id);
CREATE INDEX IF NOT EXISTS idx_share_recipient_email ON share (recipient_email);
CREATE TABLE IF NOT EXISTS feedback (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
message text,
ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_feedback_user_id ON feedback (user_id);
CREATE TABLE IF NOT EXISTS thread (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
parent_uid varchar(40) REFERENCES thread (uid) ON DELETE CASCADE ON UPDATE CASCADE,
owner_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
"schema" text,
text text NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_thread_parent_uid ON thread (parent_uid);
CREATE INDEX IF NOT EXISTS idx_thread_owner_user_id ON thread (owner_user_id);
CREATE TABLE IF NOT EXISTS user_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
metadata jsonb,
text text NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_user_comments_user_id ON user_comments (user_id);
CREATE INDEX IF NOT EXISTS idx_user_comments_uid ON user_comments (uid);
CREATE TABLE IF NOT EXISTS user_fsentry_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_id integer NOT NULL REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_comments_user_comment_id ON user_fsentry_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_comments_fsentry_id ON user_fsentry_comments (fsentry_id);
CREATE TABLE IF NOT EXISTS user_fsentry_version_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_version_id integer NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_version_comments_user_comment_id ON user_fsentry_version_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_version_comments_version_id ON user_fsentry_version_comments (fsentry_version_id);
CREATE TABLE IF NOT EXISTS user_group_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_group_comments_user_comment_id ON user_group_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_group_comments_group_id ON user_group_comments (group_id);
CREATE TABLE IF NOT EXISTS user_user_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
commented_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_user_comments_user_comment_id ON user_user_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_user_comments_commented_user_id ON user_user_comments (commented_user_id);
CREATE TABLE IF NOT EXISTS user_oidc_providers (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
provider varchar(64) NOT NULL,
provider_sub varchar(255) NOT NULL,
refresh_token text,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_user_oidc_providers_user_id ON user_oidc_providers (user_id);
CREATE INDEX IF NOT EXISTS idx_user_oidc_providers_provider ON user_oidc_providers (provider);
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_providers_provider_sub_unique ON user_oidc_providers (provider, provider_sub);
CREATE TABLE IF NOT EXISTS app_update_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
old_name varchar(50),
new_name varchar(50),
reason varchar(255)
);
CREATE INDEX IF NOT EXISTS idx_app_update_audit_app_id ON app_update_audit (app_id);
CREATE TABLE IF NOT EXISTS user_update_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
old_email varchar(256),
new_email varchar(256),
old_username varchar(50),
new_username varchar(50),
reason varchar(255)
);
CREATE INDEX IF NOT EXISTS idx_user_update_audit_user_id ON user_update_audit (user_id);
CREATE TABLE IF NOT EXISTS storage_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
is_subtract boolean NOT NULL DEFAULT FALSE,
amount bigint NOT NULL,
field_a varchar(16),
field_b varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_storage_audit_user_id ON storage_audit (user_id);
CREATE TABLE IF NOT EXISTS audit_user_to_app_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_app_permissions_user_id ON audit_user_to_app_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_app_permissions_app_id ON audit_user_to_app_permissions (app_id);
CREATE TABLE IF NOT EXISTS audit_dev_to_app_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_dev_to_app_permissions_user_id ON audit_dev_to_app_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_dev_to_app_permissions_app_id ON audit_dev_to_app_permissions (app_id);
CREATE TABLE IF NOT EXISTS audit_user_to_group_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
group_id integer REFERENCES "group" (id) ON DELETE SET NULL ON UPDATE CASCADE,
group_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(255),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_group_permissions_user_id ON audit_user_to_group_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_group_permissions_group_id ON audit_user_to_group_permissions (group_id);
CREATE TABLE IF NOT EXISTS audit_user_to_user_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
issuer_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
issuer_user_id_keep integer NOT NULL,
holder_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
holder_user_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_user_permissions_issuer_user_id ON audit_user_to_user_permissions (issuer_user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_user_permissions_holder_user_id ON audit_user_to_user_permissions (holder_user_id);
CREATE TABLE IF NOT EXISTS ai_usage (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
service_name varchar(64),
model_name varchar(128),
price_modifier varchar(40),
cost integer,
value_uint_1 integer,
value_uint_2 integer,
value_uint_3 integer,
value_uint_4 integer,
value_uint_5 integer,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_ai_usage_app_id ON ai_usage (app_id);
CREATE INDEX IF NOT EXISTS idx_ai_usage_service_name ON ai_usage (service_name);
CREATE INDEX IF NOT EXISTS idx_ai_usage_model_name ON ai_usage (model_name);
CREATE INDEX IF NOT EXISTS idx_ai_usage_price_modifier ON ai_usage (price_modifier);
CREATE INDEX IF NOT EXISTS idx_ai_usage_created_at ON ai_usage (created_at);
CREATE INDEX IF NOT EXISTS idx_ai_usage_user_timestamp ON ai_usage (user_id, created_at);
CREATE TABLE IF NOT EXISTS general_analytics (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
uid varchar(40) NOT NULL,
trace_id varchar(40),
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer,
server_id varchar(40),
actor_type varchar(40),
tags jsonb,
fields jsonb
);
CREATE INDEX IF NOT EXISTS idx_general_analytics_user_id ON general_analytics (user_id);
CREATE INDEX IF NOT EXISTS idx_general_analytics_app_id ON general_analytics (app_id);
CREATE TABLE IF NOT EXISTS monthly_usage_counts (
year integer NOT NULL,
month integer NOT NULL,
service_type varchar(40) NOT NULL,
service_name varchar(40) NOT NULL,
actor_key varchar(255) NOT NULL,
pricing_category jsonb NOT NULL,
pricing_category_hash bytea NOT NULL,
"count" integer DEFAULT 0,
value_uint_1 integer,
value_uint_2 integer,
value_uint_3 integer,
PRIMARY KEY (year, month, service_type, service_name, actor_key, pricing_category_hash)
);
CREATE TABLE IF NOT EXISTS service_usage_monthly (
"key" varchar(255) NOT NULL,
year integer NOT NULL,
month integer NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
"count" integer NOT NULL,
extra jsonb,
PRIMARY KEY ("key", year, month)
);
CREATE INDEX IF NOT EXISTS idx_service_usage_monthly_user_id ON service_usage_monthly (user_id);
CREATE INDEX IF NOT EXISTS idx_service_usage_monthly_app_id ON service_usage_monthly (app_id);
CREATE TABLE IF NOT EXISTS per_user_credit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL UNIQUE REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
amount bigint NOT NULL,
last_updated_at bigint NOT NULL
);
INSERT INTO "user" (id, uuid, username, metadata)
VALUES (1, '5d4adce0-a381-4982-9c02-6e2540026238', 'system', '{}'::jsonb)
ON CONFLICT (uuid) DO UPDATE SET username = EXCLUDED.username;
SELECT setval(pg_get_serial_sequence('"user"', 'id'), COALESCE((SELECT MAX(id) FROM "user"), 1), true);
INSERT INTO "group" (uid, owner_user_id, extra, metadata)
VALUES
('26bfb1fb-421f-45bc-9aa4-d81ea569e7a5', 1, '{"critical": true, "type": "default", "name": "system"}'::jsonb, '{"title": "System", "color": "#000000"}'::jsonb),
('ca342a5e-b13d-4dee-9048-58b11a57cc55', 1, '{"critical": true, "type": "default", "name": "admin"}'::jsonb, '{"title": "Admin", "color": "#a83232"}'::jsonb),
('78b1b1dd-c959-44d2-b02c-8735671f9997', 1, '{"critical": true, "type": "default", "name": "user"}'::jsonb, '{"title": "User", "color": "#3254a8"}'::jsonb),
('b7220104-7905-4985-b996-649fdcdb3c8f', 1, '{"critical": true, "type": "default", "name": "temp"}'::jsonb, '{"title": "Temp", "color": "#888888"}'::jsonb),
('3c2dfff7-d22a-41aa-a193-59a61dac4b64', 1, '{"type": "default", "name": "moderator"}'::jsonb, '{"title": "Moderator", "color": "#a432a8"}'::jsonb),
('5e8f251d-3382-4b0d-932c-7bb82f48652f', 1, '{"type": "default", "name": "developer"}'::jsonb, '{"title": "Developer", "color": "#32a852"}'::jsonb)
ON CONFLICT (uid) DO UPDATE SET
owner_user_id = EXCLUDED.owner_user_id,
extra = EXCLUDED.extra,
metadata = EXCLUDED.metadata;
INSERT INTO apps (
uid, owner_user_id, icon, name, title, description, index_url,
godmode, maximize_on_start, background,
approved_for_listing, approved_for_opening_items,
approved_for_incentive_program, tags, "timestamp"
)
VALUES
('app-3920851d-bda8-479b-9407-8517293c7d44', 1, NULL, 'pdf', 'PDF', '', 'https://pdf.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-5584fbf7-ed69-41fc-99cd-85da21b1ef51', 1, NULL, 'camera', 'Camera', 'Camera in the browser.', 'https://online-camera.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-11edfba2-1ed3-4e22-8573-47e88fb87d70', 1, NULL, 'player', 'Player', 'A free video player app in the browser.', 'https://simple-player.puter.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-7bdca1a4-6373-4c98-ad97-03ff2d608ca1', 1, NULL, 'recorder', 'Recorder', 'Online voice recorder in the browser with cloud storage.', 'https://voice-recorder.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-d7e9471f-e441-4d72-a5ab-75e96573b76b', 1, NULL, 'music-player', 'Music Player', 'A free music player app in the browser.', 'https://player.puter.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2026-05-10 00:00:00'),
('app-e3ac5486-da8c-42ad-8377-8728086e0980', 1, NULL, 'git', 'Git', 'Puter Git client', 'https://builtins.namespaces.puter.com/git', FALSE, FALSE, TRUE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-0b37f054-07d4-4627-8765-11bd23e889d4', 1, NULL, 'dev-center', 'Dev Center', 'This is the app that makes apps', 'https://builtins.namespaces.puter.com/dev-center', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-fbbdb72b-ad08-4cb4-86a1-de0f27cf2e1e', 1, NULL, 'puter-linux', 'Puter Linux', 'Linux emulator for Puter', 'https://builtins.namespaces.puter.com/emulator', TRUE, FALSE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-838dfbc4-bf8b-48c2-b47b-c4adc77fab58', 1, NULL, 'editor', 'Editor', 'Text editor', 'https://online-notepad.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-58282b08-990a-4906-95f7-fa37ff92452b', 1, NULL, 'draw', 'Draw', 'Image editor', 'https://draw.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'graphics', '2020-01-01 00:00:00'),
('app-0bef044f-918f-4cbf-a0c0-b4a17ee81085', 1, NULL, 'about', 'About', 'About Puter', 'https://about.puter.com', TRUE, FALSE, FALSE, FALSE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-a2ae72a4-1ba3-4a29-b5c0-6de1be5cf178', 1, NULL, 'app-center', 'App Center', 'Discover apps for Puter', 'https://app-center.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-93005ce0-80d1-50d9-9b1e-9c453c375d56', 1, NULL, 'markus', 'Markus', 'Markdown editor', 'https://markus.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-6f79ef7b-52b7-4b31-91c6-fc07b62e9396', 1, NULL, 'code', 'Code', 'Code editor', 'https://code.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-862fc09e-c7b8-4c30-b5b4-47b3cc5f5232', 1, NULL, 'memos', 'Memos', 'Notes and memos', 'https://memos.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-aeb8c03e-1144-4c57-bff5-3a1a7c17f9f0', 1, NULL, 'word-processor', 'Word Processor', 'Write documents', 'https://word-processor.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-c10a1999-22f5-4644-9960-9aaac0d4934e', 1, NULL, 'spreadsheet', 'Spreadsheet', 'Work with spreadsheets', 'https://spreadsheet.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-e9c1d58d-3d8d-4f0a-a85c-e383ef63bc29', 1, NULL, 'presentation', 'Presentation', 'Create presentations', 'https://presentation.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-fb7d9e42-8207-4fa0-b680-f0239327097f', 1, NULL, 'pdf-editor', 'PDF Editor', 'Edit PDF documents', 'https://pdf-editor.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00')
ON CONFLICT (uid) DO UPDATE SET
owner_user_id = EXCLUDED.owner_user_id,
name = EXCLUDED.name,
title = EXCLUDED.title,
description = EXCLUDED.description,
index_url = EXCLUDED.index_url,
godmode = EXCLUDED.godmode,
maximize_on_start = EXCLUDED.maximize_on_start,
background = EXCLUDED.background,
approved_for_listing = EXCLUDED.approved_for_listing,
approved_for_opening_items = EXCLUDED.approved_for_opening_items,
approved_for_incentive_program = EXCLUDED.approved_for_incentive_program,
tags = EXCLUDED.tags;
INSERT INTO user_to_group_permissions (user_id, group_id, permission, extra)
SELECT u.id, g.id, 'driver', '{}'::jsonb
FROM "user" u, "group" g
WHERE u.username = 'system'
AND g.uid = 'ca342a5e-b13d-4dee-9048-58b11a57cc55'
ON CONFLICT (user_id, group_id, permission) DO UPDATE SET extra = EXCLUDED.extra;
@@ -0,0 +1,18 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
ALTER TABLE `subdomains` ADD COLUMN `preamble_version` varchar(64) DEFAULT NULL;
@@ -0,0 +1,34 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Extend `sessions` so a single row can represent any token kind
-- (web/app/access_token/asset), carry display metadata for the
-- manage-sessions UI, and be soft-revoked.
ALTER TABLE `sessions` ADD COLUMN `kind` TEXT NOT NULL DEFAULT 'web'
CHECK (`kind` IN ('web', 'app', 'access_token', 'asset'));
ALTER TABLE `sessions` ADD COLUMN `label` TEXT;
ALTER TABLE `sessions` ADD COLUMN `parent_session_id` TEXT;
ALTER TABLE `sessions` ADD COLUMN `last_ip` TEXT;
ALTER TABLE `sessions` ADD COLUMN `last_user_agent` TEXT;
ALTER TABLE `sessions` ADD COLUMN `revoked_at` INTEGER;
ALTER TABLE `sessions` ADD COLUMN `expires_at` INTEGER;
CREATE INDEX IF NOT EXISTS `idx_sessions_user_revoked`
ON `sessions` (`user_id`, `revoked_at`);
CREATE INDEX IF NOT EXISTS `idx_sessions_parent`
ON `sessions` (`parent_session_id`);
@@ -0,0 +1,45 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Composite-key lookups + audit columns.
-- - `app_uid` : binds `kind='app'` rows to their app authorization
-- target. (user_id, app_uid) is the idempotency key.
-- - `legacy_token_uid` : keys lazy-backfilled rows to the v1 token_uid that
-- originally minted them.
-- - `created_via` : audit sentinel (e.g. 'legacy_backfill').
-- - `auth_id` : stable per-user identity that survives re-login;
-- lets manage-sessions group by identity.
ALTER TABLE `sessions` ADD COLUMN `app_uid` TEXT;
ALTER TABLE `sessions` ADD COLUMN `legacy_token_uid` TEXT;
ALTER TABLE `sessions` ADD COLUMN `created_via` TEXT;
ALTER TABLE `sessions` ADD COLUMN `auth_id` TEXT;
-- Partial unique indexes keep "at most one active row per key" without
-- breaking the soft-revoke pattern (revoked rows stay for audit).
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_app_active`
ON `sessions` (`user_id`, `app_uid`)
WHERE `kind` = 'app' AND `revoked_at` IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_legacy_token_active`
ON `sessions` (`legacy_token_uid`)
WHERE `legacy_token_uid` IS NOT NULL AND `revoked_at` IS NULL;
-- Supports manage-sessions list queries grouped by kind.
CREATE INDEX IF NOT EXISTS `idx_sessions_kind_user`
ON `sessions` (`kind`, `user_id`);
@@ -0,0 +1,29 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Let `kind='access_token'` rows be reverse-looked-up
-- from the `token_uid` claim that lives only in `access_token_permissions`.
-- Required so `POST /auth/revoke-access-token` with a raw token_uid input
-- (no JWT) can find and soft-revoke the session row, matching the JWT
-- input path's coverage. Without it, raw-uuid revoke would only drop the
-- permissions row, leaving the session-row kill switch un-flipped.
ALTER TABLE `sessions` ADD COLUMN `access_token_uid` TEXT;
CREATE INDEX IF NOT EXISTS `idx_sessions_access_token_uid`
ON `sessions` (`access_token_uid`)
WHERE `access_token_uid` IS NOT NULL;
@@ -0,0 +1,30 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Worker session uniqueness. One active kind='worker' row per
-- (user_id, app_uid, worker_name).
--
-- SQLite UNIQUE indexes treat NULLs as distinct (per the SQL standard),
-- so user-scoped workers (where `app_uid` is NULL) would otherwise be
-- allowed to duplicate. IFNULL collapses NULL `app_uid` to empty
-- string in the index expression so two user-scoped workers with the
-- same worker_name correctly conflict. MySQL handles the same case
-- via the IFNULL in mig_11's generated column.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_worker_active`
ON `sessions` (`user_id`, IFNULL(`app_uid`, ''), json_extract(`meta`, '$.worker_name'))
WHERE `kind` = 'worker' AND `revoked_at` IS NULL;
@@ -0,0 +1,31 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce case-insensitive uniqueness on user.username.
--
-- SQLite defaults varchar columns to BINARY collation, so without this
-- index `Admin` and `admin` can coexist as separate rows even though the
-- reserved-name check and adminOnly gate both treat usernames as
-- case-insensitive. Prod MySQL already uses ascii_general_ci on this
-- column; this brings self-hosted SQLite to the same invariant.
--
-- If this CREATE fails because the DB already contains case-collision
-- duplicates, resolve them manually before re-running the migration —
-- there is no safe automatic merge of two user accounts.
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_username_nocase
ON user(username COLLATE NOCASE)
WHERE username IS NOT NULL;
@@ -0,0 +1,90 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Add 'worker' to the `kind` CHECK constraint on `sessions`. Mirrors the
-- MySQL ENUM extension in mysql_mig_12.
--
-- SQLite cannot ALTER a CHECK constraint in place, so we follow the
-- standard 12-step rebuild: create `sessions_new` with the corrected
-- constraint, copy rows, drop the old table, rename, then recreate every
-- index that lived on the original (indexes are auto-dropped with their
-- table).
CREATE TABLE `sessions_new` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"user_id" INTEGER NOT NULL,
"uuid" TEXT NOT NULL,
"meta" JSON DEFAULT NULL,
"created_at" INTEGER DEFAULT 0,
"last_activity" INTEGER DEFAULT 0,
"kind" TEXT NOT NULL DEFAULT 'web'
CHECK (`kind` IN ('web', 'app', 'access_token', 'asset', 'worker')),
"label" TEXT,
"parent_session_id" TEXT,
"last_ip" TEXT,
"last_user_agent" TEXT,
"revoked_at" INTEGER,
"expires_at" INTEGER,
"app_uid" TEXT,
"legacy_token_uid" TEXT,
"created_via" TEXT,
"auth_id" TEXT,
"access_token_uid" TEXT,
FOREIGN KEY("user_id") REFERENCES "user" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
INSERT INTO `sessions_new` (
`id`, `user_id`, `uuid`, `meta`, `created_at`, `last_activity`,
`kind`, `label`, `parent_session_id`, `last_ip`, `last_user_agent`,
`revoked_at`, `expires_at`, `app_uid`, `legacy_token_uid`,
`created_via`, `auth_id`, `access_token_uid`
)
SELECT
`id`, `user_id`, `uuid`, `meta`, `created_at`, `last_activity`,
`kind`, `label`, `parent_session_id`, `last_ip`, `last_user_agent`,
`revoked_at`, `expires_at`, `app_uid`, `legacy_token_uid`,
`created_via`, `auth_id`, `access_token_uid`
FROM `sessions`;
DROP TABLE `sessions`;
ALTER TABLE `sessions_new` RENAME TO `sessions`;
CREATE INDEX IF NOT EXISTS `idx_sessions_user_revoked`
ON `sessions` (`user_id`, `revoked_at`);
CREATE INDEX IF NOT EXISTS `idx_sessions_parent`
ON `sessions` (`parent_session_id`);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_app_active`
ON `sessions` (`user_id`, `app_uid`)
WHERE `kind` = 'app' AND `revoked_at` IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_legacy_token_active`
ON `sessions` (`legacy_token_uid`)
WHERE `legacy_token_uid` IS NOT NULL AND `revoked_at` IS NULL;
CREATE INDEX IF NOT EXISTS `idx_sessions_kind_user`
ON `sessions` (`kind`, `user_id`);
CREATE INDEX IF NOT EXISTS `idx_sessions_access_token_uid`
ON `sessions` (`access_token_uid`)
WHERE `access_token_uid` IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_worker_active`
ON `sessions` (`user_id`, IFNULL(`app_uid`, ''), json_extract(`meta`, '$.worker_name'))
WHERE `kind` = 'worker' AND `revoked_at` IS NULL;
@@ -0,0 +1,85 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { preparePostgresSql } from './preparePostgresSql.js';
describe('preparePostgresSql', () => {
it('converts placeholders and backtick identifiers outside SQL literals', () => {
const prepared = preparePostgresSql(
'SELECT `user`.`id` FROM `user` WHERE `email` = ? AND `username` = ?',
);
expect(prepared).toEqual({
text: 'SELECT "user"."id" FROM "user" WHERE "email" = $1 AND "username" = $2',
parameterCount: 2,
});
});
it('leaves question marks inside strings and comments untouched', () => {
const prepared = preparePostgresSql(`
SELECT '?' AS literal, ?
-- ? in a comment
FROM \`apps\`
WHERE \`description\` = 'is this ok?'
/* and ? in a block comment */
AND \`name\` = ?
`);
expect(prepared.text).toContain("SELECT '?' AS literal, $1");
expect(prepared.text).toContain('-- ? in a comment');
expect(prepared.text).toContain('"description" = \'is this ok?\'');
expect(prepared.text).toContain('/* and ? in a block comment */');
expect(prepared.text).toContain('"name" = $2');
expect(prepared.parameterCount).toBe(2);
});
it('escapes double quotes inside converted identifiers', () => {
const prepared = preparePostgresSql(
'SELECT `odd"name` FROM `odd``table` WHERE `id` = ?',
);
expect(prepared).toEqual({
text: 'SELECT "odd""name" FROM "odd`table" WHERE "id" = $1',
parameterCount: 1,
});
});
it('does not scan placeholders inside dollar-quoted strings', () => {
const prepared = preparePostgresSql(
"SELECT $$?$$, $tag$`not_ident` ?$tag$, `id` FROM `user` WHERE `id` = ?",
);
expect(prepared).toEqual({
text: 'SELECT $$?$$, $tag$`not_ident` ?$tag$, "id" FROM "user" WHERE "id" = $1',
parameterCount: 1,
});
});
it('does not treat Postgres JSON operators as comments', () => {
const prepared = preparePostgresSql(
"SELECT * FROM `sessions` WHERE `user_id` = ? AND `meta` #>> ARRAY['worker_name'] = ? AND `expires_at` > ?",
);
expect(prepared).toEqual({
text: 'SELECT * FROM "sessions" WHERE "user_id" = $1 AND "meta" #>> ARRAY[\'worker_name\'] = $2 AND "expires_at" > $3',
parameterCount: 3,
});
});
});
@@ -0,0 +1,169 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
export interface PreparedPostgresSql {
text: string;
parameterCount: number;
}
type ScannerState =
| { kind: 'normal' }
| { kind: 'singleQuote' }
| { kind: 'doubleQuote' }
| { kind: 'backtickIdentifier' }
| { kind: 'lineComment' }
| { kind: 'blockComment' }
| { kind: 'dollarQuote'; tag: string };
const isLineCommentStart = (sql: string, index: number): boolean => {
if (sql[index] !== '-' || sql[index + 1] !== '-') return false;
const after = sql[index + 2];
return after === undefined || /\s/u.test(after);
};
const readDollarQuoteTag = (sql: string, index: number): string | null => {
const match = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/u.exec(sql.slice(index));
return match?.[0] ?? null;
};
export const preparePostgresSql = (sql: string): PreparedPostgresSql => {
let state: ScannerState = { kind: 'normal' };
let parameterCount = 0;
let out = '';
for (let i = 0; i < sql.length; i++) {
const char = sql[i]!;
const next = sql[i + 1];
switch (state.kind) {
case 'normal': {
const dollarTag =
char === '$' ? readDollarQuoteTag(sql, i) : null;
if (dollarTag) {
out += dollarTag;
i += dollarTag.length - 1;
state = { kind: 'dollarQuote', tag: dollarTag };
break;
}
if (char === "'") {
out += char;
state = { kind: 'singleQuote' };
break;
}
if (char === '"') {
out += char;
state = { kind: 'doubleQuote' };
break;
}
if (char === '`') {
out += '"';
state = { kind: 'backtickIdentifier' };
break;
}
if (isLineCommentStart(sql, i)) {
out += char;
if (char === '-') {
out += next ?? '';
i += 1;
}
state = { kind: 'lineComment' };
break;
}
if (char === '/' && next === '*') {
out += '/*';
i += 1;
state = { kind: 'blockComment' };
break;
}
if (char === '?') {
parameterCount += 1;
out += `$${parameterCount}`;
break;
}
out += char;
break;
}
case 'singleQuote':
out += char;
if (char === "'" && next === "'") {
out += next;
i += 1;
} else if (char === '\\' && next !== undefined) {
out += next;
i += 1;
} else if (char === "'") {
state = { kind: 'normal' };
}
break;
case 'doubleQuote':
out += char;
if (char === '"' && next === '"') {
out += next;
i += 1;
} else if (char === '"') {
state = { kind: 'normal' };
}
break;
case 'backtickIdentifier':
if (char === '`' && next === '`') {
out += '`';
i += 1;
} else if (char === '`') {
out += '"';
state = { kind: 'normal' };
} else if (char === '"') {
out += '""';
} else {
out += char;
}
break;
case 'lineComment':
out += char;
if (char === '\n') {
state = { kind: 'normal' };
}
break;
case 'blockComment':
out += char;
if (char === '*' && next === '/') {
out += '/';
i += 1;
state = { kind: 'normal' };
}
break;
case 'dollarQuote':
if (sql.startsWith(state.tag, i)) {
out += state.tag;
i += state.tag.length - 1;
state = { kind: 'normal' };
} else {
out += char;
}
break;
}
}
return { text: out, parameterCount };
};
@@ -0,0 +1,120 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
type ScannerState =
| { kind: 'normal' }
| { kind: 'singleQuote' }
| { kind: 'doubleQuote' }
| { kind: 'lineComment' }
| { kind: 'blockComment' }
| { kind: 'dollarQuote'; tag: string };
const readDollarQuoteTag = (sql: string, index: number): string | null => {
const match = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/u.exec(sql.slice(index));
return match?.[0] ?? null;
};
export const splitPostgresStatements = (sql: string): string[] => {
const statements: string[] = [];
let state: ScannerState = { kind: 'normal' };
let start = 0;
for (let i = 0; i < sql.length; i++) {
const char = sql[i]!;
const next = sql[i + 1];
switch (state.kind) {
case 'normal': {
const dollarTag =
char === '$' ? readDollarQuoteTag(sql, i) : null;
if (dollarTag) {
i += dollarTag.length - 1;
state = { kind: 'dollarQuote', tag: dollarTag };
break;
}
if (char === "'") {
state = { kind: 'singleQuote' };
break;
}
if (char === '"') {
state = { kind: 'doubleQuote' };
break;
}
if (char === '-' && next === '-') {
i += 1;
state = { kind: 'lineComment' };
break;
}
if (char === '/' && next === '*') {
i += 1;
state = { kind: 'blockComment' };
break;
}
if (char === ';') {
const statement = sql.slice(start, i).trim();
if (statement !== '') statements.push(statement);
start = i + 1;
}
break;
}
case 'singleQuote':
if (char === "'" && next === "'") {
i += 1;
} else if (char === '\\' && next !== undefined) {
i += 1;
} else if (char === "'") {
state = { kind: 'normal' };
}
break;
case 'doubleQuote':
if (char === '"' && next === '"') {
i += 1;
} else if (char === '"') {
state = { kind: 'normal' };
}
break;
case 'lineComment':
if (char === '\n') {
state = { kind: 'normal' };
}
break;
case 'blockComment':
if (char === '*' && next === '/') {
i += 1;
state = { kind: 'normal' };
}
break;
case 'dollarQuote':
if (sql.startsWith(state.tag, i)) {
i += state.tag.length - 1;
state = { kind: 'normal' };
}
break;
}
}
const finalStatement = sql.slice(start).trim();
if (finalStatement !== '') statements.push(finalStatement);
return statements;
};
+7 -7
View File
@@ -24,7 +24,7 @@ import type { IConfig } from '../../types';
import { PuterClient } from '../types';
import { EMAIL_TEMPLATES, type EmailTemplateName } from './templates';
// ── Types ────────────────────────────────────────────────────────────
// -- Types ------------------------------------------------------------
// nodemailer doesn't ship TS types, so declare the subset we use.
interface NodemailerTransport {
@@ -50,7 +50,7 @@ interface CompiledTemplate {
html: ReturnType<typeof template>;
}
// ── Clean-email rules ────────────────────────────────────────────────
// -- Clean-email rules ------------------------------------------------
type CleanRule = (parts: { local: string; domain: string }) => {
local: string;
@@ -90,7 +90,7 @@ const DOMAIN_ALIASES: Record<string, string> = {
'googlemail.com': 'gmail.com',
};
// ── EmailClient ──────────────────────────────────────────────────────
// -- EmailClient ------------------------------------------------------
/**
* Unified email client. Handles:
@@ -112,7 +112,7 @@ export class EmailClient extends PuterClient {
this.compileTemplates();
}
// ── Lifecycle ────────────────────────────────────────────────────
// -- Lifecycle ----------------------------------------------------
override onServerStart(): void {
const emailConf = this.config.email;
@@ -133,7 +133,7 @@ export class EmailClient extends PuterClient {
this.transport = null;
}
// ── Public API: sending ──────────────────────────────────────────
// -- Public API: sending ------------------------------------------
/**
* Render a template and send it to `to`.
@@ -174,7 +174,7 @@ export class EmailClient extends PuterClient {
});
}
// ── Public API: clean / validate ─────────────────────────────────
// -- Public API: clean / validate ---------------------------------
/**
* Normalize an email to its canonical form for dedup comparisons.
@@ -241,7 +241,7 @@ export class EmailClient extends PuterClient {
this.validators.push(fn);
}
// ── Internals ────────────────────────────────────────────────────
// -- Internals ----------------------------------------------------
private defaultFrom(): string {
return this.config.email?.from ?? '"Puter" [email protected]';
+15 -5
View File
@@ -19,7 +19,13 @@
import { extensionStore } from '../../extensions';
import { PuterClient } from '../types';
import { EventListener, EventMap, ListenKey, MatchingEvents } from './types';
import {
EventListener,
EventMap,
EventMetadata,
ListenKey,
MatchingEvents,
} from './types';
export class EventClient extends PuterClient {
#eventListeners: Partial<Record<ListenKey, EventListener[]>> = {};
@@ -52,7 +58,11 @@ export class EventClient extends PuterClient {
* scan of every listener.
*
*/
emit<T extends keyof EventMap>(key: T, data: EventMap[T], meta: unknown) {
emit<T extends keyof EventMap>(
key: T,
data: EventMap[T],
meta: EventMetadata,
) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey = (
@@ -87,7 +97,7 @@ export class EventClient extends PuterClient {
async emitAndWait<T extends keyof EventMap>(
key: T,
data: EventMap[T],
meta: unknown,
meta: EventMetadata,
) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
@@ -129,7 +139,7 @@ export class EventClient extends PuterClient {
callback: (
key: MatchingEvents<P>,
data: EventMap[MatchingEvents<P>],
meta: unknown,
meta: EventMetadata,
) => Promise<void> | void,
) {
const listeners: EventListener[] =
@@ -140,7 +150,7 @@ export class EventClient extends PuterClient {
listener: EventListener,
key: T,
data: EventMap[T],
meta: unknown,
meta: EventMetadata,
) {
try {
await listener(key, data, meta);
+20 -5
View File
@@ -14,6 +14,9 @@
* the SocketService fans out to user-scoped channels.
*/
import { Actor } from '../../core';
import { FSEntry } from '../../stores/fs/FSEntry';
// GUI write events spread an entry plus per-event metadata into `response`.
// The exact field set varies by emit site (FSController / LegacyFSController /
// WebDAVController each project a slightly different shape) so the envelope
@@ -96,8 +99,11 @@ export type EventMap = {
};
'app.from-origin': { origin: string };
'app.privateAccess.check': {
app: unknown;
actor: unknown;
appUid: string;
userUid: string;
requestHost: string;
requestPath: string;
actor?: Actor;
result: {
allowed: boolean;
reason?: string;
@@ -162,8 +168,9 @@ export type EventMap = {
sourceObjectKey: string;
copyObjectKey: string;
};
'fs.move.node': { node: unknown; fromPath: string; toPath: string };
'fs.remove.node': { node: unknown; entry: unknown; target: unknown };
'fs.move.node': { node: FSEntry; fromPath: string; toPath: string };
'fs.remove.node': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.write.file': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.storage.upload-progress': {
upload_tracker: unknown;
context: unknown;
@@ -199,6 +206,13 @@ export type EventMap = {
// ---- Subdomains ----
'subdomain.delete': { subdomain: string };
'subdomain.update': { subdomain: string };
'site.htmlServed': {
subdomain: string;
entry: unknown;
host: string;
requestPath: string;
mime: string;
};
// ---- Thumbnails ----
'thumbnail.read': {
@@ -264,8 +278,9 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
? Extract<EventKey, `${Prefix}.${string}`>
: P & EventKey;
export type EventMetadata = { from_outside?: boolean };
export type EventListener<K extends EventKey = EventKey> = (
key: K,
data: EventMap[K],
meta: unknown,
meta: EventMetadata,
) => Promise<void> | void;
+9
View File
@@ -18,6 +18,7 @@
*/
import type { IConfig, WithLifecycle } from '../types';
import type { ClickhouseClient } from './clickhouse/ClickhouseClient';
/**
* Extension-augmentable client registry. Extensions add their own client
@@ -40,6 +41,14 @@ export interface IExtensionClientInstances {
* override this for that name.
*/
[key: string]: unknown;
/**
* Optional ClickHouse analytics client. Absent by default — a production
* deployment registers it via an extension to speed up the app-stats path
* at scale (see {@link ClickhouseClient}). Always branch on its presence
* and fall back to SQL.
*/
clickhouse?: ClickhouseClient;
}
export interface IPuterClient<T extends WithLifecycle = WithLifecycle> {
+35 -3
View File
@@ -17,6 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { isAccessTokenActor, isAppActor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { driversContainers } from '../../exports.js';
import {
@@ -89,6 +90,16 @@ export class AppController extends PuterController {
// actor calls this, the app id is already on the token — clients
// don't re-send it in the body. Fall back to `actor.app.uid`
// before 400-ing for a missing body field.
//
// Authorization: only two callers are trusted to report opens —
// 1. a root user actor (plain session, no `.app` and no access
// token), e.g. the GUI launching apps on behalf of the user;
// 2. the app-under-user actor for the app being reported, i.e.
// `actor.app.uid === app_uid`.
// Everything else — access tokens (regardless of issuer), asset
// tokens, app actors reporting for a *different* app — is denied,
// otherwise any authenticated party could inflate another app's
// open count.
router.post(
'/rao',
{
@@ -96,8 +107,9 @@ export class AppController extends PuterController {
requireAuth: true,
},
async (req, res) => {
const actor = req.actor;
const bodyAppUid = req.body?.app_uid;
const actorAppUid = req.actor?.app?.uid;
const actorAppUid = actor?.app?.uid;
const app_uid =
typeof bodyAppUid === 'string' && bodyAppUid.length > 0
? bodyAppUid
@@ -108,6 +120,26 @@ export class AppController extends PuterController {
});
}
// Access tokens (and any other non-user/non-app identity,
// e.g. asset tokens) are not allowed to report opens —
// they're shared / scoped credentials and shouldn't drive
// analytics counters.
if (isAccessTokenActor(actor)) {
throw new HttpError(
403,
'Access tokens cannot report app opens',
{ legacyCode: 'forbidden' },
);
}
if (isAppActor(actor) && app_uid !== actorAppUid) {
throw new HttpError(
403,
'App actors can only report opens for their own app',
{ legacyCode: 'forbidden' },
);
}
const app = await this.appStore.getByUid(app_uid);
if (!app)
throw new HttpError(404, 'App not found', {
@@ -196,7 +228,7 @@ export class AppController extends PuterController {
},
);
// ── POST /query/app ────────────────────────────────────────
// -- POST /query/app ----------------------------------------
// Batch marketplace-style lookup by name or UID.
//
// Access rules: only apps the caller has a legitimate reason to
@@ -293,7 +325,7 @@ export class AppController extends PuterController {
},
);
// ── GET /app-icon/:app_uid(/:size) ─────────────────────────
// -- GET /app-icon/:app_uid(/:size) -------------------------
// Serve app icon — data URL decoded inline, HTTP URL redirected.
//
// ⚠ FLAG: Missing sharp-based resize pipeline; serves the original.
@@ -30,12 +30,14 @@
*/
import bcrypt from 'bcrypt';
import jwt from 'jsonwebtoken';
import { v4 as uuidv4 } from 'uuid';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import type { EventClient } from '../../clients/event/EventClient.js';
import type { Actor } from '../../core/actor.js';
import { runWithContext } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { requireUserActorGate } from '../../core/http/middleware/gates.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -793,6 +795,65 @@ describe('AuthController.handleLogout', () => {
});
});
describe('AuthController account-lifecycle route gating', () => {
const routeOptions = (method: string, path: string) => {
const proto = Object.getPrototypeOf(controller) as {
__puterRoutes?: Array<{
method: string;
path: string;
options?: Record<string, unknown>;
}>;
};
const route = (proto.__puterRoutes ?? []).find(
(r) =>
r.method.toLowerCase() === method.toLowerCase() &&
r.path === path,
);
expect(route, `route ${method} ${path} not found`).toBeDefined();
return route!.options ?? {};
};
it('POST /logout requires a human user actor', () => {
const opts = routeOptions('post', '/logout');
expect(opts.requireUserActor).toBe(true);
expect(opts.antiCsrf).toBe(true);
});
it('GET /get-anticsrf-token requires a human user actor', () => {
const opts = routeOptions('get', '/get-anticsrf-token');
expect(opts.requireUserActor).toBe(true);
});
it('requireUserActorGate rejects app-under-user and access-token actors', () => {
const gate = requireUserActorGate();
const run = (actor: Partial<Actor>) =>
new Promise<unknown>((resolve) => {
gate(
{ actor } as never,
{} as never,
(err?: unknown) => resolve(err),
);
});
return (async () => {
const appActor = await run({
user: { uuid: 'u1' },
app: { uid: 'app-1' },
} as Partial<Actor>);
expect(appActor).toMatchObject({ statusCode: 403 });
const tokenActor = await run({
user: { uuid: 'u1' },
accessToken: { uid: 'tok-1' },
} as Partial<Actor>);
expect(tokenActor).toMatchObject({ statusCode: 403 });
const human = await run({ user: { uuid: 'u1' } } as Partial<Actor>);
expect(human).toBeUndefined();
})();
});
});
// ── Token grants: user → user / app / group ─────────────────────────
describe('AuthController grant flows', () => {
@@ -2079,6 +2140,104 @@ describe('AuthController session endpoints', () => {
),
).rejects.toMatchObject({ statusCode: 403 });
});
it('rename-session: 400 when uuid param is missing', async () => {
const { actor } = await makeUserAndActor();
await expect(
controller.handleRenameSession(
makeReq({ label: 'x' }, { actor, params: {} }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rename-session: 400 when label is the wrong type', async () => {
const { actor } = await makeUserAndActor();
await expect(
controller.handleRenameSession(
makeReq(
{ label: 123 as unknown as string },
{ actor, params: { uuid: 'whatever' } },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rename-session: 400 when label field is missing entirely', async () => {
// Guards against accidental "PATCH with empty body silently clears
// the label". Type guard rejects `undefined` before reaching the
// service layer.
const { actor } = await makeUserAndActor();
await expect(
controller.handleRenameSession(
makeReq({}, { actor, params: { uuid: 'whatever' } }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('rename-session: 404 when the uuid belongs to another user', async () => {
const { user: u1 } = await makeUserAndActor();
const { actor: a2 } = await makeUserAndActor();
const sessionRes = await server.services.auth.createSessionToken(
u1,
{},
);
const uuid = (sessionRes.session as { uuid: string }).uuid;
await expect(
controller.handleRenameSession(
makeReq(
{ label: 'pwned' },
{ actor: a2, params: { uuid } },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('rename-session: success updates the row label', async () => {
const { user, actor } = await makeUserAndActor();
const sessionRes = await server.services.auth.createSessionToken(
user,
{},
);
const uuid = (sessionRes.session as { uuid: string }).uuid;
const res = makeRes();
await controller.handleRenameSession(
makeReq({ label: 'My Phone' }, { actor, params: { uuid } }),
res,
);
expect(res.body).toEqual({});
const rows = await server.clients.db.read(
'SELECT `label` FROM `sessions` WHERE `uuid` = ?',
[uuid],
);
expect((rows[0] as { label: string }).label).toBe('My Phone');
});
it('rename-session: accepts null to clear the label', async () => {
const { user, actor } = await makeUserAndActor();
const sessionRes = await server.services.auth.createSessionToken(
user,
{},
);
const uuid = (sessionRes.session as { uuid: string }).uuid;
// Seed a non-null label so the clear-to-null transition is observable.
await server.clients.db.write(
'UPDATE `sessions` SET `label` = ? WHERE `uuid` = ?',
['something', uuid],
);
await controller.handleRenameSession(
makeReq({ label: null }, { actor, params: { uuid } }),
makeRes(),
);
const rows = await server.clients.db.read(
'SELECT `label` FROM `sessions` WHERE `uuid` = ?',
[uuid],
);
expect((rows[0] as { label: string | null }).label).toBeNull();
});
});
// ── Dev-app grants/revokes ─────────────────────────────────────────
@@ -3405,4 +3564,568 @@ describe('AuthController.handleRevokeSession additional branches', () => {
const body = res.body as { sessions: unknown[] };
expect(Array.isArray(body.sessions)).toBe(true);
});
it('refuses to revoke the caller’s OWN current session row (400)', async () => {
// Invariant: a self-revoke leaves the client in an ambiguous
// identity state because the response can't write fresh auth
// state. /logout is the only path that should end the session
// you're currently authenticated under.
const { user, actor } = await makeUserAndActor();
const sessionRes = await server.services.auth.createSessionToken(
user,
{},
);
const sessionUid = (sessionRes.session as { uuid: string }).uuid;
const actorWithSession = {
...actor,
session: { uid: sessionUid },
} as Actor;
await expect(
controller.handleRevokeSession(
makeReq({ uuid: sessionUid }, { actor: actorWithSession }),
makeRes(),
),
).rejects.toMatchObject({
statusCode: 400,
legacyCode: 'bad_request',
});
});
it('still allows revoking a DIFFERENT session belonging to the same user', async () => {
// Sanity check that the self-revoke guard only blocks the
// caller's own uuid — sibling rows must still be revokable
// (that's the whole point of manage-sessions).
const { user, actor } = await makeUserAndActor();
const callerSession = await server.services.auth.createSessionToken(
user,
{},
);
const targetSession = await server.services.auth.createSessionToken(
user,
{},
);
const actorWithSession = {
...actor,
session: {
uid: (callerSession.session as { uuid: string }).uuid,
},
} as Actor;
const res = makeRes();
await controller.handleRevokeSession(
makeReq(
{ uuid: (targetSession.session as { uuid: string }).uuid },
{ actor: actorWithSession },
),
res,
);
expect((res.body as { sessions: unknown[] }).sessions).toBeDefined();
});
});
// ── handleMigrateToken ──────────────────────────────────────────────
describe('AuthController.handleMigrateToken', () => {
const TEST_ORIGIN = 'https://migrate.test.local';
// PuterServer keeps config in a private field (#config), so we go
// through the controller — IController stores it as `protected
// config` which TS marks but JS doesn't enforce, and the controller
// is the actual consumer of #isMigrateTokenOriginAllowed anyway.
const controllerConfig = () =>
(controller as { config: Record<string, unknown> }).config;
// Mints a v1-shaped JWT signed under the test server's legacy
// secret. The body matches what migrateLegacyToken expects per
// `decoded.type`.
const mintV1Token = (payload: Record<string, unknown>): string => {
const legacy = controllerConfig().jwt_secret as string | undefined;
if (!legacy) throw new Error('test config missing jwt_secret');
return jwt.sign(payload, legacy);
};
beforeAll(() => {
// Make the origin allow-check pass for these tests. We mutate
// the live config because setupTestServer is shared across the
// file; the original value is undefined (default config has no
// `origin`) so we don't need to restore.
controllerConfig().origin = TEST_ORIGIN;
});
it('rejects when the Origin header is missing', async () => {
await expect(
controller.handleMigrateToken(makeReq({}), makeRes()),
).rejects.toMatchObject({ statusCode: 403 });
});
it('rejects when the Origin header is not in config.origin or the allowlist', async () => {
const { user } = await makeUserAndActor();
const v1 = mintV1Token({
type: 'access-token',
token_uid: uuidv4(),
user_uid: user.uuid,
});
await expect(
controller.handleMigrateToken(
makeReq(
{},
{
headers: {
origin: 'https://not-allowed.example',
authorization: `Bearer ${v1}`,
},
},
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 403 });
});
it('normalizes trailing slash on the request Origin (B4)', async () => {
// The Origin header per spec doesn't carry a trailing slash, but
// a misconfigured proxy or a deployment with config.origin
// ending in `/` would otherwise force every call to reject.
const { user } = await makeUserAndActor();
const v1 = mintV1Token({
type: 'access-token',
token_uid: uuidv4(),
user_uid: user.uuid,
});
const res = makeRes();
await controller.handleMigrateToken(
makeReq(
{},
{
headers: {
origin: `${TEST_ORIGIN}/`, // trailing slash
authorization: `Bearer ${v1}`,
},
},
),
res,
);
expect((res.body as { kind: string }).kind).toBe('access_token');
});
it('normalizes case on the request Origin (B4)', async () => {
const { user } = await makeUserAndActor();
const v1 = mintV1Token({
type: 'access-token',
token_uid: uuidv4(),
user_uid: user.uuid,
});
const res = makeRes();
await controller.handleMigrateToken(
makeReq(
{},
{
headers: {
origin: TEST_ORIGIN.toUpperCase(),
authorization: `Bearer ${v1}`,
},
},
),
res,
);
expect((res.body as { kind: string }).kind).toBe('access_token');
});
it('returns 409 reauth_required for v1 web/session tokens', async () => {
// Web tokens never migrate silently — they always go through
// the interactive reauth flow. The body code is what puter.js /
// GUI key on; the 409 status is what tells SDK code "this
// isn't a generic auth failure, route through reauth".
const { user } = await makeUserAndActor();
const v1 = mintV1Token({
type: 'session',
user_uid: user.uuid,
uuid: uuidv4(),
});
await expect(
controller.handleMigrateToken(
makeReq(
{},
{
headers: {
origin: TEST_ORIGIN,
authorization: `Bearer ${v1}`,
},
},
),
makeRes(),
),
).rejects.toMatchObject({
statusCode: 409,
code: 'reauth_required',
});
});
it('does NOT set the puter_token_v2 cookie when migrating an access token (B3)', async () => {
// Access tokens are programmatic — they ride in Authorization
// headers, not browser cookies. Setting a cookie here would
// confuse cookie-only middleware downstream.
const { user } = await makeUserAndActor();
const v1 = mintV1Token({
type: 'access-token',
token_uid: uuidv4(),
user_uid: user.uuid,
});
const res = makeRes();
await controller.handleMigrateToken(
makeReq(
{},
{
headers: {
origin: TEST_ORIGIN,
authorization: `Bearer ${v1}`,
},
},
),
res,
);
expect(res.cookies.puter_token_v2).toBeUndefined();
expect((res.body as { kind: string }).kind).toBe('access_token');
expect((res.body as { token: string }).token).toBeTruthy();
});
it('sets the puter_token_v2 cookie when migrating an app-under-user token (B3)', async () => {
// App tokens DO get a cookie companion — the app runs inside an
// iframe in the GUI, and the GUI's cookie-only middleware
// authenticates subsequent calls from the iframe via the cookie
// rather than the client having to plumb Authorization through
// every request.
const { user } = await makeUserAndActor();
const appUid = `app-${uuidv4()}`;
const v1 = mintV1Token({
type: 'app-under-user',
user_uid: user.uuid,
app_uid: appUid,
});
const res = makeRes();
await controller.handleMigrateToken(
makeReq(
{},
{
headers: {
origin: TEST_ORIGIN,
authorization: `Bearer ${v1}`,
},
},
),
res,
);
expect((res.body as { kind: string }).kind).toBe('app');
const cookie = res.cookies.puter_token_v2;
expect(cookie).toBeDefined();
expect(cookie.value).toBe((res.body as { token: string }).token);
expect(cookie.opts?.httpOnly).toBe(true);
});
});
// -- auth_id preservation on forced re-login --
describe('AuthController auth_id preservation on reauth', () => {
const password = 'correct-horse-battery';
const mintReauth = (uuid: string): string =>
server.services.auth.signReauthToken(uuid);
it('handleLogin with matching reauth_token completes login as the same user', async () => {
const u = `aid_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.1`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
const seeded = await server.stores.user.getByUsername(u);
const res = makeRes();
await controller.handleLogin(
makeReq(
{
username: u,
password,
reauth_token: mintReauth(seeded!.uuid),
},
{ ip },
),
res,
);
expect(isCompleteLoginResponse(res.body)).toBe(true);
expect((res.body as { user: { uuid: string } }).user.uuid).toBe(
seeded!.uuid,
);
});
it('handleLogin with mismatched reauth_token is rejected 409', async () => {
const a = `aida_${Math.random().toString(36).slice(2, 10)}`;
const b = `aidb_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.2`;
await controller.handleSignup(
makeReq(
{ username: a, email: `${a}@test.local`, password },
{ ip },
),
makeRes(),
);
await controller.handleSignup(
makeReq(
{ username: b, email: `${b}@test.local`, password },
{ ip },
),
makeRes(),
);
const userB = await server.stores.user.getByUsername(b);
await expect(
controller.handleLogin(
makeReq(
{
username: a,
password,
reauth_token: mintReauth(userB!.uuid),
},
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({
statusCode: 409,
fields: { code: 'auth_id_mismatch' },
});
});
it('handleLogin with reauth_token for an unknown user returns 404', async () => {
const u = `aidu_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.3`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
await expect(
controller.handleLogin(
makeReq(
{
username: u,
password,
reauth_token: mintReauth(uuidv4()),
},
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('handleLogin with a non-string reauth_token returns 400', async () => {
const u = `aidb_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.4`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
await expect(
controller.handleLogin(
makeReq({ username: u, password, reauth_token: 42 }, { ip }),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('handleLogin with a forged/garbage reauth_token returns 401', async () => {
const u = `aidf_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.9`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
await expect(
controller.handleLogin(
makeReq(
{ username: u, password, reauth_token: 'not-a-jwt' },
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 401 });
});
it('handleLogin OTP branch echoes auth_id into the OTP JWT', async () => {
const u = `aidotp_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.5`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
const seeded = await server.stores.user.getByUsername(u);
await server.stores.user.update(seeded!.id, {
otp_enabled: 1,
otp_secret: 'TESTSECRETBASE32',
});
const res = makeRes();
await controller.handleLogin(
makeReq(
{
username: u,
password,
reauth_token: mintReauth(seeded!.uuid),
},
{ ip },
),
res,
);
expect(res.statusCode).toBe(202);
const body = res.body as { otp_jwt_token: string };
const decoded = server.services.token.verify(
'otp',
body.otp_jwt_token,
) as {
user_uid: string;
auth_id?: string;
};
expect(decoded.auth_id).toBe(seeded!.uuid);
});
it('handleSignup is_temp + matching reauth_token returns the SAME temp user', async () => {
const tempRes1 = makeRes();
const ip = `127.0.${Math.floor(Math.random() * 200)}.6`;
await controller.handleSignup(
makeReq({ is_temp: true }, { ip }),
tempRes1,
);
const body1 = tempRes1.body as { user: { uuid: string } };
const tempUuid = body1.user.uuid;
const tempUser1 = await server.stores.user.getByUuid(tempUuid);
expect(tempUser1).toBeTruthy();
const markerId = tempUser1!.id;
const tempRes2 = makeRes();
await controller.handleSignup(
makeReq(
{ is_temp: true, reauth_token: mintReauth(tempUuid) },
{ ip },
),
tempRes2,
);
expect(isCompleteLoginResponse(tempRes2.body)).toBe(true);
const body2 = tempRes2.body as {
user: { uuid: string; is_temp: boolean };
};
expect(body2.user.uuid).toBe(tempUuid);
expect(body2.user.is_temp).toBe(true);
const tempUser2 = await server.stores.user.getByUuid(tempUuid);
expect(tempUser2!.id).toBe(markerId);
});
it('handleSignup is_temp + reauth_token pointing at a permanent user is rejected', async () => {
const u = `aidperm_${Math.random().toString(36).slice(2, 10)}`;
const ip = `127.0.${Math.floor(Math.random() * 200)}.7`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
const seeded = await server.stores.user.getByUsername(u);
await expect(
controller.handleSignup(
makeReq(
{
is_temp: true,
reauth_token: mintReauth(seeded!.uuid),
},
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 400 });
});
it('handleSignup is_temp + reauth_token for an unknown user returns 404', async () => {
const ip = `127.0.${Math.floor(Math.random() * 200)}.8`;
await expect(
controller.handleSignup(
makeReq(
{
is_temp: true,
reauth_token: mintReauth(uuidv4()),
},
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('handleSignup is_temp rejects a forged reauth_token (401)', async () => {
const ip = `127.0.${Math.floor(Math.random() * 200)}.10`;
await expect(
controller.handleSignup(
makeReq(
{ is_temp: true, reauth_token: 'not-a-jwt' },
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 401 });
});
it('rate-limits reauth_token login attempts per IP', async () => {
const ip = `10.99.${Math.floor(Math.random() * 200)}.${Math.floor(Math.random() * 200)}`;
const u = `aidrl_${Math.random().toString(36).slice(2, 10)}`;
await controller.handleSignup(
makeReq(
{ username: u, email: `${u}@test.local`, password },
{ ip },
),
makeRes(),
);
const seeded = await server.stores.user.getByUsername(u);
for (let i = 0; i < 5; i++) {
const res = makeRes();
await controller.handleLogin(
makeReq(
{
username: u,
password,
reauth_token: mintReauth(seeded!.uuid),
},
{ ip },
),
res,
);
expect(isCompleteLoginResponse(res.body)).toBe(true);
}
await expect(
controller.handleLogin(
makeReq(
{
username: u,
password,
reauth_token: mintReauth(seeded!.uuid),
},
{ ip },
),
makeRes(),
),
).rejects.toMatchObject({ statusCode: 429 });
});
});
+436 -75
View File
@@ -26,7 +26,11 @@ import { Controller, Get, Post } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { antiCsrf } from '../../core/http/middleware/antiCsrf.js';
import { generateCaptcha } from '../../core/http/middleware/captcha.js';
import { createUserProtectedGate } from '../../core/http/middleware/userProtected.js';
import { checkRateLimit } from '../../core/http/middleware/rateLimit.js';
import {
createUserProtectedGate,
createWebSessionActorGate,
} from '../../core/http/middleware/userProtected.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import {
ROUTES_METADATA_KEY,
@@ -91,7 +95,7 @@ const RESERVED_USERNAMES = new Set([
*/
@Controller('')
export class AuthController extends PuterController {
// ── Login ───────────────────────────────────────────────────────
// -- Login -------------------------------------------------------
@Post('/login', {
captcha: true,
@@ -158,23 +162,34 @@ export class AuthController extends PuterController {
}
// Verify password
const passwordMatch = await bcrypt.compare(password, user.password);
const passwordMatch = await bcrypt.compare(
password,
user.password as string,
);
if (!passwordMatch) {
throw new HttpError(401, 'Incorrect password.', {
legacyCode: 'password_mismatch',
});
}
// OTP branching — if 2FA enabled, return a short-lived OTP JWT
const reauthAuthId = this.#extractAuthIdFromReauthToken(
req.body.reauth_token,
);
await this.#enforceAuthIdMatch(req, user, reauthAuthId);
// OTP branching — if 2FA enabled, return a short-lived OTP JWT.
// Re-bind the verified `auth_id` into the JWT so the follow-up
// OTP/recovery call can re-enforce the match without re-trusting
// a free-form claim from the client.
if (user.otp_enabled) {
const otp_jwt_token = this.services.token.sign(
'otp',
{
user_uid: user.uuid,
purpose: 'otp-login',
},
{ expiresIn: '5m' },
);
const otpClaims: Record<string, unknown> = {
user_uid: user.uuid,
purpose: 'otp-login',
};
if (reauthAuthId) otpClaims.auth_id = reauthAuthId;
const otp_jwt_token = this.services.token.sign('otp', otpClaims, {
expiresIn: '5m',
});
res.status(202).json({
proceed: true,
@@ -187,7 +202,7 @@ export class AuthController extends PuterController {
await this.#completeLogin(req, res, user);
}
// ── Login: OTP verification ─────────────────────────────────────
// -- Login: OTP verification -------------------------------------
@Post('/login/otp', {
captcha: true,
@@ -210,7 +225,11 @@ export class AuthController extends PuterController {
let decoded;
try {
decoded = this.services.token.verify('otp', token);
decoded = this.services.token.verify<{
user_uid: string;
purpose: string;
auth_id?: string;
}>('otp', token);
} catch {
throw new HttpError(400, 'Invalid token.', {
legacyCode: 'bad_request',
@@ -238,10 +257,12 @@ export class AuthController extends PuterController {
return;
}
await this.#enforceAuthIdMatch(req, user, decoded.auth_id ?? null);
await this.#completeLogin(req, res, user);
}
// ── Login: recovery code ────────────────────────────────────────
// -- Login: recovery code ----------------------------------------
@Post('/login/recovery-code', {
captcha: true,
@@ -264,7 +285,11 @@ export class AuthController extends PuterController {
let decoded;
try {
decoded = this.services.token.verify('otp', token);
decoded = this.services.token.verify<{
user_uid: string;
purpose: string;
auth_id?: string;
}>('otp', token);
} catch {
throw new HttpError(400, 'Invalid token.', {
legacyCode: 'bad_request',
@@ -288,7 +313,7 @@ export class AuthController extends PuterController {
}
const hashed = hashRecoveryCode(code);
const codes = (user.otp_recovery_codes || '')
const codes = ((user.otp_recovery_codes as string) || '')
.split(',')
.filter(Boolean);
const idx = codes.indexOf(hashed);
@@ -305,10 +330,12 @@ export class AuthController extends PuterController {
);
await this.stores.user.invalidateById(user.id);
await this.#enforceAuthIdMatch(req, user, decoded.auth_id ?? null);
await this.#completeLogin(req, res, user);
}
// ── Signup ──────────────────────────────────────────────────────
// -- Signup ------------------------------------------------------
@Post('/signup', {
captcha: true,
@@ -328,6 +355,49 @@ export class AuthController extends PuterController {
return;
}
// Temp-user reauth short-circuit: when an existing temp user is
// forced through the reauth flow, the GUI re-submits /signup with
// is_temp=true plus the server-signed reauth_token from the 401.
// Verifying the token (not a raw auth_id) means a leaked uuid alone
// can't re-attach a session to someone else's temp account.
// Permanent users must go through /login (they have credentials),
// so we reject that path here.
if (
is_temp &&
body.reauth_token !== undefined &&
body.reauth_token !== null
) {
const reauthAuthId = this.#extractAuthIdFromReauthToken(
body.reauth_token,
);
if (!reauthAuthId) {
throw new HttpError(400, 'Invalid `reauth_token`.', {
legacyCode: 'bad_request',
});
}
await this.#checkAuthIdRateLimit(req);
const existing = await this.stores.user.getByUuid(reauthAuthId);
if (!existing) {
throw new HttpError(404, 'auth_id not found.', {
legacyCode: 'not_found',
});
}
if (existing.password !== null || existing.email !== null) {
throw new HttpError(
400,
'auth_id resolves to a non-temp account; use /login instead.',
{ legacyCode: 'bad_request' },
);
}
if (existing.suspended) {
throw new HttpError(401, 'This account is suspended.', {
legacyCode: 'account_suspended',
});
}
await this.#completeLogin(req, res, existing);
return;
}
// Fill in temp user defaults
if (is_temp) {
body.username ??= await this.#generateRandomUsername();
@@ -519,7 +589,7 @@ export class AuthController extends PuterController {
let user;
if (pseudo_user) {
// ── Pseudo-user claim (convert the placeholder row) ──
// -- Pseudo-user claim (convert the placeholder row) --
await this.stores.user.update(pseudo_user.id, {
username: body.username,
password: password_hash,
@@ -560,7 +630,7 @@ export class AuthController extends PuterController {
force: true,
});
} else {
// ── New user ────────────────────────────────────────
// -- New user ----------------------------------------
const clientIp = req.ip || req.socket?.remoteAddress || null;
const proxyIpChain = req.headers['x-forwarded-for'];
@@ -605,7 +675,7 @@ export class AuthController extends PuterController {
}
}
// ── Provision FS home + default folders ─────────────────
// -- Provision FS home + default folders -----------------
// Idempotent — skips if `user.trash_uuid` is already set (pseudo
// users who went through a prior signup won't double-create).
try {
@@ -618,7 +688,7 @@ export class AuthController extends PuterController {
console.warn('[signup] generateDefaultFsentries failed:', e);
}
// ── Send email confirmation ─────────────────────────────
// -- Send email confirmation -----------------------------
if (
!is_temp &&
user!.requires_email_confirmation &&
@@ -691,16 +761,20 @@ export class AuthController extends PuterController {
await this.#completeLogin(req, res, user!);
}
// ── Logout ──────────────────────────────────────────────────────
// -- Logout ------------------------------------------------------
@Post('/logout', {
requireAuth: true,
requireUserActor: true,
allowUnconfirmed: true,
antiCsrf: true,
})
async handleLogout(req: Request, res: Response): Promise<void> {
// Clear the session cookie
res.clearCookie(this.config.cookie_name!);
// Clear the session cookie + the v2 migrate-token companion
// cookie (set after `/auth/migrate-token` for app-under-user
// iframes). authProbe reads `puter_token_v2` as a fallback, so
// a stale value would re-authenticate the next request.
res.clearCookie(this.config.cookie_name ?? 'puter_token');
res.clearCookie('puter_token_v2');
// Remove the session (fire-and-forget)
if (req.token) {
@@ -711,7 +785,9 @@ export class AuthController extends PuterController {
// same path as /user-protected/delete-own-user — so we don't
// orphan fsentries/sessions/permissions.
if (req.actor?.user && !req.actor.user.email) {
const user = await this.stores.user.getByUuid(req.actor.user.uuid);
const user = await this.stores.user.getByUuid(
req.actor.user.uuid as string,
);
if (user && user.password === null && user.email === null) {
this.#cascadeDeleteUser(user.id).catch((e) => {
console.warn('[logout] temp-user cleanup failed:', e);
@@ -722,7 +798,7 @@ export class AuthController extends PuterController {
res.send('logged out');
}
// ── Email confirmation ──────────────────────────────────────────
// -- Email confirmation ------------------------------------------
@Post('/send-confirm-email', {
subdomain: ['api', ''],
@@ -852,7 +928,7 @@ export class AuthController extends PuterController {
res.json({ email_confirmed: true, original_client_socket_id });
}
// ── Password recovery ───────────────────────────────────────────
// -- Password recovery -------------------------------------------
@Post('/send-pass-recovery-email', {
subdomain: ['api', ''],
@@ -944,7 +1020,12 @@ export class AuthController extends PuterController {
let decoded;
try {
decoded = this.services.token.verify('otp', token);
decoded = this.services.token.verify<{
user_uid: string;
email: string;
exp: number;
purpose: string;
}>('otp', token);
} catch {
throw new HttpError(400, 'Invalid or expired token.', {
legacyCode: 'token_expired' as never,
@@ -956,7 +1037,7 @@ export class AuthController extends PuterController {
});
}
const user = await this.stores.user.getByUuid(decoded.user_uid);
const user = await this.stores.user.getByUuid(decoded?.user_uid);
if (!user || user.email !== decoded.email) {
throw new HttpError(400, 'Token is no longer valid.', {
legacyCode: 'bad_request',
@@ -968,7 +1049,7 @@ export class AuthController extends PuterController {
});
}
const exp = decoded.exp;
const exp = decoded.exp as number;
const time_remaining = exp
? Math.max(0, exp - Math.floor(Date.now() / 1000))
: 0;
@@ -1001,7 +1082,12 @@ export class AuthController extends PuterController {
let decoded;
try {
decoded = this.services.token.verify('otp', token);
decoded = this.services.token.verify<{
user_uid: string;
email: string;
token: string;
purpose: string;
}>('otp', token);
} catch {
throw new HttpError(400, 'Invalid or expired token.', {
legacyCode: 'token_expired' as never,
@@ -1046,7 +1132,7 @@ export class AuthController extends PuterController {
res.send('Password successfully updated.');
}
// ── User-protected mutations ────────────────────────────────────
// -- User-protected mutations ------------------------------------
//
// The five `/user-protected/*` and `/user-protected/delete-own-user`
// routes are wired in the `registerRoutes` override below because
@@ -1344,7 +1430,7 @@ export class AuthController extends PuterController {
res.send('Email changed successfully. You may close this window.');
}
// ── Save account (convert temp user to permanent) ───────────────
// -- Save account (convert temp user to permanent) ---------------
@Post('/save_account', {
subdomain: ['api', ''],
@@ -1534,7 +1620,7 @@ export class AuthController extends PuterController {
});
}
// ── Captcha generation ───────────────────────────────────────────
// -- Captcha generation -------------------------------------------
@Get('/api/captcha/generate', { subdomain: '*' })
async handleCaptchaGenerate(_req: Request, res: Response): Promise<void> {
@@ -1545,10 +1631,12 @@ export class AuthController extends PuterController {
res.json({ token, image });
}
// ── Anti-CSRF token generation ──────────────────────────────────
// -- Anti-CSRF token generation ----------------------------------
@Get('/get-anticsrf-token', {
requireAuth: true,
// Anti-CSRF tokens are only consumed by `requireUserActor` routes,
// so issuance is scoped to the same actor kind for consistency.
requireUserActor: true,
allowUnconfirmed: true,
})
async handleGetAntiCsrfToken(req: Request, res: Response): Promise<void> {
@@ -1561,7 +1649,7 @@ export class AuthController extends PuterController {
res.json({ token });
}
// ── Permission grants ───────────────────────────────────────────
// -- Permission grants -------------------------------------------
@Post('/auth/grant-user-user', {
subdomain: 'api',
@@ -1633,7 +1721,7 @@ export class AuthController extends PuterController {
res.json({});
}
// ── Permission revokes ──────────────────────────────────────────
// -- Permission revokes ------------------------------------------
@Post('/auth/revoke-user-user', {
subdomain: 'api',
@@ -1705,7 +1793,7 @@ export class AuthController extends PuterController {
res.json({});
}
// ── Permission checks ───────────────────────────────────────────
// -- Permission checks -------------------------------------------
@Post('/auth/check-permissions', { subdomain: 'api', requireAuth: true })
async handleCheckPermissions(req: Request, res: Response): Promise<void> {
@@ -1733,7 +1821,7 @@ export class AuthController extends PuterController {
res.json({ permissions: result });
}
// ── Session management ──────────────────────────────────────────
// -- Session management ------------------------------------------
@Get('/auth/list-sessions', { subdomain: 'api', requireUserActor: true })
async handleListSessions(req: Request, res: Response): Promise<void> {
@@ -1741,12 +1829,10 @@ export class AuthController extends PuterController {
res.json(sessions);
}
@Post('/auth/revoke-session', {
subdomain: 'api',
requireUserActor: true,
allowUnconfirmed: true,
antiCsrf: true,
})
// Wired imperatively in `registerRoutes` so the cookie-only gate
// (built from `this.config`) can be composed in. Cookie-only is
// mandatory: an access token must not be able to revoke its own
// issuing web session.
async handleRevokeSession(req: Request, res: Response): Promise<void> {
const { uuid } = req.body;
if (!uuid || typeof uuid !== 'string') {
@@ -1754,7 +1840,28 @@ export class AuthController extends PuterController {
legacyCode: 'bad_request',
});
}
// The caller's own session row must go through /logout, not a
// self-revoke — otherwise the response can't write fresh auth
// state and the client ends up with an ambiguous post-revoke
// identity. /auth/revoke-all-sessions still supports a separate
// `include_current` opt-in for the nuclear case.
if (uuid === req.actor!.session?.uid) {
throw new HttpError(
400,
'Cannot revoke your current session — use /logout instead',
{ legacyCode: 'bad_request' },
);
}
// `getByUuid` returns null when the row is missing, already
// soft-revoked, or past `expires_at` — surface as 404 so a stale
// manage-sessions UI doesn't 500 when it clicks revoke on a row
// that already went away.
const session = await this.stores.session.getByUuid(uuid);
if (!session) {
throw new HttpError(404, 'Session not found', {
legacyCode: 'not_found',
});
}
if (session.user_id !== req.actor!.user.id) {
throw new HttpError(403, 'Can only revoke your own sessions', {
legacyCode: 'unauthorized',
@@ -1765,7 +1872,38 @@ export class AuthController extends PuterController {
res.json({ sessions });
}
// ── Dev app permissions ─────────────────────────────────────────
async handleRevokeAllSessions(req: Request, res: Response): Promise<void> {
const { include_current, include_apps } = req.body ?? {};
await this.services.auth.revokeAllSessions(req.actor!, {
includeCurrent: !!include_current,
includeApps: !!include_apps,
});
const sessions = await this.services.auth.listSessions(req.actor!);
res.json({ sessions });
}
async handleRenameSession(req: Request, res: Response): Promise<void> {
const uuid = req.params.uuid;
const { label } = (req.body ?? {}) as { label?: unknown };
if (!uuid || typeof uuid !== 'string') {
throw new HttpError(400, 'Missing or invalid `uuid`', {
legacyCode: 'bad_request',
});
}
if (label !== null && typeof label !== 'string') {
throw new HttpError(400, '`label` must be a string or null', {
legacyCode: 'bad_request',
});
}
await this.services.auth.setSessionLabel(
req.actor!,
uuid,
label ?? null,
);
res.json({});
}
// -- Dev app permissions -----------------------------------------
@Post('/auth/grant-dev-app', { subdomain: 'api', requireUserActor: true })
async handleGrantDevApp(req: Request, res: Response): Promise<void> {
@@ -1820,7 +1958,7 @@ export class AuthController extends PuterController {
res.json({});
}
// ── Permission listing ──────────────────────────────────────────
// -- Permission listing ------------------------------------------
@Get('/auth/list-permissions', {
subdomain: 'api',
@@ -1882,7 +2020,7 @@ export class AuthController extends PuterController {
});
}
// ── App origin resolution ───────────────────────────────────────
// -- App origin resolution ---------------------------------------
@Post('/auth/app-uid-from-origin', { subdomain: 'api', requireAuth: true })
async handleAppUidFromOrigin(req: Request, res: Response): Promise<void> {
@@ -1895,7 +2033,7 @@ export class AuthController extends PuterController {
res.json({ uid });
}
// ── App token + check ───────────────────────────────────────────
// -- App token + check -------------------------------------------
@Post('/auth/get-user-app-token', {
subdomain: 'api',
@@ -1933,7 +2071,10 @@ export class AuthController extends PuterController {
{},
);
const token = this.services.auth.getUserAppToken(req.actor!, app_uid);
const token = await this.services.auth.getUserAppToken(
req.actor!,
app_uid,
);
const missingFSPathPromise = (async () => {
// Ensure the app's per-user AppData directory exists.
@@ -1988,7 +2129,7 @@ export class AuthController extends PuterController {
token?: string;
} = { app_uid, authenticated };
if (authenticated) {
result.token = this.services.auth.getUserAppToken(
result.token = await this.services.auth.getUserAppToken(
req.actor!,
app_uid,
);
@@ -1996,7 +2137,75 @@ export class AuthController extends PuterController {
res.json(result);
}
// ── Access tokens ───────────────────────────────────────────────
// -- Access tokens -----------------------------------------------
async handleMigrateToken(req: Request, res: Response): Promise<void> {
// 1. Origin lock. Reject anything that isn't same-origin to
// `config.origin` or in the explicit per-deployment allowlist.
// No Origin header → reject (this endpoint is browser-only by
// design; server-side callers should re-auth properly).
const reqOrigin = req.headers.origin;
if (!reqOrigin || !this.#isMigrateTokenOriginAllowed(reqOrigin)) {
throw new HttpError(403, 'Origin not allowed', {
legacyCode: 'forbidden',
});
}
// 2. Extract token. Header takes precedence so body-capture logs
// (if any) never see the credential.
const authHeader = req.headers.authorization;
const headerToken =
typeof authHeader === 'string' && authHeader.startsWith('Bearer ')
? authHeader.slice('Bearer '.length).trim()
: null;
const bodyToken =
typeof req.body?.token === 'string' ? req.body.token.trim() : null;
const v1Token = headerToken || bodyToken;
if (!v1Token) {
throw new HttpError(400, 'Missing token', {
legacyCode: 'bad_request',
});
}
const result = await this.services.auth.migrateLegacyToken(v1Token, {
ip: req.ip,
userAgent:
typeof req.headers['user-agent'] === 'string'
? req.headers['user-agent']
: undefined,
});
// `puter_token_v2` is the cookie companion to v2 app tokens —
// the app runs in-browser (we already gated on Origin above) so
// the GUI's cookie-only middleware can authenticate subsequent
// calls from the same iframe without the client having to
// forward Authorization headers. Access tokens are programmatic
// (no browser cookie surface) so we deliberately skip them here.
if (result.kind === 'app') {
res.cookie('puter_token_v2', result.token, {
...sessionCookieFlags(this.config),
httpOnly: true,
});
}
res.json(result);
}
#isMigrateTokenOriginAllowed(origin: string): boolean {
// Origin headers and config values both reach us in inconsistent
// shapes (trailing slash, mixed case from misconfigured deploys,
// stray whitespace from JSON config edits). Normalize both sides
// before equality so a `config.origin` with a trailing slash
// doesn't reject every same-origin browser call.
const normalize = (raw: string | undefined): string =>
(raw ?? '').trim().replace(/\/+$/, '').toLowerCase();
const incoming = normalize(origin);
if (!incoming) return false;
if (incoming === normalize(this.config.origin)) return true;
const allowlist = (
this.config as { allow_migrate_token_origins?: string[] }
).allow_migrate_token_origins;
if (!Array.isArray(allowlist)) return false;
return allowlist.some((entry) => normalize(entry) === incoming);
}
@Post('/auth/create-access-token', {
subdomain: 'api',
@@ -2029,10 +2238,10 @@ export class AuthController extends PuterController {
res.json({ token });
}
@Post('/auth/revoke-access-token', {
subdomain: 'api',
requireUserActor: true,
})
// Wired imperatively in `registerRoutes` so the cookie-only gate
// (built from `this.config`) can be composed in. Cookie-only is
// mandatory: a leaked access token must not be able to silently
// revoke its own siblings.
async handleRevokeAccessToken(req: Request, res: Response): Promise<void> {
let { tokenOrUuid } = req.body;
if (!tokenOrUuid || typeof tokenOrUuid !== 'string') {
@@ -2049,7 +2258,7 @@ export class AuthController extends PuterController {
res.json({ ok: true });
}
// ── 2FA: configure ──────────────────────────────────────────────
// -- 2FA: configure ----------------------------------------------
@Post('/auth/configure-2fa/:action', {
subdomain: 'api',
@@ -2128,8 +2337,8 @@ export class AuthController extends PuterController {
}
await this.clients.db.write(
'UPDATE `user` SET `otp_enabled` = 1 WHERE `uuid` = ?',
[user.uuid],
'UPDATE `user` SET `otp_enabled` = ? WHERE `uuid` = ?',
[this.clients.db.booleanValue(true), user.uuid],
);
await this.stores.user.invalidateById(user.id);
@@ -2152,7 +2361,7 @@ export class AuthController extends PuterController {
});
}
// ── 2FA: disable (user-protected, wired in registerRoutes below) ─
// -- 2FA: disable (user-protected, wired in registerRoutes below) -
async handleDisable2fa(req: Request, res: Response): Promise<void> {
const user = await this.stores.user.getById(req.actor!.user.id!, {
@@ -2164,8 +2373,8 @@ export class AuthController extends PuterController {
});
await this.clients.db.write(
'UPDATE `user` SET `otp_enabled` = 0, `otp_recovery_codes` = NULL, `otp_secret` = NULL WHERE `uuid` = ?',
[user.uuid],
'UPDATE `user` SET `otp_enabled` = ?, `otp_recovery_codes` = NULL, `otp_secret` = NULL WHERE `uuid` = ?',
[this.clients.db.booleanValue(false), user.uuid],
);
await this.stores.user.invalidateById(user.id);
@@ -2182,7 +2391,7 @@ export class AuthController extends PuterController {
res.json({ success: true });
}
// ── Developer profile ───────────────────────────────────────────
// -- Developer profile -------------------------------------------
@Get('/get-dev-profile', { subdomain: 'api', requireUserActor: true })
async handleGetDevProfile(req: Request, res: Response): Promise<void> {
@@ -2212,7 +2421,7 @@ export class AuthController extends PuterController {
});
}
// ── Group management ────────────────────────────────────────────
// -- Group management --------------------------------------------
@Post('/group/create', { subdomain: 'api', requireUserActor: true })
async handleGroupCreate(req: Request, res: Response): Promise<void> {
@@ -2318,7 +2527,7 @@ export class AuthController extends PuterController {
});
}
// ── Session helpers ─────────────────────────────────────────────
// -- Session helpers ---------------------------------------------
@Get('/get-gui-token', {
requireUserActor: true,
@@ -2359,14 +2568,14 @@ export class AuthController extends PuterController {
user,
req.actor.session.uid,
);
res.cookie(this.config.cookie_name, sessionToken, {
res.cookie(this.config.cookie_name ?? 'puter_token', sessionToken, {
...sessionCookieFlags(this.config),
httpOnly: true,
});
res.status(204).end();
}
// ── Delete own account (user-protected, wired below) ────────────
// -- Delete own account (user-protected, wired below) ------------
//
// Purge S3 objects + fsentries first, then the user row. FK
// cascades on most related tables are `ON DELETE SET NULL` (not
@@ -2375,13 +2584,14 @@ export class AuthController extends PuterController {
async handleDeleteOwnUser(req: Request, res: Response): Promise<void> {
const userId = req.actor!.user.id!;
res.clearCookie(this.config.cookie_name!);
res.clearCookie(this.config.cookie_name ?? 'puter_token');
res.clearCookie('puter_token_v2');
res.clearCookie('puter_revalidation');
await this.#cascadeDeleteUser(userId);
res.json({ success: true });
}
// ── registerRoutes override ─────────────────────────────────────
// -- registerRoutes override -------------------------------------
//
// The `@Controller('')` decorator would normally install a default
// `registerRoutes` walker that iterates `prototype[__puterRoutes]`.
@@ -2424,7 +2634,7 @@ export class AuthController extends PuterController {
routerMethod.call(router, r.path, r.options, bound);
}
// ── User-protected routes (per-instance middleware) ──────────
// -- User-protected routes (per-instance middleware) ----------
const userProtectedDeps = {
config: this.config,
userStore: this.stores.user,
@@ -2522,9 +2732,77 @@ export class AuthController extends PuterController {
},
(req, res) => this.handleDeleteOwnUser(req, res),
);
const webSessionGate = createWebSessionActorGate();
router.post(
'/auth/revoke-session',
{
subdomain: 'api',
requireUserActor: true,
allowUnconfirmed: true,
antiCsrf: true,
middleware: [webSessionGate],
},
(req, res) => this.handleRevokeSession(req, res),
);
router.post(
'/auth/revoke-all-sessions',
{
subdomain: 'api',
requireUserActor: true,
allowUnconfirmed: true,
antiCsrf: true,
rateLimit: {
scope: 'revoke-all-sessions',
limit: 10,
window: 60 * 60_000,
key: 'user',
},
middleware: [webSessionGate],
},
(req, res) => this.handleRevokeAllSessions(req, res),
);
router.post(
'/auth/revoke-access-token',
{
subdomain: 'api',
requireUserActor: true,
antiCsrf: true,
middleware: [webSessionGate],
},
(req, res) => this.handleRevokeAccessToken(req, res),
);
router.patch(
'/auth/sessions/:uuid/label',
{
subdomain: 'api',
requireUserActor: true,
allowUnconfirmed: true,
antiCsrf: true,
middleware: [webSessionGate],
},
(req, res) => this.handleRenameSession(req, res),
);
router.post(
'/auth/migrate-token',
{
rateLimit: {
scope: 'migrate-token',
limit: 20,
window: 15 * 60_000,
key: 'ip',
},
},
(req, res) => this.handleMigrateToken(req, res),
);
}
// ── Private helpers ──────────────────────────────────────────────
// -- Private helpers ----------------------------------------------
async #cascadeDeleteUser(userId: number): Promise<void> {
try {
@@ -2609,6 +2887,89 @@ export class AuthController extends PuterController {
}
}
/**
* Per-IP enumeration clamp on `auth_id`-bearing auth requests. Separate
* from the route-level rate limit so a tighter ceiling applies only to
* the path that takes a uuid hint from the body — the normal login
* path stays at its more generous limit.
*/
async #checkAuthIdRateLimit(req: Request): Promise<void> {
const ip = req.ip || req.socket?.remoteAddress || 'unknown';
const ok = await checkRateLimit(
`login-with-auth-id:${ip}`,
5,
15 * 60_000,
);
if (!ok) {
throw new HttpError(429, 'Too many auth_id login attempts.', {
legacyCode: 'too_many_requests',
fields: { 'retry-after': 900 },
});
}
}
/**
* Extract the `auth_id` claim from a client-supplied reauth_token.
* Returns null when no token was supplied. Throws on invalid/expired
* tokens. The reauth_token is a server-signed JWT minted by the
* authProbe at 401 time — accepting only the signed envelope (vs. a
* raw UUID) means a leaked auth_id alone can't attach a session to
* an existing account.
*/
#extractAuthIdFromReauthToken(suppliedToken: unknown): string | null {
if (suppliedToken === undefined || suppliedToken === null) return null;
if (typeof suppliedToken !== 'string' || !suppliedToken) {
throw new HttpError(400, 'Invalid `reauth_token`.', {
legacyCode: 'bad_request',
});
}
const { authId } = this.services.auth.verifyReauthToken(suppliedToken);
return authId;
}
/**
* Enforce a verified `auth_id` against the user the credential flow
* has resolved. Caller has already extracted `auth_id` from the
* server-signed reauth_token (or from an OTP-flow JWT). When the GUI
* is forced through reauth, the 401 response embeds the reauth_token;
* the client echoes it back so we can confirm the second login lands
* on the same user row — critical for temp users, where a fresh
* signup would otherwise mint a new account and strand their files.
*
* No `authId` supplied → no-op (normal login). Unknown `authId` →
* 404 (mirrors username-not-found, avoids being an enumeration
* oracle). Mismatch against the resolved user → 409
* (`auth_id_mismatch`).
*/
async #enforceAuthIdMatch(
req: Request,
resolvedUser: { id: number; uuid: string },
authId: string | null,
): Promise<void> {
if (!authId) return;
await this.#checkAuthIdRateLimit(req);
// Common path: auth_id maps to the same uuid the credential flow
// resolved. Skip the user-table read entirely. The DB lookup is
// only needed to disambiguate 404 (unknown auth_id) from 409
// (known but mismatched) on the error path.
if (authId === resolvedUser.uuid) return;
const authIdUser = await this.stores.user.getByUuid(authId);
if (!authIdUser) {
throw new HttpError(404, 'auth_id not found.', {
legacyCode: 'not_found',
});
}
if (authIdUser.id !== resolvedUser.id) {
throw new HttpError(409, 'auth_id does not match credentials.', {
legacyCode: 'bad_request',
fields: { code: 'auth_id_mismatch' },
});
}
}
async #completeLogin(
req: Request,
res: Response,
@@ -2633,7 +2994,7 @@ export class AuthController extends PuterController {
await this.services.auth.createSessionToken(user as never, meta);
// HTTP-only cookie gets the session token
res.cookie(this.config.cookie_name, sessionToken, {
res.cookie(this.config.cookie_name ?? 'puter_token', sessionToken, {
...sessionCookieFlags(this.config),
httpOnly: true,
});
@@ -44,7 +44,7 @@ export class DesktopController extends PuterController {
}
registerRoutes(router) {
// ── Desktop background ──────────────────────────────────────
// -- Desktop background --------------------------------------
router.post(
'/set-desktop-bg',
@@ -98,7 +98,7 @@ export class DesktopController extends PuterController {
},
);
// ── Taskbar items ───────────────────────────────────────────
// -- Taskbar items -------------------------------------------
router.post(
'/update-taskbar-items',
@@ -123,7 +123,7 @@ export class DesktopController extends PuterController {
},
);
// ── Folder layout ───────────────────────────────────────────
// -- Folder layout -------------------------------------------
router.post(
'/set_layout',
@@ -149,7 +149,7 @@ export class DesktopController extends PuterController {
},
);
// ── Folder sort ─────────────────────────────────────────────
// -- Folder sort ---------------------------------------------
router.post(
'/set_sort_by',
@@ -188,7 +188,7 @@ export class DesktopController extends PuterController {
);
}
// ── Helpers ──────────────────────────────────────────────────────
// -- Helpers ------------------------------------------------------
/**
* Update columns on an actor-owned fsentry. Accepts either `item_uid`
@@ -124,11 +124,13 @@ const makeSyntheticDriver = () => ({
const buildController = (driver: ReturnType<typeof makeSyntheticDriver>) => {
// The controller reads `this.services?.permission` only when an actor
// is on the request; otherwise the services bag is unused. Empty
// objects are sufficient for this path.
// is on the request; otherwise the services bag is unused. The
// alarm client is invoked on the rate-limit / concurrency rejection
// paths, so stub a no-op `create`.
const clients = { alarm: { create: () => {} } };
return new DriverController(
{} as any,
{} as any,
clients as any,
{} as any,
{} as any,
{ syntheticDriver: driver } as any,
@@ -33,7 +33,7 @@ import type { DriverController } from './DriverController.js';
// The DriverController under test is the same instance the live request
// pipeline uses, so its iface→driver registry is populated from real
// drivers (puter-kvstore, puter-apps, puter-subdomains, …). The HTTP
// handlers (`#handleCall`, `#handleListInterfaces`, `#handleXd`) are
// handlers (`#handleCall`, `#handleListInterfaces`) are
// private — we exercise the public lookup API (`resolve` / list / get
// default) which the handlers themselves delegate to.
@@ -116,7 +116,7 @@ describe('DriverController.resolve', () => {
});
});
// ── Route handlers (#handleCall, #handleListInterfaces, #handleXd) ─
// ── Route handlers (#handleCall, #handleListInterfaces) ─────────────
// The handlers are private class fields. We capture references to them by
// invoking `registerRoutes` with a fake router whose `post`/`get` save the
@@ -189,10 +189,7 @@ const makeRes = (): MockRes => {
return res;
};
const makeReq = (
body: Record<string, unknown> = {},
actor?: Actor,
): Request =>
const makeReq = (body: Record<string, unknown> = {}, actor?: Actor): Request =>
({
body,
actor,
@@ -338,11 +335,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
res as unknown as Response,
() => {},
),
routes['POST /call'](req, res as unknown as Response, () => {}),
);
const body = res.body as {
success: boolean;
@@ -374,11 +367,9 @@ describe('DriverController.#handleCall (via captured router)', () => {
// Register it through the controller's private map by re-running
// its #buildIfaceMap path: easier to just stash it into the
// existing iface map directly via TypeScript-defeating cast.
const internalDrivers = (
controller as unknown as {
['#drivers']: Map<string, Map<string, unknown>>;
}
);
const internalDrivers = controller as unknown as {
['#drivers']: Map<string, Map<string, unknown>>;
};
// Access the actual private slot via the well-known getter
// pattern doesn't work for `#`-private fields; instead, re-run
// registerRoutes after stashing on the bag — but that's already
@@ -413,13 +404,12 @@ describe('DriverController.#handleCall (via captured router)', () => {
>;
const original = kv.set;
kv.set = async function (...args: unknown[]) {
const { Context } = await import(
'../../core/context.js'
);
const { Context } = await import('../../core/context.js');
Context.set('driverMetadata', { providerUsed: 'kv-direct' });
return (
original as (...x: unknown[]) => Promise<unknown>
).apply(this, args);
return (original as (...x: unknown[]) => Promise<unknown>).apply(
this,
args,
);
};
try {
const res = makeRes();
@@ -432,11 +422,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
res as unknown as Response,
() => {},
),
routes['POST /call'](req, res as unknown as Response, () => {}),
);
const body = res.body as {
metadata?: Record<string, unknown>;
@@ -497,31 +483,11 @@ describe('DriverController.#handleListInterfaces', () => {
});
});
describe('DriverController.#handleXd', () => {
let routes: Captured;
beforeAll(() => {
routes = captureRoutes(controller);
});
it('serves the iframe-bridge HTML with text/html content-type', () => {
const res = makeRes();
routes['GET /xd'](
makeReq(),
res as unknown as Response,
() => {},
);
expect(res.contentType).toBe('text/html');
expect(res.sentBody).toMatch(/<!DOCTYPE html>/);
expect(res.sentBody).toMatch(/\/drivers\/call/);
});
});
describe('DriverController.registerRoutes', () => {
it('registers POST /call, GET /list-interfaces, and GET /xd', () => {
it('registers POST /call and GET /list-interfaces', () => {
const routes = captureRoutes(controller);
expect(routes['POST /call']).toBeInstanceOf(Function);
expect(routes['GET /list-interfaces']).toBeInstanceOf(Function);
expect(routes['GET /xd']).toBeInstanceOf(Function);
});
});
@@ -26,8 +26,6 @@ import {
checkDriverRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import type { WithLifecycle } from '../../types';
import type { DriverMeta } from '../../drivers/meta.js';
import {
isDriverStreamResult,
@@ -35,124 +33,49 @@ import {
resolveDriverMethodConcurrent,
resolveDriverMethodRateLimit,
} from '../../drivers/meta.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import type { WithLifecycle } from '../../types';
import { PuterController } from '../types.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
// ── /drivers/xd payload ─────────────────────────────────────────────
//
// Self-contained HTML/JS shipped to the iframe consumer. Listens for
// postMessage events shaped as `{ id, interface, method, params }`,
// forwards to `/drivers/call`, and posts `{ id, result }` back to the
// originating window.
//
// Wire-shape note: the postMessage uses `params` (puter-js's historical
// name) but `/drivers/call` expects `args`; this bridge translates.
const extractUpstreamStatus = (e: {
status?: number;
statusCode?: number;
response?: { status?: number };
$metadata?: { httpStatusCode?: number };
message?: string;
}): number | undefined => {
const direct = e.status ?? e.statusCode;
if (typeof direct === 'number') return direct;
const fromResponse = e.response?.status;
if (typeof fromResponse === 'number') return fromResponse;
const fromAws = e.$metadata?.httpStatusCode;
if (typeof fromAws === 'number') return fromAws;
// Message sniff (e.g. "... failed with status 422 ...").
// Only trust if it's adjacent to a status-indicating word to
// avoid matching random 4xx/5xx-looking numbers in payloads.
const msg = e.message;
if (typeof msg === 'string') {
const m = msg.match(/\bstatus(?:\s+code)?\s*[:=]?\s*(4\d\d|5\d\d)\b/i);
if (m) return Number(m[1]);
}
return undefined;
};
const XD_SCRIPT = /* js */ `
(function () {
const call = async ({ interface_name, method_name, params }) => {
const response = await fetch('/drivers/call', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
interface: interface_name,
method: method_name,
args: params,
}),
});
return await response.json();
};
const fcall = async ({ interface_name, method_name, params }) => {
const form = new FormData();
form.append('interface', interface_name);
form.append('method', method_name);
for (const k in params) {
form.append(k, params[k]);
}
const response = await fetch('/drivers/call', {
method: 'POST',
body: form,
});
return await response.json();
};
window.addEventListener('message', async (event) => {
const { id, interface: iface, method, params } = event.data || {};
let has_file = false;
for (const k in params) {
if (params[k] instanceof File) {
has_file = true;
break;
}
}
const result = has_file
? await fcall({ interface_name: iface, method_name: method, params })
: await call({ interface_name: iface, method_name: method, params });
if (event.source) {
event.source.postMessage({ id, result }, event.origin);
}
});
})();
`;
const XD_HTML = `<!DOCTYPE html>
<html>
<head>
<title>Puter Driver API</title>
<script>
document.addEventListener('DOMContentLoaded', function () {
${XD_SCRIPT}
});
</script>
</head>
<body></body>
</html>`;
// ── Controller ──────────────────────────────────────────────────────
/**
* Routes driver RPC calls through a unified HTTP surface.
*
* - `POST /drivers/call` — invoke `<iface>.<method>(args)` on the
* registered driver after a per-actor permission + rate-limit check.
* Stream-shaped results are piped directly; everything else is
* returned as JSON.
* - `GET /drivers/list-interfaces` — enumerate registered driver
* interfaces with their default + alternate implementations.
* - `GET /drivers/xd` — legacy iframe bridge; serves an HTML page that
* proxies `postMessage` RPCs to `/drivers/call` on the same origin.
*
* Holds an internal iface → driverName → instance map built at
* construction time from `this.drivers`. Extensions that register
* additional drivers end up in that bag before this controller is
* instantiated, so they show up here automatically.
*/
/**
* Catch-all upstream-error translator for the driver boundary.
*
* Drivers that wrap a third-party SDK (OpenAI, Anthropic, etc.) often
* let the SDK's own error class bubble — those carry an HTTP `.status`
* but are plain `Error` subclasses, not `HttpError`s, so they would
* otherwise hit the global error handler as unexpected 500s and page
* PagerDuty. Repackage them with `upstream_*` legacy codes so the
* alarm gate (server.ts) treats them as upstream failures and only
* pages on the two we actually care about (rate-limit / auth).
*
* `HttpError`s thrown by drivers pass through untouched.
*/
const translateProviderError = (err: unknown): unknown => {
if (isHttpError(err)) return err;
if (!err || typeof err !== 'object') return err;
const e = err as {
status?: number;
statusCode?: number;
response?: { status?: number };
$metadata?: { httpStatusCode?: number };
message?: string;
error?: { code?: string; type?: string; message?: string };
code?: string;
};
const status = e.status ?? e.statusCode;
const status = extractUpstreamStatus(e);
if (typeof status !== 'number') return err;
const msg = e.error?.message ?? e.message ?? 'Upstream provider error';
@@ -203,7 +126,7 @@ export class DriverController extends PuterController {
this.#buildIfaceMap();
}
// ── Lookup API (used by tests / internals) ──────────────────────
// -- Lookup API (used by tests / internals) ----------------------
/** Resolve a driver by interface + optional name (default when omitted). */
resolve(interfaceName: string, driverName?: string): DriverInstance | null {
@@ -227,7 +150,7 @@ export class DriverController extends PuterController {
return this.#defaults.get(interfaceName);
}
// ── Route registration ──────────────────────────────────────────
// -- Route registration ------------------------------------------
registerRoutes(router: PuterRouter): void {
router.post(
@@ -240,14 +163,9 @@ export class DriverController extends PuterController {
{ subdomain: 'api', requireAuth: true },
this.#handleListInterfaces,
);
router.get(
'/xd',
{ subdomain: 'api', requireAuth: true },
this.#handleXd,
);
}
// ── Handlers ────────────────────────────────────────────────────
// -- Handlers ----------------------------------------------------
#handleCall = async (req: Request, res: Response): Promise<void> => {
const {
@@ -335,6 +253,19 @@ export class DriverController extends PuterController {
if (
!(await checkDriverRateLimit(req, ifaceName, method, rateLimitSpec))
) {
// De-dupe on (iface, method) so a hot loop across many users
// aggregates as occurrences on a single low-severity alarm
// instead of fanning out one per user.
this.clients.alarm.create(
`driver_rate_limit_hit:${ifaceName}:${method}`,
`Driver rate limit hit on ${ifaceName}:${method}`,
{
iface: ifaceName,
method,
userUuid: req.actor?.user?.uuid,
},
'info',
);
throw new HttpError(429, 'Too many requests.', {
legacyCode: 'too_many_requests',
});
@@ -357,6 +288,16 @@ export class DriverController extends PuterController {
concurrentSpec,
);
if (!handle.ok) {
this.clients.alarm.create(
`driver_concurrent_limit_hit:${ifaceName}:${method}`,
`Driver concurrency limit hit on ${ifaceName}:${method}`,
{
iface: ifaceName,
method,
userUuid: req.actor?.user?.uuid,
},
'info',
);
throw new HttpError(429, 'Too many concurrent requests.', {
legacyCode: 'too_many_requests',
});
@@ -433,12 +374,7 @@ export class DriverController extends PuterController {
res.json(out);
};
#handleXd = (_req: Request, res: Response): void => {
res.type('text/html');
res.send(XD_HTML);
};
// ── Internals ───────────────────────────────────────────────────
// -- Internals ---------------------------------------------------
#buildIfaceMap(): void {
const bag = this.drivers as unknown as Record<string, DriverInstance>;
@@ -434,6 +434,119 @@ describe('FSController.completeBatchWrites', () => {
expect(finalized?.wasOverwrite).toBe(true);
});
it('emits updated events with GUI metadata when overwriting via batch completion', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/overwrite-event.js`;
const firstStart = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path: target, size: 1 } }],
actor,
}),
firstStart.res,
),
);
const [firstResponse] = firstStart.captured
.body as SignedWriteResponse[];
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: firstResponse!.sessionId }],
actor,
}),
makeRes().res,
),
);
const targetEntry = await server.stores.fsEntry.getEntryByPath(target);
expect(targetEntry).not.toBeNull();
await server.stores.subdomain.create({
userId,
subdomain: `workers.puter.${username}-worker`,
rootDirId: targetEntry!.id,
});
const secondStart = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [
{
fileMetadata: {
path: target,
size: 2,
overwrite: true,
},
},
],
actor,
}),
secondStart.res,
),
);
const [secondResponse] = secondStart.captured
.body as SignedWriteResponse[];
const emitSpy = vi.spyOn(server.clients.event, 'emit');
let updatedCall:
| (typeof emitSpy.mock.calls)[number]
| undefined;
try {
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [
{
uploadId: secondResponse!.sessionId,
guiMetadata: {
operationId: 'op-123',
itemUploadId: 'item-456',
socketId: 'socket-789',
originalClientSocketId: 'socket-789',
},
},
],
actor,
}),
makeRes().res,
),
);
updatedCall = emitSpy.mock.calls.find(
([eventName]) => eventName === 'outer.gui.item.updated',
);
} finally {
emitSpy.mockRestore();
}
expect(updatedCall).toBeTruthy();
const payload = updatedCall?.[1] as {
user_id_list?: number[];
response?: Record<string, unknown>;
};
expect(payload.user_id_list).toEqual([userId]);
expect(payload.response).toMatchObject({
uid: expect.any(String),
uuid: expect.any(String),
id: expect.any(String),
path: target,
name: 'overwrite-event.js',
is_dir: false,
type: expect.stringMatching(/^application\/javascript/),
workers: [
expect.objectContaining({
subdomain: `workers.puter.${username}-worker`,
address: expect.stringContaining(`${username}-worker`),
}),
],
from_new_service: true,
operation_id: 'op-123',
item_upload_id: 'item-456',
socket_id: 'socket-789',
original_client_socket_id: 'socket-789',
});
});
it("rejects another user's session ids with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
@@ -678,6 +791,87 @@ describe('FSController.searchEntries', () => {
expect(Array.isArray(results)).toBe(true);
expect(results.some((r) => r.name === needle)).toBe(true);
});
it('scopes app-under-user actors to their AppData root', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
// User-owned entry outside AppData — must NOT appear for the app.
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: `/${username}/Documents/${needle}` },
actor: userActor,
}),
makeRes().res,
),
);
// Entry under the app's own AppData — must appear.
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: {
path: `/${username}/AppData/${appUid}/${needle}`,
create_missing_parents: true,
},
actor: userActor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.searchEntries(
makeReq({ body: { query: needle }, actor: appActor }),
res,
),
);
const results = captured.body as Array<{ name: string; path: string }>;
expect(results.length).toBeGreaterThan(0);
for (const r of results) {
expect(
r.path === `/${username}/AppData/${appUid}` ||
r.path.startsWith(`/${username}/AppData/${appUid}/`),
).toBe(true);
}
expect(
results.some(
(r) => r.path === `/${username}/Documents/${needle}`,
),
).toBe(false);
});
it('returns nothing for an app actor when no AppData entries match', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
// Only seed outside AppData — the app must not be able to find it.
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: `/${username}/Documents/${needle}` },
actor: userActor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.searchEntries(
makeReq({ body: { query: needle }, actor: appActor }),
res,
),
);
expect(captured.body).toEqual([]);
});
});
// ── /read (readEntry, validation paths) ─────────────────────────────
@@ -759,6 +953,82 @@ describe('FSController.mkdirEntry', () => {
expect(body.isDir).toBe(true);
});
it('dedupes an existing directory when dedupe_name is true', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/hello`;
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target, dedupe_name: true },
actor,
}),
res,
),
);
const body = captured.body as {
path: string;
name: string;
isDir: boolean;
};
expect(body.path).toBe(`/${username}/Documents/hello (1)`);
expect(body.name).toBe('hello (1)');
expect(body.isDir).toBe(true);
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/hello (1)`,
),
).toMatchObject({ isDir: true });
});
it('requires parent write when deduping an existing directory', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const target = `/${username}/AppData/${appUid}`;
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target },
actor: userActor,
}),
makeRes().res,
),
);
await expect(
withActor(appActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target, dedupe_name: true },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/AppData/${appUid} (1)`,
),
).toBeNull();
});
it('expands ~/ in the path to the user home', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
+52 -83
View File
@@ -20,12 +20,13 @@
import Busboy from 'busboy';
import type { Request, Response } from 'express';
import { posix as pathPosix } from 'node:path';
import { assertNormalized } from '../../services/fs/resolveNode.js';
import { pipeline } from 'node:stream/promises';
import type { Actor } from '../../core/actor.js';
import { effectiveActorApp } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import { assertNormalized } from '../../services/fs/resolveNode.js';
import type {
PreparedBatchWrite,
UploadedBatchWriteItem,
@@ -58,6 +59,7 @@ import type {
ThumbnailUploadPrepareItem,
ThumbnailUploadPreparePayload,
} from './types.js';
import { toLegacyEntry } from './legacyFsHelpers.js';
class UploadProgressTracker implements UploadProgressTrackerLike {
total = 0;
progress = 0;
@@ -421,7 +423,12 @@ export class FSController extends PuterController {
const busboy = Busboy({ headers: req.headers });
busboy.on('field', (fieldName, value, info) => {
if (info.fieldnameTruncated || info.valueTruncated) {
if (
(info as unknown as { filenameTruncated: string })
.filenameTruncated ||
info.nameTruncated ||
info.valueTruncated
) {
failParse(
new HttpError(
400,
@@ -810,7 +817,7 @@ export class FSController extends PuterController {
res.json(updatedResponses);
}
// ── Read-side routes ────────────────────────────────────────────────
// -- Read-side routes ------------------------------------------------
@Post('/stat', { subdomain: 'api', requireVerified: true })
async statEntry(req: Request, res: Response) {
@@ -910,7 +917,7 @@ export class FSController extends PuterController {
@Post('/search', { subdomain: 'api', requireVerified: true })
async searchEntries(req: Request, res: Response) {
this.#requireActor(req);
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
const body = this.#toObjectRecord(req.body);
const query =
@@ -929,6 +936,7 @@ export class FSController extends PuterController {
userId,
query,
limit ?? 200,
this.#appDataScopeForActor(actor),
);
res.json(results);
}
@@ -1007,7 +1015,7 @@ export class FSController extends PuterController {
}
}
// ── Mutation routes ────────────────────────────────────────────────
// -- Mutation routes ------------------------------------------------
@Post('/mkdir', { subdomain: 'api', requireVerified: true })
async mkdirEntry(req: Request, res: Response) {
@@ -1031,13 +1039,15 @@ export class FSController extends PuterController {
legacyCode: 'bad_request',
});
const dedupeName =
this.#toBoolean(body.dedupe_name ?? body.dedupeName) ?? false;
await this.#assertCanCreate(actor, path);
if (dedupeName) await this.#assertCanDedupeCreate(actor, path);
const entry = await this.services.fs.mkdir(userId, {
path,
overwrite: this.#toBoolean(body.overwrite) ?? false,
dedupeName:
this.#toBoolean(body.dedupe_name ?? body.dedupeName) ?? false,
dedupeName,
createMissingParents:
this.#toBoolean(
body.create_missing_parents ??
@@ -1205,7 +1215,7 @@ export class FSController extends PuterController {
res.json(shortcut);
}
// ── Read-side helpers ───────────────────────────────────────────────
// -- Read-side helpers -----------------------------------------------
#requireActor(req: Request): Actor {
const actor = req.actor;
@@ -1302,6 +1312,17 @@ export class FSController extends PuterController {
await this.#assertAccess(actor, parentForCheck, 'write');
}
async #assertCanDedupeCreate(actor: Actor, targetPath: string) {
const existing = await this.stores.fsEntry.getEntryByPath(targetPath);
if (!existing) return;
const parent = pathPosix.dirname(targetPath);
await this.#assertAccess(
actor,
parent === '/' ? targetPath : parent,
'write',
);
}
async #assertAccess(
actor: Actor,
path: string,
@@ -1649,24 +1670,6 @@ export class FSController extends PuterController {
normalizedFileMetadata.multipartPartSize = multipartPartSize;
}
const bucket = this.#firstDefined(
metadataRecord.bucket,
fallbackRecord.bucket,
);
if (typeof bucket === 'string' && bucket.length > 0) {
normalizedFileMetadata.bucket = bucket;
}
const bucketRegion = this.#firstDefined(
metadataRecord.bucketRegion,
metadataRecord.bucket_region,
fallbackRecord.bucketRegion,
fallbackRecord.bucket_region,
);
if (typeof bucketRegion === 'string' && bucketRegion.length > 0) {
normalizedFileMetadata.bucketRegion = bucketRegion;
}
const associatedAppId = this.#toNumber(
this.#firstDefined(
metadataRecord.associatedAppId,
@@ -2034,56 +2037,7 @@ export class FSController extends PuterController {
}
async #toGuiFsEntry(entry: FSEntry): Promise<Record<string, unknown>> {
const dirpath = pathPosix.dirname(entry.path);
const extension = pathPosix.extname(entry.name).slice(1).toLowerCase();
const response = {
id: entry.uuid,
uid: entry.uuid,
uuid: entry.uuid,
parent_id: entry.parentUid,
parent_uid: entry.parentUid,
path: entry.path,
dirname: dirpath,
dirpath,
name: entry.name,
is_dir: entry.isDir,
is_shortcut: entry.isShortcut ? 1 : 0,
shortcut_to: entry.shortcutTo,
type: entry.isDir ? 'folder' : extension,
writable: true,
is_public: entry.isPublic,
thumbnail: entry.thumbnail,
immutable: entry.immutable,
metadata: entry.metadata,
modified: entry.modified,
created: entry.created,
accessed: entry.accessed,
size: entry.size,
};
if (
typeof response.thumbnail === 'string' &&
response.thumbnail.length > 0
) {
const thumbnailEntry = {
uuid: entry.uuid,
thumbnail: response.thumbnail,
};
// emitAndWait — listener rewrites s3:// / legacy URLs into
// time-limited signed URLs on the payload object.
await this.clients.event.emitAndWait(
'thumbnail.read',
thumbnailEntry,
{},
);
response.thumbnail =
typeof thumbnailEntry.thumbnail === 'string' &&
thumbnailEntry.thumbnail.length > 0
? thumbnailEntry.thumbnail
: null;
}
return response;
return toLegacyEntry(this.clients.event, entry);
}
async #emitGuiWriteEvent(
@@ -2093,7 +2047,7 @@ export class FSController extends PuterController {
): Promise<void> {
const response = {
...(await this.#toGuiFsEntry(fsEntry)),
...this.#toEventGuiMetadata(guiMetadata, false),
...this.#toEventGuiMetadata(guiMetadata),
from_new_service: true,
};
await this.clients.event.emit(
@@ -2144,6 +2098,20 @@ export class FSController extends PuterController {
return pathParts.length >= 2 && pathParts[1] === 'AppData';
}
// If `actor` is effectively scoped to an app (directly or through an
// access-token issuer chain), return that app's AppData root under the
// actor's user. Returns undefined for pure user actors. The store anchors
// search results to this path so app actors can't see entries outside
// their AppData via `/fs/search`.
#appDataScopeForActor(actor: Actor): string | undefined {
const app = effectiveActorApp(actor);
if (!app) return undefined;
const username = actor.user?.username;
if (typeof username !== 'string' || username.length === 0)
return undefined;
return `/${username}/AppData/${app.uid}`;
}
#estimateDataUrlSize(dataUrl: string): number {
const commaIndex = dataUrl.indexOf(',');
const base64 =
@@ -2237,7 +2205,7 @@ export class FSController extends PuterController {
return null;
}
return { index, contentType, size };
return { index, contentType, size } as ThumbnailUploadPrepareItem;
}
async #attachSignedThumbnailUploadTargets(
@@ -2257,11 +2225,12 @@ export class FSController extends PuterController {
const payload: ThumbnailUploadPreparePayload = {
items: prepareItems.map(
(item): ThumbnailUploadPrepareItem => ({
index: item.index,
contentType: item.contentType,
...(item.size !== undefined ? { size: item.size } : {}),
}),
(item): ThumbnailUploadPrepareItem =>
({
index: item.index,
contentType: item.contentType,
...(item.size !== undefined ? { size: item.size } : {}),
}) as ThumbnailUploadPrepareItem,
),
};
// emitAndWait — listeners populate `uploadUrl` / `thumbnailUrl` on
@@ -206,6 +206,78 @@ describe('LegacyFSController.mkdir', () => {
expect(fetched?.isDir).toBe(true);
});
it('dedupes an existing directory when dedupe_name is true', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const parent = `/${username}/Documents`;
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { parent, path: 'hello' },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { parent, path: 'hello', dedupe_name: true },
actor,
}),
res,
),
);
const body = captured.body as Record<string, unknown>;
expect(body).toMatchObject({
path: `${parent}/hello (1)`,
name: 'hello (1)',
is_dir: true,
});
expect(
await server.stores.fsEntry.getEntryByPath(`${parent}/hello (1)`),
).toMatchObject({ isDir: true });
});
it('requires parent write when deduping an existing directory', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const parent = `/${username}/AppData`;
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: { parent, path: appUid },
actor: userActor,
}),
makeRes().res,
),
);
await expect(
withActor(appActor, () =>
controller.mkdir(
makeReq({
body: { parent, path: appUid, dedupe_name: true },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
expect(
await server.stores.fsEntry.getEntryByPath(
`${parent}/${appUid} (1)`,
),
).toBeNull();
});
it("rejects writing into another user's home with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
@@ -813,6 +885,57 @@ describe('LegacyFSController.search', () => {
expect(Array.isArray(results)).toBe(true);
expect(results.some((r) => r.name === needle)).toBe(true);
});
it('scopes app-under-user actors to their AppData root', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: { path: `/${username}/Documents/${needle}` },
actor: userActor,
}),
makeRes().res,
),
);
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: {
path: `/${username}/AppData/${appUid}/${needle}`,
create_missing_parents: true,
},
actor: userActor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.search(
makeReq({ body: { query: needle }, actor: appActor }),
res,
),
);
const results = captured.body as Array<{ path: string }>;
expect(results.length).toBeGreaterThan(0);
for (const r of results) {
expect(
r.path === `/${username}/AppData/${appUid}` ||
r.path.startsWith(`/${username}/AppData/${appUid}/`),
).toBe(true);
}
expect(
results.some(
(r) => r.path === `/${username}/Documents/${needle}`,
),
).toBe(false);
});
});
// ── read (validation paths only) ────────────────────────────────────
@@ -959,6 +1082,76 @@ describe('LegacyFSController.sign', () => {
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('refuses an app actor signing for a different app (403)', async () => {
// An app-under-user actor may only mint a token for its own app;
// requesting a different app's UID is rejected.
const { actor: userActor } = await makeUser();
const targetApp = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `victim-${uuidv4()}`,
title: 'Victim app',
index_url: 'https://example.test/victim.html',
},
{ ownerUserId: userActor.user!.id! },
);
const attackerActor: Actor = {
...userActor,
app: { uid: `attacker-${uuidv4()}` },
};
const { res } = makeRes();
await expect(
withActor(attackerActor, () =>
controller.sign(
makeReq({
body: {
items: [{}],
app_uid: targetApp.uid,
},
actor: attackerActor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 403, legacyCode: 'forbidden' });
});
it('lets an app actor sign for its own app', async () => {
const { actor: userActor } = await makeUser();
const ownApp = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `self-${uuidv4()}`,
title: 'Self app',
index_url: 'https://example.test/self.html',
},
{ ownerUserId: userActor.user!.id! },
);
const appActor: Actor = { ...userActor, app: { uid: ownApp.uid } };
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.sign(
makeReq({
body: { items: [{}], app_uid: ownApp.uid },
actor: appActor,
}),
res,
),
);
const body = captured.body as { token?: string };
expect(typeof body.token).toBe('string');
});
});
// ── writeFile (validation paths) ────────────────────────────────────
@@ -1038,6 +1231,85 @@ describe('LegacyFSController.openItem', () => {
});
});
// ── openItem: write_url stripping for read-only callers ─────────────
//
// Mirrors `/sign` and `/readdir`, which already strip `write_url` when the
// caller only proved read. Without these gates, /open_item handed out a
// valid write signature to read-only sharees — `/writeFile`'s own ACL
// re-check would still reject the write, but the leak shape (a signed
// write URL escaping the access boundary) is the same one those other
// endpoints already defend against.
describe('LegacyFSController.openItem (write_url stripping)', () => {
beforeAll(() => {
(
controller as unknown as { config: { api_base_url?: string } }
).config.api_base_url = 'http://api.test.local';
});
it('returns write_url for the owner (who has write)', async () => {
const { actor } = await makeUser();
const target = `/${actor.user!.username}/Documents/owned.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor }),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.openItem(
makeReq({ body: { path: target }, actor }),
res,
),
);
const body = captured.body as {
signature: { read_url?: string; write_url?: string };
};
expect(body.signature.read_url).toBeDefined();
expect(body.signature.write_url).toBeDefined();
expect(body.signature.write_url).toContain('writeFile');
});
it('strips write_url for a read-only sharee (the fix)', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const target = `/${victim.actor.user!.username}/Documents/shared-ro.txt`;
await withActor(victim.actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor: victim.actor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
await server.services.permission.grantUserUserPermission(
victim.actor,
attacker.actor.user!.username!,
`fs:${entry!.uuid}:read`,
{},
);
const { res, captured } = makeRes();
await withActor(attacker.actor, () =>
controller.openItem(
makeReq({
body: { uid: entry!.uuid },
actor: attacker.actor,
}),
res,
),
);
const body = captured.body as {
signature: { read_url?: string; write_url?: string };
};
// Sharee still gets read_url (they have read).
expect(body.signature.read_url).toBeDefined();
// …but write_url is stripped (they don't have write).
expect(body.signature.write_url).toBeUndefined();
});
});
// ── requestAppRootDir ───────────────────────────────────────────────
describe('LegacyFSController.requestAppRootDir', () => {
@@ -2032,3 +2304,197 @@ describe('LegacyFSController.readdir extras', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── suggestApps (ACL-gated) ─────────────────────────────────────────
describe('LegacyFSController.suggestApps', () => {
it('refuses to look up entries an app actor cannot see', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
...userActor,
app: { uid: `app-suggest-${uuidv4()}` },
};
const target = `/${username}/Documents/probe.txt`;
await withActor(userActor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor: userActor,
}),
makeRes().res,
),
);
const spy = vi.spyOn(
server.services.suggestedApps,
'getSuggestedApps',
);
try {
const { res } = makeRes();
await withActor(appActor, () =>
controller.suggestApps(
makeReq({ body: { path: target }, actor: appActor }),
res,
),
);
expect(spy).toHaveBeenLastCalledWith({
name: undefined,
path: undefined,
});
} finally {
spy.mockRestore();
}
});
it('forwards entry name/path to the suggester for the owning user', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/probe2.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor,
}),
makeRes().res,
),
);
const spy = vi.spyOn(
server.services.suggestedApps,
'getSuggestedApps',
);
try {
const { res } = makeRes();
await withActor(actor, () =>
controller.suggestApps(
makeReq({ body: { path: target }, actor }),
res,
),
);
expect(spy).toHaveBeenLastCalledWith({
name: 'probe2.txt',
path: target,
});
} finally {
spy.mockRestore();
}
});
});
// ── readdirSubdomains ───────────────────────────────────────────────
describe('LegacyFSController.readdirSubdomains', () => {
it('returns an empty array for app-under-user actors', async () => {
const { actor: userActor, userId } = await makeUser();
const appActor: Actor = {
...userActor,
app: { uid: `app-subd-${uuidv4()}` },
};
await server.clients.db.write(
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`) VALUES (?, ?, ?)',
[uuidv4(), `sd-${uuidv4().slice(0, 8)}`, userId],
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.readdirSubdomains(
makeReq({ body: {}, actor: appActor }),
res,
),
);
expect(captured.body).toEqual([]);
});
it("returns the user actor's own subdomain rows", async () => {
const { actor, userId } = await makeUser();
const sd = `sd-${uuidv4().slice(0, 8)}`;
await server.clients.db.write(
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`) VALUES (?, ?, ?)',
[uuidv4(), sd, userId],
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdirSubdomains(
makeReq({ body: {}, actor }),
res,
),
);
const rows = captured.body as Array<{ subdomain: string }>;
expect(rows.some((r) => r.subdomain === sd)).toBe(true);
});
});
// ── updateFsentryThumbnail ──────────────────────────────────────────
describe('LegacyFSController.updateFsentryThumbnail', () => {
it('rejects an app actor probing entries outside AppData with 404', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
...userActor,
app: { uid: `app-thumb-${uuidv4()}` },
};
const target = `/${username}/Documents/thumbme.txt`;
await withActor(userActor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor: userActor,
}),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const { res } = makeRes();
await expect(
withActor(appActor, () =>
controller.updateFsentryThumbnail(
makeReq({
body: {
uid: entry!.uuid,
thumbnail: 'data:image/png;base64,AA==',
},
actor: appActor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('updates the thumbnail for the owning user actor', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/thumbme2.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor,
}),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.updateFsentryThumbnail(
makeReq({
body: {
uid: entry!.uuid,
thumbnail: 'data:image/png;base64,AA==',
},
actor,
}),
res,
),
);
const body = captured.body as { thumbnail: string };
expect(typeof body.thumbnail).toBe('string');
});
});
+251 -115
View File
@@ -22,7 +22,7 @@ import type { Request, RequestHandler, Response } from 'express';
import { contentType as contentTypeFromMime } from 'mime-types';
import { posix as pathPosix } from 'node:path';
import type { Actor } from '../../core/actor.js';
import { isAccessTokenActor } from '../../core/actor.js';
import { effectiveActorApp, isAccessTokenActor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
@@ -57,6 +57,16 @@ type RouterCache = Map<string, RequestHandler | null>;
const additionalRoutePaths: Record<string, string> = {};
// Legacy `/batch` multipart upload caps. Each file is buffered fully into
// memory before any quota / storage check runs, so without these limits an
// authenticated caller could grow the process heap proportional to whatever
// they sent. Streaming uploads go through `/writeFile`; this path is for
// pre-v2 clients only.
const BATCH_MAX_FILE_SIZE = 100 * 1024 * 1024; // 100 MiB per file
const BATCH_MAX_FILES = 64;
const BATCH_MAX_PARTS = 256;
const BATCH_MAX_FIELD_SIZE = 1 * 1024 * 1024; // 1 MiB per operation/fileinfo JSON
async function loadAdditionalRouter(
key: string,
): Promise<RequestHandler | null> {
@@ -74,7 +84,7 @@ async function loadAdditionalRouter(
}
}
// ── Controller ──────────────────────────────────────────────────────
// -- Controller ------------------------------------------------------
export class LegacyFSController extends PuterController {
#additionalCache: RouterCache = new Map();
@@ -195,35 +205,7 @@ export class LegacyFSController extends PuterController {
res.json({ recommended, recent });
});
router.post('/suggest_apps', apiOptions, async (req, res) => {
const suggestSvc = this.services.suggestedApps;
if (!suggestSvc?.getSuggestedApps) {
res.json([]);
return;
}
const body = req.body ?? {};
// Client sends { uid } or { path } identifying a file entry.
// Resolve the entry to get its name/path for extension detection.
let entryName: string | undefined;
let entryPath: string | undefined;
if (body.uid || body.path) {
try {
const entry = await resolveV1Selector(
this.stores.fsEntry,
body,
);
entryName = entry?.name;
entryPath = entry?.path;
} catch {
// If we can't resolve, fall back to empty
}
}
const suggestions = await suggestSvc.getSuggestedApps({
name: entryName,
path: entryPath,
});
res.json(suggestions);
});
router.post('/suggest_apps', apiOptions, this.suggestApps);
// puter-js polls this to decide whether to purge its in-memory FS
// cache. SocketService bumps a per-user Redis key on every
@@ -251,70 +233,11 @@ export class LegacyFSController extends PuterController {
},
);
// ── POST /readdir-subdomains ────────────────────────────────
router.post(
'/readdir-subdomains',
apiOptions,
async (req: Request, res: Response) => {
const userId = req.actor?.user?.id;
if (!userId)
throw new HttpError(401, 'Authentication required', {
legacyCode: 'unauthorized',
});
const rows = await this.clients.db.read(
'SELECT `subdomain`, `root_dir_id`, `uuid`, `ts` FROM `subdomains` WHERE `user_id` = ?',
[userId],
);
res.json(rows);
},
);
// ── POST /update-fsentry-thumbnail ──────────────────────────
router.post('/readdir-subdomains', apiOptions, this.readdirSubdomains);
router.post(
'/update-fsentry-thumbnail',
apiOptions,
async (req: Request, res: Response) => {
const userId = req.actor?.user?.id;
if (!userId)
throw new HttpError(401, 'Authentication required', {
legacyCode: 'unauthorized',
});
const { uid, thumbnail } = (req.body ?? {}) as {
uid?: string;
thumbnail?: string;
};
if (!uid)
throw new HttpError(400, 'Missing `uid`', {
legacyCode: 'bad_request',
});
if (!thumbnail)
throw new HttpError(400, 'Missing `thumbnail`', {
legacyCode: 'bad_request',
});
const entry = await this.stores.fsEntry.getEntryByUuid(uid);
if (!entry || entry.userId !== userId)
throw new HttpError(403, 'Access denied', {
legacyCode: 'forbidden',
});
// Emit thumbnail.created so the thumbnails extension can S3-upload.
// emitAndWait is required — the extension rewrites `event.url`
// from a data URL to an `s3://` pointer, and the DB write below
// needs to see that rewrite.
const event = { url: thumbnail };
await this.clients.event.emitAndWait(
'thumbnail.created',
event,
{},
);
await this.clients.db.write(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[event.url, uid],
);
res.json({ thumbnail: event.url });
},
this.updateFsentryThumbnail,
);
for (const key of Object.keys(additionalRoutePaths)) {
@@ -328,7 +251,7 @@ export class LegacyFSController extends PuterController {
}
}
// ── Route implementations ───────────────────────────────────────────
// -- Route implementations -------------------------------------------
//
// Handlers are public arrow class fields so they auto-bind `this` and can
// be passed directly to `router.post(...)` without `.bind(this)`. Express
@@ -504,17 +427,33 @@ export class LegacyFSController extends PuterController {
const normalizedTarget = targetPath.startsWith('/')
? targetPath
: `/${targetPath}`;
const dedupeName =
getBoolean(body, 'dedupe_name', 'change_name') ?? false;
await assertCanCreate(
this.services.acl,
this.services.fs,
actor,
normalizedTarget,
);
if (dedupeName) {
const existing =
await this.stores.fsEntry.getEntryByPath(normalizedTarget);
if (existing) {
const parent = pathPosix.dirname(normalizedTarget);
await assertAccess(
this.services.acl,
this.services.fs,
actor,
parent === '/' ? normalizedTarget : parent,
'write',
);
}
}
const entry = await this.services.fs.mkdir(userId, {
path: targetPath,
overwrite: getBoolean(body, 'overwrite') ?? false,
dedupeName: getBoolean(body, 'dedupe_name', 'change_name') ?? false,
dedupeName,
createMissingParents:
getBoolean(
body,
@@ -756,8 +695,108 @@ export class LegacyFSController extends PuterController {
res.send('');
};
suggestApps = async (req: Request, res: Response): Promise<void> => {
const suggestSvc = this.services.suggestedApps;
if (!suggestSvc?.getSuggestedApps) {
res.json([]);
return;
}
const actor = this.#requireActor(req);
const body = asRecord(req.body);
let entryName: string | undefined;
let entryPath: string | undefined;
if (body.uid || body.path) {
try {
const entry = await resolveV1Selector(
this.stores.fsEntry,
body,
);
// Suggestions leak the entry's extension and existence;
// gate on `see` so app actors can't probe outside scope.
if (entry?.path) {
await assertAccess(
this.services.acl,
this.services.fs,
actor,
entry.path,
'see',
);
}
entryName = entry?.name;
entryPath = entry?.path;
} catch {
// Unresolvable or ACL-denied → empty suggestions.
}
}
const suggestions = await suggestSvc.getSuggestedApps({
name: entryName,
path: entryPath,
});
res.json(suggestions);
};
readdirSubdomains = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
// Subdomain enumeration is a user-level concern; app actors would
// otherwise see root_dir uids pointing outside their AppData scope.
if (effectiveActorApp(actor)) {
res.json([]);
return;
}
const userId = this.#getActorUserId(req);
const rows = await this.clients.db.read(
'SELECT `subdomain`, `root_dir_id`, `uuid`, `ts` FROM `subdomains` WHERE `user_id` = ?',
[userId],
);
res.json(rows);
};
updateFsentryThumbnail = async (
req: Request,
res: Response,
): Promise<void> => {
const actor = this.#requireActor(req);
const { uid, thumbnail } = asRecord(req.body) as {
uid?: string;
thumbnail?: string;
};
if (!uid)
throw new HttpError(400, 'Missing `uid`', {
legacyCode: 'bad_request',
});
if (!thumbnail)
throw new HttpError(400, 'Missing `thumbnail`', {
legacyCode: 'bad_request',
});
const entry = await this.stores.fsEntry.getEntryByUuid(uid);
if (!entry || !entry.path)
throw new HttpError(404, `Entry not found: uid=${uid}`, {
legacyCode: 'subject_does_not_exist',
});
await assertAccess(
this.services.acl,
this.services.fs,
actor,
entry.path,
'write',
);
// emitAndWait is required: the thumbnails extension rewrites
// `event.url` from a data URL to an `s3://` pointer, and the DB
// write below needs to see that rewrite.
const event = { url: thumbnail };
await this.clients.event.emitAndWait('thumbnail.created', event, {});
await this.clients.db.write(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[event.url, uid],
);
res.json({ thumbnail: event.url });
};
search = async (req: Request, res: Response): Promise<void> => {
this.#requireActor(req);
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
const body = asRecord(req.body);
const query = getString(body, 'query', 'text') ?? '';
@@ -766,7 +805,21 @@ export class LegacyFSController extends PuterController {
legacyCode: 'bad_request',
});
const results = await this.services.fs.searchByName(userId, query, 200);
// App-under-user actors only see entries within their AppData root;
// user actors are unscoped. Mirrors the ACL short-circuit in
// ACLService.check.
const app = effectiveActorApp(actor);
const username = actor.user?.username;
const pathScope =
app && typeof username === 'string' && username.length > 0
? `/${username}/AppData/${app.uid}`
: undefined;
const results = await this.services.fs.searchByName(
userId,
query,
200,
pathScope,
);
const shaped = await Promise.all(
results.map((r) => toLegacyEntry(this.clients.event, r)),
);
@@ -888,7 +941,7 @@ export class LegacyFSController extends PuterController {
return this.read(req, res, { realMime: true });
};
// ── Signed-URL + meta routes ────────────────────────────────────────
// -- Signed-URL + meta routes ----------------------------------------
/**
* POST /sign
@@ -937,7 +990,10 @@ export class LegacyFSController extends PuterController {
legacyCode: 'not_found',
});
grantApp = { uid: app.uid };
result.token = this.services.auth.getUserAppToken(actor, app.uid);
result.token = await this.services.auth.getUserAppToken(
actor,
app.uid,
);
}
for (const rawItem of items) {
@@ -1071,15 +1127,16 @@ export class LegacyFSController extends PuterController {
);
}
// `write` — multipart upload, streamed directly to the v2 write path.
// ACL re-check: if the caller is session-authenticated, verify they
// have write permission on the target. This prevents a read-only
// share recipient from using a leaked write signature.
if (operation === 'write' && req.actor) {
// ACL re-check: require an authenticated actor with write
// permission. A valid write signature alone is not sufficient —
// the caller must also pass ACL, preventing exploitation of
// leaked write URLs by read-only share recipients.
const callerActor = this.#requireActor(req);
if (operation === 'write') {
await assertAccess(
this.services.acl,
this.services.fs,
req.actor,
callerActor,
targetEntry.path,
'write',
);
@@ -1120,7 +1177,6 @@ export class LegacyFSController extends PuterController {
// ops (mkdir/rename/copy/move/delete) require a caller actor with explicit
// ACL on the affected paths — mirroring the unsigned counterparts above.
const record = asRecord(req.body);
const callerActor = this.#requireActor(req);
if (operation === 'mkdir') {
await assertAccess(
this.services.acl,
@@ -1271,12 +1327,14 @@ export class LegacyFSController extends PuterController {
}
// Directory: return a signed listing of direct children.
// The caller only proved read access, so strip write_url from
// each child to prevent privilege escalation via /writeFile.
if (entry.isDir) {
const children = await this.services.fs.listDirectory(entry.uuid);
const signedChildren = children.map((child) => ({
...signEntry(child, signingCfg),
path: child.path,
}));
const signedChildren = children.map((child) => {
const { write_url: _, ...rest } = signEntry(child, signingCfg);
return { ...rest, path: child.path };
});
res.json(signedChildren);
return;
}
@@ -1349,6 +1407,20 @@ export class LegacyFSController extends PuterController {
'read',
);
// Downgrade the envelope when the caller only proved read.
// `/writeFile`'s ACL re-check would still block the write, but
// returning `write_url` to a read-only caller is the same
// privilege-leak shape that `/sign` and `/readdir` strip.
const writeOk = await this.services.acl.check(
actor,
{
path: entry.path,
resolveAncestors: () =>
this.services.fs.getAncestorChain(entry.path),
},
'write',
);
const suggested =
(await this.services.suggestedApps?.getSuggestedApps({
name: entry.name,
@@ -1368,11 +1440,20 @@ export class LegacyFSController extends PuterController {
{},
{ reason: 'open_item' },
);
token = this.services.auth.getUserAppToken(actor, defaultAppUid);
token = await this.services.auth.getUserAppToken(
actor,
defaultAppUid,
);
}
const signingCfg = signingConfigFromAppConfig(this.config);
const signature = { ...signEntry(entry, signingCfg), path: entry.path };
const signed = signEntry(entry, signingCfg);
const signature = writeOk
? { ...signed, path: entry.path }
: (() => {
const { write_url: _, ...rest } = signed;
return { ...rest, path: entry.path };
})();
res.json({
signature,
token,
@@ -1538,7 +1619,7 @@ export class LegacyFSController extends PuterController {
};
// Helpers for writeFile
// ── GUI event emission ───────────────────────────────────────────
// -- GUI event emission -------------------------------------------
//
// Fire-and-forget `outer.gui.item.*` events so SocketService,
// BroadcastService, WorkerDriver (hot-reload), and cache-invalidation
@@ -1668,7 +1749,7 @@ export class LegacyFSController extends PuterController {
});
}
// ── Batch route ─────────────────────────────────────────────────────
// -- Batch route -----------------------------------------------------
//
// `/batch` interleaves multipart JSON operations with optional file
// uploads. puter-js uses it for `write`, `shortcut`, `mkdir`, `move`,
@@ -1977,7 +2058,46 @@ export class LegacyFSController extends PuterController {
}> = [];
const fileinfos: Array<Record<string, unknown>> = [];
let parseError: Error | null = null;
const bb = Busboy({ headers: req.headers });
const bb = Busboy({
headers: req.headers,
limits: {
fileSize: BATCH_MAX_FILE_SIZE,
files: BATCH_MAX_FILES,
parts: BATCH_MAX_PARTS,
fieldSize: BATCH_MAX_FIELD_SIZE,
},
});
// Busboy emits these `*Limit` events when a configured cap is
// hit. Capture the first one as a 413 so callers get a clean
// signal instead of a silently-truncated upload.
bb.on('filesLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
`Too many files in batch (max ${BATCH_MAX_FILES})`,
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('partsLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
`Too many parts in batch (max ${BATCH_MAX_PARTS})`,
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('fieldsLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
'Too many fields in batch',
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('field', (fieldName, value) => {
try {
@@ -2004,8 +2124,24 @@ export class LegacyFSController extends PuterController {
// For streaming uploads use the signed `/writeFile` endpoint.
bb.on('file', (_fieldName, stream, info) => {
const chunks: Buffer[] = [];
let truncated = false;
stream.on('data', (chunk: Buffer) => chunks.push(chunk));
// Busboy emits `limit` after writing the first byte past
// `fileSize`. The stream continues being drained so the
// multipart parser stays in sync, but we discard the (now
// truncated) buffer and mark the batch as failed.
stream.on('limit', () => {
truncated = true;
if (!parseError) {
parseError = new HttpError(
413,
`File in batch exceeds ${BATCH_MAX_FILE_SIZE} bytes`,
{ legacyCode: 'too_large' as never },
);
}
});
stream.on('end', () => {
if (truncated) return;
files.push({
content: Buffer.concat(chunks),
mimeType:
@@ -2065,7 +2201,7 @@ export class LegacyFSController extends PuterController {
return { error: true, status: 500, message: 'Unknown batch error' };
}
// ── Helpers ─────────────────────────────────────────────────────────
// -- Helpers ---------------------------------------------------------
#parsePositiveIntegerQuery(
query: Record<string, unknown>,
@@ -48,7 +48,7 @@ import type { IConfig } from '../../types.js';
* terse.
*/
// ── Body parsing ─────────────────────────────────────────────────────
// -- Body parsing -----------------------------------------------------
export function asRecord(value: unknown): Record<string, unknown> {
if (!value || typeof value !== 'object' || Array.isArray(value)) return {};
@@ -160,7 +160,7 @@ export async function resolveV1Selector(
return entry;
}
// ── ACL ──────────────────────────────────────────────────────────────
// -- ACL --------------------------------------------------------------
export async function assertAccess(
aclService: ACLService,
@@ -261,7 +261,7 @@ export async function assertCanCreate(
await assertAccess(aclService, fsService, actor, parentForCheck, 'write');
}
// ── Response shaping ────────────────────────────────────────────────
// -- Response shaping ------------------------------------------------
type AppRowLookup = {
getByIds: (ids: number[]) => Promise<Map<number, Record<string, unknown>>>;
@@ -461,7 +461,7 @@ export async function toLegacyEntry(
export { normalizeAbsolutePath };
// ── Signing ─────────────────────────────────────────────────────────
// -- Signing ---------------------------------------------------------
/**
* Pull the signing config off the app config. Throws if either value is
+1
View File
@@ -52,6 +52,7 @@ export interface ThumbnailUploadPrepareItem {
size?: number;
uploadUrl?: string;
thumbnailUrl?: string;
item_uid: string;
}
export interface ThumbnailUploadPreparePayload {
@@ -72,7 +72,7 @@ export class HomepageController extends PuterController {
await homepage.send({ req, res, actor }, meta, launch);
};
// ── Root + path-aliased shell routes ────────────────────────
// -- Root + path-aliased shell routes ------------------------
router.get('/', {}, (req, res) => sendShell(req, res));
@@ -86,7 +86,7 @@ export class HomepageController extends PuterController {
router.get('/@:username', {}, (req, res) => sendShell(req, res));
// ── /app/:name ─ app metadata baked into the shell ──────────
// -- /app/:name - app metadata baked into the shell ----------
router.get('/app/:name', {}, async (req, res) => {
const name = String(req.params.name ?? '');
@@ -122,7 +122,7 @@ export class HomepageController extends PuterController {
});
});
// ── /show/* ─ launch explorer with the requested file path ──
// -- /show/* - launch explorer with the requested file path --
router.get('/show/*splat', {}, (req, res) => {
const filePath = req.path.slice('/show'.length);
@@ -138,7 +138,7 @@ export class HomepageController extends PuterController {
return sendShell(req, res, {}, launch);
});
// ── Fallback static mount ───────────────────────────────────
// -- Fallback static mount -----------------------------------
// Serves lingering GUI assets (images, fonts, lib files, etc.)
// out of <gui_assets_root>/src. Falls through to the 404 handler
// when the file doesn't exist.
@@ -37,7 +37,7 @@ export class HostingController extends PuterController {
}
registerRoutes(router) {
// ── Delete site ─────────────────────────────────────────────
// -- Delete site ---------------------------------------------
router.post(
'/delete-site',
@@ -93,7 +93,7 @@ function isSameOrigin(target: string, origin: string): boolean {
*/
export class OIDCController extends PuterController {
registerRoutes(router: PuterRouter): void {
// ── GET /auth/oidc/providers ────────────────────────────────
// -- GET /auth/oidc/providers --------------------------------
// Public — list enabled provider IDs for the frontend.
router.get(
@@ -106,7 +106,7 @@ export class OIDCController extends PuterController {
},
);
// ── GET /auth/oidc/:provider/start ──────────────────────────
// -- GET /auth/oidc/:provider/start --------------------------
// Redirect user to IdP authorization endpoint.
router.get(
@@ -215,7 +215,7 @@ export class OIDCController extends PuterController {
},
);
// ── /auth/oidc/callback/login (GET + POST) ────────────────
// -- /auth/oidc/callback/login (GET + POST) ----------------
const cbOpts = {
subdomain: '',
@@ -283,7 +283,7 @@ export class OIDCController extends PuterController {
router.get('/auth/oidc/callback/login', cbOpts, loginCb);
router.post('/auth/oidc/callback/login', cbOpts, loginCb);
// ── /auth/oidc/callback/signup (GET + POST) ────────────────
// -- /auth/oidc/callback/signup (GET + POST) ----------------
const signupCb = async (req: Request, res: Response) => {
const origin = this.config.origin ?? '';
@@ -346,7 +346,7 @@ export class OIDCController extends PuterController {
router.get('/auth/oidc/callback/signup', cbOpts, signupCb);
router.post('/auth/oidc/callback/signup', cbOpts, signupCb);
// ── /auth/oidc/callback/revalidate (GET + POST) ────────────
// -- /auth/oidc/callback/revalidate (GET + POST) ------------
const revalidateCb = async (
req: Request,
@@ -404,7 +404,7 @@ export class OIDCController extends PuterController {
router.get('/auth/oidc/callback/revalidate', cbOpts, revalidateCb);
router.post('/auth/oidc/callback/revalidate', cbOpts, revalidateCb);
// ── GET /auth/revalidate-done ───────────────────────────────
// -- GET /auth/revalidate-done -------------------------------
// Landing page after revalidation; posts to opener for popup flow.
router.get(
@@ -428,7 +428,7 @@ if (window.opener) {
);
}
// ── Shared helpers ──────────────────────────────────────────────
// -- Shared helpers ----------------------------------------------
/**
* Resolve an OIDC callback to a Puter user. In order:
@@ -110,7 +110,7 @@ export class PuterAIController extends PuterController {
this.#modelDetails('aiVideo'),
);
// ── Video URL proxy ─────────────────────────────────────────
// -- Video URL proxy -----------------------------------------
// Reverse-proxies AI-generated video URLs that can't be given
// directly to the client (auth-gated provider downloads). The
// URL itself is HMAC-signed, so no additional auth gate.
@@ -226,7 +226,7 @@ export class PuterAIController extends PuterController {
};
}
// ── /openai/v1/chat/completions ─────────────────────────────────
// -- /openai/v1/chat/completions ---------------------------------
openaiChatCompletions = async (
req: Request,
@@ -394,7 +394,7 @@ export class PuterAIController extends PuterController {
});
};
// ── /openai/v1/completions ──────────────────────────────────────
// -- /openai/v1/completions --------------------------------------
openaiCompletions = async (req: Request, res: Response): Promise<void> => {
const body = asRecord(req.body);
@@ -522,7 +522,7 @@ export class PuterAIController extends PuterController {
});
};
// ── /openai/v1/responses ────────────────────────────────────────
// -- /openai/v1/responses ----------------------------------------
openaiResponses = async (req: Request, res: Response): Promise<void> => {
const body = asRecord(req.body);
@@ -832,7 +832,7 @@ export class PuterAIController extends PuterController {
);
};
// ── /anthropic/v1/messages ──────────────────────────────────────
// -- /anthropic/v1/messages --------------------------------------
anthropicMessages = async (req: Request, res: Response): Promise<void> => {
const body = asRecord(req.body);
@@ -1037,7 +1037,7 @@ export class PuterAIController extends PuterController {
});
};
// ── Internals ───────────────────────────────────────────────────
// -- Internals ---------------------------------------------------
#driver(): ChatCompletionDriver {
const driver = this.drivers.aiChat;
@@ -1049,7 +1049,7 @@ export class PuterAIController extends PuterController {
}
}
// ── Shared helpers ──────────────────────────────────────────────────
// -- Shared helpers --------------------------------------------------
const DEFAULTS = {
openaiChat: 'openai-completion',
@@ -1135,7 +1135,7 @@ const pipeNdjsonStream = (
stream.on('error', opts.onError);
};
// ── OpenAI/Anthropic shape helpers ───────────────────────────────────
// -- OpenAI/Anthropic shape helpers -----------------------------------
const extractTextContent = (content: unknown): string => {
if (content === undefined || content === null) return '';
@@ -1254,7 +1254,7 @@ const getPromptText = (prompt: unknown): string => {
return prompt;
};
// ── OpenAI /responses input → message list ──────────────────────────
// -- OpenAI /responses input → message list --------------------------
const parseJsonMaybe = (value: unknown): unknown => {
if (typeof value !== 'string') return value ?? {};
@@ -1363,7 +1363,7 @@ const responseInputToMessages = (input: unknown): unknown[] => {
return messages;
};
// ── OpenAI /responses result → output items ─────────────────────────
// -- OpenAI /responses result → output items -------------------------
const responseOutputFromResult = (
result: Extract<IChatCompleteResult, { message?: unknown }>,
@@ -1484,7 +1484,7 @@ const normalizeResponsesTool = (tool: unknown): unknown => {
return { ...(t.function as Record<string, unknown>), type: 'function' };
};
// ── Anthropic → internal messages ───────────────────────────────────
// -- Anthropic → internal messages -----------------------------------
const normalizeAnthropicTools = (tools: unknown): unknown[] | undefined => {
if (!Array.isArray(tools) || tools.length === 0) return undefined;
@@ -51,7 +51,7 @@ export class ShareController extends PuterController {
// router.post('/share', { ...api, requireAuth: true }, this.#share);
}
// // ── POST /sharelink/check ───────────────────────────────────────
// // -- POST /sharelink/check ---------------------------------------
// // Public — verify a share token from an email link.
// #check = async (req: Request, res: Response): Promise<void> => {
@@ -80,7 +80,7 @@ export class ShareController extends PuterController {
// });
// };
// // ── POST /sharelink/apply ───────────────────────────────────────
// // -- POST /sharelink/apply ---------------------------------------
// // Auth required — apply a pending share's permissions to the caller.
// #apply = async (req: Request, res: Response): Promise<void> => {
@@ -158,7 +158,7 @@ export class ShareController extends PuterController {
// res.json({ $: 'api:status-report', status: 'success' });
// };
// // ── POST /sharelink/request ─────────────────────────────────────
// // -- POST /sharelink/request -------------------------------------
// // Auth required — notify the issuer that someone is requesting access.
// #request = async (req: Request, res: Response): Promise<void> => {
@@ -207,7 +207,7 @@ export class ShareController extends PuterController {
// res.json({ $: 'api:status-report', status: 'success' });
// };
// // ── POST /share ─────────────────────────────────────────────────
// // -- POST /share -------------------------------------------------
// // Auth required — create shares for recipients (users or emails).
// #share = async (req: Request, res: Response): Promise<void> => {
@@ -355,7 +355,7 @@ export class ShareController extends PuterController {
// });
// };
// // ── Helpers ──────────────────────────────────────────────────────
// // -- Helpers ------------------------------------------------------
// /**
// * Convert share declarations into a flat permission list.
@@ -140,7 +140,7 @@ export class StaticPagesController extends PuterController {
page('&#10005;', 'Something went wrong', msg, '#e53e3e');
const ok = (msg: string) => page('&#10003;', 'Success', msg, '#38a169');
// ── /robots.txt ─────────────────────────────────────────────
// -- /robots.txt ---------------------------------------------
router.get('/robots.txt', {}, (req, res) => {
const domain = this.config.domain ?? req.hostname;
const disallowed = [
@@ -160,12 +160,12 @@ export class StaticPagesController extends PuterController {
res.type('text/plain').send(body);
});
// ── /sitemap.xml ────────────────────────────────────────────
// -- /sitemap.xml --------------------------------------------
router.get('/sitemap.xml', {}, async (req, res) => {
const domain = this.config.domain ?? req.hostname;
const origin = `${req.protocol}://${domain}`;
const apps = (await this.clients.db.read(
'SELECT `name` FROM `apps` WHERE `approved_for_listing` = 1',
`SELECT \`name\` FROM \`apps\` WHERE \`approved_for_listing\` = ${this.clients.db.booleanLiteral(true)}`,
)) as Array<{ name: string }>;
const urls = [
`<url><loc>${req.protocol}://docs.${domain}/</loc></url>`,
@@ -181,7 +181,7 @@ export class StaticPagesController extends PuterController {
res.type('application/xml').send(body);
});
// ── /unsubscribe ────────────────────────────────────────────
// -- /unsubscribe --------------------------------------------
router.get('/unsubscribe', {}, async (req, res) => {
const userUuid =
typeof req.query.user_uuid === 'string'
@@ -206,7 +206,7 @@ export class StaticPagesController extends PuterController {
res.send(ok('You have successfully unsubscribed from all emails.'));
});
// ── /confirm-email-by-token ─────────────────────────────────
// -- /confirm-email-by-token ---------------------------------
router.get('/confirm-email-by-token', {}, async (req, res) => {
const userUuid =
typeof req.query.user_uuid === 'string'
@@ -252,7 +252,7 @@ export class StaticPagesController extends PuterController {
const [dupe] = (await this.clients.db.read(
`SELECT EXISTS(
SELECT 1 FROM \`user\` WHERE (\`email\` = ? OR \`clean_email\` = ?)
AND \`email_confirmed\` = 1
AND \`email_confirmed\` = ${this.clients.db.booleanLiteral(true)}
AND \`password\` IS NOT NULL
) AS email_exists`,
[user.email, cleanEmail],
@@ -35,7 +35,7 @@ export class SystemController extends PuterController {
/** @type {import('../../core/http/PuterRouter.js').PuterRouter} */
router,
) {
// ── Healthcheck ─────────────────────────────────────────────
// -- Healthcheck ---------------------------------------------
// Delegates to ServerHealthService for the real check-based
// status. Returns `{ ok: true }` when all registered checks pass,
// or `{ ok: false, failed: [...] }` + 503 when any fail or the
@@ -51,7 +51,7 @@ export class SystemController extends PuterController {
return res.json(status);
});
// ── Version ─────────────────────────────────────────────────
// -- Version -------------------------------------------------
router.get('/version', { subdomain: '*' }, (_req, res) => {
const version =
@@ -70,7 +70,7 @@ export class SystemController extends PuterController {
});
});
// ── Contact us ──────────────────────────────────────────────
// -- Contact us ----------------------------------------------
router.post(
'/contactUs',
@@ -129,7 +129,7 @@ export class SystemController extends PuterController {
},
);
// ── GET /whoarewe ───────────────────────────────────────────
// -- GET /whoarewe -------------------------------------------
router.get('/whoarewe', {}, (_req, res) => {
res.json({
@@ -139,7 +139,7 @@ export class SystemController extends PuterController {
});
});
// ── GET /lsmod ──────────────────────────────────────────────
// -- GET /lsmod ----------------------------------------------
// Enumerates driver interfaces and their implementors.
router.get(
@@ -1,10 +1,13 @@
// This suite tests basic features of puter webdav. it is not a comprehensive webdav test suite unlike litmus
// but rather it performs some common sense checks to ensure that WebDAV support isn't irrevocably broken in puter
import type { Request, Response } from 'express';
import { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import { PuterRouter } from '../../core/http/PuterRouter.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import type { WebDAVController } from './WebDAVController.js';
let server: PuterServer;
@@ -101,6 +104,35 @@ const basicAuth = (user: string, pass: string) =>
const noop = vi.fn();
const makeUser = async () => {
const username = `webdav-${Math.random().toString(36).slice(2, 10)}`;
const created = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
created,
);
const refreshed = (await server.stores.user.getById(created.id))!;
return {
userId: refreshed.id,
username: refreshed.username,
actor: {
user: {
id: refreshed.id,
uuid: refreshed.uuid,
username: refreshed.username,
},
},
};
};
describe('WebDAVController', () => {
describe('route registration', () => {
it('registers a single catch-all use() route', () => {
@@ -396,6 +428,53 @@ describe('WebDAVController', () => {
);
expect(captured.statusCode).toBe(422);
});
it('emits GUI-safe item events without leaking numeric fsentry ids', async () => {
const { actor, userId, username } = await makeUser();
const target = `/${username}/Documents/webdav-event.txt`;
const { res, captured } = makeRes();
const req = Object.assign(
Readable.from(['hello']),
makeReq({
method: 'PUT',
path: target,
headers: { 'content-length': '5' },
actor,
}),
) as Request;
const emitSpy = vi.spyOn(server.clients.event, 'emit');
let addedCall:
| (typeof emitSpy.mock.calls)[number]
| undefined;
try {
await dispatchMiddleware(req, res, noop);
await new Promise((resolve) => setTimeout(resolve, 0));
addedCall = emitSpy.mock.calls.find(
([eventName]) => eventName === 'outer.gui.item.added',
);
} finally {
emitSpy.mockRestore();
}
expect(captured.statusCode).toBe(201);
expect(addedCall).toBeTruthy();
const payload = addedCall?.[1] as {
user_id_list?: number[];
response?: Record<string, unknown>;
};
expect(payload.user_id_list).toEqual([userId]);
expect(payload.response).toMatchObject({
id: expect.any(String),
uid: expect.any(String),
uuid: expect.any(String),
path: target,
from_new_service: true,
});
expect(typeof payload.response?.id).toBe('string');
expect(payload.response?.id).toBe(payload.response?.uuid);
expect(payload.response).not.toHaveProperty('userId');
});
});
describe('DELETE', () => {
@@ -516,7 +595,7 @@ describe('WebDAVController', () => {
await dispatchMiddleware(
makeReq({
method: 'LOCK',
path: '/lockable-file.txt',
path: '/test/lockable-file.txt',
actor: {
user: {
id: 1,
@@ -539,8 +618,28 @@ describe('WebDAVController', () => {
expect(captured.headers['lock-token']).toContain('urn:uuid:');
});
it('rejects locking a path the user has no write access to (e.g. root)', async () => {
const { res, captured } = makeRes();
await dispatchMiddleware(
makeReq({
method: 'LOCK',
path: '/',
actor: {
user: {
id: 1,
uuid: 'test-uuid',
username: 'test',
},
},
}),
res,
noop,
);
expect(captured.statusCode).toBe(403);
});
it('rejects a second exclusive lock on the same path', async () => {
const uniquePath = `/double-lock-${Date.now()}.txt`;
const uniquePath = `/test/double-lock-${Date.now()}.txt`;
// First lock
const { res: res1, captured: cap1 } = makeRes();
@@ -582,7 +681,7 @@ describe('WebDAVController', () => {
});
it('refreshes an existing lock when If header provides the token', async () => {
const uniquePath = `/refresh-lock-${Date.now()}.txt`;
const uniquePath = `/test/refresh-lock-${Date.now()}.txt`;
const { res: res1, captured: cap1 } = makeRes();
await dispatchMiddleware(
makeReq({
@@ -27,6 +27,7 @@ import type { PuterRouter } from '../../core/http/PuterRouter.js';
import { verify as verifyOtp } from '../../services/auth/OTPUtil.js';
import { expandTildePath } from '../../services/fs/resolveNode.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
import { toLegacyEntry } from '../fs/legacyFsHelpers.js';
import { PuterController } from '../types.js';
import {
createLock,
@@ -128,7 +129,7 @@ export class WebDAVController extends PuterController {
case 'MOVE':
return this.#move(req, res, actor, davPath, redis, lockToken);
case 'LOCK':
return this.#lock(req, res, davPath, redis, lockToken);
return this.#lock(req, res, actor, davPath, redis, lockToken);
case 'UNLOCK':
return this.#unlock(req, res, davPath, redis);
default:
@@ -138,7 +139,7 @@ export class WebDAVController extends PuterController {
}
}
// ── Auth ─────────────────────────────────────────────────────────
// -- Auth ---------------------------------------------------------
async #resolveActor(req: Request, res: Response): Promise<Actor | null> {
// If the global authProbe already resolved an actor, use it.
@@ -236,7 +237,7 @@ export class WebDAVController extends PuterController {
};
}
// ── OPTIONS ──────────────────────────────────────────────────────
// -- OPTIONS ------------------------------------------------------
#options(res: Response): void {
res.status(200)
@@ -250,7 +251,7 @@ export class WebDAVController extends PuterController {
.send('');
}
// ── GET / HEAD ──────────────────────────────────────────────────
// -- GET / HEAD --------------------------------------------------
async #get(
req: Request,
@@ -300,7 +301,7 @@ export class WebDAVController extends PuterController {
result.body.pipe(res);
}
// ── PROPFIND ────────────────────────────────────────────────────
// -- PROPFIND ----------------------------------------------------
async #propfind(
req: Request,
@@ -347,7 +348,7 @@ export class WebDAVController extends PuterController {
.send(wrapMultistatus(responses.join('\n')));
}
// ── PROPPATCH (stub — acknowledges but doesn't persist props) ───
// -- PROPPATCH (stub — acknowledges but doesn't persist props) ---
async #proppatch(
res: Response,
@@ -371,7 +372,7 @@ export class WebDAVController extends PuterController {
);
}
// ── MKCOL ───────────────────────────────────────────────────────
// -- MKCOL -------------------------------------------------------
async #mkcol(
req: Request,
@@ -421,7 +422,7 @@ export class WebDAVController extends PuterController {
.end();
}
// ── PUT ─────────────────────────────────────────────────────────
// -- PUT ---------------------------------------------------------
async #put(
req: Request,
@@ -497,7 +498,7 @@ export class WebDAVController extends PuterController {
.end();
}
// ── DELETE ───────────────────────────────────────────────────────
// -- DELETE -------------------------------------------------------
async #delete(
res: Response,
@@ -527,7 +528,7 @@ export class WebDAVController extends PuterController {
res.status(204).end();
}
// ── COPY ────────────────────────────────────────────────────────
// -- COPY --------------------------------------------------------
async #copy(
req: Request,
@@ -585,7 +586,7 @@ export class WebDAVController extends PuterController {
res.status(destExists ? 204 : 201).end();
}
// ── MOVE ────────────────────────────────────────────────────────
// -- MOVE --------------------------------------------------------
async #move(
req: Request,
@@ -641,17 +642,23 @@ export class WebDAVController extends PuterController {
res.status(destExists ? 204 : 201).end();
}
// ── LOCK ────────────────────────────────────────────────────────
// -- LOCK --------------------------------------------------------
async #lock(
req: Request,
res: Response,
actor: Actor,
davPath: string,
redis: unknown,
headerToken: string | null,
): Promise<void> {
const r = redis as import('ioredis').Cluster;
// ACL must succeed before any lock state is touched — otherwise
// an authenticated user could lock paths they don't own (e.g. `/`)
// and block writes for everyone else.
await this.#assertWrite(actor, davPath);
// Refresh existing lock
if (headerToken) {
const existing = await getLockIfValid(r, headerToken);
@@ -702,7 +709,7 @@ export class WebDAVController extends PuterController {
.send(lockResponseXml(token, davPath, lockScope));
}
// ── UNLOCK ──────────────────────────────────────────────────────
// -- UNLOCK ------------------------------------------------------
async #unlock(
req: Request,
@@ -733,7 +740,7 @@ export class WebDAVController extends PuterController {
res.status(204).end();
}
// ── ACL helpers ─────────────────────────────────────────────────
// -- ACL helpers -------------------------------------------------
async #assertRead(actor: Actor, path: string): Promise<void> {
const descriptor = {
@@ -759,32 +766,36 @@ export class WebDAVController extends PuterController {
});
}
// ── Event emission ──────────────────────────────────────────────
// -- Event emission ----------------------------------------------
#emitGuiEvent<T extends keyof EventMap>(
eventName: T,
entry: FSEntry,
extra?: Record<string, unknown>,
): void {
const payload = {
user_id_list: [entry.userId],
response: { ...entry, ...extra, from_new_service: true },
};
const meta = {};
void Promise.resolve()
.then(() =>
.then(async () => {
const response = {
...(await toLegacyEntry(this.clients.event, entry)),
...extra,
from_new_service: true,
};
this.clients.event.emit(
eventName,
payload as unknown as EventMap[T],
{
user_id_list: [entry.userId],
response,
} as unknown as EventMap[T],
meta,
),
)
);
})
.catch(() => {
// non-critical
});
}
// ── Misc helpers ────────────────────────────────────────────────
// -- Misc helpers ------------------------------------------------
#parseDestination(req: Request): string {
const dest = req.headers.destination as string | undefined;
@@ -801,7 +812,7 @@ export class WebDAVController extends PuterController {
}
}
// ── XML helpers ──────────────────────────────────────────────────────
// -- XML helpers ------------------------------------------------------
function escapeXml(text: string): string {
return text
+1 -1
View File
@@ -198,7 +198,7 @@ export async function hasWritePermission(
return true;
}
// ── Internals ───────────────────────────────────────────────────────
// -- Internals -------------------------------------------------------
async function getPathLockMap(
redis: Cluster,
+10
View File
@@ -97,3 +97,13 @@ export const userRelatedActor = (actor: Actor): Actor => {
if (!actor.app && !actor.accessToken) return actor;
return { user: actor.user };
};
/**
* Walk the access-token issuer chain and return the first app identity found,
* or null if no app appears anywhere in the chain.
*/
export const effectiveActorApp = (actor: Actor): ActorApp | null => {
if (actor.app) return actor.app;
if (actor.accessToken) return effectiveActorApp(actor.accessToken.issuer);
return null;
};
+4 -4
View File
@@ -46,7 +46,7 @@ import type { Actor } from './actor';
* ```
*/
// ── Well-known typed keys ───────────────────────────────────────────
// -- Well-known typed keys -------------------------------------------
export interface KnownContextFields {
/** The authenticated actor, if one was resolved by the auth probe. */
@@ -57,7 +57,7 @@ export interface KnownContextFields {
requestId: string;
}
// ── Context store ───────────────────────────────────────────────────
// -- Context store ---------------------------------------------------
interface ContextStore {
known: Partial<KnownContextFields>;
@@ -66,7 +66,7 @@ interface ContextStore {
const als = new AsyncLocalStorage<ContextStore>();
// ── Public API ──────────────────────────────────────────────────────
// -- Public API ------------------------------------------------------
/**
* Static-style context accessor.
@@ -131,7 +131,7 @@ export class Context {
}
}
// ── Internal: used by the request-context middleware ─────────────────
// -- Internal: used by the request-context middleware -----------------
/**
* Run `fn` inside a new context scope. Used by the request-context
+3 -3
View File
@@ -61,7 +61,7 @@ export class PuterRouter {
this.prefix = prefix;
}
// ── use ─────────────────────────────────────────────────────────
// -- use ---------------------------------------------------------
//
// `use` is the only method whose path is optional (global-ish
// middleware) and whose options can appear with or without a path.
@@ -77,7 +77,7 @@ export class PuterRouter {
return this;
}
// ── HTTP verbs + WebDAV ─────────────────────────────────────────
// -- HTTP verbs + WebDAV -----------------------------------------
//
// All take `(path, handler)` or `(path, options, handler)`.
@@ -231,7 +231,7 @@ export class PuterRouter {
return this.#push('move', args);
}
// ── Internals ───────────────────────────────────────────────────
// -- Internals ---------------------------------------------------
#push(method: RouteMethod, args: unknown[]): this {
const normalized = this.#parsePathArgs(args);
+3 -3
View File
@@ -42,7 +42,7 @@ import {
* directly work equally well.
*/
// ── Prototype shape helpers ─────────────────────────────────────────
// -- Prototype shape helpers -----------------------------------------
interface DecoratedPrototype {
[ROUTES_METADATA_KEY]?: CollectedRoute[];
@@ -57,7 +57,7 @@ const getOrInitRoutes = (proto: DecoratedPrototype): CollectedRoute[] => {
return proto[ROUTES_METADATA_KEY]!;
};
// ── @Controller ─────────────────────────────────────────────────────
// -- @Controller -----------------------------------------------------
/**
* Class decorator.
@@ -128,7 +128,7 @@ export function Controller(prefix: string = '') {
};
}
// ── Method decorators (@Get, @Post, ...) ───────────────────────────
// -- Method decorators (@Get, @Post, ...) ---------------------------
// eslint-disable-next-line @typescript-eslint/no-unsafe-function-type
type AnyMethod = Function;
+12 -27
View File
@@ -35,40 +35,25 @@ declare global {
// eslint-disable-next-line @typescript-eslint/no-namespace
namespace Express {
interface Request {
/**
* Populated by the global auth probe when a valid token is
* attached to the request (Bearer header, auth_token body/query,
* session cookie, or socket handshake). Absent for anonymous
* requests — route-level gates decide whether to reject.
*/
actor?: Actor;
/** The raw token string, if one was presented and parsed. */
token?: string;
/**
* Set by the global auth probe when a token was extracted from
* the request but failed to resolve to an actor (bad signature,
* expired, references a deleted session/user/app, or has a
* legacy shape v2 can't authenticate). Lets `requireAuthGate`
* distinguish "no token" from "token present but invalid" and
* emit the legacy `token_auth_failed` error so old clients
* trigger their re-login flow.
*/
tokenAuthFailed?: boolean;
/**
* Raw request body bytes, captured by the global JSON parser's
* `verify` callback. Available for any JSON request — needed by
* webhook handlers that verify an HMAC over the exact bytes the
* sender signed (e.g., AppStore prod webhooks, BroadcastService
* inter-instance hooks).
*
* Set only when the global JSON parser actually ran (request had
* `Content-Type: application/json` or one of the recognized
* JSON-as-text variants). For non-JSON requests it stays
* `undefined`.
*/
requiresReauth?: {
reason: 'token_v1' | 'session_revoked' | 'session_expired';
auth_id?: string;
/**
* Short-lived server-signed JWT that proves the bearer was
* identified as `auth_id` by the rejected session. The GUI
* must echo this back (not the raw `auth_id`) on the next
* login/signup so the controller can rebind to the same user.
*/
reauth_token?: string;
};
rawBody?: Buffer;
/** Parsed user-agent, populated by the global UA-parsing middleware. */
+1 -1
View File
@@ -64,7 +64,7 @@ export const antiCsrf = {
},
};
// ── Route middleware ────────────────────────────────────────────────
// -- Route middleware ------------------------------------------------
/**
* Middleware that requires a valid anti-CSRF token in `req.body.anti_csrf`.
@@ -34,27 +34,50 @@ import { createAuthProbe } from './authProbe';
// test wants. This is mocking at a real boundary (a service), which the
// AGENTS.md guidance explicitly allows.
type AuthResultLike =
| { actor: Actor }
| { reauth: { reason: string; auth_id?: string } }
| { invalid: true };
interface StubAuth {
service: AuthService;
seenTokens: string[];
/** What the next call to authenticateFromToken should return. */
/** Legacy setter — accepts the old Actor|null|'throw' shape. */
setNext: (next: Actor | null | 'throw') => void;
/** Set the full AuthResult to be returned by `authenticate()`. */
setNextResult: (next: AuthResultLike | 'throw') => void;
}
const makeStubAuth = (defaultActor: Actor | null = null): StubAuth => {
const seenTokens: string[] = [];
let nextResult: Actor | null | 'throw' = defaultActor;
let nextResult: AuthResultLike | 'throw' = defaultActor
? { actor: defaultActor }
: { invalid: true };
const service = {
authenticateFromToken: async (token: string) => {
// Entry point used by the probe.
authenticate: async (token: string) => {
seenTokens.push(token);
if (nextResult === 'throw') throw new Error('verify failed');
return nextResult;
},
// Back-compat wrapper for callers that still want Actor | null.
authenticateFromToken: async (token: string) => {
seenTokens.push(token);
if (nextResult === 'throw') throw new Error('verify failed');
return 'actor' in nextResult ? nextResult.actor : null;
},
// Deterministic stub — production mints a real JWT here.
signReauthToken: (authId: string) => `reauth-jwt:${authId}`,
} as unknown as AuthService;
return {
service,
seenTokens,
setNext: (n) => {
if (n === 'throw') nextResult = 'throw';
else if (n === null) nextResult = { invalid: true };
else nextResult = { actor: n };
},
setNextResult: (n) => {
nextResult = n;
},
};
@@ -522,6 +545,225 @@ describe('createAuthProbe — actor attachment + failure tracking', () => {
});
});
// ── Reauth signal + KV counters ─────────────────────────────────────
describe('createAuthProbe — reauth signal', () => {
/** Capture KV increments without a real store. */
const makeKvStub = () => {
const calls: Array<{
key: string;
pathAndAmountMap: Record<string, number>;
}> = [];
return {
calls,
store: {
incr: async (args: {
key: string;
pathAndAmountMap: Record<string, number>;
}) => {
calls.push(args);
return { res: {} as Record<string, number>, usage: 0 };
},
},
};
};
it('sets requiresReauth and counts v1 + reauth.token_v1 for a legacy v1 token', async () => {
const stub = makeStubAuth();
stub.setNextResult({
reauth: { reason: 'token_v1', auth_id: 'u-legacy' },
// Some legacy paths resolve an actor anyway (lazy-backfill).
// The probe must still set requiresReauth; gate emits 401.
actor: { user: { uuid: 'u-legacy' } },
} as never);
const kv = makeKvStub();
const probe = createAuthProbe({
authService: stub.service,
kvStore: kv.store,
});
const { req } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer v1-tok' } }),
);
expect(req.requiresReauth).toEqual({
reason: 'token_v1',
auth_id: 'u-legacy',
reauth_token: 'reauth-jwt:u-legacy',
});
// Both increments fire under the same day-bucketed key.
expect(kv.calls).toHaveLength(1);
expect(kv.calls[0].key).toMatch(/^auth-v2:metrics:\d{4}-\d{2}-\d{2}$/);
expect(kv.calls[0].pathAndAmountMap).toEqual({
v1: 1,
'reauth.token_v1': 1,
});
});
it('sets requiresReauth and counts reauth.session_revoked', async () => {
const stub = makeStubAuth();
stub.setNextResult({
reauth: { reason: 'session_revoked', auth_id: 'u-1' },
});
const kv = makeKvStub();
const probe = createAuthProbe({
authService: stub.service,
kvStore: kv.store,
});
const { req } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer tok' } }),
);
expect(req.requiresReauth?.reason).toBe('session_revoked');
expect(kv.calls[0].pathAndAmountMap).toEqual({
v1: 1,
'reauth.session_revoked': 1,
});
});
it('sets requiresReauth and counts reauth.session_expired', async () => {
const stub = makeStubAuth();
stub.setNextResult({
reauth: { reason: 'session_expired', auth_id: 'u-1' },
});
const kv = makeKvStub();
const probe = createAuthProbe({
authService: stub.service,
kvStore: kv.store,
});
const { req } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer tok' } }),
);
expect(req.requiresReauth?.reason).toBe('session_expired');
expect(kv.calls[0].pathAndAmountMap).toEqual({
v1: 1,
'reauth.session_expired': 1,
});
});
it('counts v2 verify (no reauth) for a healthy token', async () => {
const stub = makeStubAuth({ user: { uuid: 'u-1' } });
const kv = makeKvStub();
const probe = createAuthProbe({
authService: stub.service,
kvStore: kv.store,
});
const { req } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer good-tok' } }),
);
expect(req.actor).toBeTruthy();
expect(req.requiresReauth).toBeUndefined();
expect(kv.calls[0].pathAndAmountMap).toEqual({ v2: 1 });
});
it('does not increment counters when no token is presented', async () => {
const stub = makeStubAuth();
const kv = makeKvStub();
const probe = createAuthProbe({
authService: stub.service,
kvStore: kv.store,
});
await runProbe(probe, makeReq({}));
// Anonymous-OK routes pay no metrics cost.
expect(kv.calls).toHaveLength(0);
});
it('absorbs KV failures — never rejects on the hot path', async () => {
const stub = makeStubAuth({ user: { uuid: 'u-1' } });
// Failing KV: increments throw. Probe must still succeed.
const failingKv = {
incr: async () => {
throw new Error('kv unavailable');
},
};
const probe = createAuthProbe({
authService: stub.service,
kvStore: failingKv,
});
const { req, next } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer good-tok' } }),
);
expect(next).toHaveBeenCalledWith();
expect(req.actor).toBeTruthy();
});
it('works without a kvStore (counters become no-ops)', async () => {
const stub = makeStubAuth({ user: { uuid: 'u-1' } });
// Production may not always pass a KV — fresh installs without
// dynamo wired still need a functioning probe.
const probe = createAuthProbe({ authService: stub.service });
const { req } = await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer good-tok' } }),
);
expect(req.actor).toBeTruthy();
expect(req.requiresReauth).toBeUndefined();
});
it('logs `[auth-v2] reauth reason=<r> auth_id=<id>` per event', async () => {
// The log line is the human-facing forensic counterpart to the
// KV counter. Asserting the exact shape so ops can `grep
// '\[auth-v2\] reauth'` and trust the format won't drift.
const stub = makeStubAuth();
stub.setNextResult({
reauth: { reason: 'session_revoked', auth_id: 'u-grep' },
});
const infoSpy = vi.spyOn(console, 'info').mockImplementation(() => {});
try {
const probe = createAuthProbe({ authService: stub.service });
await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer tok' } }),
);
expect(infoSpy).toHaveBeenCalledWith(
'[auth-v2] reauth reason=session_revoked auth_id=u-grep',
);
} finally {
infoSpy.mockRestore();
}
});
it('logs `auth_id=-` when the reauth result has no auth_id', async () => {
const stub = makeStubAuth();
stub.setNextResult({
reauth: { reason: 'session_expired' },
});
const infoSpy = vi.spyOn(console, 'info').mockImplementation(() => {});
try {
const probe = createAuthProbe({ authService: stub.service });
await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer tok' } }),
);
expect(infoSpy).toHaveBeenCalledWith(
'[auth-v2] reauth reason=session_expired auth_id=-',
);
} finally {
infoSpy.mockRestore();
}
});
it('does not emit a reauth log line on a healthy v2 verify', async () => {
const stub = makeStubAuth({ user: { uuid: 'u-1' } });
const infoSpy = vi.spyOn(console, 'info').mockImplementation(() => {});
try {
const probe = createAuthProbe({ authService: stub.service });
await runProbe(
probe,
makeReq({ headers: { authorization: 'Bearer good-tok' } }),
);
const reauthCalls = infoSpy.mock.calls.filter((args) =>
String(args[0]).startsWith('[auth-v2] reauth'),
);
expect(reauthCalls).toHaveLength(0);
} finally {
infoSpy.mockRestore();
}
});
});
// ── Server-backed end-to-end (real AuthService + DB) ────────────────
//
// The unit tests above cover the extraction logic in isolation. This
+74 -20
View File
@@ -18,7 +18,11 @@
*/
import type { Request, RequestHandler } from 'express';
import type { AuthService } from '../../../services/auth/AuthService';
import type {
AuthService,
ReauthReason,
} from '../../../services/auth/AuthService';
import type { SystemKVStore } from '../../../stores/systemKv/SystemKVStore';
// Ensure the `Request.actor` / `Request.token` augmentation is in scope
// wherever this middleware is imported.
@@ -28,8 +32,25 @@ interface AuthProbeOptions {
authService: AuthService;
/** Name of the session cookie to inspect. Falls back to `config.cookie_name`. */
cookieName?: string;
kvStore?: SystemKVStore;
}
const authV2MetricsKey = (): string => {
const now = new Date();
const yyyy = now.getUTCFullYear();
const mm = String(now.getUTCMonth() + 1).padStart(2, '0');
const dd = String(now.getUTCDate()).padStart(2, '0');
return `auth-v2:metrics:${yyyy}-${mm}-${dd}`;
};
const bumpCounter = (
kv: Pick<SystemKVStore, 'incr'> | undefined,
pathAndAmountMap: Record<string, number>,
): void => {
if (!kv) return;
kv.incr({ key: authV2MetricsKey(), pathAndAmountMap }).catch(() => {});
};
/**
* Non-enforcing auth probe. Runs globally (installed by `PuterServer`) on
* every request, tries to locate a token in the usual places, and — if one
@@ -49,7 +70,7 @@ interface AuthProbeOptions {
* 6. Socket handshake query (for ws upgrades that pass through HTTP first)
*/
export const createAuthProbe = (opts: AuthProbeOptions): RequestHandler => {
const { authService, cookieName } = opts;
const { authService, cookieName, kvStore } = opts;
return async (req, _res, next): Promise<void> => {
// If something upstream already attached an actor, respect it.
if (req.actor) {
@@ -64,24 +85,45 @@ export const createAuthProbe = (opts: AuthProbeOptions): RequestHandler => {
}
try {
const actor = await authService.authenticateFromToken(token);
if (actor) {
req.actor = actor;
// Thread the request IP and User-Agent into authenticate so
// SessionStore.touch can refresh `last_ip` / `last_user_agent`
// when a session roams to a new network / browser.
const result = await authService.authenticate(token, {
ip: req.ip,
userAgent: req.headers['user-agent'] ?? undefined,
});
if (result.reauth) {
bumpCounter(kvStore, {
v1: 1,
[`reauth.${result.reauth.reason}`]: 1,
});
// Bind a short-lived JWT proving the rejected session
// identified this auth_id. The GUI echoes this back on
// /login or /signup; the raw auth_id is informational
// only and is not accepted as authoritative on its own.
const reauth_token = result.reauth.auth_id
? authService.signReauthToken(result.reauth.auth_id)
: undefined;
req.requiresReauth = {
reason: result.reauth.reason as ReauthReason,
auth_id: result.reauth.auth_id,
...(reauth_token ? { reauth_token } : {}),
};
console.info(
`[auth-v2] reauth reason=${result.reauth.reason} auth_id=${result.reauth.auth_id ?? '-'}`,
);
} else if (result.actor) {
bumpCounter(kvStore, { v2: 1 });
}
if (result.actor) {
req.actor = result.actor;
req.token = token;
} else {
// Token was present but didn't resolve — covers v1-shape
// tokens v2 can't authenticate (e.g. FPE-encrypted session
// UUIDs), as well as tokens whose session/user/app rows
// have been deleted. `requireAuthGate` reads this flag
// to emit `token_auth_failed` (matching v1) so clients
// trigger their re-login flow instead of retrying.
} else if (result.invalid) {
req.tokenAuthFailed = true;
}
} catch {
// Probe never rejects — invalid tokens just leave `req.actor`
// undefined. Same flag as above so the gate's response shape
// is identical regardless of whether the failure came from
// jwt.verify or a downstream lookup.
req.tokenAuthFailed = true;
}
next();
@@ -130,10 +172,22 @@ const extractToken = (req: Request, cookieName?: string): string | null => {
// arbitrary browser Origin spend an ambient session cookie against the
// credentialed API CORS surface; bearer/body/x-api-key tokens remain
// available for cross-origin SDK requests.
if (cookieName && !isCrossOriginBrowserRequest(req)) {
const cookieToken = req.cookies?.[cookieName];
if (typeof cookieToken === 'string' && cookieToken.length > 0) {
return stripBearer(cookieToken);
//
// `puter_token_v2` is the cookie companion to v2 app-under-user
// tokens set by `POST /auth/migrate-token`. We accept it under the
// same same-origin gate as the primary session cookie so a private
// app iframe can authenticate subsequent calls without re-attaching
// an `Authorization` header on every request.
if (!isCrossOriginBrowserRequest(req)) {
if (cookieName) {
const cookieToken = req.cookies?.[cookieName];
if (typeof cookieToken === 'string' && cookieToken.length > 0) {
return stripBearer(cookieToken);
}
}
const v2Token = req.cookies?.puter_token_v2;
if (typeof v2Token === 'string' && v2Token.length > 0) {
return stripBearer(v2Token);
}
}
+2 -2
View File
@@ -59,7 +59,7 @@ function readTransactionValue(result) {
return result[1];
}
// ── Public API ──────────────────────────────────────────────────────
// -- Public API ------------------------------------------------------
/** Generate a captcha image + token pair. */
export async function generateCaptcha(difficulty = 'medium') {
@@ -96,7 +96,7 @@ export async function verifyCaptcha(token, answer) {
return text === answer.toLowerCase().trim();
}
// ── Route middleware ────────────────────────────────────────────────
// -- Route middleware ------------------------------------------------
/**
* Captcha gate middleware factory.
@@ -136,6 +136,53 @@ describe('requireAuthGate', () => {
});
expectHttpError(got, 403, 'forbidden');
});
// ── Reauth signal ───────────────────────────────────────────────
it('returns 401 reauth_required for a legacy v1 token', () => {
const got = runGate(requireAuthGate(), {
requiresReauth: { reason: 'token_v1', auth_id: 'u-1' },
});
expectHttpError(got, 401, 'reauth_required');
expect((got as HttpError).fields).toMatchObject({
code: 'reauth_required',
reason: 'token_v1',
auth_id: 'u-1',
});
});
it('returns 401 reauth_required with reason=session_revoked', () => {
const got = runGate(requireAuthGate(), {
requiresReauth: { reason: 'session_revoked', auth_id: 'u-2' },
});
expectHttpError(got, 401, 'reauth_required');
expect((got as HttpError).fields).toMatchObject({
reason: 'session_revoked',
auth_id: 'u-2',
});
});
it('returns 401 reauth_required with reason=session_expired', () => {
const got = runGate(requireAuthGate(), {
requiresReauth: { reason: 'session_expired' },
});
expectHttpError(got, 401, 'reauth_required');
expect((got as HttpError).fields).toMatchObject({
reason: 'session_expired',
});
// No auth_id field at all when none was supplied (vs. set-to-undefined).
expect((got as HttpError).fields?.auth_id).toBeUndefined();
});
it('reauth_required takes priority over tokenAuthFailed', () => {
// Both flags set: the structured reauth signal wins. v2 clients
// key on `code === 'reauth_required'`; v1 clients still see a 401.
const got = runGate(requireAuthGate(), {
requiresReauth: { reason: 'token_v1', auth_id: 'u-1' },
tokenAuthFailed: true,
});
expectHttpError(got, 401, 'reauth_required');
});
});
// ── requireUserActorGate ────────────────────────────────────────────
@@ -226,6 +273,35 @@ describe('adminOnlyGate', () => {
'forbidden',
);
});
it('admits the built-ins regardless of case', () => {
// On a SQLite self-host with case-sensitive (BINARY) collation, a
// user row could exist with `Admin`/`SYSTEM` capitalization. The
// gate must still admit those — case is normalized on both sides.
for (const username of ['Admin', 'ADMIN', 'sYsTeM']) {
const got = runGate(adminOnlyGate(), {
actor: { user: { uuid: 'u-1', username } },
});
expect(got).toBeUndefined();
}
});
it('admits case-mismatched extras (allowlist lowercased on construction)', () => {
// Pass an extra in mixed case; the gate must accept the same name
// in any case — the on-disk username row may be either, depending
// on the DB's column collation.
const gate = adminOnlyGate(['Daniel']);
expect(
runGate(gate, {
actor: { user: { uuid: 'u-1', username: 'daniel' } },
}),
).toBeUndefined();
expect(
runGate(gate, {
actor: { user: { uuid: 'u-1', username: 'DANIEL' } },
}),
).toBeUndefined();
});
});
// ── requireVerifiedGate ─────────────────────────────────────────────
+23 -9
View File
@@ -24,14 +24,22 @@ import { assertVerifiedEmail } from '../verifiedEmail';
// Make sure the `Express.Request.actor` augmentation is in scope.
import '../expressAugmentation';
/**
* Build the 401 a route gate emits when `req.actor` is absent. Splits the
* legacy code so old clients can tell the two cases apart: `token_missing`
* means "send a token", `token_auth_failed` means "your token didn't work,
* re-login". Matches the v1 backend's auth-failure shape so the existing
* client retry-vs-reauth logic doesn't need to change.
*/
const rejectAuth = (req: Request): HttpError => {
if (req.requiresReauth) {
return new HttpError(401, 'Re-authentication required', {
legacyCode: 'reauth_required',
fields: {
code: 'reauth_required',
reason: req.requiresReauth.reason,
...(req.requiresReauth.auth_id
? { auth_id: req.requiresReauth.auth_id }
: {}),
...(req.requiresReauth.reauth_token
? { reauth_token: req.requiresReauth.reauth_token }
: {}),
},
});
}
if (req.tokenAuthFailed) {
return new HttpError(401, 'Authentication failed', {
legacyCode: 'token_auth_failed',
@@ -151,10 +159,16 @@ export const DEFAULT_ADMIN_USERNAMES = ['admin', 'system'] as const;
export const adminOnlyGate = (
extras: readonly string[] = [],
): RequestHandler => {
const allowList = new Set<string>([...DEFAULT_ADMIN_USERNAMES, ...extras]);
// Match the case-insensitivity guarantee of the username column
// (MySQL: ascii_general_ci; SQLite: idx_user_username_nocase). Comparing
// raw-case here would let a stored `Admin` bypass the lowercase allowlist
// on any backend that lets case-collision rows exist.
const allowList = new Set<string>(
[...DEFAULT_ADMIN_USERNAMES, ...extras].map((u) => u.toLowerCase()),
);
return (req, _res, next) => {
const username = req.actor?.user.username;
if (!username || !allowList.has(username)) {
if (!username || !allowList.has(username.toLowerCase())) {
next(
new HttpError(403, 'Only admins may request this resource', {
legacyCode: 'forbidden',
@@ -34,7 +34,7 @@ import {
normalizeHost,
normalizeHostRaw,
renderLoginBootstrapHtml,
resolvePrivateAppForHostedSite,
resolveOwnedAppForHostedSite,
resolvePrivateIdentity,
resolvePublicHostedIdentity,
subdomainFromHost,
@@ -451,7 +451,7 @@ describe('renderLoginBootstrapHtml', () => {
// ── Server-backed helpers (AuthService + DB) ─────────────────────────
//
// `resolvePrivateAppForHostedSite` queries the apps table directly;
// `resolveOwnedAppForHostedSite` queries the apps table directly;
// `resolvePrivateIdentity` / `resolvePublicHostedIdentity` rely on the
// real AuthService for token verification + cookie naming.
@@ -524,7 +524,33 @@ const baseConfig = () =>
protocol: 'http',
} as unknown as IConfig);
describe('resolvePrivateAppForHostedSite', () => {
const createApp = async (
ownerId: number,
indexUrl: string,
opts: { isPrivate?: boolean } = {},
): Promise<{ id: number; uid: string }> => {
const uid = `app-${uuidv4()}`;
await server.clients.db.write(
`INSERT INTO \`apps\` (\`uid\`, \`name\`, \`title\`, \`index_url\`, \`owner_user_id\`, \`is_private\`)
VALUES (?, ?, ?, ?, ?, ?)`,
[
uid,
`app-${uid}`,
`app-${uid}`,
indexUrl,
ownerId,
opts.isPrivate ? 1 : 0,
],
);
const row = (
await server.clients.db.read('SELECT id, uid FROM apps WHERE uid = ?', [
uid,
])
)[0] as { id: number; uid: string };
return row;
};
describe('resolveOwnedAppForHostedSite', () => {
const reqOf = (host: string): Request =>
({
hostname: host,
@@ -532,84 +558,101 @@ describe('resolvePrivateAppForHostedSite', () => {
headers: { host },
}) as unknown as Request;
it('returns the associated app immediately when it is already private', async () => {
it("matches the subdomain owner's app by index_url on the same hosting domain", async () => {
const owner = await makeUser();
const associatedApp = {
id: 1,
uid: 'app-direct',
is_private: 1,
owner_user_id: owner.id,
};
const out = await resolvePrivateAppForHostedSite({
req: reqOf('beans.site.puter.localhost'),
site: { user_id: owner.id, associated_app_id: 1 },
associatedApp,
db: server.clients.db,
config: baseConfig(),
matchedHostingDomain: 'site.puter.localhost',
});
expect(out?.uid).toBe('app-direct');
});
it('falls back to an index_url lookup when associated_app_id is unset', async () => {
const owner = await makeUser();
// Private app registered with a URL on the static hosting domain.
// The visitor lands on the equivalent private host — the helper
// should still find it.
const app = await createPrivateApp(
owner.id,
'http://beans.site.puter.localhost/',
);
const out = await resolvePrivateAppForHostedSite({
req: reqOf('beans.app.puter.localhost'),
site: { user_id: owner.id, associated_app_id: null },
associatedApp: null,
const out = await resolveOwnedAppForHostedSite({
req: reqOf('beans.site.puter.localhost'),
site: { user_id: owner.id },
db: server.clients.db,
config: baseConfig(),
matchedHostingDomain: 'app.puter.localhost',
});
expect(out?.uid).toBe(app.uid);
});
it('returns the original associated app when no index_url matches', async () => {
it('matches across hosting variants (static URL, private host request)', async () => {
const owner = await makeUser();
const associatedApp = {
id: 99,
uid: 'app-public',
is_private: 0,
owner_user_id: owner.id,
};
const out = await resolvePrivateAppForHostedSite({
req: reqOf('orphan.site.puter.localhost'),
site: { user_id: owner.id, associated_app_id: 99 },
associatedApp,
const app = await createPrivateApp(
owner.id,
'http://beans.site.puter.localhost/',
);
const out = await resolveOwnedAppForHostedSite({
req: reqOf('beans.app.puter.localhost'),
site: { user_id: owner.id },
db: server.clients.db,
config: baseConfig(),
matchedHostingDomain: 'site.puter.localhost',
});
expect(out?.uid).toBe('app-public');
expect(out?.uid).toBe(app.uid);
});
it("ignores apps owned by a different user even when index_url matches the host", async () => {
// Regression test for the `associated_app_uid` IDOR: an app owned
// by user B must never resolve as "associated" with a subdomain
// owned by user A, even when the index_url notionally matches.
const a = await makeUser();
const b = await makeUser();
await createPrivateApp(b.id, 'http://beans.site.puter.localhost/');
const out = await resolveOwnedAppForHostedSite({
req: reqOf('beans.site.puter.localhost'),
site: { user_id: a.id },
db: server.clients.db,
config: baseConfig(),
});
expect(out).toBeNull();
});
it('returns a public app when requirePrivate is not set', async () => {
const owner = await makeUser();
const app = await createApp(
owner.id,
'http://beans.site.puter.localhost/',
{ isPrivate: false },
);
const out = await resolveOwnedAppForHostedSite({
req: reqOf('beans.site.puter.localhost'),
site: { user_id: owner.id },
db: server.clients.db,
config: baseConfig(),
});
expect(out?.uid).toBe(app.uid);
});
it('filters out non-private apps when requirePrivate is set', async () => {
const owner = await makeUser();
await createApp(
owner.id,
'http://beans.site.puter.localhost/',
{ isPrivate: false },
);
const out = await resolveOwnedAppForHostedSite({
req: reqOf('beans.site.puter.localhost'),
site: { user_id: owner.id },
db: server.clients.db,
config: baseConfig(),
requirePrivate: true,
});
expect(out).toBeNull();
});
it('returns null when the site has no owner', async () => {
const out = await resolvePrivateAppForHostedSite({
const out = await resolveOwnedAppForHostedSite({
req: reqOf('beans.site.puter.localhost'),
site: { user_id: null, associated_app_id: null },
associatedApp: null,
site: { user_id: null },
db: server.clients.db,
config: baseConfig(),
matchedHostingDomain: 'site.puter.localhost',
});
expect(out).toBeNull();
});
it("returns null on a request whose host can't be parsed", async () => {
const out = await resolvePrivateAppForHostedSite({
const out = await resolveOwnedAppForHostedSite({
req: reqOf(''),
site: { user_id: 1, associated_app_id: null },
associatedApp: null,
site: { user_id: 1 },
db: server.clients.db,
config: baseConfig(),
matchedHostingDomain: 'site.puter.localhost',
});
expect(out).toBeNull();
});
@@ -632,7 +675,7 @@ describe('resolvePrivateIdentity', () => {
it('returns the sticky private-cookie identity when the token matches the expected app/subdomain/host', async () => {
const user = await makeUser();
const appUid = `app-${uuidv4()}`;
const token = authService.createPrivateAssetToken({
const token = await authService.createPrivateAssetToken({
appUid,
userUid: user.uuid,
subdomain: 'beans',
@@ -657,7 +700,7 @@ describe('resolvePrivateIdentity', () => {
it('falls through to req.actor when the private cookie is for a different app', async () => {
const user = await makeUser();
const wrongToken = authService.createPrivateAssetToken({
const wrongToken = await authService.createPrivateAssetToken({
appUid: `app-${uuidv4()}`,
userUid: user.uuid,
subdomain: 'beans',
@@ -756,7 +799,7 @@ describe('resolvePrivateIdentity', () => {
1,
],
);
const wrongAppToken = authService.getUserAppToken(
const wrongAppToken = await authService.getUserAppToken(
{ user: { id: user.id, uuid: user.uuid } } as unknown as Parameters<
typeof authService.getUserAppToken
>[0],
@@ -795,7 +838,7 @@ describe('resolvePrivateIdentity', () => {
1,
],
);
const matchedToken = authService.getUserAppToken(
const matchedToken = await authService.getUserAppToken(
{ user: { id: user.id, uuid: user.uuid } } as unknown as Parameters<
typeof authService.getUserAppToken
>[0],
@@ -830,7 +873,7 @@ describe('resolvePublicHostedIdentity', () => {
it('returns the cookie identity when present and valid', async () => {
const user = await makeUser();
const appUid = `app-${uuidv4()}`;
const token = authService.createPublicHostedActorToken({
const token = await authService.createPublicHostedActorToken({
appUid,
userUid: user.uuid,
subdomain: 'beans',
@@ -848,7 +891,7 @@ describe('resolvePublicHostedIdentity', () => {
expectedSubdomain: 'beans',
expectedHost: 'beans.site.puter.localhost',
});
expect(out.source).toBe('private-cookie');
expect(out.source).toBe('public-cookie');
expect(out.userUid).toBe(user.uuid);
expect(
(out as { hasValidPublicCookie?: boolean }).hasValidPublicCookie,
@@ -31,9 +31,10 @@ import type { Actor } from '../../actor';
*
* Covers:
* - Host/subdomain parsing against `private_app_hosting_domain(_alt)`.
* - Broader private-app detection for hosted sites whose subdomain row
* has no `associated_app_id` — falls back to an `index_url` lookup
* against the subdomain owner's private apps.
* - Owned-app detection for hosted sites: matches the subdomain owner's
* apps by `index_url` against the request host. Replaces the older
* `associated_app_id`-trusting path, which was user-writable without
* an ownership check.
* - Bootstrap-token identity resolution (`Authorization: Bearer`,
* `?puter.auth.token=`, `X-Puter-Auth-Token`, referrer query) so
* private-app visitors with a valid session token (but no cookie on
@@ -68,6 +69,7 @@ export interface PrivateHostingConfig {
export interface PrivateIdentity {
source:
| 'private-cookie'
| 'public-cookie'
| 'session-cookie'
| 'bootstrap-token'
| 'authorization'
@@ -76,13 +78,12 @@ export interface PrivateIdentity {
| 'none';
userUid?: string;
sessionUuid?: string;
/** True when resolved from the sticky `puter.private.asset.token` cookie. */
/** True when resolved from the sticky private-asset cookie. */
hasValidPrivateCookie?: boolean;
}
interface SubdomainLike {
user_id?: number | null;
associated_app_id?: number | null;
}
interface AppLike {
@@ -95,7 +96,7 @@ interface AppLike {
index_url?: string | null;
}
// ── Host helpers ────────────────────────────────────────────────────
// -- Host helpers ----------------------------------------------------
export function normalizeHost(value: string | undefined | null): string | null {
if (typeof value !== 'string') return null;
@@ -149,7 +150,7 @@ export function hostMatchesPrivateDomain(
return privateDomains.some((pd) => host === pd || host.endsWith(`.${pd}`));
}
// ── Subdomain extraction from a hosted request ─────────────────────
// -- Subdomain extraction from a hosted request ---------------------
export function subdomainFromHost(
host: string,
@@ -169,43 +170,39 @@ export function subdomainFromHost(
return host.split('.')[0] || '';
}
// ── Private-app detection fallback (v1 `resolvePrivateAppForHostedSite`)
// -- Hosted-site → owned-app resolution -----------------------------
/**
* When the subdomain row has no `associated_app_id`, v1 looked up the
* owner's private apps and matched `index_url` against the request host
* variants (this-subdomain × every hosting domain). Ports that logic.
* Resolve "what app does the subdomain owner run on this host?" by matching
* apps owned by `site.user_id` against the request's host variants. Used by
* the puter-site middleware to decide whether the request hits a private
* app (gate) or a public app (sticky cookie), and by the subdomain driver
* to populate the `associated_app` field in API responses.
*
* Returns the matched private app (if any) so the caller can run the
* normal access check — closes a gap where a private app's files could
* be served via a subdomain whose row wasn't explicitly linked.
* Ownership-anchored on `apps.owner_user_id = site.user_id` so a subdomain
* can never "claim" an app belonging to a different user. The `subdomains`
* row's own `associated_app_id` column is intentionally ignored — it was
* previously user-writable without an ownership check, so any value there
* is either legacy or planted; deriving from `index_url` makes the answer
* fall out of facts the system already verifies at app-create time.
*/
export async function resolvePrivateAppForHostedSite(opts: {
export async function resolveOwnedAppForHostedSite(opts: {
req: Request;
site: SubdomainLike;
associatedApp: AppLike | null;
db: AbstractDatabaseClient;
config: PrivateHostingConfig;
matchedHostingDomain: string;
requirePrivate?: boolean;
}): Promise<AppLike | null> {
// When the subdomain row's own `associated_app_id` points at a private
// app, that wins outright. Otherwise fall through to index_url matching
// so a private app whose canonical URL lives on one hosting variant
// (`beans.puter.site`) still resolves when the visitor hits another
// (`beans.puter.app`).
if (opts.associatedApp && Number(opts.associatedApp.is_private ?? 0) > 0) {
return opts.associatedApp;
}
if (!opts.site?.user_id) return opts.associatedApp ?? null;
if (!opts.site?.user_id) return null;
const host = normalizeHost(opts.req.hostname);
if (!host) return opts.associatedApp ?? null;
if (!host) return null;
const hostedSubdomain = subdomainFromHost(host, [
...opts.config.staticDomains,
...opts.config.privateDomains,
]);
if (!hostedSubdomain) return opts.associatedApp ?? null;
if (!hostedSubdomain) return null;
// Build host variants with AND without port, then cross each with both
// protocols. Apps store whatever URL the user typed at create time, so
@@ -244,16 +241,19 @@ export async function resolvePrivateAppForHostedSite(opts: {
}
}
const uniqueCandidates = [...new Set(urlCandidates)];
if (uniqueCandidates.length === 0) return opts.associatedApp ?? null;
if (uniqueCandidates.length === 0) return null;
const privateFilter = opts.requirePrivate
? `AND \`is_private\` = ${opts.db.booleanLiteral(true)} `
: '';
const placeholders = uniqueCandidates.map(() => '?').join(', ');
const rows = await opts.db.read(
`SELECT * FROM apps WHERE owner_user_id = ? AND is_private = 1 AND index_url IN (${placeholders}) LIMIT 2`,
`SELECT * FROM apps WHERE owner_user_id = ? ${privateFilter}AND index_url IN (${placeholders}) LIMIT 2`,
[opts.site.user_id, ...uniqueCandidates],
);
if (rows.length === 0) return opts.associatedApp ?? null;
if (rows.length === 0) return null;
if (rows.length > 1) {
console.warn('[puter-site] private_access.host_match_ambiguous', {
console.warn('[puter-site] hosted_site_app_match_ambiguous', {
requestHost: host,
matchCount: rows.length,
});
@@ -261,7 +261,7 @@ export async function resolvePrivateAppForHostedSite(opts: {
return rows[0] as unknown as AppLike;
}
// ── Bootstrap token resolution ──────────────────────────────────────
// -- Bootstrap token resolution --------------------------------------
function getAuthorizationToken(req: Request): string | null {
const header = req.headers?.authorization;
@@ -348,12 +348,20 @@ export async function resolvePrivateIdentity(opts: {
const cookies = (req as Request & { cookies?: Record<string, string> })
.cookies;
// 1. Sticky private-asset cookie.
const privateCookieName = authService.getPrivateAssetCookieName();
// 1. Sticky private-asset cookie. Prefer the v2 cookie name; fall
// back to the legacy dot-style name while the deprecation window
// is open. A v1 (legacy-secret) token verifies but is intentionally
// NOT treated as a valid sticky cookie — we fall through so the
// chain re-mints under the v2 secret on this response.
const v2CookieName = authService.getPrivateAssetCookieNameV2();
const legacyCookieName = authService.getPrivateAssetCookieName();
const privateCookieToken =
typeof cookies?.[privateCookieName] === 'string'
? cookies[privateCookieName]
: null;
(typeof cookies?.[v2CookieName] === 'string'
? cookies[v2CookieName]
: null) ??
(typeof cookies?.[legacyCookieName] === 'string'
? cookies[legacyCookieName]
: null);
if (privateCookieToken) {
try {
const claims = await authService.verifyPrivateAssetToken(
@@ -364,12 +372,15 @@ export async function resolvePrivateIdentity(opts: {
expectedPrivateHost,
},
);
return {
source: 'private-cookie',
userUid: claims.userUid,
sessionUuid: claims.sessionUuid,
hasValidPrivateCookie: true,
};
if (!claims.legacy) {
return {
source: 'private-cookie',
userUid: claims.userUid,
sessionUuid: claims.sessionUuid,
hasValidPrivateCookie: true,
};
}
/* legacy cookie: fall through so the caller re-mints v2 */
} catch {
/* fall through — stale / mismatched / logged-out cookie */
}
@@ -480,14 +491,18 @@ export async function resolvePublicHostedIdentity(opts: {
const cookies = (req as Request & { cookies?: Record<string, string> })
.cookies;
const publicCookieName = authService.getPublicHostedActorCookieName();
const publicCookieNameV2 = authService.getPublicHostedActorCookieNameV2();
const publicCookieNameLegacy = authService.getPublicHostedActorCookieName();
const publicCookieToken =
typeof cookies?.[publicCookieName] === 'string'
? cookies[publicCookieName]
: null;
(typeof cookies?.[publicCookieNameV2] === 'string'
? cookies[publicCookieNameV2]
: null) ??
(typeof cookies?.[publicCookieNameLegacy] === 'string'
? cookies[publicCookieNameLegacy]
: null);
if (publicCookieToken) {
try {
const claims = authService.verifyPublicHostedActorToken(
const claims = await authService.verifyPublicHostedActorToken(
publicCookieToken,
{
expectedAppUid,
@@ -495,12 +510,15 @@ export async function resolvePublicHostedIdentity(opts: {
expectedHost,
},
);
return {
source: 'private-cookie',
userUid: claims.userUid,
sessionUuid: claims.sessionUuid,
hasValidPublicCookie: true,
};
if (!claims.legacy) {
return {
source: 'public-cookie',
userUid: claims.userUid,
sessionUuid: claims.sessionUuid,
hasValidPublicCookie: true,
};
}
/* legacy cookie: fall through so the caller re-mints v2 */
} catch {
/* fall through */
}
@@ -564,7 +582,7 @@ export async function resolvePublicHostedIdentity(opts: {
return { source: 'none' };
}
// ── Redirect helpers ────────────────────────────────────────────────
// -- Redirect helpers ------------------------------------------------
/** Build the URL to redirect a private-app request to its private host. */
export function buildPrivateHostRedirect(
@@ -645,7 +663,7 @@ export function buildAppCenterFallback(
return `https://${config.domain}/app/app-center/?item=${encodeURIComponent(appName)}`;
}
// ── Login bootstrap HTML ────────────────────────────────────────────
// -- Login bootstrap HTML --------------------------------------------
/**
* Minimal HTML page that prompts the visitor to sign in with Puter.
@@ -629,7 +629,7 @@ describe('createPuterSiteMiddleware — file serving', () => {
expect(String(out.body)).toContain('Not Found');
});
it('returns 404 when the resolved URL points at a directory rather than a file', async () => {
it('returns 404 when the resolved URL points at a directory with no index.html', async () => {
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
@@ -645,8 +645,8 @@ describe('createPuterSiteMiddleware — file serving', () => {
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
// Documents/ exists from generateDefaultFsentries — a
// directory entry, which the file branch must refuse.
// Documents/ exists from generateDefaultFsentries but has
// no index.html — directory fallback must still 404.
path: '/Documents',
}),
res,
@@ -657,6 +657,45 @@ describe('createPuterSiteMiddleware — file serving', () => {
expect(out.contentType).toBe('text/html; charset=UTF-8');
});
it('serves <folder>/index.html when the URL resolves to a folder containing one', async () => {
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
const sub = `folderidx-${Math.random().toString(36).slice(2, 8)}`;
await server.stores.subdomain.create({
userId: owner.id,
subdomain: sub,
rootDirId: homeEntry!.id,
});
const body = Buffer.from('nested doc');
await writeFile(
owner.id,
`${homePath}/Documents/index.html`,
body,
'text/html',
);
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
// No trailing slash; folder exists and contains index.html.
path: '/Documents',
}),
res,
vi.fn(),
);
// Allow the piped stream to flush.
await new Promise<void>((resolve) => setImmediate(resolve));
expect(out.statusCode).toBe(200);
expect(out.headers['Content-Type']).toMatch(/text\/html/);
const piped = out.body as Buffer | undefined;
expect(Buffer.isBuffer(piped)).toBe(true);
expect(piped!.equals(body)).toBe(true);
});
it("serves the HTML SUBDOMAIN_404 when the subdomain's root_dir_id points to a missing entry", async () => {
// Subdomain row references a fsentry id that doesn't exist —
// earlier path resolution must catch this with the HTML 404 page.
@@ -781,3 +820,344 @@ describe('createPuterSiteMiddleware — file serving', () => {
expect(out.statusCode).toBe(404);
});
});
// ── .puter_site_config (custom error pages) ─────────────────────────
//
// Sites can drop a `.puter_site_config` JSON file at the root of their
// hosting directory to map error codes onto custom pages — the
// canonical use case is SPA fallback (404 → /index.html with status
// 200). These tests pin the contract end-to-end through the real
// FSService so we exercise parsing, path resolution, and the loop-
// prevention guard together.
describe('createPuterSiteMiddleware — .puter_site_config', () => {
const setupSiteWithConfig = async (config: unknown) => {
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
const sub = `cfg-${Math.random().toString(36).slice(2, 8)}`;
await server.stores.subdomain.create({
userId: owner.id,
subdomain: sub,
rootDirId: homeEntry!.id,
});
await writeFile(
owner.id,
`${homePath}/.puter_site_config`,
Buffer.from(JSON.stringify(config)),
'application/json',
);
return { owner, homePath, sub };
};
it('serves /index.html with status 200 on 404 when configured (SPA fallback)', async () => {
// The user's headline use case: route any unknown path through
// the SPA entrypoint so client-side routing can take over,
// while still serving HTTP 200 so search engines don't cache
// the page as a hard 404.
const { owner, homePath, sub } = await setupSiteWithConfig({
errors: {
'404': { file: '/index.html', status: 200 },
},
});
const body = Buffer.from('<html>spa-shell</html>');
await writeFile(owner.id, `${homePath}/index.html`, body, 'text/html');
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/some/client-route',
}),
res,
vi.fn(),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(out.statusCode).toBe(200);
expect(out.headers['Content-Type']).toMatch(/text\/html/);
const piped = out.body as Buffer | undefined;
expect(Buffer.isBuffer(piped)).toBe(true);
expect(piped!.equals(body)).toBe(true);
});
it('serves the configured file with the matched code when `status` is omitted', async () => {
// No explicit `status` → default to the matched error code
// (404 here). Bare `{ file: '/404.html' }` should Just Work
// for the classic "pretty 404 page" use case.
const { owner, homePath, sub } = await setupSiteWithConfig({
errors: { '404': { file: '/404.html' } },
});
const body = Buffer.from('<html>custom 404</html>');
await writeFile(owner.id, `${homePath}/404.html`, body, 'text/html');
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/no-such-page',
}),
res,
vi.fn(),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(out.statusCode).toBe(404);
const piped = out.body as Buffer | undefined;
expect(piped!.equals(body)).toBe(true);
});
it('falls back to the default 404 page when the configured error file does not exist (no infinite loop)', async () => {
// Critical loop guard: if the error page itself is missing,
// we must NOT recurse through the error config again. The
// request must terminate with the built-in 404 page rather
// than spinning errors.404 → errors.404 → … .
const { sub } = await setupSiteWithConfig({
errors: { '404': { file: '/missing.html', status: 200 } },
});
// Deliberately do not write missing.html.
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/anything',
}),
res,
vi.fn(),
);
expect(out.statusCode).toBe(404);
expect(out.contentType).toBe('text/html; charset=UTF-8');
expect(String(out.body)).toContain('Not Found');
});
it('returns 404 when a visitor requests `.puter_site_config` directly (no config leak)', async () => {
// The config file is implementation detail — hide it the same
// way any other missing path would 404, so visitors can't
// enumerate deployment shape by guessing well-known names.
const { sub } = await setupSiteWithConfig({
errors: { '404': { file: '/index.html', status: 200 } },
});
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/.puter_site_config',
}),
res,
vi.fn(),
);
// No error-page fallback because we didn't write /index.html.
expect(out.statusCode).toBe(404);
// Body shape is the default 404, not the JSON config.
expect(String(out.body)).not.toContain('errors');
});
it('rejects `errors.404.file` paths that try to climb out of the site root', async () => {
// The normalizer collapses `..` segments, so an attacker who
// can edit the config can't pivot it into reading files
// outside their own subdomain root.
const otherOwner = await makeUserWithHome();
const secretPath = `/${otherOwner.username}/secret.html`;
await writeFile(
otherOwner.id,
secretPath,
Buffer.from('SECRET'),
'text/html',
);
const { sub } = await setupSiteWithConfig({
errors: {
'404': {
// Tries to walk up to the other user's home dir.
file: `/../${otherOwner.username}/secret.html`,
status: 200,
},
},
});
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/anything',
}),
res,
vi.fn(),
);
// The collapsed path resolves under the requesting site's
// root, which doesn't have a `<username>/secret.html` file —
// so the fallback fails and we get the default 404, NOT the
// SECRET body.
expect(out.statusCode).toBe(404);
expect(String(out.body)).not.toContain('SECRET');
});
it('ignores malformed JSON configs and behaves like there is no config', async () => {
// A typo in the config file must not 5xx the request — the
// visitor still gets the default 404 on a missing path.
const owner = await makeUserWithHome();
const homePath = `/${owner.username}`;
const homeEntry = await server.stores.fsEntry.getEntryByPath(homePath);
const sub = `bad-${Math.random().toString(36).slice(2, 8)}`;
await server.stores.subdomain.create({
userId: owner.id,
subdomain: sub,
rootDirId: homeEntry!.id,
});
await writeFile(
owner.id,
`${homePath}/.puter_site_config`,
Buffer.from('this is not json {{{'),
'application/json',
);
await writeFile(
owner.id,
`${homePath}/index.html`,
Buffer.from('would-be-fallback'),
'text/html',
);
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/unknown',
}),
res,
vi.fn(),
);
expect(out.statusCode).toBe(404);
expect(String(out.body)).toContain('Not Found');
// Did NOT silently fall back to /index.html as if the config
// were valid — malformed config must be ignored, not partially
// applied.
expect(String(out.body)).not.toContain('would-be-fallback');
});
it('still serves real files normally — config only applies on 404', async () => {
// Sanity check: when the requested path exists, the config's
// error rules are not consulted, so the response is the live
// file at status 200 (not the error-page status override).
const { owner, homePath, sub } = await setupSiteWithConfig({
errors: { '404': { file: '/index.html', status: 200 } },
});
const body = Buffer.from('real-page');
await writeFile(
owner.id,
`${homePath}/page.html`,
body,
'text/html',
);
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/page.html',
}),
res,
vi.fn(),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(out.statusCode).toBe(200);
const piped = out.body as Buffer | undefined;
expect(piped!.equals(body)).toBe(true);
});
it('serves the cached config on a second request, even when the on-disk file is removed (Redis cache holds within TTL)', async () => {
// First request seeds the cache with the parsed config. We
// then delete the underlying file and fire a second request:
// if the cache is wired correctly, the SPA fallback still
// applies because we never re-read from S3. Within-TTL
// staleness is the explicit contract (60s default).
const { owner, homePath, sub } = await setupSiteWithConfig({
errors: { '404': { file: '/index.html', status: 200 } },
});
const body = Buffer.from('<html>spa-shell</html>');
await writeFile(owner.id, `${homePath}/index.html`, body, 'text/html');
const mw = buildMiddleware();
// First request → primes the cache.
const first = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/route-a',
}),
first.res,
vi.fn(),
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(first.out.statusCode).toBe(200);
// Now wipe the on-disk config. If the loader hits S3 on every
// request, the second call below would see no config and fall
// through to a default 404. The cache contract is that it
// does NOT — within the TTL, the prior parse stands.
const configEntry = await server.stores.fsEntry.getEntryByPath(
`${homePath}/.puter_site_config`,
);
if (configEntry) {
await server.services.fs.remove(owner.id, { entry: configEntry });
}
const second = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/route-b',
}),
second.res,
vi.fn(),
);
await new Promise<void>((resolve) => setImmediate(resolve));
// SPA fallback still fires → cache served the deleted config.
expect(second.out.statusCode).toBe(200);
expect(
(second.out.body as Buffer | undefined)?.equals(body),
).toBe(true);
});
it('ignores `errors` entries with status codes outside 4xx/5xx', async () => {
// A `200` key in errors is meaningless and a footgun (it
// could be used to silently override the happy path). The
// parser must drop these on the floor.
const { sub } = await setupSiteWithConfig({
errors: {
'200': { file: '/oops.html', status: 200 },
'999': { file: '/oops.html', status: 200 },
},
});
const mw = buildMiddleware();
const { res, out } = makeRes();
await mw(
makeReq({
hostname: `${sub}.site.puter.localhost`,
path: '/missing',
}),
res,
vi.fn(),
);
// No valid 404 rule survives the filter → default 404.
expect(out.statusCode).toBe(404);
expect(String(out.body)).toContain('Not Found');
});
});
+134 -41
View File
@@ -21,9 +21,9 @@ import type { RequestHandler } from 'express';
import { contentType as contentTypeFromMime } from 'mime-types';
import { posix as pathPosix } from 'node:path';
import type { puterClients } from '../../../clients';
import { FS_COSTS } from '../../../controllers/fs/costs';
import type { puterServices } from '../../../services';
import type { puterStores } from '../../../stores';
import { FS_COSTS } from '../../../controllers/fs/costs';
import type { IConfig, LayerInstances } from '../../../types';
import {
buildAppCenterFallback,
@@ -32,10 +32,16 @@ import {
buildPublicHostRedirect,
hostMatchesPrivateDomain,
renderLoginBootstrapHtml,
resolvePrivateAppForHostedSite,
resolveOwnedAppForHostedSite,
resolvePrivateIdentity,
resolvePublicHostedIdentity,
} from './privateAppGate';
import {
isSiteConfigPath,
loadSiteConfig,
resolveErrorTarget,
type SiteConfig,
} from './puterSiteConfig';
/**
* Serves user-hosted static sites on the hosting domains (`*.puter.site`,
@@ -52,9 +58,15 @@ import {
*
* Deferred (not yet implemented):
* - `.at` username-based sites (UUIDv5-keyed `/user/Public`).
* - `.puter_site_config` error rules (custom status-code → file mapping).
* - Custom domains (subdomains table `domain` column) — requires host
* validation to allow arbitrary hostnames first.
*
* Site config:
* - `.puter_site_config` at the site root (see `puterSiteConfig.ts`)
* supplies custom error pages. On a file 404, the matching rule's
* `file` is served with the rule's `status` — supports the SPA
* fallback pattern of `404 → /index.html with status 200`. The
* config file itself is hidden from public serving.
*/
const SUBDOMAIN_404 = `<div style="font-size: 20px;
@@ -192,29 +204,21 @@ export const createPuterSiteMiddleware = (
const hostingCfg = buildHostingConfig(config);
let associatedApp: AppRow | null = null;
if (
site.associated_app_id !== null &&
site.associated_app_id !== undefined
) {
associatedApp = (await layers.stores.app.getById(
site.associated_app_id,
)) as unknown as AppRow | null;
}
const privateApp = (await resolvePrivateAppForHostedSite({
// The subdomain row's `associated_app_id` column is intentionally
// not consulted — it was previously user-writable without an
// ownership check (so any value there is either legacy or planted).
// Resolve "what app does this subdomain host?" directly from
// `apps.owner_user_id = site.user_id` + `index_url` match against
// the request host.
const associatedApp = (await resolveOwnedAppForHostedSite({
req,
site: {
user_id: site.user_id,
associated_app_id: site.associated_app_id,
},
associatedApp,
site: { user_id: site.user_id },
db: layers.clients.db,
config: hostingCfg,
matchedHostingDomain: matched,
})) as AppRow | null;
const isPrivateApp = Boolean(privateApp?.is_private);
const isPrivateApp = Boolean(associatedApp?.is_private);
const privateApp = isPrivateApp ? associatedApp : null;
if (isPrivateApp) {
// Private apps must run on the private hosting domain. If a
@@ -319,15 +323,16 @@ export const createPuterSiteMiddleware = (
// visitors but still refreshes after rotation/expiry.
if (!identity.hasValidPrivateCookie) {
try {
const token = layers.services.auth.createPrivateAssetToken({
appUid: privateApp!.uid,
userUid: identity.userUid,
sessionUuid: identity.sessionUuid,
subdomain,
privateHost: host,
});
const token =
await layers.services.auth.createPrivateAssetToken({
appUid: privateApp!.uid,
userUid: identity.userUid,
sessionUuid: identity.sessionUuid,
subdomain,
privateHost: host,
});
res.cookie(
layers.services.auth.getPrivateAssetCookieName(),
layers.services.auth.getPrivateAssetCookieNameV2(),
token,
layers.services.auth.getPrivateAssetCookieOptions({
requestHostname: host,
@@ -386,15 +391,17 @@ export const createPuterSiteMiddleware = (
associatedApp?.uid
) {
const token =
layers.services.auth.createPublicHostedActorToken({
appUid: associatedApp.uid,
userUid: identity.userUid,
sessionUuid: identity.sessionUuid,
subdomain,
host,
});
await layers.services.auth.createPublicHostedActorToken(
{
appUid: associatedApp.uid,
userUid: identity.userUid,
sessionUuid: identity.sessionUuid,
subdomain,
host,
},
);
res.cookie(
layers.services.auth.getPublicHostedActorCookieName(),
layers.services.auth.getPublicHostedActorCookieNameV2(),
token,
layers.services.auth.getPublicHostedActorCookieOptions({
requestHostname: host,
@@ -436,7 +443,16 @@ export const createPuterSiteMiddleware = (
// Resolve URL path → absolute FS path under the site root.
let urlPath = req.path || '/';
if (urlPath.endsWith('/')) urlPath += 'index.html';
const decoded = decodeURIComponent(urlPath);
let decoded: string;
try {
decoded = decodeURIComponent(urlPath);
} catch {
// Malformed `%xx` escape — treat as a missing path, not a 500.
res.status(404)
.type('text/html; charset=UTF-8')
.send('<h1>404</h1><p>Not Found</p>');
return;
}
// pathPosix.normalize strips `..` segments; the join with '/' anchors
// it so traversal can't escape the site root.
const resolvedUrlPath = pathPosix.normalize(
@@ -449,10 +465,63 @@ export const createPuterSiteMiddleware = (
}
const filePath = rootPath + resolvedUrlPath;
// Best-effort site config load. A missing / malformed config
// never blocks the request — `loadSiteConfig` swallows all
// errors and returns null on any failure. The Redis cache
// keyed on `rootDirId` keeps the hot path off S3 for the
// common case where the same subdomain gets repeat visits.
let siteConfig: SiteConfig | null = null;
try {
siteConfig = await loadSiteConfig({
rootPath,
rootDirId: rootEntry.id,
fsEntryStore: layers.stores.fsEntry,
fsService: layers.services.fs,
cache: layers.clients.redis,
});
} catch (e) {
console.warn('[puter-site] loadSiteConfig threw', e);
}
// Hide the config file from public serving — visitors should
// see the same 404 as for any other missing path so the
// deployment shape isn't leaked.
const isConfigRequest = isSiteConfigPath(resolvedUrlPath);
// Subdomain hosting bypasses ACL by design: anything the owner placed
// under the registered root_dir is treated as public. Path traversal
// is blocked above by `pathPosix.normalize` anchoring at `/`.
const entry = await layers.stores.fsEntry.getEntryByPath(filePath);
let entry = isConfigRequest
? null
: await layers.stores.fsEntry.getEntryByPath(filePath);
if (entry?.isDir) {
// Folder request → fall back to <folder>/index.html, the same
// way `/` is rewritten to `/index.html` at the site root above.
entry = await layers.stores.fsEntry.getEntryByPath(
pathPosix.join(filePath, 'index.html'),
);
}
// Custom error page fallback. On a 404 we consult the site config
// for a rule and, if one resolves to an existing file, serve that
// instead. Critical: we do this exactly once — the error page
// itself never re-enters error handling, so a misconfigured
// `errors.404.file` that doesn't exist falls through to the
// default 404 page rather than looping.
let statusOverride: number | undefined;
if (!entry || entry.isDir) {
const errorTarget = resolveErrorTarget(siteConfig, 404, rootPath);
if (errorTarget) {
const candidate = await layers.stores.fsEntry.getEntryByPath(
errorTarget.absPath,
);
if (candidate && !candidate.isDir) {
entry = candidate;
statusOverride = errorTarget.status;
}
}
}
if (!entry || entry.isDir) {
res.status(404)
.type('text/html; charset=UTF-8')
@@ -461,8 +530,13 @@ export const createPuterSiteMiddleware = (
}
// Stream the file. `fsEntry.readContent` honours Range + emits
// ETag/Last-Modified when the S3 layer returns them.
// ETag/Last-Modified when the S3 layer returns them. Range
// requests are suppressed when serving a custom error page so
// the visitor always receives the full document with the
// configured status code (no 206 with a stale `bytes=...`
// header from the original request).
const range =
statusOverride === undefined &&
typeof req.headers.range === 'string'
? req.headers.range
: undefined;
@@ -479,6 +553,25 @@ export const createPuterSiteMiddleware = (
const mime =
contentTypeFromMime(entry.name) || 'application/octet-stream';
res.setHeader('Content-Type', mime);
// Fire-and-forget signal for downstream extensions
if (mime === 'text/html' || mime === 'application/xhtml+xml') {
try {
layers.clients.event.emit(
'site.htmlServed',
{
subdomain,
entry,
host,
requestPath: req.path,
mime,
},
{},
);
} catch (e) {
console.warn('[puter-site] site.htmlServed emit failed', e);
}
}
if (download.contentLength !== null) {
res.setHeader('Content-Length', String(download.contentLength));
}
@@ -489,7 +582,7 @@ export const createPuterSiteMiddleware = (
res.setHeader('Last-Modified', download.lastModified.toUTCString());
res.setHeader('Accept-Ranges', 'bytes');
res.setHeader('Access-Control-Allow-Origin', '*');
res.status(range ? 206 : 200);
res.status(statusOverride ?? (range ? 206 : 200));
// Best-effort egress metering against the site owner. The request
// itself is unauthenticated (public site visitor), so we can't use
@@ -0,0 +1,386 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { posix as pathPosix } from 'node:path';
import type { FSEntry } from '../../../stores/fs/FSEntry';
/**
* Minimal Redis surface we need — `get` / `set` with EX TTL. Typed as a
* subset of ioredis so callers can pass either the real cluster client
* or a mock without needing the full Cluster type here.
*/
export interface SiteConfigCache {
get(key: string): Promise<string | null>;
set(
key: string,
value: string,
mode: 'EX',
ttlSeconds: number,
): Promise<unknown>;
}
/**
* Site-level configuration loaded from `.puter_site_config` at the site
* root. Lets a hosted site customize how the server responds for it —
* currently limited to error-page mapping (e.g. SPA fallback that serves
* `/index.html` for any 404 with a 200 status).
*
* The on-disk JSON shape:
*
* {
* "errors": {
* "404": { "file": "/index.html", "status": 200 }
* }
* }
*
* Other static-hosting platforms (Vercel `vercel.json`, Netlify/Amplify
* `_redirects`, nginx `error_page`) express the same idea with different
* syntax. The internal `SiteConfig` is intentionally narrow so additional
* parsers can normalize to it without churning the consumer in
* `puterSite.ts` — see `SITE_CONFIG_FILENAMES` for the lookup list.
*
* Security posture (every input here originates from a user-uploaded file
* served on the open internet):
* - the on-disk file is size-capped before parse (`MAX_CONFIG_BYTES`),
* - JSON parsing is wrapped in try/catch — a malformed file silently
* falls back to default behavior, never 5xx,
* - every `file` value is normalized as if it were a URL path: it must
* start with `/`, gets `pathPosix.normalize`d so `..` is collapsed,
* and is re-anchored under the site root before any FS lookup,
* - status codes are clamped to a strict allow-list (the request side
* only honours 4xx/5xx error keys; the response side validates the
* `status` is a legitimate HTTP integer),
* - the config file itself is hidden from public serving by the caller
* (`isSiteConfigPath`) — same status/body as any other missing path,
* no separate 403 that would leak its existence.
*
* Loop safety is the consumer's responsibility: when serving an error
* page, do NOT re-consult `errors` if the error page itself is missing,
* otherwise a misconfigured site could spin a 404→404→404 cycle.
*/
export interface SiteErrorRule {
/** Absolute path under the site root (e.g. `/index.html`). */
file: string;
/** HTTP status to return when serving this error page (200–599). */
status: number;
}
export interface SiteConfig {
/** Map of HTTP status code → custom error rule. Keys are 4xx/5xx. */
errors: Record<number, SiteErrorRule>;
}
const MAX_CONFIG_BYTES = 64 * 1024;
// Cache key prefix is distinct from `subdomains:` (SubdomainStore) and
// other Redis users — keep this in sync if you rename, otherwise stale
// entries from prior deploys could be read back as configs.
const CACHE_KEY_PREFIX = 'puter-site-config:';
const CACHE_TTL_SECONDS = 60;
// Sentinel for "we looked, the site has no config" so repeated visits
// to a config-less site don't keep round-tripping S3 to confirm.
const NEGATIVE_CACHE_MARKER = '__none__';
/**
* Filenames consulted at the site root, in priority order. First file
* that parses to a non-empty config wins. Extending this list to add
* Vercel / Netlify / nginx adapters is the entrypoint for multi-format
* support — each parser receives the raw text and returns a normalized
* `SiteConfig` (or null if the file isn't valid in that format).
*/
const SITE_CONFIG_FILENAMES: ReadonlyArray<{
name: string;
parse: (text: string) => SiteConfig | null;
}> = [{ name: '.puter_site_config', parse: parsePuterSiteConfig }];
/**
* Returns true if `urlPath` (already normalized to start with `/`) names
* the site config file. Used by `puterSite.ts` to suppress direct
* serving so the deployment shape isn't leaked to visitors.
*/
export function isSiteConfigPath(urlPath: string): boolean {
const base = pathPosix.basename(urlPath);
return SITE_CONFIG_FILENAMES.some((f) => f.name === base);
}
interface LoadSiteConfigArgs {
/** Absolute site root path (e.g. `/<username>/Public`). */
rootPath: string;
/**
* Stable identifier for the site root, used as the cache key. We
* key on `rootDirId` (not the subdomain) so that renaming a
* subdomain — or pointing multiple subdomains at the same
* directory — neither orphans nor duplicates the cached entry.
*/
rootDirId: number;
fsEntryStore: {
getEntryByPath: (path: string) => Promise<FSEntry | null>;
};
fsService: {
readContent: (
entry: FSEntry,
options?: { range?: string },
) => Promise<{
body: NodeJS.ReadableStream;
contentLength: number | null;
}>;
};
/**
* Optional Redis cache. When omitted, every request re-reads the
* config from S3 — fine for tests, slow for prod. Cache failures
* are swallowed (best-effort): a transient Redis blip just falls
* through to the live read, never errors the request.
*/
cache?: SiteConfigCache;
}
/**
* Locate and parse the site config. Returns null when no config file
* exists, the file is unreadable, oversized, or fails validation —
* callers must treat null as "behave like there is no config" and never
* raise the error to the visitor. Errors are logged for the operator
* but never surfaced to the request.
*/
export async function loadSiteConfig(
args: LoadSiteConfigArgs,
): Promise<SiteConfig | null> {
const { rootPath, rootDirId, fsEntryStore, fsService, cache } = args;
if (!rootPath || rootPath === '/') return null;
// Reject non-positive-integer ids defensively — they'd produce a
// weird cache key and we'd cache something nonsensical against it.
// The store contract is positive integers, but this is a hot path
// for untrusted-origin traffic so we belt-and-brace it.
const cacheable =
Number.isInteger(rootDirId) && rootDirId > 0 && cache !== undefined;
const cacheKey = cacheable ? `${CACHE_KEY_PREFIX}${rootDirId}` : null;
if (cacheable && cacheKey) {
try {
const raw = await cache!.get(cacheKey);
if (raw === NEGATIVE_CACHE_MARKER) return null;
if (typeof raw === 'string' && raw.length > 0) {
// Trust the cached shape — it was produced by this
// same parser, validated, and the TTL is short. Still
// wrap in try/catch in case a different deploy wrote a
// legacy/unparseable value to the same key.
try {
const parsed = JSON.parse(raw) as SiteConfig;
if (parsed && parsed.errors) return parsed;
} catch {
/* fall through to a fresh load */
}
}
} catch {
// Cache transport failure — fall through to live load.
// Don't poison the next request with a half-applied state.
}
}
for (const { name, parse } of SITE_CONFIG_FILENAMES) {
const filePath = pathPosix.join(rootPath, name);
let entry: FSEntry | null;
try {
entry = await fsEntryStore.getEntryByPath(filePath);
} catch (e) {
console.warn('[puter-site] config lookup failed', {
path: filePath,
error: (e as Error)?.message,
});
continue;
}
if (!entry || entry.isDir) continue;
// Reject oversized configs before paying the S3 read. `size` can
// be null for legacy entries — accept and rely on the streaming
// byte counter below.
if (entry.size !== null && entry.size > MAX_CONFIG_BYTES) {
console.warn('[puter-site] config too large, ignoring', {
path: filePath,
size: entry.size,
});
continue;
}
let text: string | null;
try {
text = await readBoundedText(entry, fsService, MAX_CONFIG_BYTES);
} catch (e) {
console.warn('[puter-site] config read failed', {
path: filePath,
error: (e as Error)?.message,
});
continue;
}
// Null means the stream exceeded the byte cap mid-read.
if (text === null) continue;
let parsed: SiteConfig | null;
try {
parsed = parse(text);
} catch (e) {
console.warn('[puter-site] config parse threw', {
path: filePath,
error: (e as Error)?.message,
});
parsed = null;
}
if (parsed && Object.keys(parsed.errors).length > 0) {
if (cacheable && cacheKey) {
writeCache(
cache!,
cacheKey,
JSON.stringify(parsed),
CACHE_TTL_SECONDS,
);
}
return parsed;
}
}
// No file matched (or all matched files parsed to empty). Cache
// the negative result so config-less sites — which are the common
// case — don't keep paying the FS lookup on every visit.
if (cacheable && cacheKey) {
writeCache(cache!, cacheKey, NEGATIVE_CACHE_MARKER, CACHE_TTL_SECONDS);
}
return null;
}
// Fire-and-forget cache write. We never await it on the request path
// because failure is non-fatal and we don't want a slow Redis to add
// latency to the response — the next request just re-loads from FS.
function writeCache(
cache: SiteConfigCache,
key: string,
value: string,
ttlSeconds: number,
): void {
cache.set(key, value, 'EX', ttlSeconds).catch(() => {
/* swallow — cache writes are best-effort */
});
}
/**
* Resolve a custom error rule for `statusCode` into an absolute FS path
* under `rootPath`. Returns null if no rule applies or the rule's `file`
* would escape the site root after normalization. Caller is responsible
* for loop prevention (don't recurse into error handling when serving
* the error page itself).
*/
export function resolveErrorTarget(
config: SiteConfig | null,
statusCode: number,
rootPath: string,
): { absPath: string; status: number } | null {
if (!config) return null;
const rule = config.errors[statusCode];
if (!rule) return null;
// Defence-in-depth: re-normalize at use time. `parsePuterSiteConfig`
// already does this, but keeping the contract here means future
// parsers (Vercel, Netlify) only need to return raw paths and can
// rely on this final guard.
const normalized = pathPosix.normalize(pathPosix.join('/', rule.file));
if (!normalized.startsWith('/') || normalized === '/') return null;
const absPath = rootPath.replace(/\/+$/, '') + normalized;
return { absPath, status: rule.status };
}
// -- Parsers ---------------------------------------------------------
function parsePuterSiteConfig(text: string): SiteConfig | null {
let raw: unknown;
try {
raw = JSON.parse(text);
} catch {
return null;
}
if (!raw || typeof raw !== 'object') return null;
const errorsField = (raw as { errors?: unknown }).errors;
const errors: Record<number, SiteErrorRule> = {};
if (errorsField && typeof errorsField === 'object') {
for (const [k, v] of Object.entries(
errorsField as Record<string, unknown>,
)) {
const code = Number(k);
// Only 4xx/5xx are meaningful as "error pages" — accepting
// 2xx/3xx keys would let a config silently override the
// happy path, which is out of scope and a footgun.
if (!Number.isInteger(code) || code < 400 || code > 599) continue;
if (!v || typeof v !== 'object') continue;
const rule = v as { file?: unknown; status?: unknown };
if (typeof rule.file !== 'string' || !rule.file.startsWith('/')) {
continue;
}
// Default the response status to the matched error code so a
// bare `{ file: '/404.html' }` Just Works. Allow overriding
// for the SPA-fallback case where 404 should turn into 200.
let status: number;
if (rule.status === undefined) {
status = code;
} else if (
typeof rule.status === 'number' &&
Number.isInteger(rule.status) &&
rule.status >= 200 &&
rule.status <= 599
) {
status = rule.status;
} else {
continue;
}
const normalized = pathPosix.normalize(
pathPosix.join('/', rule.file),
);
// Empty/root after normalize is meaningless as an error page
// (it would resolve to the site root itself).
if (normalized === '/') continue;
errors[code] = { file: normalized, status };
}
}
return { errors };
}
// Returns null when the stream exceeds `maxBytes` (caller treats as
// "config too large, ignore"). The stream is always destroyed before
// return so the S3 connection doesn't leak on early break.
async function readBoundedText(
entry: FSEntry,
fsService: LoadSiteConfigArgs['fsService'],
maxBytes: number,
): Promise<string | null> {
const download = await fsService.readContent(entry);
const stream = download.body as NodeJS.ReadableStream & {
destroy?: () => void;
};
const chunks: Buffer[] = [];
let total = 0;
let exceeded = false;
try {
for await (const chunk of stream as AsyncIterable<Buffer>) {
total += chunk.length;
if (total > maxBytes) {
exceeded = true;
break;
}
chunks.push(chunk);
}
} finally {
stream.destroy?.();
}
if (exceeded) return null;
return Buffer.concat(chunks).toString('utf8');
}
+34 -11
View File
@@ -42,11 +42,11 @@ import { HttpError } from '../HttpError.js';
* one via the `backend` option; omitting it uses the configured default.
*/
// ── Backend names ────────────────────────────────────────────────────
// -- Backend names ----------------------------------------------------
export const RATE_LIMIT_BACKENDS = ['memory', 'redis', 'kv'];
// ── Memory backend ──────────────────────────────────────────────────
// -- Memory backend --------------------------------------------------
// Hard cap and retention bound memory in the worst case. Without them,
// one-shot keys (visited once, never again) leak forever: their single
@@ -87,7 +87,7 @@ async function checkMemory(key, limit, windowMs) {
return true;
}
// ── Redis backend ───────────────────────────────────────────────────
// -- Redis backend ---------------------------------------------------
async function checkRedis(
/** @type {import('ioredis').Cluster} */
@@ -127,7 +127,7 @@ async function checkRedis(
return true;
}
// ── KV backend ──────────────────────────────────────────────────────
// -- KV backend ------------------------------------------------------
async function checkKv(kv, key, limit, windowMs) {
const prefix = `rate:${key}:`;
@@ -151,7 +151,7 @@ async function checkKv(kv, key, limit, windowMs) {
return true;
}
// ── Concurrent in-flight backends ───────────────────────────────────
// -- Concurrent in-flight backends -----------------------------------
//
// Concurrent limiting is the *other* shape: rather than "no more than X
// hits in Y window", it's "no more than X requests in flight at once".
@@ -247,7 +247,7 @@ async function acquireKvConcurrent(kv, key, limit) {
};
}
// ── Backend registry ────────────────────────────────────────────────
// -- Backend registry ------------------------------------------------
//
// All registered backends stay live simultaneously; selection happens
// per call via the `backend` option. The `default` slot is what callers
@@ -345,7 +345,7 @@ function resolveBackend(name) {
return backends[defaultBackendName];
}
// ── Key strategies ──────────────────────────────────────────────────
// -- Key strategies --------------------------------------------------
/**
* Build a rate-limit key from the request.
@@ -406,7 +406,7 @@ function fingerprint(req) {
.slice(0, 16);
}
// ── Route middleware ────────────────────────────────────────────────
// -- Route middleware ------------------------------------------------
/**
* Express middleware factory. Reads from the materialised route option:
@@ -455,7 +455,7 @@ export function rateLimitGate(opts) {
};
}
// ── Driver-call helper ──────────────────────────────────────────────
// -- Driver-call helper ----------------------------------------------
/**
* Check rate limit for a driver call. Called from DriverController's
@@ -492,7 +492,30 @@ export async function checkDriverRateLimit(req, ifaceName, method, opts = {}) {
}
}
// ── Subscription-aware limit resolution ─────────────────────────────
// -- Imperative helper -----------------------------------------------
/**
* Imperative rate-limit check (no middleware shape). For handlers that
* need a second-axis limit after their route-level limit fires —
* e.g. `/auth/migrate-token` limits per IP at the route, then per
* `auth_id` once the v1 token is decoded. Returns true if allowed,
* false if rate-limited. Fails open on backend error, matching the
* rest of this module's policy.
*/
export async function checkRateLimit(key, limit, windowMs, backend) {
const bk = resolveBackend(backend);
try {
return await bk.rate(key, limit, windowMs);
} catch (err) {
console.error(
'[rate-limit] imperative check failed, failing open:',
err,
);
return true;
}
}
// -- Subscription-aware limit resolution -----------------------------
/**
* Per-request limit resolution shared by `rateLimitGate` and
@@ -517,7 +540,7 @@ async function resolveSubscriptionLimit(req, opts) {
}
}
// ── Concurrency gate + driver helper ────────────────────────────────
// -- Concurrency gate + driver helper --------------------------------
/**
* Express middleware factory for concurrent in-flight limiting:
@@ -63,6 +63,57 @@ export interface UserProtectedGateDeps {
tokenService: TokenService;
}
/**
* Cookie-only credential check. Rejects API tokens, GUI tokens,
* `x-api-key` headers, and query-string tokens — only a request whose
* resolved `req.token` matches the session cookie passes. Used both by
* `createUserProtectedGate` (as its first stage) and standalone by
* routes that need session-cookie-only credentials without the full
* password-revalidation gate (e.g. `/auth/revoke-*`, where an access
* token shouldn't be able to revoke its own issuing web session).
*/
export const createSessionCookieGate = (config: IConfig): RequestHandler => {
const cookieName = config.cookie_name ?? 'puter_token';
return (req, _res, next) => {
const cookieValue = req.cookies?.[cookieName];
if (!cookieValue || (req.token && req.token !== cookieValue)) {
return next(
new HttpError(401, 'Session cookie required', {
legacyCode: 'session_required',
}),
);
}
next();
};
};
/**
* Same intent as `createSessionCookieGate` ("an access token can't revoke
* its issuing web session") but identity-shape based instead of cookie-
* value based, so it works for the cross-subdomain GUI -> api.puter.com
* call path where the session cookie isn't sent. Accepts a plain user
* actor (web/session/gui-token); rejects app-under-user actors and
* access-token actors. Pair with `antiCsrf: true` on the route.
*/
export const createWebSessionActorGate = (): RequestHandler => {
return (req, _res, next) => {
const actor = req.actor;
if (
!actor?.user ||
!actor.session ||
(actor as { app?: unknown }).app ||
(actor as { accessToken?: unknown }).accessToken
) {
return next(
new HttpError(401, 'Web session required', {
legacyCode: 'session_required',
}),
);
}
next();
};
};
export interface UserProtectedGateOptions {
/** Allow temp accounts (no password + no email) through. Default: false. */
allowTempUsers?: boolean;
@@ -95,19 +146,10 @@ export const createUserProtectedGate = (
options: UserProtectedGateOptions = {},
): RequestHandler[] => {
const { config, userStore, oidcService, tokenService } = deps;
const cookieName = config.cookie_name ?? 'puter_token';
const allowTemp = !!options.allowTempUsers;
// 1. Session cookie only.
const requireSessionCookie: RequestHandler = (req, _res, next) => {
const cookieValue = req.cookies?.[cookieName];
if (!cookieValue || (req.token && req.token !== cookieValue)) {
throw new HttpError(401, 'Session cookie required', {
legacyCode: 'session_required',
});
}
next();
};
// 1. Session cookie only. Shared with the standalone cookie-only gate.
const requireSessionCookie = createSessionCookieGate(config);
// 2. Fresh user row (bypass cache to catch just-suspended accounts).
// `getById` doesn't take options; go through `getByProperty` with
+1 -1
View File
@@ -274,7 +274,7 @@ export const ROUTES_METADATA_KEY = '__puterRoutes' as const;
/** Internal: the property name used to stash a controller's path prefix. */
export const PREFIX_METADATA_KEY = '__puterControllerPrefix' as const;
// ── Type narrowing helpers ──────────────────────────────────────────
// -- Type narrowing helpers ------------------------------------------
//
// When a route declares a gate option (requireAuth, requireUserActor,
// adminOnly, allowedAppIds), the materializer guarantees the corresponding
@@ -43,6 +43,7 @@ import { XAIProvider } from './providers/xai/XAIProvider.js';
import { ZAIProvider } from './providers/zai/ZAIProvider.js';
import { AlibabaProvider } from './providers/alibaba/AlibabaProvider.js';
import { MoonshotProvider } from './providers/moonshot/MoonshotProvider.js';
import { MiniMaxProvider } from './providers/minimax/MiniMaxProvider.js';
import type {
IChatCompleteResult,
IChatModel,
@@ -217,7 +218,7 @@ export class ChatCompletionDriver extends PuterDriver {
this.#buildModelMap();
}
// ── Interface methods ───────────────────────────────────────────
// -- Interface methods -------------------------------------------
async models() {
const seen = new Set<string>();
@@ -342,7 +343,7 @@ export class ChatCompletionDriver extends PuterDriver {
}
}
// ── Credit / subscription gates (metering) ────────────────────
// -- Credit / subscription gates (metering) --------------------
// Cheap pre-flight: reject when the user can't afford even the
// approximate input cost, keep subscriber-only models gated, and
// cap `max_tokens` so output can't exceed remaining credits.
@@ -555,7 +556,7 @@ export class ChatCompletionDriver extends PuterDriver {
return streamResult as unknown as IChatCompleteResult;
}
// ── Post-completion audit event ──────────────────────────────
// -- Post-completion audit event ------------------------------
// Only for non-streaming results (streaming emits from the
// `chatStream.end` wrapper above). Extensions like prompt_block /
// prodMeteringAndBilling listen for this to log completions.
@@ -651,8 +652,18 @@ export class ChatCompletionDriver extends PuterDriver {
// thinking_tokens → output rate fallback
let rate = costs[key];
if (typeof rate !== 'number' || !Number.isFinite(rate)) {
if (key === 'thinking_tokens' && outputRate !== undefined) {
if (isOutputKey(key) && outputRate !== undefined) {
rate = outputRate;
} else if (!isOutputKey(key)) {
const inputRateRaw = costs[inputKey];
if (
typeof inputRateRaw === 'number' &&
Number.isFinite(inputRateRaw)
) {
rate = inputRateRaw;
} else {
continue;
}
} else {
continue;
}
@@ -762,7 +773,7 @@ export class ChatCompletionDriver extends PuterDriver {
);
}
// ── Provider registration ───────────────────────────────────────
// -- Provider registration ---------------------------------------
#registerProviders() {
const providers = this.config.providers ?? {};
@@ -859,6 +870,18 @@ export class ChatCompletionDriver extends PuterDriver {
);
}
const minimax = providers['minimax'];
const minimaxKey = readKey(minimax);
if (minimaxKey) {
this.#providers['minimax'] = new MiniMaxProvider(
{
apiKey: minimaxKey,
apiBaseUrl: minimax?.apiBaseUrl as string | undefined,
},
metering,
);
}
const zai = providers['zai'];
const zaiKey = readKey(zai);
if (zaiKey) {
@@ -918,7 +941,7 @@ export class ChatCompletionDriver extends PuterDriver {
this.#providers['fake-chat'] = new FakeChatProvider();
}
// ── Model map ───────────────────────────────────────────────────
// -- Model map ---------------------------------------------------
async #buildModelMap() {
const AGGREGATORS = new Set(['together-ai', 'openrouter']);
@@ -21,7 +21,7 @@ import type { IChatModel } from '../../types.js';
// Hardcoded from https://models.dev/api.json
export const ALIBABA_MODELS: IChatModel[] = [
// ── Commercial flagship ────────────────────────────────────────
// -- Commercial flagship ----------------------------------------
{
puterId: 'alibaba:qwen/qwen-max',
id: 'qwen-max',
@@ -89,7 +89,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Plus tier ──────────────────────────────────────────────────
// -- Plus tier --------------------------------------------------
{
puterId: 'alibaba:qwen/qwen-plus',
id: 'qwen-plus',
@@ -157,7 +157,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Turbo / Flash tier ─────────────────────────────────────────
// -- Turbo / Flash tier -----------------------------------------
{
puterId: 'alibaba:qwen/qwen-turbo',
id: 'qwen-turbo',
@@ -203,7 +203,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Coding models ──────────────────────────────────────────────
// -- Coding models ----------------------------------------------
{
puterId: 'alibaba:qwen/qwen3-coder-plus',
id: 'qwen3-coder-plus',
@@ -293,7 +293,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Reasoning models ───────────────────────────────────────────
// -- Reasoning models -------------------------------------------
{
puterId: 'alibaba:qwen/qwq-plus',
id: 'qwq-plus',
@@ -361,7 +361,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Open-weight Qwen3 ──────────────────────────────────────────
// -- Open-weight Qwen3 ------------------------------------------
{
puterId: 'alibaba:qwen/qwen3-235b-a22b',
id: 'qwen3-235b-a22b',
@@ -451,7 +451,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Open-weight Qwen3.5 ────────────────────────────────────────
// -- Open-weight Qwen3.5 ----------------------------------------
{
puterId: 'alibaba:qwen/qwen3.5-397b-a17b',
id: 'qwen3.5-397b-a17b',
@@ -549,7 +549,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Open-weight Qwen3.6 ────────────────────────────────────────
// -- Open-weight Qwen3.6 ----------------------------------------
{
puterId: 'alibaba:qwen/qwen3.6-35b-a3b',
id: 'qwen3.6-35b-a3b',
@@ -599,7 +599,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Vision models ──────────────────────────────────────────────
// -- Vision models ----------------------------------------------
{
puterId: 'alibaba:qwen/qwen-vl-max',
id: 'qwen-vl-max',
@@ -755,7 +755,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Legacy Qwen2.5 ────────────────────────────────────────────
// -- Legacy Qwen2.5 --------------------------------------------
{
puterId: 'alibaba:qwen/qwen2-5-72b-instruct',
id: 'qwen2-5-72b-instruct',
@@ -889,7 +889,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Omni models (text interface, audio costs excluded) ─────────
// -- Omni models (text interface, audio costs excluded) ---------
{
puterId: 'alibaba:qwen/qwen-omni-turbo',
id: 'qwen-omni-turbo',
@@ -966,7 +966,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
},
},
// ── Translation models ─────────────────────────────────────────
// -- Translation models -----------------------------------------
{
puterId: 'alibaba:qwen/qwen-mt-plus',
id: 'qwen-mt-plus',
@@ -229,10 +229,13 @@ export class ClaudeProvider implements IChatProvider {
reasoningEffort: requestedReasoningEffort,
maxTokens: max_tokens,
});
// Opus 4.7 errors on non-default sampling params; omit temperature entirely.
// Opus 4.7 and 4.8 error on non-default sampling params; omit temperature entirely.
// Other models require temperature=1 when thinking is enabled.
const isOpus47 = modelUsed.id === 'claude-opus-4-7';
const resolvedTemperature = isOpus47
const omitsTemperature = [
'claude-opus-4-7',
'claude-opus-4-8',
].includes(modelUsed.id);
const resolvedTemperature = omitsTemperature
? undefined
: thinkingConfig
? 1
@@ -21,6 +21,37 @@ import type { IChatModel } from '../../types.js';
// Hardcoded from https://models.dev/api.json
export const CLAUDE_MODELS: IChatModel[] = [
{
puterId: 'anthropic:anthropic/claude-opus-4-8',
id: 'claude-opus-4-8',
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
open_weights: false,
tool_call: true,
knowledge: '2026-01',
release_date: '2026-05-28',
aliases: [
'claude-opus',
'claude-opus-latest',
'claude-opus-4-8-latest',
'claude-opus-4.8',
'claude-opus-4-8',
'anthropic/claude-opus-4-8',
],
name: 'Claude Opus 4.8',
costs_currency: 'usd-cents',
input_cost_key: 'input_tokens',
output_cost_key: 'output_tokens',
costs: {
tokens: 1_000_000,
input_tokens: 500,
ephemeral_5m_input_tokens: 500 * 1.25,
ephemeral_1h_input_tokens: 500 * 2,
cache_read_input_tokens: 500 * 0.1,
output_tokens: 2500,
},
context: 1000000,
max_tokens: 128000,
},
{
puterId: 'anthropic:anthropic/claude-opus-4-7',
id: 'claude-opus-4-7',
@@ -30,8 +61,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
knowledge: '2026-01',
release_date: '2026-04-16',
aliases: [
'claude-opus',
'claude-opus-latest',
'claude-opus-4-7-latest',
'claude-opus-4.7',
'claude-opus-4-7',
@@ -31,7 +31,7 @@ export const GEMINI_MODELS: IChatModel[] = [
open_weights: false,
tool_call: true,
knowledge: '2025-01',
release_date: '2026-05',
release_date: '2026-05-19',
name: 'Gemini 3.5 Flash',
aliases: ['google/gemini-3.5-flash'],
context: 1_048_576,
@@ -0,0 +1,355 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import {
afterAll,
afterEach,
beforeAll,
beforeEach,
describe,
expect,
it,
vi,
type MockInstance,
} from 'vitest';
import { SYSTEM_ACTOR } from '../../../../core/actor.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { PuterServer } from '../../../../server.js';
import { setupTestServer } from '../../../../testUtil.js';
import { withTestActor } from '../../../integrationTestUtil.js';
import { MINIMAX_MODELS } from './models.js';
import { MiniMaxProvider } from './MiniMaxProvider.js';
const { createMock, openAICtor } = vi.hoisted(() => {
const createMock = vi.fn();
const openAICtor = vi.fn();
return { createMock, openAICtor };
});
vi.mock('openai', () => {
const OpenAICtor = vi.fn().mockImplementation(function (
this: Record<string, unknown>,
opts: unknown,
) {
openAICtor(opts);
this.chat = { completions: { create: createMock } };
});
return { OpenAI: OpenAICtor, default: { OpenAI: OpenAICtor } };
});
let server: PuterServer;
let recordSpy: MockInstance<MeteringService['utilRecordUsageObject']>;
beforeAll(async () => {
server = await setupTestServer();
});
afterAll(async () => {
await server?.shutdown();
});
const makeProvider = () => {
const provider = new MiniMaxProvider(
{ apiKey: 'test-key' },
server.services.metering,
);
return { provider };
};
const asAsyncIterable = <T>(items: T[]): AsyncIterable<T> => ({
async *[Symbol.asyncIterator]() {
for (const item of items) {
yield item;
}
},
});
beforeEach(() => {
createMock.mockReset();
openAICtor.mockReset();
recordSpy = vi.spyOn(server.services.metering, 'utilRecordUsageObject');
});
afterEach(() => {
vi.restoreAllMocks();
});
describe('MiniMaxProvider construction', () => {
it('points the OpenAI SDK at the MiniMax base URL with the configured key', () => {
makeProvider();
expect(openAICtor).toHaveBeenCalledTimes(1);
expect(openAICtor).toHaveBeenCalledWith({
apiKey: 'test-key',
baseURL: 'https://api.minimax.io/v1',
});
});
it('allows overriding the MiniMax API base URL', () => {
new MiniMaxProvider(
{
apiKey: 'test-key',
apiBaseUrl: 'https://example.test/v1',
},
server.services.metering,
);
expect(openAICtor).toHaveBeenCalledWith({
apiKey: 'test-key',
baseURL: 'https://example.test/v1',
});
});
});
describe('MiniMaxProvider model catalog', () => {
it('returns minimax-m2.7 as the default', () => {
const { provider } = makeProvider();
expect(provider.getDefaultModel()).toBe('minimax-m2.7');
});
it('exposes the static MINIMAX_MODELS list verbatim from models()', () => {
const { provider } = makeProvider();
expect(provider.models()).toBe(MINIMAX_MODELS);
});
it('list() flattens canonical ids and aliases', () => {
const { provider } = makeProvider();
const ids = provider.list();
for (const model of MINIMAX_MODELS) {
expect(ids).toContain(model.id);
for (const alias of model.aliases ?? []) {
expect(ids).toContain(alias);
}
}
expect(ids).toContain('minimax-m2.7');
expect(ids).toContain('MiniMax-M2.7');
expect(ids).toContain('minimax/minimax-m2.7');
});
it('uses MiniMax completion-token limits instead of the context window', () => {
for (const model of MINIMAX_MODELS) {
expect(model.context).toBe(204_800);
expect(model.max_tokens).toBe(196_608);
}
});
});
describe('MiniMaxProvider.complete request shape', () => {
const baseCompletion = {
choices: [
{
message: { content: 'hi', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: { prompt_tokens: 1, completion_tokens: 1 },
};
it('sends the case-sensitive upstream MiniMax model id', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hello' }],
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.model).toBe('MiniMax-M2.7');
expect(args.messages).toEqual([{ role: 'user', content: 'hello' }]);
expect(args.max_tokens).toBe(1000);
});
it('resolves the upstream-cased alias to the canonical API model', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'MiniMax-M2.7-highspeed',
messages: [{ role: 'user', content: 'hello' }],
}),
);
expect(createMock.mock.calls[0]![0].model).toBe(
'MiniMax-M2.7-highspeed',
);
expect(recordSpy).toHaveBeenCalledWith(
expect.any(Object),
expect.anything(),
'minimax:minimax-m2.7-highspeed',
expect.any(Object),
);
});
it('passes standard chat completion options through', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
const tools = [
{
type: 'function',
function: {
name: 'lookup',
parameters: { type: 'object', properties: {} },
},
},
];
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
tools,
tool_choice: 'auto',
max_tokens: 256,
temperature: 0.4,
top_p: 0.9,
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.tools).toBe(tools);
expect(args.tool_choice).toBe('auto');
expect(args.max_tokens).toBe(256);
expect(args.temperature).toBe(0.4);
expect(args.top_p).toBe(0.9);
});
it('clamps oversized max_tokens to the MiniMax completion limit', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.1',
messages: [{ role: 'user', content: 'hi' }],
max_tokens: 200_000,
}),
);
const [args] = createMock.mock.calls[0]!;
expect(args.model).toBe('MiniMax-M2.1');
expect(args.max_tokens).toBe(196_608);
});
it('only sets stream_options.include_usage when streaming', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce(baseCompletion);
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
stream: false,
}),
);
const [nonStreamArgs] = createMock.mock.calls[0]!;
expect(nonStreamArgs.stream).toBe(false);
expect('stream_options' in nonStreamArgs).toBe(false);
createMock.mockReturnValueOnce(asAsyncIterable([]));
await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
stream: true,
}),
);
const [streamArgs] = createMock.mock.calls[1]!;
expect(streamArgs.stream).toBe(true);
expect(streamArgs.stream_options).toEqual({ include_usage: true });
});
});
describe('MiniMaxProvider.complete output and metering', () => {
it('returns the first choice and records MiniMax usage with model costs', async () => {
const { provider } = makeProvider();
createMock.mockResolvedValueOnce({
choices: [
{
message: { content: 'hi there', role: 'assistant' },
finish_reason: 'stop',
},
],
usage: {
prompt_tokens: 100,
completion_tokens: 50,
prompt_tokens_details: { cached_tokens: 10 },
},
});
const result = await withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'hi' }],
}),
);
expect(result).toMatchObject({
message: { content: 'hi there', role: 'assistant' },
finish_reason: 'stop',
});
expect((result as { usage: unknown }).usage).toEqual({
prompt_tokens: 100,
completion_tokens: 50,
cached_tokens: 10,
});
expect(recordSpy).toHaveBeenCalledTimes(1);
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
expect(usage).toEqual({
prompt_tokens: 100,
completion_tokens: 50,
cached_tokens: 10,
});
expect(actor).toBe(SYSTEM_ACTOR);
expect(prefix).toBe('minimax:minimax-m2.7');
expect(overrides).toEqual({
prompt_tokens: 100 * 30,
completion_tokens: 50 * 120,
cached_tokens: 10 * 6,
});
});
it('rethrows errors raised by the OpenAI client unchanged', async () => {
const { provider } = makeProvider();
const apiError = new Error('MiniMax exploded');
createMock.mockRejectedValueOnce(apiError);
await expect(
withTestActor(() =>
provider.complete({
model: 'minimax-m2.7',
messages: [{ role: 'user', content: 'boom' }],
}),
),
).rejects.toBe(apiError);
expect(recordSpy).not.toHaveBeenCalled();
});
});
describe('MiniMaxProvider.checkModeration', () => {
it('throws because MiniMax provider does not implement moderation', () => {
const { provider } = makeProvider();
expect(() => provider.checkModeration('anything')).toThrow(
/not implemented/i,
);
});
});
@@ -0,0 +1,135 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { OpenAI } from 'openai';
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
import { Context } from '../../../../core/context.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import type { IChatProvider, ICompleteArguments } from '../../types.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
import { MINIMAX_MODELS } from './models.js';
type MiniMaxConfig = {
apiKey: string;
apiBaseUrl?: string;
};
export class MiniMaxProvider implements IChatProvider {
#openai: OpenAI;
#meteringService: MeteringService;
#defaultModel = 'minimax-m2.7';
constructor(config: MiniMaxConfig, meteringService: MeteringService) {
this.#openai = new OpenAI({
apiKey: config.apiKey,
baseURL: config.apiBaseUrl ?? 'https://api.minimax.io/v1',
});
this.#meteringService = meteringService;
}
getDefaultModel() {
return this.#defaultModel;
}
models() {
return MINIMAX_MODELS;
}
list() {
const modelIds: string[] = [];
for (const model of this.models()) {
modelIds.push(model.id);
if (model.aliases) {
modelIds.push(...model.aliases);
}
}
return modelIds;
}
async complete({
messages,
stream,
model,
tools,
tool_choice,
max_tokens,
temperature,
top_p,
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
const actor = Context.get('actor');
const availableModels = this.models();
const modelUsed =
availableModels.find((m) =>
[m.id, ...(m.aliases || [])].includes(model),
) || availableModels.find((m) => m.id === this.getDefaultModel())!;
messages = await OpenAIUtil.process_input_messages(messages);
const requestedMaxTokens = max_tokens ?? 1000;
const completion = await this.#openai.chat.completions.create({
messages,
model: modelUsed.apiModel,
...(tools ? { tools } : {}),
...(tool_choice !== undefined ? { tool_choice } : {}),
max_tokens: Math.min(requestedMaxTokens, modelUsed.max_tokens),
...(temperature !== undefined ? { temperature } : {}),
...(top_p !== undefined ? { top_p } : {}),
stream,
...(stream
? {
stream_options: { include_usage: true },
}
: {}),
} as ChatCompletionCreateParams);
return OpenAIUtil.handle_completion_output({
usage_calculator: ({ usage }) => {
const trackedUsage = usage
? OpenAIUtil.extractMeteredUsage(usage)
: {
prompt_tokens: 0,
completion_tokens: 0,
cached_tokens: 0,
};
const costsOverride = Object.fromEntries(
Object.entries(trackedUsage).map(([key, value]) => {
return [key, value * Number(modelUsed.costs[key] ?? 0)];
}),
);
this.#meteringService.utilRecordUsageObject(
trackedUsage,
actor!,
`minimax:${modelUsed.id}`,
costsOverride,
);
return trackedUsage;
},
stream,
completion,
});
}
checkModeration(
_text: string,
): ReturnType<IChatProvider['checkModeration']> {
throw new Error('Method not implemented.');
}
}
@@ -0,0 +1,77 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { IChatModel } from '../../types.js';
import { usdPerMToken } from '../../utils/pricing.js';
const CONTEXT_WINDOW = 204_800;
const MAX_OUTPUT_TOKENS = 196_608;
type MiniMaxChatModel = IChatModel & {
apiModel: string;
};
const minimaxModel = (
apiModel: string,
name: string,
costs: IChatModel['costs'],
): MiniMaxChatModel => {
const id = apiModel.toLowerCase();
return {
puterId: `minimax:minimax/${id}`,
id,
apiModel,
name,
aliases: [`minimax/${id}`, apiModel, `minimax/${apiModel}`],
modalities: { input: ['text'], output: ['text'] },
open_weights: false,
tool_call: true,
context: CONTEXT_WINDOW,
max_tokens: MAX_OUTPUT_TOKENS,
costs_currency: 'usd-cents',
input_cost_key: 'prompt_tokens',
output_cost_key: 'completion_tokens',
costs,
};
};
// Hardcoded from MiniMax OpenAI-compatible API docs and pay-as-you-go pricing:
// https://platform.minimax.io/docs/api-reference/text-openai-api
// https://platform.minimax.io/docs/guides/pricing-paygo
export const MINIMAX_MODELS: MiniMaxChatModel[] = [
minimaxModel('MiniMax-M2.7', 'MiniMax M2.7', usdPerMToken(0.3, 1.2, 0.06)),
minimaxModel(
'MiniMax-M2.7-highspeed',
'MiniMax M2.7 Highspeed',
usdPerMToken(0.6, 2.4, 0.06),
),
minimaxModel('MiniMax-M2.5', 'MiniMax M2.5', usdPerMToken(0.3, 1.2, 0.03)),
minimaxModel(
'MiniMax-M2.5-highspeed',
'MiniMax M2.5 Highspeed',
usdPerMToken(0.6, 2.4, 0.03),
),
minimaxModel('MiniMax-M2.1', 'MiniMax M2.1', usdPerMToken(0.3, 1.2, 0.03)),
minimaxModel(
'MiniMax-M2.1-highspeed',
'MiniMax M2.1 Highspeed',
usdPerMToken(0.6, 2.4, 0.03),
),
minimaxModel('MiniMax-M2', 'MiniMax M2', usdPerMToken(0.3, 1.2, 0.03)),
];
@@ -61,22 +61,6 @@ export class MistralAIProvider implements IChatProvider {
return ids;
}
processInputMistralQuirks(messages: any[]) {
const output = [];
for (const message of messages) {
const contents = [];
for (const content of message.content) {
if (content.type === 'image_url') {
content.image_url = content.image_url.url;
}
contents.push(content);
}
message.content = contents;
output.push(message);
}
return output;
}
async complete({
messages,
stream,
@@ -86,7 +70,6 @@ export class MistralAIProvider implements IChatProvider {
temperature,
}: ICompleteArguments): Promise<IChatCompleteResult> {
messages = await OpenAIUtil.process_input_messages(messages);
messages = this.processInputMistralQuirks(messages);
for (const message of messages) {
if (message.tool_calls) {
message.toolCalls = message.tool_calls;
@@ -20,7 +20,7 @@
import type { IChatModel } from '../../types.js';
export const MOONSHOT_MODELS: IChatModel[] = [
// ── Flagship ────────────────────────────────────────────────────
// -- Flagship ----------------------------------------------------
{
puterId: 'moonshotai:moonshotai/kimi-k2.6',
id: 'kimi-k2.6',
@@ -47,7 +47,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
knowledge: '2025-01',
},
// ── Kimi K2.5 ──────────────────────────────────────────────────
// -- Kimi K2.5 --------------------------------------------------
{
puterId: 'moonshotai:moonshotai/kimi-k2.5',
id: 'kimi-k2.5',
@@ -69,7 +69,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
knowledge: '2025-01',
},
// ── Moonshot V1 (Legacy) ───────────────────────────────────────
// -- Moonshot V1 (Legacy) ---------------------------------------
{
puterId: 'moonshotai:moonshotai/moonshot-v1-8k',
id: 'moonshot-v1-8k',
@@ -92,6 +92,7 @@ const SAMPLE_API_MODELS = [
{
id: 'openai/gpt-5-nano',
name: 'GPT-5 Nano',
created: 1714564800,
context_length: 128000,
pricing: { prompt: 0.00001, completion: 0.00003 },
top_provider: { max_completion_tokens: 16000 },
@@ -99,6 +100,7 @@ const SAMPLE_API_MODELS = [
{
id: 'anthropic/claude-haiku-4.5',
name: 'Claude Haiku 4.5',
created: 1760529600,
context_length: 200000,
pricing: { prompt: 0.000002, completion: 0.00001 },
top_provider: { max_completion_tokens: 8192 },
@@ -107,6 +109,7 @@ const SAMPLE_API_MODELS = [
// 'openrouter/auto' is filtered out — disallowed.
id: 'openrouter/auto',
name: 'Auto',
created: 1704067200,
context_length: 32768,
pricing: { prompt: 0, completion: 0 },
top_provider: { max_completion_tokens: 4096 },
@@ -224,6 +227,23 @@ describe('OpenRouterProvider model catalog', () => {
// Second call should be a cache hit, not a second axios request.
expect(axiosRequestMock).toHaveBeenCalledTimes(1);
});
it('maps OpenRouter created timestamps to release_date metadata', async () => {
const { provider } = makeProvider();
const models = await provider.models();
expect(models).toEqual(
expect.arrayContaining([
expect.objectContaining({
id: 'openrouter:openai/gpt-5-nano',
release_date: '2024-05-01',
}),
expect.objectContaining({
id: 'openrouter:anthropic/claude-haiku-4.5',
release_date: '2025-10-15',
}),
]),
);
});
});
// ── Disallowed model gate ───────────────────────────────────────────
@@ -36,6 +36,18 @@ type OpenrouterUsage = OpenAI.Completions.CompletionUsage & {
cost?: number;
};
const openRouterReleaseDate = (created: unknown): string | undefined => {
const seconds = Number(created);
if (!Number.isFinite(seconds) || seconds <= 0) {
return undefined;
}
const date = new Date(seconds * 1000);
const year = date.getUTCFullYear();
const month = String(date.getUTCMonth() + 1).padStart(2, '0');
const day = String(date.getUTCDate()).padStart(2, '0');
return `${year}-${month}-${day}`;
};
export class OpenRouterProvider implements IChatProvider {
#meteringService: MeteringService;
@@ -275,6 +287,7 @@ export class OpenRouterProvider implements IChatProvider {
tokens: 1_000_000,
...microcentCosts,
},
release_date: openRouterReleaseDate(model.created),
...overridenModel,
});
}
@@ -433,10 +433,10 @@ describe('TogetherAIProvider.complete non-stream output', () => {
completion_tokens: 50,
cached_tokens: 0,
});
// Qwen pricing: input=20, output=20 (per million).
// Qwen pricing: input=20, output=20 (per million, dollars from API → ×100 for cents).
expect(overrides).toMatchObject({
prompt_tokens: 100 * 20,
completion_tokens: 50 * 20,
prompt_tokens: 100 * 20 * 100,
completion_tokens: 50 * 20 * 100,
});
});
});
@@ -484,13 +484,13 @@ describe('TogetherAIProvider.complete streaming', () => {
cached_tokens: 0,
});
// Qwen pricing: input=20, output=20.
// Qwen pricing: input=20, output=20 (dollars from API → ×100 for cents).
expect(recordSpy).toHaveBeenCalledTimes(1);
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
expect(prefix).toBe('togetherai:Qwen/Qwen2.5-7B-Instruct-Turbo');
expect(overrides).toMatchObject({
prompt_tokens: 4 * 20,
completion_tokens: 2 * 20,
prompt_tokens: 4 * 20 * 100,
completion_tokens: 2 * 20 * 100,
});
});
});
@@ -21,8 +21,8 @@ import { Together } from 'together-ai';
import { Context } from '../../../../core/context.js';
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
import { kv } from '../../../../util/kvSingleton.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
import { IChatModel, IChatProvider, ICompleteArguments } from '../../types.js';
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
const TOGETHER_AI_CHAT_COST_MAP = {
prompt_tokens: 'input',
@@ -51,7 +51,9 @@ export class TogetherAIProvider implements IChatProvider {
let models: IChatModel[] | undefined = kv.get(this.#kvKey);
if (models) return models;
const apiModels = await this.#together.models.list();
const apiModels = await this.#together.models.list({
query: { serverless: 'true' },
});
models = [];
for (const model of apiModels) {
if (
@@ -75,7 +77,11 @@ export class TogetherAIProvider implements IChatProvider {
output_cost_key: 'output',
costs: {
tokens: 1_000_000,
...model.pricing,
...Object.fromEntries(
Object.entries(model.pricing ?? {}).map(
([k, v]) => [k, (v as number) * 100],
),
),
},
max_tokens: model.context_length ?? 8000,
});
@@ -126,9 +132,13 @@ export class TogetherAIProvider implements IChatProvider {
const actor = Context.get('actor');
const models = await this.models();
const modelLower = model.toLowerCase();
const modelUsed =
models.find((m) => [m.id, ...(m.aliases || [])].includes(model)) ||
models.find((m) => m.id === this.getDefaultModel())!;
models.find((m) =>
[m.id, ...(m.aliases || [])].some(
(id) => id.toLowerCase() === modelLower,
),
) || models.find((m) => m.id === this.getDefaultModel())!;
const modelIdForParams = modelUsed.id.startsWith('togetherai:')
? modelUsed.id.slice('togetherai:'.length)
: modelUsed.id;
Loaded 100 of 267 files, more files were not shown because too many files have changed in this diff. Show more